img/favicon.ico" />
MyNew Technologies

How to investigate a model whose accuracy has dropped

Start with a known time window when model accuracy changed and make sure the comparison is fair.

What to do

  1. Check whether input fields, units, missing values, or user populations changed around that time.
  2. Review labeled examples in each affected group and compare them with the previous evaluation set.
  3. Ask the model owner to document the cause, next check and conditions for limiting usage or testing a replacement.

Reference: NIST AI Risk Management Framework.

Related MyNew service: AI & Machine Learning.