Abstract
A machine learning model is given for classifying images according to different aspects.
Even though this model had a good accuracy on the test set, when it was deployed into
production, we (the model developers) started to face many cases with failures due to input
images that were not covered in our training or test sets. We’ve identified that these failures
commonly have smaller confidence levels, and hence applying a threshold value to exclude low
confidence predictions could work as a solution to avoid failures. In order to establish this
threshold value, we explored different threshold values with the confidence scores returned by
the model on different subsets of images. When below the defined threshold, the output is then
considered unknown (i.e., not belonging to any of the established classes) and the SDK client will
be able to define a specific behavior for these cases. Also, the threshold can be configured to be
adaptive regarding each of the classes that the model was trained to output.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Share Alike 4.0 License.
Recommended Citation
INC, HP, "ESTABLISHING AN UNKNOWN CLASS FOR CLASSIFICATION ALGORITHMS BASED ON A LOCAL OR GLOBAL CONFIDENCE LEVEL THRESHOLD", Technical Disclosure Commons, (September 13, 2021)
https://www.tdcommons.org/dpubs_series/4587