Abstract
Quality assurance for visually complex applications may be resource-intensive, and some automated methods can be inaccurate or difficult to scale. A systematic framework is described for generating specialized machine learning models for visual user interface defect detection. This approach can involve adapting large, pre-trained foundational models through a fine-tuning process that uses small, curated datasets, with each model trained to recognize a specific visual anomaly. The framework can also incorporate data augmentation and an active learning feedback loop to iteratively improve model performance by sampling and labeling challenging examples. This process may enable the creation of classifiers for quality assurance, which can result in a more automated, scalable, and repeatable workflow for developing defect detectors.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Borgnino, Juan; Wang, Fei; Li, Zening; Cheng, Shu-Wei; and Kodur, Kamakshi, "Generating Specialized Models for Visual UI Defect Detection via Fine-Tuning and Active Learning", Technical Disclosure Commons, (September 02, 2026)
https://www.tdcommons.org/dpubs_series/11580