A Method for Identifying User Interface Anomalies in Map-Based Applications Using Prompt Engineering
Abstract
Identifying user interface (UI) regressions in complex map navigation applications from video recordings may be a subjective process. This disclosure describes a method for analyzing navigation UI behavior using a generative artificial intelligence (AI) model, which can be guided by prompt engineering strategies. The method can involve decomposing the analysis into specific components, such as camera behavior, polyline rendering, and map elements. The AI model may be provided with technical specifications, including quantitative metrics and explicit failure conditions, to analyze video input. This approach can enable the AI to systematically identify deviations and report potential regressions in a structured, time-segmented format with timestamps and severity ratings, which can support a more consistent and scalable process for UI regression detection.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Cheng, Shu-Wei; Kodur, Kamakshi; Huang, Andy; Joshi, Megha; Hua, Jessica; Ding, Oscar; Borgnino, Juan; Sadasivam, Yoga; Li, Zening; and Wang, Fei, "A Method for Identifying User Interface Anomalies in Map-Based Applications Using Prompt Engineering", Technical Disclosure Commons, (September 24, 2025)
https://www.tdcommons.org/dpubs_series/8621