Abstract
Current front-end testing methods are slow, expensive, brittle, and limited to specific predefined scenarios without considering high-level tasks and user intent. This disclosure describes a large language model (LLM) powered autonomous agent designed for comprehensively testing the User Experience (UX) of a software solution by taking into account high-level tasks and user intent. The LLM used to power the agent can be trained or fine-tuned on datasets of UI state representations paired with corresponding interaction steps, task instructions paired with successful interaction sequences, examples of efficient and inefficient interaction patterns, and descriptions of principles for good UI design. The agent can be leveraged for intelligent, adaptable, and comprehensive probing of UI functionalities via automatic generation of test cases and identifying usability issues by analyzing the test results. The agent-based approach can enable proactive detection of usability issues, robust testing of UI-related regressions, informed gating of releases, and deeper understanding of the performance from the user's perspective, thus complementing the backend testing performed via traditional testing methods.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Start, Johannes and Lunney, John, "LLM-driven Autonomous Agent for Comprehensive Automated User Experience Testing", Technical Disclosure Commons, (August 06, 2025)
https://www.tdcommons.org/dpubs_series/8431