Abstract
Extending the capabilities of Large Language Models (LLMs) with functions or tools for environmental interaction has led to the emergence of the agent paradigm. In industry, training an LLM is not always feasible because of the scarcity of domain data, legal holds on proprietary customer data, rapidly changing business requirements, and the need to prototype new assistants. Artificial Intelligence (AI) agents can address such issues by relying on the zero-shot reasoning abilities of the underlying LLM and utilizing tools to explore and reason over customer data and respond to user requests. However, there are two concerns here: (I) acquiring large scale customer queries for agent testing is time-consuming, and (II) high reliance on the tool call sequence (or trajectory) followed by the agent to respond to user queries may lead to unexpected or incorrect behavior. To address this, a multi-agent framework for synthetic data generation and verification, referred to as 'MAG-V', is proposed herein that can facilitate generating a dataset of questions that mimic customer queries and reverse-engineering alternate questions from the responses for trajectory verification.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Mathur, Abhinav; Mallipeddi, Akshay; Ross, Joseph; Curtis, Kristal; and Gou, Liang, "MAG-V: A MULTI-AGENT FRAMEWORK FOR SYNTHETIC DATA GENERATION AND VERIFICATION", Technical Disclosure Commons, (November 20, 2025)
https://www.tdcommons.org/dpubs_series/8910