Abstract
While users routinely use multimodal conversational agents powered by a large language model (LLM) as tools to obtain answers related to use or troubleshooting of a product (product support queries), general purpose LLMs may on occasion provide inconsistent and/or inaccurate responses. This can lead to a poor customer experience and a higher number of escalations to a human product support team. This disclosure describes techniques to train or fine-tune a generic multimodal large language model (LLM) to function as a specialized product support agent. The custom training uses help articles and documentation, product training/ testing videos, and other product-related materials to train the LLM to understand product nuances, common customer issues, and automatically identify troubleshooting steps. The training involves knowledge ingestion through fine-tuning a base model with curated data and reinforcement learning with human feedback to ensure continuous improvement in factors such as accuracy, clarity, and tone. A highly accurate and reliable AI-powered support mechanism is provided based on the fine-tuned model that can instantly resolve a significant portion of customer inquiries, enhance customer satisfaction, and reduce the burden on human support teams. The architecture supports real-time interaction and multimodal question-answering. Evaluation metrics can include escalation rate, resolution rate, response accuracy, customer satisfaction, etc.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Selvan, Manoj Muthamizh, "Training a Multimodal LLM as a Customized Product Support Agent", Technical Disclosure Commons, (September 10, 2025)
https://www.tdcommons.org/dpubs_series/8574