Abstract
Interactive systems using large language models (LLMs) may experience inefficiencies where a user edit to a query can trigger a re-inference process, consuming resources and increasing latency. A backend system can predict potentially editable components in a user's query. An LLM may then generate a structured response containing static text and executable client-side functions or tool calls corresponding to these predicted components. A client device, such as a smartphone or personal computer, can parse this response to render an interactive interface. This can allow user modifications to interactive components to be processed locally on the client device, which may reduce latency and computational load by avoiding a re-inference by the backend model for certain adjustments.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Tran, Duc-Hieu and Hartmann, Florian, "Generating Structured Responses from Large Language Models for Client-Side Updates", Technical Disclosure Commons, ()
https://www.tdcommons.org/dpubs_series/10298