Abstract
A system for prioritizing context in large scale language model conversations includes a token importance identification module (215) which is configured to receive conversation input and determine token importance through user tagging and system-driven preference learning. A token scoring module (220) is configured to assign an importance score to each token based on the determined importance and rank each of the tokens in descending order by the importance score. A context downscaling module (225) is configured to retain tokens with importance scores above a threshold and downscale tokens with importance scores below the threshold using summarization, compression, or removal. A persistent context memory (125) is configured to store tokens identified as having high importance for subsequent language model interactions across session boundaries.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Sinha, Ravi Shankar Kumar; SARDA, VINAY; and Kumar, Sumit, "SYSTEM AND METHOD FOR PRIORITIZING CONTEXT IN LARGE-SCALE LANGUAGE MODEL CONVERSATIONS VIA TOKEN IMPORTANCE IDENTIFICATION AND DOWNSCALING", Technical Disclosure Commons, ()
https://www.tdcommons.org/dpubs_series/10926