Abstract
Multilingual large language models (LLMs) are capable of delivering responses to user queries/ prompts, including spoken queries (where the language of the audio may be ambiguous), in various languages. In many contexts, it is important that the response from the multilingual LLM adhere to a pre-configured/ specified language. This disclosure describes three approaches to improve the language adherence of an LLM - zero shot prompting, supervised fine-tuning, and chain-of-thought reasoning. In zero short prompting, a language hint that specifies the languages known to the user is provided to the LLM along with the input audio. In supervised fine-tuning, a training set with a mixture of no language hint and correct/ incorrect/ mixed (both correct and incorrect) language hint is used to fine-tune the model. SFT with such a dataset can improve the likelihood of the LLM generating the output in the correct language. SFT can be further enhanced via COT prompting, where an explicit instruction to reason about the suitable output language as a first step, prior to generating the response to the prompt is included in the prompt. In some cases, a chain-of-thought prompt that is learned by SFT can be used, optionally with an explicit language signal.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Joshi, Kandarp; Kim, Clint; Aleksic, Petar; and Zhou, Lillian, "Improving Multilingual LLM Language Adherence via Language Signals and Fine-tuning", Technical Disclosure Commons, (May 14, 2025)
https://www.tdcommons.org/dpubs_series/8125