Abstract
A system can assist large language model (LLM) outputs in adhering to specific coding styles, potentially reducing a need for manual prompt engineering or model retraining. This disclosure describes a system for automated, test-driven generation of prompts. The method may use a reference artifact, such as a source code commit, as ground-truth data. An LLM can be tasked to generate an initial prompt intended to reproduce the reference. The system can then enter an iterative loop of executing the prompt, programmatically comparing the output against the reference, and using any resulting differences to instruct the LLM to refine the prompt. This process can be repeated until the prompt reproduces the reference to a desired degree. This can result in an evaluated, reusable prompt that codifies the underlying style for application to similar tasks.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Hibner, Artur, "Test-Driven Generation of Validated LLM Prompts via Iterative Refinement", Technical Disclosure Commons, ()
https://www.tdcommons.org/dpubs_series/9938