Abstract
A user’s creative intent for image or video editing expressed in natural language, e.g., ‘make the greenery look really vibrant,’ cannot currently be directly translated into the series of discrete, technical operations required by editing libraries. This situation exists because software libraries and application programming interfaces (APIs) are not optimized for an environment where artificial intelligence (in turn tasked by end-users) is the primary customer. This disclosure describes techniques of application programming interface (API) and software library design that can bridge the gap between the abstract requests of AI and the rigid, explicit nature of libraries, making software development more tractable and responsive to AI-driven workflows. For example, rather than using AI to create shader-like effects, the techniques leverage large language models (LLMs) to actually write the shaders that run on GPUs. The techniques enable the AI-driven creation of code that can edit media at the pixel level to generate various effects described in natural language.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Pitts, Colvin, "AI-driven Special Effects Generation and Application Framework for Visual Content", Technical Disclosure Commons, (September 29, 2025)
https://www.tdcommons.org/dpubs_series/8647