Abstract
Disclosed is an Agentic NLE Interface featuring a "System-Wide Copilot" with deep context awareness and direct tool execution capabilities. The core innovation is "RAG on the Timeline," a local Retrieval Augmented Generation pipeline focused on the NLE timeline state (OTIO). The application continuously vectorizes the OTIO state (structure, metadata, semantic content) into a local vector database. This enables the AI Copilot to understand the project context and answer semantic queries ("Find the shot where..."). The RAG pipeline integrates Vision-Language Models (VLMs) for analyzing current frame pixel data. The Copilot functions as an agent with access to executable tools mapped to internal functions (e.g., trimClip), allowing the AI to directly manipulate the OTIO/CRDT data structure.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Judson, Matthew D., "The Agentic NLE Interface: System-Wide Copilot with "RAG on the Timeline" and Direct Tool Execution", Technical Disclosure Commons, (December 02, 2025)
https://www.tdcommons.org/dpubs_series/8969