Abstract
Task-specific predictive machine learning models are used in many contexts. Such models require selection of input features and training of the model for a particular task with specific parameters. Building and maintaining such models is expensive and not scalable. This disclosure describes a universal predictive model (UPM) that is implemented using a dual encoder large language model (LLM). Both input features and task specifications are represented as natural language text or multimodal fragments that can be converted into embeddings by the LLM. A prediction model generates an output based on the feature embeddings and a task embedding. Fragments can be templatized and in the case of reuse, corresponding embeddings can be cached. The UPM can perform multi-step prediction, structured prediction, and prediction for different parameter values for the same task. The UPM reduces developer time to build predictive models, since a single model can process a variety of input features and generate a response to any type of prediction task.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Herwana, Cipta; Tomar, Shivangi; Subramanian, Sandeep; and Lath, Utkarsh, "A Large Scale, Natural Language-based Universal Predictive Model", Technical Disclosure Commons, (November 11, 2024)
https://www.tdcommons.org/dpubs_series/7515