Abstract
A system and method are disclosed that eliminate model loading in machine learning inference by enabling inference processes to operate directly on a shared, already‑resident model representation, wherein trained model parameters are stored as native binary structures in operating‑system managed shared memory and are accessed by inference processes through non‑owning typed memory views that require no allocation, copying, or deserialization; the shared memory region serves as the authoritative representation of the model and includes a control header and a plurality of model slots arranged in a double‑buffered configuration, wherein model updates are performed by writing a model into an inactive slot and publishing the model by atomically switching an active slot indicator in the control header using atomic operations, such that inference processes do not observe partially written models and lock‑free, zero‑downtime hot swapping of models while inference is active is enabled; the inference processes attach to the shared memory region without loading or instantiating the model and access model parameters using non‑owning typed memory views that reference existing memory and provide structural interpretation without allocation or interaction with garbage collection, and perform inference directly on the shared model representation using zero‑copy access independent of machine learning frameworks, whereby the disclosed system and method eliminate serialization, network communication, runtime ownership, and memory duplication and provide constant‑time inference startup, deterministic low‑latency inference, cross‑process model sharing, and continuous model evolution in high‑throughput systems.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Ahmed, Faiz, "System and Method for Zero Copy Machine Learning Inference Using Non Owning Shared Memory Model Views", Technical Disclosure Commons, ()
https://www.tdcommons.org/dpubs_series/10968