Abstract
A system, method, and computer program product are disclosed for dynamically reorganizing multi-tier feature cache placement based on machine learning model deployment topology. The system includes a deployment controller, a cache controller, a cache memory, and multiple cache tiers configured to store feature data utilized during machine learning inference. Upon detection of a model deployment event, a feature dependency manifest associated with an incoming machine learning model is accessed to identify features required for inference operations. The cache controller evaluates feature placement across cache tiers and performs promotion of required features into a higher-performance cache tier while demoting non-required features to lower cache tiers. A cache-readiness condition is determined based on availability of required features within the higher-performance cache tier. Activation of the incoming machine learning model is selectively controlled based on satisfaction of the cache-readiness condition. The disclosed techniques reduce cache misses, improve feature availability, reduce inference latency, and enhance resource utilization within distributed machine learning serving environments.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Xia, Ruiyang and Qin, Hao, "SYSTEM, METHOD, AND COMPUTER PROGRAM PRODUCT FOR DYNAMICALLY REORGANIZING MULTI-TIER FEATURE CACHE PLACEMENT BASED ON MACHINE LEARNING MODEL DEPLOYMENT TOPOLOGY", Technical Disclosure Commons, (October 26, 2026)
https://www.tdcommons.org/dpubs_series/12407