Abstract
The present disclosure relates to a system and method for proactive schema cache warming triggered by failure-cooldown expiry during low-load periods in a real-time inference system.
The method comprises maintaining a cooldown tracker recording failed schema-fetch attempts and corresponding expiry timestamps. Upon expiry of a cooldown associated with a schema identifier, an internal warming event is generated instead of passively clearing the cooldown state. Proactive schema fetching is scheduled during periods of low inference load, with fetch operations prioritized based on historical request frequency. Upon successful retrieval, the schema is cached to enable immediate cache access for subsequent inference requests. If the proactive fetch fails, the cooldown is extended using exponential backoff, while the schema identifier is retained in a warming queue for processing during a subsequent low-load period. By converting failure-recovery information into a predictive cache-warming signal, the method reduces schema-fetch latency and avoids latency penalties on the first inference request after the schema becomes available.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
XIA, RUIYANG and QIN, HAO, "SYSTEM AND METHOD FOR PROACTIVE SCHEMA CACHE WARMING TRIGGERED BY FAILURE-COOLDOWN EXPIRY DURING LOW-LOAD PERIODS IN A REAL-TIME INFERENCE SYSTEM", Technical Disclosure Commons, (November 03, 2026)
https://www.tdcommons.org/dpubs_series/12704