Abstract
In computing environments with a shared graphics processing unit, concurrent applications may compete for limited memory. The described techniques can provide a cooperative, cross-process memory arbitration framework. In such a framework, applications can provide allocation-level recovery metadata to a central coordinator process. This metadata can quantify the cost of restoring a piece of memory, for instance, by specifying the time for recovery and, for each available release mode, the restore time and the host memory occupied while the piece is released.
The coordinator can use this information to make memory release decisions based on a calculated, cross-application harm analysis, aiming to satisfy memory demands while helping to reduce the aggregate performance impact, such as user-visible latency or stalls, on the system. The coordinator can also choose, for each piece to be released, a release mode whose host-memory cost fits in the available host memory, schedule the restoration of released memory, including before its predicted next use, and include unmodified applications by interposing on their memory allocations and free-memory queries.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Yadav, Karan, "Cross-Process GPU Memory Arbitration Based on Recovery Cost Metadata", Technical Disclosure Commons, ()
https://www.tdcommons.org/dpubs_series/12551