Abstract
Large volumes of data, characterized by large variety and high update velocities, pose challenges in terms of storage, application of concurrently occurring frequent updates, and serving processes that require the most accurate version of the data simultaneously. In most current schemes, it is not possible to guarantee all of these characteristics and a relaxing one or more requirements is necessary. The present disclosure describes a scalable, easy-to-maintain metadata mechanism that is fast and efficient to update, and can provide all the above guarantees on data. The metadata maintains lightweight validity markers, and simple algebra is performed thereof to surface the most up to date and accurate data while enabling constant updates to the data in a non-blocking fashion.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Pothineni, Nagaraju; Raut, Nitin; Joshi, Nikhil J.; and Menon, Nikhil, "Stateful metadata for big data", Technical Disclosure Commons, (April 10, 2019)
https://www.tdcommons.org/dpubs_series/2134