Abstract
Conflict-free replicated data types, CRDTs, (https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type) are used to maintain consistent metadata/counters (file-share/folder/container-specific space used, iops served etc..) in an eventually consistent system. These counters need to be strongly consistent (compared to the user data) since they will be used management actions like user billing, quota enforcement. For the client-initiated data path, a straightforward application of CRDTs suffices, each proxy that serves a client request stores the counter under a key corresponding to the node on which the proxy runs.
For system maintenance, a distributed system can generate background writes/deletes that are often duplicates and separated over a period of time in the order days. This invention solves the problem of efficiently de-duplicating these system-initiated counter updates when the system contains millions of counters.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 License.
Recommended Citation
Thirumalai, Dhanwa, "System and Method for Maintaining Consistent Container-Specific Counters in a Distributed Eventually Consistent Storage System Containing Billions of Objects/Files", Technical Disclosure Commons, (December 19, 2016)
https://www.tdcommons.org/dpubs_series/354