Abstract
A lightweight, embedded computational graph engine provides fault-tolerant, concurrent data retrieval and processing across heterogeneous data backends. Operational tasks and data pipelines are modeled as execution nodes organized within a Directed Acyclic Graph (DAG). Programmatically defined task dependencies are dynamically resolved using topological sorting, allowing independent nodes to execute concurrently via a thread pool executor. Data payloads, represented as state mappings, are transferred between completing nodes and downstream dependent nodes to facilitate automated aggregation, joining, and transformation. Granular fault tolerance is achieved by classifying execution nodes as either critical or non-critical. When a non-critical node fails, the failure is caught, the node status is updated to reflect the error, and execution of remaining independent graph branches continues, thereby returning partial data. Conversely, when a critical node encounters an error, pending futures are safely canceled and execution halts to prevent invalid downstream computations.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
Cazzin, Davide and Mercier, Valentin, "Fault-Tolerant Concurrent Data Retrieval and Processing Engine Using Acyclic Dependency Graphs", Technical Disclosure Commons, (September 23, 2026)
https://www.tdcommons.org/dpubs_series/11847