Inventor(s)

Abstract

A computer-implemented method (600) of processing structured data using one or more artificial intelligence models, including a large language model, is provided. A structured dataset (310) is received (602), including a plurality of records, each having a record identifier, and associated schema information. Relationships between the records are identified (330) based on the schema information. A plurality of data chunks (420, 430, 440) is generated (608) by partitioning the records in accordance with the identified relationships such that related records are preserved within a common data chunk. A chunk manifest identifying the record identifiers included in each data chunk is generated (610). Each data chunk is processed (612) using the large language model to generate chunk-level analysis results. The record identifiers are compared (614) with the chunk manifest to identify any record not included in any processed data chunk, and each identified record is reprocessed (616) using the large language model. The chunk-level analysis results are aggregated (618), duplicate and conflicting analysis results are resolved, and a completeness certificate indicating verified record coverage is generated (620). A verified analysis result is output together with the completeness certificate (622).

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.

Share

COinS