Abstract

The present disclosure relates to a method and a system for maintaining provenance-based separation of Artificial Intelligence (AI) inference data. The present disclosure suggests receiving an inference result record generated by an artificial intelligence (AI) inference platform. The present disclosure also suggests identifying a provenance attribute associated with the inference result record, the provenance attribute indicating an execution context of the artificial intelligence model. Based on the provenance attribute, the present disclosure suggests determining whether the inference result record corresponds to a production execution mode, a shadow execution mode, or an unrecognized execution mode. Upon determination, the present disclosure suggests generating a vector embedding for the inference result record. Thereafter, the present disclosure suggests routing the vector embedding corresponding to the production execution mode to a production vector index comprising a production similarity graph. Subsequently, the present disclosure suggests routing the vector embedding corresponding to the shadow execution mode to a shadow vector index physically separate from the production vector index and comprising a separate similarity graph. Further, the present disclosure suggests quarantining the inference result record having the unrecognized execution mode before generation of a vector embedding for the inference result record. Finally, the present invention suggests computing a cross-index cosine distance value between a production index centroid and a shadow index centroid as a model validation signal for determining semantic divergence between the production AI model and the experimental AI model.

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.

Download

Share

COinS