Abstract
Abstract (Part 6 of 27)
AI Agent Safety: Independent Risk Limits That Expected Profit Cannot Override
A frozen multi-agent plan can still be unsafe even when its identity and message history are stable. Different organizations face different losses. A system that maximizes expected global profit can still violate one owner’s loss limit, a residency prohibition, a safety interlock, or an unacceptable low-frequency catastrophic risk. Simple yes/no votes ignore asymmetric exposure, correlated failures, uncertain probabilities, model drift, and adverse results from earlier phases. A signed rating or persuasive narrative does not prove the facts, and an incorrect risk model can recommend continuation that no resource owner would accept.
This embodiment (Part 6 of a 27-part series; full details at https://zenodo.org/records/23940766) evaluates multi-agent risk, benefit-loss, and tail exposure under independent domain thresholds. After the plan is committed and frozen, the independent evaluator consumes authenticated consequence evidence and scenario results. Each domain publishes its own hard prohibitions, exposure caps, jurisdiction conditions, and revocation generations. The system computes expected loss, quantile and CVaR-style tail measures, maximum modeled loss, and chance-constraint breaches while avoiding silent double-counting of shared or contingent liabilities. Categorical safety, sovereignty, privacy, and physical-actuation constraints are applied before any economic optimization; a local veto is not defeated by higher group profit. Uncertainty, missing data, and uncalibrated models remain explicit. The evaluator emits a structured, non-effective next-step decision (hold, deny, revise, renegotiate, conditional proceed, substitute, or abort). Neither an expected-profit figure nor the evaluator’s output constitutes authority to act. A separately authorized issuer and each domain’s local Finality Sink must still approve any real continuation against current conditions.
The architecture therefore lets multi-agent systems measure asymmetric and tail risks across independent organizations while ensuring that no global optimization or evaluator recommendation can override a domain’s own protected limits.
Keywords: multi-agent systems, agentic AI, AI safety, AI governance, risk evaluation, benefit-loss analysis, tail-exposure evaluation, CVaR, quantile loss, independent domain thresholds, local veto, fail-closed systems, zero-trust agents, non-effective authorization, protected continuation control, cross-organizational collaboration, execution finality, asymmetric risk, catastrophic risk constraints, independent organizational risk policy, non-compensable prohibitions, risk-conditioned actor selection, tail-risk assessment for multi-agent plans, expected-profit override prevention, protected multi-agent risk boundary, domain-specific exposure caps, uncertainty-aware risk adjudication.
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License
Recommended Citation
Das, Sangam Kumar, "AI Agent Safety: Independent Risk Limits That Expected Profit Cannot Override", Technical Disclosure Commons, (November 05, 2026)
https://www.tdcommons.org/dpubs_series/12776