Abstract
The present disclosure provides a method (300) for mitigating semantic distractors in artificial intelligence (AI) systems. The method (300) includes receiving contextual information and one or more associated tasks, generating token embeddings corresponding to the contextual information, and processing the token embeddings through one or more neural-network layers to establish task understanding. The method (300) further includes deriving a task vector from one or more hidden-state representations generated by the neural network, generating relevance-gating values based on the task vector and contextual representations, and applying the relevance-gating values during attention computation to determine gated attention scores. The method (300) further includes suppressing semantically similar but task-irrelevant contextual information using the gated attention scores, dynamically updating the task vector during inference to adapt to changing tasks or subtasks, and generating one or more outputs using a language-model head operating on the gated attention outputs. The disclosed method enables task-conditioned attention allocation and semantic-distractor suppression, thereby improving contextual utilization, reasoning consistency, and inference accuracy in long-context artificial intelligence systems.
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Recommended Citation
KUMAR, SUMIT; MOHAN, SHIVAM; and SINHA, RAVI SHANKER KUMAR, "SELF-DERIVED TASK-CONDITIONED ATTENTION GATING FOR MITIGATING SEMANTIC DISTRACTORS", Technical Disclosure Commons, ()
https://www.tdcommons.org/dpubs_series/11798