Abstract

Current accessibility features on display-based entertainment systems require visually impaired users to navigate through on-screen user interface elements sequentially. This linear, item-by-item traversal through each element in the user interface is inefficient for understanding the overall context of content-rich user interfaces that may include a significant amount of information for presenting to a user.

The disclosed technology utilizes a generative artificial intelligence (generative AI) model to analyze the content of a user interface for a display-based entertainment system that is presented on a screen of a computing device (e.g., a smart television). The generative AI model may generate a concise, audible summary of the content of the user interface. The computing device may play the summary on an audio device (e.g., a speaker, a microphone/speaker system) included in or coupled to the computing device for consumption by the user, who may be visually impaired. Following the summary, the user may provide voice commands as verbal input to an audio device (e.g., a microphone, a microphone/speaker system) included in or coupled to the computing device for direct navigation to specific on-screen elements (e.g., a target element) in the user interface. The voice commands may be processed by searching through a cached accessibility node tree of the user interface to identify the target element. The disclosed technology replaces laborious, sequential navigation of a user interface of a display-based entertainment system with a holistic screen overview of the user interface and direct, voice-activated focus control, significantly reducing content discovery time for visually impaired users.

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.

Download

Share

COinS