Abstract

This document describes a system in which a computing device (e.g., smart watch, computerized glasses, smart ring, augmented reality headset, extended reality headset, hearable device, or other body-mounted devices or sensors) aids in accessibility by collecting environmental data from the user’s surroundings and providing specific feedback or direction to the user through different means of communication. This system may include a body area network of interconnected devices, such as XR glasses, a smartwatch, smartphone, and earbuds, that work in concert to provide a sensory augmentation experience. With explicit user consent, the system may collect environmental data, such as video content, audio content, biometric data, device and user location data, and proximity to other devices. The environmental data may be combined with other information, such as a user's calendar or crowdsourced information, to generate personalized, real-time guidance, direction, and suggestions, using a multi-modal predictive artificial intelligence model. This model may use techniques like a weighted-scoring algorithm to generate a confidence score for a predicted user intent, enabling proactive and anticipatory guidance. The computing device may output the information in the format that best suits the user’s abilities, such as audio cues, visual displays, or a unique "haptic language" or haptic vocabulary of vibrations. For example, users with visual impairments can receive the information through audible or haptic communication; users with hearing impairments can receive the information through a visual display or haptic notifications. In this way, the device may provide a highly accessible way for users to interact with their technology and environment.

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.

Download

Share

COinS