Smart home devices can be controlled via a smartphone or other device. However, when a user has several smart home devices in their home, identifying a particular device and accessing its controls can be cumbersome. This disclosure describes the use of augmented reality (AR) to detect and register smart home devices, and to automatically surface device control user interface for devices within a user’s view. Per the techniques, object recognition is used to detect the make and model of devices within a user’s camera view to enable the user to easily configure the device. Mesh network protocols are used to locate devices within the local network, relative to each other and with respect to the user’s smartphone or other device used for control. The user can simply select a device within the camera viewport and access the user interface to control the device.

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.