Abstract

An on-screen keyboard is operated with the eyes through a remote eye tracker. Instead of a fixed dwell time, selection is driven by one probabilistic model, shared by all components. The model combines (i) a language-model prior over the next symbol, (ii) a spatial likelihood of the gaze around each key and (iii) a model of where a saccade lands relative to its intended target. The dwell timer starts at the end of the saccade that brings the eye onto a key. The required fixation time is a monotonic function of the key's prior probability and of the spatial ambiguity of the gaze. Key areas expand invisibly towards probable keys, while each key keeps a protected core. Within one fixation only one key can be selected; the same key can be repeated only by holding the gaze on it for a longer repeat time. After a long saccade, selection of the landed key is held back only when the landing model gives a low probability that the saccade was aimed at that key, typically because a more probable key lies further along the same direction. In that case the corrective saccade has time to arrive. A pupil-based load index, compensated for the luminance of the rendered screen with a dynamic per-user light-reflex model, can lengthen all dwell times. A noisy-channel word corrector uses the same gaze and landing likelihoods. Learned phrases are proposed as ghost text and accepted only by an uninterrupted dwell that every saccade of 1° or more resets, so that reading them word by word does not accept them. The disclosure also covers the speech-output interaction and an offline replay method for evaluating selection rules on recorded sessions.

Creative Commons License

Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.

Share

COinS