Innopolis University Gets Three Patents for AI Solutions for Text and Visual Attention

Researchers at the AI Research Center of Innopolis University have obtained three patents for AI solutions related to text and visual attention. The first patent covers a system for predicting human gaze during reading, which can forecast eye movements for texts in English, Korean, German and ten other languages using a unified neural network architecture. It combines a multilingual language model and a transformer model for predicting gaze fixations, trained on eye-tracking recordings from native speakers of different languages. The system generates synthetic gaze fixation sequences and evaluates them by word skipping probability and the number of fixations during first reading. The second patent concerns the evaluation of gaze generation during reading, addressing the lack of unified protocols for comparing sequential gaze fixations on text produced by generative models. The proposed approach compares gaze trajectories by spatial and temporal characteristics and assesses different eye movement features at the level of entire text, individual words, and sentences. The third patent is for a method of training language models with synthetic gaze trajectories, integrating visual attention data into reinforcement learning from human feedback. The invention includes generating synthetic gaze fixation sequences and visual attention features, and implementing a reinforcement learning algorithm based on human feedback that accounts for human attention patterns. According to the university, this technology accelerates AI training by 1.5–2 times, reduces computational costs, reproduces human attention patterns with up to 93% accuracy, and works even for languages with low data volumes.