Real-Time Eye Movement Tracking and Prediction Using Machine Learning for Vision-Based Applications
DOI:
https://doi.org/10.67887/PD2601202003Keywords:
Eye tracking, gaze prediction, machine learning, real-time systems, mobile-net, vision applicationsAbstract
Introduction: Real-time gaze tracking is essential for human–computer interaction, automotive safety, assistive systems, and medical diagnostics. Traditional hardware-based trackers are costly and intrusive, while many machine learning (ML) models are too computationally demanding for real-time use.
Methods: This study develops and evaluates a lightweight deep learning framework for gaze direction prediction using three architectures: a baseline Convolutional Neural Network (CNN), a single-eye MobileNetV2, and a dual-eye MobileNetV2. Performance was assessed using accuracy, precision, recall, F1-score, AUC, latency, and frames per second (FPS) to determine deployment feasibility.
Results: This show that the CNN achieved 73.23% accuracy, an AUC of 0.81, and a latency of 0.56 ms per sample. The single-eye MobileNetV2 model reached 75.02% accuracy, with an AUC of 0.80, high recall for "Not Looking Ahead" (0.90), and low latency of 1.30 ms per sample, achieving a throughput of 772 FPS. The dual-eye MobileNetV2 model achieved the highest accuracy (86.35%) and AUC (0.93), with excellent precision and recall for both classes, but suffered from higher latency (5615 ms per sample) and a throughput of 0.18 FPS. The single-eye MobileNetV2 thus offers the best balance between predictive performance and real-time operation.
Conclusion: The findings demonstrate that lightweight and interpretable ML models can deliver accurate, low-latency gaze tracking suitable for AR/VR systems, driver monitoring, and healthcare applications.
References
Ozhan FO, Aygun U, Sahin A, Urey H. Dynamic accommodation measurement using Purkinje reflections and machine learning. Scientific Reports. 2023 Dec 7;13(1):21625. https://doi.org/10.1038/s41598-023-47572-0
Jin X, Chai S, Tang J, Zhou X, Wang K. Eye-tracking in ar/vr: A technological review and future directions. IEEE Open Journal on Immersive Displays. 2024 Aug 27;1:146-54. https://doi.org/10.1109/OJID.2024.3450657
Burova A, Mäkelä J, Hakulinen J, Keskinen T, Heinonen H, Siltanen S, Turunen M. Utilizing VR and gaze tracking to develop AR solutions for industrial maintenance. In Proceedings of the 2020 CHI conference on human factors in computing systems 2020 Apr 21 (pp. 1-13). https://doi.org/10.1145/3313831.3376405
Van Damme S, Vega MT, De Turck F. Human-centric quality management of immersive multimedia applications. In 2020 6th IEEE Conference on Network Softwarization (NetSoft) 2020 Jun 29 (pp. 57-64). IEEE. https://doi.org/10.1109/NetSoft48620.2020.9165335
Bozkir E, Geisler D, Kasneci E. Person independent, privacy preserving, and real time assessment of cognitive load using eye tracking in a virtual reality setup. In 2019 IEEE conference on virtual reality and 3D user interfaces (VR) 2019 Mar 23 (pp. 1834-1837). IEEE. https://doi.org/10.1109/VR.2019.8797758
Shabbir MN, Linh DT. AI and machine learning applications in wearable health devices. Wearable Technology. 2024;5(1):3123. https://doi.org/10.54517/wt3123
Bektaş K, Strecker J, Mayer S, Garcia K. Gaze-enabled activity recognition for augmented reality feedback. Computers & Graphics. 2024 Apr 1;119:103909. https://doi.org/10.1016/j.cag.2024.103909
Lu S, Tan Z, Kong S, Zhang D. Cost-effective gaze tracking system based on polymer fiber specklegrams. Optics Letters. 2024 Sep 3;49(18):5027-30. https://doi.org/10.1364/OL.531946
Hallgarten P, Sendhilnathan N, Zhang T, Sood E, Jonker TR. Gears: Generalizable multi-purpose embeddings for gaze and hand data in vr interactions. In Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization 2024 Jun 22 (pp. 279-289). https://doi.org/10.1145/3627043.3659551
Brousseau B, Rose J, Eizenman M. Hybrid eye-tracking on a smartphone with CNN feature extraction and an infrared 3D model. Sensors. 2020 Jan 19;20(2):543. https://doi.org/10.3390/s20020543
Ahmed I, Ateeb M. Enhancing Education Accessibility through Eye Gaze Technology (EGT). In 2025 19th International Conference on Ubiquitous Information Management and Communication (IMCOM) 2025 Jan 3 (pp. 1-8). IEEE. https://doi.org/10.1109/IMCOM64595.2025.10857579
Kumari N, Ruf V, Mukhametov S, Schmidt A, Kuhn J, Küchemann S. Mobile eye-tracking data analysis using object detection via YOLO v4. Sensors. 2021 Nov 18;21(22):7668. https://doi.org/10.3390/s21227668
Darapaneni N, Prakash MD, Sau B, Madineni M, Jangwan R, Paduri AR, KP J, Belsare M, Madhavankutty P. Eye Tracking Analysis Using Convolutional Neural Network. In 2022 Interdisciplinary Research in Technology and Management (IRTM) 2022 Feb 24 (pp. 1-8). IEEE. https://doi.org/10.1109/IRTM54583.2022.9791826
Chinsatit W, Saitoh T. CNN‐Based Pupil Center Detection for Wearable Gaze Estimation System. Applied Computational Intelligence and Soft Computing. 2017;2017(1):8718956. https://doi.org/10.1155/2017/8718956
Gunawardena N, Ginige JA, Javadi B, Lui G. Performance analysis of cnn models for mobile device eye tracking with edge computing. Procedia Computer Science. 2022 Jan 1;207:2291-300. https://doi.org/10.1016/j.procs.2022.09.288
Alzubaidi LH, Hussein AH, Alkhafaji MA, Shilpa N, NP T. Efficient Real-Time Eye Gaze Tracking Detection for Human-Computer Integration Using Advanced Techniques. In 2023 3rd International Conference on Mobile Networks and Wireless Communications (ICMNWC) 2023 Dec 4 (pp. 1-6). IEEE. https://doi.org/10.1109/ICMNWC60182.2023.10436021
Chen HH, Hwang BJ, Wu JS, Liu PT. The effect of different deep network architectures upon CNN-based gaze tracking. Algorithms. 2020 May 19;13(5):127. https://doi.org/10.3390/a13050127
Khan W, Ishrat M, Faisal SM, Ahmad F, Nabilal KV, Sharma YK. A robust framework for topology-based anomaly detection in attributed networks using graph attention networks, substructure analysis, and data augmentation. International Journal of Data Science and Analytics. 2025 Nov;20(7):6715-28. https://doi.org/10.1007/s41060-025-00848-2
Khan W, Ebrahim N. Anogat-sparse-tl: A hybrid framework combining sparsification and graph attention for anomaly detection in attributed networks using the optimized loss function incorporating the twersky loss for improved robustness. Knowledge-Based Systems. 2025 Feb 28;311:113144. https://doi.org/10.1016/j.knosys.2025.113144
Donuk K, Ari A, Hanbay D. A CNN based real-time eye tracker for web mining applications. Multimedia Tools and Applications. 2022 Nov;81(27):39103-20. https://doi.org/10.1007/s11042-022-13085-7
Park J, Park S, Cha H. Gazel: Runtime gaze tracking for smartphones. In 2021 IEEE International Conference on Pervasive Computing and Communications (PerCom) 2021 Mar 22 (pp. 1-10). IEEE.
https://doi.org/10.1109/PERCOM50583.2021.9439113
Martinez, J.P. Gaze Direction Detection: Dataset with separated eyes of different people, looking or not at the camera. 2025; Available from: https://www.kaggle.com/datasets/estopadilla/gaze-direction-detection.
Liu W, Duinkharjav B, Sun Q, Zhang SQ. Fovealnet: Advancing ai-driven gaze tracking solutions for efficient foveated rendering in virtual reality. IEEE Transactions on Visualization and Computer Graphics. 2025 Mar 11. https://doi.org/10.1109/TVCG.2025.3549577
Maruyama R, Takahashi S, Hagiwara T. A Virtual Reality Driving Simulator with Gaze Tracking for Analyzing Driver’s Behavior. In 2022 IEEE 4th Global Conference on Life Sciences and Technologies (LifeTech) 2022 Mar 7 (pp. 144-145). IEEE. https://doi.org/10.1109/LifeTech53646.2022.9754947
Biradarpatil RS, Chinmayee BL, Hegde S, Mallibhat K, Mudenagudi U. Eye Gaze Tracking Towards User Attention Analysis. In 2024 5th International Conference for Emerging Technology (INCET) 2024 May 24 (pp. 1-8). IEEE. https://doi.org/10.1109/INCET61516.2024.10593440
Jawalkar SK. The Metaverse: A New Paradigm for Social and Commercial Interaction. IJLRP-International Journal of Leading Research Publication.;5(2). https://doi.org/10.5281/zenodo.14982529
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Hadia Tazeem (Author)

This work is licensed under a Creative Commons Attribution 4.0 International License.