Efficient Nano‑Optical Eye Tracking for Smart Glasses

  • 1Princeton University
  • 2Google Inc, Austria
  • 3Google Inc, Switzerland
SIGGRAPH Asia 2026

Abstract

Eye tracking is the primary input method for smart glasses, and it runs continuously on a processor that is shared with SLAM, rendering, and hand tracking. Glint-based methods transfer poorly to this form factor: frame-mounted LEDs illuminate the eye at oblique angles, so their glints are easily occluded by eyelashes, and outdoor sunlight further degrades the signal. Without glints, a rotation of the eye and a slip of the glasses can each move the pupil to nearly the same position on the sensor. In this work, we resolve this ambiguity in the optics rather than in the network, by jointly optimizing a nanophotonic metasurface, the sensor pixel pitch, and the network for both accuracy and computational cost. Our method achieves a 3D gaze error of 0.75° at 480 MMACs, an accuracy comparable to that of De2Gaze at 14× less compute. We evaluated a fabricated prototype with 20 subjects, using motorized stages to emulate slippage. Under axial slippage, its gaze error is 1.17°, compared with 2.02° for a hyperbolic lens.

Citation

				
					@inproceedings{sun2026nanoeyetracking,
  title     = {Efficient Nano-Optical Eye Tracking for Smart Glasses},
  author    = {Sun, Jipeng and Yuan, Lekang and Guseinov, Ruslan and Bickel, Bernd
               and Beeler, Thabo and Auzinger, Thomas and Heide, Felix},
  booktitle = {SIGGRAPH Asia 2026 Conference Papers},
  year      = {2026},
  publisher = {Association for Computing Machinery}
}