Vision Augmentation

2025

1. IEEE TVCG

   

   J10 ChromaGazer: Unobtrusive Visual Modulation using Imperceptible Color Vibration for Visual Guidance

   Rinto Tosa, Shingo Hattori, Yuichi Hiroi, Yuta Itoh, and Takefumi Hiraki

   IEEE Transactions on Visualization and Computer Graphics, Mar 2025

   Abs Bib DOI arXiv PDF

   Visual guidance (VG) plays an essential role in directing user attention in virtual reality (VR) and augmented reality (AR) environments. However, traditional approaches rely on explicit visual annotations, which often compromise visual clarity and increase user cognitive load. To address this issue, we propose an unobtrusive VG technique based on color vibration, a phenomenon in which rapidly alternating colors at frequencies above 25 Hz are perceived as a single intermediate color. Our work explores a perceptual state that exists between complete color fusion and visible flicker, where color differences remain detectable without conscious awareness of vibration. Through two experimental studies, we first identified the thresholds separating complete fusion, this intermediate perceptual state, and visible flicker by systematically varying color vibration parameters. Subsequently, we applied color vibrations with derived thresholds to natural image regions and validated their attention-guiding capabilities using eye-tracking measurements. The results demonstrate that controlled color vibration successfully directs user attention while maintaining low cognitive demand, providing an effective method for implementing unobtrusive VG in VR and AR systems.

   @article{tosa2025chromagazer,
     month = mar,
     author = {Tosa, Rinto and Hattori, Shingo and Hiroi, Yuichi and Itoh, Yuta and Hiraki, Takefumi},
     journal = {IEEE Transactions on Visualization and Computer Graphics},
     title = {ChromaGazer: Unobtrusive Visual Modulation using Imperceptible Color Vibration for Visual Guidance},
     year = {2025},
     volume = {31},
     number = {5},
     pages = {3450-3458},
     keywords = {Visual Guidance;Imperceptible Color Vibration;Color Perception;Augmented Reality},
     doi = {10.1109/TVCG.2025.3549173}
   }

2. ACM VRST

   CF17 Saccaidance: Saccade-Aware Pattern Embedding for Gaze Guidance on High-Speed Displays

   Masahiro Nara, Ryusuke Miyazaki, Yuichi Hiroi, Takefumi Hiraki, Yuta Itoh, and Shio Miyafuji

   In Proceedings of the 2025 31st ACM Symposium on Virtual Reality Software and Technology, Nov 2025

   Abs Bib DOI PDF

   Gaze guidance is essential for directing user attention to specific areas of interest. However, conventional visual cues generate persistent visual noise that hinders concentration during tasks. We propose Saccaidance, a gaze-guidance method that appears only when users move their gaze. Saccaidance employs temporal additive color mixing and 480 Hz high-speed displays to shift the color phase of guidance patterns. This renders the patterns barely visible during fixation and makes them appear transiently when users move their gaze as a color-breaking effect. This intermittent gaze guidance appears only during gaze transitions, providing effective guidance without interfering with focused work or requiring eye-tracking hardware. We conducted experiments with 24 participants under four conditions that involved search tasks: an unmodified baseline, conventional explicit guidance, and our proposed method using oval and radial patterns. The results show that our approach effectively constrains the exploration area while preserving subjective naturalness. We also outline application scenarios of our method, including document highlighting.

   @inproceedings{nara2025saccaidance,
     pages = {1--11},
     author = {Nara, Masahiro and Miyazaki, Ryusuke and Hiroi, Yuichi and Hiraki, Takefumi and Itoh, Yuta and Miyafuji, Shio},
     title = {Saccaidance: Saccade-Aware Pattern Embedding for Gaze Guidance on High-Speed Displays},
     year = {2025},
     isbn = {9798400721182},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3756884.3766051},
     doi = {10.1145/3756884.3766051},
     booktitle = {Proceedings of the 2025 31st ACM Symposium on Virtual Reality Software and Technology},
     articleno = {36},
     numpages = {11},
     keywords = {Gaze Guidance, Temporal Additive Color Mixing, Color Vibration, Augmented Reality},
     series = {VRST '25},
     month = nov,
   }

3. Augmented Humans

   CF11 ChromaGazer-HMD: Visual Modulation using Unobtrusive Color Vibration for Gaze Guidance with Head-Mounted Displays

   Youfang Shen, Rinto Tosa, Yuji Hatada, Yuichi Hiroi, Takefumi Hiraki, and Takeshi Naemura

   In Proceedings of the Augmented Humans International Conference 2025, Mar 2025

   Abs Bib DOI PDF

   VR content often involves vast spaces for users to explore, making it uncertain whether they will notice areas or objects highlighted by content creators. While various methods for guiding users’ gaze have been proposed, those that significantly alter the appearance of the content can reduce its naturalness and negatively impact the user experience. In this study, we explored a Visual Guidance (VG) approach using color vibration, based on the human visual system’s inability to perceive rapid chromatic changes. We evaluated its applicability to HMD environments and introduced a method called complementary color vibration, where opposite-phase vibrations are applied to the left and right displays of the HMD. We further investigated optimal parameters for VG and assessed its effectiveness in search tasks. Our experiments revealed that color vibration in HMD environments significantly reduced search times and areas compared to conditions without guidance, while maintaining the naturalness of the content. Furthermore, complementary color vibration was found to preserve naturalness significantly better than conventional synchronous color vibration. These findings indicate that VG using color vibration is effective in HMD environments and that complementary color vibration is less obtrusive than traditional synchronous color vibration.

   @inproceedings{shen2025chromagazerhmd,
     author = {Shen, Youfang and Tosa, Rinto and Hatada, Yuji and Hiroi, Yuichi and Hiraki, Takefumi and Naemura, Takeshi},
     title = {ChromaGazer-HMD: Visual Modulation using Unobtrusive Color Vibration for Gaze Guidance with Head-Mounted Displays},
     year = {2025},
     isbn = {9798400715662},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     doi = {10.1145/3745900.3746090},
     booktitle = {Proceedings of the Augmented Humans International Conference 2025},
     pages = {43–54},
     numpages = {12},
     month = mar,
   }

2024

1. SIGGRAPH Poster

   

   CP9 Measurement of the Imperceptible Threshold for Color Vibration Pairs Selected by using MacAdam Ellipse

   Shingo Hattori, Yuichi Hiroi, and Takefumi Hiraki

   In ACM SIGGRAPH 2024 Posters, Denver, CO, USA, Jul 2024

   Award 1st Place, Student Research Competition (Undergraduate), SIGGRAPH 2024.

   Abs Bib DOI arXiv

   We propose an efficient method for searching for color vibration pairs that are imperceptible to the human eye based on the MacAdam ellipse, an experimentally determined color-difference range that is indistinguishable to the human eye. We created color pairs by selecting eight colors within the sRGB color space specified by the ellipse, and conducted experiments to confirm the threshold of the amplitude of color vibration amplitude at which flicker becomes imperceptible to the human eye. The experimental results indicate a general guideline for acceptable amplitudes for pair selection.

   @inproceedings{hattori2024macadam,
     pages = {1--2},
     month = jul,
     author = {Hattori, Shingo and Hiroi, Yuichi and Hiraki, Takefumi},
     title = {Measurement of the Imperceptible Threshold for Color Vibration Pairs Selected by using MacAdam Ellipse},
     year = {2024},
     isbn = {9798400705168},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3641234.3671041},
     doi = {10.1145/3641234.3671041},
     booktitle = {ACM SIGGRAPH 2024 Posters},
     articleno = {68},
     numpages = {2},
     keywords = {MacAdam ellipse, color perception, imperceptible color vibration},
     location = {Denver, CO, USA},
     series = {SIGGRAPH '24},
   }

2021

1. Augmented Humans

   CF3 CircadianVisor: Image Presentation with an Optical See-Through Display in Consideration of Circadian Illuminance

   Takumi Tochimoto, Yuichi Hiroi, and Yuta Itoh

   In Proceedings of the Augmented Humans International Conference 2021, Rovaniemi, Finland, Feb 2021

   Abs Bib DOI PDF

   In modern society, the impact of nighttime artificial lighting on the human sleep/wake cycle (the circadian rhythm), has long been an important issue. In augmented reality, such health hazards should be prevented if we are to become a society that wears optical see-through head-mounted displays (OST-HMDs) on a daily basis. We present CircadianVisor, an OST display system that controls circadian performance. Our system combines an OST-HMD with a liquid crystal (LC) shutter and a spectrometer to control the circadian illuminance (CIL, biolux) of light incident on the user’s eyes. To prevent the CIL at the eyes from exceeding the threshold, correct for the displayed image based on RGB values, and adjust the transmittance of the LC visor to pass through the environment light based on spectral measurements, we build a proof-of-concept system to evaluate the feasibility of the system’s CIL control and test it with a spectrometer installed at the user’s viewpoint. The evaluation shows that the CIL at the user’s viewpoint can be kept below the threshold.

   @inproceedings{tochimoto2021circadian,
     month = feb,
     author = {Tochimoto, Takumi and Hiroi, Yuichi and Itoh, Yuta},
     title = {CircadianVisor: Image Presentation with an Optical See-Through Display in Consideration of Circadian Illuminance},
     year = {2021},
     isbn = {9781450384285},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3458709.3458938},
     doi = {10.1145/3458709.3458938},
     booktitle = {Proceedings of the Augmented Humans International Conference 2021},
     pages = {66–76},
     numpages = {11},
     keywords = {Augmented reality, Blue-light-blocking glasses, Circadian rhythm, Head-mounted displays},
     location = {Rovaniemi, Finland},
     series = {AHs '21},
   }

2020

1. Augmented Humans

   CF2 DehazeGlasses: Optical Dehazing with an Occlusion Capable See-Through Display

   Yuichi Hiroi, Takumi Kaminokado, Atsushi Mori, and Yuta Itoh

   In Proceedings of the Augmented Humans International Conference, Kaiserslautern, Germany, Mar 2020

   Abs Bib DOI PDF

   We present DehazeGlasses, a see-through visual haze removal system that optically dehazes the user’s field of vision. Human vision suffers from a degraded view due to aspects of the scene environment, such as haze. Such degradation may interfere with our behavior or judgement in daily tasks. We focus on hazy scenes as one common degradation source, which whitens the view due to certain atmospheric conditions. Unlike typical computer vision systems that process recorded images, we aim to realize a see-through glasses system that can optically manipulate our field of view to dehaze the perceived scene. Our system selectively modulates the intensity of the light entering the eyes via occlusion-capable optical see-through head-mounted displays (OST-HMD). We built a proof-of-concept system to evaluate the feasibility of our haze removal method by combining a digital micromirror device (DMD) and an OST-HMD, and tested it with a user-perspective viewpoint camera. A quantitative evaluation with 80 scenes from a haze removal dataset shows that our system realizes a dehazed view that is significantly closer to the ground truth scene compared to the native view under a perceptual image similarity metric. This evaluation shows that our system achieves perceptually natural haze removal while maintaining the see-through view of actual scenes.

   @inproceedings{hiroi2020dehaze,
     pages = {1--11},
     month = mar,
     author = {Hiroi, Yuichi and Kaminokado, Takumi and Mori, Atsushi and Itoh, Yuta},
     title = {DehazeGlasses: Optical Dehazing with an Occlusion Capable See-Through Display},
     year = {2020},
     isbn = {9781450376037},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3384657.3384781},
     doi = {10.1145/3384657.3384781},
     booktitle = {Proceedings of the Augmented Humans International Conference},
     articleno = {3},
     numpages = {11},
     keywords = {Vision Augmentation, Occlusion-Capable HMD, Head-Mounted Displays, Haze Removal, Augmented Reality},
     location = {Kaiserslautern, Germany},
     series = {AHs '20},
   }

2. CVPR Workshop

   CW1 DehazeGlasses: Optical Dehazing with an Occlusion Capable See-Through Display

   Yuichi Hiroi, Takumi Kaminokado, Atsushi Mori, and Yuta Itoh

   In 9th IEEE International Workshop on Computational Cameras and Displays (CCD, CVPR Workshops), 2020

   Workshop presentation

   Bib Website

   @inproceedings{hiroi2020dehaze-workshop,
     author = {Hiroi, Yuichi and Kaminokado, Takumi and Mori, Atsushi and Itoh, Yuta},
     title = {DehazeGlasses: Optical Dehazing with an Occlusion Capable See-Through Display},
     booktitle = {9th IEEE International Workshop on Computational Cameras and Displays (CCD, CVPR Workshops)},
     year = {2020},
     note = {Workshop presentation},
   }

2017

1. Augmented Human

   CF1 AdaptiVisor: assisting eye adaptation via occlusive optical see-through head-mounted displays

   Yuichi Hiroi, Yuta Itoh, Takumi Hamasaki, and Maki Sugimoto

   In Proceedings of the 8th Augmented Human International Conference, Silicon Valley, California, USA, Mar 2017

   Best paper Best Paper, 3rd Place, Augmented Human 2017.

   Abs Bib DOI PDF Video

   Brightness adaptation is a fundamental ability in human visual system, and adjusts various levels of darkness and light. While this ability is continuously used, and it can mostly handle sudden lighting changes in the environment, the adaptation could still take several minutes. Moreover, during the adaptation, the color perception changes as well. This slow reactivity and perception change of the eyes could lead to mistakes for tasks performed in dazzling or temporally high-contrast environments such as when driving into the sun or during a welding process.We present AdaptiVisor, a vision augmentation system that assists the brightness adaptation of the eye. Our system selectively modulates the intensity of the light coming into the eyes via occlusion-capable Optical See-Through Head-Mounted Displays (OST-HMD). An integrated camera captures highlights and brightness in the environment via high-dynamic range capture, and our display system selectively dims or enhances part of field of views so that the user would not perceive rapid brightness changes. We build a proof-of-concept system to evaluate the feasibility of the adaptation assistance by combining a transmissive LCD panel and an OST-HMD, and test it with a user-perspective, view-point camera. The evaluation shows that the system decreases the overexposed area in a scene to 1/15th, and enhances the color by reducing majorly underexposed area to half. We also include a preliminary user trial and it indicates that the system also works for real eyes for the HMD part and to some extent for the LCD.

   @inproceedings{hiroi2017adaptivisor,
     pages = {1--9},
     month = mar,
     author = {Hiroi, Yuichi and Itoh, Yuta and Hamasaki, Takumi and Sugimoto, Maki},
     title = {AdaptiVisor: assisting eye adaptation via occlusive optical see-through head-mounted displays},
     year = {2017},
     isbn = {9781450348355},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3041164.3041178},
     doi = {10.1145/3041164.3041178},
     booktitle = {Proceedings of the 8th Augmented Human International Conference},
     articleno = {9},
     numpages = {9},
     keywords = {AdaptiVisor, augmented reality, brightness adaptation, head-mounted displays, near-eye displays, occlusive HMD, optical see-through, vision augmentation},
     location = {Silicon Valley, California, USA},
     series = {AH '17},
   }

2016

1. SIGGRAPH E-tech

   

   CD1 Laplacian vision: augmenting motion prediction via optical see-through head-mounted displays and projectors

   Yuta Itoh, Yuichi Hiroi, Jiu Otsuka, Maki Sugimoto, Jason Orlosky, Kiyoshi Kiyokawa, and Gudrun Klinker

   In ACM SIGGRAPH 2016 Emerging Technologies, Anaheim, California, Jul 2016

   Abs Bib DOI Video

   Naı̈ve physics [7], or folk physics, is our ability to understand physical phenomena. We regularly use this ability in life to avoid collisions in traffic, follow a tennis ball and time the return shot, or while working in dynamic industrial settings. Though this skill improves with practice, it is still imperfect, which leads to mistakes and misjudgments for time intensive tasks. People still often miss a tennis shot, which might cause them to lose the match, or fail to avoid a car or pedestrian, which can lead to injury or even death.As a step towards reducing these errors in human judgement, we present Laplacian Vision (LV), a vision augmentation system which assists the human ability to predict future trajectory information. By tracking real world objects and estimating their trajectories, we can improve a users’s prediction of the landing spot of a ball or the path of an oncoming car. We have designed a system that can track a flying ball in real time, predict its future trajectory, and visualize it in the user’s field of view. The system is also calibrated to account for end-to-end delays so that the trajectory appears to emanate forward from the moving object. We also conduct a user study where 29 subjects predict an object’s landing spot, and show that prediction accuracy improves 3 fold using LV.

   @inproceedings{itoh2016laplacian,
     pages = {1},
     month = jul,
     author = {Itoh, Yuta and Hiroi, Yuichi and Otsuka, Jiu and Sugimoto, Maki and Orlosky, Jason and Kiyokawa, Kiyoshi and Klinker, Gudrun},
     title = {Laplacian vision: augmenting motion prediction via optical see-through head-mounted displays and projectors},
     year = {2016},
     isbn = {9781450343725},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/2929464.2949028},
     doi = {10.1145/2929464.2949028},
     booktitle = {ACM SIGGRAPH 2016 Emerging Technologies},
     articleno = {13},
     location = {Anaheim, California},
     series = {SIGGRAPH '16},
   }