publications
2024
- IEEE TVCG
HaptoFloater: Visuo-Haptic Augmented Reality by Embedding Imperceptible Color Vibration Signals for Tactile Display Control in a Mid-Air ImageIEEE Transactions on Visualization and Computer Graphics, Nov 2024We propose HaptoFloater, a low-latency mid-air visuo-haptic augmented reality (VHAR) system that utilizes imperceptible color vibrations. When adding tactile stimuli to the visual information of a mid-air image, the user should not perceive the latency between the tactile and visual information. However, conventional tactile presentation methods for mid-air images, based on camera-detected fingertip positioning, introduce latency due to image processing and communication. To mitigate this latency, we use a color vibration technique; humans cannot perceive the vibration when the display alternates between two different color stimuli at a frequency of 25 Hz or higher. In our system, we embed this imperceptible color vibration into the mid-air image formed by a micromirror array plate, and a photodiode on the fingertip device directly detects this color vibration to provide tactile stimulation. Thus, our system allows for the tactile perception of multiple patterns on a mid-air image in 59.5 ms. In addition, we evaluate the visual-haptic delay tolerance on a mid-air display using our VHAR system and a tactile actuator with a single pattern and faster response time. The results of our user study indicate a visual-haptic delay tolerance of 110.6 ms, which is considerably larger than the latency associated with systems using multiple tactile patterns.
@article{nagano2024haptofloater, author = {Nagano, Rina and Kinoshita, Takahiro and Hattori, Shingo and Hiroi, Yuichi and Itoh, Yuta and Hiraki, Takefumi}, journal = {IEEE Transactions on Visualization and Computer Graphics}, title = {HaptoFloater: Visuo-Haptic Augmented Reality by Embedding Imperceptible Color Vibration Signals for Tactile Display Control in a Mid-Air Image}, year = {2024}, volume = {30}, number = {11}, pages = {7463-7472}, keywords = {Haptic interfaces;Vibrations;Visualization;Image color analysis;Actuators;Delays;Tactile sensors;Visuo-haptic displays;mid-air images;LCD displays;imperceptible color vibration}, doi = {10.1109/TVCG.2024.3456175}, issn = {1941-0506}, month = nov, } - IEEE TVCG
StainedSweeper: Compact, Variable-Intensity Light-Attenuation Display with Sweeping Tunable RetardersYuichi Hiroi, Takefumi Hiraki, and Yuta ItohIEEE Transactions on Visualization and Computer Graphics, May 2024Light Attenuation Displays (LADs) are a type of Optical See-Through Head-Mounted Display (OST-HMD) that present images by attenuating incoming light with a pixel-wise polarizing color filter. Although LADs can display images in bright environments, there is a trade-off between the number of Spatial Light Modulators (SLMs) and the color gamut and contrast that can be expressed, making it difficult to achieve both high-fidelity image display and a small form factor. To address this problem, we propose StainedSweeper, a LAD that achieves both the wide color gamut and the variable intensity with a single SLM. Our system synchronously controls a pixel-wise Digital Micromirror Device (DMD) and a nonpixel polarizing color filter to pass light when each pixel is the desired color. By sweeping this control at high speed, the human eye perceives images in a time-multiplexed, integrated manner. To achieve this, we develop the OST-HMD design using a reflective Solc filter as a polarized color filter and a color reproduction algorithm based on the optimization of the time-multiplexing matrix for the selected primary color filters. Our proof-of-concept prototype showed that our single SLM design can produce subtractive images with variable contrast and a wider color gamut than conventional LADs.
@article{hiroi2024stainedsweeper, author = {Hiroi, Yuichi and Hiraki, Takefumi and Itoh, Yuta}, journal = {IEEE Transactions on Visualization and Computer Graphics}, title = {StainedSweeper: Compact, Variable-Intensity Light-Attenuation Display with Sweeping Tunable Retarders}, year = {2024}, volume = {30}, number = {5}, pages = {2682-2692}, keywords = {Image color analysis;Optical filters;Filtering algorithms;Light sources;Optical imaging;Optical reflection;Optical polarization;Light attenuation display;see-through display;augmented reality;time-multiplexing;polarized color filter}, doi = {10.1109/TVCG.2024.3372058}, issn = {1941-0506}, month = may, } - IEEE TVCG
Towards Co-Operative Beaming Displays: Dual Steering Projectors for Extended Projection Volume and Head Orientation RangeHiroto Aoki, Takumi Tochimoto, Yuichi Hiroi, and Yuta ItohIEEE Transactions on Visualization and Computer Graphics, May 2024Existing near-eye displays (NEDs) have trade-offs related to size, weight, computational resources, battery life, and body temperature. A recent paradigm, beaming display, addresses these trade-offs by separating the NED into a steering projector (SP) for image presentation and a passive headset worn by the user. However, the beaming display has issues with the projection area of a single SP and has severe limitations on the head orientation and pose that the user can move. In this study, we distribute dual steering projectors in the scene to extend the head orientation and pose of the beaming display by coordinating the dual projections on a passive headset. For cooperative control of each SP, we define a geometric model of the SPs and propose a calibration and projection control method designed for multiple projectors. We present implementations of the system along with evaluations showing that the precision and delay are 1.8 ∼ 5.7 mm and 14.46 ms, respectively, at a distance of about 1 m from the SPs. From this result, our prototype with multiple SPs can project images in the projection area (20 \textmm \times 30 \textmm) of the passive headset while extending the projectable head orientation. Furthermore, as applications of cooperative control by multiple SPs, we show the possibility of multiple users, improving dynamic range and binocular presentation.
@article{aoki2024dualbeaming, author = {Aoki, Hiroto and Tochimoto, Takumi and Hiroi, Yuichi and Itoh, Yuta}, journal = {IEEE Transactions on Visualization and Computer Graphics}, title = {Towards Co-Operative Beaming Displays: Dual Steering Projectors for Extended Projection Volume and Head Orientation Range}, year = {2024}, volume = {30}, number = {5}, pages = {2309-2318}, keywords = {Mirrors;Cameras;Headphones;Calibration;Optical imaging;High-speed optical techniques;Head;Near-eye display;Augmented reality;Projectors}, doi = {10.1109/TVCG.2024.3372118}, issn = {1941-0506}, month = may, } - IEEE ISMAR
FactoredSweeper: Optical See-Through Display Integrating Light Attenuation and Addition with a Single Spatial Light ModulatorYuichi Hiroi, Takefumi Hiraki, and Yuta ItohIn 2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Oct 2024Light Attenuation Displays (LADs), a subset of Optical See-Through Head-Mounted Displays (OST-HMDs), enable image display in bright environments by filtering incident light at the pixel level. Although recent methods have proposed single-DMD light attenuation, they do not consider additive color display and background compensation, limiting their applicability in real-world scenarios. We present FactoredSweeper, a single digital micromirror device (DMD) system that incorporates both light attenuation and addition. By synchronizing the DMD, color filter, and light source, our system generates an additive virtual image, light attenuation, and occlusion through time multiplexing. To produce the target image while compensating for the background, we optimize time-multiplexed binary DMD patterns and LED/color filter schedules using perceptually-driven non-negative matrix factorization. Simulations and prototypes demonstrate that our integrated attenuation-addition single-SLM system achieves superior dynamic range and perceptual image quality compared to conventional occlusion-capable OST-HMDs using grayscale occlusion masks.
- ISMAR Poster
Beaming Display Using Thin Holographic Waveguides for Wider Head Orientation Angle RangeIn 2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Oct 2024Augmented Reality (AR) glasses face fundamental challenges related to technical trade-offs. Emerging Beaming Displays (BDs) offer a compelling solution by separating the active and passive components. However, existing BD-based AR glasses have yet to achieve a thin and lightweight design with wide incident projection angles. This work proposes an eyepiece for BDs, including a holographic waveguide with tailored in- and out-coupling gratings. The proposed design aims to achieve a millimeter-thin form factor with a wide tolerance for incident angles, thus overcoming the limitations of existing designs. We have constructed proof-of-concept passive AR glasses prototypes, all approximately 2mm thick, including one in the form of conventional eyeglasses, and demonstrated an acceptable lateral angle of incidence of up to 90 degrees.
- arXiv
PanoTree: Autonomous Photo-Spot Explorer in Virtual Reality ScenesTomohiro Hayase, Sacha Braun, Hikari Yanagawa, Itsuki Orito, and Yuichi HiroiOct 2024@misc{hayase2024panotree, title = {PanoTree: Autonomous Photo-Spot Explorer in Virtual Reality Scenes}, author = {Hayase, Tomohiro and Braun, Sacha and Yanagawa, Hikari and Orito, Itsuki and Hiroi, Yuichi}, year = {2024}, eprint = {2405.17136}, archiveprefix = {arXiv}, primaryclass = {cs.CV}, } - SIGGRAPH Poster
Measurement of the Imperceptible Threshold for Color Vibration Pairs Selected by using MacAdam EllipseShingo Hattori, Yuichi Hiroi, and Takefumi HirakiIn ACM SIGGRAPH 2024 Posters, Denver, CO, USA, Oct 20241st Place in SIGGRAPH 2024 Student Research Competition for Undergraduate Work
We propose an efficient method for searching for color vibration pairs that are imperceptible to the human eye based on the MacAdam ellipse, an experimentally determined color-difference range that is indistinguishable to the human eye. We created color pairs by selecting eight colors within the sRGB color space specified by the ellipse, and conducted experiments to confirm the threshold of the amplitude of color vibration amplitude at which flicker becomes imperceptible to the human eye. The experimental results indicate a general guideline for acceptable amplitudes for pair selection.
@inproceedings{hattori2024macadam, author = {Hattori, Shingo and Hiroi, Yuichi and Hiraki, Takefumi}, title = {Measurement of the Imperceptible Threshold for Color Vibration Pairs Selected by using MacAdam Ellipse}, year = {2024}, isbn = {9798400705168}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3641234.3671041}, doi = {10.1145/3641234.3671041}, booktitle = {ACM SIGGRAPH 2024 Posters}, articleno = {68}, numpages = {2}, keywords = {MacAdam ellipse, color perception, imperceptible color vibration}, location = {Denver, CO, USA}, series = {SIGGRAPH '24}, } - IEEE VR Workshop
Design and Implementation of Agent APIs for Large-Scale Social VR PlatformsRyutaro Kurai, Takefumi Hiraki, Yuichi Hiroi, Yutaro Hirao, Monica Perusquia-Hernandez, Hideaki Uchiyama, and Kiyoshi KiyokawaIn 2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) , Mar 2024Implementing an autonomous agent on a social VR platform where many users share space requires diverse information. In particular, it is required to recognize the distance from other users, their orientation toward each other, the avatar’s pose, and text and voice messages, and to behave accordingly. This paper proposes an API to obtain the above information on “Cluster,” a multi-device social VR platform in operation, and an agent that uses the API. We have implemented this API using a network proxy. The agent using this API can connect to ChatGPT [7] and have a conversation in real time. We measured the latency required for the conversation and confirmed that the response time was about 1 second.
@inproceedings{kurai2024agentapi, author = {Kurai, Ryutaro and Hiraki, Takefumi and Hiroi, Yuichi and Hirao, Yutaro and Perusquia-Hernandez, Monica and Uchiyama, Hideaki and Kiyokawa, Kiyoshi}, booktitle = { 2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) }, title = {{ Design and Implementation of Agent APIs for Large-Scale Social VR Platforms }}, year = {2024}, volume = {}, issn = {}, pages = {584-587}, keywords = {Three-dimensional displays;Text recognition;Oral communication;Virtual reality;Speech recognition;User interfaces;Software}, doi = {10.1109/VRW62533.2024.00112}, url = {https://doi.ieeecomputersociety.org/10.1109/VRW62533.2024.00112}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, month = mar, }
2023
- IEEE TVCG
Low-Latency Beaming Display: Implementation of Wearable, 133 μs Motion-to-Photon Latency Near-Eye DisplayYuichi Hiroi , Akira Watanabe, Yuri Mikawa, and Yuta ItohIEEE Transactions on Visualization and Computer Graphics, Nov 2023This paper presents a low-latency Beaming Display system with a 133 μ\mathrms motion-to-photon (M2P) latency, the delay from head motion to the corresponding image motion. The Beaming Display represents a recent near-eye display paradigm that involves a steerable remote projector and a passive wearable headset. This system aims to overcome typical trade-offs of Optical See-Through Head-Mounted Displays (OST-HMDs), such as weight and computational resources. However, since the Beaming Display projects a small image onto a moving, distant viewpoint, M2P latency significantly affects displacement. To reduce M2P latency, we propose a low-latency Beaming Display system that can be modularized without relying on expensive high-speed devices. In our system, a 2D position sensor, which is placed coaxially on the projector, detects the light from the IR-LED on the headset and generates a differential signal for tracking. An analog closed-loop control of the steering mirror based on this signal continuously projects images onto the headset. We have implemented a proof-of-concept prototype, evaluated the latency and the augmented reality experience through a user-perspective camera, and discussed the limitations and potential improvements of the prototype.
@article{hiroi2023lowlatency, author = {Hiroi, Yuichi and Watanabe, Akira and Mikawa, Yuri and Itoh, Yuta}, journal = {IEEE Transactions on Visualization and Computer Graphics}, title = {Low-Latency Beaming Display: Implementation of Wearable, 133 μs Motion-to-Photon Latency Near-Eye Display}, year = {2023}, volume = {29}, number = {11}, pages = {4761-4771}, keywords = {Mirrors;Low latency communication;Headphones;Head;Cameras;Tracking;Two dimensional displays;Low-Latency Display;Beaming Display;Motion-to-Photon Latency;Lateral-effect Photodiodes}, doi = {10.1109/TVCG.2023.3320212}, issn = {1941-0506}, month = nov, } - ACM UIST
Telextiles: End-to-end Remote Transmission of Fabric Tactile SensationIn Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, San Francisco, CA, USA, Nov 2023The tactile sensation of textiles is critical in determining the comfort of clothing. For remote use, such as online shopping, users cannot physically touch the textile of clothes, making it difficult to evaluate its tactile sensation. Tactile sensing and actuation devices are required to transmit the tactile sensation of textiles. The sensing device needs to recognize different garments, even with hand-held sensors. In addition, the existing actuation device can only present a limited number of known patterns and cannot transmit unknown tactile sensations of textiles. To address these issues, we propose Telextiles, an interface that can remotely transmit tactile sensations of textiles by creating a latent space that reflects the proximity of textiles through contrastive self-supervised learning. We confirm that textiles with similar tactile features are located close to each other in the latent space through a two-dimensional plot. We then compress the latent features for known textile samples into the 1D distance and apply the 16 textile samples to the rollers in the order of the distance. The roller is rotated to select the textile with the closest feature if an unknown textile is detected.
@inproceedings{kitagishi2023telextiles, author = {Kitagishi, Takekazu and Hiroi, Yuichi and Watanabe, Yuna and Itoh, Yuta and Rekimoto, Jun}, title = {Telextiles: End-to-end Remote Transmission of Fabric Tactile Sensation}, year = {2023}, isbn = {9798400701320}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3586183.3606764}, doi = {10.1145/3586183.3606764}, booktitle = {Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology}, articleno = {67}, numpages = {10}, keywords = {Haptic feedback, Machine learning, Passive haptic feedback, Self supervised learning, Tactile Display, Tactile perception, Texture, Texture perception, Texture recognition}, location = {San Francisco, CA, USA}, series = {UIST '23}, } - ACM VRST
Retinal Homing Display: Head-Tracking Auto-stereoscopic Retinal Projection DisplayHiroto Aoki, Yuichi Hiroi, Yuta Itoh, and Jun RekimotoIn Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology, Christchurch, New Zealand, Nov 2023This paper introduces Retinal Homing Display, which presents focus-free stereoscopic images via retinal projection, thus eliminating the need for the user to wear additional equipment. Traditional 3D displays, typically classified as either naked-eye stereoscopic or wearable, present inherent challenges: the former involves a compromise between resolution and accurate depth perception, while the latter imposes an additional burden on the user. Our proposed display employs optical and mechanical mechanisms to converge projector light at the user’s pupil center, simultaneously tracking eye movements. This lets the user perceive focus-free, high-resolution stereoscopic images without wearable equipment. We implemented a proof-of-concept system utilizing a robotic arm and a Dihedral Corner Reflector Array (DCRA), subsequently evaluating image quality and its eyebox. Finally, we discuss the limitations of the current prototype and outline potential directions for future research.
@inproceedings{aoki2023retinal, author = {Aoki, Hiroto and Hiroi, Yuichi and Itoh, Yuta and Rekimoto, Jun}, title = {Retinal Homing Display: Head-Tracking Auto-stereoscopic Retinal Projection Display}, year = {2023}, isbn = {9798400703287}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3611659.3615715}, doi = {10.1145/3611659.3615715}, booktitle = {Proceedings of the 29th ACM Symposium on Virtual Reality Software and Technology}, articleno = {13}, numpages = {10}, keywords = {autostereoscopic display, motion-following display, retinal projection}, location = {Christchurch, New Zealand}, series = {VRST '23}, } - IEEE VR
A Compact Photochromic Occlusion Capable See-through Display with Holographic LensesChun-Wei Ooi, Yuichi Hiroi, and Yuta ItohIn 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR), Mar 2023Occlusion is a crucial visual element in optical see-through (OST) augmented reality, however, implementing occlusion in OST displays while addressing various design trade-offs is a difficult problem. In contrast to the traditional method of using spatial light modulators (SLMs) for the occlusion mask, using photochromic materials as occlusion masks can effectively eliminate diffraction artifacts in see-through views due to the lack of electronic pixels, thus providing superior see-through image quality. However, this design requires UV illumination to activate the photochromic mate-rial, which traditionally requires multiple SLMs, resulting in a larger form factor for the system. This paper presents a compact photochromic occlusion-capable OST design using multilayer, wavelength-dependent holographic optical lenses (HOLs). Our approach employs a single digital mi-cromirror display (DMD) to form both the occlusion mask with UV light and a virtual image with visible light in a time-multiplexed man-ner. We demonstrate our proof-of-concept system on a bench-top setup and assess the appearance and contrasts of the displayed image. We also suggest potential improvements for current prototypes to encourage the community to explore this occlusion approach.
@inproceedings{ooi2023photochromic, author = {Ooi, Chun-Wei and Hiroi, Yuichi and Itoh, Yuta}, booktitle = {2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR)}, title = {A Compact Photochromic Occlusion Capable See-through Display with Holographic Lenses}, year = {2023}, volume = {}, number = {}, pages = {237-242}, keywords = {Visualization;Optical diffraction;Three-dimensional displays;Optical design;Prototypes;Holography;User interfaces;Human-centered computing-Human computer interaction (HCI)-Interaction paradigms-Mixed / augmented reality;Human-centered computing-Communication hardware;interfaces and storage-Displays and imagers}, doi = {10.1109/VR55154.2023.00039}, issn = {2642-5254}, month = mar, } - ISMAR PosterDual Beaming Display for Extended Head Orientation and Projection VolumeTakumi Tochimoto, Yuichi Hiroi, and Yuta ItohIn 2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Oct 2023
Existing near-eye displays (NEDs) come with various tradeoffs, including size, weight, and battery life. The new "beaming display" model attempts to address these by splitting the NED into a steering projector (SP) and a passive headset. However, it has limitations in terms of projection area and user head movement. In this research, we utilize two steering projectors to expand the beaming display’s capabilities. We introduce a geometric model and a control method specifically for multiple projectors. Tests indicate an accuracy of 3-19mm and a delay of 14.46 ms from a 1m distance to the SPs, allowing projection onto a 20mm x 30mm area of the headset. This method not only expands head movement possibilities but also shows potential for multiple users and improved dynamic range and binocular presentation.
@inproceedings{tochimoto2023dualbeaming, author = {Tochimoto, Takumi and Hiroi, Yuichi and Itoh, Yuta}, booktitle = {2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)}, title = {Dual Beaming Display for Extended Head Orientation and Projection Volume}, year = {2023}, volume = {}, number = {}, pages = {377-378}, keywords = {Headphones;Design methodology;Geometric modeling;Display systems;Focusing;Dynamic range;Delays;Human-centered computing;Visualization;Visualization techniques;Treemaps; Human-centered computing;Visualization design and evaluation methods}, doi = {10.1109/ISMAR-Adjunct60411.2023.00081}, issn = {2771-1110}, month = oct, } - AHs Poster
Brain-Computer Interface using Directional Auditory PerceptionYuto Koike, Yuichi Hiroi, Yuta Itoh, and Jun RekimotoIn Proceedings of the Augmented Humans International Conference 2023, Glasgow, United Kingdom, Oct 2023Best Poster Honorable Mention
We investigate the potential of brain-computer interface (BCI) using electroencephalogram (EEG) induced by listening (or recalling) auditory stimuli of different directions. In the initial attempt, we apply a time series classification model based on deep learning to the EEG to demonstrate whether each EEG can be classified by recognizing binary (left or right) auditory directions. The results showed high classification accuracy when trained and tested on the same users. Discussion is provided to further explore this topic.
@inproceedings{koike2023braincomputer, author = {Koike, Yuto and Hiroi, Yuichi and Itoh, Yuta and Rekimoto, Jun}, title = {Brain-Computer Interface using Directional Auditory Perception}, year = {2023}, isbn = {9781450399845}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3582700.3583713}, doi = {10.1145/3582700.3583713}, booktitle = {Proceedings of the Augmented Humans International Conference 2023}, pages = {342–345}, numpages = {4}, keywords = {BCI, BMI, Brain-computer Interface, Brain-machine Interface, Directional Auditory Sensation, EEG, Electroencephalogram, Electroencephalography}, location = {Glasgow, United Kingdom}, series = {AHs '23}, }
2022
- Opt. Exp.
Neural distortion fields for spatial calibration of wide field-of-view near-eye displaysYuichi Hiroi, Kiyosato Someya, and Yuta ItohOpt. Express, Oct 2022Japan’s Leading Optics Research in 2022 (Top 30), Optical Society of Japan
We propose a spatial calibration method for wide field-of-view (FoV) near-eye displays (NEDs) with complex image distortions. Image distortions in NEDs can destroy the reality of the virtual object and cause sickness. To achieve distortion-free images in NEDs, it is necessary to establish a pixel-by-pixel correspondence between the viewpoint and the displayed image. Designing compact and wide-FoV NEDs requires complex optical designs. In such designs, the displayed images are subject to gaze-contingent, non-linear geometric distortions, which explicit geometric models can be difficult to represent or computationally intensive to optimize. To solve these problems, we propose neural distortion field (NDF), a fully-connected deep neural network that implicitly represents display surfaces complexly distorted in spaces. NDF takes spatial position and gaze direction as input and outputs the display pixel coordinate and its intensity as perceived in the input gaze direction. We synthesize the distortion map from a novel viewpoint by querying points on the ray from the viewpoint and computing a weighted sum to project output display coordinates into an image. Experiments showed that NDF calibrates an augmented reality NED with 90° FoV with about 3.23 pixel (5.8 arcmin) median error using only 8 training viewpoints. Additionally, we confirmed that NDF calibrates more accurately than the non-linear polynomial fitting, especially around the center of the FoV.
@article{hiroi2022neural, author = {Hiroi, Yuichi and Someya, Kiyosato and Itoh, Yuta}, journal = {Opt. Express}, keywords = {Image metrics; Lens design; Near eye displays; Optical aberration; Optical systems; Systems design}, number = {22}, pages = {40628--40644}, publisher = {Optica Publishing Group}, title = {Neural distortion fields for spatial calibration of wide field-of-view near-eye displays}, volume = {30}, month = oct, year = {2022}, url = {https://opg.optica.org/oe/abstract.cfm?URI=oe-30-22-40628}, doi = {10.1364/OE.472288}, } - ACM VRST
NeARportation: A Remote Real-time Neural Rendering FrameworkYuichi Hiroi, Yuta Itoh, and Jun RekimotoIn Proceedings of the 28th ACM Symposium on Virtual Reality Software and Technology, Tsukuba, Japan, Oct 2022While presenting a photorealistic appearance plays a major role in immersion in Augmented Virtuality environment, displaying that of real objects remains a challenge. Recent developments in photogrammetry have facilitated the incorporation of real objects into virtual space. However, reproducing complex appearances, such as subsurface scattering and transparency, still requires a dedicated environment for measurement and possesses a trade-off between rendering quality and frame rate. Our NeARportation framework combines server–client bidirectional communication and neural rendering to resolve these trade-offs. Neural rendering on the server receives the client’s head posture and generates a novel-view image with realistic appearance reproduction that is streamed onto the client’s display. By applying our framework to a stereoscopic display, we confirm that it can display a high-fidelity appearance on full-HD stereo videos at 35-40 frames per second (fps) according to the user’s head motion.
@inproceedings{hiroi2022nearportation, author = {Hiroi, Yuichi and Itoh, Yuta and Rekimoto, Jun}, title = {NeARportation: A Remote Real-time Neural Rendering Framework}, year = {2022}, isbn = {9781450398893}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3562939.3565616}, doi = {10.1145/3562939.3565616}, booktitle = {Proceedings of the 28th ACM Symposium on Virtual Reality Software and Technology}, articleno = {23}, numpages = {5}, keywords = {remote rendering, real-time rendering, neural rendering, augmented virtuality, appearance reproduction}, location = {Tsukuba, Japan}, series = {VRST '22}, } - IEEE ISMAR
Towards Spatial Airflow Interaction: Schlieren Imaging for Augmented RealityZhang Zhibin, Yuichi Hiroi, and Yuta ItohIn 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Oct 2022This work integrates Schlieren Imaging, a unique sensing modality, into Augmented Reality (AR) to explore ways to utilize invisible airflows for AR. Schlieren imaging is an imaging technique that visualizes the flow of fluids, which is normally invisible to the eyes. Theoretically, the technique can calculate the motion, pressure, temperature, and density of the airflow in our physical world. This unique, but less applied modality may expand interaction paradigms in AR and VR. We build a proof-of-concept AR system combined with Schlieren imaging that allows real airflow to affect virtual objects. The results of quantitative analyses show that our system can integrate different types of airflow with pressure values ranging from weak breathing actions to a heat gun up to 10m/s or 0.25m3/min airflow. We also showcase AR use cases including blowing out a virtual candle and a heat gun.
@inproceedings{zhibin2022airflow, author = {Zhibin, Zhang and Hiroi, Yuichi and Itoh, Yuta}, booktitle = {2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)}, title = {Towards Spatial Airflow Interaction: Schlieren Imaging for Augmented Reality}, year = {2022}, volume = {}, number = {}, pages = {215-223}, keywords = {Heating systems;Temperature sensors;Photography;Visualization;Sensitivity;Statistical analysis;User interfaces;Schlieren imaging;augmented reality;spatial airflow interaction}, doi = {10.1109/ISMAR55827.2022.00036}, issn = {1554-7868}, month = oct, }
2021
- Opt. Exp.
Focal surface occlusionOpt. Express, Oct 2021This paper proposes focal surface occlusion to provide focal cues of occlusion masks for multiple virtual objects at continuous depths in an occlusion-capable optical see-through head-mounted display. A phase-only spatial light modulator (PSLM) that acts as a dynamic free-form lens is used to conform the focal surface of an occlusion mask to the geometry of the virtual scene. To reproduce multiple and continuous focal blurs while reducing the distortion of the see-through view, an optical design based on afocal optics and edge-based optimization to exploit a property of the occlusion mask is established. The prototype with the PSLM and transmissive liquid crystal display can reproduce the focus blur of occluded objects at multiple and continuous depths with a field of view of 14.6°.
@article{hiroi2021focalsurface, author = {Hiroi, Yuichi and Kaminokado, Takumi and Ono, Shunsuke and Itoh, Yuta}, journal = {Opt. Express}, keywords = {Head mounted displays; Microlens arrays; Near eye displays; Optical components; Optical systems; Spatial light modulators}, number = {22}, pages = {36581--36597}, publisher = {Optica Publishing Group}, title = {Focal surface occlusion}, volume = {29}, month = oct, year = {2021}, url = {https://opg.optica.org/oe/abstract.cfm?URI=oe-29-22-36581}, doi = {10.1364/OE.440024}, } - Augmented Humans
CircadianVisor: Image Presentation with an Optical See-Through Display in Consideration of Circadian IlluminanceTakumi Tochimoto, Yuichi Hiroi, and Yuta ItohIn Proceedings of the Augmented Humans International Conference 2021, Rovaniemi, Finland, Oct 2021In modern society, the impact of nighttime artificial lighting on the human sleep/wake cycle (the circadian rhythm), has long been an important issue. In augmented reality, such health hazards should be prevented if we are to become a society that wears optical see-through head-mounted displays (OST-HMDs) on a daily basis. We present CircadianVisor, an OST display system that controls circadian performance. Our system combines an OST-HMD with a liquid crystal (LC) shutter and a spectrometer to control the circadian illuminance (CIL, biolux) of light incident on the user’s eyes. To prevent the CIL at the eyes from exceeding the threshold, correct for the displayed image based on RGB values, and adjust the transmittance of the LC visor to pass through the environment light based on spectral measurements, we build a proof-of-concept system to evaluate the feasibility of the system’s CIL control and test it with a spectrometer installed at the user’s viewpoint. The evaluation shows that the CIL at the user’s viewpoint can be kept below the threshold.
@inproceedings{tochimoto2021circadian, author = {Tochimoto, Takumi and Hiroi, Yuichi and Itoh, Yuta}, title = {CircadianVisor: Image Presentation with an Optical See-Through Display in Consideration of Circadian Illuminance}, year = {2021}, isbn = {9781450384285}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3458709.3458938}, doi = {10.1145/3458709.3458938}, booktitle = {Proceedings of the Augmented Humans International Conference 2021}, pages = {66–76}, numpages = {11}, keywords = {Augmented reality, Blue-light-blocking glasses, Circadian rhythm, Head-mounted displays}, location = {Rovaniemi, Finland}, series = {AHs '21}, } - ISMAR Poster
Focus-Aware Retinal Projection-based Near-Eye DisplayMayu Kaneko, Yuichi Hiroi, and Yuta ItohIn 2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Oct 2021The primary challenge in optical see-through near-eye displays lies in providing correct optical focus cues. Established approaches such as varifocal or light field display essentially sacrifice temporal or spatial resolution of the resulting 3D images. This paper explores a new direction to address the trade-off by combining a retinal projection display (RPD) with ocular wavefront sensing (OWS). Our core idea is to display a depth of field-simulated image on an RPD to produce visually consistent optical focus cues while maintaining the spatial and temporal resolution of the image. To obtain the current accommodation of the eye, we integrate OWS. We demonstrate that our proof-of-concept system successfully renders virtual contents with proper depth cues while covering the eye accommodation range from 28.5 cm (3.5 D) to infinity (0.0 D).
@inproceedings{kaneko2021focusaware, author = {Kaneko, Mayu and Hiroi, Yuichi and Itoh, Yuta}, booktitle = {2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)}, title = {Focus-Aware Retinal Projection-based Near-Eye Display}, year = {2021}, volume = {}, number = {}, pages = {207-208}, keywords = {Three-dimensional displays;Optical variables measurement;Optical imaging;Retina;Rendering (computer graphics);Adaptive optics;Light fields;Augmented reality;accommodation sensing;retinal projection;near-eye display}, doi = {10.1109/ISMAR-Adjunct54149.2021.00049}, issn = {}, month = oct, }
2020
- IEEE TVCG
StainedView: Variable-Intensity Light-Attenuation Display with Cascaded Spatial Color Filtering for Improved Color FidelityTakumi Kaminokado, Yuichi Hiroi, and Yuta ItohIEEE Transactions on Visualization and Computer Graphics, Dec 2020We present StainedView, an optical see-through display that spatially filters the spectral distribution of light to form an image with improved color fidelity. Existing light-attenuation displays have limited color fidelity and contrast, resulting in a degraded appearance of virtual images. To use these displays to present virtual images that are more consistent with the real world, we require three things: intensity modulation of incoming light, spatial color filtering with narrower bandwidth, and appropriate light modulation for incoming light with an arbitrary spectral distribution. In StainedView, we address the three requirements by cascading two phase-only spatial light modulators (PSLMs), a digital micromirror device, and polarization optics to control both light intensity and spectrum distribution. We show that our design has a 1.8 times wider color gamut fidelity (75.8% fulfillment of sRGB color space) compared to the existing single-PSLM approach (41.4%) under a reference white light. We demonstrated the design with a proof-of-concept display system. We further introduce our optics design and pixel-selection algorithm for the given light input, evaluate the spatial color filter, and discuss the limitation of the current prototype.
@article{kamino2020stainedview, author = {Kaminokado, Takumi and Hiroi, Yuichi and Itoh, Yuta}, journal = {IEEE Transactions on Visualization and Computer Graphics}, title = {StainedView: Variable-Intensity Light-Attenuation Display with Cascaded Spatial Color Filtering for Improved Color Fidelity}, year = {2020}, volume = {26}, number = {12}, pages = {3576-3586}, keywords = {Image color analysis;Optical imaging;Optical polarization;Adaptive optics;Optical distortion;Optical modulation;Nonlinear optics;Light attenuation display;phase modulation;see-through display;vision augmentation;augmented reality}, doi = {10.1109/TVCG.2020.3023569}, issn = {1941-0506}, month = dec, } - Augmented Humans
DehazeGlasses: Optical Dehazing with an Occlusion Capable See-Through DisplayYuichi Hiroi, Takumi Kaminokado, Atsushi Mori, and Yuta ItohIn Proceedings of the Augmented Humans International Conference, Kaiserslautern, Germany, Dec 2020We present DehazeGlasses, a see-through visual haze removal system that optically dehazes the user’s field of vision. Human vision suffers from a degraded view due to aspects of the scene environment, such as haze. Such degradation may interfere with our behavior or judgement in daily tasks. We focus on hazy scenes as one common degradation source, which whitens the view due to certain atmospheric conditions. Unlike typical computer vision systems that process recorded images, we aim to realize a see-through glasses system that can optically manipulate our field of view to dehaze the perceived scene. Our system selectively modulates the intensity of the light entering the eyes via occlusion-capable optical see-through head-mounted displays (OST-HMD). We built a proof-of-concept system to evaluate the feasibility of our haze removal method by combining a digital micromirror device (DMD) and an OST-HMD, and tested it with a user-perspective viewpoint camera. A quantitative evaluation with 80 scenes from a haze removal dataset shows that our system realizes a dehazed view that is significantly closer to the ground truth scene compared to the native view under a perceptual image similarity metric. This evaluation shows that our system achieves perceptually natural haze removal while maintaining the see-through view of actual scenes.
@inproceedings{hiroi2020dehaze, author = {Hiroi, Yuichi and Kaminokado, Takumi and Mori, Atsushi and Itoh, Yuta}, title = {DehazeGlasses: Optical Dehazing with an Occlusion Capable See-Through Display}, year = {2020}, isbn = {9781450376037}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3384657.3384781}, doi = {10.1145/3384657.3384781}, booktitle = {Proceedings of the Augmented Humans International Conference}, articleno = {3}, numpages = {11}, keywords = {Vision Augmentation, Occlusion-Capable HMD, Head-Mounted Displays, Haze Removal, Augmented Reality}, location = {Kaiserslautern, Germany}, series = {AHs '20}, } - ISMAR Poster
Stencil Marker: Designing Partially Transparent Markers for Stacking Augmented Reality ObjectsXuan Zhang, Jonathan Lundgren, Yoya Mesaki, Yuichi Hiroi, and Yuta ItohIn 2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Nov 2020We propose a transparent colored AR marker that allows 3D objects to be stacked in space. Conventional AR markers make it difficult to display multiple objects in the same position in space, or to manipulate the order or rotation of objects. The proposed transparent colored markers are designed to detect the order and rotation direction of each marker in the stack from the observed image, based on mathematical constraints. We describe these constraints to design markers, the implementation to detect its stacking order and rotation of each marker, and a proof-of-concept application Totem Poles. We also discuss the limitations of the current prototype and possible research directions.
@inproceedings{zhang2020stencil, author = {Zhang, Xuan and Lundgren, Jonathan and Mesaki, Yoya and Hiroi, Yuichi and Itoh, Yuta}, booktitle = {2020 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)}, title = {Stencil Marker: Designing Partially Transparent Markers for Stacking Augmented Reality Objects}, year = {2020}, volume = {}, number = {}, pages = {255-257}, keywords = {Human computer interaction;Three-dimensional displays;Image color analysis;Stacking;Prototypes;Augmented reality;Human-centered computing;Visualization;Visualization techniques;Treemaps;Human-centered computing;Visualization;Visualization design and evaluation methods}, doi = {10.1109/ISMAR-Adjunct51615.2020.00073}, issn = {}, month = nov, }
2019
- ISMAR Poster
OSTNet: Calibration Method for Optical See-Through Head-Mounted Displays via Non-Parametric Distortion Map GenerationKiyosato Someya, Yuichi Hiroi, Makoto Yamada, and Yuta ItohIn 2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Oct 2019We propose a spatial calibration method for Optical See-Through Head-Mounted Displays (OST-HMDs) having complex optical distortion such as wide field-of-view (FoV) designs. Viewpoint-dependent non-linear optical distortion makes existing spatial calibration methods either impossible to handle or difficult to compensate without intensive computation. To overcome this issue, we propose OSTNet, a non-parametric data-driven calibration method that creates a generative 2D distortion model for a given six-degree-of-freedom viewpoint pose.
@inproceedings{someya2019ostnet, author = {Someya, Kiyosato and Hiroi, Yuichi and Yamada, Makoto and Itoh, Yuta}, booktitle = {2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)}, title = {OSTNet: Calibration Method for Optical See-Through Head-Mounted Displays via Non-Parametric Distortion Map Generation}, year = {2019}, volume = {}, number = {}, pages = {259-260}, keywords = {Optical distortion;Calibration;Cameras;Nonlinear distortion;Two dimensional displays;Decoding;optical see through head mounted display;calibration;Variational Autoencoder}, doi = {10.1109/ISMAR-Adjunct.2019.00-34}, issn = {}, month = oct, }
2018
- IEEE TVCG
HySAR: Hybrid Material Rendering by an Optical See-Through Head-Mounted Display with Spatial Augmented Reality ProjectionIEEE Transactions on Visualization and Computer Graphics, Apr 2018Spatial augmented reality (SAR) pursues realism in rendering materials and objects. To advance this goal, we propose a hybrid SAR (HySAR) that combines a projector with optical see-through head-mounted displays (OST-HMD). In an ordinary SAR scenario with co-located viewers, the viewers perceive the same virtual material on physical surfaces. In general, the material consists of two components: a view-independent (VI) component such as diffuse reflection, and a view-dependent (VD) component such as specular reflection. The VI component is static over viewpoints, whereas the VD should change for each viewpoint even if a projector can simulate only one viewpoint at one time. In HySAR, a projector only renders the static VI components. In addition, the OST-HMD renders the dynamic VD components according to the viewer’s current viewpoint. Unlike conventional SAR, the HySAR concept theoretically allows an unlimited number of co-located viewers to see the correct material over different viewpoints. Furthermore, the combination enhances the total dynamic range, the maximum intensity, and the resolution of perceived materials. With proof-of-concept systems, we demonstrate HySAR both qualitatively and quantitatively with real objects. First, we demonstrate HySAR by rendering synthetic material properties on a real object from different viewpoints. Our quantitative evaluation shows that our system increases the dynamic range by 2.24 times and the maximum intensity by 2.12 times compared to an ordinary SAR system. Second, we replicate the material properties of a real object by SAR and HySAR, and show that HySAR outperforms SAR in rendering VD specular components.
@article{hamasaki2018hysar, author = {Hamasaki, Takumi and Itoh, Yuta and Hiroi, Yuichi and Iwai, Daisuke and Sugimoto, Maki}, journal = {IEEE Transactions on Visualization and Computer Graphics}, title = {HySAR: Hybrid Material Rendering by an Optical See-Through Head-Mounted Display with Spatial Augmented Reality Projection}, year = {2018}, volume = {24}, number = {4}, pages = {1457-1466}, keywords = {Augmented realtiy;Head-mounted displays;Rendering (computer graphics);Optical reflection;Adaptive optics;Optical see-through displays;hybrid material rendering;spatial augmented reality}, doi = {10.1109/TVCG.2018.2793659}, issn = {1941-0506}, month = apr, }
2017
- Augmented Human
AdaptiVisor: assisting eye adaptation via occlusive optical see-through head-mounted displaysYuichi Hiroi, Yuta Itoh, Takumi Hamasaki, and Maki SugimotoIn Proceedings of the 8th Augmented Human International Conference, Silicon Valley, California, USA, Apr 2017Brightness adaptation is a fundamental ability in human visual system, and adjusts various levels of darkness and light. While this ability is continuously used, and it can mostly handle sudden lighting changes in the environment, the adaptation could still take several minutes. Moreover, during the adaptation, the color perception changes as well. This slow reactivity and perception change of the eyes could lead to mistakes for tasks performed in dazzling or temporally high-contrast environments such as when driving into the sun or during a welding process.We present AdaptiVisor, a vision augmentation system that assists the brightness adaptation of the eye. Our system selectively modulates the intensity of the light coming into the eyes via occlusion-capable Optical See-Through Head-Mounted Displays (OST-HMD). An integrated camera captures highlights and brightness in the environment via high-dynamic range capture, and our display system selectively dims or enhances part of field of views so that the user would not perceive rapid brightness changes. We build a proof-of-concept system to evaluate the feasibility of the adaptation assistance by combining a transmissive LCD panel and an OST-HMD, and test it with a user-perspective, view-point camera. The evaluation shows that the system decreases the overexposed area in a scene to 1/15th, and enhances the color by reducing majorly underexposed area to half. We also include a preliminary user trial and it indicates that the system also works for real eyes for the HMD part and to some extent for the LCD.
@inproceedings{hiroi2017adaptivisor, author = {Hiroi, Yuichi and Itoh, Yuta and Hamasaki, Takumi and Sugimoto, Maki}, title = {AdaptiVisor: assisting eye adaptation via occlusive optical see-through head-mounted displays}, year = {2017}, isbn = {9781450348355}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3041164.3041178}, doi = {10.1145/3041164.3041178}, booktitle = {Proceedings of the 8th Augmented Human International Conference}, articleno = {9}, numpages = {9}, keywords = {AdaptiVisor, augmented reality, brightness adaptation, head-mounted displays, near-eye displays, occlusive HMD, optical see-through, vision augmentation}, location = {Silicon Valley, California, USA}, series = {AH '17}, } - IEEE VR PosterHySAR: Hybrid material rendering by an optical see-through head-mounted display with spatial augmented reality projectionIn 2017 IEEE Virtual Reality (VR), Mar 2017
We propose a hybrid SAR concept combining a projector and Optical See-Through Head-Mounted Displays (OST-HMD). Our proposed hybrid SAR system utilizes OST-HMD as an extra rendering layer to render a view-dependent property in OST-HMDs according to the viewer’s viewpoint. Combined with view-independent components created by a static projector, the viewer can see richer material contents. Unlike conventional SAR systems, our system theoretically allows unlimited number of viewers seeing enhanced contents in the same space while keeping the existing SAR experiences. Furthermore, the system enhances the total dynamic range, the maximum intensity, and the resolution of perceived materials. With a proof-of-concept system that consists of a projector and an OST-HMD, we qualitatively demonstrate that our system successfully creates hybrid rendering on a hemisphere object from five horizontal viewpoints. Our quantitative evaluation also shows that our system increases the dynamic range by 2.1 times and the maximum intensity by 1.9 times compared to an ordinary SAR system.
@inproceedings{7892251, author = {Hiroi, Yuichi and Itoh, Yuta and Hamasaki, Takumi and Iwai, Daisuke and Sugimoto, Maki}, booktitle = {2017 IEEE Virtual Reality (VR)}, title = {HySAR: Hybrid material rendering by an optical see-through head-mounted display with spatial augmented reality projection}, year = {2017}, volume = {}, number = {}, pages = {211-212}, keywords = {Rendering (computer graphics);Dynamic range;Spatial resolution;Cameras;Optical imaging;Electronic mail;H.5.1 [Information Interfaces and Presentation]: Multimedia Information Systems — Artificial, augmented, and virtual realities}, doi = {10.1109/VR.2017.7892251}, issn = {2375-5334}, month = mar, }
2016
- ACM TEI
MARCut: Marker-based Laser Cutting for Personal Fabrication on Existing ObjectsTakashi Kikuchi, Yuichi Hiroi, Ross T. Smith, Bruce H. Thomas, and Maki SugimotoIn Proceedings of the TEI ’16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction, Eindhoven, Netherlands, Mar 2016Typical personal fabrication using a laser cutter allows objects to be created from raw material and the engraving of existing objects. Current methods to precisely align an object with the laser is a difficult process due to indirect manipulations. In this paper, we propose a marker-based system as a novel paradigm for direct interactive laser cutting on existing objects. Our system, MARCut, performs the laser cutting based on tangible markers that are applied directly onto the object to express the design. Two types of markers are available; hand constructed Shape Markers that represent the desired geometry, and Command Markers that indicate the operational parameters such as cut, engrave or material.
@inproceedings{kikuchi2016marcut, author = {Kikuchi, Takashi and Hiroi, Yuichi and Smith, Ross T. and Thomas, Bruce H. and Sugimoto, Maki}, title = {MARCut: Marker-based Laser Cutting for Personal Fabrication on Existing Objects}, year = {2016}, isbn = {9781450335829}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/2839462.2856549}, doi = {10.1145/2839462.2856549}, booktitle = {Proceedings of the TEI '16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction}, pages = {468–474}, numpages = {7}, keywords = {Laser Cutting, Marker-based, Personal Fabrication}, location = {Eindhoven, Netherlands}, series = {TEI '16}, } - SIGGRAPH E-tech
Laplacian vision: augmenting motion prediction via optical see-through head-mounted displays and projectorsYuta Itoh, Yuichi Hiroi, Jiu Otsuka, Maki Sugimoto, Jason Orlosky, Kiyoshi Kiyokawa, and Gudrun KlinkerIn ACM SIGGRAPH 2016 Emerging Technologies, Anaheim, California, Mar 2016@inproceedings{itoh2016laplacian, author = {Itoh, Yuta and Hiroi, Yuichi and Otsuka, Jiu and Sugimoto, Maki and Orlosky, Jason and Kiyokawa, Kiyoshi and Klinker, Gudrun}, title = {Laplacian vision: augmenting motion prediction via optical see-through head-mounted displays and projectors}, year = {2016}, isbn = {9781450343725}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/2929464.2949028}, doi = {10.1145/2929464.2949028}, booktitle = {ACM SIGGRAPH 2016 Emerging Technologies}, articleno = {13}, location = {Anaheim, California}, series = {SIGGRAPH '16}, }
2015
- ISMAR Poster
Remote Welding Robot Manipulation Using Multi-view ImagesYuichi Hiroi, Kei Obata, Katsuhiro Suzuki, Naoto Ienaga, Maki Sugimoto, Hideo Saito, and Tadashi TakamaruIn 2015 IEEE International Symposium on Mixed and Augmented Reality, Sep 2015This paper proposes a remote welding robot manipulation system by using multi-view images. After an operator specifies two-dimensional path on images, the system transforms it into three-dimensional path and displays the movement of the robot by overlaying graphics with images. The accuracy of our system is sufficient to weld objects when combining with a sensor in the robot. The system allows the non-expert operator to weld objects remotely and intuitively, without the need to create a 3D model of a processed object beforehand.
@inproceedings{hiroi2015remote, author = {Hiroi, Yuichi and Obata, Kei and Suzuki, Katsuhiro and Ienaga, Naoto and Sugimoto, Maki and Saito, Hideo and Takamaru, Tadashi}, booktitle = {2015 IEEE International Symposium on Mixed and Augmented Reality}, title = {Remote Welding Robot Manipulation Using Multi-view Images}, year = {2015}, volume = {}, number = {}, pages = {128-131}, keywords = {Cameras;Welding;Robot vision systems;Three-dimensional displays;Robot kinematics;Remote robot control;multi-view images;industrial robot;augmented reality}, doi = {10.1109/ISMAR.2015.38}, issn = {}, month = sep, }
