HOME >> English >> Augmented Environment Mapping

Augmented Environment Mapping: Appearance Editing of Glossy Surfaces in Spatial Augmented Reality

We propose a novel spatial augmented reality (SAR) framework to edit the appearance of physical glossy surfaces. The key idea is utilizing the specular reflection, which was a major distractor in conventional SAR systems. Namely, we spatially manipulate the appearance of an environmental surface, which is observed through the specular reflection. We use a stereoscopic display to present two appearances with disparity on the environmental surface, by which the depth of the specularly reflected visual information corresponds to the glossy surface. We refer to this method as augmented environment mapping (AEM). The paper describes its principle, followed by three different implementation approaches inspired by typical virtual and augmented reality approaches. We confirmed the feasibility of AEM through both quantitative and qualitative experiments using prototype systems.

Augmented environment mapping for manipulating the appearance of a glossy surface (a curved mirror): (a) an overview of a system using a flat panel LCD (three more motion capture cameras are used but out of the capturing camera's field of view), (b) appearance manipulation results from different observation positions, (c) another view of (b) where a displayed image on the LCD is changed according to the observation position. Note that the displayed image was rendered for monoscopic vision for a reader's better visibility.


  • Takumi Kaminokado, Daisuke Iwai, and Kosuke Sato, "Augmented Environment Mapping for Appearance Editing of Glossy Surfaces," In Proc. IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2019. [pdf]