Dynamic Multi-Projection Mapping

Projection based mixed-reality is an effective tool to create immersive visualizations on real-world objects. This is used in a wide range of applications like art-installations, education, design, stage shows and advertising. Using a multi-projection system allows us to immerse users in an altered reality without the need to wear additional head-gear. In this work, we present a novel, adaptable and real-time projection mapping system, which supports multiple projectors and high quality rendering of dynamic content on surfaces of complex geometrical shape. Our system allows for smooth blending across multiple projectors using a new optimization framework that simulates the diffuse direct light transport of the physical world to continuously adapt the color output of each projector pixel. We present a real-time solution to this optimization problem using off-the-shelf graphics hardware, depth cameras and projectors. Our approach enables us to move projectors, depth camera or objects while maintaining the correct illumination, in real-time, without the need for markers on the object.