Dense Planar SLAM

Using higher-level entities during mapping has the potential to improve camera localisation performance and give substantial perception capabilities to real-time 3D SLAM systems. We present an efficient new real-time approach which densely maps an environment using bounded planes and surfels extracted from depth images (like those produced by RGB-D sensors or dense multi-view stereo reconstruction). Our method offers the every-pixel descriptive power of the latest dense SLAM approaches, but takes advantage directly of the planarity of many parts of real-world scenes via a data-driven process to directly regularize planar regions and represent their accurate extent efficiently using an occupancy approach with on-line compression. Large areas can be mapped efficiently and with useful semantic planar structure which enables intuitive and useful AR applications such as using any wall or other planar surface in a scene to display a user’s content.

In the Press

fastcompany logo   The Oculus Rift Can Now Change Your Carpeting, 16 Sep 2014.

digitaltrends logo   This modified Oculus Rift can make the floor transform in front of your eyes (without LSD!), 16 Sep 2014.

gizmodo logo   Researchers Use Oculus Rift And Augmented Reality To Put Facebook On Your Wall, 17 Sep 2014.

slashgear logo   Oculus project puts Facebook on your wall, 17 Sep 2014.

Videos

Figures

dps_stairs

Dense Planar SLAM in action. (left) The stairs of a house have been mapped with both planar and non-planar region surfels. (middle) Normal map shows high-quality reconstruction. Observe the lower quality normals on the highlighted red area lacking planar measurements. (right) Planar-only regions.

dps_kitchen_multitask

We can take advantage of the dense and real-time nature of our system to perform novel Augmented Virtuality (a.k.a Mixed Reality) interactions with fine occlusion handling. For this, we built a hardware setup consisting of an Xtion RGB-D sensor attached to the Oculus Rift DK1.

A user can choose a set of planes to alter the original input with an application display, allowing for instance to read a Facebook Wall on a real wall. The user can still safely navigate an environment without fear of collisions and perform tasks like zooming in/out as natural as walking closer/further from surfaces.

dps_facebookwall_oculus

Paper

Renato F. Salas-Moreno, Ben Glocker, Paul H. J. Kelly and Andrew J. Davison, Dense Planar SLAM, in Proc. International Symposium on Mixed and Augmented Reality (ISMAR), IEEE, September 2014. PDF File