3D Scene Completion

Depth Completion Pipeline

A Contour Completion Model for Augmenting Surface Reconstructions

Nathan Silberman, Lior Shapira, Ran Gal, Pushmeet Kohli

To appear in the proceedings of ECCV 2014

Abstract

The availability of commodity depth sensors such as Kinect has enabled development of  methods which can densely reconstruct arbitrary scenes. While the results of these  methods are accurate and visually appealing, they are quite often incomplete. This is either due to the fact that only part of the space was visible during the data capture process or due to the surfaces being occluded by other objects in the scene. In this paper, we address the problem of completing and refining such reconstructions. We propose a method for scene completion that can infer the layout of the complete room and
the full extent of partially occluded objects. We propose a new probabilistic model, Contour Completion Random Fields, that allows us to complete the boundaries of occluded surfaces. We evaluate our method on synthetic and real world reconstructions of 3D scenes and show that it quantitatively and qualitatively outperforms standard methods. We created a large data set of partial and complete reconstructions which we will make
available to the community as a benchmark for the scene completion task. Finally,
we demonstrate the practical utility of our algorithm via an augmented-reality
application where objects interact with the completed reconstructions inferred
by our method.

Files

Paper, Supplemental, Video

Pipeline

Scene Completion Pipeline Large

(a) Pointcloud of scene (b) Detect planar surfaces in pointcloud (c) Classify planes into floor, ceiling, walls and inner surfaces (d) Use contour completion to complete room boundaries (e) Use contour completion to complete interior objects in room (f) Final result

Bibtex

@inproceedings{silberman2014a, 
 author = {Nathan Silberman and
 Lior Shapira and
 Ran Gal and
 Pushmeet Kohli},
 title = {A Contour Completion Model for Augmenting Surface Reconstructions},
 booktitle = {Proceedings of ECCV 2014},
 publisher = {Springer}, 
 year = {2014},
 isbn = {---},
 pages = {---}
}

Leave a Reply