Corpus ID: 203952114

3D Manhattan Room Layout Reconstruction from a Single 360 Image

  title={3D Manhattan Room Layout Reconstruction from a Single 360 Image},
  author={Chuhang Zou and Jheng-Wei Su and Chi-Han Peng and Alex Colburn and Qi Shan and Peter Wonka and Hung-kuo Chu and Derek Hoiem},
Recent approaches for predicting layouts from 360 panoramas produce excellent results. These approaches build on a common framework consisting of three steps: a pre-processing step based on edge-based alignment, prediction of layout elements, and a post-processing step by fitting a 3D layout to the layout elements. Until now, it has been difficult to compare the methods due to multiple different design decisions, such as the encoding network (e.g. SegNet or ResNet), type of elements predicted… Expand
3D Room Layout Estimation Beyond the Manhattan World Assumption
Experimental results show the proposed new training and post-processing method for 3D room layout estimation outperforms state-of-the-art approaches by a large margin in predicting visible room layout. Expand
SSLayout360: Semi-Supervised Indoor Layout Estimation from 360° Panorama
  • P. Tran
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2021
This work proposes the first approach to learn representations of room corners and boundaries by using a combination of labeled and unlabeled data for improved layout estimation in a 360° panoramic scene. Expand
AtlantaNet: Inferring the 3D Indoor Layout from a Single $360^\circ $ Image Beyond the Manhattan World Assumption
This work introduces a novel end-to-end approach to predict a 3D room layout from a single panoramic image, leveraging Recurrent Neural Networks (RNNs) to capture long-range geometric patterns, and exploiting a customized training strategy based on domain-specific knowledge. Expand
Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling
This paper presents a new synthetic dataset, Structured3D, with the aim of providing large-scale photo-realistic images with rich 3D structure annotations for a wide spectrum of structured 3D modeling tasks, and takes advantage of the availability of professional interior designs to automatically extract 3D structures from them. Expand
Zillow Indoor Dataset: Annotated Floor Plans With 360° Panoramas and 3D Room Layouts
A large indoor dataset with 71,474 panoramas from 1,524 real unfurnished homes, ZInD provides annotations of 3D room layouts, 2D and 3D floor plans, panorama location in the floor plan, and locations of windows and doors, which is the largest real dataset with layout annotations. Expand
Single-Shot Cuboids: Geodesics-based End-to-end Manhattan Aligned Layout Estimation from Spherical Panoramas
This work is the first to directly infer Manhattan-aligned outputs, and introduces the geodesic heatmaps and loss and a boundary-aware center of mass calculation that facilitate higher quality keypoint estimation in the spherical domain. Expand
PanoDR: Spherical Panorama Diminished Reality for Indoor Scenes
This work proposes a model that initially predicts the structure of a indoor scene and then uses it to guide the reconstruction of an empty – background only – representation of the same scene, to ensure structure-aware counterfactual inpainting. Expand
Monte Carlo Scene Search for 3D Scene Understanding
This work proposes a modification of the Monte Carlo Tree Search (MCTS) algorithm to retrieve objects and room layouts from noisy RGB-D scans and uses its adapted MCTS algorithm, which has few easy-to-tune hyperparameters and can optimise general losses, to optimise the posterior probability of objects andRoom layout hypotheses given the RGB- D data. Expand
State‐of‐the‐art in Automatic 3D Reconstruction of Structured Indoor Environments
An up‐to‐date integrative view of the field, bridging complementary views coming from computer graphics and computer vision is provided, and the structure of output models and the priors exploited to bridge the gap between imperfect sources and desired output are defined. Expand
Deep3DLayout: 3D Reconstruction of an Indoor Layout from a Spherical Panoramic Image
Figure 1. From a single cluttered panoramic image, our end-to-end deep network recovers, at interactive rates, a watertight 3D mesh of the underlying architectural structure. The graph convolutionalExpand


LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image
An algorithm to predict room layout from a single image that generalizes across panoramas and perspective images, cuboid layouts and more general layouts (e.g. "L"-shape room) is proposed, which achieves among the best accuracy for perspective images and can handle both cuboid-shaped and moregeneral Manhattan layouts. Expand
HorizonNet: Learning Room Layout With 1D Representation and Pano Stretch Data Augmentation
The proposed network, HorizonNet, trained for predicting 1D layout, outperforms previous state-of-the-art approaches and can diversify panorama data and be applied to other panorama-related learning tasks. Expand
Efficient 3D Room Shape Recovery from a Single Panorama
  • Hao Yang, H. Zhang
  • Mathematics, Computer Science
  • 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2016
An algorithm that can automatically infer a 3D shape from a collection of partially oriented superpixel facets and line segments and is efficient, that is, the inference time for each panorama is less than 1 minute. Expand
Rent3D: Floor-plan priors for monocular layout estimation
This paper exploits the floor plan to impose aspect ratio constraints across the layouts of different rooms, as well as to extract semantic information, e.g., the location of windows which are marked in floor plans, which can significantly help in resolving the challenging room-apartment alignment problem. Expand
Automatic 3D Indoor Scene Modeling from Single Panorama
A system that automatically extracts 3D geometry of an indoor scene from a single 2D panorama and uses the recovered layout to guide shape estimation of the remaining objects using their normal information is described. Expand
RoomNet: End-to-End Room Layout Estimation
This paper predicts the locations of the room layout keypoints using RoomNet, an end-to-end trainable encoder-decoder network and presents optional extensions to the RoomNet architecture such as including recurrent computations and memory units to refine the keypoint locations under the same parametric capacity. Expand
PanoContext: A Whole-Room 3D Context Model for Panoramic Scene Understanding
Experiments show that solely based on 3D context without any image region category classifier, the proposed whole-room context model can achieve a comparable performance with the state-of-the-art object detector, demonstrating that when the FOV is large, context is as powerful as object appearance. Expand
Recovering Surface Layout from an Image
This paper takes the first step towards constructing the surface layout, a labeling of the image intogeometric classes, to learn appearance-based models of these geometric classes, which coarsely describe the 3D scene orientation of each image region. Expand
DuLa-Net: A Dual-Projection Network for Estimating Room Layouts From a Single RGB Panorama
A deep learning framework, called DuLa-Net, to predict Manhattan-world 3D room layouts from a single RGB panorama that leverages two projections of the panorama at once, namely the equirectangular panorama-view and the perspective ceiling-view, that each contains different clues about the room layouts. Expand
Efficient Exact Inference for 3D Indoor Scene Understanding
This paper presents a novel branch and bound approach which splits the label space in terms of candidate sets of 3D layouts, and efficiently bounds the potentials in these sets by restricting the contribution of each individual face. Expand