# PU-Net: Point Cloud Upsampling Network

@article{Yu2018PUNetPC,
title={PU-Net: Point Cloud Upsampling Network},
author={Lequan Yu and Xianzhi Li and Chi-Wing Fu and Daniel Cohen-Or and Pheng-Ann Heng},
journal={2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition},
year={2018},
pages={2790-2799}
}
• Published 21 January 2018
• Computer Science
• 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Learning and analyzing 3D point clouds with deep networks is challenging due to the sparseness and irregularity of the data. [] Key Method Our network is applied at a patch-level, with a joint loss function that encourages the upsampled points to remain on the underlying surface with a uniform distribution. We conduct various experiments using synthesis and scan data to evaluate our method and demonstrate its superiority over some baseline methods and an optimization-based method. Results show that our…
265 Citations

## Figures and Tables from this paper

PU-Flow: a Point Cloud Upsampling Networkwith Normalizing Flows
• Computer Science
ArXiv
• 2021
A novel deep learning-based model, called PU-Flow, which incorporates normalizing ﬂows and weight prediction techniques to produce dense points uniformly distributed on the underlying surface, and outperforms state-of-the-art methods in terms of reconstruction quality, proximity-to-surface accuracy, and computation efﬁciency.
Data-driven Upsampling of Point Clouds
• Computer Science
Comput. Aided Des.
• 2019
PU-Transformer: Point Cloud Upsampling Transformer
• Computer Science, Environmental Science
ArXiv
• 2021
To activate the transformer’s strong capability in representing features, a new variant of a multi-head self-attention structure is developed to enhance both point-wise and channel-wise relations of the feature map.
PUGeo-Net: A Geometry-centric Network for 3D Point Cloud Upsampling
• Computer Science
ECCV
• 2020
This paper addresses the problem of generating uniform dense point clouds to describe the underlying geometric structures from given sparse point clouds. Due to the irregular and unordered nature,
Deep Magnification-Flexible Upsampling Over 3D Point Clouds
• Computer Science
IEEE Transactions on Image Processing
• 2021
This paper addresses the problem of generating dense point clouds from given sparse point clouds to model the underlying geometric structures of objects/scenes by taking advantage of the linear approximation theorem and designing a lightweight neural network to adaptively learn unified and sorted interpolation weights as well as the high-order refinements.
Meta-PU: An Arbitrary-Scale Upsampling Network for Point Cloud
• Computer Science
IEEE transactions on visualization and computer graphics
• 2021
This work proposes a novel method called Meta-PU" to firstly support point cloud upsampling of arbitrary scale factors with a single model and even outperforms the existing methods trained for a specific scale factor only.
SPU-Net: Self-Supervised Point Cloud Upsampling by Coarse-to-Fine Reconstruction with Self-Projection Optimization
• Computer Science
IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
• 2022
This work proposes a self-supervised point cloud upsampling network, named SPU-Net, that achieves comparable performances to state-of-the-art supervised methods and proposes a coarse-to-fine reconstruction framework, which contains two main components: point feature extraction and point feature expansion.
PU-GAN: A Point Cloud Upsampling Adversarial Network
• Computer Science
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
• 2019
A new point cloud upsampling network called PU-GAN, which is formulated based on a generative adversarial network (GAN), to learn a rich variety of point distributions from the latent space and upsample points over patches on object surfaces.
Deep Point Cloud Reconstruction
• Computer Science
ArXiv
• 2021
A deep point cloud reconstruction network consisting of a 3D sparse stacked-hourglass network as for the initial densification and denoising, and a refinement via transformers converting the discrete voxels into 3D points called amplified positional encoding is proposed.
Cascaded Refinement Network for Point Cloud Completion
• Computer Science
2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 2020
This work proposes a cascaded refinement network together with a coarse-to-fine strategy to synthesize the detailed object shapes and designs a patch discriminator that guarantees every local area has the same pattern with the ground truth to learn the complicated point distribution.

## References

SHOWING 1-10 OF 43 REFERENCES
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Computer Science
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
• 2017
This paper designs a novel type of neural network that directly consumes point clouds, which well respects the permutation invariance of points in the input and provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing.
Deep Learning on Point Sets for 3 D Classification and Segmentation
• C. Qi
• Computer Science
• 2016
This paper designs a novel type of neural network that directly consumes point clouds and well respects the permutation invariance of points in the input, and provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing.
PointCNN
• Computer Science
NIPS 2018
• 2018
This work proposes to learn a X-transformation from the input points, which is used for simultaneously weighting the input features associated with the points and permuting them into latent potentially canonical order, and calls it PointCNN.
PCPNet Learning Local Shape Properties from Raw Point Clouds
• Computer Science
Comput. Graph. Forum
• 2018
The utility of the PCPNET approach in the context of shape reconstruction is demonstrated, by showing how it can be used to extract normal orientation information from point clouds.
Representation Learning and Adversarial Generation of 3D Point Clouds
• Computer Science
ArXiv
• 2017
This paper introduces a deep autoencoder network for point clouds, which outperforms the state of the art in 3D recognition tasks, and designs GAN architectures to generate novel point-clouds.
Pointwise Convolutional Neural Networks
• Computer Science
2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
• 2018
This paper presents a convolutional neural network for semantic segmentation and object recognition with 3D point clouds, and at the core of this network is point-wise convolution, a new convolution operator that can be applied at each point of a point cloud.
PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Computer Science
NIPS
• 2017
A hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set and proposes novel set learning layers to adaptively combine features from multiple scales to learn deep point set features efficiently and robustly.
Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction
• Computer Science
AAAI
• 2018
This paper uses 2D convolutional operations to predict the 3D structure from multiple viewpoints and jointly apply geometric reasoning with 2D projection optimization, and introduces the pseudo-renderer, a differentiable module to approximate the true rendering operation, to synthesize novel depth maps for optimization.
A Point Set Generation Network for 3D Object Reconstruction from a Single Image
• Computer Science
2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
• 2017
This paper addresses the problem of 3D reconstruction from a single image, generating a straight-forward form of output unorthordox, and designs architecture, loss function and learning paradigm that are novel and effective, capable of predicting multiple plausible 3D point clouds from an input image.
PointCNN: Convolution On $\mathcal{X}$-Transformed Points
• Computer Science
• 2018
The proposed method is a generalization of typical CNNs to feature learning from point clouds, thus it is called PointCNN, and experiments show that it achieves on par or better performance than state-of-the-art methods on multiple challenging benchmark datasets and tasks.