Through-Wall Human Pose Estimation Using Radio Signals

  title={Through-Wall Human Pose Estimation Using Radio Signals},
  author={Mingmin Zhao and Tianhong Li and Mohammad Abu Alsheikh and Yonglong Tian and Hang Zhao and Antonio Torralba and Dina Katabi},
  journal={2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition},
This paper demonstrates accurate human pose estimation through walls and occlusions. [] Key Method Specifically, during training the system uses synchronized wireless and visual inputs, extracts pose information from the visual stream, and uses it to guide the training process. Once trained, the network uses only the wireless signal for pose estimation. We show that, when tested on visible scenes, the radio-based system is almost as accurate as the vision-based system used to train it. Yet, unlike vision…

Figures and Tables from this paper

HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar

A novel human pose estimation benchmark, Human Pose with Millimeter Wave Radar (HuPR), that includes synchronized vision and radio signal components is introduced and a cross-modality training framework that leverages the ground-truth 2D keypoints representing human body joints for training is proposed.

WiFi-Based Human Pose Image Generation

A framework to generate the target human pose images by combining the wireless signals with an initial optical image is proposed and results show that it achieves higher accuracy than the state-of-the-art WiFi-based pose estimation method and better visual quality than the State of theart human generation method.

Dense Human Pose Estimation From WiFi

A deep neural network is developed that maps the phase and amplitude of Wi-Fi signals to UV coordinates within 24 human regions and can estimate the dense pose of multiple subjects, with comparable performance to image-based approaches, by utilizing WiFi signals as the only input.

3D Human Pose Estimation for Free-from and Moving Activities Using WiFi

GoPose is presented, a 3D skeleton-based human pose estimation system that uses WiFi devices at home that does not require a user to wear or carry any sensors and can reuse the WiFi devices that already exist in a home environment for mass adoption.

GoPose: 3D Human Pose Estimation Using WiFi

GoPose is environment-independent and is highly accurate in constructing 3D poses for mobile users and achieves around 4.7cm of accuracy under various scenarios including tracking unseen activities and under NLoS scenarios.

Capturing Human Pose Using mmWave Radar

This paper proposes an accurate human pose estimation system using the 77GHz millimeter wave radar for the first time that people use off-the-shelf millimeterWave radar to complete such a task.

Through-Wall Human Mesh Recovery Using Radio Signals

RF-Avatar, a neural network model that can estimate 3D meshes of the human body in the presence of occlusions, baggy clothes, and bad lighting conditions, and even through walls, is presented.

Human Silhouette and Skeleton Video Synthesis through Wi-Fi signals

A novel two-branch generative neural network is presented that effectively maps radio data into visual features, following a teacher-student design that exploits a cross-modality supervision strategy.

Recovering Human Pose and Shape From Through-the-Wall Radar Images

A deep learning-based human pose and shape recovery method from TWRI images Inspired by cross-modal learning, this method follows a teacher–student learning pipeline that avoids the heavy cost of manual labeling.

DensePose From WiFi

A deep neural network is developed that maps the phase and amplitude of WiFi signals to UV coordinates within 24 human regions and can estimate the dense pose of multiple subjects, with comparable performance to image-based approaches, by utilizing WiFi signals as the only input.



2D Human Pose Estimation: New Benchmark and State of the Art Analysis

A novel benchmark "MPII Human Pose" is introduced that makes a significant advance in terms of diversity and difficulty, a contribution that is required for future developments in human body models.

RMPE: Regional Multi-person Pose Estimation

This paper proposes a novel regional multi-person pose estimation (RMPE) framework to facilitate pose estimation in the presence of inaccurate human bounding boxes and can achieve 76:7 mAP on the MPII (multi person) dataset.

Thin-Slicing Network: A Deep Structured Model for Pose Estimation in Videos

The proposed end-to-end architecture is evaluated on two widely used benchmarks for video-based pose estimation (Penn Action and JHMDB datasets) and outperforms several state-of-the-art methods.

DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation

An approach that jointly solves the tasks of detection and pose estimation: it infers the number of persons in a scene, identifies occluded body parts, and disambiguates body parts between people in close proximity of each other is proposed.

Capturing the human figure through a wall

RF-Capture tracks the 3D positions of a person's limbs and body parts even when the person is fully occluded from its sensor, and does so without placing any markers on the subject's body.

Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

We present an approach to efficiently detect the 2D pose of multiple people in an image. The approach uses a nonparametric representation, which we refer to as Part Affinity Fields (PAFs), to learn

Structured Feature Learning for Pose Estimation

A bi-directional tree structured model is proposed, so that the feature channels at a body joint can well receive information from other joints, and the proposed framework improves feature learning substantially.

Towards Accurate Multi-person Pose Estimation in the Wild

This work proposes a method for multi-person detection and 2-D pose estimation that achieves state-of-art results on the challenging COCO keypoints task by using a novel form of keypoint-based Non-Maximum-Suppression (NMS), instead of the cruder box-level NMS, and by introducing a novel aggregation procedure to obtain highly localized keypoint predictions.

DeeperCut: A Deeper, Stronger, and Faster Multi-person Pose Estimation Model

The goal of this paper is to advance the state-of-the-art of articulated pose estimation in scenes with multiple people. To that end we contribute on three fronts. We propose (1) improved body part

3D Convolutional Neural Networks for Human Action Recognition

A novel 3D CNN model for action recognition that extracts features from both the spatial and the temporal dimensions by performing 3D convolutions, thereby capturing the motion information encoded in multiple adjacent frames.