Human Activity Recognition from Wearable Sensor Data Using Self-Attention

@inproceedings{Mahmud2020HumanAR,
  title={Human Activity Recognition from Wearable Sensor Data Using Self-Attention},
  author={Saif Mahmud and M Tanjid Hasan Tonmoy and Kishor Kumar Bhaumik and A. K. M. Mahbubur Rahman and M. Ashraful Amin and Mohammad Shoyaib and Muhammad Asif Hossain Khan and Amin Ahsan Ali},
  booktitle={European Conference on Artificial Intelligence},
  year={2020}
}
Human Activity Recognition from body-worn sensor data poses an inherent challenge in capturing spatial and temporal dependencies of time-series signals. In this regard, the existing recurrent or convolutional or their hybrid models for activity recognition struggle to capture spatio-temporal context from the feature space of sensor reading sequence. To address this complex problem, we propose a self-attention based neural network model that foregoes recurrent architectures and utilizes… 
[PDF] Semantic Reader
38 Citations
Highly Influential Citations
6
Background Citations
14
Methods Citations
18
Results Citations
1

Figures and Tables from this paper

38 Citations

Hierarchical Self Attention Based Autoencoder for Open-Set Human Activity Recognition

The proposed self attention based approach combines data hierarchically from different sensor placements across time to classify closed-set activities and it obtains notable performance improvement over state-of-the-art models on five publicly available datasets.

A Deep Dilated Convolutional Self-attention Model for Multimodal Human Activity Recognition

This work proposes a self-attention based deep dilated convolution network that uses two channels, named temporal channel and spatial channel, respectively, to extract the readings- over-time and time-over-readings features from sensor signals.

Contrastive Accelerometer–Gyroscope Embedding Model for Human Activity Recognition

A two-stream convolutional neural network model is proposed that processes the accelerometer and gyroscope signals separately and is named the Contrastive Accelerometer–Gyroscope Embedding (CAGE) model, which outperforms preceding HAR models in four publicly available benchmarks.

Contrastive Self-supervised Learning for Sensor-based Human Activity Recognition

One of the most recent advances in Self-Supervised Leaning (SSL) is combined with a powerful transformer-based encoder to introduce a Contrastive Self-supervised learning approach to Sensor-based Human Activity Recognition (CSSHAR) that learns feature representations from unlabeled sensory data.

Contrastive Predictive Coding for Human Activity Recognition

This work introduces the Contrastive Predictive Coding (CPC) framework to human activity recognition, which captures the temporal structure of sensor data streams and leads to significantly improved recognition performance when only small amounts of labeled training data are available.

Dilated causal convolution with multi-head self attention for sensor human activity recognition

A network architecture based on dilated causal convolution and multi-head self-attention mechanisms that entirely dispense recurrent architectures to make efficient computation and maintain the ordering of the time steps is proposed.

Temporal Feature Alignment in Contrastive Self-Supervised Learning for Human Activity Recognition

This work upgrades a contrastive SSL framework, namely SimCLR, widely used in various applications by introducing a temporal feature alignment procedure for Human Activity Recognition by integrating a dynamic time warping algorithm in a latent space to force features to be aligned in a temporal dimension.

Dual-Branch Interactive Networks on Multichannel Time Series for Human Activity Recognition

A new Dual-branch Interactive Network (DIN) that inherits the advantages from both CNN and Transformer to handle multi-channel time series for HAR and utilizes two-stream architecture to disentangle local and global features by performing conv-embedding and patch- embedding.

Cross-Domain Activity Recognition Using Shared Representation in Sensor Data

The proposed multi-domain learning network on six real-world sensor activity datasets outperforms the existing methods by applying only 50% of the labelled data, confirming the efficacy of the proposed approach as a generic model to learn human activities from different but related domains in a joint effort to reduce the number of required models and thus improve system efficiency.

Masked reconstruction based self-supervision for human activity recognition

Masked reconstruction is introduced as a viable self-supervised pre-training objective for human activity recognition and its effectiveness in comparison to state-of-the-art unsupervised learning techniques is explored.

References

SHOWING 1-10 OF 32 REFERENCES

AttnSense: Multi-level Attention Mechanism For Multimodal Human Activity Recognition

AttnSense introduces the framework of combining attention mechanism with a convolutional neural network and a Gated Recurrent Units network to capture the dependencies of sensing signals in both spatial and temporal domains, which shows advantages in prioritized sensor selection and improves the comprehensibility.

GRU-based Attention Mechanism for Human Activity Recognition

The introduced model has achieved better performance with respect to the well-defined evaluation metrics in both uniform and imbalanced class distribution than the existing state-of-the-art deep learning based model.

Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables

This paper rigorously explore deep, convolutional, and recurrent approaches across three representative datasets that contain movement data captured with wearable sensors, and describes how to train recurrent approaches in this setting, introduces a novel regularisation approach, and illustrates how they outperform the state-of-the-art on a large benchmark dataset.

Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition

A generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which is suitable for multimodal wearable sensors, does not require expert knowledge in designing features, and explicitly models the temporal dynamics of feature activations is proposed.

On attention models for human activity recognition

This paper introduces attention models into HAR research as a data driven approach for exploring relevant temporal context and constructs attention models for HAR by adding attention layers to a state-of-the-art deep learning HAR model (DeepConvLSTM).

Understanding and improving recurrent networks for human activity recognition by continuous attention

Qualitative analysis shows that the attention learned by the models agree well with human intuition, and these two mechanisms adaptively focus on important signals and sensor modalities are proposed.

On the role of features in human activity recognition

In a systematic exploration, eight different feature extraction methods are evaluated, including conventional heuristics and recent representation learning methods, and their capabilities for effective activity recognition on five benchmarks are assessed.

Multi-modal Convolutional Neural Networks for Activity Recognition

A multi-modal CNN where 2D kernels are used in both convolutional and pooling layers, to capture local dependency over time as well as spatial dependency over sensors, compared to several state of the art methods.

Nurse care activity recognition: a GRU-based approach with attention mechanism

This work proposes a Gated Recurrent Unit (GRU) model with attention mechanism to recognize the nurse activities and obtains approximately 66.43% accuracy for person-wise accuracy, leaving out cross validation.

Activity Recognition from User-Annotated Acceleration Data

This is the first work to investigate performance of recognition algorithms with multiple, wire-free accelerometers on 20 activities using datasets annotated by the subjects themselves, and suggests that multiple accelerometers aid in recognition.