Learning With Feature Evolvable Streams

  title={Learning With Feature Evolvable Streams},
  author={Bo-Jian Hou and Lijun Zhang and Zhi-Hua Zhou},
  journal={IEEE Transactions on Knowledge and Data Engineering},
Learning with streaming data has attracted much attention during the past few years. Though most studies consider data stream with fixed features, in real practice the features may be evolvable. For example, features of data gathered by limited-lifespan sensors will change when these sensors are substituted by new ones. In this article, we propose a novel learning paradigm: Feature Evolvable Streaming Learning where old features would vanish and new features would occur. Rather than relying on… 

Figures and Tables from this paper

Storage Fit Learning with Feature Evolvable Streams

Both theoretical and empirical results validate that the proposed Storage-Fit Feature-Evolvable streaming Learning (SF2EL) approach can preserve the merit of the original feature evolvable learning i.e., can always track the best baseline and thus perform well at any time step.

Learning with Feature and Distribution Evolvable Streams

A novel discrepancy measure for data with evolving feature space and data distribution, named the evolving discrepancy is proposed, and the theory motivates the design of a learning algorithm which is further implemented by deep neural networks.

Prediction With Unpredictable Feature Evolution

This article proposes a novel paradigm: prediction with unpredictable feature evolution (PUFE) where the feature evolution is unpredictable, and gives a theoretical bound on the least number of observed entries to make the overlapping period intact.

Online Deep Learning from Doubly-Streaming Data

A novel OLD3S paradigm is proposed, where a shared latent subspace is discovered to summarize information from the old and new feature spaces, building an intermediate feature mapping relationship.

Online Learning from Capricious Data Streams: A Generative Approach

A novel algorithm, named OCDS (Online learning from Capricious Data Streams), which trains a learner on a universal feature space that establishes relationships between old and new features, so that the patterns learned in the old feature space can be used in the new feature space.

Online Learning from Data Streams with Varying Feature Spaces

A novel online learning algorithm OLVF is proposed to learn from data with arbitrarily varying feature spaces to classify the feature spaces and the instances from feature spaces simultaneously and a feature sparsity method is applied to reduce the model complexity.

Evolving Metric Learning for Incremental and Decremental Features

A new online Evolving Metric Learning (EML) model for incremental and decremental features, which can handle the instance and feature evolutions simultaneously by incorporating with a smoothed Wasserstein metric distance is developed.

Toward Mining Capricious Data Streams: A Generative Approach

A generative graphical model is proposed to model the construction process of a universal feature space that establishes relationships between old and new features, so that the patterns learned in the old feature space can be used in the new feature space.

Incremental Feature Spaces Learning with Label Scarcity

This work proposes a novel incremental Feature spaces Learning with Label Scarcity (FLLS) algorithm, together with its two variants, which combines the online passive-aggressive update rule and margin-maximum principle to jointly update the dynamic classifier in the shared and augmented feature space.

Adaptive Learning for Weakly Labeled Streams

A novel noise transition matrix estimation approach for data streams with scarce noisy labels by online anchor points identification is proposed and an adaptive learning algorithm for weakly labeled data streams via model reuse is proposed to effectively alleviate the negative influence of label noise with unlabeled data.



One-Pass Learning with Incremental and Decremental Features

The OPID approach is an one-pass learning approach, which only needs to scan each instance once and does not need to store the whole data, and thus satisfies the evolving streaming data nature.

Online Incremental Feature Learning with Denoising Autoencoders

This paper proposes an incremental feature learning algorithm to determine the optimal model complexity for large-scale, online datasets based on the denoising autoencoder, which is composed of two processes: adding features and merging features.

Heterogeneous Ensemble for Feature Drifts in Data Streams

An algorithm called HEFT-Stream is proposed that incorporates feature selection into a heterogeneous ensemble to adapt to different types of concept drifts for data streams and outperforms state-of-the-art ensemble classifiers in terms of accuracy, speed, and scalability.

Distribution-Free One-Pass Learning

This paper proposes a simple yet effective approach for distribution-free one-pass learning, without requiring prior knowledge about the change, where every data item can be discarded once scanned.

A Survey on Transfer Learning

The relationship between transfer learning and other related machine learning techniques such as domain adaptation, multitask learning and sample selection bias, as well as covariate shift are discussed.

A framework for on-demand classification of evolving data streams

The aim here is to create a classification system in which the training model can adapt quickly to the changes of the underlying data stream, and proposes an on-demand classification process which can dynamically select the appropriate window of past training data to build the classifier.

A Survey on Ensemble Learning for Data Stream Classification

This work proposes a taxonomy for data stream ensemble learning as derived from reviewing over 60 algorithms, and important aspects such as combination, diversity, and dynamic updates, are thoroughly discussed.

Online learning with kernels

This paper considers online learning in a reproducing kernel Hilbert space, and allows the exploitation of the kernel trick in an online setting, and examines the value of large margins for classification in the online setting with a drifting target.

Streamed Learning: One-Pass SVMs

A single-pass SVM which is based on the minimum enclosing ball of streaming data, and it is shown that the MEB updates for the streaming case can be easily adapted to learn the SVM weight vector in a way similar to using online stochastic gradient updates.

Online Transfer Learning