Learning Through Deterministic Assignment of Hidden Parameters

  title={Learning Through Deterministic Assignment of Hidden Parameters},
  author={Jian Fang and Shaobo Lin and Zongben Xu},
  journal={IEEE Transactions on Cybernetics},
Supervised learning frequently boils down to determining hidden and bright parameters in a parameterized hypothesis space based on finite input–output samples. The hidden parameters determine the nonlinear mechanism of an estimator, while the bright parameters characterize the linear mechanism. In a traditional learning paradigm, hidden and bright parameters are not distinguished and trained simultaneously in one learning process. Such a one-stage learning (OSL) brings a benefit of theoretical… Expand
Random Sketching for Neural Networks With ReLU
This article borrows the well-known random sketching strategy from kernel methods to transform the training of shallow rectified linear unit (ReLU) nets into a linear least-squares problem, and shows that random Sketching can significantly reduce the computational burden of numerous backpropagation algorithms while maintaining their learning performance. Expand


Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part II)
It is proved that the generalization capability of ELM with Gaussian kernel is essentially worse than that of FNN withGaussian kernel, and it is found that the well-developed coefficient regularization technique can essentially improve thegeneralization capability. Expand
Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part I)
A comprehensive feasibility analysis of ELM is conducted and it is revealed that there also exists some activation functions, which makes the corresponding ELM degrade the generalization capability. Expand
Approximation Methods for Supervised Learning
The main focus is to understand what is the rate of approximation, measured either in expectation or probability, that can be obtained under a given prior fρ ∈ Θ, and what are possible algorithms for obtaining optimal or semioptimal results. Expand
Extreme learning machine: Theory and applications
A new learning algorithm called ELM is proposed for feedforward neural networks (SLFNs) which randomly chooses hidden nodes and analytically determines the output weights of SLFNs which tends to provide good generalization performance at extremely fast learning speed. Expand
Learning with sample dependent hypothesis spaces
This paper proposes an idea of using a larger function class containing the union of all possible hypothesis spaces (varying with the sample) to measure the approximation ability of the algorithm and shows how this idea provides error analysis for two particular classes of learning algorithms in kernel methods: learning the kernel via regularization and coefficient based regularization. Expand
Distributed Learning with Regularized Least Squares
It is shown with error bounds in expectation that the global output function of this distributed learning with the least squares regularization scheme in a reproducing kernel Hilbert space is a good approximation to the algorithm processing the whole data in one single machine. Expand
Structured AutoEncoders for Subspace Clustering
This work proposes a novel subspace clustering approach by introducing a new deep model—Structured AutoEncoder (StructAE), which learns a set of explicit transformations to progressively map input data points into nonlinear latent spaces while preserving the local and global subspace structure. Expand
Concentration estimates for learning with ℓ1-regularizer and data dependent hypothesis spaces
We consider the regression problem by learning with a regularization scheme in a data dependent hypothesis space and l1-regularizer. The data dependence nature of the kernel-based hypothesis spaceExpand
Distributed Semi-supervised Learning with Kernel Ridge Regression
This paper provides error analysis for distributed semi-supervised learning with kernel ridge regression (DSKRR) based on a divide-and-conquer strategy and shows that unlabeled data play important roles in reducing the distributed error and enlarging the number of data subsets in DSKRR. Expand
Fast and Efficient Strategies for Model Selection of Gaussian Support Vector Machine
Two strategies for selecting the kernel parameter (sigma) and the penalty coefficient of Gaussian support vector machines (SVMs) are suggested and evaluated, showing that in terms of efficiency and generalization capabilities, the new strategies outperform the current methods, and the performance is uniform and stable. Expand