# Learning Through Deterministic Assignment of Hidden Parameters

@article{Fang2020LearningTD, title={Learning Through Deterministic Assignment of Hidden Parameters}, author={Jian Fang and Shaobo Lin and Zongben Xu}, journal={IEEE Transactions on Cybernetics}, year={2020}, volume={50}, pages={2321-2334} }

Supervised learning frequently boils down to determining hidden and bright parameters in a parameterized hypothesis space based on finite input–output samples. The hidden parameters determine the nonlinear mechanism of an estimator, while the bright parameters characterize the linear mechanism. In a traditional learning paradigm, hidden and bright parameters are not distinguished and trained simultaneously in one learning process. Such a one-stage learning (OSL) brings a benefit of theoretical… Expand

#### One Citation

Random Sketching for Neural Networks With ReLU

- Computer Science, Medicine
- IEEE Transactions on Neural Networks and Learning Systems
- 2021

This article borrows the well-known random sketching strategy from kernel methods to transform the training of shallow rectified linear unit (ReLU) nets into a linear least-squares problem, and shows that random Sketching can significantly reduce the computational burden of numerous backpropagation algorithms while maintaining their learning performance. Expand

#### References

SHOWING 1-10 OF 75 REFERENCES

Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part II)

- Computer Science, Medicine
- IEEE Transactions on Neural Networks and Learning Systems
- 2015

It is proved that the generalization capability of ELM with Gaussian kernel is essentially worse than that of FNN withGaussian kernel, and it is found that the well-developed coefficient regularization technique can essentially improve thegeneralization capability. Expand

Is Extreme Learning Machine Feasible? A Theoretical Assessment (Part I)

- Mathematics, Computer Science
- IEEE Trans. Neural Networks Learn. Syst.
- 2015

A comprehensive feasibility analysis of ELM is conducted and it is revealed that there also exists some activation functions, which makes the corresponding ELM degrade the generalization capability. Expand

Approximation Methods for Supervised Learning

- Mathematics, Computer Science
- Found. Comput. Math.
- 2006

The main focus is to understand what is the rate of approximation, measured either in expectation or probability, that can be obtained under a given prior fρ ∈ Θ, and what are possible algorithms for obtaining optimal or semioptimal results. Expand

Extreme learning machine: Theory and applications

- Computer Science
- Neurocomputing
- 2006

A new learning algorithm called ELM is proposed for feedforward neural networks (SLFNs) which randomly chooses hidden nodes and analytically determines the output weights of SLFNs which tends to provide good generalization performance at extremely fast learning speed. Expand

Learning with sample dependent hypothesis spaces

- Computer Science, Mathematics
- Comput. Math. Appl.
- 2008

This paper proposes an idea of using a larger function class containing the union of all possible hypothesis spaces (varying with the sample) to measure the approximation ability of the algorithm and shows how this idea provides error analysis for two particular classes of learning algorithms in kernel methods: learning the kernel via regularization and coefficient based regularization. Expand

Distributed Learning with Regularized Least Squares

- Computer Science, Mathematics
- J. Mach. Learn. Res.
- 2017

It is shown with error bounds in expectation that the global output function of this distributed learning with the least squares regularization scheme in a reproducing kernel Hilbert space is a good approximation to the algorithm processing the whole data in one single machine. Expand

Structured AutoEncoders for Subspace Clustering

- Computer Science, Medicine
- IEEE Transactions on Image Processing
- 2018

This work proposes a novel subspace clustering approach by introducing a new deep model—Structured AutoEncoder (StructAE), which learns a set of explicit transformations to progressively map input data points into nonlinear latent spaces while preserving the local and global subspace structure. Expand

Concentration estimates for learning with ℓ1-regularizer and data dependent hypothesis spaces

- Mathematics
- 2011

We consider the regression problem by learning with a regularization scheme in a data dependent hypothesis space and l1-regularizer. The data dependence nature of the kernel-based hypothesis space… Expand

Distributed Semi-supervised Learning with Kernel Ridge Regression

- Computer Science
- J. Mach. Learn. Res.
- 2017

This paper provides error analysis for distributed semi-supervised learning with kernel ridge regression (DSKRR) based on a divide-and-conquer strategy and shows that unlabeled data play important roles in reducing the distributed error and enlarging the number of data subsets in DSKRR. Expand

Fast and Efficient Strategies for Model Selection of Gaussian Support Vector Machine

- Mathematics, Computer Science
- IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)
- 2009

Two strategies for selecting the kernel parameter (sigma) and the penalty coefficient of Gaussian support vector machines (SVMs) are suggested and evaluated, showing that in terms of efficiency and generalization capabilities, the new strategies outperform the current methods, and the performance is uniform and stable. Expand