#### Filter Results:

#### Publication Year

1994

2016

#### Publication Type

#### Co-author

#### Publication Venue

#### Key Phrases

Learn More

—Stochastic approximation (SA) has long been applied for problems of minimizing loss functions or root finding with noisy input information. As with all stochastic search algorithms , there are adjustable algorithm coefficients that must be specified, and that can have a profound effect on algorithm performance. It is known that choosing these coefficients… (More)

- James C. Spall
- 1998

ultivariate stochastic optimization plays a major role in the analysis and control of many engineering systems. In almost all real-world optimization problems, it is necessary to use a mathematical algorithm that iteratively seeks out the solution because an analytical (closed-form) solution is rarely available. In this spirit, the " simultaneous… (More)

- James C. SPALL
- 2005

The Fisher information matrix summarizes the amount of information in the data relative to the quantities of interest. There are many applications of the information matrix in modeling, systems analysis, and estimation, including confidence region calculation, input design, prediction bounds, and " noninformative " priors for Bayesian analysis. This article… (More)

—Consider the problem of developing a controller for general (nonlinear and stochastic) systems where the equations governing the system are unknown. Using discrete-time measurements , this paper presents an approach for estimating a controller without building or assuming a model for the system (including such general models as differential/difference… (More)

- James C. Spall
- 2004

Stochastic optimization algorithms have been growing rapidly in popularity over the last decade or two, with a number of methods now becoming " industry standard " approaches for solving challenging optimization problems. This paper provides a synopsis of some of the critical issues associated with stochastic optimization and a gives a summary of several… (More)

Covariance calculations and confidence intervals for maximum likelihood estimates (MLEs) are commonly used in identification and statistical inference. To accurately construct such confidence intervals, one typically needs to know the covariance of the MLE. Standard statistical theory shows that the normalized MLE is asymptotically normally distributed with… (More)

- James C. Spall
- 2004

This paper extends results presented in preliminary form at the 2003 ACC to provide more theoretical justification for the Monte Carlo method (Sect. 4), to introduce the use of antithetic random numbers (Sect. 5), and to carry out a new numerical study (Sect. 6). A more complete version of this paper is available upon request. Abstract: The Fisher… (More)

It is known that a stochastic approximation (SA) analogue of the deterministic Newton-Raphson algorithm provides an asymptotically optimal or near-optimal form of stochastic search. In a recent paper, Spall (2006) introduces two enhancements that generally improve the quality of the estimates for underlying Jacobian (Hessian) matrices, thereby improving the… (More)

The simultaneous perturbation stochastic approximation SPSA algorithm has recently attracted considerable attention for optimization problems where it is diicult or impossible to obtain a direct gradient of the objective say, loss function. The approach is based on a highly eecient simultaneous perturbation approximation to the gradient based on loss… (More)