• Corpus ID: 239769298

Why Machine Learning Cannot Ignore Maximum Likelihood Estimation

  title={Why Machine Learning Cannot Ignore Maximum Likelihood Estimation},
  author={Mark J. van der Laan and Sherri Rose},
The growth of machine learning as a field has been accelerating with increasing interest and publications across fields, including statistics, but predominantly in computer science. How can we parse this vast literature for developments that exemplify the necessary rigor? How many of these manuscripts incorporate foundational theory to allow for statistical inference? Which advances have the greatest potential for impact in practice? One could posit many answers to these queries. Here, we… 

The Selectively Adaptive Lasso

This paper builds upon the theory of HAL to construct the Selectively Adaptive Lasso (SAL), a new algorithm which retains HAL’s dimension-free, nonparametric convergence rate but which also scales computationally to massive datasets.



The Highly Adaptive Lasso Estimator

  • D. BenkeserM. J. Laan
  • Computer Science, Mathematics
    2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA)
  • 2016
A novel nonparametric regression estimator is proposed that, in contrast to many existing methods, does not rely on local smoothness assumptions nor is it constructed using local smoothing techniques, and respects global smoothness constraints by virtue of falling in a class of right-hand continuous functions with left-hand limits that have variation norm bounded by a constant.

Super Learner

A fast algorithm for constructing a super learner in prediction which uses V-fold cross-validation to select weights to combine an initial set of candidate learners.

Targeted Maximum Likelihood Learning

Suppose one observes a sample of independent and identically distributed observations from a particular data generating distribution. Suppose that one is concerned with estimation of a particular

hal9001: Scalable highly adaptive lasso regression in R

The hal9001 R package provides a computationally efficient implementation of the highly adaptive lasso (HAL), a flexible nonparametric regression and machine learning algorithm endowed with several

Higher Order Targeted Maximum Likelihood Estimation

Asymptotic efficiency of targeted maximum likelihood estimators (TMLE) of target features of the data distribution relies on a a second order remainder being asymptotically negligible. In previous

Causal Models and Learning from Data: Integrating Causal Modeling and Statistical Estimation

It is argued that a formal causal framework can help in designing a statistical analysis that comes as close as possible to answering the motivating causal question, while making clear what assumptions are required to endow the resulting estimates with a causal interpretation.

Regression Shrinkage and Selection via the Lasso

A new method for estimation in linear models called the lasso, which minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant, is proposed.

Oracle inequalities for multi-fold cross validation

The results are extended to penalized cross validation in order to control unbounded loss functions and applications include regression with squared and absolute deviation loss and classification under Tsybakov’s condition.

Nonparametric bootstrap inference for the targeted highly adaptive least absolute shrinkage and selection operator (LASSO) estimator

This article establishes that the nonparametric bootstrap for the HAL-TMLE, fixing the value of the sectional variation norm at a value larger or equal than the cross-validation selector, provides a consistent method for estimating the normal limit distribution of the HAL -TMLE.

On methods of sieves and penalization

We develop a general theory which provides a unified treatment for the asymptotic normality and efficiency of the maximum likelihood estimates (MLE's) in parametric, semiparametric and nonparametric