Corpus ID: 18160591

Convergence Failures in Logistic Regression

  title={Convergence Failures in Logistic Regression},
  author={Paul D. Allison},
A frequent problem in estimating logistic regression models is a failure of the likelihood maximization algorithm to converge. In most cases, this failure is a consequence of data patterns known as complete or quasi-complete separation. For these patterns, the maximum likelihood estimates simply do not exist. In this paper, I examine how and why complete or quasi-complete separation occur, and the effects they produce in output from SAS ® procedures. I then describe and evaluate several… Expand

Figures and Tables from this paper

Correcting the Quasi-complete Separation Issue in Logistic Regression Models
Quasi-complete separation is a commonly detected issue in logit/probit models. Quasi-complete separation occurs when the dependent variable separates an independent variable or a combination ofExpand
Invariant properties of logistic regression model in credit scoring under monotonic transformations
ABSTRACT Monotonic transformations of explanatory continuous variables are often used to improve the fit of the logistic regression model to the data. However, no analytic studies have been done toExpand
This paper proposes an application of concepts about the maximum likelihood estimation of the binomial logistic regression model to the separation phenomena. It generates bias in the estimation andExpand
Standard Binary Logistic Regression Model
The logistic regression model is a type of predictive model that can be used when the response variable is binary—for example: live/die; disease/no disease; purchase/no purchase; win/lose. In short,Expand
Verifying the existence of maximum likelihood estimates for generalized linear models.
It is shown that some, but not all, GLMs can still deliver consistent estimates of at least some of the linear parameters when these conditions fail to hold, and how to verify these conditions in the presence of high-dimensional fixed effects is demonstrated. Expand
Analysis of bias-corrected and exact estimators for binomial generalized linear model parameters
Typically, small samples have always been a problem for binomial generalized linear models. Though generalized linear models are widely popular in public health, social sciences etc. In small sampleExpand
Examining the reliability of logistic regression estimation software
The results show that solely trusting the default settings of statistical software packages may lead to non-optimal, biased or erroneous results, which may impact the quality of empirical results obtained by applied economists. Expand
Finding Respondents in the Forest: A Comparison of Logistic Regression and Random Forest Models for Response Propensity Weighting and Stratification
Survey response rates for modern surveys using many different modes are trending downward leaving the potential for nonresponse biases in estimates derived from using only the respondents. TheExpand
Using Firth's method for model estimation and market segmentation based on choice data
Abstract Using maximum likelihood (ML) estimation for discrete choice modeling of small datasets causes two problems. The first problem is that the data may exhibit separation, in which case the MLExpand
Analysis of sparse data in logistic regression in medical research: A newer approach
PLR is almost equal to the ordinary logistic regression when the sample size is large and is superior in small cell values and simulation experiment shows that the estimated coverage probability of this method is near the nominal level of 95% even for small sample sizes and for a large number of covariates. Expand


A solution to the problem of separation in logistic regression.
A procedure by Firth originally developed to reduce the bias of maximum likelihood estimates is shown to provide an ideal solution to separation and produces finite parameter estimates by means of penalized maximum likelihood estimation. Expand
On the existence of maximum likelihood estimates in logistic regression models
SUMMARY The problems of existence, uniqueness and location of maximum likelihood estimates in log linear models have received special attention in the literature (Haberman, 1974, Chapter 2;Expand
Bias reduction of maximum likelihood estimates
SUMMARY It is shown how, in regular parametric problems, the first-order term is removed from the asymptotic bias of maximum likelihood estimates by a suitable modification of the score function. InExpand
The application of Firth's procedure to Cox and logistic regression
The phenomenon of separation or monotone likelihood is observed in the fitting process of a logistic or a Cox model if the likelihood converges while at least one parameter estimate diverges to ±Expand
Hierarchical Bayesian semiparametric procedures for logistic regression
SUMMARY A simple procedure is proposed for exact computation to smooth Bayesian estimates for logistic regression functions, when these are not constrained to lie on a fitted regression surface.Expand
Median Unbiased Estimation for Binary Data
Abstract This article compares the accuracy of the median unbiased estimator with that of the maximum likelihood estimator for a logistic regression model with two binary covariates. The formerExpand
Bayesian analysis of binary and polychotomous response data
Abstract A vast literature in statistics, biometrics, and econometrics is concerned with the analysis of binary and polychotomous response data. The classical approach fits a categorical responseExpand
A note on A. Albert and J. A. Anderson's conditions for the existence of maximum likelihood estimates in logistic regression models
SUMMARY This note expands the paper by Albert & Anderson (1984) on the existence and uniqueness of maximum likelihood estimates in logistic regression models. Their three possible mutually exclusiveExpand
Discharge Rates of Medicare Stroke Patients to Skilled Nursing Facilities: Bayesian Logistic Regression with Unobserved Heterogeneity
Abstract We determine factors, both hospital-specific and market area-specific, associated with hospitals' propensities for discharging Medicare stroke patients to skilled nursing facilities (SNF's)Expand