Treatment Choice With Trial Data: Statistical Decision Theory Should Supplant Hypothesis Testing

  title={Treatment Choice With Trial Data: Statistical Decision Theory Should Supplant Hypothesis Testing},
  author={Charles F. Manski},
  journal={The American Statistician},
  pages={296 - 304}
  • C. Manski
  • Published 20 March 2019
  • Psychology
  • The American Statistician
Abstract A central objective of empirical research on treatment response is to inform treatment choice. Unfortunately, researchers commonly use concepts of statistical inference whose foundations are distant from the problem of treatment choice. It has been particularly common to use hypothesis tests to compare treatments. Wald’s development of statistical decision theory provides a coherent frequentist framework for use of sample data on treatment response to make treatment decisions. A body… Expand
Probabilistic Prediction for Binary Treatment Choice: with Focus on Personalized Medicine
This paper carries further my research applying statistical decision theory to treatment choice with sample data, using maximum regret to evaluate the performance of treatment rules. The specific newExpand
Trial Size for Near-Optimal Choice Between Surveillance and Aggressive Treatment: Reconsidering MSLT-II
A refined version of that analysis applicable to trials comparing aggressive treatment of patients with surveillance, based on a trial comparing nodal observation and lymph node dissection when treating patients with cutaneous melanoma, concludes that assigning 244 patients to each option would yield findings that enable suitably near-optimal treatment choice. Expand
Bayesian Methods in Regulatory Science
  • G. Rosner
  • Medicine, Computer Science
  • Statistics in biopharmaceutical research
  • 2020
A brief review of the clinical development of new treatments and argues for the use of Bayesian methods and decision theory in clinical research are provided. Expand
Remarks on statistical inference for statistical decisions
The paper first addresses binary choice problems, where all SDFs may be viewed as hypothesis tests, and next considers as-if optimization, where one uses a point estimate of the true state as if the estimate is accurate. Expand
Statistical inference for statistical decisions
This paper motivates inference-based SDFs as practical procedures for decision making that may accomplish some of what Wald envisioned and presents specific findings concerning treatment choice and point prediction with sample data. Expand
Functional Sequential Treatment Allocation
This paper introduces and studies sequential learning algorithms when the distributional characteristic of interest is a general functional of the outcome distribution, and obtains minimax expected regret optimality results within the subclass of explore-then-commit policies, and for the unrestricted class of all policies. Expand
Econometrics for Decision Making: Building Foundations Sketched by Haavelmo and Wald
This paper proposes statistical decision theory as a framework for evaluation of the performance of models in decision making, and considers the common practice of as-if optimization: specification of a model, point estimation of its parameters, and use of the point estimate to make a decision that would be optimal if the estimate were accurate. Expand
Laboratory experiments can pre-design to address power and selection issues
Using data from two laboratory experiments where subject behavior deviated sharply from theory, it is shown that the ex-post subject allocation to maximize statistical power is closer to these ex-ante calculations relative to traditional designs that balances the number of subjects across treatments. Expand
Demystify Lindley's Paradox by Interpreting P-value as Posterior Probability
In the hypothesis testing framework, p-value is often computed to determine rejection of the null hypothesis or not. On the other hand, Bayesian approaches typically compute the posterior probabilityExpand
The A/B Testing Problem with Gaussian Priors
A risk-neutral firm can perform a randomized experiment (A/B test) to learn about the effects of implementing an idea of unknown quality. The firm’s goal is to decide the experiment’s sample size andExpand


Sufficient trial size to inform clinical practice
  • C. Manski, A. Tetenov
  • Computer Science, Medicine
  • Proceedings of the National Academy of Sciences
  • 2016
This paper develops an alternative principle for trial design that aims to directly benefit medical decision making by choosing a sample size that enables implementation of near-optimal treatment rules. Expand
Social Choice with Partial Knowledge of Treatment Response
textabstractEconomists have long sought to learn the effect of a "treatment" on some outcome of interest, just as doctors do with their patients. A central practical objective of research onExpand
Statistical treatment rules for heterogeneous populations
An important objective of empirical research on treatment response is to provide decision makers with information useful in choosing treatments. This paper studies minimax-regret treatment choiceExpand
Clinical trial design enabling {\epsilon}-optimal treatment rules
Medical research has evolved conventions for choosing sample size in randomized clinical trials that rest on the theory of hypothesis testing. Bayesians have argued that trials should be designed toExpand
Statistical Principles for Clinical Trials
  • R. Kay
  • Medicine
  • The Journal of international medical research
  • 1998
If a trial is to be well designed, and the conclusions drawn from it valid, a thorough understanding of the benefits and pitfalls of basic statistical principles is required. When setting up a trial,Expand
Statistical treatment choice based on asymmetric minimax regret criteria
This paper studies the problem of treatment choice between a status quo treatment with a known outcome distribution and an innovation whose outcomes are observed only in a finite sample. I evaluateExpand
Who should be Treated? Empirical Welfare Maximization Methods for Treatment Choice
It is shown that when the propensity score is known, the average social welfare attained by EWM rules converges at least at n^(-1/2) rate to the maximum obtainable welfare uniformly over a minimally constrained class of data distributions, and this uniform convergence rate is minimax optimal. Expand
Asymptotics for Statistical Treatment Rules
This paper develops asymptotic optimality theory for statistical treatment rules in smooth parametric and semiparametric models. Manski (2000, 2002, 2004) and Dehejia (2005) have argued that theExpand
Selecting One of Two Treatments When the Responses are Dichotomous
Abstract A decision theoretic approach to the design of a clinical trial is considered for the situation in which a total of N patients with a disease receive one of two treatments and the responsesExpand
Bayesian Approaches to Randomized Trials
It is argued that a Bayesian approach allows a formal basis for using external evidence and in addition provides a rational way for dealing with issues such as the ethics of randomization, trials to show treatment equivalence, the monitoring of accumulating data and the prediction of the consequences of continuing a study. Expand