Integrating Computer Prediction Methods in Social Science: A Comment on Hofman et al. (2021)

@article{Breznau2022IntegratingCP,
  title={Integrating Computer Prediction Methods in Social Science: A Comment on Hofman et al. (2021)},
  author={Nate Breznau},
  journal={Social Science Computer Review},
  year={2022},
  volume={40},
  pages={844 - 853}
}
  • Nate Breznau
  • Published 21 April 2022
  • Psychology
  • Social Science Computer Review
Machine learning and other computer-driven prediction models are one of the fastest growing trends in computational social science. These methods and approaches were developed in computer science and with different goals and epistemologies than those in social science. The most obvious difference being a focus on prediction versus explanation. Predictive modeling offers great potential for improving research and theory development, but its adoption poses some challenges and creates new problems… 

Figures from this paper

References

SHOWING 1-10 OF 55 REFERENCES
Integrating explanation and prediction in computational social science.
TLDR
This Perspective proposes a schema for thinking about research activities along two dimensions-the extent to which work is explanatory, focusing on identifying and estimating causal effects, and the degree of consideration given to testing predictions of outcomes-and how these two priorities can complement, rather than compete with, one another.
When 4 ≈ 10,000: The Power of Social Science Knowledge in Predictive Performance
  • S. McKay
  • Computer Science
    Socius: Sociological Research for a Dynamic World
  • 2019
TLDR
Key insights included new variables constructed according to theory, lagged values of the target variables, using predicted values of certain outcomes to inform others, and validated scales rather than individual variables.
Machine Learning for Sociology
TLDR
It is argued that, in addition to serving similar purposes in sociology, machine learning tools can speak to long-standing questions on the limitations of the linear modeling framework, the criteria for evaluating empirical findings, transparency around the context of discovery, and the epistemological core of the discipline.
Assessing Replicability of Machine Learning Results: An Introduction to Methods on Predictive Accuracy in Social Sciences
TLDR
It is found that when combining studies sampled from same population to form meta-analytic intervals, random- effects methods perform best for cross-validated measures while fixed-effects methods work best for test measures.
Measuring the predictability of life outcomes with a scientific mass collaboration
TLDR
Practical limits to the predictability of life outcomes in some settings are suggested and the value of mass collaborations in the social sciences is illustrated.
(Ir)Reproducible Machine Learning: A Case Study
TLDR
It is argued that there is a reproducibility crisis brewing in research fields that use ML methods and a few systemic interventions that could help resolve it, and a case study of the subfield of civil war prediction in Political Science is examined.
Estimating the reproducibility of psychological science
TLDR
A large-scale assessment suggests that experimental reproducibility in psychology leaves a lot to be desired, and correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.
Practical Methodological Reform Needs Good Theory
  • Will M. Gervais
  • Psychology
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2021
In the face of unreplicable results, statistical anomalies, and outright fraud, introspection and changes in the psychological sciences have taken root. Vibrant reform and metascience movements have
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
TLDR
Recommendations including weighing the environmental and financial costs first, investing resources into curating and carefully documenting datasets rather than ingesting everything on the web, and carrying out pre-development exercises evaluating how the planned approach fits into research and development goals and supports stakeholder values are provided.
I Saw You in the Crowd: Credibility, Reproducibility, and Meta-Utility
  • Nate Breznau
  • Computer Science
    PS: Political Science & Politics
  • 2021
TLDR
Crowdsourcing enables novel forms of research and knowledge production, can provide more reliable Bayesian priors for selecting models and is an untapped mode of theory production that greatly benefit social science.
...
...