The Subjective Precision of Computers: A Methodological Comparison with Human Coding in Content Analysis

@article{Conway2006TheSP,
  title={The Subjective Precision of Computers: A Methodological Comparison with Human Coding in Content Analysis},
  author={Mike Conway},
  journal={Journalism \& Mass Communication Quarterly},
  year={2006},
  volume={83},
  pages={186 - 200}
}
  • Mike Conway
  • Published 1 March 2006
  • Computer Science
  • Journalism & Mass Communication Quarterly
With the increased sophistication and speed of computers, fewer text content analysis projects utilize traditional human coding. This methodological case study was designed to compare results of human versus computer-assisted coding using the same data set. The two processes yielded significantly different results in a content analysis of newspaper coverage of a political campaign. In computer-assisted coding, several subjective steps must be taken to adapt the content to the program. Those… 

Tables from this paper

Bits versus brains in content analysis. Comparing the advantages and disadvantages of manual and automated methods for content analysis

TLDR
Two different types of content analysis methods are compared by analysing the experience obtained in using these methods for the studies carried out and reviewing the methodological issues encountered in these studies and compared them with existing literature.

The Content Analysis of Media Frames: Toward Improving Reliability and Validity

TLDR
The main purpose of this study was to shed light on methodological problems in the content analysis of media frames by proposing that previously defined frame elements systematically group together in a specific way.

Content Analysis and the Algorithmic Coder

TLDR
This article focuses on how the traditional (manual) approach to conducting a content analysis is being reconfigured, assesses what is gained and lost in turning to computational solutions, and builds on a “hybrid” approach to content analysis.

Making sense of text: artificial intelligence-enabled content analysis

TLDR
This paper is among the first to introduce, apply and compare how artificial intelligence (AI) can be used for content analysis in marketing research relative to manual and computer-aided (non-AI) approaches to content analysis.

When Scale Meets Depth: Integrating Natural Language Processing and Textual Analysis for Studying Digital Corpora

TLDR
A hybrid approach that combines the scale of computational methods with the depth of qualitative analysis is proposed, which uses simple natural language processing algorithms to extract purposive samples from large textual corpora, which can then be analyzed using interpretive techniques.

Issues and Best Practices in Content Analysis

This article discusses three issues concerning content analysis method and ends with a list of best practices in conducting and reporting content analysis projects. Issues addressed include the use

Hearing the Signal in the Noise: A Software-Based Content Analysis of Patterns in Responses by Experts and Students to a New Venture Investment Proposal

The authors apply a software-based content analysis method to uncover differences in responses by expert entrepreneurs and undergraduate entrepreneur majors to a new venture investment proposal. Data

Coding Text Answers to Open-ended Questions: Human Coders and Statistical Learning Algorithms Make Similar Mistakes

TLDR
It is found that the correlation between the estimated coding error of a statistical model and that of a human is comparable to that of two humans, and the choice of statistical model does not matter, and having a second automated coder would be redundant.

Broadening Exposure, Questioning Opinions, and Reading Patterns with Reflext: a Computational Support for Frame Reflection

TLDR
Reflext, an interactive visualization tool that leverages computational linguistic analysis to support reflection on the framing of political issues, was deployed in a field study, during which the tool was used by regular readers of political news coverage during the 2012 U.S. election campaign.

Content Analysis in an Era of Big Data: A Hybrid Approach to Computational and Manual Methods

TLDR
It is argued that an approach blending computational and manual methods throughout the content analysis process may yield more fruitful results, and a case study of news sourcing on Twitter is drawn to illustrate this hybrid approach in action.
...

References

SHOWING 1-10 OF 30 REFERENCES

Pre-Assessment of Scale Reliability A Computer Content Analysis Approach

This paper examines the relationship between certain features of the language used in scale questions and internal consistency reliability of the scale. It is hypothesized that greater inconsistency

A Content Analysis of Content Analyses: Twenty-Five Years of Journalism Quarterly

Examination of the increasing number of articles employing quantitative content analysis in 1971–95 Journalism & Mass Communication Quarterly showed primary focus on news/editorial content in U.S.

The Content Analysis Guidebook

TLDR
The Content Analysis Guidebook provides an accessible core text for upper-level undergraduates and graduate students across the social sciences that unravels the complicated aspects of content analysis.

Why Going Online for Content Analysis Can Reduce Research Reliability

This study cautions researchers to be careful when comparing content-analysis research results of online searches to hand searches. How newspapers upload information into the data bases can influence

A Basic Space Unit for Newspaper Content Analysis

This department is devoted to shorter reports on research in the communications field. Readers are invited to submit summaries of investigative studies interesting for content, method or implications

Content Analysis

TLDR
The main idea behind this project is to design a spam detection system that will be enabled to analyze the content of emails, in particular the artificially generated image sent as attachment in an email.

The Framing of Feminists and Feminism in News and Public Affairs Programs in U.S. Electronic Media

This research analyzes the representation of feminists and feminism in media (in part by comparing such representation to that of women) via content analysis of 35,000 hours of ABC, CNN, PBS, and NPR

Framing the Candidates in Presidential Primaries: Issues and Images in Press Releases and News Coverage

Media coverage of presidential primaries is crucial to voters, and candidates often complain that news coverage fails to present their positions. This study used computerized content analysis to

Social or Economic Concerns: How News and Women's Magazines Framed Breast Cancer in the 1990s

Media framing of complex diseases is important because the way these are presented may affect individuals' choices regarding health. The authors analyzed 127 articles on breast cancer from three news

Setting the agenda of attributes in the 1996 Spanish general election

We advance the central proposition of agenda-setting theory - that elements prominent in the mass media's picture of the world influence the salience of those elements in the audience's picture -