• Corpus ID: 42431955

Automated Production of High-Volume, Near-Real-Time Political Event Data

  title={Automated Production of High-Volume, Near-Real-Time Political Event Data},
  author={Philip A. Schrodt},
This paper summarizes the current state-of-the-art for generating high-volume, nearreal-time event data using automated coding methods, based on recent efforts for the DARPA Integrated Crisis Early Warning System (ICEWS) and NSF-funded research. The ICEWS work expanded by more than two orders of magnitude previous automated coding efforts, coding of about 26-million sentences generated from 8-million stories condensed from around 30 gigabytes of text. The actual coding took six minutes. The… 
Analyzing International Event Data: A Handbook of Computer-Based Techniques
This was originally part of a book-length manuscript which, like everything in the Keds project, was never published due to the lack of a commercially-viable audience, and is now reformatted in a stand-alone format which includes the relevant bibliographic citations.
Open source intelligence (OSINT) for conflict monitoring in contemporary South Africa: Challenges and opportunities in a big data context
ABSTRACT With the advent of the information age, Open Source Intelligence (OSINT) has gained special prominence in the Intelligence Community (IC). However, the era of big data has brought numerous
Enhancing the Detection of Criminal Organizations in Mexico using ML and NLP
This paper relies on Machine Learning (ML) and supervised Natural Language Processing (NLP) to generate a geo-referenced database on the violent presence of Mexican Criminal Organizations (MCOs)
Stuttgart’s Black Thursday on Twitter : Mapping Political Protests with Social Media Data
Event detection based on textual data is an approach often used in the social sciences. The method has been used predominantly in the fields of international politics (Schrodt 2010) and public
Detecting unfolding crises with Visual Analytics and Conceptual Maps Emerging phenomena and big data
This paper combines Visual Analytics with Concept Maps to support analysts in monitoring conflicts and exemplifies its use to detect shifts in political stability.
'Beating the news' with EMBERS: forecasting civil unrest using open source indicators
We describe the design, implementation, and evaluation of EMBERS, an automated, 24x7 continuous system for forecasting civil unrest across 10 countries of Latin America using open source indicators
Forecasting Political Conict in Asia and the Middle East using Latent Dirichlet Allocation Models
Latent Dirichlet allocation models are a relatively new computational classification algorithm. In its standard application to document classification, the model assumes each document to be composed
Forecasting Political Conflict in Asia using Latent Dirichlet Allocation Models ∗
Latent Dirichlet allocation models are a relatively new computational classification algorithm. In its standard application to document classification, the model assumes each document to be composed
Crowdsourcing the Measurement of Interstate Conflict
A method for analyzing news documents that uses crowdsourcing, supplemented with computational approaches is introduced, shown to be a considerable improvement over automated coding, and to cost less and be much faster than expert coding.
The goal of much of the large-N quantitative literature focusing on conflict onset has been to find generalizable empirical patterns across time and space. To do so, most studies have utilized a


An Automated Information Extraction Tool for International Conflict Data with Performance as Good as Human Coders: A Rare Events Evaluation Design
This article addresses advances in event categorization schemes and software programs that automatically produce data by “reading” news stories without human coders with a method that makes it feasible, for the first time, to evaluate these programs when they are applied in areas with the particular characteristics of international conflict and cooperation data.
Validity Assessment of a Machine-Coded Event Data Set for the Middle East, 1982-92
This paper is a study of the validity of a machine-coded event data series for six Middle Eastern actors and the United States. The series is based on Reuters newswire story leads coded into the WEIS
Integrated Data for Events Analysis (IDEA): An Event Typology for Automated Events Data Development
This article outlines the basic parameters and current status of the Integrated Data for Event Analysis (IDEA) project. IDEA provides a comprehensive events framework for the analysis of
Twenty Years of the Kansas Event Data System Project
Welcome again to The Political Methodologist. Our issue leads off with an article by Phil Schrodt describing the 20year evolution of the Kansas Event Data System Project (KEDS). We then move to two
Crisis Early Warning and Decision Support: Contemporary Approaches and Thoughts on Future Research
Military planners and other decision makers require advanced early warning of impending crises so they can devise effective mitigation plans, mobilize resources, and coordinate responses with their
International data as a national resource
Nation‐states are increasingly recognizing that banks of knowledge, including social science data, are a national resource vital for understanding the course of events. Criteria for considering a
The Conflict and Peace Data Bank (COPDAB) Project
As students of politics and political science, we should and we do care about the events which lead to war, instability, and international tension as well as about events which lead to equitable
A Comparative Study of the WEIS and COPDAB Data Sets
Data from the Conflict and Peace Data Bank (COPDAB) and the World Event Interaction Survey (WEIS) reflecting United States and USSR cooperation and conflict are examined comparatively. Using the
An Event Data Analysis of Third-Party Mediation in the Middle East and Balkans
Event data on the Israel-Lebanon and Israel-Palestinian conflicts in the Levant (1979-1999) and the Serbia-Croatia and Serbia-Bosnia conflicts in the Balkans (1991-1999) are used to test two sets of
A Conflict-Cooperation Scale for WEIS Events Data
The problem of aggregating WEIS events data, coded as discrete events, into a continuous time series representing conflict or cooperation between two nations is discussed. Past literature on the