Jonathan K. Kummerfeld | Semantic Scholar

Publications
Influence

Error-Driven Analysis of Challenges in Coreference Resolution

Coreference resolution metrics quantify errors but do not analyze them. Here, we consider an automated method of categorizing errors in the output of a coreference system into intuitive underlying… (More)

Parser Showdown at the Wall Street Corral: An Empirical Investigation of Error Types in Parser Output

Jonathan K. Kummerfeld, David Hall, James R. Curran, Dan Klein
EMNLP-CoNLL
2012

Constituency parser performance is primarily interpreted through a single metric, F-score on WSJ section 23, that conveys no linguistic information regarding the remaining errors. We classify errors… (More)

Tools for Automated Analysis of Cybercriminal Markets

Rebecca S. Portnoff, Sadia Afroz, +5 authors Vern Paxson
WWW
2017

Underground forums are widely used by criminals to buy and sell a host of stolen items, datasets, resources, and criminal services. These forums contain important resources for understanding… (More)

Improving Text-to-SQL Evaluation Methodology

Catherine Finegan-Dollak, Jonathan K. Kummerfeld, +4 authors Dragomir R. Radev
ACL
2018

To be informative, an evaluation must measure how well systems generalize to realistic unseen data. We identify limitations of and propose improvements to current evaluations of text-to-SQL systems.… (More)

Factors Influencing the Surprising Instability of Word Embeddings

Laura Burdick, Jonathan K. Kummerfeld, Rada Mihalcea
NAACL-HLT
2018

Despite the recent popularity of word embedding methods, there is only a small body of work exploring the limitations of these representations. In this paper, we consider one aspect of embedding… (More)

Spatiotemporal hierarchy of relaxation events, dynamical heterogeneities, and structural reorganization in a supercooled liquid.

Raphaël Candelier, Asaph Widmer-Cooper, +4 authors David R. Reichman
Physical review letters
2010

We identify the pattern of microscopic dynamical relaxation for a two-dimensional glass-forming liquid. On short time scales, bursts of irreversible particle motion, called cage jumps, aggregate into… (More)

Understanding Task Design Trade-offs in Crowdsourced Paraphrase Collection

Youxuan Jiang, Jonathan K. Kummerfeld, Walter S. Lasecki
ACL
2017

Linguistically diverse datasets are critical for training and evaluating robust machine learning systems, but data collection is a costly process that often requires experts. Crowdsourcing the… (More)

Dialog System Technology Challenge 7

Koichiro Yoshino, Chiori Hori, +14 authors Dhruv Batra
ArXiv
2019

This paper introduces the Seventh Dialog System Technology Challenges (DSTC), which use shared datasets to explore the problem of building dialog systems. Recently, end-to-end dialog modeling… (More)

Classification of Verb Particle Constructions with the Google Web1T Corpus

Manually maintaining comprehensive databases of multi-word expressions, for example Verb-Particle Constructions (VPCs), is infeasible. We describe a new type level classifier for potential VPCs,… (More)

An Empirical Examination of Challenges in Chinese Parsing

Jonathan K. Kummerfeld, Daniel Tse, James R. Curran, Dan Klein
ACL
2013

Aspects of Chinese syntax result in a distinctive mix of parsing challenges. However, the contribution of individual sources of error to overall difficulty is not well understood. We conduct a… (More)