Jonathan K. Kummerfeld

Publications
Influence

Improving Text-to-SQL Evaluation Methodology

Catherine Finegan-Dollak, Jonathan K. Kummerfeld, +4 authors Dragomir R. Radev
Computer Science
ACL
1 June 2018

We identify limitations of and propose improvements to current evaluations of text-to-SQL systems for mapping natural language to structured database queries. Expand

Parser Showdown at the Wall Street Corral: An Empirical Investigation of Error Types in Parser Output

Jonathan K. Kummerfeld, David Hall, J. Curran, D. Klein
Computer Science
EMNLP-CoNLL
12 July 2012

We propose a novel tree-transformation methodology for evaluating parsers that categorises errors into linguistically meaningful types. Expand

An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction

Stefan Larson, Anish Mahendran, +8 authors J. Mars
Computer Science
EMNLP/IJCNLP
4 September 2019

We introduce a new dataset that includes queries that are out-of-scope---i.e., queries that do not fall into any of the system's supported intents. Expand

Factors Influencing the Surprising Instability of Word Embeddings

Laura Burdick, Jonathan K. Kummerfeld, R. Mihalcea
Computer Science
NAACL-HLT
25 April 2018

We provide empirical evidence for how various factors contribute to the stability of word embeddings, and we analyze the effects of stability on downstream tasks. Expand

Error-Driven Analysis of Challenges in Coreference Resolution

Jonathan K. Kummerfeld, D. Klein
Computer Science
EMNLP
1 October 2013

We present a new tool that automatically classifies errors in the standard output of any coreference resolution system into intuitive underlying error types. Expand

Tools for Automated Analysis of Cybercriminal Markets

Rebecca S. Portnoff, S. Afroz, +5 authors V. Paxson
Computer Science
WWW
3 April 2017

We propose an automated, top-down approach for analyzing underground forums, first identifying posts related to transactions and then extracting products and prices. Expand

A Large-Scale Corpus for Conversation Disentanglement

Jonathan K. Kummerfeld, S. R. Gouravajhala, +6 authors Walter S. Lasecki
Computer Science
ACL
25 October 2018

We introduce a new dataset of 77,563 messages manually annotated with reply-structure graphs that both disentangle conversations and define internal conversation structure. Expand

Spatiotemporal hierarchy of relaxation events, dynamical heterogeneities, and structural reorganization in a supercooled liquid.

R. Candelier, A. Widmer-Cooper, +4 authors D. Reichman
Physics, Medicine
Physical review letters
24 September 2010

We identify the pattern of microscopic dynamical relaxation for a two-dimensional glass-forming liquid. On short time scales, bursts of irreversible particle motion, called cage jumps, aggregate into… Expand

Understanding Task Design Trade-offs in Crowdsourced Paraphrase Collection

Youxuan Jiang, Jonathan K. Kummerfeld, Walter S. Lasecki
Computer Science
ACL
19 April 2017

We present the first systematic study of the key factors in crowdsourcing paraphrase collection, including the first exploration of worker incentives. Expand

DSTC7 Task 1: Noetic End-to-End Response Selection

Chulaka Gunasekara, Jonathan K. Kummerfeld, L. Polymenakos, Walter S. Lasecki
Computer Science
2019

This paper provides an overview of (1) the task structure, (2) the datasets, (3) the evaluation metrics, and (4) system results. Expand

Recommended Authors