The idea that we live near the center of a large, nonlinear void has attracted attention recently as an alternative to dark energy or modified gravity. We show that an appropriate void profile can fit both the latest cosmic microwave background and supernova data. However, this requires either a fine-tuned primordial spectrum or a Hubble rate so low as to rule these models out. We also show that measurements of the radial baryon acoustic scale can provide very strong constraints. Our results present a serious challenge to void models of acceleration.