Understanding the effects of aging on sensory processing must be the starting point for any examination of age-related changes in cognition and behavior. Previous studies of the neurophysiological effects of healthy aging on sensory processing have attributed changes in auditory evoked responses to reduced inhibition from frontocortical areas on primary auditory processing or to a global slowing of neural responses. We investigated whether the age-related changes in visual evoked potentials can be primarily accounted for by similar deficits in frontocortical inhibition or processing speed. Using evoked potentials, we examined the primary visual processing of younger (mean age, 20.0 years) and older (mean age, 76.8 years) adults in a visual oddball paradigm. We found that the pattern of early visual responses cannot be explained via reduced frontocortical inhibition or a reduction in speed of visual processing. Rather our results point toward the existence of a robust compensatory neural response to impoverished sensory input in aging.