Dementia is an important cause of disability in the elderly. There is evidence that cognitive impairment in dementia is on a continuum with cognitive impairment in the non-demented elderly. In order to investigate this possibility, we need detailed knowledge about the population distribution of cognitive function and change in cognitive function. The aim of this study is to describe the change in different domains of cognitive function over 4 years in a population-based sample of non-demented elderly people, and to investigate the effect of sociodemographic variables and baseline cognitive function on change in each of the cognitive domains. Respondents from two group general practice lists (n = 503) were interviewed using the Cambridge Cognitive Examination (CAMCOG) at the incidence wave of the Cambridge City Over-75 Cohort Study and after a mean time period of 3.9 years. One hundred and thirty five of 212 non-demented subjects seen at follow-up completed the CAMCOG at both interviews. The annual rate of change in total CAMCOG score was -1.6 points per year (p < 0.001). There was statistically significant decline in all of the CAMCOG subscales. Greater decline in the Memory subscale was associated with less education (p = 0.03). Greater decline in the Attention/Calculation subscale was associated with manual social class (p = 0.05). Greater decline in the Perception subscale was associated with older age (p = 0.03). Decline in specific cognitive domains may indicate a reversible phase of cognitive impairment and deserves further investigation.