This study aimed to identify the predictors of decline in health-related quality of life (HRQOL) in Chinese patients with type 2 diabetes mellitus (T2DM). A prospective longitudinal observational study was conducted on 1826 Chinese T2DM patients managed in public primary care setting. HRQOL was measured at baseline, 1 and 2 years by the Short Form-12 Health Survey version 2 (SF-12v2) and Chinese (HK) Short Form-6 Dimensions (SF-6D). Linear mixed effect models with forward stepwise method were performed to select the factors associated with SF-12v2 physical (PCS) and mental component summary (MCS) scores and SF-6D value. Over a 2-year observation period, the HRQOL (PCS: −0.626; MCS: −1.869; and SF-6D: −0.017 per year) scores decreased significantly with time. Female, unmarried, current smoker, no regular exercise, obesity, comorbid hypertension, chronic kidney disease (CKD) or cardiovascular disease (CVD) and insulin use were predictors of one or more poorer HRQOL scores after 12 or 24 months. Older age had a negative impact on PCS score and SF-6D value, but had a positive impact on MCS score. The HRQOL of Chinese T2DM patients under primary care declined significantly over time. Obesity, smoking and no regular exercise were found to be modifiable risk factors of the decline in HRQOL in T2DM, which strengthened the importance of these lifestyle changes in diabetes care. More attention should be given to T2DM patients who are female, older, unmarried, or on anti-hypertensive drugs or insulin, or have comorbid hypertension, CKD or CVD in minimizing the negative impact of illness on their life.