Low bone mineral density (BMD) is common among breast cancer survivors due to acute estrogen deprivation. Soy food is a rich source of phytoestrogens, namely isoflavones, known to have both estrogenic and anti-estrogenic effects. The objective of the study was to assess the association between soy consumption and BMD in breast cancer survivors, which has not previously been evaluated. Forearm BMD was evaluated using dual-energy X-ray absorptiometry at 60 months post-diagnosis for 1,587 participants of the Shanghai Breast Cancer Survival Study. Soy intakes collected at 6, 18, and 36 months post-diagnosis were averaged, and the association with BMD, osteopenia, and osteoporosis was evaluated using linear and logistic regression. The mean (standard deviation) intake of isoflavones was 48.1 (28.0) mg/day. Soy intake was inversely associated with BMD and positively associated with osteoporosis. Compared with the lowest quartile, the highest quartile of soy isoflavone intake, ≥ 62.64 mg/day, was associated with a reduction of BMD by 1.95 % [95 % confidence interval (CI) −3.54, −0.36 %] and an increased odds ratio of 1.69 for osteoporosis (95 % CI 1.09, 2.61). The inverse association was predominantly seen among women who recently entered menopause (≤5 years). In contrast to observations from general populations, high soy intake (≥62.64 mg of soy isoflavone/day) was associated with lower proximal forearm BMD among breast cancer survivors, particularly during the early years of menopause. Our finding needs to be replicated, particularly in studies with more comprehensive bone density evaluation.