Metastasis is the most life-threatening complication in all cancers. The chemokine receptor 4 (CXCR4) is expressed at high levels in many breast-cancer tumors and may modulate metastasis. We compared the time-to-metastasis and the sites of metastasis between breast-cancer tumors expressing CXCR4 at high or low levels. We enrolled 191 early breast cancer patients in our study. The expression of CXCR4 was evaluated using immunohistochemical staining, and the patients were divided into low-level (CXCR4-) and high-level (CXCR4+) CXCR4 expression groups. Associations between the patients' level of CXCR4 expression and their basic clinical characteristics, time-to-metastasis, and metastatic sites were examined using a Cox proportional-hazards regression model. A total of 107 CXCR4+ patients (56 %) were identified. No statistical differences were evident in basic characteristics between the CXCR4+ and CXCR4- groups. The CXCR4+ group had a higher incidence of distant metastasis during the first year (10.3 % versus 1.1 %, P = 0.009) and shorter event-free survival (17.43 months versus 27.5 months, P = 0.026) than those of the CXCR4- group. The CXCR4+ group also had a higher incidence of bone metastasis (P = 0.008) than the CXCR4- group. No significant difference in metastasis sites in other organs was observed between the two groups. A high level of CXCR4 expression in breast cancer is associated with early distant and bone metastases. The CXCR4+ phenotype may be a useful predictor for the prevention of early treatment failure and bone metastasis in breast cancer patients. This retrospective study shows that a high expression of CXCR4 in breast cancer is associated with earlier distant metastasis and bone metastasis in breast cancer.