Objective To analyze the efficacy of music therapy on the rehabilitation of post-stroke cognitive impairment (PSCI) and to provide a reference for rehabilitation intervention methods for PSCI. Methods Patients hospitalized in Beijing Bo’Ai Hospital, China Rehabilitation Research Center and diagnosed with PSCI between December 2020 and July 2022 were prospectively selected. According to the random number table method, patients were divided into a music therapy group and a control group. Both groups were given conventional neurology medication, nursing care, and conventional rehabilitation. The music therapy group received additional music therapy training, and both groups received treatment for one month. The Montreal Cognitive Assessment (MoCA), National Institute of Health Stroke Scale (NIHSS), Fugl-Meyer Assessment Scale (FMA), and modified Barthel Index (MBI) were used before and after treatment to assess patients’ cognitive function, degree of neurological deficits, motor function and activities of daily live. Results A total of 48 patients were included, with 24 patients in both groups. There was no statistically significant difference in gender, age, education level, stroke type, lesion location, comorbidities, history of myocardial infarction or peripheral vascular disease, and smoking status between the two groups of patients (P>0.05). Before and after treatment, most patients in the two groups did not score in terms of language and delayed recall scores, and the difference were not statistically significant (P>0.05). There was no statistically significant difference in MoCA scores, visual space and executive function, naming, attention, calculation, abstract thinking, and orientation scores between the two groups of patients before treatment (P>0.05). After treatment, the MoCA score, visual space and executive function, naming, attention, calculation, abstract thinking, and orientation scores of the music therapy group improved compared to before treatment (P<0.05), while the MoCA score, visual space and executive function, naming, attention, and orientation scores of the control group improved compared to before treatment (P<0.05). After treatment, the improvement in MoCA scores [5.0 (3.0, 6.0) vs. 2.5 (1.0, 4.0)], attention [1.0 (0.0, 1.0) vs. 0.0 (0.0, 1.0)], and abstract thinking scores [0.0 (0.0, 1.0) vs. 0.0 (0.0, 0.0)] in the music therapy group were better than that in the control group (P<0.05). There was no statistically significant difference in NIHSS, FMA, and MBI scores between the two groups of patients before treatment (P>0.05), and both groups improved after treatment compared to before treatment (P<0.05). After treatment, there was no statistically significant difference in the improvement of NIHSS, FMA, and MBI scores between the two groups of patients (P>0.05). Conclusions Compared with conventional rehabilitation therapy, training combined with music therapy is more beneficial for improving cognitive function in PSCI patients, especially in the cognitive domains of attention and abstract thinking. However, significant advantages have not been found in improving the degree of neurological impairment, limb motor function, and daily living activities.