Objective To enhance medical quality evaluation by conducting a systematic review and evidence-based synthesis of relevant research. Methods Such databases as CBM (1978 to 2009), VIP (1989 to 2009), CNKI (1990 to 2009), WanFang Dissertation database (1997 to 2009), and relevant websites were searched to identify relevant Chinese studies for the synthetical evaluation on methods evaluating medical quality. Results A total of 216 articles including 184 observational studies, 29 traditional reviews, and 3 experts’ opinions were identified. Twenty-one synthetical evaluation methods were used among 216 articles, and the top three methods which had been quoted more than 30 times were: RSR (19.72%), TOPSIS method (19.03%), and comprehensive index (12.80%). Among the 184 observational research papers, the documents using one kind of comprehensive evaluation took up 89.13% (164/184), the ones using two kinds of evaluation methods accounted for 9.78% (18/184), and those using three kinds of methods accounted for 1.09% (2/184). Conclusion a) The joint use of two or more than two methods is not common. The majority of studies using single comprehensive evaluation method focus on RSR, TOPSIS and comprehensive index method; b) The comprehensive evaluation method is considered non-uniform, some not even standardized; c) Most of the medical literatures do not correct the value of quality assessment; and d) Considerably most studies perform retrospective evaluation with historical data.