ObjectiveTo evaluate whether and to what extent the new risk of bias (ROB) tool has been used in Cochrane systematic reviews (CSRs) on acupuncture. MethodsWe searched the Cochrane Database of Systematic Review (CDSR) in issue 12, 2011. Two reviewers independently selected CSRs which primarily focused on acupuncture and moxibustion. Then the data involving in essential information, the information about ROB (sequence generation, allocation concealment, blindness, incomplete outcome data, selective reporting and other potential sources of bias) and GRADE were extracted and statistically analyzed. ResultsIn total, 41CSRs were identified, of which 19 CSRs were updated reviews. Thirty-three were published between 2009 and 2011. 60.98% reviews used the Cochrane Handbook as their ROB assessment tool. Most CSRs gave information about sequence generation, allocation concealment, blindness, and incomplete outcome data, however, half of them (54.55%, 8/69) showed selective reporting or other potential sources of bias. Conclusion"Risk of bias" tools have been used in most CSRs on acupuncture since 2009. However, the lack of evaluation items still remains.
At present, there are many items/checklists used to assess the methodological quality of animal studies. Yet, no tool has been specifically designed for assessing internal validity of animal studies. This articles introduce and interprets SYRCLE's risk of bias tool for animal studies in detail for Chinese scholars to accurately assess the methodological quality of animal studies when they develop systematic reviews on animal studies, so as to provide references for scientific design and implementation of animal studies.
Currently there is no tool designed specifically to assess the risk of bias in the design, conduct or analysis of systematic reviews. ROBIS (Risk Of Bias In Systematic reviews), which was developed lately, aims mainly to assess the risk of bias in the conduct and result interpretation of systematic reviews relating to interventions, etiology, diagnosis and prognosis, as well as the relevance of the systematic review questions and the practice questions that their users want to address. This paper aims to introduce the ROBIS tool to Chinese systematic review developers, guideline developers and other researchers to promote the comprehension of it and its application, so as to improve the quality of systematic reviews in China.
ObjectiveTo interpret ROBIS, a new tool to evaluate the risk of bias in systematic reviews, to promote the comprehension of it and its proper application. MethodsWe explained each item of ROBIS tool, used it to evaluate the risk of bias of a selected intervention review whose title was Cyclophosphamide for Primary Nephrotic Syndrome of Children: A Systematic Review, and judged the risk of bias in the review. ResultsThe selected systematic review as a whole was rated as “high risk of bias”, because there existed high risk of bias in domain 2 to 4, namely identification and selection of studies, data collection and study appraisal, synthesis and findings. The risk of bias in domain 1 (study eligibility criteria) was low. The relevance of identified studies and the review’s research question was appropriately considered and the reviewers avoided emphasizing results on the basis of their statistical significance. ConclusionROBIS is a new tool worthy of being recommended to evaluate risk of bias in systematic reviews. Reviewers should use ROBIS items as standards to conduct and produce high quality systematic reviews.
ObjectiveTo evaluate the risk of bias and reliability of conclusions of systematic reviews (SRs) of lung cancer screening. MethodsWe searched PubMed, EMbase, The Cochrane Library (Issue 2, 2016), Web of Knowledge, CBM, WanFang Data and CNKI to collect SRs of lung cancer screening from inception to February 29th, 2016. The ROBIS tool was applied to assess the risk of bias of included SRs, and then GRADE system was used for evidence quality assessment of outcomes of SRs. ResultsA total of 11 SRs involving 5 outcomes (mortality, detection rate, survival rate, over-diagnosis and potential benefits and harms) were included. The results of risk of bias assessment by ROBIS tool showed:Two studies completely matched the 4 questions of phase 1. In the phase 2, 6 studies were low risk of bias in the including criteria field; 8 studies were low risk of bias in the literature search and screening field; 3 studies were low risk of bias in the data abstraction and quality assessment field; and 5 studies were low risk of bias in the data synthesis field. In the phase 3 of comprehensive risk of bias results, 5 studies were low risk. The results of evidence quality assessment by GRADE system showed:three studies had A level evidence on the outcome of mortality; 1 study had A level evidence on detection; 1 study had A level evidence on survival rate; 3 studies on over-diagnosis had C level evidence; and 2 studies on potential benefits and harms had B level evidence. ConclusionThe risk of bias of SRs of lung cancer screening is totally modest; however, the evidence quality of outcomes of these SRs is totally low. Clinicians should cautiously use these evidence to make decision based on local situation.
ObjectivesTo comprehensively evaluate the methodological quality and applicability of the results of systematic reviews on acupuncture treatment for primary depression.MethodsWeb of Science, EMbase, PubMed, The Cochrane Library, CNKI, CBM, WanFang Data and VIP databases were electronically searched to collect systematic reviews/meta-analyses on acupuncture treatment for primary depression from inception to December 5th, 2018. Two researchers independently screened and extracted data by using tools of AMSTAR 2 to evaluate the methodological quality, using ROBIS to assess risk of bias, and using CASP-S.R to evaluate the applicability of the results.ResultsA total of 18 systematic reviews/meta-analyses were included, and all focused on acupuncture intervention, including 2 primary outcome indicators. According to AMSTAR 2 evaluation results, there were 4 high quality studies, 12 medium quality studies and 2 low quality studies; ROBIS results found 10 high bias risk studies, 7 low bias risk studies and 1 unclear; CASP-S.R showed only 4 design studies applicable to local individuals, and there were no studies on the relationship between design benefits, hazards and costs.ConclusionsThe quality of systematic reviews/meta-analyses for acupuncture treatment of primary depression is moderate, however with a certain bias. Most studies may not directly benefit local individuals. All studies have no relationship with cost hazards. It is expected for further reviewers to strictly follow systematic evaluation method to improve research quality and reduce bias, while the applicability of the systematic review to individuals from different regions should be considered as well as the relationship between the benefit and cost hazard. In addition, more valid RCTs are required to provide higher quality evidence and explore correlated and comprehensive mechanism.
This study aims to introduce how to use the PROBAST (prediction model risk of bias assessment tool) to evaluate risk of bias and applicability of the study of diagnostic or prognostic predictive models, including the introduction of the background, the scope of application and use of the tool. This tool mainly involves the four areas of participants, predictors, outcomes and analyses. The risk of bias in the research is evaluated through the four areas, while the applicability is evaluated in the first three. PROBAST provides a standardized approach to evaluate the critical appraisal of the study of diagnostic or prognostic predictive models, which screens qualified literature for data analysis and helps to establish a scientific basis for clinical decision-making.
The COSMIN-RoB checklist includes three sections with a total of 10 boxes, which is used to evaluate risk of bias of studies on content validity, internal structure, and other measurement properties. COSMIN classifies reliability, measurement error, criteria validity, hypothesis testing for construct validity, and responsiveness as other measurement properties, which primarily focus on the quality of the (sub)scale as a whole, rather than on the item level. Among the five measurement properties, reliability, measurement error and criteria validity are the most widely used in the studies. Therefore, this paper aims to interpret COSMIN-RoB checklist with examples to guide researchers to evaluate the risk of bias of the studies on reliability, measurement error and criteria validity of PROMs.
Measurement properties studies of patient-reported outcome measures (PROMs) aims to validate the measurement properties of PROMs. In the process of designing and statistical analysis of these measurement properties studies, bias will occur if there are any defects, which will affect the quality of PROMs. Therefore, the COSMIN (consensus-based standards for the selection of health measurement instruments) team has developed the COSMIN risk of bias (COSMIN-RoB) checklist to evaluate risk of bias of studies on measurement properties of PROMs. The checklist can be used to develop systematic reviews of PROMs measurement properties, and for PROMs developers, it can also be used to guide the research design in the measurement tool development process for reducing bias. At present, similar assessment tools are lacking in China. Therefore, this article aims to introduce the primary contents of COSMIN-RoB checklist and to interpret how to evaluate risk of bias of the internal structure studies of PROMs with examples.
RoB2 (revised version 2019), an authoritative tool for assessing the risk of bias in randomized controlled trials, has been updated and improved based on the original version. This article elaborated and interpreted the background and main content of RoB2 (revised version 2019), as well as the operation process of the new software. Compared with the previous version of RoB2 (revised version 2018), RoB2 (revised version 2019) has the advantages of rich content, complete details, accurate questions, and simple operation, etc. Additionally, it is more user-friendly for researchers and beginners. The risk bias assessment of randomized controlled trials is more comprehensive and accurate, and it is an authoritative, trustworthy, and popular tool for evaluating the risk of bias in randomized controlled studies in medical practice.