ObjectivesTo develop a tool to assess the credibility of cohort studies regarding anticoagulants treatment for patients with atrial fibrillation.MethodsMEDLINE, EMbase and CBM databases were retrieved for eligible studies on the methodological quality and credibility of the cohort study. The retrieval period was from inception to December 19th, 2017, and the languages were limited to Chinese and English. Based on the search results, the items on the methodology quality of the research or the credibility of the results were collected. Through brainstorming of the core group, the collected items were expanded and sorted out to a preliminary scale. Furthermore, we conducted 4 rounds of internal expert seminars to discuss and evaluate the preliminary scale content and discuss the scale framework. Finally, we organized domestic authoritative experts to conduct 4 external evaluations on the scale and formed the final scale.ResultsA total of 7 734 literatures were obtained from primary search, in which 17 of which were included. Fifty-five tools with 780 items for the quality assessment of observational studies were collected from the included studies. After removing duplicated keywords with similar meanings, we obtained the " keyword pool” with 46 keywords, which were preliminarily formed 46 items. After discussion of the core group, which aimed to supplement the potential items, eliminate the items irrelevant to methodology, and integrate the items with partial overlapping meanings, a preliminary scale of 43 items was formed. According to the four rounds of internal expert consultative meetings, these items were integrated to form a preliminary scale of 40 items. After further evaluation by four rounds of external expert consultative meetings, a consensus was reached and a scale of 21 items from seven domains (i.e., definition of question, measurement, follow-up, confounder, missing data, statistical analysis, and results assessment) was finally formed.ConclusionsThis study developed an assessment tool for the credibility of the results from the cohort studies regarding anticoagulants treatment in patients with atrial fibrillation, which has practical clinical value. Clinicians can refer to the results of credibility assessment by using this tool to better assist clinical decision-making in clinical practice. In addition, in the preparation of relevant guidelines, this tool can be used to assess the credibility of results from cohort studies.