Aim: To assess the methodological quality in the identification and selection of studies of evidence synthesis articles cited in the 2017/2018 U.S. Department of Health and Human Services (DHHS) Human Immunodeficiency Virus (HIV) Guidelines for the Use of Antiretroviral Agents in HIV-1 Infected Adults and Adolescents using the risk of bias in systematic reviews tool (ROBIS).

Background: The number of published systematic reviews has dramatically increased in recent years. The rapid influx of systematic reviews has made keeping up and evaluating primary literature nearly impossible for health care practitioners. Additionally, there has been some concern that a proportionate amount of newly published systematic reviews can be considered low-quality. Suboptimal systematic reviews can be harmful, given their significant prestige and the influence they have on supporting clinical decision making. The concern for low-quality systematic reviews has highlighted the importance of rigorously assessing systematic reviews.

Methods: Our corpus comprised of all literature cited in the 2017/2018 DHHS HIV Guidelines. Two investigators independently conducted a title and abstract screening of the corpus of literature and identified literature that were likely to be categorized as an evidence synthesis article. Full-text screening was then performed independently by two investigators that confirmed the eligibility of evidence synthesis articles included in the study. Evidence synthesis articles were then independently assessed by two investigators for their methodological quality of identifying and selecting studies using the ROBIS tool. Investigators also assessed literature using a questionnaire that looked at the studies’ search-related elements and the methods for article selection and data extraction.

Results: We identified 26 pieces of literature cited in the DHHS HIV Guidelines as evidence synthesis articles. Eleven evidence synthesis articles (42.31%) were considered to have a low risk of bias in the identification and selection of studies by the ROBIS tool and therefore considered to be of high methodological quality. In contrast, 12 evidence synthesis articles (46.15%) were considered to have a high risk of bias in the identification and selection of studies by the ROBIS tool and therefore considered to be of low methodological quality. The remaining three evidence synthesis articles (11.54%) were categorized as unclear in their risk of bias.

Conclusion:  We attempted to establish a benchmark of what is considered the best available evidence through a current guideline.  Using the ROBIS tool, our study identified that more than half of the identified evidence synthesis articles cited by the 2017/2018 DHHS HIV have a high risk of bias in the identification and selection of studies. With these results, it is clear to see that the methodological quality should be considered when evaluating the results of evidence synthesis articles and making guideline recommendations