Wednesday, May 4, 10:30 am - 12:00 pm
Registration Required
CME Credit Hours: 1.5
The peer review process is central to the continued advancement of scientific knowledge. Critical review of scientific manuscripts helps to ensure that the most reliable evidence is disseminated within the surgical community. The volume of new material, the complexity of trial design, and the limitations of peer review prior to publication are challenges to this process, for which post-publication peer review by readers of the scientific literature is necessary. While the practicing surgeon relies on the editorial process to a great extent to separate the “wheat from the chaff”, (s)he also requires solid critical appraisal skills to ensure that evidence from published studies is trustworthy as well as being wisely applied to the proper patient care scenarios. While the editor asks, “Does this manuscript add significant knowledge to the literature?” the reader asks, “Does this manuscript add new information that warrants a change to my practice?”
There are three generic types of surgical trials: exploratory trials to assess utility, explanatory trials to assess efficacy, and pragmatic trials to assess effectiveness. Methodologies include observational studies (cohort or case control), administrative database studies, randomized controlled trials (RCT), structured reviews, and meta-analyses. Each methodology has a role defined by its strengths and limitations.
Traditionally, observational studies are viewed as presenting evidence of the least evidentiary strength. Yet, there are many instances where an observational study is the best and perhaps the only investigatory approach that is practical, especially if a disease entity or outcome is rare. Observational studies may provide relatively strong evidence when there is a large treatment effect, or when confounding factors would bias the results in opposition to the observed effect. They may also be subject to significant bias, and thus their methodology and results must be carefully examined.
Large non-randomized observational studies based on administrative databases have become very popular due to ease of data abstraction. They have the advantage of reporting on large populations and identifying trends in treatment, outcomes and rare complications. However, data collection may be incomplete or inconsistent and the data may lack the granularity to draw conclusions to answer key questions.
The RCT design is least likely to be affected by bias and is the only methodology that can identify cause and effect. Sound knowledge of study-design is needed to evaluate the many variations in structure and primary outcomes (i.e. inferiority, non-inferiority). While no study is completely void of bias it is important to determine whether bias is responsible for a significant portion of the observed effect as there is wide variation in the quality of RCTs.
The sheer volume of primary literature has increased the importance of secondary analysis or literature summaries. A systematic review of the literature may be combined with a meta-analysis to give a best estimate of effect. Although pooling the results of multiple trials increases precision by narrowing confidence intervals, a secondary analysis of poorly designed RCTs may result in a misleading conclusion. Thus, the reviewer must be familiar with the common limitations of secondary analysis and the range of conclusions that can be drawn.
This symposium is aimed at three groups: present and prospective reviewers for Diseases of the Colon & Rectum (DC&R), the practicing surgeon who wants to increase their critical appraisal skills, and authors who wish to improve their writing skills. It is designed to be hands- on. Through an interactive symposium, we will explore the most common study methodologies, identify appropriate questions for each method, while identifying the advantages, the disadvantages, and the common mistakes in study conduct, reporting and conclusions. We will also explore essential resources for additional learning in this area.
Previously published representative papers from the four common methodologies will be identified in advance from DC&R. Six weeks prior to the symposium, the originally submitted unedited manuscripts of these four papers will be distributed to each symposium participant who will have the chance to read and critique them as if they were primary reviewers. At the symposium, each participant will be assigned to a small group led by an editorial board member from DC&R. Following an introduction of the manuscript by the faculty, the editorial board members will facilitate a working discussion and critique of each manuscript within the small groups. Board members will have access to the original editorial comments and the changes that were requested by the editors prior to publication to enhance the discussion. The benefit to each individual participant will be maximized with appropriate will be maximized with appropriate preparation time prior to the symposium TIME PRIOR TO THE SYMPOSIUM.
Objectives
At the conclusion of this session, participants should be able to:
- Recognize patient scenarios where observational studies are appropriate and can potentially provide strong evidence.
- Identify the advantages, limitations and proper use of studies using administrative databases
- Recognize the potential for bias and methodological limitations involving randomized controlled trials
- Recall the components of a valuable comprehensive systematic review and meta-analysis
- Apply resources to enhance their critical appraisal skills.
Agenda
Director: Susan Galandiuk, MD, Louisville, KY
Introduction
Susan Galandiuk, MD, Louisville, KY
Observational Studies
Rodrigo Oliva Perez, MD, PhD, São Paolo, Brazil
Observational Studies Small Group Discussions
Administrative Database Studies
Alexander Hawkins MD, MPH, Nashville, TN
Administrative Database Studies Small Group Discussions
Randomized Controlled Trials
Deborah Keller, MS, MD; Sacramento, CA
Randomized Controlled Trials Small Group Discussions
Systematic Reviews & Meta-Analyses
Aaron J. Dawes, MD, PhD, Palo Alto, CA
Systematic Reviews & Meta-Analyses Small Group Discussions
Adjourn