Evidence Appraisal — Formulate clinical questions and critically appraise research to guide intervention decisions

Historical Context & Motivation

For much of the twentieth century, clinical decisions in rehabilitation and healthcare were guided primarily by professional authority, apprenticeship traditions, and individual clinical experience. Practitioners relied on what they had been taught by mentors, supplemented by anecdotal observations from their own caseloads. While this approach produced skilled clinicians, it also meant that ineffective or even harmful interventions could persist for decades without systematic scrutiny. The recognition that clinical practice needed a more rigorous foundation gave rise to a transformative movement that would fundamentally reshape how occupational therapists—and all healthcare professionals—make intervention decisions.

The concept of evidence-based practice (EBP) emerged from the broader evidence-based medicine (EBM) movement, which sought to integrate the best available research evidence with clinical expertise and patient values. In occupational therapy, this shift was not merely academic—it was a professional imperative. As healthcare systems demanded accountability, payers required justification for services, and clients expected transparency, the ability to formulate precise clinical questions and critically appraise research became an essential competency for every practitioner.

The central question that evidence appraisal seeks to answer is deceptively simple: How do we know whether a given intervention actually works for a specific client population? Answering this question requires a structured process—moving from clinical uncertainty to a searchable question, locating the best available evidence, appraising its quality, and integrating findings into client-centered practice. This lesson equips you with the frameworks and skills to navigate each stage of that process.

Core Principles of Evidence Appraisal

Evidence appraisal in occupational therapy rests on several foundational principles that guide practitioners from recognizing a knowledge gap to translating research findings into clinical action. These principles are not abstract ideals—they represent a practical workflow that every OTR can apply in daily practice, whether working in acute care, outpatient rehabilitation, school-based settings, or community health. Understanding these core ideas transforms evidence appraisal from a daunting academic exercise into a manageable, even intuitive, clinical habit.

The Evidence-Based Practice Process — Visual Framework

The evidence-based practice process in occupational therapy is best understood as a cyclical workflow rather than a linear sequence. Each stage feeds into the next, and the final evaluation stage often generates new clinical questions, restarting the cycle. The following diagram illustrates this iterative process and highlights the three domains that converge in evidence-informed decision-making: best available research evidence, clinical expertise, and client values and context.

Notice that the cycle is continuous and self-renewing. When you assess outcomes in Step 5 and find that the intervention produced unexpected results—whether positive, negative, or equivocal—that observation becomes the seed of a new clinical question, returning you to Step 1. This iterative nature ensures that evidence appraisal is not a one-time task but an ongoing professional practice that evolves alongside the research literature and your growing clinical expertise.

The PICO Framework — Formulating Clinical Questions

The PICO framework is the cornerstone tool for translating vague clinical uncertainty into a precise, searchable research question. Without a well-structured question, literature searches become unfocused, yielding either an overwhelming volume of irrelevant results or missing critical studies altogether. PICO provides a systematic template that ensures every element of the clinical scenario is captured, enabling efficient database searching and targeted evidence retrieval.

Anatomy of a PICO Question

Combining these elements produces a complete clinical question: "In adults aged 50–75 with chronic stroke and upper extremity hemiparesis (P), does constraint-induced movement therapy (I), compared to conventional occupational therapy (C), improve upper extremity function as measured by the ARAT (O)?" This question is specific enough to guide a targeted database search and broad enough to capture relevant studies across multiple research designs.

Types of Clinical Questions

Clinical questions in OT practice generally fall into two categories. Background questions seek general knowledge about a condition, intervention, or phenomenon (e.g., "What is sensory integration therapy?") and are typically answered by textbooks or review articles. Foreground questions address specific clinical decisions and are best answered by primary research—these are the questions that the PICO framework structures most effectively. As you progress from novice to experienced clinician, the proportion of foreground questions you generate will increase, reflecting deeper engagement with nuanced clinical decision-making.

Levels of Evidence and Study Design Hierarchy

Not all research evidence carries equal weight. The levels of evidence hierarchy ranks study designs according to their ability to minimize bias and establish causal relationships between interventions and outcomes. Understanding this hierarchy is essential for critical appraisal because it allows you to quickly assess the strength of evidence supporting a particular intervention. However, the hierarchy is a guide, not an absolute rule—a well-designed cohort study may provide stronger evidence than a poorly executed randomized controlled trial.

For occupational therapy intervention questions, Level I systematic reviews represent the gold standard because they synthesize findings from multiple individual studies, reducing the influence of any single study's limitations. Level II RCTs are particularly valuable because randomization minimizes selection bias and confounding, allowing stronger causal inferences about whether an intervention produced the observed outcome. However, many areas of OT practice—especially those involving complex, individualized interventions in natural contexts—may have limited RCT evidence, requiring practitioners to draw on lower levels while acknowledging the associated uncertainty.

Worked Example — From Clinical Scenario to Evidence-Informed Decision

The following worked example walks through the complete evidence appraisal process for a common occupational therapy clinical scenario, demonstrating how each step of the EBP cycle operates in practice.

Critical Appraisal Tools and Criteria

Critical appraisal is the systematic evaluation of research quality to determine how much confidence you can place in a study's findings. Several structured tools have been developed to guide this process, each tailored to specific study designs. Knowing which tool to use and what questions to ask transforms critical appraisal from a subjective judgment into a transparent, reproducible evaluation process.

The Three Core Appraisal Questions

Regardless of which tool you use, critical appraisal always addresses three fundamental questions. First, are the results valid? This concerns internal validity—whether the study was designed and conducted in a way that minimizes systematic error (bias). Key considerations include randomization procedures, allocation concealment, blinding of participants and assessors, completeness of follow-up, and whether intention-to-treat analysis was used. Second, what are the results? This examines effect size (how large was the treatment effect?), precision (how narrow are the confidence intervals?), and statistical versus clinical significance. Third, are the results applicable to my client? This addresses external validity—whether the study population, setting, and intervention protocol are sufficiently similar to your clinical context to warrant applying the findings.

Connecting Evidence Appraisal to Advanced Practice and OT Theory

Evidence appraisal does not exist in isolation—it intersects with occupational therapy theory, professional reasoning, and the broader healthcare quality improvement ecosystem. As practitioners develop advanced competency, they move from simply consuming evidence to contributing to it through practice-based evidence, program evaluation, and outcomes research. This section connects the foundational evidence appraisal skills covered in this lesson to the broader landscape of professional development and advanced clinical reasoning.

An important consideration for advanced integration is the relationship between evidence appraisal and OT-specific theoretical frameworks. The Occupational Therapy Practice Framework (OTPF) emphasizes occupation-based, client-centered practice—values that must guide how evidence is appraised and applied. A study may demonstrate that a biomechanical intervention improves range of motion, but the occupational therapist must evaluate whether that improvement translates into meaningful occupational participation for the client. This requires looking beyond statistical significance to clinical significance and occupational relevance—a distinctly OT lens on evidence appraisal that sets our profession apart from purely biomedical approaches.

Practice Problems

Evidence Appraisal — Key Concepts Review