Apply Program Evaluation Methods - Licensed Master Social Worker LMSW

What is a key distinction between reliability and validity of a measurement tool?

Reliability is consistency; validity is accuracy of measurement. Reliability ensures stable results across uses, while validity confirms the tool measures the intended construct accurately.
What is the key difference between formative and summative program evaluation?

Formative improves during delivery; summative judges outcomes after. Formative evaluation focuses on ongoing refinements to enhance program processes, while summative assesses overall impact post-implementation.
Identify the internal validity threat when an external event, not the program, influences outcomes.

History. History confounds results when concurrent events outside the program affect participant outcomes independently.
Which formula gives program completion rate if $60$ of $75$ enrolled clients finish services?

Completion rate $=\frac{60}{75}=0.80$ (80%). The completion rate formula divides successful completions by total enrollments to quantify program retention effectiveness.
What is regression to the mean as a threat to internal validity?

Extreme baseline scores tend to move toward average on retest. Regression to the mean occurs due to statistical tendencies, misleading attribution of changes to the intervention.
What is outcome evaluation designed to assess?

Whether the program achieved its intended client or system changes. Outcome evaluation measures the extent to which program goals for client or systemic improvements are met.
Which quality improvement approach uses iterative small tests of change: PDSA or RCT?

PDSA (Plan-Do-Study-Act). PDSA facilitates continuous quality enhancement through rapid cycles of testing and refining small-scale changes.
Which term refers to changes in clients or systems that result from services (short or long term)?

Outcomes. Outcomes evaluate the actual benefits or transformations resulting from program interventions on participants or communities.
What is fidelity in program evaluation?

The degree to which services are delivered as intended. Fidelity ensures program integrity by verifying adherence to the designed model, which supports reliable evaluation results.
What is selection bias in program evaluation?

Group differences at baseline that are unrelated to the intervention. Selection bias arises from non-random group assignment, leading to pre-existing differences that skew evaluation findings.
Identify the major threat to internal validity when participants improve due only to the passage of time.

Maturation. Maturation threatens validity by attributing natural developmental changes to the intervention rather than time alone.
What is process evaluation designed to assess?

Program implementation, fidelity, reach, and service delivery. Process evaluation examines operational aspects to ensure programs are executed efficiently and reach intended populations.
What does benchmarking mean in quality improvement?

Comparing performance to a recognized standard or peer programs. Benchmarking identifies performance gaps by using external standards to drive targeted quality improvements.
What is a logic model in program evaluation?

A map linking inputs, activities, outputs, and outcomes. Logic models visually represent program theory by connecting resources to actions and expected results for evaluation planning.
Which option best defines a structure measure in the Donabedian model?

Resources and organizational capacity (staffing, policies, equipment). Structure measures assess foundational elements that enable quality care in the Donabedian framework for evaluation.
Which option best defines a process measure in the Donabedian model?

What the program does (service steps, adherence, timeliness). Process measures evaluate the execution of services to ensure they align with best practices in Donabedian's model.
Which formula gives percent change if average score drops from $50$ to $40$ after intervention?

Percent change $=\frac{40-50}{50}\times 100=-20\%$ . The percent change formula calculates relative improvement or decline by comparing post- to pre-intervention values.
Which option best defines an outcome measure in the Donabedian model?

Client or population results (symptoms, functioning, quality of life). Outcome measures capture the end results of care, reflecting health and well-being impacts in Donabedian's framework.
What is a quasi-experimental design in program evaluation?

A comparison design without random assignment. Quasi-experimental designs allow causal inferences in real-world settings where randomization is impractical or unethical.
Which evaluation design is strongest for causal inference: RCT, quasi-experimental, or pretest-posttest only?

Randomized controlled trial (RCT). RCTs minimize bias through randomization, providing the highest evidence level for establishing program causality.
Which term refers to immediate deliverables of a program (for example, sessions delivered)?

Outputs. Outputs measure direct products of program activities, indicating operational productivity rather than broader impacts.
What does it mean for an outcome measure to be clinically meaningful, not only statistically significant?

The change is large enough to matter in real-world functioning. Clinical meaningfulness evaluates practical significance, ensuring statistical changes translate to tangible improvements in daily life.
What is the primary purpose of program evaluation in social work practice?

To determine effectiveness and inform decisions about services. Program evaluation assesses if interventions meet goals and guides resource allocation for optimal service delivery in social work.
What is a needs assessment used to determine before starting or revising a program?

The gap between current conditions and desired conditions. Needs assessments identify discrepancies to prioritize interventions that address specific community or client deficiencies effectively.

Study these flashcards