- A researcher develops a new intervention for panic disorder and recruits participants who have recently been hospitalized following a severe panic attack. After a six-week intervention, the average severity of their symptoms has significantly decreased. Besides the intervention itself, which phenomenon provides the most likely alternative explanation for this decrease?
Answer: Regression to the mean
The correct answer is B. Regression to the mean is a statistical phenomenon where extreme scores on a first measurement tend to be closer to the average on a second measurement. By selecting participants immediately after an extreme event (hospitalization for a severe panic attack), the researcher has selected individuals at their worst. It is statistically probable that their symptoms would naturally become less severe and move back toward their personal average level over time, regardless of any treatment.
(A) The Hawthorne effect is when participants change their behavior because they know they are being observed, which is a possible factor but less specific to this scenario than regression.
(C) The placebo effect requires the belief that one is receiving a treatment; while possible, regression to the mean is a more direct explanation given the participant selection method.
(D) The observer-expectancy effect relates to the researcher's expectations influencing the outcome, which is not the primary issue described.
- Researchers propose a study of a novel but risky psychosurgical procedure for severe, treatment-resistant obsessive-compulsive disorder. The design calls for randomly assigning patients to receive either the real surgery or a 'sham surgery,' which involves anesthesia and an incision but not the therapeutic brain intervention. An Institutional Review Board (IRB) would likely find which aspect of this study to be the most significant ethical challenge?
Answer: The inherent risks and invasive nature of the placebo condition for the control group.
The correct answer is D. The core ethical principle of 'do no harm' (non-maleficence) is paramount. While a sham surgery provides a powerful scientific control, it exposes participants in the control group to the risks of anesthesia, infection, and incision without any possibility of therapeutic benefit. An IRB would have to weigh the potential knowledge gained against this direct risk of harm to the control group, making it a profound ethical challenge.
(A) Financial cost is a practical issue, not a primary ethical one.
(B) While blinding is a methodological issue, it is possible to keep patients and assessors blind, even if surgeons are not. This is less of an ethical concern than causing potential harm.
(C) While an active control group might be a good alternative, the primary ethical red flag is the nature of the placebo itself, not the absence of another group.
- Researchers conducted a highly controlled experiment on a new medication for anxiety, using a very specific sample of college students with no other diagnosed disorders or medical conditions. This careful selection of participants likely increases the study's internal validity but may simultaneously limit its:
Answer: external validity.
The correct answer is B. External validity refers to the extent to which the results of a study can be generalized to other populations, settings, and times. By using a highly specific and homogenous sample (e.g., only college students with uncomplicated anxiety), the researchers minimize confounding variables, which boosts internal validity (the confidence that the medication caused the outcome within the study). However, this makes it uncertain whether the results would apply to older adults, people with co-occurring disorders, or other more complex, real-world populations, thus limiting external validity.
(A) Construct validity concerns whether the measures used are accurately assessing the intended psychological construct (anxiety).
(C) and (D) are forms of reliability, which refer to the consistency of measurements, not the generalizability of findings.
- In a randomized controlled trial for a new antidepressant, researchers used a wait-list control group. Participants in this group received no treatment for the duration of the study but were offered the treatment afterward. What is the primary limitation of using a wait-list control when trying to determine the specific effects of the new medication?
Answer: It fails to control for participant expectancy and the psychological effects of receiving attention.
The correct answer is B. A wait-list control group helps account for spontaneous remission (natural improvement over time), but it does not control for the placebo effect. The placebo effect stems from a participant's belief or expectation that they are receiving a helpful treatment and from the therapeutic effects of simply being in a study (e.g., receiving attention from clinicians). The wait-list group knows they are not receiving treatment, so this design cannot isolate the medication's pharmacological effects from these psychological factors.
(A) While there are ethical considerations, they are not the primary methodological limitation for determining the drug's specific effects.
(C) Since participants were randomly assigned, sampling bias is not the primary issue; rather, attrition (drop-out) might be higher in the wait-list group, but that is a different problem.
(D) While it is true that a wait-list design cannot be double-blind, the core limitation is its failure to account for the placebo effect, which is a more specific and central problem than the general issue of blinding.
- In a study testing a new form of 'energy therapy' for chronic pain, practitioners who administer the therapy are also responsible for assessing the participants' pain reduction. The practitioners are strong believers in the therapy's effectiveness. This design is most vulnerable to which specific type of bias?
Answer: Observer-expectancy effect
The correct answer is C. The observer-expectancy effect (a form of experimenter bias) occurs when a researcher's beliefs or expectations about the outcome of a study influence their observations or interactions with participants. In this case, practitioners who believe in the therapy might unconsciously perceive or rate participants' pain as being more reduced than it actually is. This is a major threat to validity when outcome assessment is not blinded.
(A) Attrition bias relates to participants dropping out of the study.
(B) The participant-expectancy effect (placebo effect) is about the participants' beliefs, not the practitioners'.
(D) Selection bias relates to how participants were assigned to groups at the beginning of the study.
- A wellness company advertises a new light therapy device for seasonal affective disorder, stating, "In our initial user study, 85% of participants reported a significant mood improvement after four weeks of daily use." From a scientific standpoint, why is this claim insufficient to establish the treatment's efficacy?
Answer: The absence of a control group means improvements could be due to placebo effects or spontaneous remission.
The correct answer is D. The most critical flaw in this claim is the lack of a control group. Without comparing the light therapy group to a group that did not receive the treatment (or received a placebo version), it is impossible to rule out alternative explanations for the improvement. These include the placebo effect (improvement due to expectation), spontaneous remission (symptoms improving on their own over time), or regression to the mean.
(A) While longer studies are often better, four weeks can be a sufficient duration to see effects for some treatments; this is a less fundamental flaw than the lack of a control group.
(B) Self-report measures are standard in psychology research. While they have limitations, they are not inherently invalid. The core issue is the study design, not the measurement type itself.
(C) The sample size is an important detail, but even a very large study that shows improvement is uninterpretable without a control group to compare it against.
- A famous actor gives a televised interview in which he passionately describes how a specific, unconventional 'neuro-vibrational therapy' cured his long-standing battle with depression. From the perspective of evidence-based practice, this actor's account should be considered:
Answer: unreliable evidence because it fails to control for alternative explanations.
The correct answer is D. This is a classic example of a testimonial or anecdotal evidence. While potentially compelling, it is scientifically unreliable because it cannot rule out numerous alternative explanations for the actor's improvement. These could include the placebo effect, a cyclical course of his depression (spontaneous remission), other lifestyle changes, or concurrent treatments. It is not a systematic investigation.
(A) This is not a case study, which would involve a systematic and detailed examination by a clinician or researcher.
(B) While it might inspire investigation, calling it 'compelling' overstates its scientific value. The key point to evaluate is its current evidentiary weight, which is low.
(C) The actor is an expert on his own experience, but not on the efficacy of treatments for depression in general; therefore, this is not expert opinion in a scientific sense.
- A study compares two different therapies for post-traumatic stress disorder (PTSD). At the end of the 12-week trial, Therapy A shows a significantly faster reduction in symptoms. However, at a one-year follow-up assessment, patients who received Therapy B report significantly lower rates of symptom relapse. This pattern of results underscores the critical importance of including which element in treatment evaluation studies?
Answer: Longitudinal follow-up assessments
When evaluating psychological treatments, researchers must consider both immediate effectiveness and long-term sustainability. This question highlights a crucial distinction: short-term symptom reduction doesn't guarantee lasting recovery.
The correct answer is A because this scenario perfectly demonstrates why longitudinal follow-up assessments are essential. Therapy A appeared superior at 12 weeks, but this initial advantage disappeared over time, while Therapy B showed better durability. Without the one-year follow-up, researchers would have concluded that Therapy A was more effective—a potentially harmful misinterpretation that could influence treatment guidelines and patient care.
Here's why the other options don't address the core issue: B) A no-treatment control group would help establish that both therapies work better than no intervention, but wouldn't reveal the different patterns of short-term versus long-term effectiveness shown here. C) Blinding therapists prevents bias in treatment delivery, but the pattern of results described could occur regardless of whether therapists knew the hypothesis. D) A large, diverse sample improves generalizability and statistical power, but sample characteristics don't explain why one therapy shows immediate benefits while another shows superior long-term outcomes.
Remember this key principle for treatment research questions: effectiveness must be measured across time, not just at treatment completion. Many psychological interventions show impressive immediate results that fade, while others build more gradually but provide lasting change. Always look for whether studies include follow-up periods when evaluating treatment research—it's often the most critical component for determining real-world clinical value.
- A researcher plans to test a new journaling technique for reducing work-related stress. The initial plan is to recruit volunteers, measure their stress levels, have them practice the technique for one month, and then measure their stress levels again. To transform this plan into a minimally acceptable controlled trial, which of the following changes is the most essential?
Answer: Adding a control group of participants who are randomly assigned to not practice the technique.
The correct answer is A. The most fundamental requirement of a controlled trial is a control or comparison group. Without it, there is no way to know if any observed changes are due to the intervention or to other factors (e.g., time passing, placebo effects). Randomly assigning participants to either the journaling group or a control group is the most crucial first step to allow for causal inference.
(B), (C), and (D) are all features of a good study, but they are refinements. Better measures (B), a longer duration (C), and a more representative sample (D) are all valuable, but none of them can substitute for the basic need for a control group, which is essential to the study's internal validity.
- A researcher is evaluating claims about a new digital therapeutic app. To do so, she statistically combines the results of 15 separate randomized controlled trials that have already been conducted on the app. This type of research is best described as:
Answer: a meta-analysis.
The correct answer is B. A meta-analysis is a statistical procedure for combining data from multiple studies to arrive at a single conclusion. It is commonly used to evaluate the overall efficacy of a treatment when numerous studies on the topic exist. By pooling data, a meta-analysis can increase statistical power and provide a more robust estimate of the treatment's effect size.
(A) A longitudinal study follows the same individuals over a period of time.
(C) A double-blind experiment is a design for a single study, not a method for combining studies.
(D) A case-control study is a retrospective, observational study that compares individuals with a condition (cases) to those without (controls).
- In a single-blind study of a new psychotropic drug, participants are unaware if they are receiving the drug or a placebo. However, the true drug has a very common and noticeable side effect of causing a persistent metallic taste. What is the most likely threat this side effect poses to the study's design?
Answer: It may effectively un-blind the participants, compromising the control for expectancy effects.
When you encounter questions about research design flaws, focus on how various factors can compromise the study's ability to control for confounding variables and maintain scientific rigor.
The metallic taste side effect creates a critical problem for this single-blind design. The whole purpose of keeping participants blind to their treatment condition is to control for expectancy effects—the phenomenon where participants' beliefs about their treatment influence their responses and outcomes. When participants can easily identify whether they're receiving the real drug (due to the distinctive metallic taste) versus the placebo, they effectively become unblinded. This compromises the study's control over expectancy effects, making it impossible to determine whether any observed benefits come from the drug itself or from participants' knowledge that they're receiving active treatment. This is exactly what answer A describes.
Answer B is incorrect because while some participants might drop out due to the unpleasant side effect, the question asks about the threat to study design, not participant retention. Answer C makes an unwarranted assumption—a common side effect doesn't necessarily indicate unsafe dosing, as many effective medications have noticeable but harmless side effects. Answer D is wrong because informed consent issues, while important ethically, don't directly threaten the study's methodological design; participants would have been informed about potential side effects during the consent process.
Remember: In research design questions, always consider how each factor affects the study's ability to isolate the variable being tested. Blinding procedures are specifically designed to control expectancy effects, so anything that compromises blinding threatens this control.
- In a study to determine the efficacy of a new form of psychodynamic therapy for depression, researchers compare it to a group receiving the current gold-standard treatment, Cognitive Behavioral Therapy (CBT). In this experimental design, the group receiving CBT is best described as what?
Answer: An active control group
The correct answer is D. An active control group (also known as a standard treatment or treatment-as-usual group) receives a treatment that is known to be effective. This design is used to determine if a new treatment is superior to, or at least as good as, the existing standard of care. It controls for general therapeutic effects beyond a simple placebo.
(A) A placebo control group would receive an inert or sham treatment designed to mimic the experimental therapy without its active components.
(B) The experimental group is the one receiving the new treatment being tested (the psychodynamic therapy).
(C) A wait-list control group would receive no treatment during the study period.
- A single, well-designed study funded by the manufacturer of a new herbal supplement finds it to be moderately more effective than a placebo in reducing anxiety. Before a clinician recommends this supplement, what would be the most important next step in the scientific validation process?
Answer: Seeking independent replication of the study's findings by a different research group.
The correct answer is C. Replication is a cornerstone of the scientific method. A single study, even if well-designed, can produce a fluke result or be subject to subtle, unintentional biases from the researchers, especially when they are funded by a party with a vested interest. Before a treatment is widely accepted, its effects must be replicated by independent researchers to ensure the findings are robust and reliable.
(A) A larger study by the same team is helpful but does not address the issue of potential researcher bias or a site-specific effect.
(B) Testimonials are anecdotal evidence and are not a valid method for scientific validation.
(D) Understanding the mechanism is scientifically interesting but is not as crucial for validating clinical efficacy as demonstrating that the effect is reliable through replication.
- An advertisement for a new self-help program claims it is a 'proven treatment' for phobias, citing data that nine out of ten initial participants reported 'feeling significantly better.' The primary reason this claim is scientifically unjustified is that the evidence fails to:
Answer: account for alternative explanations, such as placebo effects or spontaneous recovery.
The correct answer is C. The most fundamental flaw in this type of testimonial-based claim is the complete lack of a control group. Without a comparison, it is impossible to know if the improvement was caused by the program itself or by other factors like the participants' expectation of getting better (placebo effect), the simple passage of time (spontaneous recovery), or a desire to please the researchers. These alternative explanations must be ruled out by a controlled trial.
(A) While a sample size of 10 is very small, this is a secondary issue. Even with a sample of 10,000, the claim would be unjustified without a control group. This is a common distractor that traps students who focus on sample size without considering the more critical issue of study design.
(B) and (D) are also valid methodological concerns, but they are less central than the failure to account for alternative explanations, which is the core purpose of a controlled trial.
- A large clinical trial finds that a new therapy for social anxiety reduces symptoms by an average of 5 points on a 100-point scale compared to a placebo. The result is statistically significant (p < .01). When evaluating this treatment's utility, what is the most critical question a clinician should consider next?
Answer: Is a 5-point reduction in symptoms large enough to be clinically significant and meaningful for patients?
The correct answer is B. The scenario describes a result that is statistically significant, meaning it's unlikely to be due to chance. However, statistical significance does not automatically equate to practical or clinical significance. The most important next step is to evaluate whether the observed effect size (a 5-point change) represents a meaningful, real-world improvement for individuals suffering from social anxiety. A small, statistically significant effect may not be worth the cost, time, or potential side effects of the treatment.
(A), (C), and (D) are all valid and important questions in treatment evaluation, but the direct conflict presented in the stem is between the statistical result (p < .01) and the small effect size (5 points), making the question of clinical significance the most immediate and critical consideration.
- A research team conducts a double-blind, randomized controlled trial for a medication aimed at reducing obsessive-compulsive disorder symptoms. Which of the following potential biases is this specific research design most effectively able to minimize?
Answer: The influence of participants' and researchers' expectations on the study's outcomes.
The correct answer is A. A double-blind design, where neither the participants nor the researchers administering the treatment and assessing outcomes know who is in the experimental group versus the control group, is specifically intended to minimize expectancy effects. This includes participant expectations (placebo effect) and researcher expectations (observer-expectancy effect), both of which can systematically influence results.
(B) Blinding does not directly prevent attrition.
(C) Selection bias related to the overall sample's representativeness is addressed by sampling methods, not by blinding.
(D) Pre-existing differences between groups are minimized by random assignment, which is part of the design but is distinct from the function of blinding.
