- A marketing analyst suspects the results do not accurately reflect the satisfaction of the total 50,000 customers. This suspicion is best justified by the high potential for:
Answer: nonresponse bias, where individuals with extreme opinions are disproportionately more likely to respond.
The low response rate (10%) combined with the polarized results (very high or very low ratings) is a classic indicator of nonresponse bias. Customers who had a terrible experience or a fantastic one are motivated to provide feedback, while the majority who had an average, unremarkable experience are less likely to respond. This skews the average result away from the true mean of the entire population.
- Why is it problematic to conclude that Layout B is superior based on this experiment's design?
Answer: The two groups of visitors were not from a randomized sample, creating a fundamental selection bias.
The core of valid A/B testing is random assignment. In this case, the two groups are systematically different from the start. Visitors from paid ads often have a higher purchase intent than those from organic search. By assigning all paid traffic to B and all organic to A, the company has introduced a major confounding variable. The difference in booking rates could be due to the layout, the traffic source, or both. This is a form of selection bias that makes the results invalid.
- Which of the following represents the most significant threat to the validity of the conclusion that there is overwhelming interest among all potential customers?
Answer: The sampling frame excludes customers who purchase coffee but are not members of the digital rewards program, leading to undercoverage bias.
The primary flaw is that the sampling frame (the list from which the sample is drawn) is limited to members of the digital rewards program. This systematically excludes all customers who are not in the program, who may have very different preferences. This is a classic example of undercoverage bias, a type of selection bias that severely limits the generalizability of the results to the entire customer population.
- Which sampling method is being used, and what is its primary limitation?
Answer: It is quota sampling; its limitation is that the selection within each category is non-random, introducing potential selection bias.
This method is quota sampling. The population is divided into subgroups (strata), and a specific number of interviews (a quota) is required for each. However, the selection of individuals within each quota is left to the interviewer's discretion (i.e., it's non-random). This is its key difference from stratified sampling and its primary weakness, as interviewers may, for example, approach people who look friendly, introducing significant selection bias.
- This sampling method is a clear example of:
Answer: one-stage cluster sampling, where stores are the clusters.
This is the definition of one-stage cluster sampling. The population is divided into groups, or clusters (the stores). A random sample of clusters is selected. Then, all members within the selected clusters are included in the sample. This is different from stratified sampling, where a random sample is taken from within each group (or a selection of groups).
- Management is cautious about generalizing these results to their entire customer base. The most significant source of bias in this research design is that the:
Answer: sampling method systematically excludes customers who use online banking, ATMs, or visit at other times.
The most fundamental flaw is selection bias. The sample is drawn only from customers who use a specific service (tellers) at a specific time (weekday mornings). This group may largely consist of retirees or small business owners and systematically excludes customers who work traditional hours, prefer digital channels, or use ATMs. The characteristics and satisfaction levels of these excluded groups are likely very different, making the results unrepresentative of the entire customer base.
- A restaurant wants to survey customers about their dining frequency. Which of the following questions is most flawed due to the use of ambiguous wording?
Answer: Do you visit our restaurant regularly?
The word 'regularly' is highly ambiguous and subjective. For one person, it might mean once a week; for another, it might mean once a month. Because it lacks a precise, objective definition, different respondents will interpret it differently, leading to unreliable and incomparable data.
- The survey finds that 95% of respondents claim to recycle 'always' or 'often.' This result is most likely inflated due to a combination of what two types of bias?
Answer: Social desirability bias and interviewer bias.
Two main biases are at play. First, social desirability bias occurs because respondents want to appear responsible and pro-environment, so they overstate their recycling habits. Second, interviewer bias occurs because the interviewers' appearance (vests, logo) and introductory statement signal a 'correct' attitude, influencing respondents to give answers that please the interviewer. These response biases are the most likely cause of the inflated results.
- A city is considering a new tax to fund public art projects. Which survey question is most clearly worded to bias responses against the tax by using loaded language?
Answer: Should the city impose a burdensome new tax on working families to pay for unnecessary public art projects?
This question uses loaded and emotionally charged words to frame the issue negatively. 'Burdensome', 'working families', and 'unnecessary' are not neutral terms; they are designed to evoke a negative response to the tax proposal, making it a classic example of a loaded question.
- To correctly implement a systematic sampling plan, which procedure should the researcher follow next?
Answer: Select every 20th person on the list after the 12th person (i.e., the 32nd, 52nd, 72nd, and so on).
Systematic sampling involves selecting a random starting point and then picking every k-th element in succession from the sampling frame. Here, the population is 2,000 and the desired sample is 100, so the sampling interval (k) is 2000/100 = 20. After starting at the 12th person, the researcher must select every 20th person thereafter to complete the systematic sample.
- What is the primary methodological flaw in this question's design?
Answer: It is a double-barreled question, forcing a single response for two separate and potentially unrelated issues.
The question asks for a single rating for two distinct concepts: compensation and professional development. An employee might be very satisfied with their compensation but very dissatisfied with development opportunities, or vice-versa. This is a double-barreled question, and it is impossible for the respondent to provide a single, valid answer.
- What is the most critical and fundamental flaw in the campaign's statement?
Answer: The calculation of a margin of error is statistically invalid because the sample was gathered through a non-probability method.
Margin of error is a statistical calculation that quantifies the random sampling error for probability samples (like a simple random sample). The poll described is a voluntary response sample, which is a non-probability method. Because the selection is not random, the principles of probability theory that underpin the margin of error calculation do not apply. Stating a margin of error for such a sample is misleading and statistically invalid. While undercoverage is also a problem, the fundamental error is making a statistical claim that is not applicable to the data collection method.
- A researcher suggests the ordering of the questions may be distorting the results, as thinking about one's career success might influence how one feels about their education. This is an example of what type of bias?
Answer: Question order bias, where earlier questions set a context that influences answers to later questions.
Question order bias occurs when the sequence of questions affects how respondents answer. In this case, asking about career success first primes the alumni to evaluate their education through the lens of its economic and professional payoff. If the questions were reversed, they might evaluate their education based on other factors like personal growth or relationships, potentially leading to different satisfaction ratings.
- This sampling strategy is best described as:
Answer: snowball sampling, because the sample size grows through referrals from initial subjects.
Snowball sampling (or chain-referral sampling) is a non-probability technique used when subjects are hard to locate. The researcher collects data from a few initial subjects, who then refer others. The sample 'snowballs' as it grows. This is the exact method described in the scenario.
- Assuming the responses gathered were accurate, what is the most probable sampling-related explanation for the discrepancy?
Answer: The sampling frame is flawed because it systematically excludes renters, who may have different spending patterns.
The sampling frame is the list from which a sample is drawn. By using the property tax roll, the frame only includes homeowners and excludes all renters. Household composition and income levels can differ significantly between homeowners and renters, which in turn affects grocery spending. This undercoverage is a fundamental flaw in the sampling frame.
- This sampling approach is best described as:
Answer: proportional stratified sampling.
The population is first divided into distinct, non-overlapping subgroups (strata): Basic, Pro, and Enterprise users. A random sample is then drawn from each stratum. The size of the sample from each stratum is proportional to its size in the population (e.g., Basic users are 70% of the population and make up 70% of the sample: 350/500). This is the definition of proportional stratified sampling.
- A software company is surveying users about a new, complex feature. Which of the following questions is most likely to introduce bias by leading the respondent?
Answer: Given the significant time savings our new workflow feature provides, how satisfied are you with the recent software update?
This question is leading because it begins with a premise ('Given the significant time savings...') that is presented as fact. This primes the respondent to think about the feature in a positive light before asking for their satisfaction level, biasing the response towards agreement.
- The brand manager's interpretation of the research is most clearly exhibiting:
Answer: confirmation bias in the analysis phase.
Confirmation bias is the tendency to search for, interpret, favor, and recall information that confirms or supports one's pre-existing beliefs. The manager is not objectively analyzing the data; she is selectively focusing on evidence that supports her strategy and rationalizing away evidence that contradicts it. This happens during the analysis and interpretation phase, not during sampling or data collection.
- A city council member, trained in market research, argues the results are likely skewed. What is the most powerful form of bias supporting this argument?
Answer: Voluntary response bias, as individuals with strong feelings, particularly positive ones in this case, are more likely to self-select into the survey.
This is a classic example of a voluntary response sample. The sample consists of individuals who chose to participate. Such samples are prone to bias because people with strong opinions (often positive or negative) are more likely to participate than those with moderate opinions. While undercoverage and selection bias exist, the most direct and powerful distorting effect comes from the self-selection mechanism itself.
- The marketing team is concerned that the feedback may not reflect the broader market. This concern is primarily rooted in the research team's use of:
Answer: convenience sampling, which limits the sample to a non-representative subgroup that is easily accessible.
The researchers chose a location that was easy for them to access (a single store). This is the definition of convenience sampling. The problem is that the shoppers at an upscale, natural foods store in an affluent area are not likely to be representative of the general population of snack consumers, leading to significant selection bias.
