Marketing Quiz: Sampling And Research Bias

20 questions · exam conditions

0:00

Sampling And Research BiasQuestion 1 of 20

A national retail chain wants to survey its employees across 500 stores. To make the process efficient, the research team randomly selects 25 stores from the list of 500 and then interviews every single employee working in those 25 selected stores.

This sampling method is a clear example of:

stratified sampling, where each store is treated as a separate stratum.

one-stage cluster sampling, where stores are the clusters.

systematic sampling, where every 20th store from the list was selected.

quota sampling, where employees are selected to fill demographic quotas.

← Back to quizzes

Marketing Quiz

Marketing Quiz: Sampling And Research Bias

Practice Sampling And Research Bias in Marketing with focused quiz questions that help you check what you know, review explanations, and build confidence with test-style prompts.

What this quiz covers

This quiz focuses on Sampling And Research Bias, giving you a quick way to practice the rules, question types, and explanations that matter most for Marketing.

How to use this quiz

Try each quiz question before looking at the correct answer. Use the explanations to review missed ideas, then come back to similar questions until the pattern feels familiar.

All questions

Question 1

This sampling method is a clear example of:

stratified sampling, where each store is treated as a separate stratum.
one-stage cluster sampling, where stores are the clusters. (correct answer)
systematic sampling, where every 20th store from the list was selected.
quota sampling, where employees are selected to fill demographic quotas.

Explanation: This is the definition of one-stage cluster sampling. The population is divided into groups, or clusters (the stores). A random sample of clusters is selected. Then, all members within the selected clusters are included in the sample. This is different from stratified sampling, where a random sample is taken from within each group (or a selection of groups).

Question 2

To estimate the average weekly grocery spend for households in a large city, a research firm uses the city's property tax roll as its sampling frame to mail surveys. They find that the calculated average spend is significantly lower than national averages.

Assuming the responses gathered were accurate, what is the most probable sampling-related explanation for the discrepancy?

Nonresponse bias, as higher-spending households may have been less likely to return the mailed survey.
The sampling frame is flawed because it systematically excludes renters, who may have different spending patterns. (correct answer)
Social desirability bias, where respondents under-reported their spending to appear more frugal to the researchers.
Systematic sampling error, because using a list like a tax roll can introduce periodic biases that skew the results.

Explanation: The sampling frame is the list from which a sample is drawn. By using the property tax roll, the frame only includes homeowners and excludes all renters. Household composition and income levels can differ significantly between homeowners and renters, which in turn affects grocery spending. This undercoverage is a fundamental flaw in the sampling frame.

Question 3

A software company has 10,000 customers: 7,000 on a 'Basic' plan, 2,000 on a 'Pro' plan, and 1,000 on an 'Enterprise' plan. To conduct a satisfaction survey, they want a sample of 500 customers that reflects this distribution. They create three separate lists and randomly select 350 Basic customers, 100 Pro customers, and 50 Enterprise customers to survey.

This sampling approach is best described as:

proportional stratified sampling. (correct answer)
disproportional quota sampling.
multi-stage cluster sampling.
simple random sampling.

Explanation: The population is first divided into distinct, non-overlapping subgroups (strata): Basic, Pro, and Enterprise users. A random sample is then drawn from each stratum. The size of the sample from each stratum is proportional to its size in the population (e.g., Basic users are 70% of the population and make up 70% of the sample: 350/500). This is the definition of proportional stratified sampling.

Question 4

To assess the quality of its customer service, a bank invites customers to complete a short survey immediately after they use the in-person teller services between 9 a.m. and 12 p.m. on weekdays. The results show very high satisfaction scores.

Management is cautious about generalizing these results to their entire customer base. The most significant source of bias in this research design is that the:

survey suffers from social desirability bias, as customers may feel pressured to give positive feedback in person.
sample size may be too small to be statistically significant, regardless of the sampling method used.
sampling method systematically excludes customers who use online banking, ATMs, or visit at other times. (correct answer)
survey is subject to nonresponse bias from customers who are in a hurry and decline to participate.

Explanation: The most fundamental flaw is selection bias. The sample is drawn only from customers who use a specific service (tellers) at a specific time (weekday mornings). This group may largely consist of retirees or small business owners and systematically excludes customers who work traditional hours, prefer digital channels, or use ATMs. The characteristics and satisfaction levels of these excluded groups are likely very different, making the results unrepresentative of the entire customer base.

Question 5

A researcher wants to understand the purchasing habits of individuals who collect rare, vintage comic books, a small and hard-to-reach population. The researcher identifies five collectors through a specialized online forum and then asks each of them to provide contact information for other collectors they know. The sample grows as each new contact refers others.

This sampling strategy is best described as:

judgmental sampling, because the initial contacts were selected based on the researcher's judgment.
convenience sampling, because the researcher is using the most easily accessible contacts.
snowball sampling, because the sample size grows through referrals from initial subjects. (correct answer)
panel sampling, because it involves tracking a cohort of specialized individuals over time.

Explanation: Snowball sampling (or chain-referral sampling) is a non-probability technique used when subjects are hard to locate. The researcher collects data from a few initial subjects, who then refer others. The sample 'snowballs' as it grows. This is the exact method described in the scenario.

Question 6

A national coffee chain wants to gauge interest in a new plant-based milk option. They conduct a survey by sending an email link to all members of their digital rewards program. The results from the 25,000 responses indicate overwhelming interest in the new option.

Which of the following represents the most significant threat to the validity of the conclusion that there is overwhelming interest among all potential customers?

Nonresponse bias, as the members who are most passionate about new options are more likely to have completed the survey.
The sampling frame excludes customers who purchase coffee but are not members of the digital rewards program, leading to undercoverage bias. (correct answer)
Social desirability bias, as respondents may feel pressured to express interest in a product perceived as trendy and environmentally friendly.
The sample size is too large, leading to an over-representation of niche opinions that are not reflective of the general customer base.

Explanation: The primary flaw is that the sampling frame (the list from which the sample is drawn) is limited to members of the digital rewards program. This systematically excludes all customers who are not in the program, who may have very different preferences. This is a classic example of undercoverage bias, a type of selection bias that severely limits the generalizability of the results to the entire customer population.

Question 7

A political campaign conducts an online poll by placing a banner on a popular news website that invites visitors to share their opinion. The poll receives 1,000 responses. The campaign's report on the results states, 'The poll shows our candidate is favored by 55% of voters, with a margin of error of +/- 3%.'

What is the most critical and fundamental flaw in the campaign's statement?

The sample size of 1,000 is too small for a national poll, so the margin of error should be much larger.
The calculation of a margin of error is statistically invalid because the sample was gathered through a non-probability method. (correct answer)
The report fails to mention the confidence level (e.g., 95%) associated with the stated margin of error.
The poll suffers from undercoverage of voters who do not visit that specific news website, biasing the 55% figure.

Explanation: Margin of error is a statistical calculation that quantifies the random sampling error for probability samples (like a simple random sample). The poll described is a voluntary response sample, which is a non-probability method. Because the selection is not random, the principles of probability theory that underpin the margin of error calculation do not apply. Stating a margin of error for such a sample is misleading and statistically invalid. While undercoverage is also a problem, the fundamental error is making a statistical claim that is not applicable to the data collection method.

Question 8

An airline emails a satisfaction survey to 50,000 customers who flew in the last month. Only 5,000 customers respond. An analysis of the data reveals that 45% of respondents rated their experience as '1 - Very Poor' and 50% rated it as '5 - Excellent', with very few ratings in the middle.

A marketing analyst suspects the results do not accurately reflect the satisfaction of the total 50,000 customers. This suspicion is best justified by the high potential for:

undercoverage bias, because the sample excluded customers without valid email addresses on file.
interviewer bias, because the email's wording may have influenced how customers responded to the questions.
nonresponse bias, where individuals with extreme opinions are disproportionately more likely to respond. (correct answer)
sampling error, because the sample of 5,000 is not large enough to represent a population of 50,000.

Explanation: The low response rate (10%) combined with the polarized results (very high or very low ratings) is a classic indicator of nonresponse bias. Customers who had a terrible experience or a fantastic one are motivated to provide feedback, while the majority who had an average, unremarkable experience are less likely to respond. This skews the average result away from the true mean of the entire population.

Question 9

A university is surveying alumni. The first section of the survey asks a series of detailed questions about their current income, career progression, and job satisfaction. The second section asks them to rate their overall satisfaction with their education at the university. The university notices that satisfaction ratings are significantly correlated with the reported career success.

A researcher suggests the ordering of the questions may be distorting the results, as thinking about one's career success might influence how one feels about their education. This is an example of what type of bias?

Confirmation bias, where the university is seeking to prove its economic value to alumni.
Social desirability bias, where alumni feel the need to exaggerate both their success and their satisfaction.
Nonresponse bias, as successful alumni may be more likely to complete the entire survey.
Question order bias, where earlier questions set a context that influences answers to later questions. (correct answer)

Explanation: Question order bias occurs when the sequence of questions affects how respondents answer. In this case, asking about career success first primes the alumni to evaluate their education through the lens of its economic and professional payoff. If the questions were reversed, they might evaluate their education based on other factors like personal growth or relationships, potentially leading to different satisfaction ratings.

Question 10

A city's parks department posts a QR code on signs in all major parks, inviting visitors to take a survey about park maintenance and cleanliness. The survey receives 2,000 responses, with 90% rating the parks as 'excellent' or 'very good.' The department presents this as strong evidence of widespread satisfaction.

A city council member, trained in market research, argues the results are likely skewed. What is the most powerful form of bias supporting this argument?

Voluntary response bias, as individuals with strong feelings, particularly positive ones in this case, are more likely to self-select into the survey. (correct answer)
Selection bias, because the QR codes were only placed in major parks, ignoring smaller neighborhood parks which may be less maintained.
Undercoverage bias, as the survey method excludes residents who do not visit parks or do not own smartphones capable of scanning QR codes.
Acquiescence bias, as respondents are generally inclined to agree with positive statements about public services when surveyed.

Explanation: This is a classic example of a voluntary response sample. The sample consists of individuals who chose to participate. Such samples are prone to bias because people with strong opinions (often positive or negative) are more likely to participate than those with moderate opinions. While undercoverage and selection bias exist, the most direct and powerful distorting effect comes from the self-selection mechanism itself.

Question 11

A human resources department is conducting an employee satisfaction survey. One question reads: 'Please rate your satisfaction with your compensation and opportunities for professional development on a scale of 1 (Very Dissatisfied) to 5 (Very Satisfied).'

What is the primary methodological flaw in this question's design?

It uses loaded language by assuming employees should be satisfied with both aspects of their job.
It is a double-barreled question, forcing a single response for two separate and potentially unrelated issues. (correct answer)
It is a leading question that suggests a desired response, thus creating acquiescence bias.
It suffers from ambiguity because 'satisfaction' and 'compensation' are not clearly defined for the respondent.

Explanation: The question asks for a single rating for two distinct concepts: compensation and professional development. An employee might be very satisfied with their compensation but very dissatisfied with development opportunities, or vice-versa. This is a double-barreled question, and it is impossible for the respondent to provide a single, valid answer.

Question 12

A non-profit organization dedicated to environmental protection conducts a door-to-door survey about household recycling habits. The surveyors are enthusiastic volunteers who wear vests with the organization's prominent 'Save the Planet' logo and begin by saying, 'We're working to measure how much our community cares about recycling.'

The survey finds that 95% of respondents claim to recycle 'always' or 'often.' This result is most likely inflated due to a combination of what two types of bias?

Confirmation bias and leading questions.
Nonresponse bias and undercoverage bias.
Social desirability bias and interviewer bias. (correct answer)
Voluntary response bias and selection bias.

Explanation: Two main biases are at play. First, social desirability bias occurs because respondents want to appear responsible and pro-environment, so they overstate their recycling habits. Second, interviewer bias occurs because the interviewers' appearance (vests, logo) and introductory statement signal a 'correct' attitude, influencing respondents to give answers that please the interviewer. These response biases are the most likely cause of the inflated results.

Question 13

A market researcher is given a database of 2,000 loyalty program members, sorted alphabetically. The researcher needs a sample of 100 members. They randomly select the 12th person on the list as the starting point.

To correctly implement a systematic sampling plan, which procedure should the researcher follow next?

Randomly select 99 more people from the remaining 1,999 members in the database.
Divide the list into 100 groups of 20 and randomly select one person from each group.
Select every 20th person on the list after the 12th person (i.e., the 32nd, 52nd, 72nd, and so on). (correct answer)
Select the next 99 people on the list immediately following the 12th person (i.e., positions 13 through 111).

Explanation: Systematic sampling involves selecting a random starting point and then picking every k-th element in succession from the sampling frame. Here, the population is 2,000 and the desired sample is 100, so the sampling interval (k) is 2000/100 = 20. After starting at the 12th person, the researcher must select every 20th person thereafter to complete the systematic sample.

Question 14

A travel website tests a new homepage layout (Layout B) against its current one (Layout A). All visitors arriving from paid search ads are shown Layout B, while all visitors arriving from organic (non-paid) search results are shown Layout A. The test finds that Layout B has a 15% higher booking rate.

Why is it problematic to conclude that Layout B is superior based on this experiment's design?

The Hawthorne effect may have caused visitors who knew they were seeing a new layout to behave differently.
The two groups of visitors were not from a randomized sample, creating a fundamental selection bias. (correct answer)
Confirmation bias may have led the designers to interpret the booking rate data in favor of Layout B.
The 15% difference in booking rate is likely due to random sampling error rather than a true difference.

Explanation: The core of valid A/B testing is random assignment. In this case, the two groups are systematically different from the start. Visitors from paid ads often have a higher purchase intent than those from organic search. By assigning all paid traffic to B and all organic to A, the company has introduced a major confounding variable. The difference in booking rates could be due to the layout, the traffic source, or both. This is a form of selection bias that makes the results invalid.

Question 15

A restaurant wants to survey customers about their dining frequency. Which of the following questions is most flawed due to the use of ambiguous wording?

How many times have you eaten at our restaurant in the past 30 days?
Do you visit our restaurant regularly? (correct answer)
On a scale of 1 to 5, where 1 is 'less than once a month' and 5 is 'more than once a week,' how often do you visit?
Excluding this visit, when was the last time you dined at this specific location (e.g., yesterday, last week)?

Explanation: The word 'regularly' is highly ambiguous and subjective. For one person, it might mean once a week; for another, it might mean once a month. Because it lacks a precise, objective definition, different respondents will interpret it differently, leading to unreliable and incomparable data.

Question 16

A consumer packaged goods company is developing a new line of organic snacks. To gather initial feedback on taste and packaging, they conduct interviews with shoppers at a large, upscale natural foods store located in an affluent suburb.

The marketing team is concerned that the feedback may not reflect the broader market. This concern is primarily rooted in the research team's use of:

judgmental sampling, which is prone to researcher bias in selecting individuals who seem representative.
quota sampling, which may not accurately reflect the proportions of the true market segments.
stratified sampling, as shoppers at that store represent only one stratum of the overall population.
convenience sampling, which limits the sample to a non-representative subgroup that is easily accessible. (correct answer)

Explanation: The researchers chose a location that was easy for them to access (a single store). This is the definition of convenience sampling. The problem is that the shoppers at an upscale, natural foods store in an affluent area are not likely to be representative of the general population of snack consumers, leading to significant selection bias.

Question 17

What is the primary methodological flaw in this question's design?

It uses loaded language by assuming employees should be satisfied with both aspects of their job.
It is a double-barreled question, forcing a single response for two separate and potentially unrelated issues. (correct answer)
It is a leading question that suggests a desired response, thus creating acquiescence bias.
It suffers from ambiguity because 'satisfaction' and 'compensation' are not clearly defined for the respondent.

Question 18

A marketing analyst suspects the results do not accurately reflect the satisfaction of the total 50,000 customers. This suspicion is best justified by the high potential for:

undercoverage bias, because the sample excluded customers without valid email addresses on file.
interviewer bias, because the email's wording may have influenced how customers responded to the questions.
nonresponse bias, where individuals with extreme opinions are disproportionately more likely to respond. (correct answer)
sampling error, because the sample of 5,000 is not large enough to represent a population of 50,000.

Question 19

This sampling strategy is best described as:

judgmental sampling, because the initial contacts were selected based on the researcher's judgment.
convenience sampling, because the researcher is using the most easily accessible contacts.
snowball sampling, because the sample size grows through referrals from initial subjects. (correct answer)
panel sampling, because it involves tracking a cohort of specialized individuals over time.

Question 20

To correctly implement a systematic sampling plan, which procedure should the researcher follow next?

Randomly select 99 more people from the remaining 1,999 members in the database.
Divide the list into 100 groups of 20 and randomly select one person from each group.
Select every 20th person on the list after the 12th person (i.e., the 32nd, 52nd, 72nd, and so on). (correct answer)
Select the next 99 people on the list immediately following the 12th person (i.e., positions 13 through 111).