Computing Bias - AP Computer Science Principles

1. How can a programmer's personal, unconscious assumptions negatively impact a computing innovation, even if they have no intention of creating a biased system?
   Answer: By making design choices based on their own experiences, which may not represent the needs or characteristics of all users. Correct. Unconscious bias often manifests when developers make assumptions based on their own narrow worldview. For example, they might assume all names have a first and last name, or that all users have reliable high-speed internet. These assumptions get built into the software and can create unfair disadvantages or exclude certain user groups. - A describes intentional harm, not unconscious bias. - B describes a coding error unrelated to social or cognitive bias. - C describes an issue of performance optimization, not fairness.
2. Why is having a diverse development team considered a crucial step in mitigating computing bias?
   Answer: Team members from different backgrounds are more likely to identify potential biases and assumptions that others might miss. Correct. People's life experiences shape their perspectives and the assumptions they make. A development team with diverse backgrounds (e.g., gender, ethnicity, disability, socioeconomic status) is better equipped to spot how an innovation might fail or cause harm to different groups of users, leading to a more inclusive and less biased product. - A is incorrect. Diversity of background does not automatically equate to a diversity of technical skills. - B is incorrect. Team diversity is primarily about improving the quality and fairness of the product, not necessarily the speed of development. - D is incorrect. The size of a team is different from its diversity. A large, homogenous team may still suffer from collective blind spots.
3. ### Introduction Computing bias means a computer system produces unfair results for some people because of choices made by humans. In facial recognition, bias appears when a tool matches faces more accurately for some groups than others. For example, a city uses facial recognition to help identify suspects from security footage, but community reviews show the system flags more false matches for Black women than for white men. ### How Bias Emerges Bias emerges when the data used to build a system does not represent the full public. If the training photos include many light-skinned faces and fewer dark-skinned faces, the system "learns" patterns that work better for the overrepresented group. Bias also appears when designers choose goals like speed or convenience without checking who is harmed by errors. ### Societal and Personal Impacts Misidentification can lead to stressful police stops, wrongful questioning, or even arrest. People may lose trust in public safety agencies and in technology more broadly. Even when a mistake is corrected, the individual may face reputational damage, missed work, and fear of future encounters. ### Mitigation Strategies To reduce bias, agencies and companies test accuracy across demographic groups, expand and balance the photo data, and set strict rules for when facial recognition may be used. Some require a trained human reviewer and additional evidence before action is taken. Others publish audits and allow independent experts to evaluate results. ### Conclusion Facial recognition can support investigations, but it also risks unequal harm when errors cluster on specific groups. Fairness improves when systems are evaluated openly, updated with representative data, and limited by clear policies. According to the passage, how does computing bias affect Black women in facial recognition?
   Answer: It produces more false matches for them than for white men This question tests AP Computer Science Principles skills: understanding computing bias and its societal impact. Computing bias occurs when algorithms or data sets favor certain outcomes, often reflecting societal inequalities. In the passage, the impact of bias in facial recognition on Black women is highlighted, showing how it produces more false matches for them compared to white men. Choice A is correct because it accurately reflects the passage's specific example that the system flags more false matches for Black women than for white men. Choice B is incorrect because it contradicts the passage's point about unequal error rates and the existence of false matches. To help students: Encourage students to identify specific examples in passages that illustrate bias patterns, and discuss how underrepresentation in training data affects different groups unequally. Watch for: Answer choices that claim perfect accuracy or equal treatment when the passage explicitly describes disparate impacts.
4. Which of the following datasets is MOST likely to introduce bias into the algorithm?
   Answer: A dataset of historical housing program approvals from a single city known for past discriminatory housing practices. Correct. Historical data often contains the biases of the time period in which it was collected. Using data from a city with a known history of discrimination is highly likely to teach the algorithm to perpetuate those same biased patterns. - B, C, and D are incorrect because they describe methods specifically aimed at reducing bias by creating a more representative and fair dataset through synthetic generation, random sampling, or combining diverse sources.
5. ### Introduction Computing bias means a computer system produces unfair results for some people because of choices made by humans. In facial recognition, bias appears when a tool matches faces more accurately for some groups than others. For example, a city uses facial recognition to help identify suspects from security footage, but community reviews show the system flags more false matches for Black women than for white men. ### How Bias Emerges Bias emerges when the data used to build a system does not represent the full public. If the training photos include many light-skinned faces and fewer dark-skinned faces, the system "learns" patterns that work better for the overrepresented group. Bias also appears when designers choose goals like speed or convenience without checking who is harmed by errors. ### Societal and Personal Impacts Misidentification can lead to stressful police stops, wrongful questioning, or even arrest. People may lose trust in public safety agencies and in technology more broadly. Even when a mistake is corrected, the individual may face reputational damage, missed work, and fear of future encounters. ### Mitigation Strategies To reduce bias, agencies and companies test accuracy across demographic groups, expand and balance the photo data, and set strict rules for when facial recognition may be used. Some require a trained human reviewer and additional evidence before action is taken. Others publish audits and allow independent experts to evaluate results. ### Conclusion Facial recognition can support investigations, but it also risks unequal harm when errors cluster on specific groups. Fairness improves when systems are evaluated openly, updated with representative data, and limited by clear policies. In the context of the passage, how does bias influence the outcome of facial recognition use?
   Answer: It shifts errors toward groups underrepresented in the photo data This question tests AP Computer Science Principles skills: understanding computing bias and its societal impact. Computing bias occurs when algorithms or data sets favor certain outcomes, often reflecting societal inequalities. In the passage, the impact of bias in facial recognition is highlighted, showing how it shifts errors toward groups underrepresented in training data. Choice B is correct because it accurately reflects the passage's explanation that when training photos underrepresent certain groups, the system learns patterns that work better for overrepresented groups, shifting errors to underrepresented ones. Choice C is incorrect because it contradicts the passage's central point about unequal accuracy across demographic groups. To help students: Encourage analysis of how training data composition affects algorithm performance, and discuss the connection between data representation and outcome fairness. Watch for: Answer choices that claim equal outcomes when the passage explicitly describes unequal impacts.
6. What is the most likely consequence of using this system in a public setting with a diverse population?
   Answer: The system will be less accurate in identifying individuals from demographic groups that were underrepresented in the training data. Correct. The training data lacks diversity, which is a primary cause of data bias. The system will likely perform poorly for groups not well-represented in the training set, as it has not learned to identify their features effectively. - A is incorrect because the size of a dataset does not guarantee fairness or prevent bias; the representativeness of the data is critical. - C is incorrect because while camera quality can affect performance, the core issue of bias stems from the unrepresentative training data. - D is incorrect because while data privacy is a valid concern for facial recognition, the question asks about the consequence of the biased training method, which relates to accuracy and fairness.
7. What is the most likely consequence if this program is used on a broader patient population?
   Answer: The program's diagnostic accuracy will likely be lower for patients from ethnic backgrounds not included in the training data. Correct. The appearance of skin conditions can vary significantly across different skin tones. By training on data from only one group, the program has not learned to recognize these variations. This data bias will likely lead to reduced accuracy and potentially harmful misdiagnoses for underrepresented groups. - A is incorrect. Privacy restrictions are a legal and data management issue, not a direct consequence of the biased training method. - B is incorrect. The 'fundamental features' of a visual disease are often tied to how they appear on different skin types, so the model's learning is incomplete and biased. - C is incorrect. The program's speed of execution is generally not affected by the content of the image in this way; the issue is with the accuracy of its analysis.
8. Introduction: An airport adds facial recognition for boarding to speed up lines. Examples of Bias: Computing bias refers to unfair patterns in results produced by a computing system. Reports show some facial recognition tools misidentify travelers with darker skin tones at higher rates, often because the data used to build the tool includes fewer images of those travelers. How Bias Emerges: Bias can arise when a data set is not representative, when images are labeled inconsistently, or when testing focuses on only one group. Impacts: Misidentification can cause missed flights, extra screening, and embarrassment, which may fall more heavily on certain demographic groups. Mitigation Strategies: Airports can offer an opt-out option, measure error rates by group, and require staff to confirm identity before penalties. How does computing bias affect travelers with darker skin tones according to the passage?
   Answer: They are misidentified more often during boarding checks This question tests AP Computer Science Principles skills: understanding computing bias and its societal impact. Computing bias occurs when algorithms or data sets favor certain outcomes, often reflecting societal inequalities. In the passage, the impact of bias in airport facial recognition is highlighted, showing how some systems misidentify travelers with darker skin tones at higher rates. Choice A is correct because it accurately reflects the passage's statement that these travelers experience higher misidentification rates during boarding checks, which can cause missed flights and extra screening. Choice B is incorrect because it contradicts the passage by suggesting these travelers have advantages rather than disadvantages. To help students: Emphasize careful reading to identify specific groups affected by bias and how they're impacted. Watch for: Answer choices that reverse or misrepresent the actual effects of bias described in the passage.
9. A machine learning model exhibits bias against a certain demographic group. A developer decides to address this by collecting a much larger dataset. Which of the following best explains why this action might NOT eliminate the bias?
   Answer: If the method of data collection is still flawed, the new, larger dataset will likely contain the same underlying bias. Correct. According to EK DAT-2.C.5, bias is not eliminated by simply collecting more data. If the source or method of collection is biased, a larger sample will just amplify that original bias. For bias to be reduced, the data must be more representative, not just more plentiful. - A is incorrect. This is not a generally true statement about machine learning; more data typically improves accuracy if the data is of good quality. - B is incorrect. While budget is a practical concern, it doesn't explain why the approach is fundamentally flawed from a technical perspective. - C is incorrect. Storage and scalability are system design concerns, not the reason why the bias itself would persist.
10. This situation primarily demonstrates which of the following concerns in computing?
    Answer: Computing bias, where an algorithm perpetuates past human biases found in the training data. Correct. The hiring tool learned from historical data that reflected a past bias (a predominantly male workforce). The algorithm is likely to identify patterns associated with past successful (male) candidates and unfairly penalize candidates who do not fit that historical pattern, thus perpetuating the bias. - A is incorrect. The issue is about the fairness of the algorithm's output, not its security. - B is incorrect. The algorithm's efficiency is a separate concern from its fairness. - C is incorrect. The digital divide relates to access to technology, whereas this problem is about the biased processing of information by the technology itself.
11. Which of the following provides the best definition of algorithmic bias?
    Answer: Systematic and repeatable errors in a computer system that create unfair outcomes for certain groups of people. Correct. This definition captures the key elements of algorithmic bias: it is systematic (not random), repeatable, and results in unfairness or prejudice against specific subgroups. - A describes a buggy or unreliable program, not a biased one. - B describes a syntax error, which is a coding mistake that a compiler or interpreter would catch. - D describes a security vulnerability, which is distinct from the concept of fairness in a system's outcomes.
12. ### Introduction Computing bias means a computer system produces unfair results for some people because of choices made by humans. In facial recognition, bias appears when a tool matches faces more accurately for some groups than others. For example, a city uses facial recognition to help identify suspects from security footage, but community reviews show the system flags more false matches for Black women than for white men. ### How Bias Emerges Bias emerges when the data used to build a system does not represent the full public. If the training photos include many light-skinned faces and fewer dark-skinned faces, the system "learns" patterns that work better for the overrepresented group. Bias also appears when designers choose goals like speed or convenience without checking who is harmed by errors. ### Societal and Personal Impacts Misidentification can lead to stressful police stops, wrongful questioning, or even arrest. People may lose trust in public safety agencies and in technology more broadly. Even when a mistake is corrected, the individual may face reputational damage, missed work, and fear of future encounters. ### Mitigation Strategies To reduce bias, agencies and companies test accuracy across demographic groups, expand and balance the photo data, and set strict rules for when facial recognition may be used. Some require a trained human reviewer and additional evidence before action is taken. Others publish audits and allow independent experts to evaluate results. ### Conclusion Facial recognition can support investigations, but it also risks unequal harm when errors cluster on specific groups. Fairness improves when systems are evaluated openly, updated with representative data, and limited by clear policies. What is a consequence of computing bias as described in the text?
    Answer: Stressful stops or wrongful questioning after misidentification This question tests AP Computer Science Principles skills: understanding computing bias and its societal impact. Computing bias occurs when algorithms or data sets favor certain outcomes, often reflecting societal inequalities. In the passage, the impact of bias in facial recognition is highlighted, showing how it leads to misidentification with serious personal consequences. Choice C is correct because it accurately reflects the passage's explanation of bias consequences including stressful police stops and wrongful questioning after misidentification. Choice A is incorrect because the passage actually states people lose trust in public safety agencies, not gain it. To help students: Encourage careful reading to identify specific consequences mentioned in the text, and discuss how technological errors can have real-world impacts on individuals. Watch for: Answer choices that present opposite effects or benefits when the passage describes harms.
13. Introduction: A university uses facial recognition to take attendance in large lecture halls. Examples of Bias: Computing bias is a pattern of unfair outcomes produced by a system, often tied to the data it learns from and the choices people make when building it. Some facial recognition systems show higher error rates for women and for students with darker skin tones when those groups are underrepresented in the training images. How Bias Emerges: Bias can come from limited photo collections, poor image quality, and testing that does not reflect the student body. Impacts: Students may be marked absent incorrectly, which can affect grades and increase conflict with instructors. Mitigation Strategies: The university can use multiple attendance options, audit errors by group, and allow quick correction without penalties. According to the passage, what is a consequence of computing bias as described in the text?
    Answer: It can cause some students to be marked absent incorrectly This question tests AP Computer Science Principles skills: understanding computing bias and its societal impact. Computing bias occurs when algorithms or data sets favor certain outcomes, often reflecting societal inequalities. In the passage, the impact of bias in university attendance systems is highlighted, showing how facial recognition can have higher error rates for women and students with darker skin tones. Choice A is correct because it accurately reflects the passage's statement that 'Students may be marked absent incorrectly, which can affect grades and increase conflict with instructors.' Choice B is incorrect because it contradicts the passage's discussion of errors and bias. To help students: Emphasize identifying specific consequences mentioned in the text rather than idealized outcomes. Watch for: Answer choices that promise perfection or error-free performance when the passage discusses bias and errors.
14. Introduction: A state motor vehicle office uses facial recognition to detect identity fraud when people apply for driver's licenses. Examples of Bias: Computing bias refers to unfair patterns in outcomes produced by a system, often linked to the data and assumptions behind it. Facial recognition bias can produce more false matches for certain demographic groups when the system is trained on images that do not represent the full population. How Bias Emerges: Bias can come from uneven data, earlier record-keeping practices, and limited testing across groups. Impacts: False matches can delay licenses, trigger investigations, or create lasting records that are hard to correct. Mitigation Strategies: The office can require human review, allow easy appeals, and publish regular reports on error rates by group. Based on the text, which strategy is used to mitigate computing bias?
    Answer: Publish regular reports on error rates by group This question tests AP Computer Science Principles skills: understanding computing bias and its societal impact. Computing bias occurs when algorithms or data sets favor certain outcomes, often reflecting societal inequalities. In the passage, the impact of bias in motor vehicle office facial recognition is highlighted, showing how false matches can delay licenses or trigger investigations. Choice A is correct because it accurately reflects the passage's specific mitigation strategy to 'publish regular reports on error rates by group,' which helps identify and address bias patterns. Choice B is incorrect because it contradicts the passage's emphasis on human review and appeals. To help students: Emphasize identifying mitigation strategies that increase transparency and accountability. Watch for: Options that suggest removing oversight or making systems less reviewable.
15. Introduction: A sports stadium uses facial recognition to identify banned attendees at entry gates. Examples of Bias: Computing bias means a system's results are uneven in ways that can disadvantage certain groups, often because the system reflects the data it learns from. Facial recognition bias can lead to higher misidentification rates for Black attendees and women when the training images underrepresent them. How Bias Emerges: Bias can arise from unbalanced photo databases, poor image quality, and policies that treat automated flags as proof. Impacts: False flags can lead to denied entry, public confrontation, or unnecessary involvement of security. Mitigation Strategies: The stadium can use human confirmation, track false flags by demographic group, and limit consequences until identity is verified. How does computing bias affect Black attendees and women according to the passage?
    Answer: They are misidentified at higher rates than others This question tests AP Computer Science Principles skills: understanding computing bias and its societal impact. Computing bias occurs when algorithms or data sets favor certain outcomes, often reflecting societal inequalities. In the passage, the impact of bias in stadium facial recognition is highlighted, showing how the system can disadvantage certain demographic groups. Choice A is correct because it accurately reflects the passage's statement that 'Facial recognition bias can lead to higher misidentification rates for Black attendees and women when the training images underrepresent them.' Choice C is incorrect because it contradicts the passage's entire discussion of how bias creates uneven outcomes. To help students: Emphasize careful reading to identify which groups are specifically mentioned as experiencing higher error rates. Watch for: Options that claim systems are neutral when the passage explicitly discusses bias and uneven outcomes.
16. What is the primary source of the algorithm's bias?
    Answer: The historical data used to train the algorithm contained and reflected past societal biases. Correct. The algorithm learned patterns from data that was generated during a time of discriminatory practices. It is therefore perpetuating this historical bias by treating new applications in a similar way, even if the discriminatory policies are no longer in place. The data itself is the source of the bias. - A is incorrect. Complexity and transparency are issues, but the root cause of the bias is the data. - C is incorrect. This response blames the victims of the bias rather than identifying the flaw in the system. - D is incorrect. Security is an important but separate concern from algorithmic fairness.
17. What is the most plausible technical explanation for this bias?
    Answer: The algorithm was trained on a massive text dataset from the real world, which contains statistical biases about gender and professions. Correct. Language models learn from the patterns in the text they are trained on. If the training data (e.g., books, articles, websites) more frequently associates "doctor" with male pronouns and "nurse" with female pronouns, the model will learn and replicate this statistical correlation. This is a classic example of a computing innovation reflecting biases present in its training data. - A is incorrect. The bias is in the core translation logic, not the user interface. - B is incorrect. This is not a syntax error, but a logical outcome based on the model's training. - C is incorrect. This is a software and data issue, not a hardware limitation.
18. ### Introduction Computing bias means a computer system produces unfair results for some people because of choices made by humans. In facial recognition, bias appears when a tool matches faces more accurately for some groups than others. For example, a city uses facial recognition to help identify suspects from security footage, but community reviews show the system flags more false matches for Black women than for white men. ### How Bias Emerges Bias emerges when the data used to build a system does not represent the full public. If the training photos include many light-skinned faces and fewer dark-skinned faces, the system "learns" patterns that work better for the overrepresented group. Bias also appears when designers choose goals like speed or convenience without checking who is harmed by errors. ### Societal and Personal Impacts Misidentification can lead to stressful police stops, wrongful questioning, or even arrest. People may lose trust in public safety agencies and in technology more broadly. Even when a mistake is corrected, the individual may face reputational damage, missed work, and fear of future encounters. ### Mitigation Strategies To reduce bias, agencies and companies test accuracy across demographic groups, expand and balance the photo data, and set strict rules for when facial recognition may be used. Some require a trained human reviewer and additional evidence before action is taken. Others publish audits and allow independent experts to evaluate results. ### Conclusion Facial recognition can support investigations, but it also risks unequal harm when errors cluster on specific groups. Fairness improves when systems are evaluated openly, updated with representative data, and limited by clear policies. According to the passage, what is one reason facial recognition bias emerges?
    Answer: Training photos overrepresent some groups and underrepresent others This question tests AP Computer Science Principles skills: understanding computing bias and its societal impact. Computing bias occurs when algorithms or data sets favor certain outcomes, often reflecting societal inequalities. In the passage, the emergence of bias in facial recognition is highlighted, showing how it stems from unrepresentative training data. Choice A is correct because it accurately reflects the passage's explanation that bias emerges when training photos include many light-skinned faces and fewer dark-skinned faces, creating overrepresentation and underrepresentation. Choice B is incorrect because it attributes bias to hardware issues rather than data representation problems. To help students: Encourage understanding of how training data composition directly affects algorithm performance, and discuss the importance of representative datasets. Watch for: Answer choices that blame hardware or speed when the passage focuses on data representation issues.
19. Which of the following describes the most significant ethical concern related to computing bias in this scenario?
    Answer: The historical arrest data may reflect biases in policing, leading the program to unfairly target certain neighborhoods and create a feedback loop. Correct. This highlights a critical societal impact of computing bias. If historical data reflects that certain neighborhoods were policed more heavily (regardless of actual crime rates), the algorithm will learn to identify these areas as high-risk. This can lead to increased police presence, more arrests, and a reinforcement of the original bias in future data. - A is incorrect. Data privacy is a valid ethical concern, but the question is specifically about bias in the program's function. - B is incorrect. Inaccuracy is a general problem for any predictive model, but the key ethical concern here is the systematic and unfair nature of the inaccuracy due to bias. - D is incorrect. User interface design is a usability issue, not the core ethical problem of systemic bias.
20. This phenomenon is a direct illustration of which of the following?
    Answer: Algorithmic bias Correct. The algorithm is producing systematically different and unfair outcomes for different demographic groups (male vs. female users). This is a clear and direct example of algorithmic bias, likely learned from biased historical data about job applications or societal roles. - A is incorrect. Symmetric key encryption is a cryptographic technique for securing data. - C is incorrect. Fault tolerance is a principle of network design that ensures reliability. - D is incorrect. Crowdsourcing is a method of gathering information or work from a large group of people.

Start this quiz