Understanding Aggregation Bias
Machine learning plays a more significant role in shaping decisions that directly affect people’s lives, from determining job opportunities to influencing law enforcement practices. These algorithms learn patterns from past data and apply them to new situations. However, when there are flaws in the data or the way the model is built, it can lead to unintended and sometimes harmful consequences. We have already seen examples of this in areas like predictive policing and facial recognition.
One common issue is aggregation bias. This happens when we try to use a single, “one-size-fits-all” model for data that includes distinct groups, each with its own unique characteristics. By not accounting for these differences, the model can give inaccurate or unfair results, often disadvantaging certain groups (Suresh & Guttag, 2019).
Types of Aggregation Bias
According to Walker & Catrambone (1993), there are three types of aggregation bias:
- Aggregation over Trials
This aggregation type combines multiple observations of the same task into a single average value. For example, if you are measuring reaction times across several trials, averaging these times helps reduce the influence of random factors like attention lapses or learning effects during individual trials. The goal is to remove these sources of variability (known as error variance) to create a result that better reflects the actual performance. While this method reduces noise, it can also overlook meaningful differences between trials.
2. Aggregation over Subjects
In this case, data is averaged across multiple individuals to eliminate the effects of individual differences, such as varying abilities or skill levels. Using the group’s average performance instead of individual scores, the model simplifies the data and reduces variance related to personal characteristics. This approach assumes that individual differences do not interact with the studied variables, but it risks losing valuable insights about how different people respond to the same treatment or task.
3. Aggregation over Predictor Variable Values
This occurs when data is grouped into broader categories of a predictor variable. For instance, if you are studying how household income affects spending on transportation, incomes might be grouped into ranges (e.g., $10,000–$20,000). By doing this, variability directly tied to individual income levels is smoothed out. While this simplifies the analysis, it also removes potentially important details about how the predictor variable (e.g., income) influences the outcome.
Each aggregation type has benefits, such as reducing noise, but it also comes with trade-offs, potentially obscuring meaningful patterns or introducing bias. It is important to use these techniques thoughtfully, keeping the context of the data and goals in mind.
Example of Aggregation Bias in Machine Learning
Patton et al. [38] studied tweets from gang-involved youth in Chicago, uncovering the importance of understanding the local and cultural context behind the data. By bringing in domain experts from the community to interpret and annotate tweets, they highlighted how general NLP tools often need to be revised.
For instance, certain emojis or hashtags used by this group carried meanings that a broad, non-specific model trained on general Twitter data would completely miss. Even more striking, phrases or words that might seem aggressive in other contexts were lyrics from a local rapper, reflecting cultural expression rather than conflict [19].
Without accounting for this context and instead relying on a “one-size-fits-all” model for all social media data, these tweets could easily be misclassified, leading to harmful or unfair conclusions about this population. This example underscores the critical need to design models that respect the nuances and unique characteristics of the communities they aim to serve (Suresh & Guttag, 2021).
Effects of Aggregation Bias on Machine Learning
Aggregation bias can have far-reaching and deeply personal consequences, especially when machine learning models fail to account for the diversity within the data they are built upon. When a model is optimised for the dominant group or tries to serve everyone with a “one-size-fits-all” approach, it often ends up being suboptimal for all groups. This is particularly problematic when representation bias is also at play, further compounding the issue.
Think of model training as a simplification process—compressing complex relationships into patterns that the algorithm can use. While this is necessary, it also means that important nuances in the data can be lost, especially for underrepresented or minority groups. When these differences are oversimplified, the model may fail to make accurate predictions or provide fair outcomes for these groups. This is what’s known as “quality-of-service harm,” where minority groups are left with less accurate, less fair, or even harmful results.
The impact of aggregation bias doesn’t stop there. It can lead to the misallocation of critical resources, the creation of policies that miss the mark, and the further marginalisation of communities that are already at a disadvantage. Addressing this bias isn’t just about improving model performance—it’s about ensuring fairness, equity, and inclusivity in how AI systems impact real lives.
Designing Mitigation for Aggregation Bias in Machine Learning
Aggregation bias arises when conclusions are generalised from observations of a larger group, overlooking the unique characteristics of smaller subgroups within that population. This results in datasets that need to accurately represent these subgroups’ specific traits and needs, leading to potential inaccuracies and inequities in outcomes. Addressing this issue requires a holistic approach considering the diversity and context of all subgroups in the data. Here are some strategies in the AI/ML you can adopt to mitigate aggregation bias:
Tackling aggregation bias requires a systematic, proactive approach. You can get started with these resources:
Free Resources for Aggregation Bias Mitigation
AI Bias Mitigation Package – £999
The ultimate resource for organisations ready to tackle bias at scale starting from problem definition through to model monitoring to drive responsible AI practices.
Mitigate and resolve 15 Types of Bias specific to your project with detailed guidance from problem definition to model monitoring. Packed with practical methods, research-based strategies, and critical questions to guide your team. Comprehensive checklists with +75 design cards for every phase in the AI/ ML pipelineCustomised AI Bias Mitigation Package – £2499
Mitigate and resolve Mitigate and resolve 15 Types of Bias specific to your project with detailed guidance from problem definition to model monitoring. Packed with practical methods, research-based strategies, and critical questions specific to your use case. Customised checklists and +75 design cards for every phase in the AI/ ML pipeline
Source
Chen, C.F., Napolitano, R., Hu, Y., Kar, B. and Yao, B., 2024. Addressing machine learning bias to foster energy justice. Energy Research & Social Science, 116, p.103653.
Chu, C., Donato-Woodger, S., Khan, S.S., Shi, T., Leslie, K., Abbasgholizadeh-Rahimi, S., Nyrup, R. and Grenier, A., 2024. Strategies to Mitigate Age-Related Bias in Machine Learning: Scoping Review. JMIR aging, 7, p.e53564.
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K. and Galstyan, A., 2021. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6), pp.1-35.
Fahse, T., Huber, V. and van Giffen, B., 2021. Managing bias in machine learning projects. In Innovation Through Information Systems: Volume II: A Collection of Latest Research on Technology Issues (pp. 94-109). Springer International Publishing.
Suresh, H. and Guttag, J.V., 2019. A framework for understanding unintended consequences of machine learning. arXiv preprint arXiv:1901.10002, 2(8), p.73.
Suresh, H. and Guttag, J., 2021, October. A framework for understanding sources of harm throughout the machine learning life cycle. In Proceedings of the 1st ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (pp. 1-9).
Walker, N. and Catrambone, R., 1993. Aggregation bias and the use of regression in evaluating models of human performance. Human factors, 35(3), pp.397-411.
Weerts, H.J., 2021. An introduction to algorithmic fairness. arXiv preprint arXiv:2105.05595.