The Ripple Effect Trap in Machine Learning

Understanding Ripple Effects

The ripple effect occurs when there is a failure to understand how machine learning (ML) solutions can unintentionally impact various interconnected elements. Experience shows that making software changes without fully understanding their effects on pre-existing software or AI/ML systems, people, organizations, or social structures can result in numerous challenges, including poor effort estimates, delays in release schedules, degraded software design, unreliable software products, and even the premature retirement of the software system.

This phenomenon often starts with a seemingly minor change that ripples throughout the system, leading to major unintended consequences in:

• Pre-existing AI/ML systems or other systems: Affecting behaviours, values, or interactions.
• People and organizations: Disrupting workflows or altering societal norms.
• Social structures: Impacting behaviours, priorities, or embedded systemic values.

Example for the Ripple Effect in Machine Learning

Example 1 highlights how social structures, behaviours, or priorities embedded within systems can be unintentionally impacted. 

Example 1: “While automated vehicles (AVs) are known to affect traffic, road, and even infrastructure design, most technical research has focused on incorporating these as features to be modelled rather than questioning the status of AVs as the dominant form of future mobility. Engagement across the entire sociotechnical stack requires understanding social phenomena like the “reinforcement politics” of dominant groups using technology to remain in power and “reactivity” like gaming and adversarial behavior“ Andrus et al (2020).

The example reflects the failure to consider how AVs interact with and reshape social phenomena beyond their technical implementation. Example 2 shows how people can be unintentionally impacted, particularly through changes in opinions and societal norms. 

Example 2: “If LLMs produce content disproportionately, say preferring one political opinion over another, it would be a matter of concern to what extent they may influence people’s opinions. Jakesch et al. (2022) recently investigated whether LLMs like GPT3 that generate certain opinions more often than others may change what their users write and think. The authors found that interactions with opinionated language models changed users’ opinions systematically, and unintentionally” Dhole (2023).

These examples underscore the need for proactive engagement with ML’s ripple effects.

Guide to Addressing the Ripple Effect Trap in ML

Beyond technical optimisation, it is critical to address societal impacts, ensure diverse stakeholder involvement, and implement safeguards to prevent unintended consequences like bias amplification, power reinforcement, or adversarial behaviours.

Eliminating unintended consequences may not be possible, but you can minimise the risks by addressing critical decisions during a technology’s development.

Tackling representation bias requires a systematic, proactive approach. You can get started with these resources:  

Free Resources for mitigating The Ripple Effect Trap

Best practices and design considerations for mitigating The Ripple Effect Trap (click Free Downloads).

 

AI Bias Mitigation Package – £999

The ultimate resource for organisations ready to tackle bias at scale starting from problem definition through to model monitoring to drive responsible AI practices.

Mitigate and resolve 15 Types of Bias specific to your project with detailed guidance from problem definition to model monitoring.

Packed with practical methods, research-based strategies, and critical questions to guide your team.

Comprehensive checklists with +75 design cards for every phase in the AI/ ML pipeline

Get Bias Mitigation Package– (Delivery within 2-3 days)

Customised AI Bias Mitigation Package – £2499

We’ll customise the design cards and checklists to meet your specific use case and compliance requirements—ensuring the toolkit aligns perfectly with your goals and industry standards.

Mitigate and resolve 15 Types of Bias specific to your project with detailed guidance from problem definition to model monitoring.

Packed with practical methods, research-based strategies, and critical questions specific to your use case.

Customised checklists and +75 design cards for every phase in the AI/ ML pipeline

Get Customised AI Bias Mitigation Package– (Delivery within 7 days)

Sources

Andrus, M., Dean, S., Gilbert, T.K., Lambert, N. and Zick, T., 2020, November. AI development for the public interest: From abstraction traps to sociotechnical risks. In 2020 IEEE International Symposium on Technology and Society (ISTAS) (pp. 72-79). IEEE.

Bhat, A., Coursey, A., Hu, G., Li, S., Nahar, N., Zhou, S., Kästner, C. and Guo, J.L., 2023, April. Aspirations and practice of ml model documentation: Moving the needle with nudging and traceability. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (pp. 1-17).

Bilal, H. and Black, S., 2005, March. Using the ripple effect to measure software quality. In SOFTWARE QUALITY MANAGEMENT-INTERNATIONAL CONFERENCE- (Vol. 13, p. 183).

Chancellor, S., 2023. Toward practices for human-centered machine learning. Communications of the ACM, 66(3), pp.78-85.

Dhole, K., 2023, December. Large language models as SocioTechnical systems. In Proceedings of the Big Picture Workshop (pp. 66-79).

Hoel, T., Chen, W. and Pawlowski, J.M., 2020. Making context the central concept in privacy engineering. Research and Practice in Technology Enhanced Learning, 15(1), p.21.

Khlaaf, H., 2023. Toward comprehensive risk assessments and assurance of ai-based systems. Trail of Bits, 7.

Li, M., Wang, W. and Zhou, K., 2021. Exploring the technology emergence related to artificial intelligence: A perspective of coupling analyses. Technological Forecasting and Social Change, 172, p.121064.

Mordaschew, V., Herrmann, J.P. and Tackenberg, S., 2023. Methods of change impact analysis for product development: A systematic review of the literature. Proceedings of the Design Society, 3, pp.2655-2664.

Selbst, A.D., Boyd, D., Friedler, S.A., Venkatasubramanian, S. and Vertesi, J., 2019, January. Fairness and abstraction in sociotechnical systems. In Proceedings of the conference on fairness, accountability, and transparency (pp. 59-68).

Weerts, H.J., 2021. An introduction to algorithmic fairness. arXiv preprint arXiv:2105.05595.

Wexler, J., Pushkarna, M., Bolukbasi, T., Wattenberg, M., Viégas, F. and Wilson, J., 2019. The what-if tool: Interactive probing of machine learning models. IEEE transactions on visualization and computer graphics, 26(1), pp.56-65.