Imagine prompting a text-to-image generator for a simple “mugshot of criminals”, only for it to return a grid consisting entirely of Black and Latinx men.
To some, the image is a clear-cut case of structural bias and a perpetuation of a harmful societal stereotype. To others, looking at it through a purely mechanical lens, it might simply appear to fit the literal request. If you ask a single, centralized team of policy experts to grade that image, you will get one answer. But if you ask the people actually being targeted by that stereotype, you get a completely different story.
This is the core challenge of generative AI safety today. Global models are built and they are used by billions of people across thousands of distinct cultures, yet the safety guardrails are largely monolithic. Current AI alignment models rely on narrow sets of human feedback to define “safe” or “harmful”, ultimately treating human perspective as a monolith rather than a complex tapestry.
At the Nordic Data Science and Machine Learning (NDSML) Summit, Charvi Rastogi, a Research Scientist on the Responsible Research team at Google DeepMind, delivered a compelling talk addressing this exact crisis. She introduced a groundbreaking framework for shifting from single-lens safety to pluralistic alignment.
If you missed her session, here is a glimpse into why this shift is vital for the future of AI—and why you’ll want to watch the full presentation.
The Three Pillars of Pluralistic Alignment
True pluralistic alignment requires a systematic overhaul of the AI development pipeline. In her talk, Charvi breaks this down into three essential pillars:
[Disentangle] ──> [Understand] ──> [Integrate]
- Disentangling Perceptions: Actively capturing how different groups perceive harm, recognizing that image safety is deeply subjective.
- Understanding Divergence: Isolating where group viewpoints disagree and distinguishing meaningful cultural divergence from random noise.
- Integrating Scalably: Finding ways to inject these diverging perspectives directly into the machine learning pipeline without fracturing the model.
To put this framework to the test, the DeepMind team built DIVE (Demographically Intersectional Visual Evaluations) which is a massive, highly controlled text-to-image safety dataset that tracks how diverse human populations evaluate AI outputs.
Three Insights That Rewrite the AI Safety Structure
The findings from the DIVE dataset challenge some of the most common assumptions in current AI alignment:
1. High-Level Demographics Aren’t Enough
Evaluating bias solely across broad buckets like “gender” or “age” is a fundamental flaw, as demonstrated by the DIVE dataset. By tracking a Group Association Index (GAI), DeepMind discovered that intersectional groups are significantly more cohesive.
2. Experts Disagree with the Public on Bias
When DeepMind compared the ratings of professional policy experts against diverse crowd raters, they found high agreement on clear-cut violations like violence or sexually explicit content. However, on issues of bias and stereotyping, expert raters diverged drastically from the public. Traditional safety teams often lack the lived experience or cultural context required to spot a subtle, systemic stereotype that a targeted group notices immediately.
3. The Target Group Must Have the Final Say
When evaluating whether an image is harmful, the identity of the person being depicted matters immensely. The research highlighted fascinating nuances: sometimes a targeted group finds an image explicitly unsafe while outsiders miss the harm entirely. Other times, outsiders take offense on behalf of a group that actually finds the content benign. Alignment cannot succeed unless we explicitly account for who is in the crosshairs of the AI’s output.
Scalability: The Future of “Pluralistic Auto-Raters”
The biggest counter-argument to pluralistic alignment is always scale. How can an AI engineering team possibly consult thirty different intersectional demographic groups every time they update a model?
This is where the final act of Charvi’s presentation becomes very interesting, especially for ML practitioners. She reveals how the team leveraged LLMs (experimenting with Gemma and Gemini) as diverse judges. By prompting these models with specific demographic context and few-shot examples from the DIVE dataset, they created “pluralistic auto-rators”.
The results is a fascinating look at how steerable modern AI models actually are when it comes to emulating diverse human values, opening the door for scalable, multi-perspective safety testing before deployment.
The Future of Responsible AI
AI is expanding rapidly into every corner of human life, and with that power comes the responsibility to ensure these tools respect the diverse world they serve.
How accurately did Gemini 1.5 Flash mirror human demographic groups? What are the mathematical guardrails for defining “meaningful disagreement” versus noise?
The full talk Towards Pluralistic Alignment in GenAI Safety from Charvi Rastogi, a Research Scientist on the Responsible Research team at Google DeepMind from the NDSML Summit dives deep into the statistical architecture, the specific prompt-image edge cases, and the future of DeepMind’s responsible AI pipeline.
Gain access to the full NDSML video library and unlock the complete session. Secure your ticket for the NDSML summit on the official website.
