Responsible AI: The Business Analyst's Role in Ethics and Governance

Disclaimer: The views and opinions expressed in this article are those of the author and may not reflect the perspectives of IIBA.

This article is part of AI Wednesdays, an ongoing 2026 series that explores how business analysis professionals are using AI in real, practical ways. Each article is written by a practitioner and shares experience-based insights to help you use AI with greater confidence—starting small, building familiarity over time, and applying it where it adds real value. 

I was in a stakeholder workshop last year when a senior product manager said something that stuck with me.

We were reviewing requirements for an AI-assisted decisioning tool, and he said, "We'll figure out the governance stuff later. Right now we need to get the model working." That sentence captures the problem with how most organizations approach responsible AI.

Governance gets treated as a finishing touch, something to bolt on before go-live. By that point, the fundamental design decisions have already been made, the data has already been selected, and the model is already producing outputs that nobody has tested for fairness.

Every few weeks, another headline appears about an AI system that made a biased hiring decision, denied a loan to the wrong demographic, or generated content that crossed a legal line. These failures rarely stem from bad intentions. They stem from a gap between what the AI can do and what the organization has bothered to specify about how it should behave.

PwC's 2025 Responsible AI survey puts some numbers around this: 60% of executives say responsible AI boosts ROI and efficiency, while 55% reported an improved customer experience.

Yet nearly half of those same respondents admitted that turning responsible AI principles into actual operational processes has been an ongoing struggle. The will is there. The execution is not.

For business analysis professionals, this should feel like familiar territory. We’ve spent our careers translating ambiguous goals into structured, testable, deliverable requirements. We sit between business stakeholders who want outcomes and technical teams who build solutions. We run workshops, manage competing priorities, and document what good looks like.

These are precisely the skills that responsible AI demands. The trouble is that most governance frameworks are written by lawyers, risk professionals, or data scientists. They rarely speak the language of user stories, acceptance criteria, or sprint backlogs.

That’s the gap I want to address in this article.

Why This Matters Now

Responsible AI has moved well beyond ethical handwringing. It’s becoming a legal, financial, and operational reality.

The EU AI Act, the world's first comprehensive AI legislation, becomes fully enforceable for most obligations by August 2026. It classifies AI systems by risk level and imposes strict requirements for high-risk applications across financial services, healthcare, employment, education, and public services. These requirements cover risk management, data governance, transparency, human oversight, robustness, and detailed technical documentation.

The fines are serious: up to 35 million euros or 7% of global annual turnover, whichever is higher.

Europe isn’t acting alone. Colorado's AI Act took effect in February, targeting high-risk AI in employment, housing, and financial services decisions. New York City already mandates annual bias audits for automated hiring tools. California signed the Transparency in Frontier Artificial Intelligence Act in September 2025, imposing disclosure requirements on generative AI systems.

As the regulatory patchwork grows, organizations operating across borders can’t afford to treat compliance as a regional concern.

And then there’s the less quantifiable but equally real business case. Organizations known for responsible AI practices consistently report higher customer retention and stronger brand trust. Those that cut corners face regulatory action, reputational damage, and the kind of operational disruption that comes when you have to pull a system out of production because nobody thought carefully enough about what it was doing.

I’ve seen this play out firsthand in financial services, where a poorly governed model doesn’t just create a PR problem. It creates a regulatory investigation.

Why Business Analysts Are Better Positioned Than They Think

Here’s something I find most business analysis professionals underestimate: our existing skill set maps almost perfectly onto the requirements of responsible AI governance. We just haven’t been framing it that way.

Think about elicitation and requirements definition. When a regulator asks an organization, "How do you ensure this system does not discriminate?", the answer should trace back to a requirement that a business analysis professional documented, validated with stakeholders, and placed in the backlog. Defining ethical boundaries, fairness criteria, and transparency requirements is requirements work.

It just happens to be requirements work that most teams haven’t yet recognized as part of the analyst’s scope.

Think about stakeholder management. Responsible AI involves navigating genuinely competing interests. Legal teams worry about compliance exposure. Data science teams focus on model performance metrics. Business sponsors want speed to market. End users and affected customers want fairness and trust.

Holding these perspectives together and finding a workable path forward is what business analysis professionals do every day. The context may be new, but the skill isn’t.

Think about process modelling. A well-drawn process model that shows where AI decisions happen, who is affected, what human oversight exists, and where escalation occurs is already a governance artefact. Add decision points for review thresholds and audit checkpoints, and you have something a compliance officer can use.

Think about traceability and documentation. The EU AI Act requires comprehensive technical documentation, design history, and evidence of testing. Business analysis professionals already build and maintain traceable requirements that link business objectives to delivered solution components. That discipline translates directly.

The point is straightforward. Responsible AI doesn’t require business analysis professionals to retrain as ethicists or data scientists. It requires us to apply the rigour we already bring to a domain that desperately needs it.

Five Practical Responsibilities for the Responsible AI Business Analyst

1. Lead the AI Impact Assessment

Before any AI system reaches development, someone needs to think carefully about who it affects and what could go wrong. This is natural business analysis territory.

An AI impact assessment should explore who is affected by the system's decisions, including customers, employees, job applicants, or the general public. It should consider what happens if the system produces incorrect or biased outputs and how severe the consequences might be. It should examine whether the system processes sensitive data or makes decisions in regulated domains. It should also evaluate what existing protections or controls are already in place and where the organization's risk appetite sits for this type of decision.

I’ve found that the most effective way to run this is as a structured workshop with business owners, legal, risk, and technology stakeholders in the room together. When these groups only see each other's input through documents, important tensions get smoothed over or missed entirely. Face-to-face facilitation surfaces the real disagreements. 

The output becomes a foundational artefact that shapes requirements, test plans, and monitoring. And it needs to happen before the team has committed to a solution, not after the model has been trained and everyone is emotionally invested in launching it.

2. Define Fairness and Bias Requirements

Bias in AI typically originates from historical data that reflects real-world inequalities. A credit scoring model trained on decades of lending decisions may perpetuate patterns of discrimination that existed in those decisions. A recruitment tool trained on past hiring data may systematically disadvantage candidates from underrepresented groups, not because anyone designed it to, but because that’s what the data shows.

Business analysis professionals can address this by writing explicit fairness requirements. These should specify which protected characteristics, such as gender, ethnicity, age, or disability status, must be tested for differential treatment. They should establish what level of performance disparity across groups is acceptable and what triggers a formal review. They should require that training data is assessed for representativeness and historical bias before the model is built. And they should mandate ongoing monitoring of model outputs to detect emerging disparities after deployment, because a model that’s fair at launch can drift over time as the data changes.

Without these requirements, "be fair" remains a principle with no accountability. With them, data science teams have clear, testable criteria. That’s the difference a business analysis professional makes.

3. Specify Transparency and Explainability

Under the EU AI Act, users must be informed when they interact with AI, they must understand the system's capabilities and limitations, and decisions must be explainable. For business analysis professionals, this translates into concrete, implementable requirements.

When a system interacts directly with a person, your requirements should specify how and when that person is told AI is involved. When a system makes or influences a consequential decision, you need to define what explanation is provided, to whom, in what format, and at what level of detail. When a system generates content, you should specify how that content is labelled. When a decision is challenged or a customer complains, you need to define what audit trail exists so the organization can reconstruct how the decision was reached.

I want to stress that explainability requirements aren’t just regulatory paperwork. They’re a trust mechanism. When a customer can understand why their application was declined, or when a regulator can trace the logic of an automated decision, trust increases. When outputs feel like a black box, trust erodes.

As business analysis professionals, we should think of transparency requirements the way we think about any good user experience: the goal is to give people the information they need, when they need it, in a way they can actually use.

4. Design Human Oversight Into the Process

One of the most important governance principles is that humans remain meaningfully in the loop for consequential AI decisions. I want to emphasise the word "meaningfully." Rubber-stamping an AI recommendation that arrives pre-approved with a green tick is not genuine oversight. The human reviewer must have sufficient information, genuine authority, and realistic time to review, question, and override the system.

This means your requirements need to specify which decisions require human review and at what threshold, what information the reviewer receives alongside the AI recommendation, how the reviewer overrides or modifies the output and what record is kept when they do, and what training reviewers need to exercise effective judgement rather than simply deferring to the machine.

This is fundamentally process design work. It requires understanding the decision context, the cognitive demands on the reviewer, the time pressures of the workflow, and the practical limits of both the AI system and the humans working alongside it. In my experience, the biggest risk is designing oversight that looks good on a process diagram but collapses under real operational conditions because the reviewer is handling 200 cases a day and can’t meaningfully assess each one.

Business analysis professionals need to design for the reality of the operational environment, not the ideal.

5. Make Governance Visible in the Backlog

This is possibly the single most impactful thing a business analysis professional can do. If bias testing, audit logging, transparency notices, and human review workflows don’t appear as backlog items, they won’t get built. They’ll be deferred when delivery pressure mounts, descoped when timelines slip, or quietly forgotten until something goes wrong.

Governance requirements should be written with the same rigour as functional requirements. They need clear acceptance criteria, sprint assignments, and validation during testing. A responsible AI checklist can help ensure every AI feature includes a bias and fairness test plan, transparency and disclosure requirements, human oversight design with escalation paths, audit and logging requirements, an incident response procedure, and a post-deployment monitoring plan with a defined review cadence.

When governance is visible in the backlog, it becomes part of the team's definition of done. When it lives in a separate policy document that nobody reads during sprint planning, it becomes what one governance practitioner memorably described as "governance theatre": the appearance of responsibility without the substance.

Common Pitfalls

Working across financial services organizations, I’ve watched several patterns undermine responsible AI efforts repeatedly.

The first is treating governance as a one-off exercise. Teams complete a risk assessment before launch, tick the box, and move on. But AI systems drift. Data distributions change. User behaviour evolves. Upstream systems get modified.

A model that was fair and accurate at launch can become problematic six months later without anyone noticing. Your requirements should include post-deployment monitoring with specific triggers for reassessment.

The second is assuming that fairness is purely a data science problem. Data scientists can measure statistical fairness metrics, but defining what fairness actually means in a specific business context requires business judgment and stakeholder input.

Should a lending model aim for equal approval rates across demographic groups, or equal accuracy? Those are different things, and the choice has real consequences for real people. That’s a business decision, not a technical one, and business analysis professionals should be facilitating it.

The third is confusing compliance with genuine responsibility. An organization can meet the minimum requirements of the EU AI Act while still deploying a system that its customers experience as unfair or harmful. Legal compliance sets a floor, not a ceiling.

What your brand promises, what your values claim, and what your stakeholders expect should set a higher bar. Business analysis professionals can help organizations think beyond the regulatory minimum.

The fourth is governance without authority. I’ve seen organizations publish beautifully written AI ethics policies and appoint governance committees that meet quarterly, and none of it matters because the committee has no power to delay or modify a deployment. If governance decisions can’t be reflected in the backlog with real authority, they’re performative. Business analysis professionals can help by ensuring governance findings translate into actual requirement changes, not just meeting minutes.

Questions for Your Next AI Project

If you’re working on an AI initiative right now, here are questions to put to your team. If you can’t answer them clearly, you have work to do.

• Who is affected by this system's decisions, and have we assessed the potential impact on them? 
• Have we defined explicit fairness requirements and tested for bias across protected characteristics? 
• Do affected users know they’re interacting with AI, and can they understand why a decision was made? 
• Is there meaningful human oversight for consequential decisions, with real authority to intervene? 
• Are governance requirements in the backlog with acceptance criteria and assigned ownership? 
• Do we have a plan for ongoing monitoring, incident response, and periodic reassessment after launch? 
• Could we demonstrate compliance to a regulator if they asked us tomorrow?

These are analytical, organizational, and human. They’re business analysis questions.

Conclusion: Governance Is Design Work

The organizations that get responsible AI right will be those that treat governance as a design discipline. They will embed fairness into requirements from the start, build transparency and oversight into their processes, test rigorously, and monitor outcomes continuously after launch.

Business analysis professionals are well-positioned to lead this work. We have the stakeholder relationships, the analytical frameworks, the documentation discipline, and the cross-functional credibility to turn principles into practice. The regulatory environment is tightening. The business case is clear. And the skills required sit at the core of what we have always done.

Responsible AI needs someone who asks the right questions, captures the answers with precision, and makes sure they get delivered. That’s our job. The question is whether we step up and claim it.

Join us on June 17 for the member webinar on AI governance—a practical next step for the analysts ready to lead.

---

About the Author