AI audit services

Q: What drives the price of an AI audit?

Four things move the number:how many systems fall inside the scope;whether our auditors reach the code and the training data or only the API;how many groups and segments the bias work covers;whether remediation sits inside the engagement.Access depth moves it most. An audit with the repository, the training data, and a staging environment reaches conclusions in the first week. An audit limited to the production API reaches fewer of them and needs more runs to get there.Most audits reach their first findings and a priced remediation list within 14 days of access being granted. When you want the range before committing, contact us and we will walk the four variables against your systems.

Q: Can you audit a system your own engineers built?

Yes for an internal audit or a technical review, and no for a statutory independent bias audit. NYC Local Law 144 bars an auditor who developed, tested, or trained the tool, and bars anyone with a direct or material indirect financial interest in the outcome. Where our team built the system, we supply the measurement work to the auditor you appoint, or take the remediation side and let another firm sign.

Q: We only have API access to the vendor model. How much can an audit see?

An audit from outside the model measures outcomes, selection rates, robustness to changed input, and consistency across groups. Those four cover most of what the regulations ask an operator to show.Two things need more than an API:attributing a result to the training data needs the training set;testing a counterfactual the API refuses needs the model;Every finding carries a confidence level, and the findings that need the vendor arrive as a written question list, phrased so the vendor’s answer settles the point either way.Systems your own teams built, and systems a previous vendor left behind, get the same treatment. Our auditors read the documentation that exists, test what the system does, and mark each documented claim as supported, partly supported, or unverified.

Q: What does one finding contain, and who closes it?

A finding arrives as a record your engineers can act on without a follow-up call. It names the system, the test that produced the result, the data slice the test ran on, the severity, the business consequence in one line, and the commands that reproduce the result on your side.Reproduction is the part that decides how fast a finding closes. A described finding starts an argument about whether the number is right. A finding with the query attached moves straight to the fix, because your engineer sees the same result our auditor saw.Closing sits with a named owner on your side, and the name goes into the register at handover so no finding waits for someone to claim it. Our auditors run the retest against the original test and mark the finding closed when the result changes, which keeps closure a measurement and keeps it out of the realm of opinion.Where you want the fix written as well as verified, Relevant merges the remediation as pull requests. Where an independent bias audit is in scope, the law keeps our engineers out of the code, so your team writes the fix and our auditors run the retest.

Q: Should we start with an audit or with a read on the whole estate?

It depends on whether a system has already been named. When a regulator, a customer, or your own risk committee points at something specific, audit that system and start measuring this month.When the question is which of forty systems deserve attention, an AI maturity assessment scores the estate across governance, data, infrastructure, and talent, and returns a ranked list. Auditing forty systems at once may cost more than the answer returns, and the ranking usually shows three of them carrying most of the exposure.

Every number in an audit report traces back to a query, a test set, and a threshold. Relevant Software audits your models and agents, then hands over the code that produced them, so next year’s audit can start as a re-run.

Book a free consultation

13+

Years in businessLarge expertise as a software development company

246

Projects deliveredEngineering excellence that solves industry challenges

99%

On-time deliveryMilestones land when we say they will

92%

Senior talentTop software engineers on our team

98%

Client satisfactionCustomers trust us with new projects

Our AI audit services

AI system audit
AI bias audit
Independent bias audit under NYC Local Law 144
ISO/IEC 42001 internal audit
EU AI Act conformity readiness audit
AI documentation review
Model validation for regulated use
Audit suite implementation

AI system audit
AI bias audit
Independent bias audit under NYC Local Law 144
ISO/IEC 42001 internal audit
EU AI Act conformity readiness audit
AI documentation review
Model validation for regulated use
Audit suite implementation

AI system audit

Our auditors test a production model or agent against what its documentation claims, covering accuracy on the cases that matter, behaviour at the edges, and the explanations it gives for its own output. A different question, what someone could make the system do on purpose, is answered by AI security services.

Performance measured on the segments and edge cases the system decides about
Robustness checks against malformed, adversarial, and out-of-distribution input
Explainability review covering what the system can show about a single decision

AI bias audit

Selection rates, impact ratios, and outcome differences tell you how a system treats the groups it decides about, and our auditors produce each number with the query behind it.

Selection rates by sex and by race or ethnicity, plus intersectional categories
Impact ratios compared against the four-fifths threshold
Sub-group performance differences reported with confidence intervals

Independent bias audit under NYC Local Law 144

The law asks for an auditor who did not develop, test, or train the tool and who holds no financial interest in its outcome. Relevant audits tools our engineers did not build, and where we did build one, we supply the measurement work to the auditor you appoint.

The published summary in the form the law expects, with a named auditor of record
Annual scheduling with the data pull documented for the next round
A methodology note your legal team reads before the audit starts

ISO/IEC 42001 internal audit

Certification bodies expect the management system to have audited itself first, and our auditors run that round clause by clause, then close findings with the people who own them. The structure being tested comes from our AI governance implementation work when your team wants both.

Clause-by-clause testing of the AI management system against ISO/IEC 42001
Sampled records showing each procedure ran, with dates and signatures
A corrective action log closed before the certification body books stage 1

EU AI Act conformity readiness audit

Before a conformity assessment, the technical documentation, the logging, the human oversight step, and the risk management file all have to exist and match the system. Our auditors test each one and mark it against the article that asks for it, using the classification set through our AI compliance services.

Technical documentation tested against the system it describes
Logging and record-keeping checked for retention and completeness
Human oversight design reviewed against how the system runs day to day

AI documentation review

A model card makes claims, and our auditors read each one against the system it describes, marking it supported, partly supported, or unverified.

Claim-by-claim verification against code, configuration, and observed behaviour
Provenance check on the datasets each model was trained and evaluated on
A rewrite list for the claims that need different wording

Model validation for regulated use

Banks and insurers run model validation as a standing function, and our auditors work inside that framework: conceptual soundness, data quality, outcomes analysis, and ongoing monitoring, documented the way your validation group files it.

Conceptual soundness review of the method against the stated use
Outcomes analysis and benchmarking against a challenger model
An ongoing monitoring plan with thresholds your validation group approves

Audit suite implementation

Every measurement in the audit ships as code, and our engineers put it where your team runs it again. MLOps engineering wires the same suite into your pipeline when you want the checks running between audits.

The measurement suite in your repository, with its test data and thresholds
A scheduled run that reproduces the audit’s baseline on demand
A comparison view showing what moved since the last round

What you keep after an AI audit

A findings register with owners and dates
Each finding arrives with its severity, the system it belongs to, the steps to reproduce it, and a named owner. Your engineers work from the reproduction steps, so the first fix starts before the report is signed.
The measurement suite that produced the numbers
Every metric in the report comes out of code our engineers hand over with it. Your team re-runs the same suite next quarter against the same baseline, so the next audit starts from a diff.
Bias results in the form the regulation asks for
Selection rates by sex and by race or ethnicity, intersectional categories, and impact ratios against the four-fifths threshold arrive in the layout a filing expects. The underlying queries come with them, so a reviewer can check the arithmetic.
A conformity read per system
A line-by-line comparison sets what each system's documentation claims against what the audit observed. Where the two agree, you hold evidence. Where they part, you hold a scoped item with an estimate against it.

Name the system your risk committee asks about, and the audit starts there. We scope it on the first call.

Start with one system

Our AI audit services from scoping to retest

A scoped system list and an audit plan you approve
Scoping covers which systems fall inside, which regime each one answers to, and what counts as a finding. You approve the plan and the price before any testing begins.
Access and data pulls documented as you grant them
Our auditors record the read-only access, the sample data, and the production traffic slice as they arrive, and write down the extraction steps.
Results as they are produced, with the query attached
Each result lands as the test generates it, with the data slice it ran on. Your team reads the query and the slice, so a disputed number gets settled the same day.
The report, with every number traceable
The written audit sets each claim in your documentation against the observed result, and each number points to the code that produced it.
The suite in your repository and a retest date
Handover moves the measurement code, the thresholds, and the test data into your repository, with the commands to run them. You finish holding the findings, the report, and the ability to produce next year’s numbers yourselves.

Why companies bring AI audit services to Relevant

Numbers you can trace to their source

Every figure in the report links to the query, the test set, and the threshold that produced it. A reviewer who disputes a result opens the code and checks it.

Auditors who read the code

Our team inspects the repository, the configuration, and the production traffic alongside the paperwork. A claim that holds in a model card and fails in the code gets caught at the source.

Faster delivery on every engagement

Every audit runs with our AI-assisted development practice behind it, which is where up to 50% faster delivery and up to 20% fewer bugs come from, with senior code review on every release.

Independence stated up front

Relevant tells you at scoping which parts of an audit our team may sign and which need an auditor you appoint. The rule that decides it comes from the regulation, and you see it before the engagement starts.

Findings that close

Where independence allows it, the engineers who found a problem write the fix and the acceptance test. The correction carries the same understanding that produced the finding.

Certified handling of the data an audit touches

ISO 27001, GDPR, and HIPAA-aligned delivery covers the sample sets, the traffic slices, and the personal records an audit has to read. The handling rules land in the audit plan you approve.

What clients say about Relevant

4.9 is our Clutch
average

‌

‌
‌

‌
‌
‌
‌
‌

‌

‌
‌

‌
‌
‌
‌
‌

‌

‌
‌

‌
‌
‌
‌
‌

‌

‌
‌

‌
‌
‌
‌
‌

‌

‌
‌

‌
‌
‌
‌
‌

‌

‌
‌

‌
‌
‌
‌
‌

‌

‌
‌

‌
‌
‌
‌
‌

‌

‌
‌

‌
‌
‌
‌
‌

Questions that come up when scoping AI audit services

What drives the price of an AI audit?

Can you audit a system your own engineers built?

We only have API access to the vendor model. How much can an audit see?

What does one finding contain, and who closes it?

Should we start with an audit or with a read on the whole estate?

Which AI system would you audit first?

Our AI audit services

AI system audit
AI bias audit
Independent bias audit under NYC Local Law 144
ISO/IEC 42001 internal audit
EU AI Act conformity readiness audit
AI documentation review
Model validation for regulated use
Audit suite implementation

AI system audit
AI bias audit
Independent bias audit under NYC Local Law 144
ISO/IEC 42001 internal audit
EU AI Act conformity readiness audit
AI documentation review
Model validation for regulated use
Audit suite implementation

Why companies bring AI audit services to Relevant

Numbers you can trace to their source

Every figure in the report links to the query, the test set, and the threshold that produced it. A reviewer who disputes a result opens the code and checks it.

Auditors who read the code

Our team inspects the repository, the configuration, and the production traffic alongside the paperwork. A claim that holds in a model card and fails in the code gets caught at the source.

Faster delivery on every engagement

Every audit runs with our AI-assisted development practice behind it, which is where up to 50% faster delivery and up to 20% fewer bugs come from, with senior code review on every release.

Independence stated up front

Findings that close

Where independence allows it, the engineers who found a problem write the fix and the acceptance test. The correction carries the same understanding that produced the finding.

Certified handling of the data an audit touches

ISO 27001, GDPR, and HIPAA-aligned delivery covers the sample sets, the traffic slices, and the personal records an audit has to read. The handling rules land in the audit plan you approve.

Which AI system would you audit first?

AI audit services