Eradicating Bias: The Explainable ROI of AI in Sourcing

Explainable AI (XAI) can make some model behavior easier to inspect, but explainability alone does not remove bias, prove a hiring criterion is job-related, or guarantee a financial return. Results depend on the job, data, workflow, and people using the tool. This guide outlines a more defensible way to assess sourcing systems and measure their value. It is general information, not legal advice.

What explainability can and cannot do

An explanation describes aspects of how a model produced an output. It can help reviewers investigate influential inputs, identify unexpected behavior, and document questions for a vendor. It does not establish that the inputs are appropriate, the model is valid for a particular job, or outcomes are fair. NIST treats explainability and interpretability, validity and reliability, transparency, and fairness with harmful bias managed as distinct characteristics of trustworthy AI. They need to be evaluated together, not treated as substitutes for one another.

Employment discrimination laws also apply when employers use AI. The EEOC notes that a seemingly neutral selection procedure can raise disparate-impact concerns if it unjustifiably screens out a protected group. A vendor's explanation or fairness report is therefore evidence to assess, not a legal certification or proof of compliance.

How to assess a sourcing system

Start by documenting the system's intended use: which roles it supports, what information it receives, what recommendation it produces, and how recruiters use that output. Validate the system for that use rather than relying on a general claim about model performance. Review data quality, job-related criteria, accessibility and accommodation needs, and outcomes across relevant groups where lawful and appropriate. Record the evaluation method, sample sizes, limitations, and changes to the model or workflow.

A human reviewer should have enough context and authority to question or disregard a recommendation. Monitor what happens in practice; a model that appears acceptable in a test may behave differently after data, job requirements, or recruiter practices change. No single audit, score, or explanation demonstrates that a system is bias-free.

Which outcomes should be measured?

Choose measures before a pilot and define exactly how each is calculated. Useful operational measures may include time spent on sourcing and review, candidate response rates, funnel conversion, recruiter capacity, and time-to-fill. For quality, use job-related criteria and a stated follow-up period instead of an undefined label such as "quality of hire." For fairness, examine selection patterns at relevant stages and assess whether observed differences are meaningful given sample sizes and uncertainty. A single ratio is a diagnostic signal, not a complete legal analysis or proof of fairness.

Compare like with like: the same job families, locations, hiring stages, and time windows where possible. Track baseline performance and a comparison group, and note changes in pay, labor-market conditions, hiring volume, or process design that could affect results. This helps separate the tool's contribution from other causes.

How to calculate ROI responsibly

Use measured, attributable benefits and include the full cost of the system. A basic formula is:

ROI (%) = (verified benefits - total program costs) / total program costs x 100

Depending on the pilot, costs may include licenses, integration, data preparation, training, recruiter review, accessibility work, validation, independent audits, monitoring, and vendor management. Count time saved as financial value only when the organization can explain how that capacity was used or how it changed costs. Treat avoided bad hires or improved retention as benefits only when the measurement method, baseline, time horizon, and attribution are defensible.

There is no universal percentage by which AI reduces bad hires, time-to-hire, or recruiter workload. Published figures may describe a single vendor, customer, job type, or survey and may not establish causation. Do not present such estimates as expected results for every employer. Report the source, population, period, comparison, and limitations—or omit the number.

A practical pilot

1. Define the decision the system supports and the jobs in scope.
2. Set job-related success, fairness, accessibility, and operational measures in advance.
3. Establish a baseline and a comparison method before deployment.
4. Record all program costs and preserve human authority over decisions.
5. Review results and uncertainty with qualified HR, technical, and legal stakeholders.
6. Continue monitoring after deployment and reassess when the model or workflow changes.

Conclusion

Explainability can support investigation and oversight, but it is not a bias-removal method on its own. Evaluate validity, outcomes, accessibility, privacy, and human use together. Measure ROI with organization-specific evidence and communicate estimates with their limits instead of promising that AI is unbiased or guarantees savings.