Measure AI value by comparing worthwhile, accepted work with a credible alternative. Include the effort to prepare inputs, check outputs and correct mistakes. Then establish whether any improvement changed a result the organization cares about, such as turnaround, service quality, available capacity or cost.
Tool use helps explain what happened. It cannot, by itself, establish that the work improved or that the investment paid off.
This guide is for the adoption lead and workflow owner deciding what to continue, change or expand. Start with one workflow you can observe. If that choice is still unclear, use the task-discovery worksheet before building a dashboard.
Name the result before choosing the measure
“Get more people using AI” describes a rollout ambition. It leaves the value question open. A team might use a tool frequently because it helps, because use is required or because getting an acceptable result takes repeated attempts.
Choose a result that matters to the people doing or receiving the work. For example, an online retailer's customer-service team may want shoppers with delayed orders to receive accurate delivery information sooner. Check whether the reply matches the order and carrier records, how long the customer waits, whether they need to contact the team again and how much staff effort the answer takes. A prompt count cannot stand in for all of them.
Write down:
- The work. Which tasks and people are included?
- The accepted result. What must be true before the output can be used?
- The intended improvement. What should become better, and for whom?
- The decision owner. Who can judge the result and authorize the next step?
The Australian National AI Centre's guidance similarly starts with the problem, expected outcome and signs of progress. It also includes quality and capacity alongside financial benefits. Keep those categories visible in your own evaluation, so a worthwhile improvement does not need to be disguised as cash saved.
Keep use and outcomes in separate columns
A useful measurement approach connects different kinds of evidence without treating them as interchangeable. McKinsey's measurement framework makes a related distinction between technical performance, adoption, operations and financial impact. It is practitioner guidance, not proof that improving one level necessarily improves the next.
Use this smaller map to identify the question your current evidence can answer.
| Evidence | Question it helps answer | What it cannot establish alone |
|---|---|---|
Access and readiness | Can the intended people use the approved tool on suitable work? | Whether they use it or benefit from it. |
Use over time | Who returns to it, for which tasks and opportunities? | Whether their finished work is better. |
Accepted task results | Did quality, total effort or turnaround improve? | Whether the organization captured the benefit. |
Operational and financial outcomes | Did service, capacity, costs or another agreed outcome change? | Whether AI caused the change rather than something else. |
Keep the population and time window attached to each measure. “Half the team used it” means little without knowing who had access, who had relevant work and when the observation happened. A specialist with a monthly task should not be judged against someone with daily opportunities.
Use the AI adoption metrics reference to define the counted activity, denominator, reporting window and decision owner before setting a target.
When use is low, investigate access, task fit and support through the people and change guide. When use is high but results are weak, inspect the work. More training or more prompts may not resolve the actual problem.
Compare work that meets the same standard
For the checks a colleague should perform before accepting an AI-assisted result, use the guide to human review of AI output. It covers original evidence, omissions and what to do when a claim cannot be verified.
Agree the quality check before comparing speed. For a retailer's delayed-order reply, that means the right order, a confirmed delivery date or an honest statement that it is still unknown, and a clear next step for the customer. For a finance analyst's expense commentary, it means amounts traceable to the ledger, the record of transactions, and explanations supported by those entries. The people who rely on the output should help define acceptance.
Then compare similar work under conditions you can describe. Record task difficulty, available information and who performed the work. Include failed attempts and corrections. A collection of successful demonstrations will give a different picture from the ordinary queue.
A Reddit commenter describing a financial-institution content project explained that the team had established a manual page-review baseline before an AI pilot, while retaining human validation. They intended to compare review effort and quality consistency. This was a plan entering its first phase, with no reported results. A reply asked who owned the financial baseline and return on investment (ROI) calculations; that question remained unanswered in the visible discussion.
The practical lesson is to settle both the comparison and its owner early. Ask someone to be responsible for each of these checks:
- Quality. Review outputs against the same criteria. Where practical, keep reviewers unaware of which method produced them.
- Effort. Include preparation, generation, checking, correction and work passed to colleagues.
- Turnaround. Measure waiting and handoffs separately from active work time.
- Comparison. Record changes in task mix, staffing, demand or process that could explain the result.
A before-and-after observation can inform a decision, but other changes may account for it. Where the investment warrants stronger evidence, plan a suitable comparison group or randomized rollout with someone able to design the evaluation. Avoid comparing enthusiastic volunteers with everyone else and calling the difference an AI effect. The guide to running a credible AI pilot explains assignment, missing outcomes and the limits of the conclusion.
Read findings at the level they measured
A large experiment can produce a useful result without answering every business question.
In Shifting Work Patterns with Generative AI, researchers studied randomized access to Microsoft 365 Copilot across 66 large firms and 7,137 knowledge workers. Their main analysis examined the later months of a six-month rollout.
1.4 hours
less weekly Outlook session time after access to Copilot
The randomized-access estimate uses 6,441 workers in the email analysis during months four through six. It measures application activity, not financial savings.
The study did not find a significant average reduction in meeting time. It also lacked direct measures of work content or productivity, and application sessions could overlap other activity. Its evidence supports a change in observed email work; it does not establish an equivalent increase in profit or a universal saving for other organizations.
Treat employee estimates as another kind of evidence. In its May 2026 technical-worker survey, METR distinguished perceived speed gains from perceived gains in valuable work. The researchers explicitly describe surveys as useful but flawed complements to other methods. Their convenience sample and uncertain counterfactual estimates are reasons to avoid treating self-reported gains as audited results.
Ask employees for concrete examples of what changed, what still needed checking and what they did next. Use those accounts to choose observations worth making. Preserve the difference between “people report that this helps” and “we observed this improvement under these conditions.”
For a case-level record and calculation, use the guide to measuring net time saved by AI. It separates recurring effort, unattended waiting and setup.
For company accounts and commissioned reports, use the success-story disclosure questions to separate the evidence a publisher provides from the assumptions your own decision still needs.
Follow the benefit beyond the task
In a public account of his software work, architect Farrukh Naveed Anjum described faster generation shifting attention toward review, testing and release queues. His post offers a practitioner observation, without a measured organizational result. It suggests a useful question for your own evaluation: where does the output go next?
Suppose a freight company's account team uses AI to prepare weekly delivery updates for its retail customers. Each update identifies late shipments, confirmed arrival dates and unresolved questions from approved shipment records. An account manager checks those details before sending it. For a comparable update that passes those checks, the team records the effort below.
The difference is five minutes per accepted update. Across twelve comparable customer delivery updates, that would release one hour of capacity before training, setup or other program costs. The drafting improvement alone would overstate the gain. When local gains are real but delivery stays flat, investigate why personal productivity may not improve company results.
What happens to that hour matters. If the account team uses that hour to answer waiting delivery enquiries, check whether more customers receive checked replies. If the objective is to send weekly updates earlier, check whether they still wait in the account manager's approval queue. If people finish closer to their normal working hours, record that outcome rather than inventing a payroll reduction.
Capacity can be valuable while staffing expenditure stays unchanged. Claim a cash saving only when there is an actual cost change within the measurement period. Agree the treatment with the person responsible for the budget, and do not count the same released hour twice as both a cost saving and extra output value.
Include the full cost of achieving the result. The National AI Centre calls attention to training, testing, data preparation and ongoing oversight alongside visible software costs. Your evaluation should also retain checking and correction effort, even when that work moves to a different team.
For a staffed operating plan, use the full-cost adoption budget guide to separate additional cash, existing staff time and shared commitments.
For a funding request, use the guide to building an AI adoption business case to compare alternatives, expose assumptions and assign responsibility for realizing benefits.
Make the next decision explicit
Bring the evidence back to the person who can act on it. A short record is enough to start:
- Result sought. The workflow, people, accepted output and intended benefit.
- Comparison made. The period, task mix and method used to establish what changed.
- Observed result. Quality, effort, turnaround and any verified operational or financial effect.
- Limits. Missing information, alternative explanations and conditions under which the result may differ.
- Decision. Continue, change, extend the evaluation or stop, with an owner and a review point.
Separate those choices. A promising but uncertain result may justify a better evaluation. A quality failure may require correcting the workflow before further use. A well-supported gain in one team may justify testing transfer to another, without assuming it will carry over unchanged.
The AI adoption strategy guide connects this decision to ownership, support and rollout. Measurement should help the team choose what to do, not merely supply a more impressive slide.
Questions about measuring AI value
Does time saved equal AI ROI?
No. Return on investment compares benefits and costs over a stated period; time released is only part of that account. If a freight team saves an hour preparing checked customer updates, find out whether it answers more delivery enquiries, sends updates earlier or reduces extra working hours. Those are different outcomes, and none automatically reduces payroll spending. Include training, setup and review costs, and agree any financial treatment with the budget owner. The total-effort example shows how to trace that distinction.
Can usage data show that AI is valuable?
Usage data can show who uses a tool, when they return and which work deserves investigation. It cannot establish that the output is accurate or useful. A customer-service adviser might generate several delivery replies because the early drafts contain wrong dates. Pair usage with checks of the finished reply, the time taken and what happened for the customer. Use the evidence map to distinguish participation from task results and business outcomes.
What if we did not collect a baseline?
State that the earlier comparison is missing and begin a consistent comparison now. For weekly freight-customer updates, record the preparation, checking and correction time for similar updates that meet the same acceptance standard. Trustworthy historical records may help, but note differences in shipment complexity, staffing or available information. Do not turn remembered time estimates into precise measured savings. The comparison checks explain what to record and why a before-and-after change alone cannot establish an AI effect.
What if quality improves but the work takes longer?
The change may still be worthwhile if the better result matters enough to justify the extra effort. A retailer's delivery reply that takes longer to check may give the customer a reliable date and prevent another enquiry. Verify those outcomes rather than assuming them: define what counts as a correct reply, record the total effort and check whether repeat contacts change. Use the result-definition questions with the service owner to agree the tradeoff before treating speed as the sole test.
What should a small team measure first?
Start with one recurring piece of work and a clear definition of an acceptable result. A freight account team could compare weekly customer delivery updates, checking shipment details and confirmed dates while recording preparation, review and correction time. Follow the update through approval to see whether customers receive it sooner. Add measures only when they help the team decide what to change. Use the short decision record to keep the result, comparison, limits, owner and next action together.
