Measure growth and better decisions by following one consequential choice from the evidence available when it was made to the action taken and the result observed later. Keep those three records separate. A well-supported decision can have a disappointing outcome, and a renewal can arrive for reasons unrelated to AI.
For an adoption lead and business-unit owner, the useful question is what changed in the choice, what happened because someone acted on it, and how much of the commercial result the comparison can support. The guide to measuring AI value covers the broader measurement program. Here, the focus is a decision whose business outcome takes time to arrive.
Name the choice that could change the result
Suppose Jamie Lee manages customer renewals at a business-software supplier. An approved assistant helps Jamie review permitted support records, product-use information and contract details before proposing a renewal plan. The hope is that better choices about customer support will improve retention.
“AI-assisted renewals” is too broad to evaluate. Choose the decision the trial will change. For example, should an account receive a targeted training session, a narrower subscription plan or ordinary renewal support? State who can approve each commitment. Generating a useful account brief and authorizing a change to the customer's contract are different responsibilities.
Then write the proposed connection in plain language: the account review identifies an unresolved training need; the team offers suitable help; the customer can use the software more successfully; that improvement may affect the renewal. Each transition is a question to investigate. A larger pile of account briefs establishes none of them.
Name the competing explanation too. A customer might renew because switching is inconvenient, because a procurement deadline passed or because a discount changed the offer. Capturing these conditions makes the later result more interpretable. It also gives Jamie something specific to learn before another renewal season.
Assess the decision before learning who renewed
The Society of Decision Professionals’ core principles distinguish the quality of a decision at commitment from its outcome under uncertainty. Its framework considers the decision's framing, alternatives, information, values, reasoning and commitment to action. These are professional principles, rather than evidence that an AI tool improves a company's decisions.
Apply that distinction to the renewal choice. Before Jamie learns the result, have someone who understands the account examine the proposed action and the evidence available at that point.
| Question at the time of the choice | Evidence to retain for Jamie's account |
|---|---|
Was this the right problem to address? | Customer reports and usage records distinguish a training need from a missing product capability. |
Were meaningful alternatives considered? | The record compares training, a narrower plan and the current service, including their costs and limits. |
Does the reasoning support the commitment? | The proposed session addresses the observed difficulty; unresolved facts and approval conditions stay visible. |
Could the team carry out the choice? | A service owner confirms the available help, timing and authority to make the offer. |
Use the same standard for comparable decisions made with and without AI assistance. Preserve the initial assessment before the renewal outcome is known. Otherwise, a reviewer may reconstruct a persuasive explanation around whichever customer stayed.
This goes beyond checking whether the account brief is accurate. The output-quality guide covers that prerequisite. An accurate brief can still support a poor choice if it ignores an alternative or promises help the team cannot deliver.
The distinction appears in practitioners’ frustrations. In a May 2026 Reddit discussion, a Reddit user described product requirements arriving faster while teams skipped the customer-problem work beneath them. Another Reddit user suggested sampling those documents and asking their authors for the problem, evidence and cost of delay. The original poster replied that leadership priorities and different levels of AI competence complicated the change. These anonymous accounts do not prove better outcomes, but they expose why counting finished documents can miss the decision the program hopes to improve.
Follow the choice through delivery and customer response
A decision record needs an implementation record alongside it. If Jamie proposes training but no trainer is available, the customer has not received the intervention. If the session happens but covers the wrong problem, completion alone is insufficient.
Three questions for Jamie’s renewal plan
Proposed connections to test, with separate evidence at each stage
Does training fit this account?
Compare the observed customer difficulty with training, a narrower plan and ordinary support. Retain the reasoning before the renewal result.
Did the customer receive useful help?
Record the approved session, delivery and response. An undelivered offer cannot demonstrate that training helped.
Did retention change against a credible comparison?
Follow eligible accounts to their renewal outcome. Include pending results, concessions and the cost of support.
For this trial, record whether the approved help was offered and delivered, what the customer did afterward and whether the original difficulty remained. Keep declined offers and failed delivery in the account population. They reveal whether the proposed approach works under ordinary operating conditions.
Record changes to the plan as well. A customer may request a different session, reject the narrower subscription or disclose that its budget has already been cut. Those developments help explain the result without turning the original recommendation into a fictional success.
A better decision process is an early finding. A delivered service is another. Neither is a measured renewal gain. Report each at its own level while the commercial outcome remains pending. If nothing changes beyond the account brief, investigate that break before scaling the program on the assumption that revenue will follow.
Compare commercial results for accounts that have had time to decide
Define the eligible accounts and the renewal observation window before looking at results. A cohort is simply the group followed under those agreed conditions. State which contracts have reached their decision date and which remain unresolved; a late-year renewal should not be counted as a retained customer in an early review.
Choose measures that match the claim:
- Customer retention. Among the eligible accounts with an observed renewal outcome, how many renewed? Also report pending and missing outcomes so selective exclusion cannot flatter the rate.
- Retained revenue. What recurring revenue was retained from the eligible population? Separate expansion, contraction, concessions and cancellations rather than treating every renewal as equivalent.
- Cost of the intervention. What did preparation, review, training delivery and additional support require? A retained account with an expensive service promise can have a different economic result from one renewing on ordinary terms.
Agree the accounting with finance. The AI ROI guide explains how to avoid treating retained revenue, margin and saved capacity as independent benefits to add together.
Next ask what would have happened without the changed renewal process. Where feasible, use a pre-agreed randomized rollout or another defensible comparison, following the credible AI pilot guide. Comparing enthusiastic users’ accounts with everyone else's can mix the effect of the workflow with differences in account difficulty, manager experience or willingness to seek help. A historical comparison also needs to account for pricing, product and market changes.
Actual commercial research provides a reason to test rather than assume the direction of improvement. In a field experiment published in Management Science in July 2026, Otis and colleagues randomized access to an AI business adviser among 640 Kenyan entrepreneurs. The approximately five-month experiment ran in 2023 and measured self-reported revenues and profits. It did not establish a positive average performance effect; results differed with baseline business performance.
The researchers’ analysis pointed toward differences in the changes entrepreneurs chose to implement. They also noted limited statistical power, reporting limitations and alternative explanations. This microenterprise setting is not a renewal benchmark for a larger company. Its useful challenge is narrower: access to plausible advice does not establish commercial benefit, and an average can conceal important differences.
Decide what the evidence permits you to fund next
At the review date, make a claim that matches the furthest observed link. If account decisions are better supported but delivery is unreliable, fund a bounded delivery repair if the case warrants it. If the service reaches customers but renewal outcomes are still immature, keep the commercial claim pending and set a review around the relevant contract cycle.
Use a short investment record:
- Decision population. Which renewal choices and accounts were eligible, and which were excluded before the trial?
- Observed change. What improved in the choice, delivered help or mature customer result, against which comparison?
- Uncertainty and downside. Which outcomes are pending, which accounts fared worse and which other changes could explain the difference?
- Resources and authority. What extra review, support and concessions were required, and who can approve the next commitment?
- Next decision. Continue within the same scope, repair a named weakness, stop or expand under stated conditions, with an owner and review date.
Agree those conditions with the business owner before results arrive. There is no universal renewal-rate improvement or number of weeks that makes expansion worthwhile. The threshold depends on the size and reliability of the benefit, the cost of delivering it and the consequences of being wrong.
Questions about measuring growth and better decisions
Can higher revenue prove that AI improved decisions?
Higher revenue alone cannot establish that AI improved decisions. Customer mix, pricing, product changes and market conditions can move revenue at the same time. Define the decision process being changed, retain its implementation evidence and compare outcomes for a pre-agreed eligible population. The commercial comparison explains why maturity, costs and alternative explanations belong beside the result.
How can we assess decision quality before the outcome arrives?
Assess the reasoning using the information available when the decision was made. For a customer renewal, examine whether the proposed help addresses the observed problem, considers feasible alternatives and has an owner able to deliver it. Preserve that assessment before the renewal outcome becomes known. It is evidence about the decision process, with its own limits, rather than a forecast that the customer will stay.
How long should we wait for growth results?
Choose an observation window that matches the outcome and gives comparable cases time to mature. A training session can be assessed before an annual contract renews, so report delivery and commercial results on separate schedules. Identify pending and missing outcomes explicitly. A slow result does not guarantee a later payoff; agree what interim evidence and costs justify continuing the bounded trial.
Can we count an avoided cancellation as a benefit?
An avoided cancellation is a claim about what would otherwise have happened. A customer renewing after training does not establish that it would have cancelled without that training. Retain evidence about the original issue and the action taken, use a defensible comparison where possible, and distinguish an estimated avoided loss from observed retained revenue. Have finance review the assumptions and prevent double counting.
Do adoption rates still matter in a growth trial?
Adoption rates help establish whether the changed workflow reached the intended work. Report who could use it, which eligible renewal decisions received support and why others did not. Pair that reach with evidence of decisions, delivery and customer results. Usage cannot substitute for those outcomes, and an appropriately rejected recommendation may reflect sound judgment rather than failed adoption.
