Assess AI readiness by asking what your team can demonstrate for a specific piece of work. Agree the proposed use, inspect the information and permissions it needs, watch how people would check the result, and record the gaps that prevent the next step. A useful assessment ends with a decision and named follow-up actions.
A questionnaire can help people prepare for that conversation. Its score becomes less useful when different answers cancel each other out: an enthusiastic sponsor cannot compensate for missing permission to use supplier data, and a well-written policy cannot create time for review.
For an adoption lead in a company of a few hundred people, the practical challenge is often resolving those differences. Here is how to turn competing claims of readiness into a decision the workflow owner can explain.
Decide what being ready would permit
Start by writing the decision you are assessing. “Ready for AI” leaves too much open. “Ready to test a tool that drafts a supplier-quote comparison for a buyer to check” names work, an output and a human responsibility.
Suppose Jamie Lee manages purchasing at a 180-person industrial wholesaler. Buyers compare emailed quotes before ordering replacement parts. Jamie wants to test whether an approved AI tool can extract the quoted item, quantity, unit price and delivery terms into a comparison sheet. The buyer would verify every field against the original before using it.
That proposal contains several possible permissions. Keep them separate:
- Prepare an offline test using permitted sample quotes, with no live purchasing decision attached.
- Assist a buyer on current work using approved supplier information and a staffed checking process.
- Place or change orders through a connected purchasing system.
Evidence for the first does not establish readiness for the others. Write down the users, information and actions included in the decision, along with what remains outside it.
NIST's voluntary AI Risk Management Framework 1.0 makes context the basis for an initial decision about whether to proceed. Its MAP guidance includes intended use, requirements and human oversight. This supports asking a bounded question; it does not supply a universal readiness grade.
If the team cannot yet name useful work, start with discovering use cases from everyday tasks. An assessment of an unspecified project will produce equally unspecified actions.
Inspect the evidence with the people doing the work
Invite the buyer, purchasing manager and relevant technical and data owners to examine the same examples. Ask each person to distinguish what they have inspected from what they expect to be true. Involve the people responsible for additional risks when the proposed use requires their expertise.
The UK government's AI Playbook recommends observing and speaking with people involved in a process to understand their needs and establish a baseline. It is public-sector guidance, but that research method is useful here: watch a buyer complete the comparison before deciding what assistance should change.
For Jamie's proposed test, use questions that lead to something the group can inspect.
| Readiness claim | Evidence to inspect | What an unresolved finding changes |
|---|---|---|
We understand the task | A buyer completes a recent comparison and explains exceptions | Observe more work before specifying the output |
The inputs are usable | Permitted quotes with different layouts, pack sizes and delivery terms | Add missing cases or narrow the supported inputs |
We may use this information | The applicable tool approval and supplier-data handling decision | Hold use of those inputs until their owner resolves permission |
Buyers can check the output | A buyer traces extracted fields to the source and catches a planted discrepancy | Improve the review method before relying on it |
Checking fits the workload | The manager identifies who checks, when, and what work makes room for it | Revise capacity or reduce the trial scope |
We can judge whether it helps | Current comparison effort, corrections and a definition of acceptable work | Establish a baseline and acceptance criteria before claiming a benefit |
Treat this as a starting set of questions for this task. A tool that can place orders needs additional checks around action authority, system behavior, support and recovery.
Actual records can contradict a confident summary. In an April 2026 Reddit account, a Reddit user describing client work reported that duplicate account records had led an AI deployment to give patients conflicting balances. The client was unnamed, and the account was not independently verified. It illustrates a reason to inspect ambiguous records; it does not establish how often this happens or validate the poster's readiness scoring system.
Resolve disagreements before assigning a score
Suppose Jamie says the supplier information is standardized because every quote includes a price. A buyer points out that some prices are per item and others per box. Both observations can be accurate. They imply different things about whether a tool can produce a meaningful comparison.
Do not settle the disagreement by averaging their confidence ratings. Ask what would resolve it. In this case, the team needs examples of both arrangements, an agreed way to represent the unit, and a check that refuses to compare prices when the unit is unclear.
Self-scoring also attracted a useful objection in a public X discussion. Mark Ajzenstadt, who promotes an AI engineering service, proposed an eight-question readiness assessment for portfolio companies. In a reply, Eriks Briedis questioned how much a room of respondents could establish by scoring itself.
Six of the eight are self-scored in the room, so two or three are probably generous.
A reply to a consultant's readiness proposal. This is a reader's objection to self-assessment, not a measured finding about assessment accuracy.
Keep a disputed finding open until the relevant evidence or decision arrives. A simple sequence helps avoid turning the meeting into a contest between the most confident speakers.
Resolve one disputed finding
State the claim
Supplier prices can be compared using the extracted fields.
Inspect a difficult case
Compare a per-box quote with a per-item quote and identify the missing unit.
Agree the consequence
Require explicit units and buyer verification; hold comparisons with unresolved units.
Keep the evidence attached
Save the example, the decision owner and the condition that would change the finding.
NIST's MEASURE guidance calls for documenting risks that cannot be measured and assessing performance under conditions similar to deployment. It also includes independent assessment. Use those principles to keep uncertainty visible and bring in someone outside the build team when the decision warrants it. NIST AI RMF core.
Separate unresolved conditions from useful experiments
Some uncertainty is the reason to run a test. Jamie does not yet know whether extracting quote fields will reduce total comparison effort. An offline evaluation can investigate that question if the sample information is permitted and the team has a way to judge the output.
Other gaps prevent the proposed test from starting. If nobody has established whether supplier quotes may enter the selected tool, calling it a pilot does not resolve that permission. If no buyer is available to check the results, the team cannot evaluate a process whose design depends on buyer review.
For the wholesaler, a reasonable next decision might be to prepare a permitted offline sample while leaving live quote processing and order placement out of scope. State who must confirm sample use, what counts as a correct comparison, and which failures need attention before the next review. Do not describe this preparation decision as approval for deployment.
A readiness questionnaire remains useful when it helps people find those questions. NIST explicitly describes its Playbook as voluntary guidance rather than a checklist to complete in full. Choose checks for the proposed work, and ask anyone offering a predictive score to show what outcome it predicts, on which population, and how that prediction was validated.
Write a decision someone can revisit
End the assessment with a short record in the team's existing planning system. Include:
- The decision and its scope. Which activity may proceed, with which users, inputs and actions?
- The evidence inspected. Link examples, approval records and test results, with dates or versions.
- The unresolved findings. Preserve disagreements and unknowns, including the consequence for the proposal.
- The next actions. Name an owner and describe what completion looks like for each gap.
- The next review. Set a date suited to the work and the changes that require earlier reconsideration.
For Jamie, “improve data readiness” is too broad to close. “The purchasing lead agrees how pack size and price unit appear in the test set, then a buyer checks each expected comparison against its source quotes” names observable work. A separate action should resolve tool permission; completing one does not complete the other.
Ask the person who owns the workflow decision to confirm the result. If review work has no place in the schedule, use the learning-time agreement to make the capacity commitment concrete. Carry the resulting decision into the wider AI adoption strategy.
Questions about AI readiness assessments
What should an AI readiness assessment include?
An AI readiness assessment should examine the task, permitted information, system access, people's ability and time to review outputs, and evidence needed to judge the result. Select additional checks according to what the proposed system can do and whom it affects. Begin with a specific permission or investment decision, then use the evidence questions to identify what the team must inspect.
Do we need an AI maturity score?
You do not need a composite score to decide whether a bounded test may proceed. A consistent rubric can organize discussion, provided its evidence and limits remain visible. Keep required conditions separate so strengths elsewhere cannot conceal an unresolved approval or absent reviewer. Before treating a score as a predictor of success, ask for validation relevant to your intended use. The disagreement process helps turn conflicting ratings into concrete checks.
Who should assess AI readiness?
Include the workflow owner, people who do the work, relevant technical and data owners, and specialists responsible for the proposal's risks. Assign the final decision to someone with the authority to make it. Seek independent challenge when the stakes, missing expertise or conflicts of interest warrant it. Each participant should bring evidence from their own responsibilities; a sponsor should not answer for everyone's operating conditions.
Can we start a pilot before every gap is closed?
A bounded test can investigate uncertainty within an approved scope, such as whether permitted sample quotes can be compared accurately. It should not proceed past a missing prerequisite for that scope, such as unresolved input permission or unavailable review capacity. Record the conditions, stop points and evidence needed for the next decision. The offline-test example shows why preparation, live assistance and automated ordering require separate judgments.
