Public agencies should evaluate employee AI use against the service people need, the burdens it creates and the resources it consumes. Profit is not required to make a disciplined decision. Define a useful public outcome, check who can actually receive it and decide which failures would make the proposed use unacceptable. Staff time is part of that assessment, alongside the experience of the person seeking help.
This article concerns employee assistance in ordinary service communication. A decision about someone's entitlement, admission or access to an essential service needs its own safeguards and evaluation. The roles and industries guide explains why the actual task matters more than a broad sector label.
Name the service outcome before choosing a measure
Suppose Sam Taylor works at a municipal library. A branch will temporarily change its opening hours, and staff receive questions about when people can visit and where they can go instead. The library is considering an approved assistant to help staff draft replies from the current branch schedule and verified alternative contact information. Sam checks each reply before sending it.
Alex Morgan, the service manager, could measure drafts per hour. But the resident's task is to plan a usable visit. A fast reply with the wrong date fails that task. So does a correct reply that leaves someone unable to find an accessible way to obtain the information they need.
Alex starts with a service question: can staff help people understand the temporary arrangements, with less avoidable effort for residents and colleagues? That gives the trial a purpose beyond demonstrating use of a new tool. It also leaves room to discover that changing the published notice or improving the source schedule would solve more of the problem.
The UK Government Digital Service's Service Standard on defining success combines performance measures with user research and asks teams to consider online and offline channels. It is useful context for this library question, rather than a claim that every public agency follows the same service rules.
Write the outcome in language the service owner and a resident could both understand. “People can work out when and where to visit” is a better starting point than “increase AI adoption”. Tool use may explain how the work happened. It does not establish whether the service improved.
Keep different forms of value visible
The government's first-iteration AI impact evaluation framework distinguishes deployment impact from model evaluation and includes service, workforce and wider effects. Its suggested measures are illustrative. A library should choose evidence for its own service question rather than adopt an entire catalogue of indicators.
For Alex's temporary-hours enquiry, the following questions would expose different ways the trial could succeed or fail:
| Service question | Evidence the library could examine | What the measure cannot show alone |
|---|---|---|
Is the answer accurate and usable? | Review replies against the current schedule, branch details and agreed alternatives | Correct facts do not prove the resident understood them |
Does the answer arrive in time? | Track elapsed time to an accepted reply relative to the planned visit | A quick first draft may still wait for review |
Can people use the available channels? | Observe relevant phone, email and printed-information journeys with users | A successful email exchange does not establish access for everyone |
Is avoidable resident effort falling? | Investigate repeat contacts, unresolved questions and abandoned attempts | Fewer calls might reflect people giving up |
What happens to staff work? | Include drafting, checking, corrections and work passed to other colleagues | Sam's saved minutes may become another person's checking time |
Are the resources justified? | Record ongoing costs and the capacity used to deliver an acceptable answer | Released capacity is not automatically a budget saving |
These are separate questions, not ingredients for an automatic score. A lower cost should not conceal an answer that directs a resident to a closed branch. Nor should a favorable average make a recurring barrier for people using the telephone disappear.
In an October 2026 LinkedIn account, Emily Campbell-Ratcliffe, Director of AI Assurance at Frontier Economics, described gaps in post-deployment evaluation and communicating uncertainty. She was discussing her consultancy's evidence response, not reporting a measured public-service benefit. The practical challenge for Alex is similar: a result from a technical test must still be connected to the service decision it is meant to support.
Test the route by which people receive the answer
A service can be accurate at its source and difficult to use at its destination. Suppose Sam's email includes the correct temporary hours, but the enquiry began by telephone and the resident needs the details read back. Counting the completed email as a resolved enquiry would hide the remaining task.
Before testing drafting speed, Alex and Sam trace the relevant routes through the service. They ask where a person learns about the change, how they ask a question and what happens if the first answer is insufficient. They include people with relevant access needs in appropriate user research, rather than treating a colleague's rehearsal as proof of accessibility.
Agree which problems require correction before continuing. For this example, Alex could propose conditions such as:
- Wrong destination or date. A reply contradicts the current approved branch schedule.
- An unusable route. The proposed response leaves the person without an appropriate way to receive or clarify the information.
- An invented commitment. A draft promises an alternative service that the library has not confirmed.
- Unresolved checking responsibility. Staff cannot tell who must verify changed information before it reaches a resident.
The service owner must approve the actual conditions and the response to each failure. Keep them visible beside the outcome measures. A trial report should say when a condition failed, what changed and what was rechecked, even if the overall drafting average improved.
Confidence deserves separate attention too. Someone may say an answer feels reassuring while its date is wrong. Another person may receive an accurate answer and still distrust the service. Ask useful experience questions and check the service facts; satisfaction cannot stand in for both.
Compare the service with a credible alternative
Before the trial, find out how the library handles these enquiries now. Preserve the starting information while there is still a chance to collect it:
- Which schedule do staff consult?
- What checks do they perform before replying?
- Which channels do people use?
- What commonly leads to a second contact?
The Treasury's guidance on evaluating AI interventions, updated in May 2026, emphasizes early baseline planning and examining differences between groups. Work with an evaluator to choose a comparison that fits the service and the decision. A simple before-and-after change cannot isolate AI's contribution if staffing, demand or the quality of the branch notice changed at the same time.
The comparison should cover the complete task. In an October 2026 Reddit discussion, a self-described cataloguer explained using AI to find possible subject headings, then checking every suggestion in an authoritative source. Another cataloguer replied that this verification made the work slower for them. Their roles, employers and results are unverified, and these are individual experiences of a different library task. Their disagreement makes the measurement problem concrete: producing a suggestion and completing dependable work can take different amounts of effort.
For the temporary-hours trial, record consequential changes in the workflow as they happen. If Alex rewrites the notice halfway through, the final report needs that fact. If another colleague starts checking every reply, include that person's work. Avoid attributing all improvement or deterioration to the assistant.
Look at relevant differences without overclaiming. A small number of phone enquiries cannot support a confident percentage comparison with email. An absence of recorded complaints does not show that everyone could use the service. Plan appropriate coverage and handling of personal information with the responsible colleagues before collecting additional data.
Make the funding decision explicit
Public value does not make cost irrelevant. Alex still needs to understand what the service consumes and whether a simpler change could achieve the same result. Account for the work required to maintain the schedule, support staff and correct problems, alongside the cost of the approved tool.
If staff spend less time drafting, describe what that capacity can realistically support. Perhaps Sam can answer more enquiries during a closure period or give more attention to difficult questions. Check that the proposed use of capacity is feasible. Do not describe saved minutes as cash released from the budget unless the budget consequence is real and supported.
The decision note should distinguish four possible conclusions:
- Stop this use. A material service problem remains unresolved, or the proposed benefit does not justify the work and cost.
- Repair and retest. A specific source, channel or review problem has a plausible correction that still needs evidence.
- Extend the evaluation. The early findings are useful, but the coverage or duration cannot answer the funding question.
- Expand within an agreed scope. The evidence supports the proposed service use, with continuing ownership and monitoring.
State the evidence and uncertainty behind the chosen conclusion. A limited trial may justify a better-informed next test without justifying an organization-wide purchase. Equally, a useful improvement in residents' ability to plan a visit need not be converted into speculative profit to deserve consideration.
Questions about public-service AI evaluation
Can a public agency justify AI without a cash saving?
A public agency can consider a supported improvement in service quality, timeliness or access alongside its cost. For a library enquiry, helping a resident plan a usable visit is a relevant outcome even if the budget does not shrink. The decision still needs evidence of that improvement, an account of ongoing resources and a comparison with practical alternatives.
Are fewer calls a sign that the service improved?
Fewer calls can mean that people found clear information elsewhere, but they can also mean that people could not get through or gave up. A public-service trial should investigate the reason for the change. Combine contact data with unresolved-enquiry checks and relevant user research before treating lower volume as a benefit.
How should an agency check whether an AI-assisted service is inclusive?
An agency should examine how relevant groups and channels experience the actual service task, with appropriate user research and evaluation expertise. A successful email trial does not establish that telephone or printed-information routes work. Plan suitable coverage and data handling, investigate barriers and report uncertainty when the evidence is too small to support a comparison.
What can a short public-service pilot establish?
A short pilot may reveal whether staff can use the proposed assistance, where review work arises and which service failures need correction. It may provide too little evidence about less frequent enquiries, sustained access or longer-term confidence. Define the decision the pilot can support and name the evidence still required before wider use.
How should staff time appear in a public-value assessment?
Include the complete work needed to deliver an acceptable service outcome, including review, rework and effort passed to colleagues. Then explain what any released capacity can realistically support. Keep that capacity separate from a cash saving, and assess it alongside resident effort, service quality and continuing cost.
