An internal assistant should withhold an answer when the information it is allowed to use does not support the claim the employee needs. That can mean declining one conclusion while answering the rest of the question. A return approval can establish that a replacement was authorized. It cannot establish that the replacement left the warehouse.
Abstention should preserve a route to finishing the work. Tell the employee what is known, which conclusion remains unsupported and how to obtain the missing evidence through an approved route. Ask a clarifying question when the employee can supply a necessary detail. If an appropriate source is available, check it rather than refusing prematurely.
Decide what the question needs to establish
Suppose Sam Taylor coordinates customer service at an office-equipment distributor. A customer needs a replacement printer before a training session. Sam asks the company's assistant, “Has the replacement printer shipped?” The permitted return ticket shows that the replacement was approved, but the assistant has no dispatch record for it.
An answer such as “Yes, your replacement is on its way” would convert an authorization into a shipping event. “No, it hasn't shipped” would make the opposite unsupported claim. Either could lead Sam to give the customer an inaccurate update or arrange an unnecessary replacement.
The first decision is therefore about the evidence required for this claim. The assistant needs to identify the right order and find an applicable dispatch record. A relevant document about returns does not supply that proof.
Joren and colleagues' Sufficient Context research distinguishes information relevant to a question from information sufficient to answer it. Their question-answering experiments also show why providing sufficient context does not guarantee a correct answer. These are public benchmark results, not a test of your company's assistant. The useful implication is to inspect what the supplied material establishes, rather than treating relevance or the presence of a citation as acceptance.
Vinodkumar Rathod describes the practical frustration in a personal assistant project using synthetic data. His first version produced fluent policy answers that the supplied material did not support. He reports adding evidence-limited answers alongside retrieval improvements and known-answer tests. His account illustrates the problem; it does not establish a production success rate or a recipe that guarantees correctness.
Clarify, check or withhold the unsupported conclusion
These responses solve different problems. Choose according to what is missing, rather than making every uncertain question receive the same apology.
The same shipment question needs different responses
Sam asks whether the replacement printer has shipped.
Ask for the order number
The missing identity can be supplied by Sam. Do not ask Sam to provide the shipping answer they came to find.
Withhold the shipment claim
Approval confirms authorization, not dispatch. Explain the limit and direct the unresolved check to the warehouse desk.
Answer within the record’s scope
Confirm the correct order and what the permitted record establishes. Do not turn dispatch into an unsupported delivery promise.
In the middle case, asking Sam whether the printer shipped would return the original task to the person seeking help. A useful clarification would ask for an order number if that is missing. Once the order is identified, the unresolved issue is evidence of dispatch.
A specific permitted lookup can resolve that issue. Repeating broad searches without a reason to expect the dispatch record can add material without answering the question. Xie and colleagues' EACL 2026 study found that search could improve performance on answerable benchmark questions while making appropriate abstention harder on unanswerable ones. That does not mean search is harmful in general. It means more retrieved text is not, by itself, evidence that an unsupported question has become answerable.
Microsoft's architecture guidance likewise distinguishes missing context, partial information and ambiguous questions. Those distinctions need to become observable behavior in your workflow. An instruction to avoid guessing is only a starting point; test what the assistant actually does with each condition.
Keep the access boundary intact. A missing source is not permission to search a colleague's private material or move customer records into another account. The permissions guide explains why an available connection and an appropriate use are separate decisions.
Make the reply useful without overstating progress
For Sam, a bounded reply could read:
This reply gives Sam a confirmed fact, a precise limit and a next action. It neither invents shipment status nor declares that no dispatch record exists anywhere. The wording should match the assistant's actual access and search, rather than imply a company-wide investigation.
Agree the ordinary route with the service owner before relying on it. In this example, Jamie can check the warehouse record through the approved process. If that person is absent, Sam needs the team's existing backup route. An assistant that names an unavailable individual has supplied a suggestion, not a completed handoff.
Keep these states distinct:
- A route is available. The reply explains who can check and what Sam should provide.
- A request was sent. The system has evidence that the permitted request was actually created or delivered.
- Someone accepted the work. The receiving team has acknowledged responsibility under the agreed process.
- The question was resolved. An authorized person or source supplied the dispatch status for the right order.
Do not let a reassuring sentence silently move the work from the first state to the last. If the assistant only gives instructions, it should not say “I've escalated this.” If it can create a request, confirm the actual result and leave the unresolved shipment question visible.
Avoid a broad refusal that discards useful information. Sam may still be able to tell the customer that the replacement was approved and that shipping status is being checked, provided the check is actually underway. Any delivery promise needs its own support.
Test missed answers as well as unsupported answers
A cautious final reply can be appropriate for the information the assistant received while exposing a failure elsewhere. If the dispatch record was available through the approved route but never reached the assistant, withholding the shipment claim avoids guessing. It does not make the overall service successful.
A Reddit discussion about abstention testing makes that distinction explicit. The original poster proposes separating genuinely missing information from information that retrieval failed to find. A reply describing a testing practice recommends retaining the expected source identifiers and checking what was actually retrieved before judging the answer. These are practitioner accounts and proposals, not independently verified company results. Their useful challenge is that a refusal score can look better while useful answers become harder to obtain.
Ask the business owner to label a small set of real, permitted questions with the evidence they require. For each question, distinguish the answer decision from the system's ability to find that evidence and the employee's eventual resolution.
| What to examine | What would count as a problem in Sam's task? |
|---|---|
Unsupported conclusions | The assistant claims shipped or not shipped from approval alone. |
Unnecessary withholding | The right dispatch record is supplied, but the assistant still declines to answer. |
Missed evidence | The approved source contains the dispatch record, but the ordinary lookup does not retrieve it. |
Unhelpful clarification | The assistant asks Sam to provide shipment status instead of the missing order identity. |
Unfinished work | The reply suggests a warehouse check, but nobody receives or owns the request. |
This avoids treating every non-answer as either success or failure. Keep the denominators visible when reporting results: accuracy among answered questions leaves out withheld questions, while coverage tells you how many questions were answered. Report both, along with unsupported answers and missed answerable work. A system that refuses everything cannot become useful by achieving a flattering score on the answers it never gave.
Set the acceptance conditions for the task's consequences. A shipping update that drives a customer commitment needs evidence of dispatch for the correct order. A general explanation of how to request a replacement can use the approved procedure. The same confident tone or numerical score cannot substitute for those different requirements.
Rehearse the unresolved case before rollout
Start with one recurring employee question and the people responsible for its sources and receiving workflow. The knowledge-ownership guide helps distinguish who settles the business answer from who maintains the technical source route.
- Define the required proof. For Sam's question, agree which record establishes dispatch and how to identify the correct order.
- Try contrasting evidence conditions. Run an approval-only case, a case with the applicable dispatch record and a case missing the order identity. Record the actual replies and source access.
- Follow the unresolved work. Use the ordinary warehouse route. Check whether the recipient gets enough permitted detail, can accept the request and knows how to return the answer.
- Review the failure and retest. Correct the source, lookup, reply or receiving process that failed. Recheck the contrasting cases so fixing unsupported answers does not introduce needless refusals.
Repeat after material changes to sources, access or assistant behavior. If the warehouse queue cannot absorb the unresolved work, keep the workflow limited while its owner repairs that dependency. Polite abstention is valuable, but it does not provide missing service capacity. For the broader choices around an internal assistant, return to the tools and workflows guide.
Questions about internal assistants withholding answers
Should we use a universal confidence threshold?
An internal assistant should not use one numerical cutoff as proof that every kind of answer is safe to use. The required evidence and consequence vary by task. Define what establishes a shipment, policy entitlement or procedural instruction with the responsible owner, then test both unsupported answers and unnecessary withholding. Any score used to help make that decision needs validation for those conditions.
Can an assistant give part of an answer?
Yes, when the supported part is useful and clearly separated from the missing conclusion. A return approval can be reported without claiming that the replacement shipped. State the limit next to the confirmed fact and give the approved next step. Avoid a partial answer that invites the employee to infer the very conclusion the assistant cannot support.
What if the answer exists but the assistant cannot find it?
Withhold an unsupported claim in the current reply, then investigate the missing evidence as a service failure. Confirm the permitted source and record what the ordinary lookup retrieved. Compare that with the evidence required for the question, using the company-answer diagnostic. Do not count a missed answer as successful abstention simply because the final wording was cautious.
Should the assistant answer if the employee asks again?
Repeating a question does not supply missing evidence. The assistant can clarify the order, check a specific permitted source or explain the unresolved condition more clearly. It should not turn an unsupported shipping claim into a fact because the employee insists. If time is short, use the agreed human route and make the deadline clear without inventing an answer.
Who owns the work after an assistant abstains?
The workflow owner must agree who receives unresolved questions, what information they need and how the employee gets a response. The assistant's technical owner investigates retrieval failures; the appropriate business owner settles the answer. Test the receiving route in a rehearsal. A suggested contact is not evidence that someone accepted or completed the work.
