An AI assistant can answer a general question well and still fail on your company's own work. The answer may be missing from the approved sources, inaccessible to the employee, absent from the material retrieved for that question, or misinterpreted after retrieval. Trace one failed answer back to its governing source before buying another tool or prescribing more prompt training.
For an enablement lead and internal service owner, the useful result is a specific finding someone can act on. “The assistant doesn't know our business” is a starting complaint. “The North branch procedure is current and readable by this employee, but the assistant returns the retired checklist” identifies a different next check.
In a public manufacturing discussion, a Reddit user wanted Copilot to find equipment and cost details in company agreements stored in Word files. They asked what prompts to use and whether an agent was needed. Replies suggested connecting or narrowing the sources, but the discussion did not establish a successful resolution. The question illustrates why it helps to inspect where the company information is available before treating wording as the whole problem.
Establish which company answer should exist
Choose a question whose correct answer a responsible colleague can establish. Record the assistant's actual reply, the employee's approved account, the time and the source it cited, if any. Keep confidential examples inside the approved support route.
Then ask the person who owns the subject to identify the applicable source. A document's recent modification date does not settle whether it governs this branch, customer, contract period or task. If two current documents disagree, someone with authority must resolve that conflict. The assistant should not invent the company's decision.
Suppose Claire Bennett coordinates equipment transfers for a rental company. She asks what North branch needs before accepting an internal transfer request. Its current procedure requires an equipment identifier, receiving branch, requested date and transport owner. An older generic checklist names only the equipment and date.
Claire's first check is whether the current North branch procedure really applies. If operations cannot settle that, changing the model will leave the underlying uncertainty intact. If the answer is settled, the current procedure becomes the reference for investigating the assistant.
Separate access from retrieval
Ask the technical owner to trace the configured route, rather than assuming that a file somewhere in the company is available to every assistant. Retrieval means finding and supplying relevant material for the answer. It is separate from the model's general training.
Work through these checks with the affected employee:
- Open the exact source. Can that employee open the current document through the approved work account? An administrator's successful attempt does not answer this question. If access is missing, ask the information owner whether it should be granted. Do not broaden permissions simply to make a demonstration work.
- Check the assistant's source route. Is that location included in this assistant's configured sources? Is it a live connection, an uploaded copy or something else? Ask what evidence shows that the relevant revision has reached the system.
- Inspect the material returned. Where the tool exposes search results, citations or diagnostic records, check whether the relevant passage was found. A connected location or a healthy status badge alone does not establish that this question received the right information.
Microsoft's Copilot Studio troubleshooting documents several ways SharePoint knowledge can return no answer, including indexing, permissions, protected content and configuration. It also describes a particularly useful distinction: a user may be able to open an encrypted document while the agent cannot use its content for an answer. Those are product-specific conditions to investigate, not a reason to remove a protection your organization needs.
For Claire, finding the old checklist in a citation is evidence worth preserving. It does not yet tell the technical owner whether the current procedure was excluded, not yet processed or simply missed during retrieval. Keep the diagnosis provisional until the configured route has been checked.
Compare a known passage with the ordinary question
If the approved tool supports selecting a document or supplying an excerpt, compare a tightly scoped attempt with the normal company-wide question. Use only material permitted in that tool and retain enough surrounding text to preserve its conditions. This comparison helps separate finding the information from using it.
For the rental example, use a fictional practice request and the applicable procedure. Ask which details the request needs. Then compare the result with an ordinary question that leaves the assistant to find its own sources.
| What happens | What it suggests | What it does not establish |
|---|---|---|
The supplied passage produces a sound answer; the ordinary query does not | Investigate source selection, retrieval scope and which revision was supplied | That the connection is the only fault, or that all other questions will work |
Both attempts miss a requirement that is explicit in the passage | Investigate interpretation, instructions and the surrounding conversation | That every model would fail, or that the source is unusable |
The passage itself lacks a settled answer | Return to the subject owner for clarification | That more elaborate prompting can supply an authorized company rule |
Run a fresh conversation as well as the original one, and record meaningful differences. Change one condition at a time where practical. A sequence of increasingly detailed hints may help someone finish today's task while concealing why the normal route still fails.
Do not make direct document selection a permanent workaround without considering its cost and permitted use. If employees must locate and paste every answer themselves, inspect whether the assistant adds enough value to justify that extra work.
Check the answer after the right source appears
Finding the correct document is progress. The reply still needs to preserve its conditions, important omissions and uncertainty. Open the cited source and compare the relevant passage with the answer; a citation is a route to evidence, not acceptance of the result.
In Claire's example, an answer that names the receiving branch but omits the transport owner is still incomplete. A reply based on the current procedure can also overreach by saying the request is approved when the procedure only lists what must be submitted. The human-review guide explains how to make those checks specific to the work.
The distinction between supplied information and its successful use also appears in research. Liu and colleagues' 2023 study varied where relevant information appeared among documents supplied to older language models. Performance depended on its position. That controlled question-answering study does not measure today's company assistants, but it challenges the assumption that adding more context necessarily produces a better answer.
Useful company context can work. In a January 2026 post, Microsoft and GitHub product manager Kayla Cinnamon described comparing a PowerToys specification in SharePoint with its repository implementation using Copilot. Her supplier-affiliated account is a reported successful task, not a guarantee for another organization's sources. Check your own question and accepted result with the same specificity.
Close the failure with evidence
Give the finding to an owner who can change the relevant part of the arrangement. The subject owner settles a conflicting procedure; the system owner investigates retrieval; the workflow owner decides whether the resulting answer is suitable for use. The ongoing-support guide covers how to keep that handoff from becoming an unowned ticket.
Keep a compact record:
- Question and expected answer. Include the governing source, applicable version and essential conditions.
- Observed failure. Preserve the actual reply and relevant retrieved passage, where available.
- Comparison. Record the account, source scope and what changed between attempts.
- Finding and owner. Separate a confirmed cause from a hypothesis still being investigated.
- Recheck. Repeat the affected question and a nearby exception after the repair, through the interface employees use.
For Claire, that exception might be a request for another branch whose procedure differs. Fixing North branch by forcing its answer onto every request would create a new error. Check that an unresolved or out-of-scope question reaches the appropriate person instead of receiving a plausible rule.
Microsoft's generative answers FAQ notes that responses can contain mistakes and that source information is not accuracy-checked. Instructions to use company knowledge do not replace source ownership or acceptance checks.
Start with one question that matters to ordinary work. Establish its source, locate the break and verify the repair. Use the tools-and-workflows guide for wider source preparation and system purchasing decisions, informed by what this investigation actually found.
Questions about company-specific AI answers
Can a better prompt fix missing company knowledge?
A better prompt can clarify the task or help select an available source. It cannot supply an authoritative company rule that has not been provided or settled. First identify the source that should answer the question and confirm that its use is permitted. Then compare a known-passage attempt with the ordinary query. Keep any successful workaround separate from evidence that the normal retrieval route has been repaired.
Why can I open a document while the assistant cannot answer from it?
Your permission to open a document and the assistant's ability to retrieve usable content are different checks. The configured source route, processing status or file protection may prevent the content from reaching the answer. Ask the technical owner to inspect the exact document and account rather than testing only as an administrator. Follow the access and retrieval checks, keeping required information protections intact.
Does a citation prove that a company answer is correct?
No. Open the cited document and check whether it applies to this question and supports the reply's actual claims. An old checklist, a different branch's procedure or a passage missing a required condition can all leave the answer unsuitable. Name the details a reviewer must verify, and hold the answer when the governing source is unresolved. The rental example shows why finding the right document is only part of acceptance.
Should we replace the model when internal answers fail?
Consider a model change after investigating the actual failure, including how the model uses an applicable passage it is permitted to receive. A replacement will not itself settle conflicting company rules or repair an excluded source. If interpretation remains unreliable, compare candidate arrangements on representative questions and important exceptions with the same acceptance requirements. The failure record gives that comparison a concrete starting point instead of a general impression.
