One Question, Three Conflicting Answers in Your Docs

The Problem Is Rarer Than You Think — and More Damaging

Your pricing page says the free plan includes five seats. Your onboarding guide says three. A blog post from eighteen months ago says unlimited seats during beta. A customer asks your AI agent how many seats they get, and the answer depends on which document surfaces first.

This isn't a documentation hygiene problem in the abstract. It's a live support liability. A customer who got the wrong seat count from your agent will escalate, and the escalation will land on a human who has to backpedal.

Rewriting all three docs immediately sounds like the right fix. It usually isn't, because you often don't know which version is correct until you've talked to Product, checked the billing system, and confirmed what's actually enforced. That conversation takes time you don't have right now.

Here's how we handle it operationally.

Step One: Identify the Conflict Before It Fires

Chattering surfaces the sources it used to construct every answer. When a conversation is flagged — either by a human reviewer or by a low-confidence signal — open the source citations. If you see two or three docs cited in the same answer with contradictory details, that's your signal.

You can also catch this proactively. Run your most common questions through the agent in a staging or test workspace. Any question about pricing, limits, permissions, or integration specs is a candidate for conflict. Flag each one manually if the answer cites more than one doc on the same factual point.

Step Two: Create a Canonical Answer in the Knowledge Layer

Don't touch the source docs yet. Instead, write a single authoritative answer and add it directly to Chattering's knowledge base as a custom entry. This entry sits above your connected docs in the retrieval priority order.

Be explicit in the entry. Write something like: As of [current quarter], the free plan includes five seats. Earlier documentation may reference different limits; the current limit is five. That framing does two things: it gives the agent a clear answer to return, and it acknowledges the inconsistency so a customer who has read conflicting docs doesn't feel gaslit.

Pin that entry to the relevant question intents. In Chattering, you can associate a knowledge entry with specific query patterns so it fires reliably on the questions you've identified.

Step Three: Suppress or Deprioritize the Conflicting Sources

A canonical entry helps, but if the three conflicting docs are still live and connected, the agent may still pull fragments from them and blend them into the answer. Use Chattering's source management to either disconnect the specific docs from retrieval or mark the conflicting sections with an exclusion note in the doc itself (a pattern like <!-- chattering-exclude --> on the outdated paragraph, depending on your doc format).

If the docs live in a shared help center or on a third-party platform where you can't annotate individual sections, the cleaner move is to temporarily disconnect the whole document from the agent and rely on the canonical entry until the doc is updated.

Step Four: File the Doc Rewrite as a Tracked Task, Not a Someday Item

This is where most teams drop the ball. The canonical entry is a patch, not a fix. If it lives in the knowledge base indefinitely while the source docs stay wrong, you're accumulating invisible debt — the next person who edits one of those docs won't know the canonical entry exists, and the conflict will re-emerge.

Create a ticket or task immediately. Link to all three conflicting docs and the canonical entry you created. Assign it to whoever owns documentation. Give it a due date no more than two weeks out. If you're using Chattering's shared inbox, you can flag conversations related to this topic so the ticket author can see real customer language before they rewrite.

Step Five: Validate the Fix Before Closing the Loop

After the canonical entry is live, test the original question again. Check the agent's answer and the sources it cites. If the canonical entry is the only source surfaced, you're done for now. If conflicting docs still appear in citations, revisit your suppression settings.

Once the source docs are rewritten and republished, reconnect them, remove the canonical entry, and test again. The goal is to get back to a state where the docs themselves are the source of truth and the knowledge layer isn't doing remedial work.

Why This Sequence Works

The canonical-entry-first approach keeps your agent accurate while your team does the slower, careful work of fixing the underlying docs. It also forces you to commit to a single answer before you touch anything — which is exactly the conversation that should happen with Product or your billing team anyway.

Rewriting all three docs simultaneously without that conversation risks enshrining a different wrong answer in all three places. Slow down the doc work; speed up the patch.