Decision Desk · Issue 04 · complete public learning edition
Can Your AI Find the Right Answer?
Decide which internal questions a retrieval system may answer, from which authoritative sources, for which users—and when it must abstain.
Learning time75–90 minutes
Teaching11 cumulative lessons
InstrumentAnswerability Test
DefaultAbstain on authority gaps
Answerability chain
Question → permission → authority → evidence → correction.
Retrieval quality is only one part of a reliable internal answer. Every link must remain inspectable.
Gate
Confirm the user may access the source
Identify current controlling authority
Return proposition-level evidence
Maintain feedback, correction, and retesting
Complete issue curriculum
Eleven cumulative lessons.
Use fictional source descriptions only; do not connect or upload an internal repository.
01Issue 04Define one question class
Open
Define one question class
‘Answer questions from our knowledge’ is not a bounded use. Name the users, question class, consequence, source collection, and prohibited questions.
02Issue 04Establish source authority
Open
Establish source authority
Availability is not authority. Identify the record owner, approval status, scope, effective date, and source that controls when materials disagree.
03Issue 04Inherit permissions correctly
Open
Inherit permissions correctly
Retrieval must not collapse repository, document, paragraph, matter, or role permissions. The answer cannot reveal what the user could not access at the source.
04Issue 04Control freshness and version
Open
Control freshness and version
Record effective dates, supersession, review cadence, and change triggers. An indexed answer can remain fluent after its source becomes wrong.
05Issue 04Resolve conflicting records
Open
Resolve conflicting records
A policy, operating procedure, client promise, and wiki may disagree. Define hierarchy, escalation, and abstention rather than letting similarity choose authority.
06Issue 04Require inspectable evidence
Open
Require inspectable evidence
Return the source, locator, version, and retrieval time needed to inspect the answer. A citation must bind to the proposition, not decorate it.
07Issue 04Design abstention
Open
Design abstention
No authoritative source, permission mismatch, unresolved conflict, missing citation, ambiguity, or high consequence should produce a bounded refusal and next route.
08Issue 04Protect reliance boundaries
Open
Protect reliance boundaries
State what the answer can establish, what it cannot, who must decide, and when a user must verify before acting.
09Issue 04Own feedback and correction
Open
Own feedback and correction
Name who receives challenges, repairs the source, corrects the answer, notifies affected users, preserves the record, and retests.
10Issue 04Evaluate representative failure
Open
Evaluate representative failure
Test stale policy, conflicting authority, inaccessible material, ambiguous identity, missing source, unsupported inference, and a plausible but wrong answer.
11Issue 04Issue a question-class disposition
Open
Issue a question-class disposition
Choose do not answer, repair first, human-led retrieval, or a bounded cited-answer pilot. Do not authorize universal ‘chat with everything.’
Critical stop conditions
A plausible answer still fails.
- no authoritative source or named source owner
- a user/source permission mismatch
- unresolved conflict between controlling records
- an answer without inspectable source and locator
- stale-content risk without a monitoring owner
- no correction, notification, or retest path
- employee, client, privileged, or security-sensitive facts outside the approved class
Interactive instrument · device-local
Knowledge Answerability and Source-Authority Test
Evaluate one internal question class against an approved-source collection. Do not upload documents or enter internal facts, employee matters, client information, credentials, or security secrets.
0/30gates evaluated
Missing authority, permission, freshness, citation, or correction requires abstention.
A fluent answer is not evidence that the system may answer.
A fluent answer is not evidence that the system may answer.
| Question class | Authoritative source exists | User may access source | Source is current | Conflicts are resolved | Answer cites evidence | Correction owner exists |
|---|---|---|---|---|---|---|
| Benefits policy | ||||||
| Security procedure | ||||||
| Client promise | ||||||
| Outdated wiki conflict | ||||||
| Employee case question |
Restoring your local test…
Completion boundary
The educational product and reusable test are complete.
A real system still requires approved sources, exact permission behavior, configuration evidence, representative evaluation, security/privacy review, correction operations, and organizational approval.