Let's find the work AI should be doing in your organization.
Whether you are exploring an idea or improving a system already in use, we’ll help you decide what is worth doing next.
Thoughtful Comics
Why Shouldn’t I Treat Every Confident AI Answer As A Fact?
Why fluent answers need evidence checks, especially when a claim could have serious consequences.
The takeaways
- 01A confident-sounding answer can still contain an unsupported claim.
- 02Verification needs evidence from sources, tools, or people.
- 03Match the strength of the evidence check to the risk of the claim.
- 04Separate facts from assumptions and inspect the original source.
Read this comic as text
Reader The answer sounded certain. Why is the statistic not real?
Robot Fluent language can come from a likely prediction, even when the claim is unsupported.
Takeaway: A confident-sounding answer can still contain an unsupported claim.
Reader Why not automatically check every claim?
Robot An LLM generates text. Verification requires trusted sources, retrieval tools, or a human check.
Takeaway: Verification needs evidence from sources, tools, or people.
Reader Do I need to verify every sentence?
Robot Match the check to the consequence. High-impact claims need stronger evidence than low-risk ideas.
Takeaway: Match the strength of the evidence check to the risk of the claim.
Reader I will ask for sources and separate facts from assumptions.
Robot Then inspect the original source before you rely on the claim.
Takeaway: Separate facts from assumptions and inspect the original source.
- Hallucination
- A confident-looking AI output that is unsupported, inaccurate, or invented.
- Verification
- Checking a claim against reliable evidence.
- Retrieval
- Finding relevant information from an external source before answering.
- Source
- The original evidence used to support a claim.