MeKat field guide
How to check an AI answer before you use it.
An AI answer is something to examine. Confident wording, a polished table and a list of links do not establish that its claims are correct. Start by separating the claims, then compare each one with a source responsible for the information.
Pick out the claims
Consider this fictional, prepared example. Before making plans, separate the answer into things you could verify. Checking a short list is more useful than asking whether the whole answer sounds right.
The Riverside Library book club meets every Thursday at 6 p.m. No booking is needed, and all ages can attend.
- Does it happen every Thursday?
- Does it start at 6 p.m.?
- Can people attend without booking?
- Is there an age restriction?
Find the source responsible for the information
For an event, start with the organizer’s current event page. For product instructions, start with the maker’s documentation. For a quotation, find the original publication or recording.
A search-result snippet can help locate a page, but open the page itself. Check the organization, event name, location and date. Two similarly named events may have different arrangements.
For this fictional example, imagine that the library’s current listing says the following. It supports one meeting at 6 p.m. It does not support “every Thursday,” “no booking” or “all ages.”
Riverside Library book club: Thursday, October 8, 6–7 p.m. Booking required. Ages 16 and older.
Keep only what the evidence supports
The corrected version keeps the supported details and leaves the unanswered question visible. You do not need to fill every gap.
We can go any Thursday without booking.
The listing gives one meeting on October 8, from 6–7 p.m., for ages 16 and older. Booking is required. I still need to check whether there are other dates.
Follow the link all the way to the claim
If an AI answer includes citations, open them. Does each source actually state the nearby claim? A link to the library homepage could establish that the library exists without proving the book club’s booking rules.
Watch for a narrow source becoming a broad conclusion: one event becomes a weekly schedule, one person’s experience becomes “everyone,” or an old announcement becomes a current rule.
Asking the same AI to confirm its answer can help uncover a mistake, but repetition is not independent evidence. Another AI agreeing does not replace checking original information. OpenAI documents that models can make factual mistakes and invent references.
Decide what to do with uncertainty
If the current page is missing or two official pages disagree, pause the claim. Contact the organizer or say what remains unknown. The useful outcome may be a better question to ask a person, rather than another confident answer.
Your turn
Try it yourself.
An AI says: “Every workshop includes materials and accepts walk-ins.” Your fictional source says: “Saturday drawing workshop: pencils and paper provided. Reserve a place before attending.” Which parts can you keep?
Show answer guidance
Materials are provided for this particular drawing workshop. The source requires a reservation and says nothing about every other workshop. A supported rewrite is: “The Saturday drawing workshop provides pencils and paper. Reserve a place before attending.”
Sources and further reading
The worked examples are original, prepared exercises. These primary sources explain the underlying guidance:
- OpenAI: does ChatGPT tell the truth?
Models can produce inaccurate facts and invented citations; check important information against sources.