Currently Available: Need a skilled Software Developer for your next project?
Categories
SEO, GEO and AEO

How to Inspect AI Answer Sources and Inline Citations

When an AI service returns an answer with source links or inline citation markers, the visible text is only part of the record. The saved API response may also contain structured fields that connect an answer span, meaning a portion of the answer, to a URL or source entry. Those fields help you inspect how the service represented its citations. They do not prove that a source supports the claim or that the claim is true.

To check an answer, save its raw response and use the format for the API that produced it to extract citation records. Match each record to the answer text and source it refers to, then compare the source page with the exact claim. OpenAI, Anthropic, and Google use different response structures, so follow the documentation for the specific API and response format when writing a parser.

Save the response before parsing it

Keep the original structured API response before turning it into HTML, Markdown, or a database record. Save it alongside the prompt, endpoint, model identifier, tool configuration, timestamp, and response or conversation ID. This context helps a reviewer identify which request produced each citation.

A copied answer or screenshot can show links but omit the metadata that connects them to text. A structured response can also include citation details that the display leaves out. If you have only rendered text, do not infer exactly which claim a citation supports unless the saved text has enough information to show the connection.

A URL list does not necessarily include every page the service searched or considered. Treat it as the sources cited in the response unless the API documentation says the list records every page retrieved.

Find citations in the format your API returns

First find the assistant's final text. Then inspect the citation fields attached to that text or its source records. A URL or number in the response is not automatically a citation. Use the citation or annotation types defined in the API documentation to identify citations.

API and response format Where to inspect Useful fields and what they show
OpenAI Responses API response.output[] → assistant message → content[] text item → annotations[] A url_citation annotation includes fields such as url, title, start_index, and end_index. The URL and title identify the cited resource; the indices locate the associated text in that content item. The Responses API reference documents the response fields, and the web search guide describes URL citations.
Anthropic Messages API with web search Assistant content[] blocks and citation data associated with text Web-search citation fields include url, title, and cited_text. The excerpt is limited to up to 150 characters, so it may not include the surrounding context needed to assess a claim. encrypted_index is a reference used in multi-turn conversations, not readable evidence. See the Claude web search documentation.
Anthropic with supplied search-result blocks A citation block that refers to a search_result block Fields such as search_result_index, start_block_index, end_block_index, cited_text, source, and title describe a citation tied to supplied search-result content. Resolve the index according to the citation type; do not assume it works like a web-search citation. The Messages API reference describes these message types.
Gemini GenerateContent API candidates[].groundingMetadata groundingChunks[] holds source details such as web.uri and web.title. groundingSupports[] associates a text segment with one or more groundingChunkIndices[]. The segment offsets are measured in bytes, so use byte-aware boundary checks, especially when the text contains non-ASCII characters. See the GenerateContent API reference.

Google documents the GenerateContent response separately from the Interactions API, which represents tool activity in a timeline. Check the documentation for the endpoint that produced your saved output rather than assuming the same field paths apply to both formats. The Interactions API migration notes describe that difference.

Parse each provider's format separately. A normalized record, which puts data into a consistent structure, can make audits easier to search. Keep the original citation type and fields in that record. The type distinguishes a URL citation from a supplied search-result reference. It also distinguishes both from a file citation.

Map each citation to its answer text and source

For each final assistant text block, extract citation records with their types and resolve them using the provider's rules. With OpenAI, check that each annotation's indices fall within the text item it refers to. With Gemini, match each support record's chunk indices to the groundingChunks[] in the same candidate, then inspect the linked segment. With Anthropic, use the citation type to tell whether the reference points to a web source or a supplied search-result block.

Flag a citation if its source reference is missing, its index falls outside the valid range, or you cannot identify its mapped span with confidence. Do not use one provider's offset rules for another. Gemini's byte offsets need byte-aware checks. Ordinary string slicing can select the wrong span when characters use different numbers of bytes.

A provider-neutral audit record can preserve the shared information while keeping each provider’s original fields available:

for each saved response:
    preserve raw_response and request_metadata

    for each final assistant text block:
        extract text and typed citation records

        for each citation:
            resolve its provider-specific source reference
            resolve any provider-specific answer span
            check source reference and span bounds

            save:
                answer_text
                citation_type
                cited_url_or_source_id
                cited_title
                cited_text_or_segment
                span_or_chunk_mapping
                parse_status
                reviewer_verification

This outline is a parsing method, not a universal API schema. Keep the raw response so you can revisit a mapping if your parser or the provider’s response format changes.

Check the page against the exact claim

After resolving a citation, open its destination and confirm that the page matches the citation. Record any redirects and the date you checked the page, since it can change after the answer was generated. Read the passage in context. Compare the answer's wording and scope with the passage, including its date and qualifications.

Provenance: Does the saved record identify a real source and map it to the intended answer text? Support: Does the cited page substantiate that precise statement? Truth: Is the statement accurate and current when checked against suitable evidence? A citation can pass the provenance check and still fail the support or truth check. For consequential claims, check an authoritative primary source where one exists and seek independent corroboration.

What citation fields establish

Depending on the API, structured citation fields identify a URL or source block, provide a title or excerpt, and link a source to part of the answer. Gemini's optional confidenceScores[] correspond to support references and range from 0 to 1; they do not give the general probability that the answer is true.

Citation fields alone do not show that the service read the whole source, that the cited passage supports the answer, or that the service cited every relevant source. They also do not show whether a source has changed. Missing citation metadata does not prove that the service did not search. A visible citation does not prove that a claim is correct.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com

Email updates

Usually a new article and a few links I found interesting.

No spam. Unsubscribe with one click.

Leave a Reply

Your email address will not be published. Required fields are marked *