Building a Search Console API Integration for Internal Tools
For an internal tool that reports website search performance, a Search Console integration provides Google’s data on how a site appears in…
When Search Console sends performance data to BigQuery, it writes each day's data into date-based partitions, which keep each date's rows together. Search Console exports its site-level and URL-level performance tables separately, so a partition in one table does not confirm that the other table has finished exporting that day.
Use the ExportLog table to check the latest successful export recorded for each table and date. Each log entry includes an epoch_version, which increases when Search Console republishes data for that table and date, and a publish_time, which records when that export completed. Together, these fields show the latest export recorded when you check. They do not guarantee that Search Console will never revise the date again.
Search Console exports daily, but the export time varies and can differ between tables. The data_date field identifies the reporting day in Pacific Time; publish_time records when Search Console completed the export. Check the two expected table names separately: searchdata_site_impression and searchdata_url_impression.
The following query returns the latest successful entry recorded for each table on one date. Replace the project and dataset names, and set target_day to the Search Console reporting date you want to inspect.
DECLARE target_day DATE DEFAULT DATE '2026-11-15';
WITH expected_tables AS (
SELECT namespace
FROM UNNEST([
'searchdata_site_impression',
'searchdata_url_impression'
]) AS namespace
),
latest_success AS (
SELECT
namespace,
ARRAY_AGG(
STRUCT(epoch_version, publish_time)
ORDER BY epoch_version DESC
LIMIT 1
)[SAFE_OFFSET(0)] AS latest
FROM <code>your-project.your_dataset.ExportLog
WHERE agenda = 'SEARCHDATA'
AND data_date = target_day
GROUP BY namespace
)
SELECT
target_day AS data_date,
e.namespace,
l.latest.epoch_version AS latest_epoch_version,
l.latest.publish_time AS latest_publish_time,
l.latest IS NOT NULL AS has_successful_export
FROM expected_tables AS e
LEFT JOIN latest_success AS l
USING (namespace)
ORDER BY e.namespace;
A successful result for both table names confirms that Search Console recorded an export for each table and date. Epoch 0 marks the first saved version; a higher epoch records a later update. Treat the reported epoch and publish time as the latest successful state recorded when you ran the query.
If has_successful_export is false for a table, the query shows that no successful export is recorded for that table and date. It does not tell you whether the export is still pending, failed, or will be abandoned. Search Console does not record unsuccessful attempts in ExportLog; check the bulk export status in Search Console settings and use Cloud Logs Explorer to investigate errors. For failure recovery details, see how Search Console handles failed bulk exports.
A missing partition or log entry alone does not show that an export failed. The two performance tables export separately, and their schedules do not share a guaranteed completion time. A date with a successful entry for one table can still lack a successful entry for the other.
Search Console retries transient errors, meaning temporary errors such as connection problems, immediately, without notifying you about them. After a write error that is not temporary, Search Console retries during a later scheduled export and includes missed data it still retains. Google says it retains a failed day for about a week before dropping that day’s data. After about a month of failed attempts, it stops the bulk export. These are approximate limits, so investigate a missing date promptly rather than treating them as exact cutoffs. See Google’s bulk export monitoring and error guidance.
BigQuery’s INFORMATION_SCHEMA.PARTITIONS view shows partition IDs, row counts, and the most recent write time. Use it to confirm that a partition exists and to inspect when BigQuery last wrote to it. These details do not replace ExportLog: a recent write time is not a Search Console-specific completion or finality flag.
DECLARE target_day DATE DEFAULT DATE '2026-11-15';
SELECT
table_name,
partition_id,
total_rows,
last_modified_time
FROM <code>your-project.your_dataset.INFORMATION_SCHEMA.PARTITIONS
WHERE table_name IN (
'searchdata_site_impression',
'searchdata_url_impression'
)
AND partition_id = FORMAT_DATE('%Y%m%d', target_day)
ORDER BY table_name;
Compare the metadata with the log entries for the same date and tables. If a partition appears but its table has no matching successful log entry, investigate the discrepancy instead of treating the partition alone as proof of a completed export. BigQuery documents the fields in its INFORMATION_SCHEMA.PARTITIONS view.
Search Console can correct data it has already exported. Its documentation says it updates the affected day’s data atomically in a single commit and records the correction in ExportLog with an incremented epoch. That behavior means historical updates should not be assumed to arrive only as appended rows.
The documentation does not establish that Search Console revises both performance tables in one shared transaction. Check the epoch for each table and date independently. Also avoid treating a row count or metric total as a completeness test: Search Console performance data can contain repeated keys, and the row count alone does not indicate whether a date reflects its latest published version. For query design, see how to preserve Search Console aggregation logic in BigQuery.
A date is validated as the latest successfully exported state recorded for a specific table when ExportLog shows its highest epoch and corresponding publish time. Check both expected tables before treating the date as fully confirmed for the export.
Search Console documents rare historical corrections, but it provides no fixed waiting period after which a date is guaranteed never to change. Record validation results as “latest version recorded as of” the check, rather than as an irreversible final state. If a later correction appears, the higher epoch and new publish time show that Search Console published a newer version.
Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.
Take a look at vroni.com