Deduplicating Joins Across Citation, Referral, and CRM Data
Website owners comparing citation reports with referrals and customer relationship management (CRM) outcomes need to keep the three record types separate. A…
When a website removes a page, its server must tell browsers and search crawlers what happened to the URL. The HTTP status code is that machine-readable response. Use 404 Not Found when the resource is missing but its future is uncertain. Use 410 Gone when you know the resource was deliberately removed and is likely to stay unavailable. If a similar page has replaced it, redirect the old URL to the replacement instead.
For search engine optimization (SEO), Google treats both 404 and 410 as error responses that keep a URL out of its index. Google’s documentation does not establish that 410 improves rankings or removes a page faster. Choose the response that accurately describes the URL, then check what your server returns.
The HTTP standard defines 404 as a response that says the server cannot find a current version of the requested resource. It does not say whether the absence is temporary or permanent. A 410 says the server knows the resource is unavailable and that the absence is likely permanent. The HTTP semantics standard describes 410 as useful when a site intentionally removes a resource and wants remote links to it cleaned up.
Return 404 when someone mistypes a URL, when the page never existed, or when you do not know whether a missing page will return. Choose 410 when you have intentionally retired the page and do not expect to restore it.
When the content has moved to a genuinely similar destination, use a permanent redirect, such as a 301, to send visitors and crawlers to that page. A redirect tells them where the resource moved. Do not redirect every deleted URL to your home page: an unrelated destination does not represent a replacement for the removed content.
If the page still needs to serve visitors but should not appear in search, keep it available and consider a noindex instruction. Google must be able to crawl the page to see that instruction, so do not block the URL from crawling at the same time. Google explains this requirement in its documentation on blocking search indexing with noindex.
Google treats 404 and 410 as 4xx responses, the HTTP status codes for client errors. It does not index content returned with these responses, and it removes previously indexed URLs over time. Google also says crawling of known URLs that return 4xx responses declines over time. Its HTTP status code guidance does not describe a distinct ranking benefit or faster removal schedule for 410 compared with 404.
Google also calls a 404 a strong signal not to crawl a known URL again. That guidance describes reduced crawling, not a guarantee that Google will never request the URL again. Nor does it establish that choosing 410 changes the outcome. Google recommends either 404 or 410 for removed content without a similar replacement; for a page that has moved or has a clear replacement, it recommends a redirect.
A status code communicates what happened to a URL. It does not guarantee how quickly search results will update, recover traffic, or transfer value from links pointing to the old page. Google processes crawling and indexing over time, and its documentation gives no fixed removal timeline for either response.
A page that displays “not found” does not necessarily return a 404. Some sites show an error message while sending a 200 OK response, which tells crawlers that the request succeeded. Google may classify such a page as a soft 404, meaning it looks like an error page but returns a success status, and exclude it from search. The status code must match the page’s actual state.
Check the exact URL and its response headers, the metadata the server sends with its reply. For example, this command displays the response headers without printing the page body:
curl -sS -D - -o /dev/null https://example.com/old-page
Confirm whether the URL returns 404, 410, or a redirect, and follow the redirect to check its final destination. Check meaningful URL variants too, such as versions with and without a trailing slash. If a content delivery network (CDN) or proxy sits between the origin server and visitors, check what it returns as well. The visitor’s response can differ from the origin’s response.
Then remove the retired URL from your sitemap and update internal links that still point to it. If a relevant replacement exists, link directly to that replacement. For sites with many pages or frequent changes, Google’s crawl budget guidance explains how to keep crawl activity focused. Google says this guidance mainly applies to very large or rapidly changing sites; most sites should keep their sitemaps current and review the Page Indexing report.
Use Search Console’s URL Inspection tool to check what Google received for a specific URL, and review the Page Indexing report for patterns such as soft 404s. If you need to know whether Googlebot requested a particular URL, server logs can help because Search Console does not provide a crawl history that can be filtered by URL or path.
Do not return 404 or 410 during a temporary outage or maintenance period. Those responses tell crawlers that the resource is missing. Google treats server errors differently: its guidance says crawlers slow down temporarily in response to 5xx errors, and persistent errors can eventually lead to indexed URLs being dropped. Make sure the response reflects whether the page was removed or the server is temporarily unable to serve it.
Also account for caching. The HTTP standard allows both 404 and 410 responses to be cached heuristically. If you restore a page after returning one of these codes, check your cache controls and purge relevant CDN or proxy caches. Changing the origin response alone does not ensure that every visitor or crawler immediately receives the new response.
Status codes help crawlers determine whether a URL still serves content they can index. Google’s guidance covers Google Search crawling and indexing. It does not show that AI-generated answers favor 410 over 404 or that either code changes the likelihood of a citation. Return an accurate status and maintain a redirect only when a relevant replacement exists.
Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.
Take a look at vroni.com