How Noindex Tags Affect Yoast XML Sitemaps

On a WordPress site, Yoast SEO uses search-visibility settings to decide which content belongs in its XML sitemaps. An XML sitemap is a file that lists URLs to help search engines discover them. By default, Yoast leaves a page out of its sitemap when it marks the page noindex.

The two signals do different jobs. A noindex directive tells a search engine not to show a page in search results. A sitemap listing helps search engines find a URL; it does not guarantee that they will crawl or index it. To check a page, review its Yoast setting and live output, then inspect the sitemap your site serves.

How Yoast connects search visibility and sitemap entries

Yoast’s search-visibility settings control whether content appears in search results. For an individual post or page, open the Yoast SEO Advanced tab and check Allow search engines to show this content in search results. Selecting No sets the item to noindex; Yoast’s standard sitemap behavior then excludes it. Yoast documents this individual URL setting.

A broader setting controls an entire content type, taxonomy (a way WordPress groups content), or archive. In Yoast SEO → Settings, inspect the relevant section for a control such as Show [type] in search results? Turning off search visibility also removes that content group from Yoast’s sitemap. This affects the group, rather than just one item, as described in Yoast’s guidance on customizing the sitemap index.

Use the individual setting to remove one URL. Use the broader setting to keep an entire content type out of search results and the sitemap. If the Advanced tab is missing, check the WordPress user’s role: Yoast says editors and administrators have access to it by default.

What to check in the generated output

First find out which plugin or WordPress feature generates the sitemap you are viewing. Yoast notes that WordPress core can also provide a sitemap, so a familiar-looking sitemap URL does not by itself confirm that Yoast generated the file. In current Yoast guidance, the sitemap control is under Yoast SEO → Settings → Site features → Technical SEO → XML sitemaps; use its view link to open the Yoast sitemap. See Yoast’s sitemap settings guidance.

Yoast’s sitemap index, a file that links to separate child sitemaps for eligible content, points to the sitemaps for content Yoast can include. Check the index and the relevant child file. Yoast leaves out a child sitemap when it has no eligible entries. If the child file exists, it might still omit the URL you are checking. Yoast’s functional specification says that pages with a noindex directive in a meta tag or HTTP header should be excluded.

Then inspect the page itself. Check its rendered HTML for a robots meta tag or its HTTP response headers for a noindex directive. Google recognizes both forms, and it must be able to crawl the page to read the instruction. If robots.txt blocks crawling, Google might not see the page-level directive. Google’s robots meta tag documentation explains how these directives work.

A sitemap can also carry a noindex header. Yoast says it sends X-Robots-Tag: noindex, follow with its XML sitemap responses so search engines do not index the sitemap file itself. That header applies to the sitemap response, not to every page listed inside it. Check the sitemap entry and the page-level noindex directive separately.

Why a URL may be missing or still appear

A URL missing from the sitemap does not prove that the page is noindex. Yoast also excludes some pages for other reasons, including a manually set canonical URL that differs from the default, redirects, and private or password-protected content. A canonical URL identifies the preferred URL for a page. If an expected URL is missing, check the page’s canonical setting as well as its visibility setting. Yoast describes these cases in its sitemap troubleshooting guidance.

If a URL appears even though you expect it to be noindexed, verify the live page’s directive and confirm that you are inspecting Yoast’s sitemap rather than a sitemap from WordPress core or another plugin. Custom code and plugins can also change Yoast sitemap output. Yoast provides developer filters to exclude posts. They also cover post types and taxonomies; an exclusion changes sitemap output without necessarily adding noindex to the page.

A sitemap entry helps search engines discover a URL, but it does not prove that the URL can be indexed or has been indexed. Google says submitting a sitemap does not guarantee that it will crawl or index the listed URLs. Removing a URL from a sitemap is not a request to remove it from Google search results. Use the page-level directive when search exclusion is the goal, and verify sitemap output separately.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com