{"id":66,"date":"2019-07-18T12:16:30","date_gmt":"2019-07-18T11:16:30","guid":{"rendered":"https:\/\/organicdigital.co\/blog\/?p=66"},"modified":"2026-09-21T12:16:08","modified_gmt":"2026-09-21T11:16:08","slug":"how-to-implement-robots-noindex-tag-with-google-tag-manager","status":"publish","type":"post","link":"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/","title":{"rendered":"How to: Noindex a Page &#8211; A Guide to the Robots Noindex Tag"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><strong>TL;DR:<\/strong> <br>A <code>noindex<\/code> directive tells search engines not to include a page in their search results. For HTML pages, the usual method is a robots meta tag such as <code>&lt;meta name=\"robots\" content=\"noindex\"><\/code>, while non-HTML files can use an <code>X-Robots-Tag<\/code> HTTP header. Don\u2019t block a noindexed page in <code>robots.txt<\/code>, because search engines need to crawl it to see the directive. If you can\u2019t modify the HTML or server configuration directly, Google Tag Manager can also be used as a fallback to add <code>noindex<\/code> with JavaScript.<\/p>\n\n\n\n<div id=\"ez-toc-container\" class=\"ez-toc-v2_0_90 counter-hierarchy ez-toc-counter ez-toc-grey ez-toc-container-direction\">\n<div class=\"ez-toc-title-container\">\n<p class=\"ez-toc-title\" style=\"cursor:inherit\">Table of Contents<\/p>\n<span class=\"ez-toc-title-toggle\"><a href=\"#\" class=\"ez-toc-pull-right ez-toc-btn ez-toc-btn-xs ez-toc-btn-default ez-toc-toggle\" aria-label=\"Toggle Table of Content\"><span class=\"ez-toc-js-icon-con\"><span class=\"\"><span class=\"eztoc-hide\" style=\"display:none;\">Toggle<\/span><span class=\"ez-toc-icon-toggle-span\"><svg style=\"fill: #999;color:#999\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" class=\"list-377408\" width=\"20px\" height=\"20px\" viewBox=\"0 0 24 24\" fill=\"none\"><path d=\"M6 6H4v2h2V6zm14 0H8v2h12V6zM4 11h2v2H4v-2zm16 0H8v2h12v-2zM4 16h2v2H4v-2zm16 0H8v2h12v-2z\" fill=\"currentColor\"><\/path><\/svg><svg style=\"fill: #999;color:#999\" class=\"arrow-unsorted-368013\" xmlns=\"http:\/\/www.w3.org\/2000\/svg\" width=\"10px\" height=\"10px\" viewBox=\"0 0 24 24\" version=\"1.2\" baseProfile=\"tiny\"><path d=\"M18.2 9.3l-6.2-6.3-6.2 6.3c-.2.2-.3.4-.3.7s.1.5.3.7c.2.2.4.3.7.3h11c.3 0 .5-.1.7-.3.2-.2.3-.5.3-.7s-.1-.5-.3-.7zM5.8 14.7l6.2 6.3 6.2-6.3c.2-.2.3-.5.3-.7s-.1-.5-.3-.7c-.2-.2-.4-.3-.7-.3h-11c-.3 0-.5.1-.7.3-.2.2-.3.5-.3.7s.1.5.3.7z\"\/><\/svg><\/span><\/span><\/span><\/a><\/span><\/div>\n<nav><ul class='ez-toc-list ez-toc-list-level-1 ' ><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-1\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#What_you_need_to_know_before_using_noindex\" >What you need to know before using noindex<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-2\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#What_is_a_noindex_tag\" >What is a noindex tag?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-3\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#Noindex_vs_nofollow_whats_the_difference\" >Noindex vs nofollow: what&#8217;s the difference?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-4\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#Which_pages_should_you_noindex\" >Which pages should you noindex?<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-5\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#Dont_use_noindex_as_a_substitute_for_canonicalisation\" >Don&#8217;t use noindex as a substitute for canonicalisation<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-6\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#How_to_add_a_noindex_tag_to_a_page\" >How to add a noindex tag to a page<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-7\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#How_to_noindex_non-HTML_files_with_X-Robots-Tag\" >How to noindex non-HTML files with X-Robots-Tag<\/a><ul class='ez-toc-list-level-3' ><li class='ez-toc-heading-level-3'><a class=\"ez-toc-link ez-toc-heading-8\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#Dont_block_a_noindexed_page_in_robotstxt\" >Don&#8217;t block a noindexed page in robots.txt<\/a><\/li><\/ul><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-9\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#How_to_add_a_noindex_tag_with_Google_Tag_Manager\" >How to add a noindex tag with Google Tag Manager<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-10\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#A_real_example_of_implementing_noindex_with_GTM\" >A real example of implementing noindex with GTM<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-11\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#Create_the_noindex_tag_in_Google_Tag_Manager\" >Create the noindex tag in Google Tag Manager<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-12\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#Choose_which_pages_should_fire_the_noindex_tag\" >Choose which pages should fire the noindex tag<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-13\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#Test_the_noindex_tag_before_publishing\" >Test the noindex tag before publishing<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-14\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#How_to_check_the_noindex_tag_is_working\" >How to check the noindex tag is working<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-15\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#Should_you_use_Google_Tag_Manager_to_add_noindex\" >Should you use Google Tag Manager to add noindex?<\/a><\/li><li class='ez-toc-page-1 ez-toc-heading-level-2'><a class=\"ez-toc-link ez-toc-heading-16\" href=\"https:\/\/daveashworth.co\/blog\/how-to-implement-robots-noindex-tag-with-google-tag-manager\/#Final_thoughts_on_using_noindex\" >Final thoughts on using noindex<\/a><\/li><\/ul><\/nav><\/div>\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_you_need_to_know_before_using_noindex\"><\/span>What you need to know before using noindex<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">If you don&#8217;t want a page to appear in Google&#8217;s search results, one of the main ways to control this is with a <code>noindex<\/code> directive.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The most common implementation is a robots meta tag added to the <code>&lt;head&gt;<\/code> of the page:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>&lt;meta name=\"robots\" content=\"noindex\"&gt;<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Once Google crawls the page and sees the directive, the page can be removed from Google&#8217;s search results.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But there are several important details to understand before adding <code>noindex<\/code> across a website. You need to know which pages should actually be noindexed, the difference between <code>noindex<\/code> and <code>nofollow<\/code>, why blocking a page in <code>robots.txt<\/code> isn&#8217;t the same thing, and how to check that Google can see your directive.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">And sometimes you can&#8217;t edit the HTML or CMS templates directly. I&#8217;ve encountered exactly that problem when working as a <a href=\"https:\/\/daveashworth.co\/\">website optimisation consultant<\/a>, so later in this guide I&#8217;ll also show the Google Tag Manager workaround I&#8217;ve used to add a robots <code>noindex<\/code> tag when the normal implementation wasn&#8217;t available.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"What_is_a_noindex_tag\"><\/span>What is a noindex tag?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A <code>noindex<\/code> directive tells search engines that a page should not be included in their search results.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The most common way to implement it is with a robots meta tag in the <code>&lt;head&gt;<\/code> of the page:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&lt;meta name=\"robots\" content=\"noindex\"&gt;<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">You can also target Google specifically using:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&lt;meta name=\"googlebot\" content=\"noindex\"&gt;<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">In most situations, however, the general <code>robots<\/code> directive makes more sense because it applies to search engines that support it rather than targeting Google alone.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It&#8217;s important to understand that <code>noindex<\/code> doesn&#8217;t stop a search engine from <strong>crawling<\/strong> the URL. The search engine needs to be able to crawl the page and discover the directive before it can act on it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is why <code>noindex<\/code> and <code>robots.txt<\/code> shouldn&#8217;t be treated as interchangeable. If you block a URL from being crawled in <code>robots.txt<\/code>, Google may be unable to access the page and see the <code>noindex<\/code> directive you&#8217;ve added to it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Noindex_vs_nofollow_whats_the_difference\"><\/span>Noindex vs nofollow: what&#8217;s the difference?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><code>noindex<\/code> and <code>nofollow<\/code> control different things.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>noindex<\/code> tells a search engine not to show the page in its search results. <code>nofollow<\/code> tells supported search engines not to follow the links on the page.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">You&#8217;ll often see older examples written like this:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&lt;meta name=\"robots\" content=\"noindex, follow\"&gt;<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">However, Google treats <code>follow<\/code> as the default behaviour, so you don&#8217;t normally need to specify it. For Google, this is generally sufficient:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&lt;meta name=\"robots\" content=\"noindex\"&gt;<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Similarly, don&#8217;t automatically use <code>noindex, nofollow<\/code> just because you don&#8217;t want a page indexed. If links on that page are useful for discovering other pages on your site, <code>nofollow<\/code> is a separate decision.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Which_pages_should_you_noindex\"><\/span>Which pages should you noindex?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A <code>noindex<\/code> directive is useful when a page needs to exist for users or site functionality but <strong>doesn&#8217;t need to appear as a standalone result in search engines<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Examples can include internal search results, account or login pages, staging or utility pages, certain filtered or parameterised URLs, and duplicate or near-duplicate pages that you deliberately don&#8217;t want appearing in search.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But I wouldn&#8217;t automatically noindex a page simply because it looks thin, is paginated, belongs to a taxonomy or contains URL parameters.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Some category, tag, filter and pagination URLs can serve a useful search or discovery purpose. The decision should be based on <strong>what the page does, whether there&#8217;s search demand for it, whether it provides unique value, and whether you actually want it competing for visibility in Google<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Before adding <code>noindex<\/code>, I&#8217;d therefore ask a fairly simple question:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Would I ever want somebody to land on this page from Google?<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If the answer is genuinely no, <code>noindex<\/code> may be appropriate. If the answer is yes, or potentially yes, removing it from the index probably isn&#8217;t the solution.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Dont_use_noindex_as_a_substitute_for_canonicalisation\"><\/span>Don&#8217;t use noindex as a substitute for canonicalisation<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\"><code>noindex<\/code> and <code>rel=\"canonical\"<\/code> also solve different problems.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A canonical tag indicates which URL you consider representative when multiple URLs contain the same or very similar content. <code>noindex<\/code> tells search engines that you don&#8217;t want a particular page appearing in search results.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So I wouldn&#8217;t automatically <code>noindex<\/code> every duplicate URL that has a canonical pointing elsewhere. First decide whether you&#8217;re trying to <strong>consolidate duplicate URLs<\/strong> or <strong>remove a page from search altogether<\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That distinction is particularly important on ecommerce sites and other large websites where filters, parameters and faceted navigation can create many variations of the same underlying content.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_to_add_a_noindex_tag_to_a_page\"><\/span>How to add a noindex tag to a page<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The simplest and most reliable way to noindex an HTML page is to add a robots meta tag directly to the <code>&lt;head&gt;<\/code> of the page:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&lt;meta name=\"robots\" content=\"noindex\"&gt;<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Once a search engine crawls the page and processes the directive, it can remove the URL from its search results.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you&#8217;re using a CMS such as WordPress, you may not need to edit the HTML directly. Your SEO plugin or CMS may provide a page-level setting that adds the robots meta tag for you.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Whichever method you use, check the rendered HTML afterwards to make sure the directive is actually present on the pages you intended to noindex.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_to_noindex_non-HTML_files_with_X-Robots-Tag\"><\/span>How to noindex non-HTML files with X-Robots-Tag<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A robots meta tag works for HTML pages because it can be added to the page&#8217;s <code>&lt;head&gt;<\/code>. For other resources, such as PDF files, you can use the <code>X-Robots-Tag<\/code> HTTP response header instead.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>X-Robots-Tag: noindex<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">This allows you to apply a <code>noindex<\/code> directive at HTTP header level rather than adding HTML to the resource itself.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>X-Robots-Tag<\/code> can also be used with HTML pages, although for a normal webpage I&#8217;d generally use the robots meta tag unless there&#8217;s a technical reason to control indexing through the server response.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Dont_block_a_noindexed_page_in_robotstxt\"><\/span>Don&#8217;t block a noindexed page in robots.txt<span class=\"ez-toc-section-end\"><\/span><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">If you want Google to discover a <code>noindex<\/code> directive, it needs to be able to crawl the URL.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Don&#8217;t add <code>noindex<\/code> to a page and then block the same URL in <code>robots.txt<\/code>. If Google can&#8217;t crawl the page, it may not be able to see the <code>noindex<\/code> directive.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_to_add_a_noindex_tag_with_Google_Tag_Manager\"><\/span>How to add a noindex tag with Google Tag Manager<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Adding <code>noindex<\/code> directly to the page or through your CMS is preferable because the directive is present in the HTML when the page is served.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But sometimes you don&#8217;t have that level of access.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I&#8217;ve worked on sites where technical limitations meant I couldn&#8217;t easily add or change robots directives on the specific URLs I wanted to control. In situations like that, Google Tag Manager can provide a workaround by adding the robots meta tag using JavaScript.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">I wouldn&#8217;t use GTM as my first choice for implementing <code>noindex<\/code>, but it&#8217;s a useful option when changing the underlying HTML or CMS isn&#8217;t practical.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"A_real_example_of_implementing_noindex_with_GTM\"><\/span><strong>A real example of implementing noindex with GTM<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">To demonstrate how this works, I originally used the <a href=\"https:\/\/daveashworth.co\/contact\/\">contact page on this site<\/a> as a guinea pig.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Before making the change, the page was indexed by Google. I could confirm this in Google Search Console and by checking the robots directives being output on the page.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" width=\"772\" height=\"354\" src=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/site-contact.png\" alt=\"site search command\" class=\"wp-image-68\" srcset=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/site-contact.png 772w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/site-contact-150x69.png 150w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/site-contact-300x138.png 300w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/site-contact-768x352.png 768w\" sizes=\"(max-width: 772px) 100vw, 772px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">You can also use Google Search Console to check a page\u2019s\nindexed status:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" width=\"644\" height=\"423\" src=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/gsc-status.png\" alt=\"GSC Status\" class=\"wp-image-69\" srcset=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/gsc-status.png 644w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/gsc-status-150x99.png 150w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/gsc-status-300x197.png 300w\" sizes=\"(max-width: 644px) 100vw, 644px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">When viewing the source code, you can see the robots index\ntag in place:<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" width=\"544\" height=\"323\" src=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/view-source.png\" alt=\"Inspect Code\" class=\"wp-image-70\" srcset=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/view-source.png 544w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/view-source-150x89.png 150w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/view-source-300x178.png 300w\" sizes=\"(max-width: 544px) 100vw, 544px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Create_the_noindex_tag_in_Google_Tag_Manager\"><\/span><strong>Create the noindex tag in Google Tag Manager<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This assumes Google Tag Manager is already installed on the site. If it isn&#8217;t, implement and test GTM before using it to control something as important as indexing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Create a new <strong>Custom HTML<\/strong> tag containing the JavaScript that adds the robots meta tag to the document <code>&lt;head&gt;<\/code>.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>&lt;script&gt;\n  var metaRobots = document.querySelector('meta&#91;name=\"robots\"]');\n\n  if (!metaRobots) {\n    metaRobots = document.createElement('meta');\n    metaRobots.setAttribute('name', 'robots');\n    document.head.appendChild(metaRobots);\n  }\n\n  metaRobots.setAttribute('content', 'noindex');\n&lt;\/script&gt;<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">This script first checks whether the page already contains a robots meta tag.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If one doesn&#8217;t exist, it creates one and adds it to the <code>&lt;head&gt;<\/code> of the page. It then sets the content of the robots tag to <code>noindex<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is preferable to my original implementation, which simply removed the existing robots tag and created a new one using jQuery. The updated version doesn&#8217;t require jQuery and handles both situations, whether a robots meta tag already exists or needs to be created.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Google can process robots meta tags added using JavaScript, but I still consider this a fallback rather than the preferred way to implement <code>noindex<\/code>. If you can add the directive directly through the HTML, CMS or HTTP response, I&#8217;d do that instead. Google itself recommends avoiding JavaScript for changing meta tags where possible.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" width=\"694\" height=\"481\" src=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/custom-html-1.png\" alt=\"Custom HTML\" class=\"wp-image-2330\" srcset=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/custom-html-1.png 694w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/custom-html-1-150x104.png 150w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/custom-html-1-300x208.png 300w\" sizes=\"(max-width: 694px) 100vw, 694px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Choose_which_pages_should_fire_the_noindex_tag\"><\/span><strong>Choose which pages should fire the noindex tag<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The trigger is the most important part of this implementation because it determines exactly which pages receive the <code>noindex<\/code> directive.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Create a trigger that only matches the URLs you deliberately want to remove from search.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For example, if you wanted the tag to fire on every URL within:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>\/blog\/tags\/<\/code><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">you could create a condition where <strong>Page Path contains <code>\/blog\/tags\/<\/code><\/strong>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you only want to noindex a specific page, use a more precise condition that matches that page rather than a broad URL pattern.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Do not configure the tag to fire on every page.<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A mistake here could add <code>noindex<\/code> across large parts of the site, potentially removing important pages from Google&#8217;s search results.<\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" width=\"922\" height=\"367\" src=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/trigger-configuration.png\" alt=\"Trigger Configuration\" class=\"wp-image-72\" srcset=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/trigger-configuration.png 922w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/trigger-configuration-150x60.png 150w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/trigger-configuration-300x119.png 300w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/trigger-configuration-768x306.png 768w\" sizes=\"(max-width: 922px) 100vw, 922px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Test_the_noindex_tag_before_publishing\"><\/span><strong>Test the noindex tag before publishing<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">I cannot stress this enough, test the tag thoroughly before publishing it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Use GTM&#8217;s <strong>Preview<\/strong> mode to check pages that should receive the <code>noindex<\/code> directive and, just as importantly, pages that shouldn&#8217;t.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Once you have tested the arse off this across various pages and are happy that the tag only fires on the desired pages, publish your changes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"How_to_check_the_noindex_tag_is_working\"><\/span><strong>How to check the noindex tag is working<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Because the robots meta tag is being added with JavaScript, you won&#8217;t see the change if you simply use <strong>View Source<\/strong>. That shows the HTML originally returned by the server, before Google Tag Manager has modified the page.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Instead, open your browser&#8217;s developer tools and inspect the rendered HTML. Look inside the <code>&lt;head&gt;<\/code> and check that the robots meta tag has been added:<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><code>&lt;meta name=\"robots\" content=\"noindex\"&gt;<\/code><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img decoding=\"async\" width=\"1024\" height=\"398\" src=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/inspect-source-code-1024x398.png\" alt=\"Inspecting Source Code\" class=\"wp-image-76\" srcset=\"https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/inspect-source-code-1024x398.png 1024w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/inspect-source-code-150x58.png 150w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/inspect-source-code-300x117.png 300w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/inspect-source-code-768x299.png 768w, https:\/\/daveashworth.co\/blog\/wp-content\/uploads\/2019\/07\/inspect-source-code.png 1345w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Once the tag is live, you can also use <strong>URL Inspection in Google Search Console<\/strong> to check how Google sees the page.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Remember that adding <code>noindex<\/code> doesn&#8217;t instantly remove a URL from Google&#8217;s index. Google needs to crawl the page and process the directive first. If you&#8217;ve made the change to an individual URL, you can use URL Inspection to request recrawling.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Once Google has processed the directive, Search Console may report the URL as excluded because of a <code>noindex<\/code> tag.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If the page remains indexed, check that Google can crawl the URL, that it isn&#8217;t blocked by <code>robots.txt<\/code>, and that the <code>noindex<\/code> directive is present in the rendered page when Google processes it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Should_you_use_Google_Tag_Manager_to_add_noindex\"><\/span>Should you use Google Tag Manager to add noindex?<span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">If you can add a robots meta tag directly to the HTML, configure it through your CMS, or use an <code>X-Robots-Tag<\/code> HTTP header, I&#8217;d choose one of those methods before using Google Tag Manager.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">GTM is useful when those options aren&#8217;t available, and I&#8217;ve successfully used it as a workaround on sites where I didn&#8217;t have sufficient control over the underlying templates.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">But you&#8217;re using JavaScript to control whether pages can appear in search, so there are more moving parts and more opportunities for something to go wrong.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That&#8217;s why the trigger configuration and testing are so important. A GTM rule that accidentally adds <code>noindex<\/code> to the wrong pages can cause considerably more damage than the indexing problem you were trying to solve.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><span class=\"ez-toc-section\" id=\"Final_thoughts_on_using_noindex\"><\/span><strong>Final thoughts on using noindex<\/strong><span class=\"ez-toc-section-end\"><\/span><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><code>noindex<\/code> is a useful way to control which pages you want appearing in search results, but it isn&#8217;t something I&#8217;d apply indiscriminately to every duplicate, thin, filtered or parameterised page.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">First decide whether the page genuinely has no value as a search result. If it doesn&#8217;t, add the <code>noindex<\/code> directive using the simplest and most reliable method available to you.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">For most HTML pages, that&#8217;s a robots meta tag. For non-HTML resources such as PDFs, an <code>X-Robots-Tag<\/code> HTTP header may be more appropriate. And if you can&#8217;t modify the page or server configuration directly, Google Tag Manager can provide a useful workaround.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Whichever method you use, test the implementation carefully and make sure you&#8217;re only excluding the URLs you actually intend to remove from search.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you&#8217;re dealing with a larger indexing problem and aren&#8217;t sure which pages should and shouldn&#8217;t be appearing in Google, that&#8217;s something I regularly investigate as part of my <a href=\"https:\/\/daveashworth.co\/services-and-consultancy\/\">website optimisation and SEO consultancy<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Technical SEO changes can often be difficult to implement without developer support.  Use Google Tag Manager to add code such as robots meta tags to a website with ease.<\/p>\n","protected":false},"author":1,"featured_media":67,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,4,13],"tags":[],"class_list":["post-66","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tag-manager","category-crawl-rate","category-on-page-seo"],"_links":{"self":[{"href":"https:\/\/daveashworth.co\/blog\/wp-json\/wp\/v2\/posts\/66","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/daveashworth.co\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/daveashworth.co\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/daveashworth.co\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/daveashworth.co\/blog\/wp-json\/wp\/v2\/comments?post=66"}],"version-history":[{"count":0,"href":"https:\/\/daveashworth.co\/blog\/wp-json\/wp\/v2\/posts\/66\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/daveashworth.co\/blog\/wp-json\/wp\/v2\/media\/67"}],"wp:attachment":[{"href":"https:\/\/daveashworth.co\/blog\/wp-json\/wp\/v2\/media?parent=66"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/daveashworth.co\/blog\/wp-json\/wp\/v2\/categories?post=66"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/daveashworth.co\/blog\/wp-json\/wp\/v2\/tags?post=66"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}