Skip to main content

Stop search indexing for Netlify Deploy Previews and Branch Deploys

Posted in Development and Serverless

Now that you’re cool with Netlify Deploy Previews and Branch Deploys, you might be worried that search engines will start indexing them and you end up with duplicate content problems.

Although Deploy Previews and Branch Deploys are unlikely to be linked to publicly, there’s a small chance one might make it out into the wild web; after all, I linked to one in my Deploy Previews article

Here’s how to fix it:

Add a command to your npm scripts

Search engines use a robots.txt file to find out what they should and shouldn’t be indexing. I have an npm script called noIndex:

scripts": {
"build": "this is whatever my build command is",
"noIndex": "npm run build && echo 'User-agent: *\nDisallow: /' > dist/robots.txt"
},

This builds the site as normal, then overwrites the contents of my default production robots.txt file with instructions not to allow any pages to be indexed.

Run the command for Deploy Previews

Then all you have to do is tell Netlify to deploy using that command instead of the default build command you’ll’ve set up for your production site:

[context.deploy-preview]
command = "npm run noIndex"

[context.branch-deploy]
command = "npm run noIndex"

Checking it all works

In the Netlify control panel, head to Deploys and keep an eye on the deploy log for your Deploy Preview or Branch Deploy. As the deployment rus, you should spot something like this if it has worked:

Different build command detected, going to use the one specified in the toml file: ‘npm run noIndex’ versus ‘npm run build’ in the site

Then, if you open your deploy preview or staging site in your browser and navigate to https://deploy-preview-or-staging-subomain.netlify.com/robots.txt, you’ll see it looks like this:

User-agent: *
Disallow: /

So now you don’t have to worry about anything other than your production site being indexed by search engines!

Subscribe

I send a newsletter on the last day of every month, containing:

  • A roundup of the articles I’ve posted
  • A hot pick from my archives
  • Some interesting posts from around the web

I don’t collect any data on when, where or if people open the emails I send them. Your email will only be used to send you newsletters and will never be passed on. You can unsubscribe at any time.

More posts

Here are a couple more posts for you to enjoy. If that’s not enough, have a look at the full list.

  1. It’s good to make mistakes

    As a designer and developer I’ve made countless mistakes, but that’s part of the reason I’m good at what I do.

  2. Focus appearance explained

    There’s some great stuff coming up in WCAG 2.2, but there’s one rule that’s particularly difficult to understand, so here it is in a bit more detail.