Skip to main content

Stop search indexing for Netlify Deploy Previews and Branch Deploys

Posted in Development and Serverless

Now that you’re cool with Netlify Deploy Previews and Branch Deploys, you might be worried that search engines will start indexing them and you end up with duplicate content problems.

Although Deploy Previews and Branch Deploys are unlikely to be linked to publicly, there’s a small chance one might make it out into the wild web; after all, I linked to one in my Deploy Previews article

Here’s how to fix it:

Add a command to your npm scripts

Search engines use a robots.txt file to find out what they should and shouldn’t be indexing. I have an npm script called noIndex:

scripts": {
"build": "this is whatever my build command is",
"noIndex": "npm run build && echo 'User-agent: *\nDisallow: /' > dist/robots.txt"
},

This builds the site as normal, then overwrites the contents of my default production robots.txt file with instructions not to allow any pages to be indexed.

Run the command for Deploy Previews

Then all you have to do is tell Netlify to deploy using that command instead of the default build command you’ll’ve set up for your production site:

[context.deploy-preview]
command = "npm run noIndex"

[context.branch-deploy]
command = "npm run noIndex"

Checking it all works

In the Netlify control panel, head to Deploys and keep an eye on the deploy log for your Deploy Preview or Branch Deploy. As the deployment rus, you should spot something like this if it has worked:

Different build command detected, going to use the one specified in the toml file: ‘npm run noIndex’ versus ‘npm run build’ in the site

Then, if you open your deploy preview or staging site in your browser and navigate to https://deploy-preview-or-staging-subomain.netlify.com/robots.txt, you’ll see it looks like this:

User-agent: *
Disallow: /

So now you don’t have to worry about anything other than your production site being indexed by search engines!

Get them delivered!

If you enjoyed this and want all the latest articles delivered to your inbox every month, pop your email in the form below.

I don’t collect any data on when, where or if people open the emails I send them. Your email will only be used to send you newsletters and will never be passed on. You can unsubscribe at any time.

More posts

Here are a couple more posts for you to enjoy. If that's not enough, have a look at the full list.

  1. Respecting people’s privacy

    Collecting data, whether needed or not, is high on a lot of companies’ agendas. Me, I don’t track users and I collect the bare minimum data.

  2. Accessibility issues when removing list markers

    If we remove the list markers from an ordered or unordered list, we’re likely to run into some issues with VoiceOver.