Does nofollow prevent indexing and if not, what's the best way?
 by Staff

Does nofollow prevent indexing and if not, what's the best way?

  • Can I prevent Google from indexing a page by adding nofollow to the URL?

Answer:

No — Using nofollow will not prevent Google or any other search engine from indexing the page. Here's the definitive answer to that question from Google's Gary Illyes...

To prevent Google from indexing a page you must use either robots.txt or the noindex meta tags as explained below.

How to use Robots.txt (not the best way)

As you probably know, robots.txt is a plain text file located in the root directory of your site that tells a spider/bot/web crawler which parts of the site should and shouldn’t be crawled. If properly configured, the file prevents search engines from crawling (i.e., scanning) and indexing specific directories (i.e., sub-folders) or pages (i.e., files) on your site.

Whenever you have duplicate content or pages that you wish to keep private, robots.txt can be used to prevent bots from indexing such content. Or, if your site is in staging or maintenance mode, robots.txt can (and should) be used to restrict bots from crawling sections or even all of your entire site.

An easy way to check your robots.txt file is to view it by adding robots.txt after your site’s URL like this...

domain.com/robots.txt

The robots.txt file can usually be edited through your domain-hosting control panel’s file manager. Otherwise you can use an FTP client to access and edit it.

Configuring robots.txt to prevent bots from indexing a subdirectory on your site looks like this:

User-agent: Googlebot 
Disallow: /example-subfolder/

User-agent specifies the type of bot...

TO READ THE FULL ARTICLE