What are Robots Tags?

Robots tags are directives(rules) that control how search engines crawl and index the content of a website.

Note:  The absence of robots tag directives will result in the default "index", "follow" values.

There are others, but here are the directives/rules primarily used for SEO purposes.

Directive Meaning
none Equal to noindex, nofollow
noindex Do not show this page in search results and do not show a "Cached" link in search results.
nofollow Do not follow the links on this page.
all There are no restrictions for indexing or serving. Equivalent to index, follow.
Note: this directive is the default value and has no effect if explicitly listed.


Note:  The absence of robots tag directives will result in the default "index", "follow" assumption.