What are Robots Tags?
Robots tags are directives(rules) that control how search engines crawl and index the content of a website.
Note: The absence of robots tag directives will result in the default "index", "follow" values.
There are others, but here are the directives/rules primarily used for SEO purposes.
| Directive | Meaning |
|---|---|
none |
Equal to noindex, nofollow |
noindex |
Do not show this page in search results and do not show a "Cached" link in search results. |
nofollow |
Do not follow the links on this page. |
all |
There are no restrictions for indexing or serving. Equivalent to index, follow. Note: this directive is the default value and has no effect if explicitly listed. |
Note: The absence of robots tag directives will result in the default "index", "follow" assumption.