About Features Downloads Getting Started Documentation Events Support GitHub

Site Tools


administration:robots.txt

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
administration:robots.txt [2020/06/04 15:05]
demiankatz
administration:robots.txt [2020/06/04 15:06] (current)
demiankatz [More Information]
Line 52: Line 52:
 ===== More Information ===== ===== More Information =====
  
-Google offers some [[https://​support.google.com/​webmasters/​answer/​7424835?​hl=en|answers to frequently asked questions]]. This explains some of Google'​s crawling behavior in more detail, including some important information such as the fact that a robots.txt "​Disallow"​ directive may be ignored based on other criteria, and the "​noindex"​ robots meta tag may be a stronger way to hide content.+Google offers some [[https://​support.google.com/​webmasters/​answer/​7424835?​hl=en|answers to frequently asked questions]]. This explains some of Google'​s crawling behavior in more detail, including some important information such as the fact that a robots.txt "​Disallow"​ directive may be ignored based on other criteria, and the "[[https://​support.google.com/​webmasters/​answer/​93710?​hl=en|noindex]]" robots meta tag may be a stronger way to hide content.
administration/robots.txt.txt · Last modified: 2020/06/04 15:06 by demiankatz