diff options
| author | Dan Halbert <halbert@halwitz.org> | 2020-08-17 09:48:17 -0400 |
|---|---|---|
| committer | Dan Halbert <halbert@halwitz.org> | 2020-08-17 09:49:46 -0400 |
| commit | e1a843878e4272334869b1db4a8222ae344f5cb8 (patch) | |
| tree | 9202013b31b97469aed72acdc396fea4e120abf7 /docs | |
| parent | dcbd3b4994bdbaa3d6b14788191b6d95524f0e66 (diff) | |
add robots.txt to specify doc versions to appear in search engines
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/robots.txt | 6 |
1 files changed, 6 insertions, 0 deletions
diff --git a/docs/robots.txt b/docs/robots.txt new file mode 100644 index 000000000..39a4eaeed --- /dev/null +++ b/docs/robots.txt @@ -0,0 +1,6 @@ +User-agent: * +Allow: /*/latest/ +Allow: /en/latest/ # Fallback for bots that don't understand wildcards +Allow: /*/5.3.x/ +Allow: /en/5.3.x/ # Fallback for bots that don't understand wildcards +Disallow: / |