rock.ru/legacy_rock
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Disallow 'internal' links link topic=XXXX.msgYYYYY or topic=XXXX.fromZZZZZZZZZZ for bots. The actual pages are still accessible, of course

Browse Source
This commit is contained in:
Aleksei Miheev
2014-02-16 16:23:17 +00:00
parent 4c6046822b
commit 2c79c59213
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
robots.txt
View File

@@ -2,3 +2,5 @@ User-Agent: *
Disallow: /ads/
Disallow: /go/
Disallow: /forum/index.php?action=profile;*
Disallow: /forum/index.php?topic=*.from*
Disallow: /forum/index.php?topic=*.msg*