www.wysiwygwebbuilder.com • How does robot.txt work and how to use it?

Page 1 of 1

Posted: Mon Apr 19, 2010 5:04 pm

by fervid

Also:

http://www.webmasterworld.com/robots_txt/3310622.htm
http://googlewebmastercentral.blogspot. ... psorg.html
http://www.sitemaps.org/protocol.php
http://en.wikipedia.org/wiki/Robots_exclusion_standard