robots.txt file and ranking?
Pear asked:
I just read this on http://www.caffeinemarketing.com/?p=96 “If you’ve submitted your web site to the search engines than the spider robots will be searching for your robots.txt file. If they do not find one the robot generates a ‘404 File Not Found’ error everytime. These robots might be searching for that file several times a day. Adding the file to your root directory will help eliminate these errors and boost your rankings.”
I just read this on http://www.caffeinemarketing.com/?p=96 “If you’ve submitted your web site to the search engines than the spider robots will be searching for your robots.txt file. If they do not find one the robot generates a ‘404 File Not Found’ error everytime. These robots might be searching for that file several times a day. Adding the file to your root directory will help eliminate these errors and boost your rankings.”
Is this true? Can robots.txt help with search engine optimization?
Internet Marketing Las Vegas
