Search Engine USER AGENT Robot Info
Search Engine USER AGENT Spider Info
{embed='se-news/.last_update'}
The following info is intended to assist you in identifying the search engine spiders that visit your site. Details are based on information you can obtain by viewing your site's visitor logs.
Use the User Agent to identify a particular search engine in your site's visitor logs, then use the Robots.txt Identifier in your robots.txt file if you'd like to block that spider from accessing parts of your site. For more information on blocking any of these spiders using your robots.txt file, see http://www.robotstxt.org/wc/exclusion.html.