Googlebot

Google's web crawler. Respects robots.txt and crawl-delay directives.