Site Search

Services

Portfolio

Pricing

Free Quote

Company

Contact Us

FAQs

Services :: Website Development :: E-Commerce Development :: Website Management :: Website Maintenance :: Website Upgrades :: Website Search :: Custom Scripts :: Search Engine Optimization :: WEBSITE HOSTING :: WEBSITE DIRECTORY Website Tools :: Website Search :: Keyword Density :: HTTP Header Viewer :: Reverse DNS Lookup :: DNS Lookup :: Newsletter Portfolio Pricing Free Quote Contact Us Company FAQs Links Recent Articles Census Data Resources Google Toolbar PageRank not Displaying Affiliate Data Feeds Future date with PHP Lookup domain names from an IP address How to change web hosts Mozilla Googlebot directs regular Googlebot/2.1 Evaluating Web Hosting Reviews Web Host Review Google's oligarchy of websites Google: 301 Redirects reappear in index after site banned Googlebot/2.1 Mozilla/5.0 not obeying robots.txt MySQL Select Random Row Fast SEO - The Other Side Search Engine Submission Tips SEO after Google's Florida Update Read more at Matt Gregg Blog

Home  ::  Articles  ::  Mozilla Googlebot directs regular Googlebot/2.1 Host 6 Domains on 1 Bluehost Account $6.95 Per Month Mozilla Googlebot directs regular Googlebot/2.1 Last Updated: 2005-04-04 01:07:09 We have been watching the Mozilla Googlebot "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" for some time now. It appears that this bot is a sort of pre-sorting mechanism for the older Googlebot "Googlebot/2.1 (+http://www.google.com/bot.html)" To us, this makes sense. Spidering the web is a very complex task and highly server-intensive. Why not send out a simplier bot (Mozilla Googlebot) that can visit sites and determine how to schedule the main bot (Googlebot/2.1). We believe that the Mozilla Googlebot is something of a datebook and calculator for the real Googlebot. The Mozilla Googlebot can also used to do the job of the Googlebot for less important spider visits (e.g. supplemental results). In essence, the Mozilla Googlebot seems to be a filter to help Googlebot around the billions of pages on the web. On a side note, including the name "Mozilla" in the user agent would support this hypothesis. Having the word Mozilla in the user agent would help to find poorly cloaked pages where it is popular as an easy cloaking solution to use Mozilla in the user agent. This would thus filter out content that Googlebot would not then need to visit. More Articles... Census Data Resources Google Toolbar PageRank not Displaying Affiliate Data Feeds Future date with PHP Lookup domain names from an IP address How to change web hosts Evaluating Web Hosting Reviews Web Host Review Google's oligarchy of websites Google: 301 Redirects reappear in index after site banned Googlebot/2.1 Mozilla/5.0 not obeying robots.txt MySQL Select Random Row Fast SEO - The Other Side Search Engine Submission Tips SEO after Google's Florida Update Return to Article Menu

Gregg Website Tools Search Enable Website Add a search feature to any website. Keyword Density Check the keyword density of your website. HTTP Header Viewer View the HTTP headers of any webpage. Reverse DNS Lookup Get the host name of an IP Address. DNS Lookup Get the IP Address of a website. Spider Simulator Simulate a SE Spider. Subscribe Today Gregg Website Development Newsletter Featured Sites EyeBike EyeBike.com offers an online search engine and price comparison for bicycle parts featuring many online vendors. Lake Tahoe Real Estate iPods Hesperia Real Estate Host 6 Domains on 1 Bluehost Account $6.95 Per Month

Services :: Portfolio :: Pricing :: Free Quote :: Company :: Contact Us :: FAQs

We're located in the High Desert, Southern California Copyright © 2004, Gregg Website Development, Privacy Policy PO Box 400308, Hesperia, CA 92340