Google's oligarchy of websites

Google continually favors large, well established sites more and more as the internet grows. This is making it more and more difficult for small business startups, without a very large marketing budget, to get much of any decent traffic from Google beyond from obscure multi-word searches.

This attitude by Google indicates to us that all the information that is valuable to people has either been found and indexed or if new information comes about it will do so through the large, well established sites. This attitude closes the internet off to significant quality growth by new sites that may, in the long run, have more to contribute and be more feature rich and user friendly than the well established sites. For example, with Google, it seems they have the idea that sites A, B, and C cover topic 1 well enough and also cover topics similar to 1, 1a, 1b, 1c, so these sites are all that any searcher will really need. If site D comes along a few years later with more information about topic 1, it will never be ranked with sites A, B, and C because Google has already decided that only A, B, and C are allowed to be authoritative sites on topics related to topic 1. Thus the information contained at site D, doesn't belong at site D, it really belongs under sites A, B, or C for it to have any value to the searcher.

Each main topic that a searcher would search for seems to have an oligarchy of authoritative websites whos authority is dictated by Google. These sites usually got their start somewhere around 1994-1998. At this time, the internet was a completely different place. Pretty much anybody could be authoritative on a particular topic because sites were limited and use of the internet was also limited. These authoritative sites from this time period have grown to the mega-sites of today, not because they were or are great sites with user friendly functionality, but because they were the first or one of the first sites on that subject. Any links from other sites on that subject would be directed to these sites which would later help to increase their rank (link relevancy) when search engines, such as Google, began to base rankings on link popularity instead of solely on keyword density.

What this all has lead to is boring results in Google. Searches on a particular topic or realm of topics will always yield these authoritative sites even if there are lesser sites with better on-topic information. Not only does this make it very difficult for a small site to grow, it helps the authoritative sites grow faster and faster, larger and larger, leaving the small sites in a place where they have an even bigger hole to dig out of.

Remember the good days of Google?
Google's main advantage when it began over other search engines at the same time was that when a searcher searched for a phrase, particularly noticeable on a phrase without many results, the information the searcher was looking for, if it existed, would show up right there on the first page of the search results. With other search engines of the time, searchers would have to go through 5, 10, 20, or more pages of results and visit almost every site to find if it contained the correct information. Sometimes the most relevant information would be buried deep in the search results pages. Google allowed a searcher to "read the page without clicking" by pulling excerpts from the page containing the keywords in the search phrase and by focusing their ranking on the least common word in the search phrase to put the most relevant information on the most specific aspect of the search at the beginning of the results pages.

This was Google then. Google today does similar to this except it adjust results with a huge bias toward to the authority sites. Filter 1: get the most relevant results, Filter 2: put the big sites above the small to help contain any new information and growth to the big sites.

Why is this a problem for the internet as a whole?
Google is the primary search engine for searches on the internet today. People were drawn to Google because of the relevance of the results in days past. Because it was so much better than other search engines of the time, people stuck with it. Today, it is one of the most biased search engines, banning sites left and right becuase there's already a site A, B, or C that in their opinion does it better. Good luck to anyone who wants to create a site to help people find information. Google thinks they do this the best and any other such site is a waste of space. Such sites will either be banned from Google or ranked so poorly that they will not be found. Without a top marketing budget, all such sites will fail, courtesy of Google and the general publics' lack of knowledge about how biased Google's results are and just how much more higher quality information exists that will never be found by searches at Google because it is not posted on an authoritative site.

What is the solution?
Ranking by link popularity needs to seriously worked over. This, as a method to rank search results, has become seriously flawed especially because newer sites with an endless marketing budget can buy their way up the search results with a combination of SEO, content development, and proper linking techniques. What is really needed is an open source, unbiased search engine with pattern matching capabilities. This would allow all sites to be indexed but would not allow the junk-type pages that clog most search engines to be prevalent unless they were asked for by the searcher. We are looking forward to the Nutch Open source Search Engine Project.

For Google to say "sites A, B, and C, do it good enough, we're not going to allow site D," hurts the internet as a whole by promoting big business, demoting small business, and blocking new and unique ideas unless they are thought of by sites A, B, or C. Google is causing circular problems for themselves and it is all based on their PageRank ranking system (PageRank is a trademark of Google). PageRank blocks small sites from competing for high SERP ranking for popular search terms regardless of the quality of the information contained at the small site. Small site owners, realizing this, have turned to SEO methods to try to beat the search engines and rank higher, as some of them rightfully should. Small sites have turned to SEO companies for the quick fix and have left sometimes with thousands of additional pages on their site that help to bring in additional traffic from unpopular multi-keyword searches, the idea being, if they have enough pages about enough multi-keywords, which small sites can compete for in Google because they are unpopular phrases, that small sites can bring in enough traffic to stay afloat in the whole game. This technique has littered Google and other search engines with junk on top of junk on top of junk. Google's solution to rid their results of this junk is to block the site D's without question if they at all appear to have more content that a site of their age should. By doing this they are ridding the internet of the "pesty" small business so that the large can thrive and the searches are left low on information, options, and choices.

We very much need some sort of open source search that is biased only on the quality of the information and its relevancy to the search phrase, as search engine giants like Google are getting further and further from answering this call.

	Site Search