As search engines success, many users have noticed that they are overloaded with information. Despite the common errata, the amount of information available has not increased. The information overload is due to fact that search engines have brought data more accessible. And as companies, organisations etc. transfer their archives to public electronic format, the problem will only get worse.

This may lead to severe problems, since many good, very relevant sites get buried somewhere in result. Users have no means to tell whether or not the data they received is relevant, and in many cases they notice the existence of irrelevant results. A large part of SEO and SEM is making content more accessible on search engines. The purpose of this article series is to describe how and why search engines work the way they do and how to implement this information on SEO / SEM strategies.

Directory versus search engine

• Directories are hierarchical index lists of sites, they list sites by topic. They are widely used and in many cases offer an extremely great source of information. However, they have few problems.

• Hierarchies are very vulnerable. Data and its classifications change constantly. This also leads to changes in hierarchy. A good example of this is DMOZ, worlds largest directory. Several subcategories are created, removed or deleted each day.

Most directories rely on human intelligence and are manually edited. They can never compete with search engines in amount of information. However, quantity is never as important as quality. Unlike directories, which show the hierarchy of data, search engines have only one purpose: tell user where to find the most relevant data. Search engines rely to mere computing power instead of human intelligence that directories offer. They crawl web pages, analyze and split them to smaller pieces. As user enters search phrase, search engine returns pages containing searched words sorted by relevance. This relevance is determined by something called algorithm.

In theory search engines sound picture perfect solution, but sorting out only the relevant results is something that no search engine has yet managed to do. Although there is clear evolution in algorithms search engines use, the journey ahead is long and rocky.