The Church of the Search Engines

Do you expect web developers to hold qualifications in computer science? By the same account, you should expect search engine optimisation (SEO) specialists to hold a degree in statistics or game theory. Or computer science, in fact.

Ever since I set up Mauve Internet, it has been asserted on the website that SEO is a myth. In recent weeks I have brushed up on my understanding of the realm of SEO so as to defend Mauve Internet's practices. What I have encountered could reasonably be described a religion. Scant evidence is mused over, formulated into doctrine, and memorized by rote. The priests of SEO wield power in the eyes of the faithful, they preach their beliefs to others and they have heated religious debates about which beliefs are important.

Building a site which is genuinely more popular than the competition is the crux of search engine ranks and the responsibility for that lies entirely with the site owner. There are also a wealth of accessibility techniques for removing barriers to spidering, and there are some common sense techniques, like canonicalising URLs so as not to divide the weight of the page. But these are within the remit of the developer, who, if they are any good, will have done them as standard. More importantly, these are done once and for all. These do not yield incremental improvements and they do not need to be continually revised.

I don't believe SEO specialists stick to this territory, although hopefully many now pay attention to it. SEO specialists I have corresponded with carve out a niche where they can remain unchallenged, a territory of keyword density, meta tags, link depth, link penalties and link juice shaping, the application of ill-defined theories which are unproven (in some cases, disproven) and which they can continue to charge for as they tweak in response to the latest webstats.

The assertion that SEO is a racket can be easily substantiated. If website owners could, by invoking SEO voodoo, position themselves arbitrarily highly in the natural listings of search engines, then the search results would be determined by website owners as a function of time and money. The usefulness of the search would quickly degenerate and users would migrate to other search engines who provide better quality results. Therefore, search engines would not make as much money from sponsored links. Search engines like making money from sponsored links, so they won't allow this to happen.

This isn't some abstract scenario I've imagined. It actually happened in the late 1990's to the search engine Altavista. Altavista's search results had become a free-for-all and it haemorraged users, primarily to Google, whose search results were vastly superior and clean of link farms. I watched it happen; in fact I was one of Altavista's users who switched to Google.

The one thing we know for certain about the ranking systems of search engines is that they are extremely complex and closely guarded secrets. They don't have to be scrutable or even produce optimal results: they merely need to produce good results - which implies being hardened against exploitation.