Eight Search Engine "C" Changesby Tara Calishain, author of Google Hacks
Hello. My name is Tara Calishain and I'm a search engine addict.
Not just mildly interested. Not just pretty good at finding things. I mean obsessed. I mean hours laboriously deconstructing Google result URLs. I mean all kinds of wacky experiments (with accompanying maniacal laughter) whenever AltaVista changes its syntaxes.
Ever since I wrote the first edition of the Official Netscape Guide to Internet Research in 1996, I've been fascinated with how search engines work. What makes them go, and what makes them go faster? Over time, that fascination has expanded to specialty search engines, databases, and other online data collections. Since 1998, I've tried to cover the world of online search engines with a weekly newsletter called ResearchBuzz, but alas, I can only scrape the surface.
Now is a wonderful time to be a search engine addict. More and more state and country sites are putting databases of material online--everything from professional license information to restaurant health-inspection scores. Extensive genealogy collections are appearing from both professional organizations and amateur enthusiasts. And Google, arguably the Internet's most popular search engine, continues to push the envelope with a variety of new offerings.
Busy? Absolutely. Exciting? You bet. But uncertain as well. Four or five years ago, searching was fairly straightforward. The Internet was useful, not yet ubiquitous. Search engines were still evolving and experimenting. There was a great surge of growth. Now the growth is not so extensive; surfers as a whole have developed a sense of what they expect when they visit a search engine. Instead, search engine companies have several problems to face in order to develop an infrastructure that's both sustainable (from a financial standpoint) and acceptable (from a user's standpoint).
There are lines from The Tempest that always remind me of the Internet:
Nothing of him that doth fade But doth suffer a sea-change Into something rich and strange.
Financial, cultural, and technological forces are all combining to push against search engines and turn them into something rich and strange indeed. But since computers and water don't mix, why don't we call them "C changes"? Eight of them have been occupying my thoughts a lot lately.
As I noted, more and more collections are going online. But while many of them have their own site-search engines, they're sometimes hard to find via traditional full-text search engines like Google. It may be because the database consists of dynamic content, which is difficult to index because it's password-protected, or for another reason. Search engines are going to have to determine how to get to this content, and how to present it to the searcher.
Once upon a time, search engines were either technology showcases (that's occasionally still the case) or they were supported with banner ads. Banner ads don't do the trick anymore, however; click-through rates have dropped through the floor and instead, ad-blockers are doing a booming business. Search engines have to come up with a way to generate revenue, and pronto.
How are they going to do it? Will it be via pay-for- inclusion (PFI) programs, or sponsored result listings that appear at the top of search results? Will it be the pay-per-click (PPC) that's been so good to Overture, or will it be the AdWords that Google's made available for even the smallest Internet advertiser? Or perhaps search engines will consider offering subscription services that deal directly with the searcher. (Yahoo does offer several "premium" services, but for the most part, the services don't deal with searching per se.)
Despite the ten years that have passed since the Web first got rolling, search engines are still indexing only a fraction of the available pages. That's not to say search engines haven't advanced greatly; they have. But the Web is growing faster than search engines can keep up. Will the growth curve ever flatten? Even if search engines could index the entire Internet, would that be useful or appropriate? Does there need to be a better way to separate out the signal from the noise before search engines try to encompass more of the Web?
Different countries have different levels of access. Germany has already demanded that Google remove certain items from its index (you can see an article about this at PCWorld.com, and it's been noted that China has blocked huge lists of sites from their users (an article on that is available from the San Diego Union Tribune).
Is it too much to hope that one search engine could be developed that would encompass all levels of information access, or are search engines going to have to maintain several different versions of their index to please everyone?
Pages: 1, 2