Chapter 6. Searching Systems

Before we delve into searching systems, we need to make a point: think twice before you make your site searchable.

Your site should of course support the finding of its information. But don't assume a search engine alone will satisfy all users' information needs. While many users want to search a site, some just want to browse it.

Now that we've got our warnings and threats out of the way, we'll discuss when to implement searching systems, and how you can make them work better.

Most web sites, as we know, aren't planned out in much detail before they're built. Instead, they grow organically. This may be all right for smaller web sites that aren't likely to expand much, but for ones that become popular, more and more content and functional features get added haphazardly, leading to a navigation nightmare.

There's a good analogy of physical architecture. Powell's Books (http://www.powells.com), which claims to be the largest bookstore in the world, covers an entire city block (43,000 square feet) in Portland, Oregon. We guess that it originally started as a single small storefront on that block, but as their business grew, they knocked a doorway through the wall into the next storefront, and so on, until they occupied the whole block. The result is a hodgepodge of chambers, halls with odd turns, and unexpected stairways. This chaotic labyrinth is a charming place to wander and browse, but if you're searching for a particular title, good luck. It will be difficult to find what you're looking for, although you might serendipitously stumble onto something better.

Yahoo! once was a Web version of Powell's. Everything was there, but fairly easy to find. Why? Because Yahoo!, like the Web, was relatively small. At its inception, Yahoo! pointed to a few hundred Internet resources, made accessible through an easily browsable subject hierarchy. No search option was available, something unimaginable to Yahoo! users today. But things soon changed. Yahoo! had an excellent technical architecture that allowed site owners to easily self-register their sites, but Yahoo!'s information architecture wasn't very well-planned, and couldn't keep up with the increasing volume of resources that were added daily. Eventually, the subject hierarchy became too cumbersome to navigate, and the Yahoo! people installed a search engine as an alternative way of finding information in the site. Nowadays it's a decent bet that more people use Yahoo!'s search engine instead of browsing through all those hierarchical subject categories, although the browsable categories remain useful as a supplement to the searching process (and, in fact, are included in search results).

Your site probably doesn't contain as much content as Yahoo! does, but if it's a substantial site, it probably merits a search engine. There are good reasons for this: users won't be willing to browse through your site's structure. Their time is limited, and their cognitive overload threshold is lower than you think. Interestingly, sometimes users won't browse for the wrong reasons; that is, they search when they don't necessarily know what to search for. Even though they would be better served by browsing, they search anyway.

You should also consider creating a searching system for your site if it contains highly dynamic content. For example, if your site is a Web-based newspaper, you could be adding dozens of story files daily. For this reason, you probably wouldn't have the time each day to maintain elaborate tables of contents, browsable indices, and other browsing systems. A search engine can help you by automatically indexing the contents of the site once or many times per day. Automating this process ensures that users have quality access to your site's content, and you can spend time doing things other than manually indexing and linking the story files.