6.5.1. Indexing the Entire Site

However, searching works much better when the information space is defined narrowly and contains homogeneous content. In other words, the more you search through indices that combine apples and oranges, the worse your retrieval results will be. After all, when you search a site, you're probably looking for apples only, not oranges. As already discussed, a site's content is usually a mix of apples, oranges, kumquats, bell peppers, chainsaws, and Barbie dolls to begin with. So, when you tell your search engine to index your entire site, the site's users will be performing searches against all kinds of stuff -- navigation, destination, and other kinds of pages -- all at once. What they retrieve can often be ugly.

• 58 documents are Welcome to Netscape Navigator version X.X pages for just about every version of Netscape Navigator and include information about plug-ins.

• 16 documents are in German (a language I don't read).

• 6 documents contain the potentially relevant term application in their titles, but 5 of these 6 have exactly the same title (Netscape Handbook: Application Features).

• 2 documents actually contain plug-in in their titles.

• 18 other assorted documents may be relevant, but are not labeled in a way that indicates whether this is the case.

Analyzing these search results, we find two common problems. First, we are presented with documents that clearly don't belong. If the site had been selectively indexed with audience differences in mind, 16% of the results would not have been displayed at all. Second, regarding relevant documents, it's not clear why we need 58 versions of the same type of document. It would have been useful to index pages more selectively, such as files relevant to Windows or Macintosh users, or recent versions versus older versions of the software. Are very many people still interested in old Netscape Beta versions? So, our search is less successful than it could have been; it gave us a lot of irrelevant documents, and too many that could be relevant.

Our search performed poorly because all the content in the site was indexed together. By doing so, the site's architects chose to ignore two very important things: that the information in their site isn't all the same, and that it makes good sense to respect the lines already drawn between different types of content. For example, it's clear that German and English content are vastly different and that their audiences overlap very little (if at all), so why not create separately searchable indices along those divisions?

Figure 6-15. Three pull-down menus perform a brief reference interview sufficient to help users download the appropriate software product.

6.5.2. Search Zones: Selectively Indexing the Right Content

Figure 6-16. Microsoft's site employs search zones to help focus the user's search before submitting a query to the search engine.

You've got many options to review, but you can quickly find the Internet Explorer area of the site where you'd want to look for plug-ins. Consider how well the effort the user expends in reviewing and selecting from this menu compares to the much greater effort of searching the entire site and then sifting through a tremendously larger retrieval set. Also note the Full Site Search option; sometimes it does make sense to maintain an index of the entire site, especially for users who are unsure where to look, who are doing a comprehensive leave-no-stones-unturned search, or who just haven't had any luck searching the more narrowly defined indices.

How is search zone indexing set up? It depends on the search engine software used. Most support the creation of search zones, but some provide interfaces that make this process easier, while others require you to manually provide a list of pages to index. In either case, search zone indexing requires more work on your part than simply pointing the search engine at the entire site: you'll need to review and mark each page that should be indexed. To make this easier, you might design your site so that pages that should be indexed together are located in the same directory; that way, you would mark for indexing a directory (and, implicitly, its contents) instead of its individual pages. You may also be working with pages that are generated from a database. In this case, you could design the database to include a field for each record denoting which index the generated page should belong to.

You can create search zones in many ways. Examples of four common approaches are:

• by content type

• by audience

• by subject

• by date