Think LIke a Web Page: 5 Tips for Smarter Search Engine Searching

Think Like a Web Page: 5 Tips for Smarter Search Engine Searching

Rita Vine, Workingfaster.com
September, 2003

Even serious web searchers turn to their favorite search engine for quick-and-dirty retrieval of good-enough information. Search engines are easy, fast, and deliver information for practically any query. When searchers think about web searching, they usually have a topic in mind, and input keywords that represent their topic into the search box.

For certain types of searches, this habit of keyword-as-topic searching in search engines is good enough. When you want to find something that you know is on the web already, typing in keywords that represent the thing you want —say, the name of an association or organization ("special libraries association") -- will usually turn up the web site of the organization or group within the first 10 results. Searches for distinctively-named things usually result in that "aha!" moment in web searching, when you can look at a results page and, without even clicking on the link, know simply from the title and excerpt that this is the page to go to.

For more general topical searches, particularly those with flexible terminology (e.g. health of Canada’s aboriginal peoples; laws restricting tobacco advertising) keyword-as-topic searching is less successful. True, a search of laws smoking advertising will likely turn up results, but the searcher is always left wondering what they missed.

A portion of what the search engine searcher misses is invisible web documents which can’t be indexed by search engine spiders. But many web pages that are indexed by a search engine might be left undiscovered simply because the keywords searched weren’t the ones that actually appeared on the relevant web page. Most searchers respond to this by trying other search terms, with similar results. By using methods that constantly bring up the same pages over and over again, searchers think they found everything on a topic, give up or run out of time.

Don’t enter the keywords that represent your subject – instead enter the keywords that you think ought to appear on pages that will be relevant to your search.

For example, if you’re looking for the definition of hatha yoga, you might have better luck searching the phrase "hatha yoga is" than the keywords hatha yoga. By thinking like a web page, you considered that a definition of hatha yoga might include the phrase "Hatha yoga is ..." followed by a definition. In the second example, you’re betting that a definition of hatha yoga includes the word definition – something that the creator of the web page might not necessarily include.

Travel sites make good use of geographic words because place names are essential components of travel web pages. But many information sites unrelated to travel – like education, government sites, and business directories for a region – don’t necessarily use geographic qualifiers on all web page links.

Many searchers outside of the U.S. try to qualify their searches with a country, region, or city in order to narrow their results to pages from that region. But if you think like a web page, you’ll know that not all web page creators think about putting a place name indicator in the page content. And even if they did, they might put in the country’s 2-letter abbreviation rather than the full name. If you really need to find resources from a particular country, avoid search engines in favor of regional directories and, for businesses, web-based yellow page listings.

Phrase searching, which is accomplished in most search engines with the use of "double quotations" surrounding the phrase (e.g. "to be or not to be") can be a great way of restricting your results. But it should only be used for terms where the words are not normally separated, typically proper names of organizations or associations (e.g. "sierra club").

Even proper names encased in phrases like "george bush" would miss any reference to George W. Bush. Proximity searching, which would enable more flexible phrase searching to within x words, isn’t always possible with search engines. At this writing, only AltaVista contains a NEAR-TO search function. Google allows a rudimentary version of proximity searching within phrases ("george * bush" would retrieve results with george separated from bush by exactly one word).

Consider using domain-name and doc-type limits when appropriate.

The Advanced Search feature of Google enables searchers to limit retrieval of their search keywords to selected domain names (including top-level domains) as well as document types. Limiting by domain names or top-level domains can be a good way of removing .com web sites from a search. If you’re primarily interested in scholarship on a topic, consider limiting to .edu sites (which will limit to US higher education domains), but recognize that this is a rudimentary and often unsatisfactory way of narrowing results.

Thinking like a web page might lead you to think that statistics on a certain topic might be available as Excel spread sheets, or that user guides to using OVID online might appear as Adobe PDF files. In either case, you can use Google’s advanced search template to limit a simple keyword search to specific document types.

Remember, no matter how hard you try, the search engine will win.

It’s important to remember just how commercial even the best search engines are. Search engines are businesses first and search tools only incidentally. They make their money by selling advertising – and lots of it. In the long run, this advertising affects the way all of us think about search resources and also dramatically affects the rank ordering of links on a search engine’s results page.

In Google, for example, PageRank (Google’s algorithmic ranking scheme) favors pages that are linked often by other pages on the web. So Google is really delivering the most popular pages, not the most relevant pages as is generally believed. The ability of web pages to reach the top of a search result list approximates a power law, where a relatively small number of sites receive the majority of links. The competition for links on the web is fierce in certain areas, particularly entertainment, consumer electronics, and publications. For more information and some science on the impact of that competition on search results, see David Pennock et al, "Winners don’t take all: Characterizing the competition for links on the web, " Proceedings of the National Academy of Sciences, 99(8): 5207-5211 (April 2002). A summary of findings is available at http://modelingtheweb.com.

--------------------------------------------------------------------------------

_{Return
to previous page}

All contents copyright © 1999-2005 Workingfaster.com. All rights reserved.
Keep It Simple Searching and Search Portfolio are trademarks of Workingfaster.com. All rights reserved.

Phone/Fax: (416) 928-1405 Toll-free U.S. & Canada: (800) 343-9878

Send comments and corrections to the webmaster.