SiteLines - Ideas About Web Searching

Yahoo, Ask Jeeves and Infospace Lead Internet Property Consolidations

There has been a flurry of purchases in the last month by Yahoo!, Ask Jeeves and Infospace. Yahoo just bought Europe's largest shopping comparison engine, Kelkoo, which will enable it to effectively capture the major European web shopping market; Infospace bought Switchboard, a leading Internet-based yellow pages service; Ask Jeeves purchased Interactive Search Holdings, a privately held company which included a variety of search properties including My Way, My Search, My Web Search, iWon, and Excite.

In this week's Silk Road Weekly, Safa Rashtchy, Internet analyst for investment firm Piper Jaffrey, offers some thoughts on summarizes why he believes that these new consolidations will continue in earnest through 2004 and 2005:

1. The advertising market is strong, suggesting that these rapidly expanding companies, which earn their revenue from advertising, will likely be profitable.
2. These surviving companies (Yahoo, AskJeeves, Infospace) are clearly the winners and are gaining strength. So this is a fertile landscape for acquisitions as a key strategy for growth.
3. Web search is now directly tied with market share and traffic, so a higher percentage of reach almost guarantees a higher percentage of revenues. In other words, if you buy the properties, and you get the all-important traffic, which means a lot.
4. The "Google Factor" -- Google's search dominance has created huge competitive pressure for nearly everyone in the industry, fueling an acquisition trend to strengthen position. Google "even without public stock, has probably more cash than many smaller comapnies and a currency that may be viewed as equally valuable as a public one, [which] creates more pressure on the potential consolidators."

As consolidation of search properties increases, the utility of searching multiple popular commercial search tools will continue to yield fewer distinctive results. We saw evidence of this last week when Alltheweb, a popular search engine previously recommended by several web experts (including me), abandoned both its independently spidered index and many of its advanced search features, and now essentially delivers Yahoo search results under the Alltheweb.com brand. (See Greg Notess' post in Search Engine Showdow for more on the Alltheweb shift.)

Subtle Look & Feel Changes to Google Favor Advertisers

Google launched some subtle changes to its desktop today. Although they are minor, most of them provide for increased visibility of ads (though Google calls this "enhanced usability" and "easier access to frequently used tools and services"). Ads on right are now separated by a vertical line, not boxes; a larger ad font is used; the ads that are placed above the results have a uniform background color rather than different colors for each ad; removal of the tabs that led to the Google Directory, Newsgroups and Google News, replaced by above-the-search-box links to all of those plus Froogle, Google's shopping site.

Keeping Up With Health Trends Online Through E-Letters

In the March issue of LLRX.com, Lois Ambash contributes an annotated roundup of recommended newsletters to help health professionals and researchers keep up to date with health-related issues, events policies and trends.

Medical Editors Fight US Treasury Ruling

In response to reports in February of a ruling by the US Treasury Department that editing scientific manuscripts from Iran, Libya, Sudan and Cuba violated existing trade embargos with those countries, the World Association of Medical Editors (WAME) issued an expected defense of scientific submission criteria. In its March 23 statement titled "Geopolitical Intrusion on Editorial Decisions," WAME struck out against the decision and encouraged editors to resist decisions based on "nationality, ethnicity, political beliefs race, or religion of the authors."

(Small sidebar: medical librarians will find much of interest at the WAME web site, including discussions on impact factors, authorship disputes, and free access to medical research.)

Although many in the library community have interpreted the Treasury Department rulings as a ban on publishing manuscripts from these countries, the rulings may actually prohibit substantial editing, translation, or corrections of manuscripts -- not quite the same thing as a publication ban, but of concern nonetheless. A New York Times article quotes Tara Bradshaw, a Treasury Department spokeswoman, who confirmed the restrictions on manuscripts to include "collaboration on and editing of the manuscripts, the selection of reviewers, and facilitation of a review resulting in substantive enhancements or alterations to the manuscripts."

Yet accounts of statements made by other Treasury Department officials suggest broader applcation of the embargo. Some of these statements are reported in Nature, February 19 2004 (427): 663. In that article, David Mills of the Treasury Dept. is quoted as suggesting that anyone wanting to publish papers from Iran should seek a license to do so from the Treasury Department, and that US scientists collaborating with Iranians could face prosecution.

Journal of Religion and Society Devotes an Issue to Mel Gibson's Passion of Christ

I haven't seen the film, but those who have (or who support student research assignments on this hot topic) will surely be interested in the January 2004 issue of the Journal of Religion and Society. The issue presents several articles from a January 2004 symposium surrounding some of the issues featured in Mel Gibson's film The Passion of Christ. The issue also includes "A Viewer's Guide to Contemporary Passion Plays," which provides background material and contemporary interpretations. (Spotted on LII New This Week)

Internet Resources for Non-Profit Organizations

In the February 2004 issue of Searcher, Hazel Cameron, a librarian at Western Washington University, has contributed an excellent review essay on web resources related to the not-for-profit sector.

Hot Paper Topics

Hot Paper Topics is a great little site to help writers quickly find high quality pre-selected resources on about 30 different term paper topics. For each topic, librarians at St. Ambrose University's O'Keefe Library select and briefly annotate a handful of high quality free web resources. Topics covered are typical of late high school/college assignments and include subjects such as censorship, attacks on America, same-sex marriage, and welfare reform. The site also links to other lists of "hot topic" link lists on the web.

Resources on Off-Shoring

The South Asia Journalists Association has produced a useful list of links to articles and news reports on the topic of off-shore outsourcing, particularly on contracting of work to India. Designed principally to help journalists cover the topic more effectively, the site includes links to cover stories, news, opinion pieces, analysis, weblogs/discussion boards and books.

Recommended Resource: National Image Library from the US Fish & Wildlife Service

The National Image Library contains a substantial (over 5000) collection of public domain photographs of wildlife, plants and natural scenery, as well as photographs of wildlife management work. The collection is keyword searchable - metadata subjects are assigned to each image. All images are provided by various state and federal agencies of the US government.

Details on Google's SEC Disclosure

Spotted on Resourceshelf.com, "Google facing public filing as it grows" by Steve Gelsi from CBS Marketwatch provides more details on the possible/likely requirement that Google will have to disclose certain financial details to the Securities and Exchange Commission even if it decides not to file an initial public offering. Gelsi notes that they hype surrounding the Google IPO is tremendous: "Nevertheless, expectations continue to be high for Google's IPO, which could fetch as much as $4 billion in proceeds as the biggest Internet deal of all time."

New from the AARP - Internet Resources on Aging

This is a new collection developed by the American Association of Retired Persons (AARP). I previously featured AgeSource in an earlier issue of SiteLines: this is not the same thing. Internet Resources on Aging contains 600 web sites of interest to older adults in the areas of health, caregiving, housing, services, and more. AgeSource includes aging-related research resources such as databases and statistical sources.

AARP's AgeLine database also recently added a customized feature "AgeLine Research to Go." Users can click on any of over 50 high-interest aging-related topics and get up to 20 recent citations/abstracts from AgeLine. New topics are added regularly. A companion features "AgeLine Searches to Go" covers consumer literature written for the general public interested in aging-related topics. (Thank you Margaret Eccles of the AARP for clarifying the differences between these tools!)

More than 55M Europeans Search with Google

In a press release from Nielsen Net Ratings, European searchers rely on Google most. Over 47% of Europeans polled use Google as their search engine, well above the 23.1% that rely on MSN Search, the second choice.

Because Europe lagged somewhat behind North America during the 90's in embracing the Internet, there are fewer high-quality made-in-Europe directories and other alternative choices for information searchers who want to go beyond Google. Europeans who want a Europe-focused search tool must rely more heavily than North Americans on commercial search tools, Yahoo-lookalike commercial portals, and existing made-in-the-USA directory search tools.

THE INFOGRAPHY - Best sources of information on selected subjects

Spotted on RLG's Shelflife, an organiztion called The Fields of Knowledge has created a Web service called The Infography (in-fóg-ra-phy) as a reference tool that facilitates the identification of "superlative sources of information about a subject of inquiry, viewed through the lens of expert opinion." This is a commercial venture, and it appears that ad revenues will support the site and also provide for some (probably modest) royalties for those subject specialists who elect to contribute resource lists to this tool.

The best way to think about the site is as a searchable list of reading lists. Many of the lists include relevant web links as well as book and journal citations. Though questions remain about how often the lists will be updated (particularly for scientific and technical topics in rapidly changing fields), this site is definitely worth consideration for libraries that support high school and college students.

It's not clear just how many lists are included in the database, but it's substantial -- I searched the keyword literature across the full text of the site and got 192 lists that contained the word somewhere in the full text. A search of marketing delivered 31 lists, including topics such as "Mass Media - Uses and Effects" and "Economics of the Arts." One quibble - the only statement of responsibility on each list is too generic. For example, on the Economics of the Arts page, we see the statement, "A professor whose research specialty is the economics of the arts recommends these sources." Surely fuller attribution is warranted and required.

Is it time to detach from our reliance on search engines?

Consider the reality of relying on your favorite search engine. You're applying a pretty dumb technology (search algorithms) against a huge, undifferentiated pile of randomly selected, unorganized content; then adding billions of dollars of keyword-matched ads to the sorted output. Moreover, the effect over time of persistent ad placement in search results is to push those web resources that lack the capacity or interest in placing ads further down the search results list and out of sight of most searchers.

This is no recipe for search success. And it's folly to assume that reliance on ad-supported search engines is just fine as long as searchers understand how results are derived and ranked. From long experience training business and information professionals to search the web more effectively, I can tell you that almost no one fully understands just how pervasively ads influence search results without receiving additional information and training.

Given the facts, I'm puzzled by the extent to which information professionals continue to believe that commercial ad-supported search tools will deliver relevant, high quality information results. And I'm troubled by how much buzz the new crop of new-to-the-marketplace ad-supported search tools is receiving among the library community, as if some "new Google" will magically produce better results applying the same ad-driven business model to giant indexes. It's time for the library community to fully understand how unsupportable the notion of finding persistently relevant information in free, ad-supported search tools really is.

The answer is better search tools which, though they may be supported by ads around the periphery of the page, have no ads placed inside of search results. It is not going to make anyone rich, but it is a model that has been used to a limited extent by individual libraries and consortia worldwide, in an effort to develop organized, selective directories of best-of-breed resources in a discipline or of a materials type.

We have a few fine models already, like Librarians Index to the Internet (http://www.lii.org) and the Resource Discovery Network (http://rdn.ack.uk). Both receive funding from libraries or consortia and have no ads inside of search results. Using a slightly different model, Genie Tyburski's excellent business and legal search starter site, the Virtual Chase (http://www.virtualchase.com) is supported by a law firm, but has no secondary delivery of ads either inside or around the periphery of the site.

There should be hundreds of these types of high quality, carefully maintained starter sites on the web, but there aren't. There used to be more, but I suspect that many were abandoned because their creators either ran out of steam, or time, or money, or felt that search engines were doing the job of delivering relevant results so well that their personal efforts were considered unnecessary. Perhaps it is time to resuscitate these efforts; for libraries to join together to form cooperative ventures, to develop and deploy high quality, discipline or format-specific collections of free web resources that continue to exist on the web, but that can no longer be found with commercial search engines.

CIL 2004: Some Key Themes

At this year's annual Computers in Libraries conference, there seemed to be a couple of key recurring themes.

Trend 1: Search engine bashing, or the increasing irrelevance of commerial ad-supported search engines.

Some experts are shifting their views on search engine capbility. -- rather than telling us about new ways to manipulate search engines and other commercial free search tools, some experts are suggesting that search engines aren't doing their job very well any more. (I heard the term "Google-bashing" a lot from conference attendees, who seemed to catch the same trend.)

David Seuss, CEO of Northern Light, provided a thoughtful keynote on web search, indicating that after 10 years, the web search challenge is nowhere near solved. Seuss highlighted the increasing inability of search engines to deliver relevant search results to many queries; that as search engine databases grow, relevance declines as algorithms increasingly fail to recognize what researchers are truly looking for; that improvements in relevance algorithms can't really make a difference in an ad-driven search engine world. Seuss envisioned a world of fragmented, selective, specialized datasets/databases that serious searchers could select from and pay for --which sounded a lot like the old DIALOG search options of many years ago, and likely forshadowed Northern Light's new focus. View his presentation online at http://www.infotoday.com/cil2004/presentations/default.shtml

Search Engine Showdown's Greg Notess delivered similar warnings, reminding us that while the search engines promote their databases and search capability, they are increasingly turning into ad machines. I echoed those same views in my presentation on the business of search engines.

Trend: Still no serious takeup of RSS

Although several presentations on RSS capabilities were featured at CIL, it was abundantly clear that, among the library community, there is hardly any takeup of RSS. It's possible that the presentations assumed a level of understanding that just wasn't there, but there is a lot of work to be done at the introductory level just to help people get the basic concepts.

Computers in Libraries 2004

I'll be taking a brief break from publishing SiteLines during the week of March 8 to participate as a trainer and speaker at Information Today's annual Computers in Libraries conference. Notes from my portion of the March 10 panel discussion "Tips for Keeping Up" with Steve Cohen of Librarystuff.net, and Gary Price of ResourceShelf.com, are now available.

FaganFinder: Search By File Format

Ontario university student Michael Fagan has produced another handy meta-search template for searching the major search engines that offer file-type limits. This beta version file format searcher enables users to select the file format type (only one at a time), and then select any one of the search engines that permit the search. Google, AlltheWeb, Yahoo!, MSN Search, Gigablast, AltaVista, and Elsevier's Scirus are included. Readers should note that not all formats can be searched in all of the tools.

My Search Index is Bigger than Your Search Index

It appears that the war for search engine content supremacy is on. Although many observers originally thought that Yahoo! would choose not to compete head-on with Google's "serious search" focus, and might focus instead as a shopping engine, it appears that Yahoo! intends to promote the value of its search index.

This week, Yahoo! announced that it is seeking to enhance its search index through a new Content Acquisition Program (CAP). Through this program, it will enter into mainly non-exclusive arrangements with content providers such as the Library of Congress and National Public Radio. CAP will also extend to other, commercial providers of information, although this was not featured in the announcement.

This action sounds similar to Google's recent announcement that it would acquire content from OCLC for addition to its index.

These efforts will enlarge the search indexes of both properties, although it is difficult at this early stage to assess the impact of these expanded indexes on search quality or relevance. Searchers need to remember that the sheer size of a search engine index isn't really much of a consideration in determining relevancy: it's the sorting algorithm that has the biggest impact on what the searcher ultimately sees on the results page.

While many searchers will applaud an even larger search index, it is also important to remember the commercial potential of a persistently growing web index consisting of billions -- possibly tens of billions -- of web pages. Because relevant information will become more difficult to find as sorting algorithms begin to fail to achieve relevancy goals, there will be even greater pressure on content providers to advertise in order to get favored placement in search results. In recent weeks, we have seen at least two incidences of Canadian government content providers (HealthyOntario.com and Culture.ca) place web ads in Google to ensure that their URL comes upon the first page following selected keyword searches.

Searching by File Types

In "Fiddling with Filetypes", Greg Notess examines how to search using file type limiters in Google, Alltheweb, and Gigablast. Notess also points out the little-known fact that PDF conversion tools often interpret text strangely, and that initial letters could be separated from the remainder of the word. Notess suggests leaving off the first letter of the word in a doc-type search (for example: try using nalyze filetype:pdf.

Lost Internet Citations

As students and researchers refer to web pages and web sites more often in their work, tracing -- or even finding -- citations for later review can be difficult or impossible. Online materials can move; they may be revised (yet remain at the same web address); or disappear completely without warning.

How big a problem is loss of Internet references? In "Going,
Going, Gone: Lost Internet References" (Science 302/5646, October 31, 2003, pp. 787-788) the authors examined occurances of Internet references in three important U.S. scientific journals: New England Journal of Medicine, The Journal of the American Medical Association (JAMA), and Science. They found that "Internet references accounted for 2.6% of all references and in articles 27 months old, 13% of Internet references were inactive." The article is available online at http://www.sciencemag.org/cgi/content/full/302/5646/787 (subscription required for full access)