Ballardvale Research

Contact Us

Market Trends - Search:  History and Future


The enterprise search market has gone through three stages and is about to enter its fourth:

  1. 1985 - 1994:  Expensive Proprietary Searching of Subject-Specific Databases
  2. 1995 - 1998:  The Web Changes Everything:  Keywords, All Subjects -- and Free
  3. 1999 - 2004:  Academia Tackles Search -- Both Web and Enterprise
  4. 2005 - 2009:  Embedded Personal Search

1985 – 1994: Expensive Proprietary Searching of Subject-Specific Databases

The seeds of the enterprise search market were planted in the mid-80s, when computer terminals started to become commonplace in corporate libraries.  At that time, content suppliers offered clients the ability to search subject-specific databases -- examples include Lexis-Nexis (law), WestLaw (law), and ComputerSelect (computer software).  The search languages, although powerful, were proprietary and arcane, and typically the corporate librarian or other trained person made the search, not a business end user.  Furthermore, these solutions were expensive, costing tens of thousands of dollars a year; usually a company footed the bill, rather than a department.

1995 – 1998: The Web Changes Everything:  Keywords, All Subjects -- and Free

By the mid-90s, several years after the arrival of the Mosaic browser, the Web was starting to catch on.  AltaVista, a skunk works project at DEC, developed a Web search engine that suddenly became all the rage.  All of a sudden, a business user -- using keywords, rather than arcane commands -- could search for information on any subject -- and for free.  Everyone became a searcher.

Many of the lessons learned in the decade before were forgotten in the euphoria of the Web.  The sheer joy of quickly finding information meant that users often overlooked the fact that the quality of information was uneven.  In addition, the idea of optimizing the up-front search syntax or the behind-the-scenes storage structure for a specific subject was lost.  Universal search was good; tuned search was bad.

1999 – 2004: Academia Tackles Search -- Both Web and Enterprise

By the mid- to late-90s, university computer science departments and think tanks such as Xerox Parc began analyzing how to improve search and categorization.  The universality of the Web and the need to index millions of documents suddenly made the problem intellectually interesting.  Out of this research sprang a number of search companies.  Examples include Autonomy (Cambridge University, founded in 1991, IPOed in 1998), Google (Stanford University, founded in 1998),  Endeca (MIT, founded in 1999), and iPhrase (MIT, founded in 1999).

The early 00s were punctuated with intense debates about the underlying technology -- e.g., "Is Bayesian inference better than semantic analysis?" -- as well as the mixing of Web search and enterprise search.

However, as time went on, the academic debates softened, as search vendors eventually realized that users did not care about the technological underpinnings, but just wanted the software to work.  In addition, search companies ultimately decided to concentrate on either Web search or enterprise search, but not both (Google and Microsoft are the exceptions).  For example, Ask Jeeves sold its enterprise search unit to Kanisa in July 2003 to concentrate on Web search; FAST decided to specialize on enterprise search -- it sold its Web search business to Overture in April 2003 and acquired AltaVista's enterprise search business in June 2003.

This recession era also meant that the more successful companies bought up point technologies.  Autonomy purchased Virage (September 2003); Google acquired Applied Semantics (April 2003), Kaltix Corp. (September 2003), and Picasa (July 2004); iPhrase bought Banter (January 2004); while Verity purchased NativeMinds' assets (March 2004).

2005 – 2009: Embedded Personal Search

Search is about to enter a new era.  Rather than an installed application, search will become embedded within the lowest levels of the system, calling into question the long-term viability of companies that currently sell search applications.

At this point, it is often easier to find a document on the Web than it is on your PC's hard drive.  This dichotomy is not lost on either Google or Microsoft, and both well-heeled companies have gone on record saying that they are developing personal search tools.  As this occurs, the dividing lines between Web search, enterprise search, and personal search will increasingly blur.  A user will enter a search term, and the system will find the relevant documents no matter where they reside.

One mechanism that will make this easier is Microsoft's rewriting of the Windows file system in its forthcoming Longhorn operating system release.  Rather than applying metatag pointers to an 8.3 DOS file system, Microsoft is embedding metatags within the file system.  Put another way, Microsoft is swapping out a file system for an information system.  This much richer view of information storage will fundamentally change search capabilities and functionality.  Indexing of content will become automatic, and the other side of the relevance equation -- the user -- will receive focus.  By watching a user's previous search behavior, the system will be able to decide whether it should interpret a search for "ATM" as "asynchronous transfer mode" or "automated teller machine."

In short, search will once again become tuned to the task -- now just for a much larger audience and content universe than was ever conceived possible in 1985.

July 2004

© 2004, Ballardvale Research. All rights reserved. Reproduction without the express written consent of Ballardvale Research is strictly prohibited. Information is based on best available resources. Opinions reflect the analystís judgment at the time and are subject to change.

Print this page  |    |  Add this page to Favorites
Company  |  Privacy  |  Contact Us  |  Site Map

Please send comments and corrections to the Webmaster.

© 2004-2006, Ballardvale Research.  All rights reserved.