eDiscovery Search Predictions for 2008

As I continued my education eDiscovery search platforms over the past couple of weeks and subsequent update to my eDiscovery Paradigm Shift Blog, I came across a really interesting update by Stephen E. Arnold on the current state of the search market in general titled "Search Rumor Round Up, Summer 2008". And, although it is not specific to search in the eDiscovery space, his overview of search is outstanding and very applicable to what we have to look forward to in eDiscovery.

As I pointed out in my post titled "eDiscovery Search Case Law Emerging", the courts are starting to catch up in regards to the value and impact of search in the eDiscovery process along with the subtle nuances of search technology. And, as I talk to law firms and the legal departments of many of the Fortune 500 about their eDiscovery issues, I am finding that the topic of search and more recently conceptual search is being raised more often in the context of culling, de-duping, finding potentially responsive and privileged docs and gaining a better understanding of their data in general. However, I have also found a complete lack of understanding of current search technology and it applicability to the real needs of the litigators and their litigation services consultants.

Further, I am curious to understand the foundation for this weeks acquisition of Attenex by FTI. Was this a fire sale because the conceptual search market has not matured and therefore Attenex has not been able to reach its full potential as a conceptual search based review platform? Or, was this a brilliant move by FTI to add yet another leading edge technology to its roster based upon accelerating market demand? Just for the record, I happen to think that it is the latter. But, there are some intriguing arguments for the former.

The observations and comments in Mr. Arnold's article that I believe are most applicable to eDiscovery are as follows:

Rumor 1: More Consolidation in Search
As eDiscovery technology matures and the obvoius winners in technology and the appropriate strategy/formula for success begins to emerge, we are seeing the same basic consolidation in eDiscovery in general and will continue to see even more eDiscovery technology consolidation over the remainder of 2008 and in to 2009.

Rumor 5: Search Will Become a Commodity
I believe this prediction to be true at the desk top / SaaS based eDiscovery platform level where the user is wanting to search several 100,000 docs in a short period of time. However, at the service center production level where the document pool is terabytes of data, I believe that there is still room for several search technology leaders that can figure out how to get these massive search project done in hours as apposed to days.

Rumor 6: Search Is a Component of Other Enterprise Software
I believe that there is no doubt that user are quickly going to expect sophisticated search to become a seamlessly integrated part of any eDiscovery platform.

Rumor 9: Key Word Search Is Dead
This is my favorite prediction for eDiscovery as I can visualize all the current eDiscovery vendors cringing.

Rumor 10: A Hardware Maker Will Put Search on a Chip
Coming from a background of integrating the appropriate and ripe software technologies into firmware / hardware solutions, I absolutely agree with this prediction. And, it fits really nicely with the "behind the firewall appliance" direction that some of the archiving vendors are going.

All of this being said, the full text of Mr. Arnold's article is as follows:

I am fortunate to receive a flow of information, often completely wacky and erroneous, in my redoubt in rural Kentucky. The last six months have been a particularly rich period. Compared to 2007, 2008 has been quite exciting.

I’m not going to assure you that these rumors have any significant foundation. What I propose to do is highlight several of the more interesting ones and offer a broader observation about each. My goal is to provide some context for the ripples that are shaking the fabric of search, content processing, and information retrieval.

The analogy to keep in mind is that we are standing on top of a jello dessert like this one.

The substance itself has a certain firmness. Try to pick it it up or chop off a hunk, and you have a slippery job on your hands. Now, the rumors:

Rumor 1: More Consolidation in Search
I think this is easy to say, but it is tough to pull off in the present economic environment. Some companies have either investors who have pumped millions into a search and content processing company. These kind souls want their money back. If the search vendor is publicly traded, the set up of the company or its valuation may be a sticky wicket. There have been some stunning buy outs so far in 2008. The most remarkable was Microsoft’s purchase of Fast Search & Transfer. SAS snapped up the little-known Teragram. But the wave of buy outs across the more than 300 companies in the search and content processing sector has not materialized.

Rumor 2: Oracle Will Make a Play in Enterprise Search
I receive a phone call or two a month asking me about Oracle SES10g. (When you access the Oracle Web site, be patient. The system was sluggish for me on June 14, 2008.)The drift of these calls boils down to one key point, “What’s Oracle’s share of the enterprise search market?” The answer is that its share can be whatever Oracle’s accountants want it to be. You see Oracle SES10g is linked to the Oracle relational database and other bits and pieces of the Oracle framework. Oracle’s acquisitions in search and retrieval from Artificial Linguistics more than a decade ago to Triple Hop in more recent times has given Oracle capability. As a superplatform, Oracle is a player in search. So far this year, Oracle has been moving forward slowly. An experiment with Bitext here and a deployment with Siderean Software there. Financial mavens want Oracle to start acquiring search and content processing companies. There are rumors, but so far no action, and I don’t expect significant changes in the short term.

Rumor 3: Microsoft Will Tidy Up Its Search Operations
This rumor suggests that Microsoft, a giant company with many barons and dukes controlling fiefdoms, can deploy one search solution. I don’t think that will happen quickly. The Certified Gold Partners who make better search systems than those available from Microsoft can rest easy for the foreseeable future. Search is too complicated in general and within Microsoft for a one-size-fits-all solution. I anticipate more search options, not fewer. Coveo, Exalead, ISYS Search Software, and others will benefit from the Microsoft approach to search for months, if not years.

Rumor 4: Semantic Search Will Unseat Google
Semantic technology is now within reach of almost any search and content processing vendor. The technology is relatively well known and the processing power is available at a reasonable cost. By itself, semantic search will not be enough to shift the market share that Google is amassing in the consumer search and enterprise markets. Google’s been chugging along for a decade, and it has yet to meet significant competition other than itself. Semantic technology is a component, not a Google killer in the hands of a competitor at this time.

Rumor 5: Search Will Become a Commodity
No, as I described in my Web log post on May 12, 2008, about the “search elephant”, search has too many different meanings for one solution to sweep the board. Each unit of a company has many different search and content processing needs. It is, therefore, difficult to convince the legal department to use the open source Lucene tool for eDiscovery. The legal eagles will want to use a service from Brainware or Stratify. Down the hall, the chemical engineers need to find chemical structure. Search consists of niches, and these will bump heads, overlap, and been quite confusing to sort out. In that confusion, consultants and different vendors thrive.

Rumor 6: Search Is a Component of Other Enterprise Software
This is a rumor related to “search will become a commodity”. True, enterprise software vendors will include more robust search and content processing systems in their software, market the heck out of the enhancements, and bundle it with whatever the client wants to buy. But enterprise applications open the door to point solutions that meet specific needs. So search certainly will become ubiquitous and the ecosystem will spawn new species of information access. Nope, search is going to be with us for a long, long time.

Rumor 7: The Google Search Appliance Doesn’t Work
False. The GOOG has more than 10,000 licensees, a fleet of partners, and the OneBox API that can make the Google Search Appliance work like Roy Roger’s prescient horse, Trigger. The GOOG has had an impact on the enterprise search market. It’s easy to complain about the Google Search Appliance. It’s harder to explain how a company with such an interesting approach to sales can sell such a large number of units. Obviously a certain sector of the market wants these Google boxes.

Rumor 8: Social Search Will Revolutionize Enterprise Search
Nope. Social functions can be useful, but in regulated industries, there are some challenges associated with social search. Social search is the equivalent of a restaurant’s weekly special. If the customers gobble enough of the dish, the special could be promoted to a specialty. It’s early days for social search in an enterprise, but it’s not too soon for law enforcement and military intelligence people to embrace the concept. Social search is quite useful in certain areas, but one needs to have lots of social data to crunch to see the technology in full flower.

Rumor 9: Key Word Search Is Dead
Key word search is hard for many people. Alternatives and options are needed. But key word search is too useful in certain types of research to go the way of the dodo. Investors like to think that a whizzy interface without a search box is the next big thing in search. Interfaces are becoming more important by the hour. But an interface without a way to look for words and phrases won’t carry the day.

Rumor 10: A Hardware Maker Will Put Search on a Chip
What’s happening is research and investigation. The Exegy appliance could be boiled down to a smaller gizmo. At some point in the future, any search appliance could be reduced to firmware. I think the likelihood of search on a chip is high, but it’s not something you will be able to buy in 2008. Intel invested in Endeca for a reason. Intel had a brief love affair and then a messy divorce with search vendor Convera years ago. Other chip-centric outfits are poking around in this area as well. On the horizon, yes, but appliances will be about as close to search in a single package that we will have in 2008.

Observations
Most people don’t realize that search is like a giant jello dessert. There is a shimmery, attractive quality to the whole thing. When you start to pick it apart, the substance becomes slippery and tough to pin down. It’s easy to be fooled by surface changes like semantic search and social search, which are like a squirt of whipped topping on the jello. Do you have candidates for rumors you think I should have included in my round up. If so, use the comments section of this Web log to post your favorites. To avoid legal hassles, I may have to edit some of your inputs. Ah, life in the modern world is so rewarding.

Labels: Chip based Search, Conceptual Search, eDiscovery, eDsicovery Appliances, Google, Keyword Search, Oralce, Search

posted by Charles Skamser @ 9:36 AM