Web 3.0 in eDiscovery

Over the past year I have been following discussions about Web 3.0, the next generation of the Internet. And, even though my wife is yelling at me to get ready to go Christmas shopping, I wanted to get this Blog posting done today as I believe that Web 3.0 is going to have a profound effect on eDiscovery in 2010 and beyond. Not only in terms of what we have to "Discover" but also in terms of how we "Discover". And, I am going to use Christmas shopping in my example of how to use Web 3.0. I am not sure what that says about my priorities. However, here goes.

From a SciFi stand point, Web 3.0 moves us closer to a more "Hal 900" like computer experience. So, instead of asking your computer to help you "find information" by typing keywords into Google, you will ask your computer to help you "complete a more conceptual task" such as as last minute Christmas shopping for my family that includes a stop for lunch with burgers and very large HDTV to watch the Packers play the Steelers. To put this into perspective, experts in the world of search would describe this approach as maturing from keyword search to conceptual semantic based search. And, the process of storing data on the Web in a form that machines can naturally understand, or converting it to that form, is called the "Semantic Web", a phrase attributed to Tim Berners-Lee, inventor of the Web (it wasn't Al Gore) and Director of the World Wide Web Consortium (W3C).

Web 1.0 was basically just a static library of information that user could access and utilize much like checking a book out of a library. Web 2.0 (the current state of the Internet), enables users to actually change the content of pages with additional information effectively making 'Web pages" living pages that grow through interaction with visitors. Web 2.0 is also characterized by social networking such as facebook and Linkedin, mass sharing of information via application such as Twitter and access via devices beyond the traditional computer.

Berners-Lee disputes the existence of Web 2.0, calling it nothing more than meaningless jargon [source: Register]. Berners-Lee maintains that he intended the World Wide Web to do all the things that Web 2.0 is supposed to do. His vision of the future Web is similar to the concept of Web 3.0. Unfortunately, the Web's current structure of storing information has been designed to work best with the the keyword approach. And therefore, accept is a few isolated instances, Web 3.0 isn't really even a possibility in 2009. For the realities of Hal9000 interactive with the Web, there have to be some changes made to way information is stored and processed. Web 3.0 will have to have collections of information called ontologies, a file that defines the relationships among a group of terms.

Back to our Christmas shopping task, in order for Web 3.0 to be able to actually provide the information that I need to have a successful shopping trip, eat lunch and watch the game, Web 3.0 will have to have an ontology that understand my family, what they want for Christmas, the best places to get those gifts, restaurants that serve hamburgers that will also be Packer game, etc., etc. etc. Basically, much more information than is available today and more information that will be available anytime soon on the Web. However, this doesn't mean that the technology is not available. Semantic search research and development has been underway for many years and is actually already being used successfully in eDiscovery and other information management markets . It's use is just not very widespread and there is still a tremendous amount of confusion about how it works and the validity of the results (please note that I have written about conceptual/semantic search in eDiscovery on this blog several times).

So, what will eDiscovery look like in the world of Web 3.0. First of all, it won't be a digital transition (i.e. we won't wake up one morning with Web 3.0). It will be a more gradual analog transition with application providing the best ROI emerging first and the rest of the world following behind (i.e. classic early adaptors vs. laggers). And, as already indicated, eDiscovery is already the perfect arena for semantic search as users can isolate information, develop ontologies and then ask the Hal900 type questions like, "what relationships exist between these custodians that are not apparent?". Or, "tell me about potentially important relationships that may exist outside of the known custodian?". Eventually, as the ontologies and the yet unknown upstream next generation ontology type technologies emerge, new search technologies will be able to find just about any information that lawyers require to ensure they understand everything that they need to know to proceed with their matters with a very high level of confidence. And, more importantly, a much higher level of confidence that what could ever be provided by completing the same analytics manually.

Other markets that are also already benefiting from this technology are consumer marketing (uncovering buying patterns), anti-crime and anti-terrorism and drug research (find unknown drug interations), just to name a few.

What is even of more interest to me is how our laws will evolve in this new world under Web 3.0. Just as Hal900 will enable law enforcement and the legal system to better understand and analyze information, so to will this same technology enable criminal elements to become more sophisticated in their approaches. I could very easily envision scenarios where Hal900 is asked the question about the most efficient and least risky way to commit a crime.

It should all be very ineresting and exciting to watch.

Labels: Conceptual Search, eDiscovery 3.0, Ontology, Semantic Search, Web 3.0

posted by Charles Skamser @ 10:24 AM