This Page

has been moved to new address

The eDiscovery Paradigm Shift

Sorry for inconvenience...

Redirection provided by Blogger to WordPress Migration Service
----------------------------------------------------- Blogger Template Style Name: Snapshot: Madder Designer: Dave Shea URL: / Date: 27 Feb 2004 ------------------------------------------------------ */ /* -- basic html elements -- */ body {padding: 0; margin: 0; font: 75% Helvetica, Arial, sans-serif; color: #474B4E; background: #fff; text-align: center;} a {color: #DD6599; font-weight: bold; text-decoration: none;} a:visited {color: #D6A0B6;} a:hover {text-decoration: underline; color: #FD0570;} h1 {margin: 0; color: #7B8186; font-size: 1.5em; text-transform: lowercase;} h1 a {color: #7B8186;} h2, #comments h4 {font-size: 1em; margin: 2em 0 0 0; color: #7B8186; background: transparent url( bottom right no-repeat; padding-bottom: 2px;} @media all { h3 { font-size: 1em; margin: 2em 0 0 0; background: transparent url( bottom right no-repeat; padding-bottom: 2px; } } @media handheld { h3 { background:none; } } h4, h5 {font-size: 0.9em; text-transform: lowercase; letter-spacing: 2px;} h5 {color: #7B8186;} h6 {font-size: 0.8em; text-transform: uppercase; letter-spacing: 2px;} p {margin: 0 0 1em 0;} img, form {border: 0; margin: 0;} /* -- layout -- */ @media all { #content { width: 700px; margin: 0 auto; text-align: left; background: #fff url( 0 0 repeat-y;} } #header { background: #D8DADC url( 0 0 repeat-y; } #header div { background: transparent url( bottom left no-repeat; } #main { line-height: 1.4; float: left; padding: 10px 12px; border-top: solid 1px #fff; width: 428px; /* Tantek hack - */ voice-family: "\"}\""; voice-family: inherit; width: 404px; } } @media handheld { #content { width: 90%; } #header { background: #D8DADC; } #header div { background: none; } #main { float: none; width: 100%; } } /* IE5 hack */ #main {} @media all { #sidebar { margin-left: 428px; border-top: solid 1px #fff; padding: 4px 0 0 7px; background: #fff url( 1px 0 no-repeat; } #footer { clear: both; background: #E9EAEB url( bottom left no-repeat; border-top: solid 1px #fff; } } @media handheld { #sidebar { margin: 0 0 0 0; background: #fff; } #footer { background: #E9EAEB; } } /* -- header style -- */ #header h1 {padding: 12px 0 92px 4px; width: 557px; line-height: 1;} /* -- content area style -- */ #main {line-height: 1.4;} {font-size: 1.2em; margin-bottom: 0;} a {color: #C4663B;} .post {clear: both; margin-bottom: 4em;} .post-footer em {color: #B4BABE; font-style: normal; float: left;} .post-footer .comment-link {float: right;} #main img {border: solid 1px #E3E4E4; padding: 2px; background: #fff;} .deleted-comment {font-style:italic;color:gray;} /* -- sidebar style -- */ @media all { #sidebar #description { border: solid 1px #F3B89D; padding: 10px 17px; color: #C4663B; background: #FFD1BC url(; font-size: 1.2em; font-weight: bold; line-height: 0.9; margin: 0 0 0 -6px; } } @media handheld { #sidebar #description { background: #FFD1BC; } } #sidebar h2 {font-size: 1.3em; margin: 1.3em 0 0.5em 0;} #sidebar dl {margin: 0 0 10px 0;} #sidebar ul {list-style: none; margin: 0; padding: 0;} #sidebar li {padding-bottom: 5px; line-height: 0.9;} #profile-container {color: #7B8186;} #profile-container img {border: solid 1px #7C78B5; padding: 4px 4px 8px 4px; margin: 0 10px 1em 0; float: left;} .archive-list {margin-bottom: 2em;} #powered-by {margin: 10px auto 20px auto;} /* -- sidebar style -- */ #footer p {margin: 0; padding: 12px 8px; font-size: 0.9em;} #footer hr {display: none;} /* Feeds ----------------------------------------------- */ #blogfeeds { } #postfeeds { }

Saturday, January 16, 2010

The Seven Deadly Sins of eDiscovery

The Seven Deadly Sins, also known as the Capital Vices or Cardinal Sins, is a classification of the most objectionable vices which has been used since early Catholic times to educate and instruct followers concerning (immoral) fallen man's tendency to sin. The final version of the list consists of wrath, greed, sloth, pride, lust, envy, and gluttony.

Comparing the tendencies of the players in the complex and high stakes world of eDiscovery in 2010 to The Seven Deadly Sins may be a bit over the top. However, there are some "Deadly" tendencies occurring within eDiscovery that are worth noting. Please note that this list does not necessarily correspond to the list of Seven Deadly Sins and several of the items on my list cover multiple sins and I decided to leave wrath off as it is the inevitable result from our clients when we commit any of the other sins.

eDiscovery Sin # 1: Thinking that eDiscovery is Just the Latest Fad
Whether it's pride or sloth, there are some lawyers that believe that eDiscovery is just the latest technology fade designed by the technology vendors to complicate and increase the cost of the legal process. I have heard many within the legal community express the view the principles of evidence have not changed, the process of discovery has not changed and that eDiscovery is not something that should change the basic tenants of litigation.This is not to say that intricate processes of litigation and the associated unique knowledge of how to best navigate that process that litigators have spent years learning and perfecting are no longer valid. However, litigators need to "change with the times", educate themselves regarding what effect eDiscovery will have on their practice and adjust, mature and evolve appropriately.

With the volume of Electronically Stored Information continuing to increase at an accelerating rate and the associated changes to the Federal Rules of Civil Procedure (FRCP) and now many of the State and Local Rules of Civil Procedure, eDiscovery has already changed litigation and will continue to be the driving factor behind additional dramatic changes to the way in which we litigate.

eDiscovery Sin # 2: Lusting after eDiscovery Technology
Although pride and sloth may be clouding the judgment of some within the legal community to the realities of eDiscovery, lust for eDiscovery technology to magically solve all of the issues within the eDiscovery lifecycle may be just as bad.The best example of this is the current debate that is going on within the Early Case Assessment (ECA) arena. Many are lusting after ECA technology as the "Silver Bullet" that is going to magically uncover/produce all of the requested/required document and reduce the cost of eDiscovery with the simple click of a button.Of course on the opposite side of this argument are the prideful and in some cases slothy (is that a word?) litigators that believe that technology has no place in litigation and that all Electronically Stored Information (ESI) should be TIFFed and filed in legal boxes so that lawyers can start to apply sticky notes.The best use of ECA technology is obviously some middle ground where litigators educate themselves regarding the value of ECA technology, possibly employ some ECA technology experts (maybe the ECA vendors and consultants) and utilize ECA technology as a tool.

eDiscovery Sin # 3: Hoarding Data
Gluttony, derived from the Latin gluttire meaning to gulp down or swallow, means over-indulgence and over-consumption of food, drink, or intoxicants to the point of waste. As the saying goes, "if a little bit is good, a whole lot has to be better? " In the world of eDiscovery this has translated into over collection or an overly long and intrusive data retention policy.Obviously if you hoard your data and/or over collect, then you will never be accused of either purposeful or inadvertent destruction of potential responsive data. The obvious danger with this approach is that if you keep everything than it could eventually be "discoverable". And, having watched the TV show called "Hoarders" as few times, many of the houses and people that Hoarders crew works with remind me of the enterprises and the IT and or data management executives that I work with in regards to their data retention policies. The similarities are surprising.On the opposite end of this argument , I have heard many General Counsel and several outside counsel point out that if we don't have it then we don't have to produce it in regards to a very aggressive data retention policy.

Obviously, the best approach is to develop a data retention policy for your organization that follows the standard guidelines for your industry and then let common sense rule the gray areas of when and when not keep data.

eDiscovery Sin # 4: Charging Too Much for eDiscovery Services and Technology
The world of eDiscovery for the past 5 years as been the equivalent of the wild west and has in most cases provided a license to steal for eDiscovery consultants, technology vendors and the law firms that they have been working with (I guess I won't be making many friends in the industry with that comment?). This is obviously the sin of Greed at its finest. Unfortunately, the result has been that only the very wealthy have been able to afford access to the legal system due to the extreme cost of even evaluating the merits of a case.

Don't get me wrong in regards to where I stand on the issue of open markets and capitalism as I believe very strongly that markets will always self regulate and provide the best prices for all involved.And, as I have pointed out on this Blog, I think that the cost of eDiscovery is coming down and will continue to drop as uses become more educated in regards to "what it should cost", new technology replaces older technology and renegade providers introduce dramatically different pricing models such as fixed prices for processing data.

eDiscovery Sin # 5: Unreasonable Requests to Produce Documents
I believe that "unreasonable requests for production" has been one of the most outrageous (and therefore sinful in the context of this Blog post) and unfortunately trendy legal maneuvers currently employed in litigation. Even before the advent of Electronically Stored Information (ESI), it was not uncommon for counsel on both sides to try and overwhelm the opposing party with requests to produce. And, with the exponential increase in the amount of data now available and the known issues with collecting and producing this data, unreasonable requests have become even more prevalent. With some help from the changes to the Federal Rules of Civil Procedure and subsequent case law, it is becoming increasingly difficult to pull this stunt. However, I still see parties suffer the results of this legal maneuver every week. Hopefully the courts will finally catch on in 2010 and put a stop this approach once and for all.

eDiscovery Sin # 6: Thinking that eDiscovery is Just for Lawyers and the Legal Departments
I am not sure what category the sin of myopic behavior fits into. However, it is prevalent throughout eDiscovery. As I have posted on this Blog many times, I believe that eDiscovery is actually a subset of the broader arena of Governance, Risk and Compliance (GRC). Basically, I contend that the enterprise should collect ESI "one time", store it in a central repository based on reasonable and well thought out data retention policies and provide access to "that data" to everyone within the enterprise that needs access.Unfortunately, what I see is the same data being collected multiple times by multiple different organizations within the same enterprise through multiple and redundant systems. This is not a new phenomena to the world of enterprise Information Technology. I can't even count how many times I have sold completely redundant system to different organizations within the same enterprise for political reasons or because each thought that they we so unique that their needs couldn't possibly be met by another departments systems. And, I attribute most of this to political envy and hoarding.

However, in 2010 with cloud computing, cheap storage, open systems integration and Software-as-as-Service (SaaS) based applications, enterprises no longer have an excuse for redundant systems. And, as a stock holder in the Global 2000 and a consumer, its about time to see some of the cost savings trickle down and turn into lower prices and/or higher margins.

eDiscovery Sin # 7: Not Changing Your Business Model to Encompass eDiscovery
Again, whether it's pride or sloth, or in this case, ignorance, not changing your business model to encompass eDiscovery, may be the biggest miscalculation of sin that you can commit. eDiscovery is not an "art" as some would have you believe. It should not happen behind the curtain like some Wizard of Oz magic trick. eDiscovery is nothing more than another business process and therefore can be defined, managed and tracked just like any other business process within the enterprise. And therefore, eDiscovery and it bigger brother (Governance, Risk and Compliance) should be integrated into the overall business process of the enterprise. And, in some cases the business models should be changed to take into consideration eDiscovery things like legal holds and data retention policies, etc.

Just ask one of your Lean Six Sigma buddies if you don't believe me. And, BTW - if you don't have a Lean Six Sigma buddy, contact me and I will introduce you to one because you are probably going to need one.

Comparing the tendencies of the players in the complex and high stakes world of eDiscovery in 2010 to The Seven Deadly Sins may be a bit over the top. However, there is always room for improvement in any system and in many cases just acknowledging that you have a problem many be the first step to making that improvement. Hopefully, my list will at least provide the foundation for thinking about what needs to be improved in eDiscovery. And, I would bet that every single of you has an even better list of The Seven Deadly Sins of eDiscovery.

Labels: , ,

Tuesday, January 12, 2010

Computer Aided Review Has Arrived

Over the past several months I have been blogging about the emergence of Hal 900 like experiences in eDiscovery and Web 3.0. As such, I have been following the work of Dr. Herb Roitblat regarding the utilization of semantic and related technology to reduce the cost of document review through computer aided document categorization.

In a Blog posting on the Orcatec Blog, "Information Discovery", on Monday January 11, 2009 titled, "Computer Assisted Document Categorization in eDiscovery ", Dr. Roitblat announces that the January issue of the Journal of the American Society for Information Science and Technology, 61(1):1–11, 2010, has an article by Roitblat, Kershaw, and Oot describing a study that compared computer classification of eDiscovery documents with manual review. It found that computer classification was at least as consistent as human review was at distinguishing responsive from non responsive documents. If having attorneys review documents is a reasonable approach to identifying responsive documents, then any system that does as well as human review should also be considered a reasonable approach.

Dr. Roitblat reports in his Blog posting that the study compared an original categorization, done by contract attorneys in response to a Department of Justice Second Request with one done by two new human teams and two computer systems.

He further reported that the two re-review teams were employees of a service provider specializing in conducting legal reviews of this sort. Each team consisted of 5 reviewers who were experienced in the subject matter of this collection. The two teams independently reviewed a random sample of 5,000 documents. The two computer systems were provided by experienced eDiscovery service providers, one in California, and one in Texas.

The documents used in the study were collected in response to a "Second Request" concerning Verizon's acquisition of MCI. The documents were collected from 83 employees in 10 US states. Together they consisted of 1.3 terabytes of electronic files in the form of 2,319,346 documents. The collection consisted of about 1.5 million email messages, 300,000 loose files, and 600,000 scanned documents. After eliminating duplicates, 1,600,047 items were submitted for review. The attorneys spent about four months, seven days a week, and 16 hours per day on the review at a total cost of $13,598,872.61 or about $8.50 per document. After review, a total of 176,440 items were produced to the Justice Department.Accuracy was measured as agreement with the decisions made by the original review team. The level of agreement between the two human review teams was also measured. The two re-review teams identified a greater proportion of the documents as responsive than did the original review. Overall, their decisions agree with the original review on 75.6% and 72.0% of the documents. The two teams agreed with one another on about 70% of the documents.About half of the documents that were identified as responsive by the original review were identified as responsive by either of the re-review teams. Conversely, about a quarter of the documents identified as nonresponsive by the original review were identified as responsive by the new teams. Although the original review and the re-reviews were conducted by comparable people with comparable skills, their level of agreement was only moderate.

Dr. Roitblat reported that they did not know whether this was due to variability in the original review, or was due to some other factor, but these results are comparable to those seen in other situations where people make independent judgments about the categorization of documents (for example, in the TREC studies). A senior attorney reclassified the documents on which the two teams disagreed. After this reclassification, the level of agreement between this adjudicated set and the original review rose to 80%. The two computer systems identified fewer documents as responsive than did the human review teams, but still a bit more than were identified by the original review.

One system agreed with the original classification on 83.2% of the documents and the other on 83.6%. Like the human review teams, about half of the documents identified as responsive by the original review were similarly classified by the computer systems.

Dr, Roitblat commented that, "As legal professionals search for ways to reduce the costs of eDiscovery, this study suggests that it may be reasonable to employ computer-based categorization. The two computer systems agreed with the original review at least as often as a human team did. "

If this is indeed true, the laborious and expensive task of document review could indeed be in the process of evolving to the next level of computer aided automation. Add in come simple computer aided voice technology and a Hal 900 experience may not be that far away.

For more information about this paper or to discuss computer aided document categorization, you may contact Dr. Roitblat at:

Labels: , , ,