This morning I read an absolutely outstanding Blog post on the Clearwell eDiscovery 2.0 Blog by Dean Gonsowski titled, “Automated Review in Electronic Discovery Re-visited”. I don’t agree with Dean’s conclusion that “the automated review dog is still not ready to hunt”.

However, I think that he did an tremendous job framing the issues and was spot on in regards to his assessment that litigators are both risk adverse and generally slow to adopt new technology approaches.

I could go through the 7 points that Dean makes and discuss how today’s “categorization” or automated review technologies such as Equivio>Relevance, CategorIx and new technology from Orcatec, to name just a few, can address each point in a legally defensible and cost effective way. However, in the essence of time, I would prefer to point out that I am aware of numerous case studies in which automated review dramatically reduced the volume of documents that had to be reviewed and therefore the subsequent overall cost of eDiscovery of and in other cases proved in a statistically significant manner that the manual review process had been a complete failure.

There is no doubt that litigators are both risk adverse and generally slow to adopt new technology approaches. However, I believe that the automated review dog is absolutely ready to hunt and will prove itself to be a indispensible tool to the field of eDiscovery.

Manual review is still the major expense in the eDiscovery process and therefore any litigator that is not trying to find new ways to reduce those costs is doing his/her clients a disservice. Given this assumption, I think that the real debate should be whether or not the old legal dogs that won’t adopt new technology such as automated review and are therefore are no longer capable of hunting at the levels of the new dogs, should be retired? I suspect that it will end up just being a natural process of selection.

The full text of Dean’s post is as follows:

Almost two years ago I wrote one of my first blog posts entitled “Review-less E-Discovery Review.” Despite the tongue twister of a title, the post posited that “there is a very real possibility that we’re on the cusp of computers taking over a significant e-discovery task for attorneys.” I’d like to take a look and see how much (if at all) my prognostications have materialized.

A cynic might think that this is the moment where E-Discovery 2.0 jumps the shark. But no, this isn’t one of those sitcom episodes where they flashback to previous shows as an easy way to recycle content. Instead, it seems useful to see how the legal market has evolved from a litigation workflow perspective, particularly with some vendors touting the benefits of review-less technologies like predictive coding.

In the original blog, I noted that there was a “scenario where a non-manual review methodology may make sense” (while importantly noting that “this approach is not without risk”). Since my last post there has been the successful adoption of Evidence Rule 502,which makes this methodology (at least conceptually) safer.

But again (imagine dreamy flashback mode), here were the guidelines I previously proffered:

Large data set. This may sound a bit obvious, but a non-manual approach is best suited for large, unwieldy data sets. The corpus doesn’t need to be in the terabytes, but the data set should be evaluated in term of discovery processing costs and attorney review estimates.

Short Production Timelines. Once the above calculations are conducted, the next step is to determine if a human based review could even conceivably be conducted in the given time frame. In many instances, an eyes-on review process just won’t be feasible since there won’t be enough bodies to throw at the problem.

Next Gen “PAR” Tools. In order to pull this “review-less” review process off, both safely and quickly, the responding party needs to have access to fast, robust processing, analysis and review (“PAR”) tools. Certainly, it’s possible to have this scenario work with an e-discovery service provider, if they have the capability.

Relatively Small Amount in Controversy. For the time being, this approach should not be considered for any “bet the company” litigation, nor anything with significant downside risk (governmental inquiries, punitive damages, class actions, 2nd requests, etc.). Yet, for many standard commercial lawsuits, corporate investigations, HR claims, etc. this review-less approach may be worth considering.

Ability to Use a Clawback Provision. Entering into a clawback provision with the opposition is mandatory in this methodology since the chances of an inadvertent production are statistically ever-present. Yet, until Evidence Rule 502 is resolved, there will always be a risk that the clawback won’t be enforceable against 3rd parties.

Non-governmental Production. Most information in governmental productions becomes part of the public record, meaning that a clawback isn’t going to be feasible. Here, trade secret information, personally identifiably data and the like would be disastrous if pushed out into the public domain.

The goal of this post is to see if this dog is any more ready to hunt than it was two years ago. The short answer (right now) appears to be: No.

We all know that litigators are both risk adverse and generally slow to adopt new technology approaches. This is particularly true when there’s a perception that they won’t have insight into the technological black box behind automated coding/tagging decisions. Litigators are understandably sensitive about the ability to prove up the reasonability of their search and review processes. This “reasonableness” requirement lines up both with the Victor Stanley requirements and FRE 50(b), which eliminates the chance of a waiver only “if the holder of the privilege or work product protection took reasonable precautions to prevent disclosure.”

Given this ongoing hesitancy, the question remains shouldn’t we be seeing more movement in automated review than the glacial progress that’s been achieved to date, particularly with the known shortcomings of the eyes-on review process? Most are familiar with the 1985 STAIRS study by Blair and Marion where the percentage of relevant documents lawyers thought they had found using Boolean Keyword searches was 75% – when the percentage they actually found was 20%.

But, despite the known deficiencies of eyes-on review it follows into the “go with the devil you know” mindset that often makes sense when dealing with judges and juries who aren’t likely to grok newer-fangled approaches.

In addition to these high-level, almost dogmatic challenges, there is one other tactical element I’d add to my previous list (of 6 factors).

7. All documents processed up-front (no rolling collection). I’ve heard some in the trenches e-discovery experts claim that they’ve never had a case that didn’t involve at least some level of incremental data collections. Whether this is an overstatement is immaterial. The fact is that a large number of e-discovery projects involve ESI that is collected (and then processed) in dribs and drabs. This if often a good thing, largely attributable to the incremental (start slowly) nature of a well thought out e-discovery project where a smaller number of initial custodians are processed, then ECA is conducted and only then is the additional ESI added to the corpus. This common methodology causes some significant heartburn for a review-less methodology since the ever changing nature of the corpus makes it difficult/impossible for a sample to be truly extensible to what will eventually be the entire data set. For this reason, the review-less approach should be limited to where the entire corpus is collected and processed at once.

In sum, the seven foregoing factors appear to still be largely valid and create an environment where an automated, review-less methodology will only make sense in a relatively rare set of circumstances. This may change in the future, but given the risk adverse DNA of most litigators I can’t imagine this tipping point happening any time soon

Labels: Automated Review, CategorIx, Categorization, Clawbacks, eDiscovery, Equivio Relevance

posted by Charles Skamser @ 10:49 AM