This Page

has been moved to new address

Percieved Value of Technology Assisted Review (TAR)

Sorry for inconvenience...

Redirection provided by Blogger to WordPress Migration Service
----------------------------------------------------- Blogger Template Style Name: Snapshot: Madder Designer: Dave Shea URL: / Date: 27 Feb 2004 ------------------------------------------------------ */ /* -- basic html elements -- */ body {padding: 0; margin: 0; font: 75% Helvetica, Arial, sans-serif; color: #474B4E; background: #fff; text-align: center;} a {color: #DD6599; font-weight: bold; text-decoration: none;} a:visited {color: #D6A0B6;} a:hover {text-decoration: underline; color: #FD0570;} h1 {margin: 0; color: #7B8186; font-size: 1.5em; text-transform: lowercase;} h1 a {color: #7B8186;} h2, #comments h4 {font-size: 1em; margin: 2em 0 0 0; color: #7B8186; background: transparent url( bottom right no-repeat; padding-bottom: 2px;} @media all { h3 { font-size: 1em; margin: 2em 0 0 0; background: transparent url( bottom right no-repeat; padding-bottom: 2px; } } @media handheld { h3 { background:none; } } h4, h5 {font-size: 0.9em; text-transform: lowercase; letter-spacing: 2px;} h5 {color: #7B8186;} h6 {font-size: 0.8em; text-transform: uppercase; letter-spacing: 2px;} p {margin: 0 0 1em 0;} img, form {border: 0; margin: 0;} /* -- layout -- */ @media all { #content { width: 700px; margin: 0 auto; text-align: left; background: #fff url( 0 0 repeat-y;} } #header { background: #D8DADC url( 0 0 repeat-y; } #header div { background: transparent url( bottom left no-repeat; } #main { line-height: 1.4; float: left; padding: 10px 12px; border-top: solid 1px #fff; width: 428px; /* Tantek hack - */ voice-family: "\"}\""; voice-family: inherit; width: 404px; } } @media handheld { #content { width: 90%; } #header { background: #D8DADC; } #header div { background: none; } #main { float: none; width: 100%; } } /* IE5 hack */ #main {} @media all { #sidebar { margin-left: 428px; border-top: solid 1px #fff; padding: 4px 0 0 7px; background: #fff url( 1px 0 no-repeat; } #footer { clear: both; background: #E9EAEB url( bottom left no-repeat; border-top: solid 1px #fff; } } @media handheld { #sidebar { margin: 0 0 0 0; background: #fff; } #footer { background: #E9EAEB; } } /* -- header style -- */ #header h1 {padding: 12px 0 92px 4px; width: 557px; line-height: 1;} /* -- content area style -- */ #main {line-height: 1.4;} {font-size: 1.2em; margin-bottom: 0;} a {color: #C4663B;} .post {clear: both; margin-bottom: 4em;} .post-footer em {color: #B4BABE; font-style: normal; float: left;} .post-footer .comment-link {float: right;} #main img {border: solid 1px #E3E4E4; padding: 2px; background: #fff;} .deleted-comment {font-style:italic;color:gray;} /* -- sidebar style -- */ @media all { #sidebar #description { border: solid 1px #F3B89D; padding: 10px 17px; color: #C4663B; background: #FFD1BC url(; font-size: 1.2em; font-weight: bold; line-height: 0.9; margin: 0 0 0 -6px; } } @media handheld { #sidebar #description { background: #FFD1BC; } } #sidebar h2 {font-size: 1.3em; margin: 1.3em 0 0.5em 0;} #sidebar dl {margin: 0 0 10px 0;} #sidebar ul {list-style: none; margin: 0; padding: 0;} #sidebar li {padding-bottom: 5px; line-height: 0.9;} #profile-container {color: #7B8186;} #profile-container img {border: solid 1px #7C78B5; padding: 4px 4px 8px 4px; margin: 0 10px 1em 0; float: left;} .archive-list {margin-bottom: 2em;} #powered-by {margin: 10px auto 20px auto;} /* -- sidebar style -- */ #footer p {margin: 0; padding: 12px 8px; font-size: 0.9em;} #footer hr {display: none;} /* Feeds ----------------------------------------------- */ #blogfeeds { } #postfeeds { }

Wednesday, June 20, 2012

Percieved Value of Technology Assisted Review (TAR)

eDiscovery Solutions Group (eDSG) ran a public poll the week of June 11, 2012 asking respondents to vote on "The Number One Reason for Using Technology Assisted Review (TAR)".  The poll was open for 7 days and there were 476 responses.

eDSG marketed this poll to the eDiscovery and legal community through specific and targeted LinkedIn groups such as The Computer Assisted eDiscovery GroupThe eDiscovery Solutions Group, The Early Case Assessment Association and via Twitter at @eDiscoveryGroup

Respondents did not have to identify their position within the eDiscovery market and were able to vote more than once.
The results of the poll are as follows:

Check the Accuracy of Human Review = 14%
Reduce the Document Set for Human Review = 29%
Replace Human Reviewers = 57%

To see a graphic representation of this poll and of all eDSG polls, please go to the eDSG website:

Comment and Analysis on Results

Most of the recent interest with Technology Assisted Review (TAR) or Predictive Coding, as it is sometimes called, stems from Da Silva Moore (2012 U.S. Dist. LEXIS 23350 (S.D.N.Y. Feb. 24, 2012)), a decision rendered by Magistrate Judge Andrew Peck and confirmed by U.S. District Judge Andrew Carter (11 Civ. 1279 (ALC)(AJP)).  However, TAR is not new and its value and reliability has been studied for years.

As an example, in a 2009 study published in the Journal of the American Society for Information Science and Technology by Roitblat, Kershaw and Oot, titled, "Document Categorization in Legal Electronic Discovery: Computer Classification vs. Manual Review", two TAR systems each agreed with an original human review on about 83% of the documents reviewed. By comparison, two new human teams only agreed on about 73% of the documents.   As a result, the Roitblat, Kershaw and Oot study concluded that TAR was, "no worse than using human review."

Other studies have also addressed the “myth” that human document review is somehow inherently more reliable than could be obtained with TAR (e.g., Grossman & Cormack, 2011; Baron, et al., 2009).    As such, the evidence has been clear for some time that TAR can be very effective.

However,  in an industry known for evolving and embracing new technology at the pace of a snail, with a built in prejudice and comfort level for human review,  I am not convinced that the market has agreed upon the best practice for using TAR.  As an example, I have witnessed users utilizing TAR to cull down very large data sets with statistical "hit rate" settings in the high 90%, leaving the "important" review work to humans.  I have also seen users utilizing TAR to "spot check" the accuracy of human reviewers with the necessary "rework" going back to a different set of human reviewers.   During this early adaptor phase in the evolution of TAR, neither of the uses that I sited for TAR are necessarily wrong.  I just believe that the research and evidence already justifies the more productive and financially rewarding use of TAR is to replace human reviewers.

Given all of this, I was actually very pleased to see that 57% of the voters in this eDSG poll agreed that  the number one reason for using Technology Assisted Review (TAR) was to replace human reviewers.  However, more in line with what I would have thought, 43% of the respondants voted otherwise. I would suspect that these are the mainstream buyers and the laggards (see Crossing the Chasm).  Or maybe these were the voters that own and operate offshore document review organizations?

In any case, maybe TAR is going to be the "tipping point" technology that finally drives the legal industry to explore and possibly embrace to the value of technology?  Let's hope so.

Labels: , , , , ,


Post a Comment

Subscribe to Post Comments [Atom]

Links to this post:

Create a Link

<< Home