Results are in from the first case where a judge mandated the use of predictive coding, despite initial objections from the plaintiff that the technology is not as effective as human review. The Wall Street Journal Law Blog reports predictive coding used in the case found 81% of relevant documents.
The ABA Journal compares these results to the 60% accuracy rate for human reviewers estimated by a 2011 Richmond Journal of Law & Technology article. Some claim the comparison proves the superiority not only of this particular review technique, but also of artificial intelligence over lawyers.
This conclusion might be jumping the smoking gun, literally. What if the most relevant data is contained in the missing 19%? Initially, the Global plaintiff argued that all relevant evidence must be produced, which is, after all, the goal of defensible discovery.
Predictive coding, also known as Technology Assisted Review (TAR) and Computer Assisted Review (CAR), is a very new review method. Contrary to what eDiscovery headlines may lead you to believe, TAR is currently used by only a small portion of the legal community. Industry standards are not set, and there is little agreement about when and how it should be used. This disagreement can lead to expensive and often futile negotiations with opposing counsel.
When predictive coding is employed, a team of highly-paid human experts, specializing in statistics, law and technology, must train the computer in what is relevant by coding a sampling of documents (in the Global case, 5,000 documents). The coding is then applied to the rest of the documents to determine relevancy. If the initial sample is not correctly coded, the results will be drastically inaccurate. Therefore, human intelligence is still essential to the process.
In addition, Global is a single case with a relatively small sample size. Even if predictive coding continues to provide consistent, reliable results, it is not a one-size-fits-all approach for document review. If it is widely adopted in the future, predictive coding will never replace other search technologies or human intelligence; it will simply be another option in your litigation toolkit.
In most cases, a carefully planned discovery process combined with a multimodal search approach yields the most accurate results for the least cost.
Drawbacks of predictive coding
- Small mistakes early in coding can create false positives and miss relevant documents.
- Accidental disclosure of privileged information is more likely.
- For the sake of transparency, CAR/TAR/predictive coding often requires disclosure of all non-privileged training documents and coding to opposing counsel, regardless of relevancy. How much do you want opposing counsel to know?
- The cost of predictive coding/CAR/TAR negotiations can cancel out any savings the approach might otherwise generate.
- Predictive coding/CAR/TAR only makes sense for extremely large sets of ESI. How big is your data?
Technology should be used to reduce review volume, not to replace human review. Human intelligence cannot be fully automated. A concept-clustering search approach combines the best of human and artificial intelligence.
Contact a WarRoom litigation support consultant to help you determine the strategy that minimizes your risk and cost.