Predictive Coding Beats Human Document Review Lawyers
Robots have taken many of the assembly-line jobs previously occupied by humans, and now it looks like computers may be able to take over what amounts to legal assembly-line work: document review. A Virginia judge ordered predictive coding to determine the relevance of over 1.3 million documents. The parties estimate the computer found about 81% of all relevant documents, which is a bit better than the 77% accuracy predicted by a 2011 article in the Richmond Journal of Law & Technology (pdf). By contrast, the same study estimated humans accuracy at about 60%.
I am not sure how much stock to put in those numbers, since the sample size is about as small as it gets. But I don’t find it hard to believe that a computer can outdo a bunch of low-paid temp lawyers shoved into a cubicle farm. I can’t imagine having to focus on page after page of bulk document production for an hour, much less 80 docs an hour, hour after hour, day after day.
But if computers are cheaper and more accurate, that would imply that a lot of document review lawyers won’t even have low-paid temp jobs. How many lawyers will this affect? I could not find any information on the number of lawyers doing document review work, but it’s safe to say that there will be a lot less work available to them. And since document review is not exactly great preparation for other lawyering work, I think it is also safe to say that many lawyers who had been doing document review will be forced to drop out of legal work entirely.
This may not be all that significant, since the law already has a glut of unemployed (and probably unemployable) law school graduates. A few thousand more will be barely noticed among the swelling ranks of unemployed lawyers.