Weigh Your Cases on Their Merits, Not Statistics

When a criminal defendant is deciding whether to go to trial or accept a guilty plea, there are a number of things to consider. The nature of the proposed plea and the likelihood of an acquittal at trial are two of the largest factors. But a defendant also has to take into consideration the sentence that could be imposed at trial. After all, if the chances of acquittal are low, but the offer is the same as the mandatory sentence, why not take a risk? All trials are, inherently, risks.

To mitigate the level of risk in civil cases, there are databases to consult that accumulate data on jury awards in cases based on certain criteria. In a recent post, Joshua Baron argued that lawyers should use an outside perspective, in the form of statistical data, to advise their clients about whether to proceed to trial. In the context of criminal cases, Mr. Baron writes that “[r]ather than focusing on the specifics of their case, [attorneys] should try to compare their case to the available statistical information about similar cases.” But I don’t think that’s spot on.

Defending a criminal case requires more than simply plugging numbers into a computer to see what the case is worth. Unlike civil cases, where there is a cold monetary figure to assess, criminal charges can involve incarceration, probation, and any other number of restrictions on the client’s freedom. Defendants must decide whether they want to gamble with a verdict of guilty and a potentially harsh sentence, or simply take a reasonable plea offer.

The question Mr. Baron presents is whether criminal attorneys should attempt to gather some kind of statistical data on similar trials to use as the focal point of advice to clients when deciding on a plea. The problem with that data is that it can be extremely misleading. After all, statistics can be made to say anything, right? This could just feed into the recommendation of the attorney, and the data could be misconstrued to bolster the argument of an attorney who just already has her mind made up and thinks the attorney should go to trial.

Verdict Predictability

Setting aside the potential for malleability of the data, let’s look at how predictable the data can actually be. First of all, the data cannot incorporate the inherent unpredictability of twelve people in a room. In addition to the normal unpredictability of juries, the data would not be able to account for many factors that play into success or failure at a trial. For instance, how do you measure success? Can a database reflect that a defendant admitted to the simple assault as part of the trial strategy, in order to argue against the aggravated assault? Or does that just show up as a conviction for simple assault?

Beyond tactical admissions, the data cannot predict other factors, such as:

  • Evidentiary rulings
  • Credibility of witnesses
  • Relative skill of the attorneys
  • Jury bias

These factors can vary greatly from case to case, even with similar charges involved. But an attorney must weigh these factors when making a recommendation to the client. Really, the attorney should discuss these factors with the client, so she is making a completely advised opinion on whether to proceed to trial. As Wade Coye points out in the comments on Mr. Baron’s post, “A fact here or fact there can mean the difference in a trial. Be prudent when approaching the decision to go to trial.”

The unpredictable nature of trial outcomes makes any kind of data on the issue somewhat suspect. If enough data was gathered and coded appropriately, it could be useful. But, as I discuss below, the statistics should not be the main thrust of the attorney’s advice to the client.

Sentence Predictability

Turning to the use of data to predict sentencing, defense attorneys run into many of the same problems as they would when predicting verdicts. Again, there are numerous factors a database would have a hard time accounting for, such as:

  • The judge’s discretion in sentencing
  • That particular prosecutor’s recommendation
  • Victim impact statements
  • The defendant’s criminal history
  • The defendant’s potentially mitigating personal history
  • Aggravating factors of the crime
  • Mitigating factors of the crime
  • Credibility of witnesses

Considering all of these factors, it seems that statistical data shouldn’t be weighed as heavily as Mr. Baron suggests in terms of predicting a sentence either.

To make the data more relevant, a lot of data must be collected. Because even if you narrow the data down to a particular state, that isn’t enough. Individual cities, counties, and judges can all vary somewhat in their sentencing schemes. For instance, in Western Pennsylvania where I practice, all DUIs have state mandated minimum sentences. But some judges impose those minimums with house arrest, while others require incarceration on a plea bargain. Moreover, if defendants take those cases to trial, some judges are more likely to impose a harsher sentence, while others will still give the mandatory minimum (albeit rarely). That means for the data to be valuable, the database would have to include information on what kind of sentences a judge accepts from a plea bargain, as well as the sentences imposed after trial.

The Weight of the Data

But, let’s assume for a moment that a database can somehow gather this amount of data, so that it may be useful to criminal attorneys. Let’s also take out the lack of a financial incentive and assume the database gets created, because I agree that there is not as much of an incentive for criminal defense attorneys to invest in this kind of data gathering. But should criminal attorneys use this data as a large part of their decision making process when advising their clients? I don’t think so.

Even prosecutors can see that all defendants should not be lumped together into one impersonal category. If defense attorneys use these comparative statistics as the main basis for a recommendation, they are failing in their duty.

A defense attorney’s duty is precisely the opposite of what Mr. Baron suggests. A defense attorney should focus on the specifics of each given client’s case in order to appropriately advise that client. By lumping all defendants together, we inch ourselves one step closer to becoming jaded hallway attorneys. That’s certainly not where I want to end up.



  1. Avatar James says:

    While I think that statistics do play an important role in deciding to take a case to trial or not, I agree with this post that statistics can be very misleading because they can be interpreted differently by different people. Statistics can be misinterpreted to bolster the argument of an attorney who already has there mind made up and thinks the attorney should go to trial. I think it is important to have a balance of data and merits when deciding to go to trial or not.

  2. Avatar Joshua Baron says:


    I love your post. Great criticisms.

    I agree that a defense lawyer has to take into account the specifics of their case. I don’t think I made that clear enough in my original post. But Kahneman argues that people tend to be overly optimistic the more specifics they know about an issue. For example: construction projects are almost late and more expensive than originally expected. If people were totally rational, we would expect people to learn from that repeated error and to correct it. Instead, there is a cognitive bias that leads to repeated errors.

    You are also right that a database that doesn’t have enough of the relevant factors included won’t be very useful. But I think it would be helpful to know if 90% of DUI trials result in convictions or the average sentence for felony theft is 45 days. Kahneman suggests that you set a spectrum of likely outcomes and then compare your case to that range: “Is our case average, above-average, or below average?” His research indicates that a decision taken with that process is less likely to be way off and more likely to be accurate.

    Once again, great post and thank you for reading my post so carefully. I’ve been following your columns and they are great.

Leave a Reply