Judged by AI

AI is increasingly used to predict our tastes in entertainment and media to ensure the next “big thing” isn’t missed. And now it is capable of predicting legal cases’ outcomes. 

The algorithm or "AI judge” has examined 584 cases relating to privacy, fair trials, and torture or degrading treatment; all three violations of the European Convention on Human Rights. In 79% of the cases its verdict matched the one that was given by the real court, yielding an impressive accuracy rate.

The algorithm analysed case text, searching for patterns and classifying each case as a “violation” or “non-violation”. An equal number of “violation” and “non-violation” cases were selected to prevent any chance of bias. It is a notable landmark as the first time an AI has been able to make a prediction based on the data that was submitted. Previous studies have made predictions based on the nature of the crime or the policy position of each judge.

Critics have claimed that an AI could never truly grasp the subtle nuances of a case in the way that a human can, though the success of an AI to win at the game Go shows that it is capable of learning “instinctive” behaviour and that it could have a chance in the future. The cases that it tended to get wrong were those where there were two similar cases, supporting the criticism that it couldn’t detect the finer subtleties.

For now, it is in no danger of replacing human lawyers and judges. Instead, it’s being highlighted as a useful tool for examining large sets of data and highlighting cases that will most likely be violations of the European Convention on Human Rights.

Next up for the AI judge is further testing with more data while the researchers refine the algorithm.