Jusletter IT

I first met Erich Schweighofer in the early nineties through the DEXA series of conferences. His work then was mainly concerned with exploring the use of AI techniques to support legal tasks¹. His collaboration in this work was the classic AI and Law partnership of a legal expert, Erich, and a computer specialist, Werner Winiwerter. Pairings of a legal specialist with a computer specialist was a fruitful source of AI and Law research at the time. Examples of such partnerships include Don Berman and Carole Hafer², John Zeleznikow and Dan Hunter³ and Henry Prakken and Giovanni Sartor⁴. The idea of close cooperation between a domain expert and a computer specialist was central to Good Old Fashioned AI and Law⁵ (GOFAIL), which was largely working in the expert systems tradition⁶ in which knowledge engineers elicited knowledge from domain experts and represented it in executable form.

Today, however, the need for expertise is no longer recognised to the same extent. Increasingly «Artificial Intelligence» has come to mean the application of learning algorithms to large amounts of data⁷. Thus, in Chess, whereas Deep Blue, the first computer to beat a world champion⁸ relied on an evaluation function provided by chess experts, Alphazero, the currently most powerful chess program, developed by Deepmind, was trained solely via self-play⁹, and placed no reliance at all on the body of knowledge that has been developed in Chess over centuries. Even more remarkably, it was possible to learn the much more complicated game of Go in the same way.

Given the success of such techniques it is unsurprising that there should be a desire to apply them in law. Large amounts of data in the form of decided cases are available, and so it seems that it should be possible to train an algorithm to predict the outcome of future cases. An example of this work in the academic realm is a program designed to identify examples of human rights violations found by the European Court of Human Rights.¹⁰

Given this revolution in what is possible using learning algorithms on big data sets, should we not abandon GOFAIL in favour of the new AI? Perhaps not: it has always been argued by AI and Law practitioners that the decision is of secondary importance, and what matters is the explanation, the argument. The only form of explanation offered by Aletras et al was «the 20 most frequent words, listed in order of their SVM [Support Vector Machine] weight». These do not, however, look immediately promising: the list for topic 23 of article 6 predicting violation, for example, is:

«court, applicant, article, judgment, case, law, proceeding, application, government, convention, time, article convention, January, human, lodged, domestic, February, September, relevant, represented»

One doubts whether this would be the list expected by a legal expert, or that pointing to the presence of these words would be found a persuasive argument by the European Court of Human Rights.

I wish to argue that there are a number of differences between legal problems, especially those regarding making, or predicting, decisions on cases, and problems suitable for machine learning and knowledge discovery, which mean that we still need arguments based on legal knowledge, and the involvement of legal experts to produce and assess them.

In this section I will point to several differences between case law and domains in which machine learning is successful, to explain why GOFAIL still has a place in AI and Law.

Machine learning is retrospective (trained on past decisions), whereas case law is prospective (intended to influence future decisions). When deciding a case we are not discovering something common to the previous cases, we are creating a rule to decide a particular case, and which is intended to constrain future cases. This rule should be consistent with previous cases¹¹, but the new case may lead to a new theory and a reinterpretation of the existing cases¹². A new rule may broaden or narrow existing features, introduce new features distinguishing previous cases, which may lack the new feature or not have considered the feature, or represent a shift in the common values of society¹³ which the court is supposed to express¹⁴. Thus a rule which can be induced from the previous cases, and which a machine learning system would be expected to derive, may not be the rule which is actually applied to the current case (and to future cases). One of the great strengths of law is that its interpretation is dynamic, able to meet new situations and to adapt as society changes. But how it will develop cannot be predicted from a consideration of the past. A legal expert might be able to conjecture some trends¹⁵, but that art is beyond the capacity of current machine learning algorithms.

A Machine Learning system is considered successful if it correctly classifies enough future cases. The program in Aletras et al was deemed a success on the grounds that it got 79% of its predictions right. But percentages cannot be applied to the decisions of human judges. A new judge-made rule always classifies the case it which it is made correctly since the judge is empowered to say what is so. There is no independent «fact of the matter». Whether the rule is successful is shown by its future treatment: it succeeds if it is endorsed by a relevant consensus, and it survives any appeal process challenging it. This will require not just a rule, but also a convincing justification for that rule, in terms that will be understood by, and able to persuade, legal professionals. Law also aspires to a very high success rate. Whereas around 80% is reported as successful for most prediction programs, using a program knowing that 20% of legal cases would be wrongly decided would not be acceptable¹⁶. This is certainly true in decision making and prediction systems, although a less demanding standard may be applied to particular legal tasks. For example, for E-discovery¹⁷, which acts as a first pass filter, a lower precision may be acceptable. But for systems intended to support the application of law, there should be something closer to the 90%+ often achieved using GOFAIL techniques¹⁸. Improvements in the performance of the machine learning systems may be possible, but the 80% ceiling has not been raised for a long time, whereas the GOFAIL approaches consistently perform better.

Machine learning often does not provide explanations. Some ML systems do produce explanations, but others, like the European Human Rights prediction system of Aletras et al, do not. In Law the explanation is what matters. This has always been recognised in the AI and Law community. Even where black box techniques such as neural networks were used in AI and Law, an effort was made to extract meaningful rules from the trained network¹⁹. Note, however, that Bench-Capon (1993) showed that highly accurate performance (let alone 80%) was no guarantee that a complete set of accurate rules had been discovered. Rules have also been produced by inductive logic programming²⁰ and using data mining techniques²¹. However, the rules they produce need to be assessed for credibility: a successful rule will have a convincing justification. A legal explanation is normally not simply in terms of a rule, but why that rule is the rule that should be followed. These reasons can readily be obtained from legal experts, or from written decisions, but a machine learning system will tend to quote the numbers of cases supporting it, which provides no more than a probabilistic justification, which will always leave room for doubt as to whether it applies in a particular case.

In machine learning we typically need a large dataset (e.g. of precedent cases) which we are trying to classify, whereas legal decisions are primarily concerned with a single case and a handful of relevant precedents. Machine Learning requires large data sets because its rules have only a statistical justification, but case law systems can work with a few landmark cases, on which intellectually satisfying arguments can be based. HYPO²² used fewer than 30: CATO²³ used 148, IBP²⁴ used 186, and reasonable theories have been developed for the wild animals domain with only half a dozen²⁵. Machine Learning and Data Mining derive their authority from their being based on large quantities of data whereas case law derives its authority from the status of the court or of the status of the judge or the cogency of the argument. Machine Learning needs many ordinary cases, so that the core of the concepts can be established. Case law is driven by hard cases, which push (and so define) the boundaries of the concepts. It is at these boundaries that legal disputes arise: easy cases in the core of the concept do not really require support to settle them.

Past data may not be homogeneous. Attitudes and case law change over time and later cases are preferred to older cases, so that it can be difficult to determine the degree of relevance of a decision. Although case law is often considered in systems without a notion of sequence²⁶, in fact sequence is important, and domain theories evolve over time as landmark cases appear²⁷. This means that some of the cases may be applying superseded doctrine or were heard in a social situation that no longer applies. Some decisions may even have been explicitly overturned. Thus deciding what cases to use for training the algorithm needs a great deal of care so as not to include decisions that may mislead. This itself requires a good deal of expertise and an understanding of the domain which GOFAIL would have encoded directly. Moreover, the resulting dataset may, once potentially misleading cases have been weeded out, be too small for the algorithms to perform optimally.

It is not unusual for a substantial number of past decisions to be simply wrong. This is particularly true of routine welfare benefit decisions. Groothuis and Svensson²⁸ drew attention to this in connection with the Netherlands General Assistance Act, and reported experiments which suggested that an error rate of more than 20% was typical. The problem is not confined to the Netherlands: The US National Bureau of Economic Research on US Disability Insurance reports «inconsistencies that suggest a potentially high rate of errors.»²⁹ In the UK, an official UK Publication produced by the Committee of Public Accounts reported «Too few decisions are right first time, with a error rate of 50% for Disability Living Allowance. There are also regional differences in decision making practices that may lead to payments to people who are not eligible for benefits.»³⁰ Thus large quantities of past data may contain a significant amount of erroneous material, and it is important that any algorithm be robust in the face of such noise. This was an explicit point in both Mozina et al (2003) and Warda et al (2009)³¹, but does not often receive attention in more recent work such as Aletras et al (2016)³². The problem is less acute for GOFAIL, which, rather than working with cases decided, often under time pressure, by low level adjudicators, is working with high level, often landmark, cases, which are heard by the very best judges with enough time to be careful, and which are often subject to a great deal of scrutiny and comment. But if we understand that our data contains erroneous decisions it is essential that the errors are not perpetuated in an AI system. Ironically it is just this sort of low level, high volume, routine cases where errors are most likely, that are most likely to be seen as suitable for decision using AI. Certainly these cases were seen as the most ripe for the application of GOFAIL techniques³³

In addition to simple errors, it has been shown that past decisions also contain bias and prejudices, consious and unconscious³⁴. Training systems on data containing bias and prejudices will present problems. A striking example from a UK newspaper is

«A stunning report claimed that a computer program used by a US court for risk assessment was biased against black prisoners. The program, Correctional Offender Management Profiling for Alternative Sanctions (Compas), was much more prone to mistakenly label black defendants as likely to reoffend ֠wrongly flagging them at almost twice the rate as white people (45% to 24%), according to the investigative journalism organisation ProPublica.» ³⁵

The possibility of bias presents something of a dilemma for Machine Learning systems. If the aim is to predict decisions and these decisions are subject to bias, the bias will need to be reflected in the program in order to achieve a satisfactory success rate. On the other hand, it is acknowledged that the bias should not exist, and so should not be reflected in the program. We do not want the system to learn prejudices, but to eliminate them³⁶. Note that GOFAIL does not suffer from this problem: its explicit rules can be examined, and biased rules rejected.

Case law is not simply a matter of identifying and applying an appropriate rule. On the contrary much of a legal dispute is taken up with arguments as to what rule should be applied. This is particularly clear in the US Supreme Court where much of the Oral Hearing stage is taken up with attempting to formulate an appropriate rule³⁷. Also there is often no consensus: we often have majority and minority opinions arguing for different rules. These arguments are often not about the facts of the case, but rather reflect ideas of purpose or value³⁸. Sometime these arguments require consideration of trade-offs between and balancing of values³⁹. It is hard to see these kinds of arguments emerging from contemporary machine learning algorithms.

The last section discussed some reasons why Machine Learning techniques might not be as successful as it currently hoped in applications intended to support legal decision making, the traditional province of GOFAIL.

There are, however, other legal tasks to which Machine Learning may well prove the most suitable technique. One is e-discovery⁴⁰ where many thousands of documents must be examined to find those which are relevant. In this task false positives are less of a problem, and so the high success rate demanded by legal decisions can be relaxed. Moreover, the documents selected can be assessed according to a gold standard, and so problems of differing arguments do not arise and the performance of the system can be objectively evaluated. Change is also less of an issue: the topic may remain constant even if the law changes.

Other clustering and information retrieval tasks may also be susceptible to Machine Learning techniques. It is to Erich’s credit that while others⁴¹ were attempting to apply techniques such as neural networks to legal decision making, he was looking at other legal tasks such as automated indexing and document clustering⁴².

In March 2015 the British Broadcasting Corporation presented a programme entitled When Robots Steal Our Jobs⁴³. The theme of the programme was that the new AI which used algorithms to learn from large amounts of data meant that AI would start to replace white collar workers, just as a previous generation of robots had replaced workers in manufacturing. The general promise of, and the hype surrounding, the new AI made practioners of GOFAIL such as myself (and perhaps Erich) wonder whether ours might be one of the jobs under threat. After all, if Alphazero could teach itself Chess and Go, why should there not be an AI program capable of teaching itself law? And then GOFAIL would be as obsolete a skill as making flint arrowheads.

In this paper I have advanced some reasons why Erich and I should not worry. Although the new AI may have some legal applications, the heart of AI and Law, support for legal decisions, is likely to remain resistant to its progress. The nature of legal data, subject to disputed interpretation, change and reinterpretation, is so very different from the data in other fields that one should not generalise too hastily from success in other fields to success in law. Law is a social construct, not a natural phenomenon. In fact, GOFAIL remains alive as well, and computer scientists continue to collaborate with lawyers on successful applications which outperform the best that the new AI can offer⁴⁴. So Erich (and I) can rest secure in the knowledge that GOFAIL is likely to last longer than our careers.