Jusletter IT

The potential of AI & law methods in law has been mostly not properly used. Thus, too many traditional lawyers consider this branch of legal science as irrelevant, at least for the time being. A promising way out may be a highly developed and semi-automatic analysis of the legal system using a better systematisation of knowledge and analytical tools for lawyering. The goal of legal data science is to complement the existing methodology of law with the new computer-based methods, and to bring it into a theoretical framework. In previous research, we have developed the 8 views/4 methods/4 syntheses approach of legal data analysis.¹ The need is evident, considering that a textual representation is insufficient for today’s needs of the legal profession. Paraphrasing the words of Layman Allen in 1957², legal data science provides razor sharp tools for the analysis of the legal system. «The fight for justice and law» – in the sense of Rudolf von Jhering³ – goes on, but now including legal data science.

In this paper, we will focus on the delivery of the desired products of legal knowledge representation using also AI & law methods. The options of knowledge can be systemized as follows: Besides the text (multimedia) corpus, 7 other views are now – more or less – required: metadata, citation network, user view, logic view, ontological view, visualisation view and argumentation view. Further, lawyers like a synthesis of the views in a practice-oriented form: commentary or manual, Dynamic Electronic Legal Commentary (DynELC)⁴, citizen information system and case-based synthesis. A lot of this analysis is done manually but the lack of sufficient resources becomes more and more evident. A strong co-operation between man and machine is required.

Legal information retrieval has been around for a long time but the analysis has to be improved in order to allow higher recall and better ranking. A semi-automatic analysis for metadata, citations and ontologies can be obtained by computational linguistics and machine learning. Further information can be derived with document categorization and the semi-automatic generation of thesaurus descriptors; also considering multilingual jurisdictions. The tricky but manageable automatic generation of hypertext links should be supplemented by an efficient ranking. The automatic generation of temporal relationships allows the production of consolidated versions. Data-friendly generation of user behaviour gives strong insights of their assessment. The semi-automatic generation of summaries of documents allows a much easier and more efficient analysis of documents. Highly needed tools for segmentation of texts into logical rules and arguments exist but have to be improved. Visualization tools allow an easier understanding of complex rules. The user needs are taken into account in sophisticated ranking algorithms: document vs. requirements, document in the corpus, document in the citations network, document in the timeline etc.

Legal data analysis is still insufficiently developed. Experts do not use such tools in order to generate synthesis tools like commentaries. This paper describes the potential, the state of the art and shows existing strengths and lacunae.

Any comprehensive theory of legal data science requires a definition of the law as a textual or multimedia representation. With Hans Kelsen⁵ the law is seen as a normative order that governs the behaviour of people. A norm belongs to this system when «man should behave in certain ways because of the norm.» In cyberspace with its multilevel regulatory structure, an extension is needed. Law is understood as the basic consensus of the stakeholders of the legal systems on «ought». Legal informatics describes this body of legal rules, e.g. the legislative processes and the respective documents. These documents are the core of the legal retrieval system (legal information system). This is still primarily done in a textual way, but is increasingly being enriched with multimedia elements. This corpus is huge: at least several gigabytes of data, millions of documents over a million records in the database dictionary, etc. Recently, an extension of legal materials to implementation practice or social web data can be observed or is strongly considered.

The goal of legal data science is to complement the existing methodology of law with the new computer-based methods, and to bring it into a theoretical framework. For a long time, work on legal knowledge has been only in the focus of legal theory and information science. Only since the late 1950s – with the start of research on legal information retrieval – appropriate progress can be noted (term retrieval, text retrieval, metadata, citations, search technologies, user interface, telecommunications, etc.). The theory distinguishes 8 views – text view, metadata view, network view, user view, logic view, ontological view, visualisation view and argumentation view.

In the last decades, text corpora with search engines have become the standard of knowledge representation but are now considered as insufficient. The «views theory» of Lu and Conrad⁶ – documents view, annotation view, citation view and user view describes this enlargement and constitute a basis for this work. However, it should be extended by four other views – legal logic, legal ontologies, legal visualisation and legal argumentation. Further, the works of Sowa⁷, Fiedler⁸, Zeleznikow/Hunter⁹, the authors of the book collection of Yearwood/Stranieri¹⁰ as well as our own work on knowledge representation of the law¹¹ have been taken into account.

The formalization of the law has started in the 1970s; the research is well documented in the proceedings of the conferences ICAIL¹² and JURIX¹³. In this paper, we argue for a stronger integration of methods of information retrieval, AI & law, legal language processing but also using all methods of data analysis useful for the legal discipline.

Knowledge representation of law is not just about the documentation itself; each view represents further insights on the law itself. The use of all available media and methods – language, meta-knowledge, visualisation, structure, mathematics and statistics, logic, ontologies, formal structures, etc. – implies also that legal «knowledge sources» must be integrated into this knowledge structure, however, keeping the holistic and harmonious picture. Language, visualisation, structure, etc. are also essential methods of human thinking itself. The principle applies that more quantity of methods and analysis ultimately results in higher quality. The more comprehensively all media, factors and methods are used, the better is the structural analysis of the law.

The text corpus (Lu/Conrad: textual set of evidence) consists of all relevant documents of the legal system: statutes, regulations, court decisions, literature, administrative practice, as well as complementary legal practice such as briefs or draft legislation. The basis is the triad of norms, court decisions and literature, however, to be extended by the now huge body of «soft law». Nowadays, there are hundreds of types of documents in any jurisdiction. Recently, implementation practice and social networks are getting higher relevance.

The description view contains the metadata of the law (Lu/Conrad: annotation view). Information science has developed a good methodology to describe materials in order to classify and summarize important content or extract relevant information. The quality of this description is highly variable (e.g. the high-level West’s Key Number System¹⁴ or the formidable CELEX metadata system¹⁵). In general, metadata is only useful for those with sufficient familiarity. However, AI & law methods allow intelligent use of metadata, e.g. setting the proper context of the communication, e.g. persons, roles, purpose, topic etc. without much user input.

The citation network (Lu/Conrad: citation network view) describes the relations between the documents. It has always been a key topic of legal documentation and is and will remain indispensable. Formally, it is documented if a document cites others ((out-bound (cited) sources) and if it is quoted by others (in-bound (citing) sources)). It is important to specify citations, e.g. referring also to the structuring elements of a document, e.g. articles, sections, paragraphs, lists, etc. The main types of citations are: basis of the act, cited acts in the document, citations in the operative part of the judgment, document amending other documents, document is amended by other acts, etc.¹⁶.

The user’s perspective takes into account the opinions of legal professions, business and civil society about the document collection (Lu/Conrad: user view). Modern search engines can (semi)automatically generate user’s perspectives. The search engine can collect information how often a particular document has been opened for viewing, how many times it has been printed, how many times it has been tested for its legal validity, etc. Thus, information about the document relevancy is available that is otherwise difficult to obtain. Sufficient data protection has to be implemented, e.g. anonymization of user data.¹⁷

The logical representation is based on the first order logic, complemented by a representation of temporal layers¹⁸ and the personal scope¹⁹. The decisive factor is the rapid and efficient execution of an at least five-digit number of rules (i.e. more than 10,000 rules). These requirements have been met in practice but are still insufficiently used.²⁰ Logic programming restricts the complexity of the logic view but allows the handling of «standard cases» in high volumes at low cost. Deontic logic tries to improve the theoretical basis for advanced logical representations, e.g. moving to solve also «hard cases». So far, only a logical representation of argumentation structures is considered as possible.

The conceptual representation can draw on the experience of «concept jurisprudence» (Begriffsjurisprudenz) that has intensively dealt with the conceptualization and systematisation and was dominant in the 19th century. The conceptual jurisprudence is a tool of thought for a better understanding of the legal system. The legal ontology, however, brings the automation option and a much more powerful systemisation. Since the 1990s, many legal ontologies have been developed²¹; now a strong standard exists with LRI Core, LKIF²² and LegalRuleML²³. Much research has been reused for a more convenient representation of legal knowledge²⁴, ²⁵. The respective elements of the concepts have to be transposed into a computer-readable structure, e.g. header, definition, relations (upper/lower term, antonym, related term, synonym, homonym, polysem etc.), pre-subsumption (relation between the concept of law and facts element) and comments. Facts, e.g. a world ontology, can be taken from projects like Cyc. This project is to provide automated applications with a formal knowledge base of «common sense» knowledge. Currently, more than three million facts and rules are formally represented in the Cyc knowledge base, provided as OpenCyc.²⁶ A major advantage of an ontology is the easier representation of relations between facts and legal concepts in a form of pre-subsumption.

The theory development of legal ontologies and also its practical implementation are sufficiently strong, however, in practice, few applications exist. Legal ontologies require a deep analysis of the legal domain that can only be done by the legal expert. So far, very few experts have moved into this new form of representation of knowledge. Thus, legal ontologies share the fate of the Semantic Web²⁷. Semantic mark-up is a strong tool for a co-operative analysis of legal text corpora but requires a more efficient collaboration between knowledge experts and knowledge engineers that rarely exists.

Following Colette R. Brunschwig, legal visualization concerns the use of graphics, images and videos for visual representation of the law.²⁸ Visualisation draws attention to the essentials, makes these memorable, increases understanding and shows hidden connections. The potential of visualizations for dialogue with the laity is undisputed. Graphical notations are also a support for the formalization of the law. The key features are represented by images or graphics, even in cases where the necessary level of abstraction for formalization is not yet reached.

The focus of the argumentation view is twofold: representation of legal arguments but also the structure of each document in its elements and logical structure: factual information, evidence, arguments, conclusions, etc.²⁹ An emphasis has been placed on the formalization of argumentation in the 1990s, after the modest success of legal logic. Following the legal theory work on theories on legal argumentation (e.g.³⁰), formalisation of legal arguments stays in the focus since the 1990s. The elements of argumentation are systematized in a logical argumentation structure. Arguments are an essential tool of legal work, because thesis, antithesis and synthesis are relevant in each case due to the dialectical roles of plaintiff, defendant and judge.

For producing this enormous structural analysis, a lot of resources are needed in case of manual work, e.g. for searching, reading, interpreting and understanding of the legal text corpus. The standard methodology is to locate, read, interpret and understand the «legal stuff», taking into account the legal interpretation and reasoning methods in a dynamic world of concepts. Other elements are also considered, e.g. social context, legal authorities, sophisticated methods of interpretation etc. At present, financial constraints restrict deep analysis to areas of legal disputes or theoretical interest. Stronger use of AI & law can be the solution to this knowledge acquisition problem.

The most significant add-on of legal informatics is the revolution in legal search by the use of search engines. Legal information providers follow law-making, rendering of new judgements or decisions, publishing of new literature etc. They take care of all relevant changes of the particular legal system, index these materials in full-text and, if possible, add metadata and provide powerful search engines. The resulting – in the best case – 4 views are very helpful for the lawyer but do not substitute a full legal analysis, e.g. a conceptual and logical analysis using all interpretation methods. Here, the fundamental statement of Sowa applies: the terminology is to be developed and to be brought into a strong logical structure.

AI & law methodology allows a representation according to the different needs of the users: Handbook, Dynamic Electronic Legal Commentary (DynELC), citizen information or case-related synthesis. AI & law methods can produce information elements for a comprehensive picture. Starting from the text corpus in XML format, legal data science methods drawn from many fields within the broad areas of mathematics, statistics, and information technology:  pattern recognition and learning, machine learning, probability models, statistical learning,  visualization, data warehousing, etc. AI & law has developed and refined these methods for legal output. It has to be noted that the required accuracy for practical use has very often not been achieved yet.

Document categorisation: An assignment of one of the more than 200 document types is relevant for the role of the document in the legal systems and determines some of its attributes (e.g. preparatory act, law, implementing act etc.). This categorisation can be further developed, e.g. in doing also privilege classification³¹.

Document segmentation: A very important but often neglected part is the segmentation of a legal document into its structural elements, e.g. author, title, material rules, procedural rules, facts, arguments, rulings etc. The semantic meaning is much stronger if the document part is also taken into account.

Citations: The automatic generation of hypertext links describes the network between documents but also its relevance. Here, the strongest use of AI & law can be noticed in practice (e.g. the Australasian Legal Information Institute AustLII uses hyperlinks as a main method of analysis support besides full text retrieval³².

Temporal relations: The automatic generation of temporal relationships analyses the temporal dimensions: existence, force, efficacy and applicability. It is standard in information retrieval but not sufficiently implemented in the AI & law environment. A concept exists with the work of Scharf³³.

Document description and summary: A lexical ontology, e.g. a thesaurus, remains critical for a condensed short content description. Multilingual versions like Eurovoc or the Swiss legal thesaurus are very helpful for first handling of documents in a foreign language. Much work has been done already in the 1990es (e.g. FLEXICON and KONTERM, see for an analysis³⁴).

Logical and argument analysis: Given the document segmentation, textual rules and arguments can be also formally represented in a pre-defined model, e.g. in very useful practical flow charts (activity diagrams) or argumentation structures. Here, much work has still to be done.

Visualisations: Recently, traditional pictorial representations are supplemented with model-based visualizations. Visualisations are very helpful for representation of «soft» relations between documents, concepts and arguments³⁵.

User assessment: Modern search engines can (semi)automatically generate user’s perspectives.³⁶

Due to space reasons, only a short overview on the state of the methodology can be given. The focus is on pattern recognition, thesaurus generation, machine learning, natural language processing, ranking, legal ontologies, legal logic, legal argumentation and visualization.

The standard representation of documents in XML format has to be improved by use of a standard part-of-speech tagger and a parser. Thus, pattern matching, machine learning and natural language processing³⁷ can be strongly improved for document categorisation, document segmentation, citation analysis³⁸, temporal relations³⁹ and thesaurus generation.

It is a hermeneutic process of knowledge generation and re-use, using a knowledge base and the text corpus. Patterns of document types, document segments, citation structures, temporal relations and concepts are matched with the text corpus resulting in a raw analysis of a document. The knowledge base has to be refined and improved using machine learning techniques. A learning circle exists once such a system is implemented for daily use.

The output is presented in a pre-defined formal model (e.g. types of citations with relevance ranking). Further, this semi-automatic approach delivers also relevant feedback from authors and users for improving the knowledge base⁴⁰. The research on the quality of legislation is also relevant⁴¹. A strong support can be expected from the study of natural language features⁴².

Model-based visualizations are based on grammars, i.e. strict specifications for composing visualizations based on sets of pre-defined elements and relation (e.g. the PICTMOD of Fill⁴³).

Ranking is used as a mechanism for establishing a relationship of relevance for a particular information or analysis. In law, it is used in legal information retrieval, however, with still insufficient success compared to the Google algorithm (cf.⁴⁴). Ranking gives some guidance concerning the relations of a document vs. requirements, document in the corpus, document in the citations network, document in the timeline etc.

The logical analysis and the argumentation analysis require a rule model and an argumentation model. Some support can be achieved if text is transferred into logical rules (e.g. the well-known example of Softlaw/Oracle⁴⁵ or recently⁴⁶). Flow-charts (activity diagrams) are also very helpful in practice. Recent work on argumentation analysis can be found in the ICAIL2015 proceedings⁴⁷. The logic programming is powerful enough to be a decisive support tool for the implementation of laws. Well-structured flowcharts help in decision-making, esp. when thousands of legal rules must be processed. However, two important elements should be added: a model of the layers of time⁴⁸; further, a differentiation according to distinct applicability of norms to legal subjects.

At the moment, a transitional period can be noticed characterised by semi-automation. Computers can be very supportive but in the very end a human being has to check the proper context and quality of the work. A fruitful cooperation between men and machines has to be developed in order to deploy and improve the methodology of legal data science.

So far, all these tools of data analysis are insufficiently developed and used in the legal domain. It is sad that legal authors rarely use such tools in order to generate synthesis works like commentaries. In our view, the AI & law methodology is sufficiently strong that legal publishers should move on and provide such a toolbox for its authors.

Legal data science looks back to more than 50 years of experience in the analysis of legal text corpora (albeit under different names, in particular legal information retrieval or AI & law). Further, the long tradition of legal theory research provides some guidance. Present analysis consists in creating 8 views using AI & law methodology for producing automatically or semi-automatically related input for further intellectual work. The stronger use of computational methods speeds up legal analysis and allows a faster and more efficient assessment of the legal materials. In this transitional period, a fruitful cooperation between men and machines has to be developed in order to deploy and improve the methodology of legal data science. A huge potential remains unused for the more efficient analysis of legal text corpora.

This paper was also presented at the JURIX2015 CEILI Workshop on Legal Data Analysis in December 2015.