Jusletter IT

The law appears in legal texts, on the one hand, and on the other hand, in legal situations, which are mostly typed. The dominance of textuality began in modern times with the reception of Roman law. Previously, customary law was situationally dominant. With the advancement of socially and legally used machines, the importance of textuality will decline and that of legal programming will increase; see e.g. [Boehme-Neßler 2017]. The visualization of law fits into this trend. On the one hand, our paper deals with the communication technology of visualization and, on the other hand, with the peculiarities of law as an object of visualization. At present, there is still no standard and comprehensive model of legal visualization, but there are some very promising approaches. A distinctive feature of the law in connection with its visualization consists in the abstractness of legal texts. The legal texts reside in an abstract frame, which is not linguistically structured. It is the task of legal visualization, in addition to the types of legal situations, to make these pre-textual interdependencies of legal terms visible [Lachmayer/Hoffmann 2005].

Related works. In recent decades, legal visualization has been studied by multiple authors. Moreover, a separate section on legal visualization runs yearly in IRIS. A 20-year IRIS multi-methodical literature analysis provides a large network of authors [Schoorman et al. 2017]. In our paper, however, we narrow the focus to only situational legal visualization. We have written about situations [Čyras/Lachmayer 2013] and Structural Legal Visualization [Čyras et al. 2015]. Commenting on a variety of visual legal communication practices would extend beyond the scope of this paper.

Volker Boehme-Neßler (2011) writes about the «visualisification of the law» and its multiple facets, including the medium of television. He notes the complementarity of text and image and the different characteristics and functions [Boehme-Neßler 2011, 86 ff]. Techno-images such as «structures, relationships or dynamic processes are often understood more readily when presented as maps, diagrams models, building plans or computer simulations». A reason for this is that they «are created by causal mechanisms» (ibid., p. 56–57). However, the latter are not prevalent in law.

A general schema regarding visualization is shown in Figure 1. This schema consists of seven layers. Layer 2 is the primary one and depicts a phenomenon such as law to be cognised and described. This phenomenon is an object of cognition, in other words, a thing. The phenomenon consists of different elements and can be viewed from different perspectives. Similarly, a metaphorical cube consists of 8 vertices, 12 edges and 6 faces. Layer 2 corresponds to Aristotle’s teaching that things are primary to universals (universalia in re).

Layer 1 depicts the realm of ideas. It corresponds to Plato’s teaching that ideas are primary to things (universalia ante rem). Next are Layers 3 to 5, which serve as description layers. Textual and visual descriptions are distinguished. Layer 3 refers directly to the thing on Layer 2, and hence, describes it. The thing can be described with text or pictures, photographed, etc., but not reflected.

Figure 1: A general schema of visualization

Layer 4 serves as a theory. It structures naïve descriptions on Layer 3. This theory can be described textually or graphically. Layer 4 is on a meta-level in respect of Layer 3.

Layer 5 serves as a meta-theory and is on a meta-level in respect of Layer 4.

Layer 6 concerns a model of a person. The left brain is linked with text/language, whereas the right brain is linked with visualization. Harmony is a goal. Layer 7 concerns emotions. This layer is additional to the six layers.

Layer 6 corresponds to the nominalists» teaching of universalia post rem. The nominalists hold that science is produced in the brain. However, Aristotle would disagree. We hold that both teachings are suited as theories of facts. Plato taught that ideas for structuring come before things. We also hold that Plato’s teaching can be called a theory of products.

This section compares verbal writing and pictorial writing in human communication. Verbal writing has its roots in the Latin language. Examples of pictorial writing are Chinese characters and the icons in public airports or in Olympic game arenas. The theme Colorizing Chinese Characters is initiated by Lachmayer and Weng.¹

Assume a human sender speaking to a human receiver (Figure 2). In addition to speech (Figure 2, element 2a), verbal writing (2b) can be added. This verbal writing is parallel to speech and has certain advantages. However, a disadvantage of speech communication is that it cannot be distributed in time. Visualization (2c) can also be added, but is mostly auxiliary.

Figure 2: Verbal writing and pictorial writing in human-to-human communication

Next comes the sender’s semantic link to the thing (Figure 2, element 3) and its semantics (element 4). The semantic link can also be autonomous and have no association with the sender. Pictorial writing is a kind of magic. Examples of this are Chinese characters and emoticons (pictorial representations of a facial expression). In the link to semantics, pictorial writing dominates, and textual descriptions such as picture captions are auxiliary (Figure 2, 3b). Pictorial characters refer to the type of the thing. In the semantic link, pictorial writing provides understanding. Textual captions provide efficiency and clearness; cf. «clare et distincte», «clear and distinct», which is René Descartes’s criterion of truth.

Communication is governed by rules (Figure 2, element 5). Both speaking (Figure 2, element 2a) and verbal writing (2b) are strict because of strict speech rules and writing rules. Children’s languages and a Babylonian confusion of languages are not desirable. Pictorial writing rules, however, are elastic. In the arts, for example, multiple interpretations are allowed.

There is no general model of visualising the legal domain. However, certain aspects of law can be explained better with visualizations. Further, several examples are provided (Figure 3). They stem from Friedrich Lachmayer’s image of visualising insights, ideas and texts; see various examples on the web (http://jusletter-it.weblaw.ch/visualisierung.html, http://www.legalvisualization.com). These visualizations are logical pictures (logische Bilder) in the sense of Röhl/Ulbrich (2007).

Figure 3: Examples of visualizations: a) legal stage, b) hierarchy of legal sources, c) trial as a ping-pong process with a court, d) legal institution such as a sales contract, e) thesaurus/ontology, f) legal framing, g) granularity entities, h) analogy of methods in Begriffsjurisprudenz and legal ontologies

We treat each visualization in Figure 3 as a graphical representation of a concept in law (bildliche Darstellung von eine Figur in Recht). Thus, a graphical concept in law would supplement a verbal concept in law. In linguistics and semiotics, supplementing a figure of speech such a metaphor, e.g. the metaphor of a bridge, with a visualization would be similar.

The term «situational visualization» is also used in computing. For example, Krum et al. [2001] introduce a style of visualization called Situational Visualization in augmented and virtual reality systems. They note that «[m]any tasks require a detailed knowledge of the local environment as well as an awareness of rapidly changing and interacting events. This awareness is termed situational awareness or situation awareness.» [Krum et al. 2001, 143] They quote Endsley [1988, 792] for a formal definition of situation awareness: «Situation Awareness is the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future.» Krum et al. [2001] note that «this type of spatio-temporal knowledge awareness is important in many tasks, including… law enforcement».

Legal machine or an expert system? Complete understanding of a situation is a prerequisite for the design of a legal machine which governs the situation. A legal machine can be defined as a machine whose actions have legal importance and legal consequences. Legal machines are legal actors capable of triggering institutional facts. Legal machines distribute permissions and obligations. An example is a traffic light, which distributes permission to cross the road. The simpler the situation, the simpler is the legal machine, for instance, an automatic barrier to a parking lot.

In a study of situational visualization, the concept of the situation is of primary importance. Our thesis is that the situation dominates in a legal machine, whereas the case dominates in an expert system. The success of a legal machine is determined by the situation to be governed. In the implementation, open texture and nondeterminism are minimised. Examples of successful legal machines are traffic lights, road radars and car parking barriers. A legal machine in a complicated situation risks taking a wrong action, for example, a decision-making machine in a highly bureaucratic domain.

Further, we follow our earlier study «situation versus case» [Čyras/Lachmayer 2013]. Situations and cases can be attributed with different methods of legal informatics. A situation stands for a type of behaviour, and a case stands for an instance. Situations are governed primarily by the principle: «Roles, not rules.» An example of a situation is a crossroad description (Figure 4). The roles comprise pedestrians and different types of drivers (car, bus, ambulance, etc.). The description of the situation constitutes part of the situation’s teleology.

Figure 4: Situation vs. case in situative jurisprudence

Situations and cases are characterised differently:

1. Type. A situation constitutes a generic behaviour pattern, whereas a case – a concrete one.
2. Ex-ante/ex-post. A situation is related with ex-ante analysis, whereas a case – with ex-post.
3. Time. A situation concerns the future, whereas a case – the past.
4. Alternatives. In a situation, alternatives are possible, and this is essential. There are no alternatives in a case. A concrete past behaviour is concerned. However, alternatives can appear in hypothetical evaluations, such as «Should the actors perform another manoeuvre, the accident would not happen.»
5. Language. Situations have no language at all.
   1. A situation is mentally – visually, acoustically, sensibly – interpreted. Suppose a driver is in a crossroad. A mental language is non-textual and non-professional. Sensual (visual, aural, etc.) comprehension dominates, and textual descriptions appear on the periphery. Hence, a situational language is non-professional. A communication language does not need to be textual; cf. gestures. Therefore, a situational language is loosened and differs from case languages.
   2. Roles are inherent in situations, e.g. «pedestrian» or «driver». The actors» legal status may be implicit because rights and obligations are comprised by their roles.
   3. Artificial agents can use formal languages. As an example, suppose multi-agent systems. The agents» beliefs, desires and intentions are represented in computers 
   Case. Witnesses use a non-professional language, and jurists – a professional one.
   1. Cases are explicitly formulated in documents. Quod non est in actis, non est in mundo – «What is not in the documents does not exist». Cases are textually available. The major facts are described in an investigation report. However, statements about facts can be defeated during the argumentation in the litigation. Visual descriptions such as schemes are supplementary and appear on the periphery.
   2. There are two kinds of languages: first, the non-professional language of witnesses and, second, the professional juristic language. Legal subsumption serves as a bridge.
6. Placing onto the Is and Ought stages
   Situation
   1. Situations are assigned to Is. A situation is always real and factual. As an example, suppose a crossroad with the red light on. You would like to cross, but do not want to show your children a bad example, as they learn the customary law from your behaviour.
   2. In contrast, the type of a situation is assigned to Ought. A situation type allows visual representations such as a schema. This appears in technical devices.
   Case
   1. Cases are also assigned to Is. Every case has passed. The reference range is not important. A case is fixed in the text.
   2. A case is on the Is stage but can be viewed from two perspectives. First, the case is assigned to the subjective law. Secondly, the case is assigned to a legal proceeding, and hence, to the objective law. Here, argumentation arises. The players are assigned roles in the legal proceeding such as plaintiff, defendant, witness, expert, etc.
7. Web applications. e-Government application examples in Austria:
   For situations, see www.help.gv.at. For cases, see www.ris.bka.gv.at.
8. Legal instruments. Distinct legal instruments are concerned:
   Situation: (i) the roles of actors; (ii) assumptions (hypothetical facts); (iii) rules which govern the situation; (iv) additional regulations which govern the situation.
   Case: (i) claim; (ii) evidence; (iii) attacks; (iv) litigation can consist of several cases (e.g. criminal and civil).
9. Formalisms. Distinct legal instruments are concerned:
   Situation: deontic logic, abstract normative systems, etc.
   Case: case modelling approaches, factors, etc.
10. Customary law, machine law and statutory law
    Situation: customary law and machine law are in the foreground. As an example, suppose a zebra crossing. Pedestrians aim to cross it. Statutory law (the road rules) regulates this situation. However, ordinary people are governed primarily by the customary law which is superim­posed. And finally, the situation is governed by traffic lights – machine law steps in.
    Cases: the traditional hierarchies of legal sources prevail.

Different Conceptualisations for Situations and Cases. Situations and cases are described by different concepts. Figure 5 shows the core elements which make the difference. A situation is viewed as a state in the world and is not finished. Different scenarios can evolve from a sole state. In terms of logic, the different worlds that can be accessed from the given situation are central. In terms of normative systems, the legal status of the players (normative positions) is key.

Figure 5: Situation and case: attention and subsumption

Cases are finished, and the outcome is known, e.g. «Mr. Lammers is dead, shot with a gun». Thus, the scenario is finished. However, the players may have different stories, and this is central. For example, the plaintiff and the defendant may render contrary arguments, e.g. «Rijkbloem shot him» and «Mrs. Lammers shot him».

Which elements are important in the constellations of situations and cases?

• Situation. Here, attention is the most important element. Attention can be compared to a cursor that can move to different positions. The players reside similarly as on a stage and create scenes like people around a table. A script assigns the players their roles.
• Case. Here, legal subsumption, i.e. bringing the text under norms, is in the forefront. It is important that the case elements are relevant to the norms. The elements of the issue have to be named in a professional legal language. Cases are marked by verbalisations. Here, the relationships – references – of the text to the relevant norms are addressed.

Notation for Situation. A situation appears on the horizontal Is stage and is described by the following entities (Figure 6):

1. Situational elements. These elements are the constituents of the situation. They are denoted by small letters, e.g. a, b, c, driver, pedestrian, etc. They exist in time and space.
2. Relations. These are the relations between the situational elements. There are many kinds of relations: causal (^c→), teleological (^te→), instrumental (^instr→), contextual (^contx→), etc. These relations are comprised by both legal relations such as debt, but also by empirical non-legal relations. The relations represent different perspectives.

The notation for a causality relation is a ^c→ b. A predicate language can serve to represent situations, cf. the block worlds in early artificial intelligence. The situational elements are represented by constants and relations – by predicates R(a₁, a₂,…, a_n).

Human beings comprehend a situation with their senses, and the brain makes a decision about which action to take. In a situation, a human being reasons primarily in terms of roles, not legal rules. Predicate logic is not in the forefront in human decision making unless a computer decision support system is employed. Computers in multi-agent systems use computer knowledge representation formats. Computers can be more effective than humans in specific tasks, for instance, emergency action based on instrumental sensors. In a situation like the crossroad in Figure 4, both drivers and unmanned vehicles could appear. Each actor would use a different representation of the situation and different decision making.

Figure 6: Situation vs. case in situative jurisprudence

We presented a schema for the visualization of phenomena such as law and the differences between verbal writing and pictorial writing. Thus, the preliminaries of a theory on visualization are introduced. Further, the differences in reasoning in legal situations and in legal cases are tackled. We hold that situative visualization contributes to the development of the legal machines which govern a certain situation.

Three to four decades after the term «expert systems» was coined, we comment on a reason why the expert systems in the legal domain have failed to meet expectations. Complete knowledge of the regulation in a non-trivial domain is unlikely to be achieved and represented in computers. However, a specific situation can be understood and equipped with a legal machine.

Boehme-Neßler, Volker, Pictorial Law: Modern Law and the Power of Pictures, Springer, Berlin Heidelberg, 2011.

Boehme-Neßler, Volker, Die Macht der Algorithmen und die Ohnmacht des Rechts, Neue Juristische Wochenschrift, volume 42, 2017, pp. 3031–3037.

Čyras, Vytautas / Lachmayer, Friedrich, Situation versus Case and Two Kinds of Legal Subsumption. In: Schweighofer, Erich / Kummer, Franz / Hötzendorfer, Walter (Eds.), Abstraction and Application, Proceedings of the 16th International Legal Informatics Symposium IRIS 2013, books@ocg.at, Vienna 2013, pp. 347–354.

Čyras, Vytautas / Lachmayer, Friedrich / Lapin, Kristina, Structural Legal Visualization, Informatica, volume 26, issue 2, 2015, pp. 199–219. http://www.mii.lt/informatica/pdf/INFO1059.pdf.

Endsley, Mica R., Situation Awareness Global Assessment Technique (SAGAT). In: Proceedings of the IEEE 1988 National Aerospace and Electronics Conference, volume 3, pp. 789–795. http://www.ieeeexplore.ws/stamp/stamp.jsp?tp=&arnumber=195097.

Krum, David M. / Ribarsky, William / Shaw, Christopfer D. / Hodges, Larry F. / Faust, Nickolas, Situational awareness. In: Proceedings of the International Conference on Virtual Reality, 2001, pp. 143–150.

Lachmayer, Friedrich / Hoffmann, Harald, From legal categories towards legal ontologies. In: Lehmann, Jos / Biasiotti, Maria Angela / Francesconi, Enrico / Sagri, Maria Teresa (Eds.), LOAIT – Legal Ontologies and Artificial Intelligence Techniques, volume 4 of the IAAIL Workshop Series, Wolf Legal Publishers, Nijmegen 2005, pp. 63–69.

Röhl, Klaus F. / Ulbrich, Stefan, Recht anschaulich. Visualisierung in der Juristenausbildung, Halem, Köln 2007.

Schoormann, Thorsten / Hofer, Julien / Behrens, Dennis / Knackstedt, Ralf, Rechtsvisualisierung in 20 Jahren IRIS – eine multimethodische Literaturanalyse. In: Schweighofer, Erich / Kummer, Franz / Hötzendorfer, Walter / Sorge, Christoph (Eds.), Trends and Communities of Legal Informatics. Proceedings of the 20th International Legal Informatics Symposium IRIS 2017, books@ocg.at, Vienna 2017, pp. 369–376.