Jusletter IT

Picture-Text Cooperation: Lettering in Legal Visualization

  • Authors: Vytautas Čyras / Friedrich Lachmayer / Kristina Moroz Lapin
  • Category: Articles
  • Region: Lithuania, Austria
  • Field of law: Legal Visualisation, Multisensory Law
  • Collection: Conference Proceedings IRIS 2015
  • Citation: Vytautas Čyras / Friedrich Lachmayer / Kristina Moroz Lapin, Picture-Text Cooperation: Lettering in Legal Visualization, in: Jusletter IT 26 February 2015
In this paper, we investigate picture-text cooperation in lettering that explains the figures of selected scientific articles in the legal domain. As samples, we survey the structured diagrams, including the logical pictures, from JURIX papers. We see the need for picture-text cooperation in the joint work of engineers and jurists. A motive is that engineers think structurally, whereas jurists think textually. However, we see no uniform framework for picture-text cooperation in legal visualizations. Therefore, this is a subject for discussion. We hold that a unified visual notation, strict compositional rules and strict visual semantics in law are not easy to achieve. Here, semi-formal representations are stronger, because representing legal meaning is a problem. However, more formal visual notations could contribute toward the creation of logical pictures.

Table of contents

  • 1. Introduction
  • 2. Repertoire in Lettering
  • 3. Legal Visualization
  • 4. Legal Meanings – a Specific Feature of Legal Visualization
  • 5. Relations – the Core of Logical Pictures
  • 6. Conclusions
  • 7. Acknowledgement
  • 8. References

1.

Introduction ^

[1]
This paper studies picture-text cooperation in lettering, which is found in the figures of scientific papers in the legal domain. We are concerned primarily with structured diagrams that include logical pictures (logische Bilder [Röhl & Ulbrich 2007, pp. 139–166]), such as architectural diagrams, flow diagrams, etc. We surveyed pictures from JURIX1 articles, because the annual JURIX conferences are among the most important for legal informatics. The selected articles are concerned with various subjects of legal informatics, such as e-Government applications, argumentation models, judicial reasoning, etc.
[2]
Motivation. Jurists think textually, whereas engineers think structurally. More precisely, jurists think normatively. In contrast, engineers can barely understand the meaning of «ought» in its entirety. However, the interprofessional cooperation of jurists and engineers is needed in various tasks. In this article, we tackle the production of lettering in visualizations in the legal domain.
[3]
Cooperation between different professions is a subject of IRIS 2015. More specifically, cooperation is observed in information systems development, formulating legal requirements, designing legal machines, etc. Cooperation appears in various software applications, such as tax management systems, electronic form proceedings, and e-Justice projects. Legal machines produce legal facts (institutional facts) and not purely brute facts. An argument for visual representations is that they are believed to convey information more effectively to non-technical people than text (Fig. 1).

Figure 1: Communication from a sender to a non-technical recipient. «Clare et distincte», «clear and distinct», refers to René Descartes’s criterion of truth. Visualizations make the text easier to understand

[4]
Methodology. The examination in this paper is conducted on the reflexive level of legal informatics and also continues our earlier investigation [Čyras & Lachmayer 2015]. The empirical question is how these visualizations are constructed and what types can be found therein. Such an analysis may also affect the future design of visualizations in legal informatics, especially as corresponding design principles are not yet in the canon.
[5]
Picture-text cooperation has links to information visualization [Card 2008; Fill 2009]. The text explains what is visualized. This is relevant for the usability of legal machines.

2.

Repertoire in Lettering ^

[6]
A visual notation consists of a set of graphical symbols (visual vocabulary), a set of compositional rules (visual grammar), and definitions of the meaning of each symbol (visual semantics) [Moody 2009, p. 756]. Graphical symbols usually include textual labels. The structure of labels and their meaning (semantics) depends on the problem domain.
[7]

We carry out an empirical study of the usage of textual labels. We further survey selected pictures from JURIX 2012 and JURIX 20132 papers. Previously, in [Čyras & Lachmayer 2015], the structural elements of the diagrams were discussed, such as colors, dimensions, focus, mindmapping, mixed shapes, quantity, relationships, tables, traditional diagrams with predefined semantics, such as argument graphs, and vertical and horizontal axes. Currently, we systematically discuss different text usage in the pictures. Basic graphical symbols, such as rectangles, ellipses, arrows, etc. allow lettering in the form of unformatted or formatted text (bold, italic, subscripts, various fonts, etc.). This applies to sub-elements and compositions, aggregations and layers. The repertoire of texts comprises different types: numbers, symbols, words, and sentences. Texts can be structured, e.g., 1:n, and can have various functions and semantics. Very different artifacts are represented in diagrams: argument graphs, inference graphs, judicial reasoning chains, work or knowledge flow diagrams, ontologies, system architectures, etc. We hold that it is important for a figure caption to name the artifact type clearly. The types of readers should be taken into account: jurists or engineers. Below, we list criteria for text analysis that may be very different:

  • Formal signs. The names of laws and article numbers can be included in diagrams, see [Winkels & Hoekstra 2012, p. 160]. Thus, citations and references to laws can be included.
  • Words. They usually label instances of a graphical symbol type. This is a usage in (semi)formal diagrams, such as the Aristotelian square of opposition and the deontic square [Philipps 2012, p. 70]. In this way, concepts and relations are labeled in ontologies [Poudyal & Quaresma 2012, p. 116]. Word combinations can be used in system architecture diagrams; see the architectural parts of the Argument Analysis System [Kubosawa et al. 2012, p. 62]. Different user roles are labeled textually in a use case diagram, which shows the Legilocal platform [Amardeilh et al. 2013, p. 12]. We find that normativity in system architecture diagrams is expressed as follows: a proposed architecture represents a norm To-Be (Norms To-Be and norms To-Do are distinguished [Sartor 2007, p. 446].) A diagram can show that a concept appears in documents and software applications of different levels [Winkels & Hoekstra 2012, p. 158]. A graph, which is syntactically simple (6 tasks, 7 knowledge assets and flow relations between them), but is not easy to understand semantically, is included in [Conetta & Schafer 2013, pp. 70–71]. Here, a directed graph represents the CommonKADS function structure for Transaction Configuration (TC). Knowledge flow between the decomposed tasks of the TC function is also explained in pseudo-code and text (the LKIF language, Legal Knowledge Interchange Format).
  • Abbreviations. They shorten non-uniform wording and improve the readability of diagrams. For example, FA – factual assertion, LA – legal assertion, FLR – FA to LA rules, LA1 – «Hayashi is liable…», LA2 – «Hayashi is not liable…» in [Araszkiewicz & Šavelka 2012, p. 8]. Here, normativity is intrinsic in the whole paper, and their Figure 1 provides a condensed visualization, showing a constraint network in the Popov v. Hyashi case. Rectangles represent different types of elements, such as FAs, LAs, and rules. A chain of (bold, dashed) lines, each representing a constraint relation, models judicial reasoning in a coherence as constraint satisfaction (CaCS) framework. Other examples are defeat graphs [Prakken 2012, p. 128], where GCi and BCi abbreviate the i’th application of the argument scheme from good to bad consequences, respectively, in defeat graphs. A balance of abbreviations and their explanations should be achieved, because the meaning of abbreviations is not always understood from the first glance at a figure and its caption, and therefore, a reading of the whole article is required [Szőke et al. 2012, pp. 150–154].
  • Symbols (textual). Greek letters, α, β,…, or capital letters, A, B (e.g., system A and system B), serve to label in different alphabets, see e.g., the argument analysis in [Kubosawa et al. 2012, p. 66]. In the modern language of science, algebraic symbols and letters with subscripts/superscripts have denoted abstract concepts since the time of Isaac Newton. Examples include inference graphs and argument graphs [Liang & Wei 2012, pp. 74, 79], strength diagrams for interacting parties [Pace & Schapachnik 2012, p. 111], and interacting legal specifications [Li et al. 2013, p. 111]. In argument component trees, where the issues, facts, and factors, and the relations between them, are made explicit, certain characters (∨ «or», ∧ «and») provide semantics to the tree nodes [Al-Abdulkarim et al. 2013, pp. 6, 8].
  • Formulations. These are clauses of various lengths. They explain graphical symbols and contribute to an understanding of semantics. Examples include argument graphs and (mind)maps, such as classification maps (failure maps, damage maps, legal risk maps) and argumentation maps (legal analysis maps, legal design maps, insurance maps) [Contissa et al. 2013, p. 75]. Such maps present legal concepts and norms to lawyers and non-lawyers (engineers, software developers) within the cooperative design and assessment of new technologies for air traffic management. Formulations are also used in callouts [Kubosawa et al. 2012, p. 69].
  • Sentences. They represent whole statements, which are longer than formulations. Examples include arguments in argument diagrams [Lynch et al. 2012, p. 84], [Prakken 2012, p. 127]. Statements with two outcomes, «yes» or «no», are used in decision flow diagrams [Kahlig 2008, p. 189]. Here, the statements are labeled with article numbers from the Austrian Act on Tenancy Law (Mietrechtsgesetz, MRG).
  • Formatting. Different fonts, boldface, and white text in a black foreground serve to attract the reader’s attention properly and to focus it hierarchically [Buchanan et al. 2012, pp. 36, 38], where the architecture and the context are shown semi-formally. Formatting and the vertical/horizontal direction of the text conveniently structure the depicted artifacts, such as document models [Francesconi 2012, p. 47], where different shapes of different brightnesses are used.
  • Structuring. Labels can be structured. There are examples of parameter = value lists in [Robaldo et al. 2012, pp. 137–139]. Such structural representations represent text as a graphical element.
  • Numbering. This provides a sequence to interpret a diagram. Due to numbering, graphical workflow diagrams can be structured informally in a two-dimensional space [Conetta & Schafer 2013, p. 70; Governatori 2013, p. 78]. Thus, non-experts in graphical notations obtain freedom of expression.
[8]
The repertoire above shows that there is no unified framework in legal visualization. Therefore, picture-text cooperation is open for discussion.
[9]
Avoid textual differentiation of graphical symbol types. Textual differentiation of symbol types is a cognitively inefficient way of dealing with excessive graphic complexity and should be avoided. Such usage violates the principles of semiotic clarity and graphic economy. However, UML3 [Booch et al. 2005], for example, frequently uses text and typographic characteristics (bold, italics, underlining) to distinguish between element and relationship types (Fig. 2).

Figure 2: Textual differentiation: UML uses labels and typographical characteristics to distinguish between symbols (adapted from [Moody 2009, p. 764])

[10]
Labels should be used at the sentence (diagram) level (Fig. 3) in distinguishing between symbol instances, and not at the language level [Moody 2009, pp. 763–764]. However, this requirement is not easy to achieve in the case of a large number of types.

Figure 3: Language level and sentence level of visual notations (adapted from [Moody 2009, p. 757], where the focus is on the top left hand quadrant (visual syntax) and excludes semantic issues and sentence-level issues (bottom)

3.

Legal Visualization ^

[11]
In (semi)formal representations of legal norms, there are, on the one hand, formal notations, which go beyond the textual ones; on the other hand, there are visual representations that also occur in competition with the text. Two different types of visualizations can be distinguished: first, the visualizations formed according to strict formal rules; second, the more intuitive pictures which can depict situations better (Fig. 1). We hold that a unified visual notation, strict compositional rules and strict visual semantics in the legal domain can rarely be achieved. Here, semi-formal representations are stronger. Jurists do not need to be experts in formal visual notations, such as UML, which is used in the field of software engineering.
[12]
A detailed study of visualization in the legal domain is provided, and the motivations behind using it are explained in Röhl and Ulbrich (2007). The lack of pictures in jurisprudence is a learning obstacle (ibid., pp. 15–17). The starting position is «Law is text», and therefore, law is always textual for jurists. Hence, there are reasons for jurists’ reluctance regarding visualizations. The use of pictures is accompanied by the risk of drawbacks, such as redundancy, a low level of abstraction, trivialization, and emotions (ibid., pp. 18–25, 100–102). However, the use of logical pictures can have advantages. Metaphors and symbols can be employed to represent norms, and hence, pictorial two-dimensional representations emerge (ibid., pp. 42–62). To summarize, the combination of the words «law and visualizations» contains a kind of a paradoxical contradiction.

4.

Legal Meanings – a Specific Feature of Legal Visualization ^

[13]
In visualizations in the legal domain, we emphasize the following two specific features. First, legal visualization is characterized by specific raw data. It cannot be limited to a specific norm or law, and it covers legal sources, legal doctrine, legal science, and other elements.
[14]
Second, the object of visualization is a legal meaning. This differentiates legal visualization from data visualization and information visualization [Card 2008], where computer-supported interactive visual representations are important. However, this is not the case in legal visualization. Information visualization can be defined as «the use of computer-supported, interactive, visual representations of abstract data in order to amplify cognition». Hence, amplifying cognition is the purpose of information visualization [Card 2008, p. 515], and it is equally the purpose of legal visualization.
[15]
In picture-text cooperation, communicating the meaning of law to the human user is of primary importance. Here, diagrams serve well as legal norm visualizations [Rechtsnormbilder, Röhl & Ulbrich 2007, pp. 109–111].
[16]
Three functions of instructive pictures can be distinguished [Röhl & Ulbrich 2007, p. 91]:
  1. Pointing function (e.g., an anatomy atlas)
  2. Situational function («A picture is worth a thousand words»)
  3. Construction (structure, design) function (the picture helps the viewer to build a mental model in her mind)
[17]
Our experience shows the importance of the latter two functions for legal meaning visualization.
[18]
The semantics conveyed by a visualization, i.e., the meaning of the representation, is addressed in [Fill 2009, p. 163], in the chapter which is devoted to the analysis of visualizations:
[19]
«Visualisation semantics are therefore related to questions such as What may a user associate with the resulting graphical representation? or Is the intended meaning of the visualization correctly transferred to the user or would another type of representation better fit?»
[20]
Legal visualizations contribute to knowledge explication. Here, we refer to Fill (2009, p. 172), who holds that «the goal of knowledge explication … is to explicate knowledge that resides in the heads and minds of people and express it by a visualization» and lists four basic aims of visualizations: knowledge explication, knowledge transfer, knowledge creation, and knowledge application. A subsequent aim, the knowledge transfer, can be achieved by the following tasks: Diverge, Converge, Organize, Elaborate, Abstract, Evaluate, and Build Consensus [Fill 2009, pp. 173–174].

5.

Relations – the Core of Logical Pictures ^

[21]
Logical pictures in legal visualizations represent relations that have a legal meaning and are not easily understood. There is no unified formal language for legal logical pictures. Therefore, we hold that legal visualization can gain from visual notations in software engineering [Moody 2009], where relations have been a concern for decades.
[22]

We hold that a relation can be managed analogously to atomic entities. An entity–relationship model (ER model) refers to the techniques proposed in the seminal work of Peter P. Chen (1976).4 However, variants of the idea existed previously. Chen holds that «[t]he entity-relationship model adopts the more natural view that the real world consists of entities and relationships. It incorporates some of the important semantic information about the real world».

[23]
Chen’s notation for entity–relationship modeling uses rectangles to represent entity sets, and diamonds to represent relationships appropriate for first-class objects: They can have attributes and relationships of their own. If an entity set participates in a relationship set, they are connected with a line. There are related diagramming convention techniques, such as UML class diagrams. When speaking about the problem of visual dialects, ER modeling exists in a variety of dialects, such as the Chen notation, which is the most commonly used in the academic context, the Information Engineering (IE) notation, which is the most commonly used in practice, the Bachman notation, the IDEF1X notation, etc. [Moody 2009, p. 758].
[24]
UML is becoming a de facto standard for modeling. There is a question about the extent to which the UML graphical notation should become a standard outside of the field of software engineering and extend to law.
[25]
Graphical symbols are used to symbolize (or perceptually represent) semantic constructs, typically defined by a meta-model: The meanings of symbols are defined by mapping them to the constructs they represent [Moody 2009, p. 757]. Various graphical symbols are well-suited to represent different types. Next, to represent instances of a type, a textual alphabet for labels is an effective method, e.g., A, B, C, property1, property2, relation1, etc. The latter instances can be further interpreted as types. In this way, the initial diagram can serve as a meta-model in the three levels thus obtained: meta-meta-model – meta-model – model.

6.

Conclusions ^

[26]
Presently, we see no uniform framework for picture-text cooperation in legal visualizations. Therefore, this is a subject for open discussion. However, we see the need for picture-text cooperation in the joint works of engineers and jurists. Visual notations could contribute to the improvement of logical pictures and human communication.

7.

Acknowledgement ^

[27]
V. Čyras and K. Lapin have been supported by the project «Theoretical and engineering aspects of e-service technology development and application in high-performance computing platforms» (No. VP1-3.1-ŠMM-08-K-01-010), which is funded by the European Social Fund.

8.

References ^

Al-Abdukarim, Latifa; Atkinson, Katie; Bench-Capon, Trevor, From oral hearing to opinion in the U.S. Supreme Court. In: Ashley, Kevin D. (Ed.), Legal Knowledge and Information Systems, JURIX 2013: The Twenty-Sixth Annual Conference, IOS Press, Amsterdam, 1–10 (2013).

Amardeilh, F.; Bourcier, D.; Cherfi, H.; Dubail, C.-H.; Garnier, A.; Guillemin-Lanne, S.; Mimouni, N.; Nazarenko, A.; Paul, È.; Salotti, S.; Seizou, M.; Szulman, S.; Zagayouna, H., The Légilocal project: the local law simply shared. In: JURIX 2013, 11–14 (2013).

Araszkiewicz, Michał; Šavelka, Jaromir, Refined coherence as constraint satisfaction framework for representing judicial reasoning. In: Schäfer, Burkhard (Ed.) Legal Knowledge and Information Systems, JURIX 2012: The Twenty-Fifth Annual Conference, IOS Press, Amsterdam, 1–10 (2012).

Booch, Grady; Rumbaugh, James; Jacobson, Ivar, The Unified Modeling Language User Guide, 2nd edition, Addison-Wesley Professional (2005).

Buchanan, William; Fan, Lu; Lawson, Alistair; Schafer, Burkhard; Scott, Russell; Thuemmler, Christoph; Uthmani, Omair, Computational data protection law: trusting each other offline and online. In: JURIX 2012, 31–40 (2012).

Card, Stuart, Information visualization. In: Sears, Andrew; Jacko, Julie A. (Eds.), The Human-Computer Interaction Handbook, 2nd edition, Lawrence Erlbaum Associates, Taylor & Francis Group, New York, 509–543 (2008).

Chen, Peter P., The Entity-Relationship Model – toward a unified view of data. In: ACM Transactions on Database Systems 1(1):9–36 (1976).

Conetta, Orlando; Schafer, Burkhard, LKIF in commercial legal practice: transaction configuration from eurobonds to copyright. In: JURIX 2013, 63–72 (2013).

Contissa, Giuseppe; Laukyte, Migle; Sartor, Giovanni; Schebesta, Hanna, Assessing liability with argumentation maps: an application in aviation law. In: JURIX 2013, 73–76 (2013).

Čyras, Vytautas; Lachmayer, Friedrich, Towards multidimensional rule visualizations. In: Araszkiewicz, Michał et al. (Eds.), Problems of Normativity, Rules and Rule Following, Springer, 445–455 (2015).

Fill, Hans-Georg, Visualisation for Semantic Information Systems. Gabler, Wiesbaden (2009).

Francesconi, Enrico, Supporting transnational judicial procedures between European member states: the e-Codex project. In: JURIX 2012, 41–50 (2012).

Governatori, Guido; Lam, Ho-Pun; Rotolo, Antonino; Villata, Serena; Gandon, Fabien, Heuristics for licenses composition. In: JURIX 2013, 77–86 (2013).

Kahlig, Wolfgang, Modeling Law in e-Government: Case Examples for Legistic (Rechtsmodellierung im e-Government: Fallbeispiele zur Legistik), VDM Verlag Dr. Müller, Saarbrücken (in German) (2008).

Kubosawa, Shumpei; Lu, Youwei; Okada, Shogo; Nitta, Katsumi, Argument analysis with factor annotation tool. In: JURIX 2012, 61–70 (2012).

Li, Tingting; Balke, Tina; De Vos, Marina; Padget, Julian; Satoh, Ken, Legal conflict detection in legal systems. In: JURIX 2013, 107–116 (2013).

Liang, QingYin; Wei, Bin, An argumentation model of evidential reasoning with variable degrees of justification. In: JURIX 2012, 71–80 (2012).

Lynch, Collin; Ashley, Kevin; Falakmasir, Mohammad H., Comparing argument diagrams. In: JURIX 2012, 81–90 (2012).

Moody, Daniel L., The «physics» of notations: towards a scientific basis for constructing visual notations in software engineering. In: IEEE Transactions on Software Engineering, 35(5):756–778 (2009).

Pace, Gordon J.; Schapachnik, Fernando, Types in two-party systems: a formal analysis. In: JURIX 2012, 105–114 (2012).

Philipps, Lothar, On deontic squares – cubes – hypercubes (Von deontischen Quadraten – Kuben – Hyperkuben). In: Philipps, Lothar, Finite Legal Concepts with Infinite Limits: Essays on Legal Logic (Endliche Rechtsbegriffe mit unendlichen Grenzen: Rechtlogische Aufsätze), Anthologia, Editions Weblaw, Bern, 69–81 (in German) (2012).

Poudyal, Prakash; Quaresma, Paulo, An hybrid approach for legal information extraction. In: JURIX 2012, 115–118 (2012).

Prakken, Henry, Formalising a legal opinion on a legislative proposal in the ASPIC+ framework. In: JURIX 2012, 119–128 (2012).

Robaldo, Livio; Lesmo, Leonardo; Radicioni, Daniele P., Compiling regular expressions to extract legal modifications. In: JURIX 2012, 133–141 (2012).

Röhl, Klaus F.; Ulbrich, Stefan, Law Graphically. Visualization in the Education of Jurists (Recht anschaulich. Visualisierung in der Juristenausbildung), edition medienpraxis 3, Halem, Köln (in German) (2007).

Sartor, Giovanni, Legal Reasoning: A Cognitive Approach to the Law. A Treatise of Legal Philosophy and General Jurisprudence, vol. 5, Springer, Heidelberg (2007).

Szőke, Ákos; Förhécz, András; Strausz, György, A unified change management of regulations and their formal representations based on the FRBR framework and the direct method. In: JURIX 2012, 147–156 (2012).

Winkels, Radboud; Hoekstra, Rinke, Automatic extraction of legal concepts and definitions. In: JURIX 2012, 157–166 (2012).

 

Vytautas Čyras, Associate Professor, Vilnius University, Faculty of Mathematics and Informatics, Naugarduko 24, 03225 Vilnius, Lithuania, Vytautas.Cyras@mif.vu.lt; http://www.mif.vu.lt/~cyras/

 

Friedrich Lachmayer, Professor, University of Innsbruck, Innrain 47, 6020 Innsbruck, Austria, Friedrich.Lachmayer@uibk.ac.at; http://www.legalvisualization.com

 

Kristina Lapin, Associate Professor, Vilnius University, Faculty of Mathematics and Informatics, Naugarduko 24, 03225 Vilnius, Lithuania, Kristina.Lapin@mif.vu.lt; http://www.mif.vu.lt/~moroz/

  1. 1 http://jurix.nl/conferences/, accessed 7 January 2015.
  2. 2 JURIX 2012, the 25th International Conference on Legal Knowledge and Information Systems, http://justinian.leibnizcenter.org/jurix/, University of Amsterdam, 17–19 December 2012. JURIX 2013, http://sinatra.cirsfid.unibo.it/jurix2013/, University of Bologna, 11–13 December 2013 (accessed 7 January 2015).
  3. 3 The Unified Modeling Language (UML) is a general purpose modeling language in the field of software engineering, which is designed to provide a standard way to visualize the design of a system [Booch et al. 2005]. See also the webpage of Object Management Group, http://www.omg.org/spec/UML/, accessed 7 January 2015.
  4. 4 In software engineering, an entity–relationship model (ER model) is a data model for describing a database in an abstract way. Diagrams created to design the entities and relationships are called entity–relationship diagrams or ER diagrams (http://en.wikipedia.org/wiki/Entity%E2%80%93relationship_model, accessed 7 January 2015).