Jusletter IT

Performing and Evaluating Inferences in JUDIPRO – A Legal Support System in Divorce Cases

  • Authors: Michał Araszkiewicz / Agata Łopatkiewicz / Tomasz Zurek
  • Category: Articles
  • Region: Poland
  • Field of law: Advanced Legal Informatics Systems and Applications
  • Collection: Conference Proceedings IRIS 2016
  • Citation: Michał Araszkiewicz / Agata Łopatkiewicz / Tomasz Zurek, Performing and Evaluating Inferences in JUDIPRO – A Legal Support System in Divorce Cases, in: Jusletter IT 25 February 2016
JUDIPRO is a legal expert system providing support in divorce cases. The novelty of the system lies not only in its capability of making inferences, but also in the ability to assess the quality of legal argumentation in context. In this paper we focus on arguments based on factors which may promote or demote certain conclusions in analysed cases. The parameter of strength of the promoting and demoting relations is introduced. In addition, we take into consideration that factors may be present in cases to different degrees. The structuring of the model is illustrated by a real life example.

Table of contents

  • 1. Introduction and Outline
  • 2. CALAS
  • 3. Modelling of an example
  • 4. Conclusions and discussion
  • 5. References

1.

Introduction and Outline ^

[1]

This paper extends and partially formalises the JUDIPRO project – a legal expert system designed to perform inferences in the field of Polish divorce proceedings. The choice of the analysed domain was inspired by previous contributions to the modelling of family law disputes in AI and Law research [Stranieri, Zeleznikow, Gawler and Lewis 1999, Bellucci and Zeleznikow 2005, Bellucci 2008] and, primarily, by the works on the Parenting Plan Support System, co-authored with Adam Zienkiewicz [Araszkiewicz, Łopatkiewicz and Zienkiewicz 2014, Araszkiewicz, Łopatkiewicz, Zienkiewicz and Zurek 2015a, Araszkiewicz, Łopatkiewicz, Zienkiewicz and Zurek 2015b]. However, neither the structure of the knowledge base nor the inference engine of JUDIPRO are domain-dependent. The choice of the domain was dictated mainly by its features, such as the presence of many open-textured concepts, a significant number of judicial opinions issued in the field as well as the social importance of the facilitation of judicial decision-making in this context.

[2]

Unlike the work indicated above, JUDIPRO is not focused on the Alternative Dispute Resolution context, but on judicial reasoning exclusively. It has been designed to model the situations of a dispute between the parties which has to be solved by the court. The JUDIPRO project is partially inspired by the philosophical research on epistemic contextualism, a view according to which the truth conditions of assertion of a knowledge ascription sentence (in the form: S knows that P) depends on the context of utterance of such a sentence [Araszkiewicz and Łopatkiewicz 2015, DeRose 2009]. Hence, JUDIPRO does not simply generate legal conclusions, but it rather asserts the level of justification of a judicial decision with respect to the factual and legal context of its issuance. The module responsible for this evaluation is called the Knowledge Ascription Module (KAM). The robust structure of KAM was described in [Araszkiewicz and Łopatkiewicz 2015].

[3]

The contribution of this paper is to provide an extension and partial formalisation of KAM by introducing: (1) a formalisation of the notion of argument; (2) the degrees of justification of the elements of inference and (3) the degrees of strength of argumentative links. This part of KAM may be referred to as the calculus of argument strength (CALAS).

[4]

The outline of the investigations is as follows: In Section 2, we present the details of CALAS. The third section is devoted to the discussion of the legal context concerning judicial decision-making in Polish proceedings. Section 4 presents a formalisation of an actual case by means of the presented framework. Section 5 includes a discussion of the obtained results and conclusions.

2.

CALAS ^

[5]

The Calculus of Argument Strength (CALAS) is based on the Stanford Certainty Factors Algebra [Buchanan and Shortliffe 1984] and its modifications presented in [Niederliński 2011], but it is interpreted differently. The Strength of Argument does not represent the certainty of conclusion, but the level of promotion or demotion of one proposition by another proposition.

Definition 1. Argument

[6]

Let P be a set of propositional elements. Let L ⊆ P x P be a set of binary links between propositional elements. Set L is divided into two disjoint subsets L+ (representing supporting links) and L- (representing demoting links).

[7]

An argument is an ordered pair of propositional elements (reason, conclusion) which belong either to L+ or to L-.

[8]

We will represent arguments by using the following notation convention:

 

Reason => Conclusion,

 

although arguments in this account should not be equalled with defeasible rules, as arguments may also represent both much weaker types of links, such as factor-based links, and stronger ones, such as strict implication.

[9]

The classification of propositional elements into reasons and conclusions is functional in the sense that one and the same propositional element may stand as the reason in one argument and the conclusion in another.

Definition 2. Link valuation function.

[10]

The link valuation function is function v: L ↦ R: <-1, 1>.

[11]

The strength of the link is parameterised by real numbers ranging from -1 to 1.

[12]

If the value of the function is between -1 and 0, it means that the reason element of an argument demotes its conclusion. Otherwise, the link between the reason and the conclusion is a supporting one. The value of function v of a concrete link L will be designated as v(L). Obviously, the strength of the link between unrelated propositions equals 0.

Definition 3. Proposition valuation function.

[13]

The proposition valuation function is function w: P ↦ R: <-1, 1>.

[14]

Each proposition is assigned its value ranging from -1 to 1. If a proposition obtains value 1, it means that it is justified to the highest possible extent, given the relevant context. If a proposition is assigned value -1, its negation is justified in the same manner. In case of ineliminable doubt concerning acceptance or rejection of a proposition, it is assigned the value of 0. The value of function w of a concrete proposition p will be designated by w(p). When we intend to refer to the weight of types of propositions depending on the role they play in an argument, we write w(Reason) or w(Conclusion) respectively.

[15]

We assume here that both the quality of reasons as well as the nature of the relation between the reason and the conclusion of an argument have some bearing on the degree of justification of its conclusion. Hence, we introduce another parameter, to which we refer to as the Strength of Argument (SoAarg). The value of this parameter with regard to a concrete argument should be calculated in accordance with the following equation:

 

SoAarg = w(Reasonarg) * v(Larg),

 

where Reasonarg is the first element of the argument in question and Larg is the link between the reason and the conclusion of the argument.

[16]

Let us note that the abovementioned equation enables us to represent a variety of different argumentative situations:

[17]

First, let us note that if the value of either w(Reasonarg) or v(Larg) equals 0, then the SoAarg comprising any such components is assigned the same value. This consequence is intuitive since completely unrelated propositions cannot form any intelligible argument and it is difficult to ascribe any argumentative force to propositions we are totally agnostic about.

[18]

Second, if w(Reasonarg) obtains either of its maximal values (1 or -1), it means that Reasonarg, or its negation are justified to the maximal extent possible, given the context. However, let us note that even such a proposition can be related in a better or worse manner to the conclusion of an argument. Although the presented version of the model does not allow us to analyse the internal structure of arguments, we may observe an important property of the equation: if v(Larg) is positive and maximal, then SoAarg preserves the value of its reason. If v(Larg) equals -1, then the conclusion of the argument is demoted in line with the value of its reason.

[19]

Third, if both w(Reasonarg) and v(Larg) have negative values, it means that SoAarg obtains a positive value. This consequence is very intuitive: if an unjustified statement is incompatible with a certain conclusion and there is such a link between them that one of them should be accepted, then it creates a degree of justification for the latter proposition.

[20]

The values of propositions may be determined in several ways. First, the values of certain propositions, especially those with which we start the process of inference, may be assigned arbitrarily. Second, the propositions which are considered evident in a particular context may be assigned the maximal (positive or negative) value. Third, propositions which are not deemed evident, but which serve as assumptions for the process of inference, may be given some initial positive, yet not maximal, value. The value of propositions which are the conclusion of only one argument is identical to the value of strength of this argument.

[21]

As noted above, in the model presented here we do not analyse the structure of reasons of arguments. If a given argument is supported by a set of propositions in conjunction (the so-called linked argument, see the recent contribution of Selinger [2014, 381]), in this model we treat such a conjunction as a single reason, presuming that the strength of such a reason equals the strength of the weakest element of this conjunction. However, the model is able to represent convergent arguments too; that is, situations when one and the same conclusion is supported by independent reasons. In this context we speak about a cumulation of arguments.

[22]

If there is more than one argument supporting (L+) a given conclusion and there are no arguments demoting it, the total value of such a conclusion can be calculated on the basis of the following equation:

 

SoASUP_Con = SoAarg1 + SoAarg2 – SoAarg1* SoAarg2

[23]

Where: SoASUP_Con is the cumulated strength of arguments supporting conclusion Con (CP or intermediate concept), SoAarg1 is the strength of the first argument supporting the conclusion, SoAarg2 is the strength of the second argument supporting the conclusion. For a higher number of supporting arguments, cumulation is first performed on any two arguments, then the global strength for those two arguments is cumulated with the strength of the third argument, etc. If there are no arguments attacking the conclusion, then the final valuation of the conclusion is equal to the cumulated strength of the arguments supporting the conclusion:

 

w(Con_arg1, arg2,...)=SoASUP_Con

[24]

If there is more than one argument demoting (L-) a given conclusion and there are no arguments supporting it, the total value of such a conclusion can be cumulated on the basis of the following equation:

 

SoADEM_Con = SoAarg1 + SoAarg2 + SoAarg1* SoAarg2

[25]

Where: SoADEM_Con is the cumulated strength of attacking arguments, SoAarg1 is the strength of the first argument attacking the conclusion, SoAarg2 is the strength of the second argument attacking the conclusion. For a higher number of attacking arguments, cumulation is first performed on any two arguments, then the global strength for those two arguments is cumulated with the strength of the third argument, etc. If there are no arguments supporting the conclusion, then the final valuation of the conclusion is equal to the cumulated strength of the arguments attacking the conclusion:

 

w(Con_arg1, arg2,...)=SoADEM_Con

[26]

Finally, if there are both arguments supporting and demoting a given conclusion, its total value should be determined after calculating the strength of arguments supporting and demoting it, by means of the following equation:

 

w(Con_ arg1, .., arg(n)) = (SoAsup_Con + SoAdem_Con) / (1-min(|SoAsup_Con| + |SoAdem_Con|))

[27]

For example: if we have two arguments supporting and one demoting a given conclusion C:

 

Arg1: A => C with v(L)=0.5
 
Arg2: B => C with v(L)=0.8
 
Arg3: D => C with v(L)=-0.5
[28]

A, B and D are facts with strength equalling 1, then

 

SoAarg1_c = 0.5
 
SoAarg2_c = 0.8
 
SoAarg3_c = -0.5
 
the cumulated strength of the argument supporting C will equal:
 

 

SoAsup_C = SoAarg1_C + SoAarg2_C – SoAarg1_C* SoAarg2_C= 0.5 + 0.8 – 0.5*0.8= 0.9
[29]

The cumulated strength of all arguments supporting and attacking proposition C can be calculated from the following equation:

 

 

SoAC = (SoAsup_C + SoAarg3_c) / (1-min(|SoAsup_C| + |SoAarg3_c|))= (0.9 + (-0.5))/ (1-min(|0.9|, |-0.5|)) = 0.2

[30]

Eventually, w(C_arg1, arg2, arg3) will equal 0.2, that is the value of SoAC.

3.

Modelling of an example ^

[31]

The details of the legal background of Polish divorce proceedings are discussed in more detail in [Araszkiewicz and Łopatkiewicz 2015], making it redundant to adduce them here. It is sufficient to recall that the basic necessary condition for granting a divorce is «the complete and irretrievable breakdown of marital life». Also, the court will typically determine which of the parties is guilty of this breakdown. As these concepts are highly open-textured and context-sensitive, it is not possible to elaborate an exhaustive set of the necessary and sufficient circumstances for ascertaining that the conditions are met. A set of such circumstances assessed as sufficient in case A may be not seen as such, or even evaluated as irrelevant, in case B. On the other hand, there are some stereotypical fact patterns, representable by intermediate concepts, which facilitate the judicial reasoning in concrete cases. In AI and Law literature, such fact patterns are typically referred to as factors [Ashley 1990; Aleven 1997]. JUDIPRO makes use of factor-based knowledge representation, assuming a bottom-up hierarchy going from the propositions expressing evidence gathered in a case though the layers of intermediate concepts up to the level of statutory predicates, which are referred to as Central Propositions – the propositions which are the ultimate grounds for court decisions.

[32]

The example modelled below is based on an actual case decided on the 21st of November 2013 by the Appellate Court in Kraków, No. I ACa 1122/13, which is one of the cases forming the JUDIPRO knowledge base. The decision was made as a result of an appellation filed by the plaintiff (the wife) who was dissatisfied with the judgment of the 1st instance court who granted the divorce, but at the same time it declared that both spouses were guilty with regard to the breakdown of marital life.

[33]

The case was annotated manually in order to identify the crucial natural language expressions referring to: first (1) evidence-related proposals (that is, the statements which are determined on the basis of proof as submitted by the parties) and (2) the set of doctrinal intermediate concepts. As regards the latter, the doctrine of Polish family law identifies three types of «bonds» the lack of which is typically seen as sufficient for the determination of the complete and irretrievable breakdown of marriage, that is: the emotional, physical and economic bonds. The continuation of any of these three bonds will typically lead to the rejection of the plaintiff’s claim and the divorce will not be granted. Therefore, it is necessary to identify convincing evidence supporting the claims that each of these bonds has ceased to exist.

[34]

In the modelled case it is possible to identify the following set of evidence-related propositions:

 

EVcase = {evchange_of_husbands_behaviour, evhusband_drinks_alcohol, evhusband_has_frequent_contacts_with_another_woman, evhusband_ignores_family_needs, evhusband_sells_family_assets, evhusband_functions_without_a_sense_of_commitment_to_family, evhusband_lives_out_of_home, evlack_of_love_declaration, evlack_of_will_to_renew_marital_life, evwife_wanted_to_help_husband, evwife_partially_forgave_husband, evwife_is_not_a_conflict_person, evwife_recorded_husband}

[35]

We assume that all evidence propositions have the strength of 1 as reasons in arguments adopting intermediate concepts as their conclusions. The list of arguments may be presented as follows:

 

arg1: evchange_of_husband’s behaviour => icemotional_bond v(L1)=-0.6
 
arg2: evhusband_drinks_alcohol => icemotional_bond v(L2)=-0.5
 
arg3: evhusband_drinks_alcohol => icphysical_bond v(L3)=-0.4
 
arg4: evhusband_drinks_alcohol => iceconomic_bond v(L4)=-0.2
 
arg5: evhusband_has_frequent_contacts_with_another_woman => icemotional_bond v(L5)=-0.8
 
arg6: evhusband_has_frequent_contacts_with_another_woman => icemotional_bond v(L6)=-0.7
 
arg7: evhusband_ignores_family_needs => iceconomic_bond v(L7)=-0.4
 
arg8: evhusband_sells_family_assets => iceconomic_bond v(L8)=-0.6
 
arg9: evhusband_functions_without_a_sense_of_commitment_to_family => icemotional_bond v(L9)=-0.4
 
arg10: evhusband_lives_out_of_home => icphysical_bond v(L10)=-0.6
 
arg11: evlack_of_love_declaration => icemotional_bond v(L11)=-0.8
 
arg12: evlack_of_will_to_renew_martial_life => icemotional_bond v(L12)=-0.8
 
arg13: icemotional_bond => cpbreakdown_of_marriage v(L16)=-0.6
 
arg14: icphysical_bond => cpbreakdown_of_marriage v(L17)=-0.6
 
arg15: iceconomic_bond => cpbreakdown_of_marriage v(L18)=-0.6
[36]

As it is assumed here that all evidences have the strength of 1, the SoAs of arguments 1-12 will be identical to the value of their links.

[37]

On the basis of the above, we can calculate the cumulated strength of intermediate concepts related to the Central Proposition of breakdown of marital life. For example, w(ic_emotional_bond) is attacked by 5 arguments (arg1, arg2, arg5, arg9, arg11). First, we calculate the cumulated strength of arguments arg1 and arg2:

 

SoAarg1,arg2= SoAarg1 + SoAarg2 + SoAarg1 * SoAarg2 = (-0.6)+(-0.5) + (-0.6)*(-0.5) = -0.8

[38]

In the next step we cumulate SoAarg1,arg2 with SoAarg5:

 

SoAarg1,arg2,arg5= SoAarg1,arg2 + SoAarg5 + SoAarg1,arg2 * SoAarg5 = (-0.8)+(-0.8) + (-0.8)*(-0.8) = -0.96
[39]

Then, we cumulate SoAarg1,arg2,arg5 with argument arg9, and finally with arg11. The strength of the intermediate concepts related to the Central Proposition of breakdown of marital life will be equal to:

 

w(ic_emotional_bond) = SoA_icemotional_bond= -0.999
w(ic_physical_bond) = SoA_icphysical_bond = -0.928

 

w(ic_economic_bond) = SoA_iceconomic_bond = -0.808
[40]

Finally, we calculate the cumulated strength of the Central Proposition «breakdown of marriage», which amounts to 0,909.

[41]

Similarly, we calculate the cumulated strength of the Central Proposition «wife’s guilt»:

 

arg16: evwife_recorded_husband => icdisloyal_act v(L16)=0,8
 
arg17: evwife_wanted_to_help_husband => cpwife’s_guilt v(L13)=-0.4
 
arg18: evwife_partially_forgave_the_husband => cpwife’s_guilt v(L14)=-0.5
 
arg19: evwife_is_not_a_conflict_person => cpwife’s_guilt v(L15)=-0.3
 
arg20: icdisloyal_act => cpwife’s_guilt v(L20)=0.4
[42]

Similar to the above, all evidences have the strength equal to 1, hence the strength of the intermediate concept icdisloyal_act is equal to 0,8 (w(icdisloyal_act) = 1*v(L16)). To calculate the cumulated strength of the Central Proposition «wife’s guilt», first we have to cumulate the strength of arguments attacking the given CP (arg17 to arg19) and, second, we need to cumulate the strength of attacking arguments 17, 18, and 19 with argument 20 which supports «wife’s guilt»:

 

SoADEM_wife’s_guilt = SoAarg17,arg18,arg19 = -0.79
 
SoASUP_wife’s_guilt = SoAarg20 = w( icdisloyal_act) * v(L20) = 0.32
 
SoAwife’s_guilt = (SoASUP_wife’s_guilt + SoADEM_wife’s_guilt) / (1-min(|SoASUP_wife’s_guilt| + |SoADEM_wife’s_guilt|))= (0.32 + (-0.79))/ (1-min(|0,32|, |-0.79|)) = -0.723

4.

Conclusions and discussion ^

[43]

The CALAS module of JUDIPRO enables it to perform inferences and calculate the strength of the obtained conclusions, both intermediate and final ones. The values of propositions, links and the total strength of arguments are parameterised by means of real numbers ranging from -1 to 1.

[44]

CALAS may be seen as a kind of bipolar abstract argumentation framework (AAF, see [Dung 1995] for an elaboration of the concept of AAFs and [Cayrol and Lagasquie-Schiex 2005] for bipolarity, that is, the introduction of attacking and supporting relations between nodes). Unlike classical AAFs, it does not merely assess the arguments as attacked or supported, but it also calculates the value of such an attack or support. A question arises then how the value of the initial propositions and of the given links can be determined? In the example discussed above, such values have been assigned to a high extent in an arbitrary manner by means of a set of simple heuristics (e.g. evidence is assigned value 1; no argumentative link is assigned value 1 or -1 if it’s not deductive, etc.). However, it should be emphasised that the KAM part of JUDIPRO as outlined in [Araszkiewicz and Łopatkiewicz 2015] shall encompass much more structures concerning the quantitative features of argumentation. We particularly aim to extend CALAS into a meta-level argumentation framework, capable of putting into question also the valuations of propositions and links. Generally, one of the most promising lines of future research seems providing for mapping the model presented here onto the existing AAFs.

[45]

Another question which may be posed in connection with this contribution is whether the chosen dense parameterisation makes sense. The answer is that the project is not designed to be descriptively adequate with regard to the language used for valuation of arguments; it is assumed that it has to be as flexible as possible in order to enable its implementation in different environments using quantitative measures (such as neural networks and Bayesian networks). However, a translation to less fine-grained, qualitative scale is also possible. Let us consider the classical tripartite scale of promotion/demotion as introduced to legal theory by Alexy [2002] and fruitfully used in the field of AI and Law for instance by Grabmair and Ashley [2010]. Hence, the following mapping seems quite natural: -1 to -0,(6) – strong demotion; -0(6) to -0(3) – moderate demotion; -0(3) to 0 (excluding 0) – light demotion; the parts of the scale for support can be defined accordingly. Obviously, the less fine the granularity is, the potentially less complete the ordering between the compared elements.

[46]

As regards the knowledge representation structures used here, the hierarchy of propositional elements ranging from evidence through intermediaries to Central Propositions resembles the structures used in CATO [Aleven 1997] or IBP [Brüninghaus and Ashley 2003]. However, the links between propositional elements are given real number values and are context-sensitive, hence we do not assume that in different cases the links between the evidence propositions and intermediate concepts would be identical.

[47]

In summing up the above considerations, let us indicate that the future investigations concerning the JUDIPRO project will concern not only the analysis of its formal features (as indicated above), but also testing its capabilities on a large corpus of judicial opinions and providing a more fine-grained structure of the knowledge base of the system.

5.

References ^

Aleven, Vincent, Teaching case-based argumentation through a model and examples. (Unpublished doctoral dissertation). University of Pittsburgh Graduate Program in Intelligent Systems, 1997.

Alexy, Robert, A Theory of Constitutional Rights, trans. Rivers, John, Oxford University Press, Oxford 2002 (repr. 2010, 1st edition in German 1986).

Araszkiewicz, Michał/Łopatkiewicz, Agata, Zienkiewicz, Adam, Factors and Dimensions in Parent Plan Support System, In: Schweighofer, Erich / Kummer, Franz / Hötzendorfer, Walter (Hrsg.), Transparenz / Transparency, Tagungsband des 17. Internationalen Rechtsinformatik Symposions IRIS 2014, Österreichische Computer Gesellschaft, Wien 2014, pp. 103–112.

Araszkiewicz, Michał/Łopatkiewicz, Agata, A Multi-Level Architecture of a Judicial Decision Support System in Divorce Proceedings (the JUDIPRO). In: Schweighofer, Erich / Kummer, Franz / Hötzendorfer, Walter (Hrsg.), Kooperation / Co-operation, Tagungsband des 18. Internationalen Rechtsinformatik Symposions IRIS 2015, Österreichische Computer Gesellschaft, Wien 2015, pp. 109–116.

Araszkiewicz, Michał/Łopatkiewicz, Agata/Zienkiewicz, Adam/Zurek, Tomasz, Representation of an actual divorce dispute in the parenting plan support system. In: Sichelman, Ted/Atkinson, Katie (eds.) Proceedings of the 15th International Conference on Artificial Intelligence and Law, ICAIL 2015, ACM New York 2015 (2015a), pp. 166–170.

Araszkiewicz, Michał/Łopatkiewicz, Agata/Zienkiewicz, Adam/Zurek, Tomasz, Framing a Knowledge Base for a Legal Expert System Dealing with Indeterminate Concepts, The Scientific World Journal 2015, Article ID 985425 (2015b), doi:10.1155/2015/985425.

Ashley, Kevin, Modeling legal argument: Reasoning with cases and hypotheticals. MIT Press, Cambridge, Mass, 1990.

Bellucci, Emilia, Development of a mediation tool in Family Law: AssetDivider. In: Francesconi, Enrico/Sartor, Giovanni/Tiscornia, Daniela (eds.), Legal Knowledge and Information Systems – JURIX 2008: The Twenty-First Annual Conference on Legal Knowledge and Information Systems, IOS Press, Amsterdam 2008, pp. 88–97.

Bellucci, Emilia/Zeleznikow, John, Developing negotiation decision support system that support mediators: A case study of the Family_Winner system, Artificial Intelligence and Law 2005, vol. 13, pp. 233–271.

Buchanan, Bruce/Shortliffe, Edward (eds.), Rule Based Expert Systems: The Mycin Experiments of the Stanford Heuristic Programming Project, Addison-Wesley, Reading, Mass. 1984.

Cayrol Claudette/Lagasquie-Schiex Marie-Christine, On the Acceptability of Arguments in Bipolar Argumentation Frameworks. In: Godo, Lluís (ed.), Symbolic and Quantitative Approaches to Reasoning with Uncertainty, 8th European Conference, ECSQARU 2005, Barcelona, Spain, July 6–8, 2005. Proceedings, Lecture Notes in Computer Science 2005, vol. 3571, pp. 378–389.

DeRose, Keith, The Case for Contextualism: Knowledge, Skepticism, and Context, Vol. 1, Clarendon Press, 2009.

Dung, Phan Minh, On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-Person Games. Artificial Intelligence Vol 77(2), pp. 321–358 (1995).

Grabmair, Matthias/Ashley, Kevin, Argumentation with Value Judgments – An Example of Hypothetical Reasoning. In: Winkels, Radboud (ed.), Legal Knowledge and Information Systems – JURIX 2010: The Twenty-Third Annual Conference on Legal Knowledge and Information Systems, IOS Press, Amsterdam 2010, pp. 67–76.

Niederliński, Antoni, rmes Rule- and Model-Based Expert Systems, Second edition, 2011. Online: http://www.rmes.pl/download/rmes_book.rar.

Selinger, Marcin, Towards Formal Representation and Evaluation of Arguments, Argumentation 2014, vol. 28, pp. 379–393.

Stranieri, Andrew/Zeleznikow, John/Gawler, Mark/ Lewis, Bryn, A hybrid rule–neural approach for the automation of legal reasoning in the discretionary domain of family law in Australia, Artificial Intelligence and Law 1999, vol. 7, pp. 153–183.