Jusletter IT

The Calculemus-FLINT method is a method for normative analysis. It consists of a step-by-step guide for the interpretation and use of sources of norms (Calculemus) and a domain specific language for expressing normative states, and normative transitions (FLINT). The method has been used to extract the tasks of the Dutch Immigration Service (IND) from a interpretation of the Dutch Aliens Act [Doesburg 2019–2], solving a normative dispute on the policies of the Dutch Tax administration concerning the withdrawal of objections based on telephone conversations [Doesburg 2018–2], comparing the application of exclusion clauses in the UN Refugee Convention in the UK and the Netherlands [Doesburg 2018–1] and on archetypical study casus used in the field of artificial intelligence (AI) and Law [Doesburg 2019–1].

While the primary goal of the development of Calculemus-FLINT is the support of large administrative organizations to build norm compliant case assessment systems, we always saw this as a steppingstone towards building and operating a normative multi-agent systems (NMAS). We intend to use NMAS to evaluate policy drafting, policy implementation, policy enforcement and dispute resolution in individual cases. In this paper, we will assess the Calculemus-FLINT method for its usefulness for that purpose. We’ll do this using the ten guidelines for the use of NMAS in computer science by Guido Boella, Gabriella Pigozzi and Leendert van der Torre [Boella 2009] as a framework. The assessment presented in this paper, will be used for the further development of the Calculemus-FLINT method and for creating instruments to support the method.

Although some readers might be familiar with our work we give a short introduction of Calculemus-FLINT in Section 2. In Section 3, we give a short introduction of the ten guidelines for the use of NMAS. In Section 4, we will reflect on the Calculemus-FLINT method using the ten guidelines for the use of NMAS in computer science. In Section 5, we discuss the outcome of the analysis and we will comment on our experience using the guidelines for NMAS.

Over the last few years we developed a method that aims to make interpretations of sources of norms that can be used to:

1. make specifications for normative tasks performed by people or machines,
2. be able to explain normative decisions to layman,
3. support decision-making and dispute resolution in administrative organizations and courts, and
4. support the implementation and evaluation of policies in large administrative organizations.

The final goal of this work is to create a method for the explicit interpretation of sources of norms, resulting in formal specifications that can be used to regulate the behavior of human and artificial agents, as part of a NMAS. The method differs from the usual deontic approach because of its focus on actions, their validity and results, rather than on rights and duties.

Kanger and Kanger [Kanger 1966], Lindahl [Lindahl 1977], Makinson [Makinson 1986], Jones and Sergot [Jones 1996] advocated the formalization of normative relations not only in a deontic dimension, i.e. obligations, prohibitions and permissions, but also in a potestative dimension consisting of statements on power or competence, based on the work of Salmond [Salmond 1902, 217–38] and Hohfeld [Hohfeld 1913]. Giovanni Sileno noticed recently that the search for a formalization of the notion of power, is still work in progress [Sileno 2016].

The progress in our search for an action-based formalization for normative relations can be found in earlier publications [Doesburg 2016] [Doesburg 2018–1] [Doesburg 2018-2] [Doesburg 2019–1] [Doesburg 2019–2] [Dymitruk 2018]. In this paper we will only give a short overview of the Calculemus method and the FLINT language. In Section 4, we will elaborate on this overview in order to assess the Calculemus-FLINT method, using the ten guidelines for the use of NMAS as a reference. A comprehensive publication on the Calculemus-FLINT method and its applications will be published later this year.

The Calculemus approach to normative questions is a simple step-by-step plan for solving normative questions. These steps, see below, can be used in combination with any method for expressing the meaning of sources of norms.

1. Express a normative question.
2. Collect sources of norms relevant for answering the question.
3. Express your interpretation of sources of norms in a representation that can be discussed with all people that have an interest related to the question.
4. Apply interpretations in order to answer the question.
5. Compare your answer with those of others and make a structured assessment of disputes.

For the explicit interpretation of sources of norms, we have developed what we have developed the «formal language for the interpretation of sources of norms» (FLINT). In its current stage it is a semi-formal language that is evolved from working on real-life cases. The formalization of this language, requiring a formal semantics is work in progress, but a sub-set of its constructs has been formalized and tested already. The language consists of three types of frames: «act frames», «duty frames» and «fact frames».

The FLINT language and its relation to both sources of norms, and individual cases is shown in Figure 1. The approach we have developed is based on two fundamental theories, respectively the theory of law as described by John Salmond [Salmond 1902] and Wesley Hohfeld [Hohfeld 1913], and second the work of Searle on the construction of social reality [Searle 1969] [Searle 1995] [Searle 2010]. The description of institutional norms floating on brute reality using status functions in order to represent facts and to find out whether they correspond with reality is based on the work of Searle. The separation between sources of norms, the interpretation of norms and representations of the world is an example of using status functions to build a normative system.

Figure 1: Making and using FLINT interpretations of sources of norms

The FLINT language is representing normative relations based on the views of Salmond on Jurisprudence. Salmond introduced the four beneficial rights: claims¹, liberties, powers and immunities, with their correlative duties, no claim², liabilities, and disabilities. He defined claim-duty relations and power-liability relations as the two fundamental normative relations. He noticed that the liberty-no claim relation is equal to the absence of a claim-duty relation, and that the immunity-disability relation is equal to the absence of a power-liability relation. Salmond also introduced five element that are involved in every right, and which all can be found in the FLINT act frames [Salmond 1902, 224–5]:

1. A person having the right.
2. The person bound by that right, having a correlative relation with the person having the right.
3. An act in favor of the person having the right.
4. Something to which the act relates, an object.
5. A title, that is to say, certain facts or events from which the right derives its origin.

The only elements in FLINT act frames that are missing in Salmonds list of the constituting elements of all rights are: the precondition (the condition that must be true in order to make an act valid), and the postcondition (the result of the performance of a valid normative act).

Wesley Hohfeld can be credited for constructing his analysis as a practical tool [Dickey 1971], also it was Hohfeld’s work that draw attention from scientists in the field of philosophy of law and (later) computer science.

In 2009 Guido Boella, Gabriella Pigozzi, and Leendert van der Torre introduced ten guidelines for the use of NMAS in computer science [Boella 2009]. A short version of this paper was presented at Jurix 2009. The guidelines will be introduced shortly below.

Guideline 1: Definition of normative multiagent system used.

In literature there are many definitions of the concept «norm». Address the explicit representation of norms, norm compliance and norm implementation.

Guideline 2: Make explicit why norms are a kind of (soft) constraints deserve special analysis.

Is non-compliant (illegal) behavior is possible in the system? Boella et al. observe that normative systems, unjustly, often just rule out illegal behavior by specification.

Guideline 3: Explain why and how norms can be changed at runtime.

Guideline on the possibilities to change the norms of the system. Norms can be changed by agents in the system and agents outside of it.

Guideline 4: Discuss the use and role of norms as a mechanism in a game-theoretic setting.

Emphasizes the game-theoretic model and the notion of a norm as a mechanism. Mechanisms in a game-theoretic setting, used by agents to coordinate themselves, have become more important in normative systems than representation issues. For this reason, normative systems need additional functionalities, e.g. the possibility to communicate or distribute norms among agents, and the possibility to detect the compliancy of agents, or the lack thereof [Boella 2006] [Boella 2009].

Guideline 5: Clarify the role of norms in the multiagent system.

Organizational issues are more and more introduced in the definition of NMAS. Norms are no longer seen only as the mechanism to regulate behavior of the system. They are also seen as a part of a larger institution. This raises questions on the role of norms in these institutions.

Guideline 6: Relate the notion of «norm» to the legal, social, or moral literature.

There is a shift of interest from the more static legalistic view of norms (where power structures are fixed) to the more dynamic interactionist view of norms (where agent interaction is the base for norm related regulation). Boella et al. put the legalistic and interactionist view in the context of five levels in the development of NMAS [Boella 2009]: off-line norm design (1), norm representation (2), norm manipulation (3), social reality (4), moral reality (5).

Guideline 7: The use of norms to resolve dilemma’s and to regulate interaction among agents.

Use norms not only to distinguish right from wrong, but also to resolve dilemmas, and use norms not only describe violations, but in general to coordinate, organize, guide, regulate or control interaction among agents.

Guideline 8: Distinguish norms from obligations, prohibitions and permissions.

Deontic logic traditionally is about obligations, prohibitions and permissions. Boella et al. make distinctions between these concepts and concepts like «violation», «norms» and «imperatives». Boella et al. notice that conditional aspects of normative behavior are typically left implicit in deontic logic. They refer to phenomena like contrary-to-duty dilemma’s, and Jorgensens dilemma to illustrate the importance for normative systems to be able to handle conditional norms. They also advise to use real life sources of norms, e.g. the European Constitution, to assess whether all aspects of such a source can be expressed within the normative system.

Guideline 9: Use the deontic paradoxes only to illustrate the normative multiagent system.

The so-called «paradoxes of deontic logic» are usually dismissed as consequences of the simplifications of Standard Deontic Logic (SDL). Contrary-to-duty paradoxes have been discussed for fifty years in deontic logic because a lot of normative reasoning is directly or indirectly related to violations, just like in defeasible reasoning a lot of reasoning is directly or indirectly related to exceptions. Therefore, it is important to assess the handling of these paradoxes if one is to analyze the working of a normative system.

Guideline 10: Consider regulative norms in relation to other kinds of norms and concepts.

Boella et al. distinguish regulatory norms and constitutive norms, where regulative norms are obligations, prohibitions and permissions, and constitutive norms state what counts as institutional facts in a normative system. Constitutive norms are based on the notion that «X counts-as Y in context C» and are used to support regulative norms by introducing institutional facts in the representation of legal reality.

In this section, the usefulness of the Calculemus-FLINT method for building NMAS is discussed, using the ten guidelines for normative systems as a framework.

Guideline 1: Definition of normative multiagent system used.

A normative system in the context of the Calculemus-FLINT method has the following characteristics:

1. A normative multiagent system is a system of agents whose behavior is regulated by norms, where every agent has its own normative system, and can interact with other agents.
2. All normative systems have an initial state.
3. Norms are derived by interpretation of sources of norms by an agent.
4. The only way to change normative states in a normative system is by qualification of behavior in the system by executing a valid normative act.
5. A normative act consists of the action of an actor on an object. A precondition must be true in order to constitute a postcondition in which normative facts and/or duties are created and/or terminated. A recipient is bound by the action of the actor.
6. Any agent can have an individual set of norms, consisting of interpretations of sources of norms, every agent can share norms with one or more other agents, disputes on the interpretation of a source between agents can exist.
7. Every state of a normative system can be derived from the initial state by instantiations of normative acts available in that system.
8. Sources of norms can be changed or added to the system at any time, using a procedure regulated by norms.
9. At any time any agent can make interpretations of new sources of norms, or can change interpretations of existing sources.
10. Agents in the system can have disputes on the interpretation of sources of norms, the description of states of the system, or the qualification of instantiations of valid normative acts.

Guideline 2: Make explicit why norms are a kind of (soft) constraints deserving special analysis.

The most important principle in computer science, and according to Dijkstra of all intelligent thinking is «separation of concerns», the willingness to study in depth an aspect of one’s subject matter in isolation for the sake of its own consistency, all the time knowing that one is occupying oneself only with one of the aspects. [Dijkstra 2016]. As a consequence, all concepts deserve special analysis, including norms. In the Calculemus-FLINT method norms are not considered to be constraints of behavior. Norms define the normative consequences of a valid act, or define acts that «ought to be» performed in the future (duties). Duties represent a situation in which a duty holder is bound to see to it that in the future, an act is performed, under circumstances that make that act valid. Performing of the act terminates the duty. Neglecting a duty can lead to the performance of enforcing acts that become accessible if a duty is not fulfilled.

Norms are necessarily, more or less, soft constraints on the behavior of agents. Normative behavior requires agents that can choose to comply, or not. Of course, in a democratic state of law the freedom to choose is limited by a judicial system and a state monopoly on the legitimated use of physical force [Weber 1992]. Norms are expressions to describe what agents «ought to do». If agents in a system «must» comply and if violation of norms is impossible, then the behavior of those agents is not normative. Under those circumstances the deontic logical modalities do not apply. The alethic modalities «possible», «impossible» and «necessary» should be used instead.

Norms are expressions that determine the normative results of valid normative. It is up to the agent to determine what to do, given a set of norms, believes, desires and intentions held by the agent.

Guideline 3: Explain why and how norms can be changed at runtime.

Norms are interpretations of sources of norms. Changes in sources of norms, should have an effect on the normative system. These changes are triggered by interpretations that result in machine readable norms. Even if the sources of norms remain unchanged, the agents in a normative system might change their interpretation of existing sources. If agents in the system are having a dispute on the correct interpretation of sources, changes towards a shared set if interpretations, are an essential condition for dispute resolution.

In a normative system that is described using the Calculemus-FLINT method, the validity of all norms is set by timestamps. Sources of norms also have timestamps for validity. As a result the relevant norms for any normative act can be determined at any given time. The outcome of a normative analysis of an action performed on a specific moment in time, may be different if the assessment is done by another agent, or at another moment in time. In order to be able to create established normative states in a system, the system should have norms that regulate dispute resolution and norms for creating and changing sources of norms.

Guideline 4: Discuss the use and role of norms as a mechanism in a game-theoretic setting.

In a game-theoretic setting, norms are used to set a framework for the qualification of behavior of agents in the system. The game has a goal that the agents want to achieve. Discussing the role of norms in a game-theoretic setting, requires a clarification of the role of norms in the multiagent system (see guideline 5).

Guideline 5: Clarify the role of norms in the multiagent system

The role of norms in a normative system is:

1. Norms (acts and duties) represent explicit interpretations of sources of norms by an agent in a system, enabling other agents in the system to share that interpretation, or not.
2. Valid acts constitute the only way to change states in a normative system.
3. The postconditions of acts can be used by agents to define a path that they believe, will bring them closer to achieving a certain goal.
4. Norms can be used to evaluate the behavior of agents, e.g. qualifying compliancy, or evaluating the degree of success of an agent in pursuing its goals.
5. Norms can include acts that can be used to punish incompliant behavior, create new sources of norms, interpret sources of norms, or regulate dispute resolution.

Calculemus-FLINT is, at this point, supporting the collection of relevant sources, the interpretation of sources, the assessment of normative states, and simple forms of dispute resolution.

Calculemus-FLINT does not yet contain functionalities to support goal selection, the selection of paths towards achieving goals, nor for the evaluation of successful pursuit of goals. These functionalities are a necessity to model behavior in a game-theoretic setting.

Guideline 6: Relate the notion of «norm» to the legal, social, or moral literature.

The relation of norms to legal and social literature is presented in Section 2. Moral literature is considered to be a part of literature on social behavior and jurisprudence.

The Calculemus-FLINT method is based on a interactionist view of norms, using a legalistic approach as a savior in last resort for dispute resolution. It has level 3 in the development of normative multiagent systems as described in [Boella 2009, 10]: agents can add and remove norms following the rules of the normative system (norm manipulation) We are working towards level 4 (social reality) by creating a NMAS that meets the challenges posed by the interactionist viewpoint as presented by [Boella 2009, 10].

Guideline 7: The use of norms to resolve dilemma’s and to regulate interaction among agents.

Boella et al. are not very careful in their use of the terms paradox and dilemma as synonyms. Contrary to duty obligations (CDT) are classified both as a dilemma and a paradox. Jorgensens’ dilemma is also referred to as Jorgensens’ paradox. Before we illustrate how these phenomena are represented in Calculemus-FLINT, we will discuss the difference between a dilemma and a paradox.

A dilemma is a situation in which a difficult choice has to be made between two or more alternatives, especially equally undesirable ones. A paradox is a statement or proposition that, despite sound (or apparently sound) reasoning from acceptable premises, leads to a conclusion that seems senseless, logically unacceptable, or self-contradictory. A normative system is supposed to support agents in making a motivated choice in case of a dilemma. Supporting the choice of an agent in a dilemma is about clarifying the alternatives in order to determine the preferable choice. Classifying the state of a system as a paradox, is merely the same as declaring the normative system has a deficiency. A paradox is a situation in which a normative system is not able to come to a sensible, logically acceptable, conclusion that is not self-contradictory.

Jorgensens’ dilemma is caused by the fact that imperatives and norms have no truth value. Therefore, an argument containing an imperative or a norm as premise, has no conclusion. Calculemus-FLINT solves this dilemma by rephrasing a norm in a form that has a truth value, e.g. «doing X is a valid normative act in state Y». This is a premise that does hold a truth value. The question whether «doing X» is better than NOT «doing X» is unanswered. That, however, is not a normative question but a matter of setting goals, and evaluating the preference between different paths that (might) lead towards the accomplishment of that goal, see discussion under guideline 5.

Chisholm’s paradox is the original version of the contrary-to-duty paradox. It consists of four sentences about Jones and his neighbors [Chisholm 1963]:

1. It ought to be that Jones goes to the assistance of his neighbors.
2. It ought to be that if Jones goes to the assistance of his neighbors, then he tells them he is coming.
3. If Jones doesn’t go to the assistance of his neighbors, then he ought not tell them he is coming.
4. Jones does not go to their assistance.

It is widely thought that Chisholm’s paradox is constituted by a mutually consistent and logically independent set of sentences: all four might be true at once, and none is a deductive consequence of the others [Hansen 2007]. However, the outcome (Jones does not go to their assistance) is in some circumstances undesirable, or even senseless and wrong, e.g. in an emergency the choice not to assist may count as negligence, which is unlawful.

The reason why these sentences lead to a paradox, is that sentences can be only true of false, right or wrong. There are no conditional norms. Calculemus-FLINT solves this problem by expressing norms as acts. The act is valid if a precondition is met, making the norm conditional. The performance of a valid act results in a postcondition. This allows Jones to compare the state where he is not going to the assistance of his neighbors, with the state where he goes, without telling them he is coming.

Of course, this requires extra information on the situation in which Jones has to decide on assisting his neighbors and the results of his actions. But then, one could hold the opinion that without additional information it is impossible to for Jones, or any other agent, to act in a way that makes any sense at all in real life. And that is the ultimate goal of a normative system: doing the right thing in real life.

Boella et al. suggests to use norms not only to distinguish right from wrong, but also to coordinate, organize, guide, regulate or control interaction among agents. The normative aspects of these acts can be represented in act frames. But coordination, organization, guiding, regulating and controlling also require functionalities like goal selection, the selection of paths towards achieving goals, and the evaluation whether the pursuit of a goal was successful. These functions are not yet available in Calculemus-FLINT.

Guideline 8: Distinguish norms from obligations, prohibitions and permissions.

In Section 2 we described the necessity of using deontic norms, and potestative norms. We also described how these phenomena can be represented using «act frames» and «duty frames».

Guideline 9: Use the deontic paradoxes only to illustrate the normative multiagent system.

This guideline is a specification is a specification of guideline 7 on the use of norms to resolve dilemma’s, see guideline 7.

Guideline 10: Consider regulative norms in relation to other kinds of norms and concepts.

Norms can be constitutive or regulative. Constitutive rules have the form «X counts as Y in context C». Regulative rules regulate the behavior of agents. In [Searle 1969, 33–4] Searle states that all norms are regulative, and some are constitutive, e.g. the rules of chess constitute the game of chess and regulate the game. The rule «drive on the right-hand side of the road» is not considered constitutive by Searle, because driving can exist prior to the existence of the rule to drive on the right-hand side of the road.

In [Hage 2013] Hage claims that all norms are constitutive and regulative rules are a subtype of constitutive rules. Regulatory rules, according to Hage, constitute the possibility to comply to the regulation. Being compliant to the regulatory rule to drive at the right-hand side of the road, then constitutes the fact that the driver is compliant. It is unclear whether Hage holds the opinion that constitutive rules that do not regulate behavior exist. The authors consider, like Searle, that all normative rules are regulative. Like Hage, we believe that the application of any rule constitutes institutional facts describing compliance to or violation of that rule, giving all rules also a constitutive aspect.

In Calculemus-FLINT norms are either describing acts or future acts (duties). Acts have a regulative aspect because an act must be performed in a state that meets a precondition in order to be valid, and constitutive because the performance of a valid act constitutes a postcondition. The authors have found no reason to introduce other types of norms then acts and duties.

In this Section we will discuss the assessment of the Calculemus-FLINT method and the experience we gathered by using the ten guidelines for NMAS. Based on our experiences we will suggest an adjusted set of guidelines for building NMAS.

Using the ten guidelines was helpful for assessing the current state of the Calculemus-FLINT method and making plans for further development of the method. However, we do believe the guidelines could be more structured by addressing the components of a normative multiagent system more explicitly. The guidelines could be more supportive in reviewing the readiness for diverse applications of NMAS as well as for comparing methods for making and using NMAS.

We will illustrate this by discussing the assessment of the Calculemus-FLINT method using the guidelines.

As is described in Section 2 the Calculemus-FLINT approach is founded on the recognition of fundamental normative relation by Salmond and Hohfeld. As a result, the guidelines on the definition of norms (guidelines 1, 9 and 10), the workings of norms by constraining behavior (guideline 2), the relation to legal and social literature (guideline 6), and the use of norms in solving dilemmas and deontic paradoxes (guidelines 7 and 9) are well covered in the Calculemus-FLINT method. The same goes for making changes in sources of norms (guideline 3), which is a basic feature of normative systems in a legal or social perspective.

The guidelines on the broader role of norms in normative multiagent agent systems is, as yet, not fully covered in the Calculemus-FLINT method. Calculemus-FLINT was initially focused on the interpretation of sources of law, the definition of fundamental normative concepts, the recognition of non-compliancy and the possibility to reach shared interpretations of sources of norms. Right now we are combining Calculemus-FLINT with methods for describing agent-behavior in order to be able to goal setting and evaluate agent behavior while attempting to reach those goals. This will also enable us to model norm enforcement, including the potential for punishment of incompliant behavior, the creation of new sources of norms, the interpretation of sources of norms, and the formal regulation of dispute resolution.

The ten guidelines can be grouped in three categories: guidelines on the definition of norms (guidelines 1, 2, 6, 8, and 10), a guideline on changing norms in runtime (guideline 3), guidelines on interaction between agents and game-theoretical mechanisms (guidelines 4 and 5), and guidelines on the use of norms and the resolving of dilemma’s and paradoxes (guideline 7 and 9).

To improve the method we will comment on some individual guidelines and the order of the guidelines. Subsequently, we will suggest an adjusted list of guidelines, based on our experiences while using the original ones.

We believe guideline 4 (Discuss the use and role of norms as a mechanism in a game-theoretic setting) and guideline 5 (Clarify the role of norms in the multiagent system) can be merged. If there are reasons to keep these guidelines separated, we recommend changing the order because the discussing a mechanism in a gametheoretic setting, requires clarification of the role of norms in the multiagent system.

Guideline 9 on the use of deontic paradoxes to illustrate the working of NMAS is a specification of guideline 7 on the use of norms to resolve dilemmas and to regulate interaction among agents. We recommend combining both guidelines. It is unclear why a discussion on the regulative and constitutive aspects of norms (guideline 10) needs a separate guideline. But then, in our opinion about norms, every norm has both regulative and constitutive aspects.

Having applied the guidelines on the Calculemus-FLINT method, we propose some adjustments, resulting in the following guidelines:

1. Give a definition of the concept «norms» and introduce the language you use for the representation of normative relations in a normative multiagent system.
2. Introduce the method you use for the interpretation of sources of norms.
3. Introduce the language you use for representing action-based state transitions.
4. Introduce the method you use for dispute resolution.
5. Introduce the method you use for goal selection by agents.
6. Introduce the method you use for goal-path selection and planning
7. Introduce the method you use for making agents physically perform actions
8. Introduce the method you use for evaluating actions of goal seeking agents.
9. Introduce the method you use for changing sources of norms.
10. Compare your approach for making a normative multiagent system with those of others.

This version of the guidelines for making and using NMAS provides more general rules. Guidelines might also contain principles and pieces of advice. We suggest adding these aspects in explanatory notes accompanying the guidelines. We will use these guidelines in the development of a Calculemus-FLINT based NMAS and will report on our experiences.

Boella, Guido/Torre, Leendert van der/Verhagen, Harko. Introduction to NMAS. Computation and Mathematical Organizational Theory, Special Issue on NMAS, 12(2–3):71–79, 2006.

Boella, Guido/Pigozzi, Gabriella/Torre, Leendert van der. Normative Systems in Computer Science. Ten Guidelines for Normative Multiagent Systems. In Normative Multi-Agent Systems, Dagstuhl Seminar Proceedings 09121, 2009.

Chisholm, Roderick. Contrary-to-Duty Imperatives and Deontic Logic. In Analysis, 24:33–66, 1963.

Dickey, Anthony. Hohfeld’s Debt to Salmond. In UWALawRw 4 10(1), University of Western Australia Law Review, 10, 59–64, 1971

Dijkstra, Edsger. On the role of scientific thought. In Selected Writings on Computing: A Personal Perspective. New York, NY, USA: Springer-Verlag, pp. 60–66, 2016

Doesburg, Robert van/Storm, Tijs van der/Engers, Tom van, CALCULEMUS: Towards a Formal Language for the Interpretation of normative Systems, Artificial Intelligence for Justice Workshop, ECAI, 2016.

Doesburg, Robert van/Engers, Tom van. Using Formal Interpretations of Legal Sources for Comparing the Application of Exclusion Clauses of the UN Refugee Convention. In Jusletter IT (Feb. 2018), 175–184, 2018.

Doesburg, Robert van/Engers, Tom van. 2018. Arguments on the Interpretation of Sources of Law. In AI Approaches to the Complexity of Legal Systems. AICOL 2015, AICOL 2016, AICOL 2016, AICOL 2017, AICOL 2017. Lecture Notes in Computer Science, vol 10791. Springer, Cham, 487–492, 2018.

Doesburg, Robert van. The False, the Former, and the Parish Priest (extended version), Doctoral Consortium ICAIL 2019, 2019

Doesburg, Robert van/Engers, Tom van. Explicit Interpretation of the Dutch Aliens Act, In: Proceedings of the First AIAS 2019, Montréal, Canada, 2019.

Dymitruk, Maria/Markovich, Rèka/Liepiņa, Rūta et al. 2018. Research in Progress: Report on the ICAIL 2017 Doctoral Consortium. Artif. Intell. Law 26, 1 (March 2018), 49–97, 2018.

Hage, Jaap, Why Regulative Rules are Constitutive Rules, presented at the Rules 2013 conference in Krakow. 2013.

Hansen, Jörg/Pigozzi, Gabriella/Torre, Leendert van der. Ten Philosophical Problems in Deontic Logic. In NMAS, volume 07122 of Dagstuhl Seminar Proceedings. Internationales Begegnungs- und Forschungszentrum für Informatik (IBFI), Schloss Dagstuhl, Germany, 2007.

Hohfeld, Wesley. Some Fundamental Legal Conceptions as Applied in Judicial Reasoning. In Yale Law Journal 23(1), 16–59, 1913.

Jones, Andrew/Sergot, Marek. A Formal Characterisation of Institutionalised Power. Journal of IGPL 4, 3, 427–443, 1996.

Kanger, Stig/Kanger, Helle. «Rights and Parliamentarism», Theoria 32, 85–115, 1986

Lindahl, Lars. Position and Change: a Study in Law and Logic, Reidel Publishing Company, Dordrecht, Holland, 1977.

Makinson, David. On the Formal Representation of Rights Relations: Remarks on the Work of Stig Kanger and Lars Lindahl, Journal of Philosophical Logic, Vol. 15, No. 4, 403–425, 1986

Salmond, John. Jurisprudence or the theory of the law (1st. ed.). Stevens and Haynes, London, UK, 1902.

Searle, John. Speech Act: An Essay in the Philosophy of Language, Cambridge University Press, Cambridge, UK, 1969.

Searle, John, The Construction of Social Reality, The Free Press, New York, 1995.

Searle, John, Making the Social World, Oxford University Press, New York, 2010.

Sileno, Giovanni, Aligning Law and Action: A Conceptional and Computational Inquiry, Ph.D. Dissertation University of Amsterdam, SIKS Dissertation, 2016.

Weber, Max. Politik als Beruf. MWG Bd. 17, hrsg. von Wolfgang J. Mommsen, 1992.