Jusletter IT

Modern facial recognition technologies are based on machine learning. The basis for the development of all facial recognition technologies are corresponding data sets. The selection of appropriate data sets is a core problem. Although a plethora of datasets are available on the Internet, not every one of them is equally well suited for this purpose. In addition to practical and technical considerations, legal and ethical considerations are required to avoid liability. The recent decisions of various data protection authorities on FRT¹ were a stark reminder, that compliant data use is of utmost importance.²

However, especially in the area of research on facial recognition technologies, there are still many legal “grey areas” – issues, that have not yet been resolved and the wording in the law that is open for interpretation. This contribution gives an overview of the legal issues with regard to the interpretation of existing law and mainly on the European General Data Protection Regulation (hereinafter: “GDPR”³).⁴

The first central legal issue that developers of FRTs are facing when choosing an image data set, is the question of the applicable (data protection) law.

While the GDPR – as an EU “Regulation” that is directly applicable in all Member States⁵ – has largely harmonized the rules of data protection law in the EU, there are still many areas of data protection law that remain within the competence of the member states. Even though the GDPR is meant to be a “general” regulation, the legislator intentionally included such “opening clauses” to account for the different (often historically founded) approaches that existed in the national member state law.

Seeing as the GDPR provides such “opening clauses” in the area of research as well, national law must be taken into consideration. If scientific research is conducted in an international consortium, consisting of various legal entities in different member states, the first question that arises is which national law should be applied. It is also conceivable that several national data protection provisions potentially apply.

Complexity increases if research is not just confined to EU member states but also includes institutions from outside the EU, which only partially fall within the territorial scope of the GDPR.⁶

While this might seem a prevalent issue, the literature regarding the question of the applicable “national” data protection law – even within the scope of the GDPR – is limited.⁷ There are two different approaches in literature to determine which of two (or many) different national data protection rules apply:

According to one approach, the determination is made on the basis of “private international law”, while the other approach draws an analogy to the jurisdiction rules of the GDPR.⁸ Both approaches can lead to the applicability of several national provisions.

In view of the current case law of the Austrian data protection authority, this ambivalence of applicable laws could lead to territorially applied “research privileges”⁹ can only be applied territorially.¹⁰

Since the development of FRTs within the context of research is mostly conducted not by a single legal entity, but by various different entities, questions regarding the “role allocation” arise. Whenever processing of personal data is conducted, it is important to qualify each entity’s “role” under the data protection regime, and especially determine the controller of a processing activity.

Possible “roles”¹¹ an entity could take on during the processing of personal data include the controller¹², the processor¹³, the data subject¹⁴ and – if none of these apply – the “third party”¹⁵ . The most important part of the role allocation is of course the determination of the controller, since the controller is the main addressee of the obligations in the GDPR.

It should be stressed, that the controller must be determined for each processing activity, which requires that these processing activities be determined beforehand. The collection of data to train a model for facial recognition is part of one processing activity, while the actual training (and testing) of the model are different processing activities. However, this separation is not always carried out strictly. The fact that the assessment of the data protection role is not carried out for every specific processing activity is arguably also due in part to the case law of the European Court of Justice (hereinafter: “ECJ”).

In particular, the (landmark) decision of the ECJ in the case of “Wirtschaftsakademie Schleswig Holstein”¹⁶ has led many scholars to apply the institute of “joint controllership” far more broadly than before that decision, and to include entities as “joint controllers” that would simply have been considered “third parties” (in the sense that they are not part of that processing activity, and therefore no obligations under the GDPR apply).

To reiterate: a controller under Art. 4(7) GDPR is the person/entity, “which, alone or jointly with others, determines the purposes and means of the processing of personal data; [...]”¹⁷

In short, the ECJ qualified the user of a Facebook fanpage as joint controller with Facebook with regard to certain processing activities (including the placing of “cookies” on visitors of the fanpage). The ECJ had to determine to which extent an administrator of a fan page (hosted on Facebook) determines, jointly with Facebook, the purposes and means of processing the personal data that is conducted. That processing is concerned personal data about the visitors to the fan page.¹⁸

The ECJ considered that 1) there was a contract between the fanpage administrator and Facebook and 2) that the intention for placing of cookies and the processing of personal data was primarily to enable Facebook to improve its system of advertising transmitted via its network but also to enable the fan page administrator to obtain statistics produced by Facebook from the visits to the page for the purposes of managing the promotion of its activity.

As a result of the decision, the question arose as to whether someone should be qualified as a controller – albeit “jointly” with another entity – based on the fact that that “controller” “profits” or “makes use” of a processing activity and also at least partially supports that processing activity.¹⁹ In other words: would that be enough to consider this entity as “determining purposes and means” of the processing?

This is an important issue for the development of FRTs as well, seeing as it could be argued that the developer “makes use” of already collected and published image data sets and should therefore be considered to be a controller with regard to that reasoning of the ECJ in the Wirtschaftsakademie Schleswig-Holstein case.

However, as a keen observer may notice, in contrast to that decision, if the initial collection of image data has been conducted already (by a different entity), without encouragement or even any involvement or knowledge by the developer, then it seems unfitting to consider the developer a “joint controller”.

This argument is supported by the subsequent decisions of the ECJ in the cases Fashion ID (C40/17) and Jehovan todistajat (C25/17), where in both cases the ECJ based its qualification of joint controllership on supporting activities with regard to the processing activity.²⁰ In addition, in the Fashion ID case the ECJ pointed out that the qualification as a controller can only be done for a specific processing activity.

So, if the determination of the controller (and the allocation of roles) is conducted not for the overall development process, but separately for each processing activity – and if the reasoning of the ECJ is applied correctly – the collection of an already existing image data set does not make the developer a joint controller with the controller of the initial collection of said data. This might seem an obvious conclusion, but considering the examples given in the more recent guidelines of the European Data Protection Board (hereinafter “EDPB”), it has to be pointed out that for a joint controllership to be considered, there has to be at least some link between both entities, with both of them having (at least) supporting roles. The subsequent use of an already collected image data set cannot lead to a person/entity being retroactively qualified as a joint controller.

However, even if the use of a publicly available image data set to develop FRT does – in general – not lead to a joint controllership with regard to the collection of that images, this does not mean, that the manner in which the initial collection took place may not play an important role when determining the lawfulness of use of that personal data.

This section raises certain relevant legal questions with regard to the lawfulness of the processing of training data sets under the GDPR. This means that this section is a “snap shot” of two specific legal issues that are associated with the training of FRT.

The usual approach in developing FRT is to use existing face image data sets to train the AI. This means that the developers of FRT do not collect face image data themselves, but rather use existing training data sets as well as evaluation data sets. These data sets are generally made freely available to the scientific community.²¹

The question arises, whether or not these training data sets can be used in a lawful manner (i.e. in compliance with the GDPR) if these data sets have been collected outside the European Union and perhaps even under legislation that has a lower standard of data protection than EU legislation. This question becomes all the more pressing, if the collection of the data set was apparently unlawful (either in view of the GDPR or in view of the legislative framework that was applicable to the initial collection).

Even though this question appears to be extremely important for assessing the lawfulness of the use of these training data sets, there is little literature on the question whether or not the initial collection would have to be “lawful” under the GDPR. If that was the case, the processing of data that have been collected “unlawfully” under the GDPR would be unlawful itself.

Although this issue has not yet been discussed before the ECJ, two decisions on a national level by the respective Supervisory Authority of a member state have come across the question how the lawfulness or unlawfulness of the initial collection of data can impact the further processing of that data and came to very different conclusions:

The Belgian Data Protection Authority (hereinafter “DPA”), for example, has more recently ruled against the use of personal data as evidence in civil law proceedings, if these data have been obtained without a legal basis under Art. 6 GDPR.²² In this case further processing of personal data, even in the context of legal proceedings, was deemed unlawful and was therefore prohibited.

While some practices of obtaining data used as evidence are dubious, in this case it appears to have been a simple mistake. The first defendant sent an e-mail to a third person as a result from his habit of sending e-mails to both the plaintiff and the second defendant, whereas he could have sent an e-mail to both the plaintiff and the second defendant concerning their notary practice and a separate e-mail to the plaintiff only concerning her personal company.²³ The DPA argued that even if unintentional, the transfer of personal data constituted processing of personal data and since the first defendant could have sent a separate e-mail, not including the second defendant, he did not adhere to the data minimisation principle and therefore the processing activity was deemed unlawful.²⁴

The second defendant, who was the recipient of these e-mails, then went on to send these to his legal counsel. The DPA neither accepted the argument that the communication was privileged as attorney-client correspondence, nor did it deem the further processing of the data in pending or future legal proceedings possible without violation of the principles of lawfulness, loyalty and transparency.

While this decision of a national DPA cannot represent the EU as a whole, the opinion of a national DPA can still have an EU-wide influence if adopted by the EDPB. Nevertheless, at the current state it is still a singular decision that is contrasted to different national case law of the member states.

Another example would be the decision of the Maltese court that, in a similar case, stated that the “fruit of the poisonous tree” doctrine adopted by the US jurisprudence according to which unlawfully obtained evidence was rendered inadmissible in court proceedings, was alien to Maltese law.²⁵ More importantly the Maltese Court stated, that data protection law, even under the GDPR, does not create or provide for such an “exclusionary rule” in case of illegally obtained evidence.²⁶ Also, the Maltese Court referenced the European Court of Human Rights’ case law according to which the use of illegally obtained evidence is not in breach of the right to a fair trial (Art. 6 ECHR).

The GDPR does in fact, not explicitly state, that in order to process data lawfully, the controller must ensure that the processed data are (initially) obtained lawfully (e.g., in a manner compliant with the GDPR). This could indicate that this is not a requirement for lawful processing of personal data, i.e. that it could also be lawful, to (further) process data that has initially been obtained unlawfully. This would also fit in with the fact that the legal grounds for processing data in Art. 6 (1) GDPR refer to the current processing activity (and not the previous processing activities) as well as the fact that the existence of Art. 14 GDPR indicates, that personal data must not always be obtained by the data subject itself.²⁷

In addition, if we consider the legal bases under Art. 6 (1) GDPR, the further processing of personal data (that initially have been unlawfully obtained) could not be based on consent given by the data subject²⁸ or the performance of a contract with the data subject²⁹, seeing as this concerns the initial collection of the data and not the further processing. It could nevertheless be possible, that the national law contains a legal obligation³⁰ to process personal data even if they have been initially obtained unlawfully.³¹ It could be argued that such a provision (or its interpretation in that manner) would not fulfil the requirement of Art. 6 (3) GDPR, according to which legal obligations in the sense of Art. 6 (1) (c) and (e) GDPR shall meet an objective of public interest and be proportionate to the legitimate aim pursued, but – according to the authors – such a strict interpretation would impose a serious restriction for national legislators and would be in stark contrast to Art. 23 GDPR.

With regard to these considerations, it could be argued that the GDPR requires to determine the lawfulness of the processing for each processing activity separately, without consideration whether these personal data have been collected or in other way processed lawfully before that point.

However, the fact that the GDPR does not explicitly state the requirement of lawfulness in all stages of processing personal data, could also be an indication that it is just implicitly taken for granted that the initial collection of personal data is required to be lawful under the GDPR. As a matter of fact, the GDPR does require that personal data must be “processed lawfully, fairly and in a transparent manner in relation to the data subject (‘lawfulness, fairness and transparency’)”.³² This seems to indicate a general requirement of lawful processing of personal data throughout their “life-cycle” (i.e. from obtaining the data through every processing activity until its deletion).

Also, the fact that personal data have been unlawfully obtained would have to be considered within the balancing of interests under Art. 6 (1) (f) GDPR. In that regard, the fact that personal data have been illegitimately obtained would have to be taken into account and would most likely shift the balance considerably against the further processing of such data by another controller.

It appears that the right answer is somewhere in the middle ground between these doctrinal approaches, one being that processing data that is not collected in accordance with the GDPR to always be unlawful and the other one disregarding the factor altogether. This is elaborated in the conclusions.

There are different approaches in the literature regarding the nature of Art. 6 (4) GDPR. While some argue that it is a separate legal basis for processing, others argue that Art. 6 (4) GDPR contains additional requirements for “further processing” of personal data, i.e. the processing of personal data that has already been collected for a specific purpose for another (“further”) purpose.³³

While an elaboration on the discussion in the literature would go too far, we can safely assume that also in this case, the right answer is somewhere in the middle ground between these doctrinal approaches:

While it would seem unconvincing to consider processing of personal data “lawful”, based on the single fact that it is “compatible” with the (perhaps unlawful) purpose of its collection, it also would be strange to have an additional condition on further processing of already collected data that goes beyond the requirements of a new collection of the very same data.

As has been discussed, the use of already existing image datasets to develop facial recognition technologies (FRTs) based on machine learning can entail various rather complex legal questions with regard to data protection, that are discussed in literature still only to a very limited degree.

Complex issues arise with regard to the applicable national data protection law, which is of importance where different entities work together in the development stage (e.g. in an international context, but also within the scope of the GDPR, if the processing falls under one of the “opening clauses”).

A key issue for any compliance evaluation under the GDPR is the “role allocation”. This contribution has elaborated why the use of an already existing image data set by a developer does – in general³⁴ and by itself – not lead to a “joint controllership”, since even under the current case law of the ECJ, a retroactive controllership cannot be constructed, if there is no indication of an entity to support the (initial) processing activity in any way.

However, a developer should take reasonable steps to ensure that the personal data of an image data set is not collected in an unlawful manner. According to the authors, the developer should use an image data set only if that image data set has not been obviously collected in contradiction to the rules of the GDPR. This requires a consideration of relevant factors of the initial collection and the publication (e.g. the initial collector as an entity, the purposes of the initial collection, the manner in which it was collected, the restriction of access to the data set, etc.).

While the fact that personal data has been collected unlawfully does not prohibit its use per se, the fact that personal data that is part of an image data set that contains images which were not collected in accordance with the GDPR should be considered when balancing either public interests (e.g Art. 6 (1) (e) GDPR) or (private) legitimate interests (Art. 6 (1) (f) GDPR) with the rights of the data subject. The aforementioned reasonable steps for the developer to take are to ensure they– to the best of their knowledge – choose an image data set that has been collected in accordance with the GDPR.

This work has been partially supported by the European CHIST-ERA program via the Austrian Research Promotion Agency (FFG) within the XAIface project (grant agreement CHIST-ERA-19-XAI-011).³⁵