BACKGROUND 

The European Union Artificial Intelligence Act (The EU AI Act) was proposed essentially with  the idea of laying down harmonised rules on artificial intelligence. Artificial Intelligence (AI) is a  quickly evolving branch of technology that can bring a wide array of economic and societal benefits  across the socio-economic spectrum. Improving prediction, optimising operations and resource  allocation and personalising service delivery, the use of artificial intelligence can support  beneficial social and environmental outcomes, providing key competitive advantages to  companies and the European economy. However, the same elements and techniques that power  the socio-economic benefits of AI can also bring about new risks or negative consequences for an  individual and in turn society at large. Considering the speed of technological change and possible  challenges, the EU committed to come up with a balanced approach.  

The European Council called for a ‘sense of urgency to address emerging trends’, in 2017,  including ‘issues such as artificial intelligence …, while at the same time ensuring a high level of  data protection, digital rights and ethical standards’ . In its 2019 “Conclusions on the Coordinated  Plan on the development and use of artificial intelligence”, the European Council highlighted the  necessity of ensuring that its citizens rights are fully respected, in turn calling for a review of the  existing relevant legislation in this field to make it more relevant with the new opportunities and  challenges raised by AI. It proposed further clarifications and determination of the AI  applications that should and would be considered high-risk. The most recent conclusions further  called for addressing the opacity, complexity, bias, degree of unpredictability and partially  autonomous behaviour of certain AI systems, to ensure that they align and are compatible with the  fundamental rights of citizens, and can also be brought under proper jurisdiction. 

OBJECTIVES OF THE PROPOSAL 

• ensuring that AI systems placed in the European Union market are used safely, respecting  the existing laws on fundamental rights and Union values 

• ensuring legal certainty in facilitating investment and innovation in AI and research related  to it 

• enhancing governance and effective enforcement of existing laws on fundamental rights  and the safety requirements that may be applicable to AI systems 

• facilitating the development of a singular market for lawful, safe and trustworthy AI  applications to prevent the fragmentation of major and minor markets.

To achieve these objectives the EU AI Act proposal also presented a balanced regulatory approach  to AI, that entails the minimum necessary requirements to address the risks and problems linked  to AI, without unnecessarily constraining or hindering technological advancements and research  in this field. It puts in place a proportionate regulatory system that revolves around a well-defined  risk-based regulatory approach that does not create unnecessary restrictions to trade, whereby legal  intervention is tailored to concrete situations, where there may be a justified cause for concern, or  where such concern can reasonably be anticipated soon. The legal framework at the same time  includes flexible mechanisms that enable it to be dynamically adapted as AI evolves and new  concerning situations emerge. The promotion of AI-driven innovation is also closely linked to the  Data Governance Act, the Open Data Directive and other initiatives under the EU strategy for data,  which will establish trusted mechanisms and services for reusing, sharing and the pooling of data,  which are essential for the development of data-driven AI models of high quality. 

STAKEHOLDER ANALYSIS  

According to the Brussels proposal on Regulation of the European Parliament and of the council,  titled “Laying down harmonised rules on artificial intelligence (Artificial Intelligence Act) and  amending certain union legislative acts” dated 21^st April 2021,  

“In total, 1215 contributions were received, of which 352 were from companies or business  organisations/associations, 406 from individuals (92%individuals from EU), 152 on behalf of  academic/research institutions, and 73 from public authorities. Civil society’s voices were  represented by 160 respondents (among which 9 consumers’ organisations, 129 non-governmental  organisations and 22 trade unions), 72 respondents contributed as ‘others’. Of the 352 business  and industry representatives, 222 were companies and business representatives, 41.5% of which  were micro, small and medium-sized enterprises. The rest were business associations. Overall,  84% of business and industry replies came from the EU-27. Depending on the question, between  81 and 598 of the respondents used the free text option to insert comments. Over 450 position  papers were submitted through the EU Survey website, either in addition to questionnaire answers  (over 400) or as stand-alone contributions”.  

Thus, it can be noted that the general stakeholders all collectively agree on a need for action  pertaining to the arbitrary future of AI and its advancements.  

NOTION OF BIOMETRIC DATA 

The notion of remote biometric identification system used in this Regulation should be defined  functionally. It is defined as “an AI system intended for the identification of natural persons  at a distance through the comparison of a person’s biometric data with the biometric data  contained in a reference database, and without prior knowledge whether the targeted person  will be present and can be identified, irrespectively of the particular technology, processes  or types of biometric data used.”  

A classification should be made between ‘real-time’ and ‘post’ remote biometric identification  systems. 

• In ‘real-time’ systems, the biometric data is captured, and the comparison and identification  all occur instantaneously, near-instantaneously or at most without a significant delay. In  this case, there should be no scope of finding a way around the rules of this Regulation on  the ‘real-time’ use of the AI systems in question by providing for minor delays. 

• In ‘post’ systems, the biometric data have already been captured and the comparison and  identification occur only after a significant delay. This involves data, which has been  generated before the use of the system in respect of the natural pupils concerned. 

The use of Real Time Biometric Systems for the purpose of law enforcement should be prohibited,  except in three exhaustively listed and narrowly defined situations: 

1. Search for potential victims of crime, including missing children 

2. Investigation regarding certain threats to the life or physical safety of natural persons or of  a terrorist attack 

3. Detection, localisation, identification or prosecution of perpetrators or suspects of the  criminal offences referred to in Council Framework Decision 2002/584/JHA.  Only after specific authorisation by a judicial authority or an independent administrative authority  of a Member State.  

NOTION OF PUBLICLY ACCESSIBLE SPACES  

The notion of publicly accessible spaces, referring to “any physical place that is accessible to  the public, irrespective of whether the place in question is privately or publicly owned.”  Hence the notion “does not cover places that are private in nature and normally not freely  accessible for third parties, including law enforcement authorities, unless those parties have been  specifically invited or authorised, such as homes, private clubs, offices, warehouses and factories.  Online spaces are not covered either, as they are not physical spaces.”  

Determination of whether a given space is accessible to the public should however be determined  on a case-by-case basis, having regard to the specificities of the individual situation at hand.  

NOTION REGARDING AI SYSTEMS FALLING UNDER THE REGULATION 

Considering the digital nature, particular AI systems should fall within the scope of this Regulation  even when they are : 

• not placed on the market 

• not put into service 

• not used in the Union 

This is the case where an operator established in the EU contracts certain services to an operator  established outside the EU, in relation to ‘high risk’ AI systems that may hold impact on the  European citizen. To prevent the circumvention of this Regulation and ensuring an effective  protection of natural persons located in the Union, “this Regulation should also apply to providers 

and users of AI systems that are established in a third country, to the extent the output produced  by those systems is used in the Union.”  

NOTION REGARDING HIGH-RISK AI SYSTEMS 

“High-risk" AI systems will include systems that pose a threat or unacceptable risks to Union  Public interests provided and protected by Union Law. 

These systems can only be placed in the Union market or into service if they comply with certain  mandatory requirements enlisted in the EU AI Act. However, AI systems identified as high-risk  should be limited only to those that have a significant harmful impact on the health, safety and  fundamental rights of persons in the Union, minimising any potential restriction to international  trade, if any. 

Following AI Systems will be considered high risk: 

• ‘Real-time’ and ‘post’ remote biometric identification systems should be classified as high risk considering the risks that they pose and should be subject to specific requirements on  “logging capabilities and human oversight.” 

• Systems managing and operating critical infrastructure 

• AI systems used in education or vocational training, for determining access or assigning  people to educational and vocational training institutions or to evaluate people on tests as  part of or as a precondition for their education. 

• AI systems used in employment, workers management and access to self-employment,  notably for the recruitment and selection of persons or for making decisions on promotion  and termination and for task allocation, monitoring or evaluation of persons in work-related  contractual relationships 

• AI systems used to evaluate the credit score or creditworthiness of an individual 

• AI systems provide access to and enjoyment of certain essential private and public  services and benefits necessary for people to fully participate in society or to improve one’s  standard of living in the Member States. 

• AI Systems designed to take actions by law enforcement authorities involving certain uses  of AI systems are characterised by “a significant degree of power imbalance” and may lead  to surveillance, arrest or deprivation of a natural person’s liberty as well as other adverse  impacts on fundamental rights guaranteed in the Charter. 

• AI systems used in migration, asylum and border control management. 

• AI systems intended for the administration of justice and democratic processes 

High quality training, validation and testing data sets require the implementation of appropriate  data governance and management practices. The training, validation and testing of data sets should  be relevant, representative and free of errors and complete in view of the intended purpose of the  system along with having the appropriate statistical properties. 

The stakeholders also agreed upon a risk-based framework as a better option than blanket  evaluation and regulation of all AI Systems. The types of risks and threats should be based on 

sector by sector and in turn case by case approach. The risks shall also be calculated after  considering the impact on human rights and safety of the EU citizen. 

NOTION REGARDING TRANSPARENCY OF HIGH-RISK AI SYSTEMS 

Having information on how high-risk AI systems have been developed and how they perform  throughout their lifecycle is essential. Thus, keeping records and proper technical documentation  becomes mandatory. Such information should include the “general characteristics, capabilities and  limitations of the system, algorithms, data, training, testing and validation processes used as well  as documentation on the relevant risk management system.” along with up-to-date technical  documentation. Users should be able to interpret the system output and use it appropriately.  Therefore, the AI Systems concerned should be accompanied by relevant documentation and  instructions of use and include concise and clear information (including in relation to possible risks  to fundamental rights and discrimination). Cybersecurity will play a crucial role in ensuring that  AI systems are resilient against attempts to alter their use, behaviour, performance or compromise  their security properties by malicious third parties exploiting the system’s vulnerabilities. 

The legal person, defined as the provider, who had taken the responsibility for the placing of the  AI System in the market or into service, regardless of whether that natural or legal person is the  person who designed or developed the system, will be held responsible for it. To ensure a high  level of trustworthiness of high-risk AI systems, those systems should be subject to a conformity  assessment prior to their placing on the market or putting into servic. It is also considered  appropriate that an AI system undergoes a new conformity assessment whenever a change occurs  which may affect the compliance of the system with this Regulation or when the intended purpose  of the system changes. Providers will be required to register their high-risk AI system in an EU  database, established and managed by the European Commission. 

High-risk AI systems should bear the CE marking to indicate their conformity with this Regulation  so that they can move freely within the internal market. However, under certain conditions, rapid  availability of innovative technologies may be crucial for health and safety of persons and for  society, thus making it appropriate that under “exceptional reasons of public security or protection  of life and health of natural persons and the protection of industrial and commercial property”,  Member States could authorise the placing on the market or putting into service of AI systems  which have not undergone a conformity assessment. 

NOTIONS REGARDING AI RESEARCH AND DEVELOPMENT 

To ensure a legal framework that is innovation and research friendly, future-proof and resilient to  disruption, competent national authorities from one or more Member States should establish  artificial intelligence regulatory sandboxes to facilitate the development and testing of innovative  AI systems under strict regulatory oversight before these systems are placed on the market or into  service. The objectives of the regulatory sandboxes should be “to foster AI innovation by  establishing a controlled experimentation and testing environment in the development and premarketing phase with a view to ensuring compliance of the innovative AI systems with this Regulation and other relevant Union and Member States legislation”. To promote and protect  innovation, it is important that the interests of small-scale providers and users of AI systems are  considered as well. 

NOTIONS REGARDING SETTING UP OF AN “EUROPEAN ARTIFICIAL INTELLIGENCE  BOARD” 

In order to facilitate a smooth, effective and harmonised implementation of this Regulation a  European Artificial Intelligence Board would be established and would be responsible for a  number of advisory tasks, including “issuing opinions, recommendations, advice or guidance on  matters related to the implementation of this Regulation, including on technical specifications or  existing standards regarding the requirements established in this Regulation and providing advice  to and assisting the Commission on specific questions related to artificial intelligence.” 

The Board shall be composed of the national supervisory authorities, represented by the head or  equivalent high-level official of that authority, and the European Data Protection Supervisor.  (Other national authorities may be invited to the meetings if the discussed issues are of relevance  to them) 

OTHER NOTIONS 

• Effective protection of rights and freedoms of individuals across the Union, the rules  established by this Regulation would apply to providers of AI systems in a non discriminatory manner, irrespective of whether they are established within the Union or in  a third country, and to users of AI systems established within the Union. 

• The Regulations apply to “Union institutions, offices, bodies and agencies”, providing AI  Systems to the citizens. (Excluding military purposes defined under the Act)

 • Research for legitimate reasons in relation to high risk AI systems should not be stifled by  the Regulation, However, “such research does not amount to use of the AI system in  human-machine relations that exposes natural persons to harm and such research is carried  out in accordance with recognised ethical standards for scientific research.” 

• AI systems related to products that are not high-risk in accordance with this Regulation  will not be required to comply with the requirements set out under the act and are nevertheless  safe when placed on the market or into service. 

GENERAL PROVISIONS 

The Regulation mentioned in the EU AI Act apply to:

• Providers of the AI systems in the Union (irrespective of whether those providers are  established within the Union or in a third country) 

• Users of these AI Systems based within the Union 

• providers and users of AI systems that are in a third country, where the output produced by  the system is used in the Union 

The Regulations however do not apply to: 

• AI systems developed or used exclusively for military purposes by the Union  • public authorities in a third country nor to international organisations falling within the  scope of the EU AI Act 

PROHIBITED ARTIFICIAL INTELLIGENCE PRACTICES 

The prohibited practices pertaining to AI will remain prohibited under the Act: 

• Putting into service AI Systems that deploy subliminal techniques beyond an individual's  consciousness, that may in turn cause physical or psychological harm to the concerned  person 

• AI Systems that segregate and exploit the vulnerabilities of an individual or community of  people based on their differences. 

• Unfavourable treatment of certain people or groups in social contexts which are unrelated  to the contexts in which the data was originally generated or collected by the AI Systems • the use of ‘real-time’ remote biometric identification systems in publicly accessible spaces  for the purpose of law enforcement (unless otherwise excluded in the EU AI Act)