🇪🇺 EU AI Act: Comments on the Third Code of Practice Draft 🇪🇺
Yesterday marked the publication of the third draft of the EU Code of Practice for GPAI developers. It’s been a long road here with amazing work from all chairs, and there’s still a way to go! So let’s take stock of the current state of the draft, the most recent development, and from a Hugging Face perspective: how it’s shaping up in terms of open and collaborative development and governance of AI systems!
Let’s start with the short version: some promising updates, but choices to enshrine two of the most problematic categories of systemic risk while diminishing transparency requirements overall are concerning for smaller actors across categories of stakeholders (including but not limited to developers), and seem to run counter to the code of practice’s ambition to remain future-proof and support responsible development of and with GPAI systems.
GPAI models with systemic risk (GPAISR)
Many of the updates in the systemic risk section reflect substantial work going in a positive direction. We particularly appreciate the effort put into enabling more process transparency for release decisions (Measure II.1.2 on risk acceptance criteria), defining evidence criteria with strong scientific grounding that include consideration of past incidents (Measure II.4.1), avoiding duplication of effort for derived models (II.4.2), and a greater focus on model-independent information that recognizes the role of broad design decision, especially training data, in shaping risks (II.4.3). Increased guidance on what constitutes best efforts for assessing models as part of a system – with due consideration for the capacity of SMEs and pointing to a supporting role for the AI Office – is also helpful (II.4.7). Most importantly, the introduction of Appendix 2 paves the way for regular updates of the code of practice. This process is essential in our view, as it should allow companies to focus on risks that present more immediate concerns, rather than try to predict the state of the technology years in the future after what will likely be significant changes in algorithms, deployment, hardware, and global data flows.
Unfortunately, the selected types of systemic risks in Appendix 1.1, which we had already expressed our concerns about in the first draft, still represent a fundamental issue and are set to exclude smaller actors and researchers from participating in the development of the technology. The current version requires developers to systematically assess risks related to cyber offense, CBRN, “harmful manipulation”, and “loss of control”. Of the four, cyber offence is by far the most concrete, and corresponds to the kind of systemic risks that can be measured and mitigated, with specific plausible harm scenarios to guide both. While the cost of doing so might initially present a barrier for smaller developers, we know from experience that collaboration and focused efforts on efficiency can bring those costs down to manageable levels. In contrast, including the other three in the selected risks at this point presents open developers with an impossible challenge. As per a recent study on biorisks: “current LLMs and BTs do not pose an immediate risk, and more work is needed to develop rigorous approaches to understanding how future models could increase biorisks”. For “harmful manipulation”, not only is there currently no convincing evidence that state-of-the-art model performance plays a major role in phenomena like platform-enabled misinformation, the definitions of both “harmful” and “manipulation” are also currently left to the discretion of the developers; whose commercial interests may at times be at odds with public health. Finally, “loss of control” as defined is still, to the best of our knowledge, closer to a science-fiction scenario than to a hazard with any level of plausibility.[1] In practice, this means that there is no way to prove meaningful due diligence for those risks following an established scientific consensus. While well-resourced developers with less transparent development and evaluation practices may be able to produce results that are somewhat thematically related to the high-level concerns captured in the three phrases describing those risks, smaller and open developers who need to rely on collaborative and scientifically grounded approaches cannot.[2]
One argument put forward in the FAQ to assuage concerns about the negative impact of these requirements on open and collaborative development is the expectation that the GPAISR classification should only apply to “5-15 companies”, with the intention to facilitate a lighter process for any newcomers that might be less well-resourced. We do not share this perspective, as we are increasingly seeing smaller and more focused teams developing systems with state-of-the-art performance, including by leveraging open artifacts and focusing efforts on different parts of the development stack; and while we welcome the reference to proportionate requirements for smaller actors, we note that the current text mostly uses the term in reference to the risk, not the developer type. This raises concerns both for the risk of furthering concentration of the technology and for the sustainability of its governance. Not only do these open systems support broader innovation and economic impact, as has been well documented for open source software, the more transparent ones also play an important role in enabling robust and verifiable research, including on the very risks under consideration in this document.
The discouragement of sharing evaluations and related research data in Measure II.4.10, despite the proposed recital in II.4.5 stating its importance, is also a disappointing shift. In contrast to the previous draft, which explicitly encouraged non-SMEs to share such information, this version backtracks. A strong, open exchange of evaluation results and data plays an indispensable role in improving safety across the field and fostering a robust evaluation ecosystem; and is particularly critical to addressing the concerns outlined above on the definition of risks outside of an established scientific consensus.
Transparency and Copyright
General transparency commitments are also severely diminished compared to the last version of the Code. For open-source models without systemic risks, we have argued that some of the requirements were redundant while potentially adding an additional administrative burden, a concern which is addressed in this draft by exempting them from using the proposed format. For GPAIs that do not meet this exemption, information about the composition of a model’s training data no longer needs to be disclosed to downstream providers. This disclosure would have been valuable to e.g. evaluate systems in a new setting without fear of data contamination, or to GPAI developers that do want to document their training data but might fear disproportionate legal scrutiny without a level playing field; thankfully, the information may still be provided by way of the “sufficiently detailed training data summary” template currently developed by the AI Office. It is more concerning however to see that the disclosure of energy information, measures taken to address personal or prohibited data, and specific evaluations – all of which have a role to play in enabling downstream providers to know in which settings a model may safely be deployed and what additional evaluations or risk mitigation strategies need to be applied – has been limited to National Competent Authorities. Overall, a stronger commitment to public disclosure, even within the already narrow set of transparency categories, would do far more to support public safety and governance, innovation, and fair competition.
The copyright measures on the other hand seem to be moving in a more promising direction, offering some useful indications on how they could be technically implemented. We appreciate in particular that measures 1.2.2 and 1.2.3-(1)(b) point to projected EU resources and processes for defining what constitutes widely accepted opt-out mechanisms. Smaller developers benefit significantly from clear guidelines and requirements that can be followed without the support of significant legal resources and without undue risk of involuntary non-compliance. Still, some aspects of this section present a disproportionate burden for open developers. The drawing up of a copyright policy to “comply with Union law on copyright and related rights” presents a unique challenge to organizations who choose to release artifacts for open-ended uses spanning different jurisdictions. The proportionality aspect is also less clear than in the second version. While the previous draft explicitly included SME exemptions, this version fails to provide sufficient clarity on when and where SMEs would be exempt, leaving them in a precarious legal position.
Overall, the balance of this proposal feels misaligned with the impact and maturity of the topics addressed in the different sections. Transparency and copyright issues are immediate and can be meaningfully shaped based on our current understanding of AI technology. Their impact will be widespread, not only affecting most models but also shaping the practices of major developers, many of whom already serve multiple versions of models they do not classify as “frontier” - and would likely argue are not GPAISR. These are the areas where clear, enforceable commitments could have the most meaningful societal impact; yet they seem severely deprioritized in the latest draft, which we strongly hope will be remedied in the next.
[1] The work cited in the FAQ to support those concerns is primarily developed by commercial entities whose success depends on the perceived performance of the models, shows strong anthropomorphization bias in its framing, and should be understood to be of marginal value at best given the lack of transparency on the systems tested, especially in terms of training data.
[2] Note however that minimally moving these from the selected types of systemic risks (Appendix 1.1) to the “other types” list (Appendix 1.2) in the next draft, at least until the next review in 2 years, would address some of these issues while allowing larger developers with particular concerns to keep investing in this research.