Confidence-based safety properties in CAISAR H/F

Job description: Through the recent developments of AI, the use of models produced by machine learning has become widespread, even in industrial settings. However, studies are flourishing showing the dangers that such models can bring, in terms of safety, privacy or even fairness. We can for example cite the adversarial attacks, small perturbations invisible to naked eyes which can drastically change the output of our AI. To face these dangers and improve trust in AI, research works are constantly emerging, resulting into several tools like ERAN (Gehr et al. 2018) or Marabou (Katz et al. 2019). In the past couple of years, the AISER team has developed CAISAR (Alberti et al. 2022), an open-source platform that focuses on the specification and verification of AI systems’ robustness and safety. In particular, CAISAR provides a language for formally specifying AI properties, and integrates various state-of-the-art tools for establishing their validity in a guaranteed way. The concept of a confidence-based safety property has been recently introduced by Athavale et al. (Athavale et al. 2024) to recast robustness and fairness properties in terms of the confidence score with which a neural network generates its outcomes. The main objective of this internship will be to investigate, design, and implement a support for confidence-based safety properties in the CAISAR, the AISER’s open-source platform for characterizing AI systems’ safety and robustness. The broad internship goals are: • familiarization with the state-of-the-art on formal approaches to properties for AI safety (Casadio et al. 2022) • familiarization with the work on confidence-based safety properties (Athavale et al. 2024) • getting started with the CAISAR platform • design and implementation of confidence-based safety properties in CAISAR • identification and evaluation against benchmarks

Institute description: The French Alternative Energies and Atomic Energy Commission (CEA) is a key player in research, development, and innovation. Drawing on the widely acknowledged expertise gained by its 16,000 staff spanned over 9 research centers with a budget of 4.1 billion Euros, CEA actively participates in more than 400 European collaborative projects with numerous academic (notably as a member of Paris-Saclay University) and industrial partners. Within the CEA Technological Research Division, the CEA List institute addresses the challenges coming from smart digital systems. Among other activities, CEA List’s Software Safety and Security Laboratory (LSL) research teams design and implement automated analysis in order to make software systems more trustworthy, to exhaustively detect their vulnerabilities, to guarantee conformity to their specifications, and to accelerate their certification. The lab recently extended its activities on the topic of AI trustworthiness and gave birth to a new research group: AISER (Artificial Intelligence Safety, Explainability and Robustness).

Your profile: The candidate will work at the crossroads of formal verification and artificial intelligence. As it is not realistic to be expert in both fields, we encourage candidates that do not meet the full qualification requirements to apply nonetheless. We strive to provide an inclusive and enjoyable workplace. We are aware of discriminations based on gender (especially prevalent on our fields), race or disability, we are doing our best to fight them. One of our team member is formally trained against psychological harassment and sexual abuse. Minimal requirements master student or equivalent (2nd/3rd engineering school year) in computer science some knowledge of the OCaml programming language (or any other functional language) some knowledge of the Python programming language ability to work in a team Preferred requirements notions in formal methods (i.e. abstract interpretation, SAT/SMT solving, etc.) notions in machine learning and, in particular, neural networks

Level of qualification studied: Bac+5 - Master 2

Languages: English Fluent,French Fluent