Junior Investigator - 3 year fellowship - Innovate New Processing Paradigms, Data Intelligence, Methodologies, and Tools for Frugal Digital

🎓 Qualification : PhD • 🤝 Employment type : 3 year contract • 📍 Location : Paris area, France

CEA-List is seeking an accomplished junior investigator to contribute novel research ideas in processing paradigms, data intelligence, methodologies, and tools to make technology more frugal and sustainable. The winner of this three-year Science Impulse fellowship will manage their own groundbreaking research project and have an opportunity to work with CEA research teams.

Background

Over the past five decades, digital technology has seen unprecedented efficiency gains. But due to exponential growth in the use of technology, these gains have not reduced overall energy consumption. Novel approaches will be required to respond to the urgent need for more sustainable, frugal technologies. In this challenge lies a wide array of research opportunities spanning architectures, algorithms, and applications. Successful research must address these topics holistically and optimize all levels of the technology stack.

Scope and applications

The proposed research may address one or a combination of the following topics and/or their applications, and proposals on related topics will also be considered:

1.Performing complex tasks with data.Deep learning can be used to build efficient machine learning models, but with high data and computing budgets. Transfer learning techniques leverage the relatedness of tasks to avoid full, complex-model training for new tasks, but there is still much room for improvement. This will require breakthroughs in:

• Formalizing and quantifying task relatedness
• Measuring the tradeoff between discriminating efficiency and transferability of induced representations
• Understanding the relationship between adversarial robustness and transferability
• Analyzing how knowledge is distributed across deep model components and how this can be used to optimize learning and transfer
• Characterizing the ability of a deep model to unfold the underlying data manifold and replacing some components of the model by unlearned transforms based on geometrical insights
  
  

2. Investigating smart frugal solutions that do not necessarily involve learning,like parsimonious models, lean models, and other approaches such as Bayesian, hyper dimensionality, or self-supervised learning. Continuous learning and updating without full relearning and dynamic adaptation and detection of new events and classes could also be addressed. Specific topics could include:

• Understanding “winning ticket” identification (subnetworks of complex dense models than can be trained to full accuracy according to the lottery ticket hypothesis) and transferability of winning tickets for different downstream tasks des billets de loterie)et la transférabilité des billets gagnants pour différentes tâches en aval
• Investigating data-free or one-shot model pruning based on network structural properties
• Analyzing deep links between model pruning and network architecture search
• Disentangling “innate” architecture-related network capability for a given task and “acquired” training-related knowledge
• Going beyond gradient descent-based model optimization
• Co-designing hardware and sparse models
• Studying model pruning transferability across different classes of models
• Developing multi-mode aggregation of different input modalities

3. Methods and tools for frugal digital engineering.Eco-design has been identified as one of the main ways to contain the digital world’s environmental impacts. And the earlier we intervene (i.e. during the design phase), the greater the potential gains across the digital service lifecycle. The topics addressed could include:

• Holistic, systemic eco-design approaches to reduce the IT resources (servers, network, terminals, etc.) needed for a given functional unit (digital service)
• Design methods and tools to support these approaches, including collaborative lifecycle analysis (LCA) methods and tools that hardware, software, and infrastructure experts can use to reduce environmental impacts across the entire digital service lifecycle
  
  

Specific applications of the topics mentioned above in non-destructive testing (NDT), structural health monitoring (SHM), computer vision, and natural language processing (NLP) could also be addressed by the research.

In terms of NDT and, especially, SHM, a variety of industries from energy (wind turbine and nuclear power plant monitoring) to transportation (rail, aeronautics, maritime) to civil engineering (bridges) are hungry for SHM solutions capable of estimating the remaining useful life of an asset and facilitating long-term maintenance planning with its significant financial and safety benefits. In terms of environmental performance, a CEA-List study showed that the carbon balance of SHM implementation is positive, even when it lengthens asset lifetimes very little. Artificial intelligence has the capacity to make SHM even more powerful. Because the sensors and other devices used in SHM systems are usually battery-powered, frugal AI will be crucial. A holistic and eco-design approach to both AI algorithms and hardware will be needed to design a complete viable SHM system based on AI.

Applications in computer vision and NLP could address potential alternatives to the current paradigm in which powerful models are produced by a few giants and adapted for specific tasks by users—at the cost of huge data and computing budgets. Here, research could address solutions to reduce model complexity, limit the amount of training data needed, and maintain the level of performance required for the target application, from exploiting foundation models (controlling bias, reducing complexity) to developing alternative frugal-by-design architectures. More disruptive approaches based on alternatives to deep learning will also be considered.

Job description

CEA-List is seeking an accomplished junior investigator to contribute to the study and development of new and innovative solutions in the field of efficient and frugal processing. The candidate will have opportunities to utilize previous CEA-List hardware and software developments as well as resources like the SACHEMS SHM implementation and testing platform at CEA-List.

As principal investigator, the candidate will be in charge of the project and must be capable of working in a very multidisciplinary environment. In addition, a holistic and, therefore, multidisciplinary approach to the proposed topic or topics (see above in scope and applications) will be expected. The end goal is to achieve acceptable tradeoffs between efficient processing and frugality through research that is a natural fit for the programs currently underway at CEA-List.

Biblio references

1. Fefferman, C., Mitter, S., & Narayanan, H. (2016). Testing the manifold hypothesis. Journal of the American Mathematical Society, 29(4), 983-1049.
2. Frankle, Jonathan, and Michael Carbin. « The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. » International Conference on Learning Representations. 2018.
3. Tanaka, H., Kunin, D., Yamins, D. L., & Ganguli, S. (2020). Pruning neural networks without any data by iteratively conserving synaptic flow. Advances in Neural Information Processing Systems, 33, 6377-6389.
4. Gaier, A., & Ha, D. (2019). Weight agnostic neural networks. Advances in neural information processing systems, 32.
5. BORDAGE, Frédéric. The environmental footprint of the digital world. Report for GreenIT. fr, 2019.Aujoux, C; Mesnil, O., « Environmental impact assessment of guided wave based structural health monitoring » Structural Health Monitoring, SAGE Publications Sage UK: London, England, 2022
6. Aujoux, C; Mesnil, O., « Environmental impact assessment of guided wave based structural health monitoring » Structural Health Monitoring, SAGE Publications Sage UK: London, England, 2022
7. http://www-list.cea.fr/en/media/news/2020/471-october-2-2020-sachems-shm-development-and-testing-platform