Data, Computing and Digital Research Infrastructures

Gaps and Needs in the Domain

 

While there has been a number of successful initiatives linking RIs and e-infrastructures, there is still significant potential to enhance and intensify the collaboration. Sharing competencies, tools and software development efforts, for example, can significantly reduce overlapping work. Wider utilisation of European joint resources, such as those provided by the EuroHPC centres and other European or national centres, results in cost efficiency and potential to scale the application performance much higher. Data in various forms – also as capability to analyse and store it – is a key asset for the future, which can be addressed through the European RI and e-infrastructure collaboration.

Today, computing systems have passed the exaflop milestone. However, prioritising comprehension of results holds greater significance than mere computational speed. For that reason, the focus should be put increasingly in competence building, for example in application development and scaling/porting work of scientific applications. In many areas, even if the supercomputer would calculate fast in theory, only a fraction of this power can be utilised due to limitations in methodology or scaling. This trend of processors advancing in speed while applications lag behind underscores the necessity for competent experts and an enhanced education system.

There are also huge opportunities in establishing a closer interaction between EuroHPC and EOSC. They share stakeholders and customer base, e.g. European research and industry. Indeed, all elements of this ecosystem – HPC, AI, data, networks, applications, competence etc. – interact and link together.

The success of the future EOSC as a European Data Space for Science, Research and Innovation does not only depend on the possibility to access valuable data types, including research data, scientific publications and other research outputs; it also hinges on providing researchers with the most advanced computing and data management resources at scale to process and analyse these data, allowing them to make new discoveries and insights.

While HPC and advanced computing resources have traditionally been allocated through scientific peer-review, there’s a need to actively promote the introduction of new, open, and flexible allocation methods to cater to a wider set of users. These methods should encompass diverse usage scenarios, ranging from high-throughput to high-performance computing, from virtual machines running on a few cores to more demanding operations running on a larger number of servers. Furthermore, these methods should support Machine Learning and AI use cases on state-of-the-art accelerator devices.

 

TURNING HPC AND AI TO A COMPETITIVE ADVANTAGE FOR EUROPE

HPC and AI are poised to become the two cornerstones of the next Digital Europe Program (DEP). Each has seen substantial advancements as independent fields in recent years. Increased investments in both areas hold the promise of enhancing Europe’s competitiveness in the global digital economy and bolstering its technological autonomy. However, the real breakthrough is likely to come from the convergence and the joint power of these two domains: Europe needs to act in a coordinated way to harness the full potential of this revolution.

In order to make the most of these assets, it is important to adopt a holistic approach supporting the convergent use of HPC and AI infrastructures. This requires additional interaction between the HPC and AI communities to discuss common issues, such as how to organise the provision of large-scale, on-demand computing resources at the European level to boost AI developments; or how to support skills development and training so that the next generation of data scientists can make the most of the new technology. Data handling is another crucial aspect. Fast access to relevant data and safe storage of large sets of data and algorithms need to be organised, in compliance with GDPR and following the FAIR principles (Findable, Accessible, Interoperable, and Reusable).

 

DECREASING THE GAPS

Fast development in digital infrastructures results in a number of gaps and needs that must be promptly addressed to fully benefit from the rapidly advancing technology. Closing these gaps may involve investing in critical areas, such as:

  • Competence development;
  • Synergy and collaboration between EuroHPC and EOSC;
  • Software development and scaling;
  • Sustainability and green transition;
  • Quality and usability of data.