Science Projects Technical Alignment

Rolling minutes: https://docs.google.com/document/d/1dO9hWn4_m1ZmXTMA0tZwkIeei0kxJD-yKH2IxxqJAto/edit 

---------------

EOSC Future WP3 - T3.3

WG Documented Outputs Template

WP3 Task 3.3 Information

Link to Public Wiki webpage

Working Group Information

Chair: 

• Jonathan Tedds/ELIXIR/EMBL-EBI
• Giuseppe La Rocca/EGI.eu

Members: 

The Technical Alignment with Science Projects Working Group was composed by:

• Representatives from all the Science Clusters (ENVRI-FAIR, ESCAPE, Life Science, PANOSC, SSHOC)
• Members from the INFRAEOSC-07 projects including C-SCALE, DICE, EGI-ACE, GEANT, OpenAIRE-Nexus and RELIANCE
• Technology providers (from WP4 and WP5)

The full list of WG members is available here.

Start Date: 01 Sep 2021 

Finish Date: on-going

Working Group Links

Link to charter document

Link to Public Wiki webpage: Science Projects Technical Alignment

Link to minutes: https://docs.google.com/document/d/1dO9hWn4_m1ZmXTMA0tZwkIeei0kxJD-yKH2IxxqJAto/edit

Link to folder: https://go.egi.eu/eosc-sp 

WG Background (Pre-operational phase)

General description of the need  Working Group - specifically for EOSC 

Research Infrastructures, such as the ones on the ESFRI road-map, are mainly characterized by the huge data volumes they generate and handle, and by the innovative applications and tools that they bring together in order to ensure effective research data exploitation by thousands of researchers across scientific disciplines and research projects. In the EOSC ecosystem, the ESFRI Research Infrastructures are organised in five thematic Science Clusters. These Science Clusters play an important role for the provision of data and services to their research communities. 

The EOSC Future project has selected Science Projects from the thematic Science Clusters to demonstrate how, by facilitating their integration with the core and horizontal services offered by EOSC, they can address major science and societal challenges and promote cutting-edge research, innovation and knowledge transfer in Europe, and as a consequence, support Horizon Europe's missions within the EOSC. From a technical perspective, the integration of these Science Projects with the EOSC core and horizontal services will provide important assets to enrich the EOSC Exchange through research data, workflows, thematic data analytics tools and advanced data management solutions.

Problems the Working Group was to address 

The main objectives of the Technical Alignment with Science Project WG was to create a discussion forum between the technical partners of the project and the Science Projects to: 

• Assess the technical requirements of the Science Projects.
• Identify gaps in the project solutions and EOSC architecture.
• Pilot integrations with EOSC Core and horizontal services.
• Keep Science Projects informed about the technical achievement of the project (from all technical WPs).
• Oversee how the science projects adopt project technical outputs.

Solutions the Working Group intended to output/provide recommendations on

• EOSC-Life - MetaCOVID: Proposal for metadata interoperability (expected delivery April-May 2023).

WG Summary of Actions (During the operational phase)

Working Group - operational activity

Meeting frequency, number of meetings, and all other operational details of relevance.

The Technical Alignment with Science Projects WG officially kick-started its operational phase in September 2021. A dedicated mailing-list (sp-wg@eoscfuture.eu) was set-up to facilitate the communication with members of the WG. A shared folder (see above) was also prepared to store relevant slide-decks, and presentations used during the meetings.

In order to facilitate the technical integration of the EOSC Future Science Projects with the EOSC core and the horizontal services provided by the INFRAEOSC-07 projects, technical experts (alias e-Infrastructure contacts) were assigned to each of the Science Projects. The overall responsibility of these experts was to liaise with the Science Projects, map the use cases to proper service providers, oversee the integration plans, and report about achievements, lessons learnt and other outcomes. During the monthly meetings, Science Projects and e-infrastructure contacts were invited to report the progress with the integration plans and raise blocking issues (if any). 

During the project’s lifetime, the scope of this WG was further expanded inviting members of:

• T6.2 - “Integration of EOSC-Core Services into European Research Practice” to provide regular updates on the status of the on-boarding of services in EOSC,
• WP10 to provide updates about the status of the dissemination plans.

Since the kick-start of the WG, a total of 18 online meetings have been organised.

Working Group - work undertaken during the operational phase

The WG, in collaboration with the assigned e-Infrastructure experts and Technology providers (from WP4 and WP5), provided assistance to support the technical integration with the EOSC core and horizontal services provided by the INFRA-EOSC-07 projects.

Working Group - challenges

The main challenges faced by the WG during the operation phase are listed below:

Integration plans:

• Different level of maturity of the Science Projects.
• For some of EOSC Future Science Projects, their integration plans to integrate with the EOSC core/horizontal services were not crystal clear, or not foreseen in the short term. 

Amendment:

• The delay with the signing of the project amendment has impacted on the official kick-start of the work plan of the SSHOC Science Projects.

On-boarding in EOSC:

• Several issues were raised during the on-boarding of services in EOSC. In particular the main issue was related to the upload of the service providers' privacy policy and terms of use which were considered optional in the previous release of the portal, but that were requested in order to complete with the registration workflow. These issues have been solved with the recent new release of the on-boarding procedure and the update of the workflow.
• The overall procedure for on-boarding services in EOSC still remains complex and difficult to follow. More flexibility on the categories classification (data resource vs resource catalog) would be needed.
• The services developed by many of the Science Clusters are the main result of collaborative efforts between different partners sometimes formalized in a Memorandum of Understanding (MoU) but without a specific legal entity formally responsible for the service. For this reason, the on-boarding of new services in EOSC under the umbrella of a single entity is still an issue and a show-stopper.

Change of the leadership of the PaNOSC Science Projects:

• In Jan. 2023 the principal investigator of the PaNOSC Science Projects left ESRF. As a consequence the integration plans were put on hold for a while. A new reference contact was assigned in March.

WG Highlighted Outputs/Recommendations (Post-operational phase)

Working Group summary of outputs and recommendations 

The main outputs produced by the WG, grouped by categories, are listed below:

1. SHOWCASE INTEGRATION STORIES (SISs):
   1. ENVRI - Climate Change Impact on Biodiversity and Ecosystems in Europe (1)
   2. ENVRI - Dashboard on the State of the Environment (2) 
   3. EOSC-Life - Imaging Data in EOSC - COVID-19 as Demonstrator (4)
   4. ESCAPE - Dark Matter SIS (5)
   5. ESCAPE - Extreme Universe and Gravitational Waves (6)
   6. PaNOSC -  Tracing Biostructures (7)
   7. PaNOSC - Serial Crystallography SIS (8)
   8. SSHOC - Access Management for Distributed RIs (10)
      
      
2. STATUS OF THE INTEGRATION WITH THE EOSC CORE AND HORIZONTAL SERVICES:
   1. PaNOSC - Serial Crystallography
      1. AAI: completed
      2. Cloud resources (offered by EGI-ACE): completed
   2. ESCAPE - Dark Matter
      1. ESCAPE AAI federated with the EOSC AAI: completed
      2. Cloud resources (offered by EGI-ACE): completed
         
   3. ENVRI-FAIR - Dashboard on the State of the Environment
      1. AAI: completed
      2. Helpdesk, Monitoring: in progress
      3. Cloud resources (offered by EGI-ACE): completed
   4. EOSC-Life - EU-OpenScreen
      1. EGI Notebooks (offered by EGI-ACE): completed
         
   5. EOSC-Life - MetaCOVID
      1. Performed interviews with the RIs on the use of contextual metadata. 
      2. Completed (preparatory TC + interview):
         1. ECRIN - interview final notes here
         2. BBMRI - interview final notes here
         3. CESSDA (TAU/FSD) - interview final notes here
         4. CLARIN (& UA) - interview draft notes here
         5. EATRIS - interview draft notes here
         6. EU-OPENSCREEN - interview final notes here
      3. EOSC-Life - MetaCOVID: Proposal for metadata interoperability (expected delivery April-May 2023).
      4. OpenAIRE and ECRIN defined a crosswalk between the ECRIN model and the OpenAIRE guidelines (see this spread-sheet).
         1. ECRIN is implementing an API that could be used by OpenAIRE to harvest the metadata records.
      5. EuroBioImaging in contact with the OpenAIRE Aggregation team.
   6. EOSC-Life - EuroBioImaging
      1. Developed Tool and Workflow Development to Support Next Generation File Format OME-Zarr (https://github.com/Euro-BioImaging/BatchConvert)
      2. The tool is going to be onboarded to WorkflowHub.
         
   7. EOSC-Life - Instruct-ERIC
      
      1. Completed the integration of the 3DBionotes-WS and COVID19 Structural Hub with the Helpdesk. 
      2. The integration with the Monitoring service is on-going.
   8. SSHOC - ARIA Science Project
      
      1. Initial pool of resources have been allocated at IN2P3-IRES.
         
         
3. ON-BOARDING IN EOSC
   1. ESCAPE: (IN PROGRESS)
      1. ESCAPE Virtual Observatory, and the 
      2. ESCAPE Open-source Scientific Software and Service Repository
   2. ENVRI-FAIR - Dashboard on the State of the Environments (COMPLETED)
   3. EOSC-Life - Instruct-ERIC (COMPLETED)
      1. 3DBionotes-WS on-boarded in the EOSC Marketplace.
      2. BioModels Monkey Pox integrated as Research Data Source.
   4. EU-OpenScreen:
      1. KG resources onboarded to EOSC services (COMPLETED):
         1. Monkeypox Knowledge Graph: A semantic resource embedding biological and chemical entities
         2. COVID-19 Knowledge Graph: A semantic resource embedding biological and chemical entities
            
            
4. DISSEMINATION
   1. ENVRI-FAIR - Dashboard on the State of the Environment: 
      1. Dissemination at ENVRI-week and EGU2023
      2. Presented a demo during the 2nd. Project review in Nov. 2022
   2. ESCAPE - Dark Matter:
      1. Presented a demo during the 2nd. Project review in Nov. 2022

WG Summary Paragraph + Further work needed

Since September 2021, the Technical Alignment with Science Projects WG has organized a total of 18 online meetings, involving representatives from the Representatives from all the Science Clusters (ENVRI-FAIR, ESCAPE, EOSCLife, PANOSC, SSHOC), members from the INFRAEOSC-07 projects including C-SCALE, DICE, EGI-ACE, GEANT, OpenAIRE-Nexus and RELIANCE, and technology providers (from WP4 and WP5). During the online meetings, organized on a monthly basis, members of the Science Projects were invited in a round-table to report the status of the integration plans of the Science Projects with the EOSC core and horizontal services provided by the INFRAEOSC-07 projects. Experts from the projects (WP4, and WP5) and service providers from the INFRAEOSC-07 projects provided assistance during the piloting activities. 

After 18 months of running activities, the WG achieved the following results:

• Integration with the EOSC core services: 
  • PaNOSC and ESCAPE completed the integration with the EOSC Federation AAI. 
• Integration with the horizontal services:
  • PaNOSC, ESCAPE, ENVRI-FAIR integrated with the Cloud Compute resources of EGI.
  • EOSC-Life - EU-OpenScreen completed the integration with the EGI Notebooks service.
• On-boarding:
  • ENVRI-FAIR completed the onboarding of the “EOSC Future Dashboard for Environmental Indicators”.
  • EOSC-Life - Instruct-ERIC completed the on-boarding of:
  • 3DBionotes-WS on-boarded in the EOSC Marketplace.
  • BioModels Monkey Pox integrated as Research Data Source.
  • The on-boarding of the ESCAPE Virtual Observatory and the ESCAPE Open-source Scientific Software and Service Repository is in progress.
• No. 8 Showcase Integration Stories (SISs) were produced.
• ENVRI-FARI and ESCAPE successfully presented demos during the 2nd. project review that took place in November 2022.
• EOSC-Life - MetaCOVID is finalizing a proposal for metadata interoperability  (expected delivery April-May 2023).

For the remaining months, the WG will:

• Progress with the integration activities of the EOSC Science Projects with the EOSC core/horizontal services.
  
• Finalize the Showcase Integration Stories for the upcoming reviews.