Addressed
| Title | EOSC Data Transfer: Architecture and Interoperability Guidelines |
|---|---|
| Published URI | |
| Revised version links | |
| Link to Metadata | https://docs.google.com/spreadsheets/d/1fjxArO-PBbvUd2bsTVXa6D8pI_9i9Vd463bn4YI0y9c/edit# |
| Date Issued to EIAC | 18.04.2023 |
| Submitted by | Andrea Manzi and Levente Farkas (EGI Foundation) |
| Outcome of EIAC review | approved at meeting 30.05.2023 |
| Date Issued to EIAB | 09.06.2023 (Anke), requested for inclusion on agenda for 04.07.2023 TCB Agenda (Michelle) |
| Outcome of EIAB review | Approved (at 04.07.2023 meeting) |
| On portal? | yes |
Overview
The proposed document has been submitted for inclusion in the EOSC Interoperability Framework.
Comments should be made either on eiac@eoscfuture.eu, where they will be archived for future reference or added to the change log below. Comments posted to other discussion methods will not be included in the review.
Statements of Support/Deny
All members of the EIAC are encouraged to make a statement of support or dissension in consideration of this proposal.
Name | Organisation | Support or Deny | Reason (if denied) |
|---|---|---|---|
Jorik van Kemenade and Paul van den | SURF | support | |
Sabeel Shah | OPENAIRE | support | |
Thanassis Mantes | ATHENA | support | |
Kostas Koumantaros | GRNET | support |
Comment/Change Log
Please fill in any comments or proposed changes below or add them as issues on the github repository.
Number | Current Text | Proposed Text / Query | Commenter | Action |
|---|---|---|---|---|
| 1. | Section 1: Intended audience | This is still the template text, I think this part should mention the intended audience for this guideline. | Jorik van Kemenade (SURF) | Addressed |
| 2. | Section 2: From reading this and the previous paragraph it is unclear to me what this guideline covers. Is it a service that people can buy or link to? Is it a core component of the EOSC Portal that people can integrate with? If possible I would like to read about a specific problem and how this solution solves this problem. In the description, I think it is important to focus on large functional components that the user or service has to interact with. So the first paragraph is a good start. | Jorik van Kemenade (SURF) | Addressed | |
| 3. | Section 3: The EOSC Data Transfer provides a way to efficiently transfer datasets from repositories registered in EOSC to Compute nodes via a Web interface abstracting over a particular technology to transfer data. | Move or copy to section 2. It helps understand the purpose of this component. | Jorik van Kemenade (SURF) | Addressed |
| 4. | Section 3: By providing an abstract layer that could be implemented by data transfer providers, we ensure service composability and the openness of the EOSC Exchange. | The risk of offering an abstraction layer over existing protocols is that we are effectively introducing yet another protocol. Maybe we can reflect on what this abstraction layer offers over deciding to implement a specific existing transfer protocol. | Jorik van Kemenade (SURF) | Addressed |
| 5. | Section 4: The Authentication and Authorization is based on the EOSC AAI Federation for the authentication/authorization towards a particular instance of a Data Transfer Service | I can't find the AAI in Figure 1, might be good to add this. | Jorik van Kemenade (SURF) | Addressed |
| 6. | Section 5: This makes the API flexible, by allowing multiple destination storages that use the same protocol to be handled by different transfer services, but at the same time it also allows an entire protocol (e.g. S3) to be handled by a specific transfer service. | I feel that explaining the why should not be part of the definition. I think a different place, e.g. section 10 of the document, might be more suitable. | Jorik van Kemenade (SURF) | Addressed |
| 7. | Section 6: The EOSC Data Transfer API is licensed under Apache Licence 2.0 | Is this the license that is relevant for interoperability? Are people expected to run the API? | Jorik van Kemenade (SURF) | Addressed |
| 8. | Section 8: RESTful API are API that conform to the design principles of the REST, or representational state transfer architectural style. | I think that if we want to describe what RESTful means, it might be good to implement this as an extra protocol. | Jorik van Kemenade (SURF) | Addressed |
| 9. | Section 8: EOSC Data Transfer REST API | If I understand correctly, I am currently looking at the interoperability guide that describes this API? Or is this meant as a link to the full API spec, where this document only covers parts of the spec? | Jorik van Kemenade (SURF) | Addressed |
| 10. | Section 4: The following components are part of the architecture | I think it would be good if each of the integration components in part 9 is at least described in less (or more) detail in the high-level architecture. | Jorik van Kemenade (SURF) | Addressed |
| 11. | Section 9.2: Implementation of the Java interface(s) | What are the Java interfaces? Are they part of a framework that people are integrating into their applications? Is it possible for me to implement in for example Python as long as I implement the correct API schema? | Jorik van Kemenade (SURF) | Addressed |
| 12. | Section 10.1 In order to extend the capabilities of the EOSC Data Transfer service, implementation of one or more Java interfaces is needed, followed by the redeployment (or a new deployment) of the EOSC Data Transfer API. | Is this a prerequisite? To me, it seems that this is the action of integrating and the purpose of this document - but I might be misunderstanding something. | Jorik van Kemenade (SURF) | Addressed |
| 13. | Section 10.2.1: Implement the Java interface ParserService in a class of your choice: | We can consider adding links to the interfaces that need to be implemented rather than giving the interface definition. This way the guideline does not have to be updated when there are changes in the API. | Jorik van Kemenade (SURF) | Addressed |
| 14. | Section 10: Please note that the Interoperability Guidelines are not intended to replace instruction manuals, help, or documentation. | I like that we include this, perhaps we can link each of the integration options below to relevant sections of the documentation and minimize the implementation details in this document? | Jorik van Kemenade (SURF) | Addressed |
| 15. | Section 9: Three different types of interoperability | I think it is a good idea if we spent some time on explaining what a data transfer service or a storage element is. We have a nice definition in table 1, but this might not be enough for people to understand the scope of the elements. | Jorik van Kemenade (SURF) | Addressed |
| 16. | Section 10.2.3: Each transfer service that gets integrated can optionally implement this functionality. | Maybe it is good if we help users with what seems like a design decision. Why would I implement this? Is there also a reason why I would not want to implement this? | Jorik van Kemenade (SURF) | Addressed |
| 17. | Section 12: Examples of solutions implementing the specifications | Maybe we can offer some links to the source, this way people can see what it actually means in terms of work to integrate. | Jorik van Kemenade (SURF) | Addressed |
| 18. | General comment | Overall I think this is a good first draft of the guideline. The focus is on the interface that the implementer has to interface with, this is very good! I am missing some depth in the overall architecture and solution. After going through the entire document I am still not completely sure about the design decision I need to take if I want to integrate my service into the transfer service and what I need to do to achieve this. I have given some comments and suggestions that can make this more clear for the implementer, but I think it is good if the guideline is looked at again from that standpoint. | Jorik van Kemenade (SURF) | Addressed |
| 19. | General comment | All of my comments have been addressed | Sabeel Shah (UniBi) | Addressed |