| Title | AI/ML Recommender System Interoperability Guideline |
|---|---|
| URI | https://doi.org/10.5281/zenodo.7849178 |
| Revised version links | |
| Link to Metadata | |
| Date Issued to EIAC | 24.04.2023 |
| Submitted by | John Shepherdson, CESSDA/WP5 |
| Outcome of EIAC review | approved at meeting 30.05.2023 |
| Date Issued to EIAB | 09.06.2023 (Anke), requested for inclusion on agenda for 04.07.2023 TCB Agenda (Michelle) |
| Outcome of EIAB review | Approved (at 04.07.2023 meeting) |
| On portal? | yes, published |
Overview
The proposed document has been submitted for inclusion in the EOSC Interoperability Framework.
Comments should be made either on eiac@eoscfuture.eu, where they will be archived for future reference or added to the change log below. Comments posted to other discussion methods will not be included in the review.
Statements of Support/Deny
All members of the EIAC are encouraged to make a statement of support or dissension in consideration of this proposal.
Name | Organisation | Support or Deny | Reason (if denied) |
|---|---|---|---|
| Thanassis Mantes | ATHENA RC | SUPPORT | |
| Kostas Koumantaros | GRNET | support | |
| Paul Gondim van Dongen | SURF | support | |
| Jorik van Kemenade | SURF | support |
Comment/Change Log
Please fill in any comments or proposed changes below or add them as issues on the github repository.
Number | Current Text | Proposed Text / Query | Commenter | Action |
|---|---|---|---|---|
| 1 | 13.1.1 Marketplace database events Suggestion to give an example of the calls creating those responses (CURL?) and to explain what those JSON are used for. The same applies to all following subparagraphs. | Aggelikki Gkamiliari | REJECTED Those messages are described in detail here: Databus - the document that is many times mentioned/included in this article. If we wanted to describe each message from the queue that is important from the point of view of our services, we could write a whole, separate article on this subject, in which many other organizations would be involved. | |
| 2 | 13.1.1 Examples Request to include relevant API calls | Aggelikki Gkamiliari | ||
| 3 |
The intended audience for the AI/ML Recommender System Interoperability Guideline is technical experts who would like to learn how to design their services to be interoperable or to integrate with the EOSC Recommender System (EOSC-RS) and/or who would like their resources (data sets) to be integrated within the EOSC-RS architecture. | Maybe we can also stress the bi-directionality and include ext catalogues which would like to get RS data (e.g., aggregate stats, recommendations per service, etc) out of the existing RS service. | George Papastefanatos | REJECTED For technical reasons, any client must be in the same private network as the EOSC-RS, so external catalogues are excluded |
| 4 | 3. Response to Community Need From the user perspective, recommendations can help to improve overall user satisfaction within a web site. | instead of referring to the web site, refer to the EOSC portal. Not a website in general | George Papastefanatos | REJECTED This paragraph is referring to the general case of user satisfaction, rather than being EOSC-specific. This is in keeping with the referenced source. |
| 5 | 3. Response to Community Need ...from the user perspective, recommendations can help to improve overall user satisfaction within a web site. For example, a user who receives service recommendations will be more satisfied with the experience and is more likely...
| Again. Maybe you can refer to "a researcher who receives recommendations about services will be more satisfied ..." | George Papastefanatos | ACCEPTED Added ‘relevant’ before 'recommendations’ |
| 6 | 3. Response to Community Need ...For example, recommendations generated from observing a user's behaviour on the web site could be substantially different from the ones that use information about the selected content or relationships with other users with similar interests. | ...For example, recommendations generated from observing a user's behaviour on the EOSC portal could be.... | George Papastefanatos | REJECTED This paragraph is referring to the general case of user satisfaction, rather than being EOSC-specific |
| 7 | Figure 4.1 | a) All arrows are biderctional, making the flow of data a bit "unclear" I would propose to label the edges and show the type of data\ or actions performed at each edge. | George Papastefanatos | ACCEPTED The diagram has been changed to show key data sent between modules |
| 8 | Figure 4.1 | b) The core part is the RS, but I miss the internals. Maybe it can be placed at the center and enlarge, showing also some of its internals subcompoenents (e.g. are these described in the appendix?) | George Papastefanatos | ACCEPTED The recommender module has been enlarged and the key module for this document, i.e. RS Facade, has been marked in it. |
| 9 | Figure 4.1 | c)EOSC Monitoring is not connected nor described. What is its purpose there. | George Papastefanatos | ACCEPTED EOSC Monitoring was removed. |
| 10 | 4.1 Evaluation framework The Recommender Metrics Framework is an independent 'metrics framework as a service' used to support the evaluation and adaptation of recommendation mechanisms. | Although independent, It would be good to have it in the overall picture of 4.1 architecture. Even as standalone service, which has input from the RS. | George Papastefanatos | ACCEPTED Recommender Metrics Framework was added to the diagram. |
| 11 | 9 Integration Options There are three main scenarios for integrating other components with the EOSC-RS. The first scenario targets presentation (i.e UI) components of the EOSC ecosystem, which can get and utilise a set of recommended (and relevant) resources for a given user.... | I miss the output of the RS. Dont we consider scenario, which an EOSC Node (external catalogue) would like to has access to rs (aggregated) data? | George Papastefanatos | REJECTED For technical reasons, any client must be in the same private network as the EOSC-RS, so external catalogues are excluded |
| 12 | 2. Description and main features The EOSC-RS delivers two main types of recommendations: recommendations for Consumers (researchers) and recommendation for (resource) Providers.
| IMO, an example of what is an actual recommendation would make the whole section clearer | Thanassis Mantes | ACCEPTED Added explanatory text. |
| 13 | 10.1.1 Integration with RS Facade Integration with the RS Facade component allows a client to obtain various types of recommendations via its REST API. I
| Some cURL-like examples per case in the appendix would be helpful | Thanassis Mantes | ACCEPTED Examples of bodies of requests have been added in the appendix BW: same as No.2: example calls could help the readers to understand the interface |
| 14 | 2. Description and main features ...The EOSC-RS is available to other EOSC Core Services components through a dedicated API (the RS Facade), but it doesn't communicate with end-users directly.
| It's not clear what is meant here. If it doesn't communicate directly, does that mean that it communicates indirectly? | Pavel Weber | ACCEPTED Added explanatory text |
| 15 | 2. Description and main features ... The generated recommendations need to be closely related to the user's implied interests. Specifically, the following use cases are delivered to authenticated users:
| all these use cases are delivered with the same weights? E.g. a general popularity is available for non-logged users, is't also available for logged and how it intertwines with other recommendation types? | Pavel Weber | REJECTED BW: From my point of view, the relative importance of recommendations presented to the users should depend on their status (logged-in/not-logged-in) |
| 16 | 2. Description and main features ...Specifically, the following use cases are delivered to authenticated users: .... | I guess authenticated users is the same as Consumers, Researchers etc. May be some general approach should be chosen and better to stick just to one notation? | Pavel Weber | ACCEPTED Changed ‘users’ to ‘Consumers’ |
| 17 | 3. Response to Community Need ...Any feature allowing seamless access to data will improve scientific processes, ensuring higher quality research in less time, and could result in more publications of higher quality... | It's not clear which feature is meant here. The sentence is obvious, but I don't see any connection to recommender system. The RS also doesn't allow seamless access, but better to say facilitate it. | Pavel Weber | ACCEPTED Changed ‘allowing’ to ‘facilitating’ and added ‘such as relevant recommendations’ |
| 18 | 3. Response to Community Need ...Finally, providing the user with an explanation for why a particular item is recommended is often as useful as the recommendation itself. For example, recommendations generated from observing a user's behaviour on the web site could be substantially different from the ones that use information about the selected content or relationships with other users with similar interests. | The connection between these 2 sentences is not clear to me. How the example given in 2nd sentence explains the first? | Pavel Weber | REJECTED The text paraphrases the cited article 'Explaining the user experience of recommender systems' which found a correlation between the two |
| 19 | 10.1.1 Integration with RS Facade ...In order to get recommendations for a specific user, the client sends a request to the RS Facade API containing all required fields and identifies itself by setting its CLIENT_NAME: ... | For Helpdesk I was asked to add a section on GDPR compliance and actions in case of sharing of sensetive information. Is it also the case here? Is anyone can get recommendation for any user? Are there any restrictions in terms of GDPR to be mentioned?
| Pavel Weber | REJECTED |