Description and main features
One of the formative aspects of the EOSC Portal vision was to facilitate the collaboration between EOSC end-users and EOSC resource providers to stimulate the uptake of EOSC Resources. By providing different levels of support for various EOSC Resource access methods the EOSC Portal addresses the inherent diversity of the EOSC ecosystem. Among other functionalities supporting the resource accessibility, EOSC Portal provides end-user features to order resources, monitor user requests and communicate with resource providers. For the providers in turn, the EOSC Portal offers various interoperability patterns to integrate the order management process in alignment with a vision of a federated system of systems. The implemented interoperability patterns are aimed at individual providers and provider communities adopting the guidelines to integrate their own resource provisioning mechanisms. EOSC Portal order management process engages also the EOSC Portal Operations Team who play a key role in the EOSC Portal CRM (Customer Relationship Management), end-users support and guidance through the composability of resources in the EOSC ecosystem.
EOSC Marketplace facilitates the ordering process, being a connection point between resources and order management systems (OMSes) for resource providers. It is also an entry point for the users looking to advance their research projects by using EOSC Resources. They can follow the entire path from resource discovery to order fulfillment in a single portal, meaning a coherent experience from their side.
On the other hand, providers and communities are met with several flexible options to integrate their ordering process. Firstly, they can specify offerings for their resources. Providers can configure their offerings using an interoperable Offering API or ergonomic UI. Both are flexible enough to support various offering use cases. Secondly, they can handle orders placed in the system in several interoperable ways, either utilizing the existing SOMBO system (Service Order Management Back Office), or using the Marketplace Ordering API. The latter allows to integrate existing Order Management Systems (OMSes) while providing an out-of-the-box support for Jira-based solutions as a reference implementation of the integration.
High-level Service Architecture
|
| Figure 1: EOSC Order Management Interoperability Patterns |
The EOSC Marketplace is the central part that facilitates the ordering processing and provides various ways to interface with it.
The end-users utilise the main component available under https://marketplace.eosc-portal.eu to access both the catalogue (discovery and access) and the user space (order management and support).
The Marketplace exposes Offering API and Ordering API, for use of providers, communities and the EOSC Portal Operations Team and with the use of different interoperability patterns provides several methods of order handling.
OMSes are a separate group of components that integrate closely with the Ordering API, but are free to have additional functionalities towards their users. An example of such a component is EOSC Portal OMS, i.e. SOMBO. On one hand, it has its own UI catering to its users (mostly providers and operations team), and on the other it bridges over to the EOSC MP through the Ordering API.
The diagram above presents facets of the OMSes oriented towards EOSC MP, but they may be parts of a more complex local ecosystem of services. Example could be the Community MP (a marketplace service established by a community of providers), which also integrates with the Offering API, thus freeing their providers from having to use the EOSC MP directly (either its UI or API).
|
| Figure 2: EOSC Order Management as a message bus for use in ordering |
More specifically, the functionalities provided by the Ordering API form a message bus that allows a tri-directional communication (as far as roles are related) between the users, the communities & providers, and the EOSC Portal Operations Team. By using a standardized approach, all can communicate using their systems of choice.
Adopted Standards
| Standard | Short Description | References |
|---|---|---|
REST | Representational state transfer, a standardized approach for WWW services architecture. | https://en.wikipedia.org/wiki/Representational_state_transfer |
OpenAPI Specification | A specification describing RESTful APIs, using JSON or YAML format. | https://oai.github.io/Documentation/ https://marketplace.eosc-portal.eu/api_docs/swagger/index.html |
JSON Schema | A specification to describe allowed API payload in JSON format. Relays information about the common data models. |
| Protocol/API | Short Description | References |
|---|---|---|
| HTTPS | TLS secured HTTP | https://tools.ietf.org/html/rfc2818 |
Interoperability Guidelines
Describe the procedures to integrate a service with this core service (e.g. integrate an helpdesk with the EOSC Core Helpdesk, integrate a catalogue with the EOSC Resource Catalogue, etc.)
Define 1 procedure for each integration options defined above.
The first section describes in general the steps needed to integrate Order Management with EOSC. The following section goes into more details on different interoperability patterns in Order Management.
Order Management integration steps
The Order Management is a complex process which consists of several integration fields. All of them have to be addressed so that end user is met with a consistent experience.
The steps to integrate are laid down in the following sections.
Setting up resource ordering
The main prerequisite for setting up ordering for a specific resource is the completion of Resource Onboarding in the EOSC Portal. The Onboarding Process, supporting user interfaces (https://providers.eosc-portal.eu/) and APIs that can be used to onboard resources are covered by other Interoperability Guidelines and other providers' documentation (https://eosc-portal.eu/providers-documentation).
Each resource can be configured to allow for direct ordering in EOSC Portal via the EOSC Marketplace. Two ways to manage the ordering configuration are available for the use of resource providers. Ordering may be configured either via the EOSC Marketplace user interface for authenticated provider administrators or via the Marketplace Offering API. Both methods allow for creating resource offers, configuring technical order parameters and selecting Order Management System that will be used to handle resource orders.
In order to process the resource orders a provider can choose a desired method from a selection of tools following the Interoperability Guidelines for Order Management. Service Order Management Backoffice (SOMBO) is available for all EOSC resource providers and allows to instantly manage resource orders. Although, in order to support other provider-specific or provider community-specific order processing methods other interoperability patterns exist. The next paragraphs describe the process of order management integration and interoperability patterns that can be selected to integrate other Order Management Systems.
References:
- https://marketplace.eosc-portal.eu/api_docs
- https://marketplace.eosc-portal.eu/api_docs/swagger/index.html?urls.primaryName=Offering%20API%20V1%20Docs
Setting up order management process integration
The configuration of resource ordering allows to instantly manage resource orders with the use of the SOMBO. However, as it is described in the following sections (Order Management interoperability patterns), the ordering process is much more flexible and offers integration methods allowing to meet specific needs of resource providers. The process of preparing integration for other Order Management Systems is independent from resource ordering configuration (described in the previous section) and both may happen in parallel.
The first step of integration is to choose a desired integration method from a selection of the available interoperability patterns.
The main entity allowing to set up the integration with the EOSC Marketplace, in order to make other systems interoperate with the ordering process, is an instance of the Order Management System (OMS) registration.An OMS registration is an EOSC Marketplace entity used for authentication and authorisation in the Marketplace Ordering API. In order to set it up please follow the steps documented here (https://marketplace.eosc-portal.eu/api_docs?subsection=integration_methods)
Once the registration is complete, one of the described interoperability patterns may be utilised for order processing by the integrating OMS. The newly registered OMS will also be available for certain providers (or provider communities, depending on the OMS type), and the Marketplace will allow to configure it as the resource Order Management System while setting up the resource ordering. For further details concerning the supported methods of integration and the instructions for implementation please refer to the section Order Management interoperability patterns.
References:
- https://marketplace.eosc-portal.eu/api_docs
- https://marketplace.eosc-portal.eu/api_docs/swagger/index.html?urls.primaryName=Ordering%20API%20V1%20Docs
Order Management interoperability patterns
The flexible Ordering API allows for different levels of integration by third parties, fitting their needs and expectations. Three integration levels presented below should cover most of the use-cases.
|
| Figure 3: The three levels of EOSC Order Management integration |
As mentioned before, EOSC Marketplace offers flexible ways to integrate ordering to improve interoperability. It has an API that is general enough to handle various provider use-cases. The integration interface consists of a REST API and an optional asynchronous update trigger endpoint.
There are several ways that providers can integrate their order management processes. One is to use SOMBO to handle user orders and it doesn’t incur any implementation cost.
A provider can also use a reference Jira Adapter (available under https://github.com/cyfronet-fid/oms-adapter-jira/) to integrate their Jira instance. This implementation can also be used as a reference to facilitate implementing in-house integrations, such as in the last case. A community integrates their own existing OMS with the EOSC Marketplace ordering process there.
To ensure interoperability, all involved systems must use compatible representations of projects, project items (mostly orders, but not limited to) and messages, that are general enough to meet their needs. The EOSC Marketplace strives to make as few assumptions about the exchanged data as possible, so as to allow differing use-cases.
All of the mentioned integration patterns are elaborated in the following subsections.
Service Order Management Backoffice (SOMBO)
The Service Order Management Back Office (SOMBO) is the orchestration tool between Marketplace, service providers, service requesters and shifters/operators. It is designed to track all the orders received by the Marketplace and to propose different actions on these orders.
|
| Figure 4: SOMBO architecture |
The aim is to facilitate the daily operations made by shifters, ease the communication between all parties, facilitate the negotiation between service requesters and service providers, provide facilities to sign SLA/OLA.
The SOMBO application provides a complete dashboard to shifters to list all service orders per status (on-going, accepted, rejected) and to operate them.
The shifters can access the details of the service orders in order to:
- update some fields and exchange comments with users requesting resources,
- request more information from the customer,
- analyse the service orders and approve or reject them,
- assign a service order to different resource providers and negotiate with these providers (exchange of information, validation or rejection of the resource request),
- generate OLAs and SLA documents for EGI services and start a negotiation process.
Currently the global workflow is the following:
- A service order (project item) is created in the Marketplace.
- It is translated into a Jira issue.
- In SOMBO, it is immediately visible in the dashboard.
- Shifters evaluate the validity of the order, asking for additional information if necessary with messages automatically transferred from SOMBO to the Marketplace and viceversa.
- Shifters contact Service Providers through a form.
- Service Providers accept or reject the initial request through a validation page.
- SLA and OLA documents could be generated to finalize the agreement between third parties.
Order Management process adapters for existing Order Management Systems / reference JIRA adapter
Using OMS adapter is an interoperability pattern utilising a simple service as a mediator in communication between EOSC Marketplace and an existing, possibly closed-source, system (e.g. ticketing system) to manage resource orders. Reference implementation supports integration with JIRA system but any integrator can develop its own compatibility layer with their current ticketing system.
EOSC Marketplace asynchronously informs the Adapter about new activity using an HTTP trigger. Every other communication attempts are handled proactively by the OMS adapter. Usually a ticket management system with which integration occurs informs an OMS adapter to notify the Markteplace about changes, then the adapter calls the Marketplace's API. In case a change occurs in the Marketplace (new Project / Project Item is created, Message is posted) the Marketplace triggers OMS adapter (every registered OMS adapter has a corresponding trigger configuration) and adapter performs any necessary calls to the integrating system (ticketing system) to enact that change.
It is in the scope of the OMS adapter to store mappings between Marketplace's IDs and ticketing system IDs.
Provided reference implementation based on the JIRA system (https://github.com/cyfronet-fid/oms-adapter-jira) contains Marketplace API client which is responsible for communication with the Marketplace Ordering API. It also contains a thin translation layer which maps Project Item parameters to fields in JIRA. The core philosophy enshrined in the system is that it should not, under any circumstances, lose information about ordering changes. To fulfill this goal all actions are scheduled in the underlying message queue (Celery). This way they can be retried if network errors occur, or in case they fail, and they can be reviewed manually by an operator.
Marketplace API for Order Management Systems
The most sophisticated integration pattern is to implement direct communication with the EOSC Marketplace. In order to support use-cases specific to a certain provider or community of providers, Marketplace Ordering API can be used to manage resource orders. This solution is suitable for setting up interoperability between EOSC Marketplace and other existing marketplaces, gateways, infrastructures and services that need the highest flexibility and deepest integration with EOSC Portal. This integration pattern allows to create a tailor made resource order handling solutions and map sophisticated resource provisioning workflows to EOSC Portal-based interaction with its end-users.
As before, the reference implementation should help with integration (https://github.com/cyfronet-fid/oms-adapter-jira). It utilizes a more general EOSC Marketplace API client, that is the preferred way to integrate: https://github.com/cyfronet-fid/marketplace-api-client.
References:
- https://marketplace.eosc-portal.eu/api_docs
- https://marketplace.eosc-portal.eu/api_docs/swagger/index.html?urls.primaryName=Ordering%20API%20V1%20Docs
- https://github.com/cyfronet-fid/marketplace-api-client/
Examples of solutions implementing this specification
The existing integration is set up with SOMBO https://opsportal.eosc-portal.eu/.
The code of a reference OMS adapter is available for inspection and use under https://github.com/cyfronet-fid/oms-adapter-jira.
The use of the Marketplace Ordering API Python client library can also facilitate faster integrations development: https://github.com/cyfronet-fid/marketplace-api-client.