Brings the information necessary for the front to work correctly.

Functional overview

Brings the information necessary for the front to work correctly.
It has for Favorites end-point :
- get favorites
- insert a new favorite
- delete a favorite

Technical overview

should have a mongodb lounched on 27017 port : docker run -p 27017:27017 –name mongo-dev mongo:bionic
to launch : – go run .

Api

Get all workflows
Get one workflow
Get workflows by user
Create workflow
Update workflow
Delete workflow
Execute workflow
Get all executions
Get one execution
Get executions status count
Get all favorites
Add favorite
Delete favorite
Get provider in catalog by id
Search in catalog

Get All workflows

URL: /api/generic-backend/workflows

METHOD: GET

Path Params: None

Query Params: None

Success Response

Code: 200 OK

Content: AllWorkflowResponseContainer model

Error Response

Code: 500 Internal Server Error

Sample Call

  curl 'http://localhost:9000/api/generic-backend/workflows' \
  -H 'Accept: application/json, text/plain, */*' \
  --compressed \
  --insecure

Get One Workflow

URL: /api/generic-backend/workflows/{id}

METHOD: GET

Path Params:

id
- description: id of the workflow we want to get
- required: true
- type: string
- restriction: None

Query Params: None

Success Response

Code: 200 OK

Content: Workflow model

Error Response

Code: 404 Not Found if no workflow with this id exists

Sample Call

  curl 'http://localhost:9000/api/generic-backend/workflows/60db108756f8733c30548560' \
  -H 'Accept: application/json, text/plain, */*' \
  --compressed \
  --insecure

Get Workflow By User

TODO: This route should be merged with Get all workflows

URL: /api/generic-backend/workflows/user

METHOD: GET

Path Params: None

Query Params:

user
- description: The id of the user
- required: true
- type: string
- restriction: None

Success Response

Code: 200 OK

Content: FoundWorkFlow[] model

Error Response

Code: 404 Not Found

Sample Call

  curl 'http://localhost:9000/api/generic-backend/workflows/users?user=60db108756f8733c30548560' \
  -H 'Accept: application/json, text/plain, */*' \
  --compressed \
  --insecure

Create Workflow

URL: /api/generic-backend/workflows

METHOD: POST

Path Params: None

Query Params: None

Request Body: Workflow model

Success Response

Code: 200 OK

Content:

  {
  "id": "61a9dd77587e8bd849fa2a79",
  "name": "demo",
  "user": "",
  "blocks": [],
  "links": null,
  "runConfig": {
    "computeProvider": {
      "providerId": "computerprovider1",
      "providerType": "computing",
      "description": "enregistrement computer provider ",
      "originClusterName": "gsm-identity",
      "endpointLocation": "http://34.228.18.168/api/compute/",
      "name": "computerProvider",
      "workflowStructure": {
        "arguments": null,
        "baseCommand": null,
        "class": "",
        "doc": "",
        "id": "",
        "inputs": null,
        "label": "",
        "outputs": null,
        "permanentFailCodes": null,
        "requirements": null,
        "stderr": "",
        "stdin": "",
        "stdout": "",
        "successCodes": null,
        "temporaryFailCodes": null
      },
      "isLocal": true,
      "metadata": {
        "kubernetesApi": "https://108.192.30.222",
        "resources": {
          "cpu": 6,
          "ram": 6255353856
        },
        "s3": {
          "accessKey": "access_key",
          "secretKey": "secret"
        }
      },
      "costLimitCluster": 0
    }
  }
}

Error Response

Code: 500 Internal Server Error

Code: 400 Bad Request

Sample Call

  curl 'http://localhost:9000/api/generic-backend/workflows' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'Content-Type: application/json' \
  -H 'Accept-Language: fr-FR,fr;q=0.9,en-US;q=0.8,en;q=0.7' \
  --data-raw '{"name":"demo","blocks":[],"runConfig":{"computeProvider":{"isLocal":true,"name":"computerProvider","providerId":"computerprovider1","providerType":"computing","description":"enregistrement computer provider ","originClusterName":"gsm-identity","endpointLocation":"http://34.228.18.168/api/compute/","workflowStructure":{"label":"","class":"","id":"","stdin":"","stdout":"","stderr":"","doc":"","successCodes":null,"temporaryFailCodes":null,"permanentFailCodes":null,"baseCommand":null,"arguments":null,"inputs":null,"outputs":null,"requirements":null},"metadata":{"kubernetesApi":"https://108.192.30.222","resources":{"cpu":6,"ram":6255353856},"s3":{"accessKey":"access_key","secretKey":"secret"}}}}}' \
  --compressed \
  --insecure

Update Workflow

URL: /api/generic-backend/workflows/{id}

METHOD: POST

Path Params:

id
- description: id of the workflow we want to update
- required: true
- type: string
- restriction: None

Query Params: None

Request Body: Workflow model

Success Response

Code: 200 OK

Content: Workflow model

Error Response

Code: 422 Unprocessable entity

Code: 500 Internal Server Error

Sample Call

  curl 'http://localhost:9000/api/generic-backend/workflows/61a9dd77587e8bd849fa2a79' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'Content-Type: application/json' \
  --data-raw $'{"id":"61a9dd77587e8bd849fa2a79","name":"demo","user":"","blocks":[{"index":0,"inputPorts":[],"outputPorts":[{"selected":false,"index":0,"name":"downloadedFile"}],"payload":{"kind":"treatment-payload","cwl":{"label":"Sentinel","class":"CommandLineTool","id":"","stdin":"","stdout":"","stderr":"","doc":"","successCodes":null,"temporaryFailCodes":null,"permanentFailCodes":null,"baseCommand":["wget"],"arguments":[{"isFilled":true,"position":0,"prefix":"","separate":true,"itemSeparator":"","valueFrom":"--tries=1","shellQuote":true},{"isFilled":true,"position":0,"prefix":"","separate":true,"itemSeparator":"","valueFrom":"-O","shellQuote":true},{"isFilled":true,"position":0,"prefix":"","separate":true,"itemSeparator":"","valueFrom":"image.tif","shellQuote":true}],"inputs":{"boundingBox":{"label":"","doc":"","streamable":false,"loadContents":false,"inputBinding":{"isFilled":false,"position":0,"prefix":"","separate":false,"itemSeparator":"","valueFrom":"","shellQuote":false},"format":null,"type":{"type":"boundingBox"},"inputType":"definition"},"dateRange":{"label":"","doc":"","streamable":false,"loadContents":false,"inputBinding":{"isFilled":false,"position":0,"prefix":"","separate":false,"itemSeparator":"","valueFrom":"","shellQuote":false},"format":null,"type":{"type":"dateRange"},"inputType":"execution"},"url":{"label":"","doc":"","streamable":false,"loadContents":false,"inputBinding":{"isFilled":true,"position":1,"prefix":"","separate":true,"itemSeparator":"","valueFrom":"","shellQuote":true},"format":null,"type":{"type":"string"},"inputType":"orchestrator"}},"outputs":{"downloadedFile":{"label":"","streamable":false,"doc":"","format":null,"binding":{"loadContents":false,"glob":["image.tif"],"outputEval":""},"type":{"type":"file"}}},"requirements":[{"type":"DockerRequirement","dockerPull":"mwendler/wget:latest","dockerLoad":"","dockerFile":"","dockerImport":"","dockerImageId":"","dockerOutputDirectory":""}]},"id":-1,"parameters":{},"providerType":"data","endpointLocation":"http://34.228.18.168/api/provider/data/sentinel","computeProvider":{"isLocal":true,"name":"computerProvider","providerId":"computerprovider1","providerType":"computing","description":"enregistrement computer provider ","originClusterName":"gsm-identity","endpointLocation":"http://34.228.18.168/api/compute/","workflowStructure":{"label":"","class":"","id":"","stdin":"","stdout":"","stderr":"","doc":"","successCodes":null,"temporaryFailCodes":null,"permanentFailCodes":null,"baseCommand":null,"arguments":null,"inputs":null,"outputs":null,"requirements":null},"metadata":{"kubernetesApi":"https://104.197.31.223","resources":{"cpu":6,"ram":6255353856},"s3":{"accessKey":"access_key","secretKey":"secret"}}},"costLimitBlock":0,"costLimitCluster":0,"s3Provider":{"providerId":"computerprovider1","providerType":"computing","description":"enregistrement computer provider ","originClusterName":"gsm-identity","endpointLocation":"http://34.228.18.168/api/compute/","name":"computerProvider","workflowStructure":{"arguments":null,"baseCommand":null,"class":"","doc":"","id":"","inputs":null,"label":"","outputs":null,"permanentFailCodes":null,"requirements":null,"stderr":"","stdin":"","stdout":"","successCodes":null,"temporaryFailCodes":null},"isLocal":true,"metadata":{"kubernetesApi":"https://107.199.31.223","resources":{"cpu":6,"ram":6255353856},"s3":{"accessKey":"access_key","secretKey":"secret"}},"costLimitCluster":0},"clusterName":"Local (gsm-identity)"},"name":"Provider d\'image sentinel","x":29,"y":67.39999389648438}],"links":[],"runConfig":{"computeProvider":{"providerId":"computerprovider1","providerType":"computing","description":"enregistrement computer provider ","originClusterName":"gsm-identity","endpointLocation":"http://34.228.18.168/api/compute/","name":"computerProvider","workflowStructure":{"arguments":null,"baseCommand":null,"class":"","doc":"","id":"","inputs":null,"label":"","outputs":null,"permanentFailCodes":null,"requirements":null,"stderr":"","stdin":"","stdout":"","successCodes":null,"temporaryFailCodes":null},"isLocal":true,"metadata":{"kubernetesApi":"https://107.199.31.223","resources":{"cpu":6,"ram":6255353856},"s3":{"accessKey":"OHgVCyljJZbQB07DHtqX","secretKey":"YPqm5AMKWUFlLmBNwbmue9LvuqptTIvyoMCQnlnK"}},"costLimitCluster":0}}}' \
  --compressed \
  --insecure

Delete Workflow

URL: /api/generic-backend/workflows/{id}

METHOD: DELETE

Path Params:

id
- description: id of the workflow we want to delete
- required: true
- type: string
- restriction: None

Query Params: None

Request Body: None

Success Response

Code: 200 OK

Content: None

Error Response

Code: 404 Not Found

Sample Call

  curl 'http://localhost:9000/api/generic-backend/workflows/61a9e059587e8bd849fa2a7a' \
  -X 'DELETE' \
  -H 'Accept: application/json, text/plain, */*' \
  --compressed \
  --insecure

Execute a Workflow

URL: /api/generic-backend/workflows/{id}/start

METHOD: POST

Path Params:

id
- description: id of the workflow we want to start
- required: true
- type: string
- restriction: None

Query Params: None

Request Body: workflow model

Success Response

Code: 200 OK

Content: execution model : the execution that started

Error Response

Code: 500 Internal Server Error Something went wrong when starting the workflow, check PipelineOrchestrator logs

Code: 422 Unprocessable entity The body is not a valid workflow

Code: 404 Not Found The id doesn’t exists

Get All Executions

URL: /api/generic-backend/executions

METHOD: GET

Path Params: None

Query Params:

workflow
- description: filter executions by workflow id
- required: false
- type: string
- sample: http://localhost:9000/api/generic-backend/executions?workflow=workflow_id

Success Response

Code: 200 OK

Content: Execution[] Execution model

Error Response

Code: 500 Internal Server Error

Code: 422 Unprocessable entity

Sample Call

  curl 'http://localhost:9000/api/generic-backend/executions' \
  -H 'Accept: application/json, text/plain, */*' \
  --compressed \
  --insecure

Get One Execution

URL: /api/generic-backend/executions/{id}

METHOD: GET

Path Params:

id
- description: Id of the execution we want to get
- required: true
- type: string
- restriction: None

Query Params: None

Success Response

Code: 200 OK

Content: Execution

Error Response

Code: 404 Not Found

Sample Call

  curl 'http://localhost:9000/api/generic-backend/executions/6167e8458cc6bdd012912f10' \
  -H 'Accept: application/json, text/plain, */*' \
  --compressed \
  --insecure

Get Executions Status Count

URL: /api/generic-backend/executions/count

METHOD: GET

Path Params: None

Query Params:

workflow
- description: filter executions by workflow id
- required: false
- type: string
- sample: http://localhost:9000/api/generic-backend/executions/count?workflow=workflow_id
startTime
- description: filter executions returning only the ones started after start time
- required: false
- type: string
- restriction: A valid date time according to RFC3339
- sample: http://localhost:9000/api/generic-backend/executions/count?startTime=2021-12-02T09:56:48.828Z

Success Response

Code: 200 OK

Content:

  {
  "succeeded": 12,
  "failed": 0,
  "running": 4
}

Error Response

Code: 500 Internal Server Error

Code: 422 Unprocessable Entity

Sample Call

  curl 'http://localhost:9000/api/generic-backend/executions/statusCount?startTime=2021-12-02T09:56:48.828Z' \
  -H 'Accept: application/json, text/plain, */*' \
  --compressed \
  --insecure

Get All Favorites

URL: /api/generic-backend/favorites

METHOD: GET

Path Params: None

Query Params: None

Success Response

Code: 200 OK

Content: getFavoriteResponse[] model

Error Response

Code: 500 Internal Server Error

Sample Call

  curl 'http://localhost:9000/api/generic-backend/favorites/' \
  -H 'Accept: application/json, text/plain, */*' \
  --compressed \
  --insecure

Add Favorite

URL: /api/generic-backend/favorites

METHOD: POST

Path Params: None

Query Params: None

Request Body: Favorite

Example:

{
  "providerId": "computerprovider1-103.152.70.51",
  "name": "computerProvider",
  "providerType": "computing",
  "description": "enregistrement computer provider ",
  "endpointLocation": "/api/compute/",
  "workflowStructure": "",
  "originClusterName": "OfflineLocalCluster",
  "isLocal": true,
  "metadata": {}
}

Success Response

Code: 201 CREATED

Content: The added Favorite

 {
  "name": "computerProvider",
  "providerId": "computerprovider1-103.152.70.51",
  "providerType": "computing",
  "description": "enregistrement computer provider ",
  "originClusterName": "OfflineLocalCluster",
  "endpointLocation": "/api/compute/",
  "workflowStructure": "",
  "isLocal": true,
  "metadata": {}
}

Error Response

Code: 500 Internal Server Error

Code: 400 Bad Request The server was not able to parse the request body

Sample Call

  curl 'http://localhost:9000/api/generic-backend/favorites/' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'Content-Type: application/json' \
  --data-raw '{"providerId":"computerprovider1-104.155.72.251","name":"computerProvider","providerType":"computing","description":"enregistrement computer provider ","endpointLocation":"/api/compute/","workflowStructure":"","originClusterName":"OfflineLocalCluster","isLocal":true,"metadata":{}}' \
  --compressed \
  --insecure

Delete Favorite

URL: /api/generic-backend/favorites/{providerId}

METHOD: DELETE

Path Params:

providerId
- description: ProviderId of the favorite we want to delete
- required: true
- type: string

Query Params: None

Success Response

Code: 200 OK

Content: None

Error Response

Code: 500 Internal Server Error

Code: 400 Bad Request Unable to parse provider id

Sample Call

  curl 'http://localhost:9000/api/generic-backend/favorites/computerprovider1-104.155.72.251' \
  -X 'DELETE' \
  -H 'Accept: application/json, text/plain, */*' \
  --compressed \
  --insecure

Get Provider In Catalog By Id

URL: /api/generic-backend/catalog/{providerType}/find/{id}

METHOD: GET

Path Params:

providerType
- description: Type of the provider
- required: true
- type: “data” | “service” | “computing”
id
- description: id of the provider
- required: true
- type: string

Query Params: None

Success Response

Code: 200 OK

Content:

  {
  "id": "61644bd899ed40dfab763104",
  "providerId": "SentinelProviderDefault",
  "providerType": "data",
  "version": "0.1",
  "name": "Provider d'image sentinel",
  "description": "Description du provider sentinel",
  "metadata": null,
  "endpointLocation": "{{CLUSTER_HOST}}/api/provider/data/sentinel",
  "workflowStructure": {
    "label": "Sentinel",
    "class": "CommandLineTool",
    "id": "",
    "stdin": "",
    "stdout": "",
    "stderr": "",
    "doc": "",
    "successCodes": null,
    "temporaryFailCodes": null,
    "permanentFailCodes": null,
    "baseCommand": [
      "wget"
    ],
    "arguments": [
      {
        "isFilled": true,
        "position": 0,
        "prefix": "",
        "separate": true,
        "itemSeparator": "",
        "valueFrom": "--tries=1",
        "shellQuote": true
      },
      {
        "isFilled": true,
        "position": 0,
        "prefix": "",
        "separate": true,
        "itemSeparator": "",
        "valueFrom": "-O",
        "shellQuote": true
      },
      {
        "isFilled": true,
        "position": 0,
        "prefix": "",
        "separate": true,
        "itemSeparator": "",
        "valueFrom": "image.tif",
        "shellQuote": true
      }
    ],
    "inputs": {
      "boundingBox": {
        "label": "",
        "doc": "",
        "streamable": false,
        "loadContents": false,
        "inputBinding": {
          "isFilled": false,
          "position": 0,
          "prefix": "",
          "separate": false,
          "itemSeparator": "",
          "valueFrom": "",
          "shellQuote": false
        },
        "format": null,
        "type": {
          "type": "boundingBox"
        },
        "inputType": "definition"
      },
      "dateRange": {
        "label": "",
        "doc": "",
        "streamable": false,
        "loadContents": false,
        "inputBinding": {
          "isFilled": false,
          "position": 0,
          "prefix": "",
          "separate": false,
          "itemSeparator": "",
          "valueFrom": "",
          "shellQuote": false
        },
        "format": null,
        "type": {
          "type": "dateRange"
        },
        "inputType": "execution"
      },
      "url": {
        "label": "",
        "doc": "",
        "streamable": false,
        "loadContents": false,
        "inputBinding": {
          "isFilled": true,
          "position": 1,
          "prefix": "",
          "separate": true,
          "itemSeparator": "",
          "valueFrom": "",
          "shellQuote": true
        },
        "format": null,
        "type": {
          "type": "string"
        },
        "inputType": "orchestrator"
      }
    },
    "outputs": {
      "downloadedFile": {
        "label": "",
        "streamable": false,
        "doc": "",
        "format": null,
        "binding": {
          "loadContents": false,
          "glob": [
            "image.tif"
          ],
          "outputEval": ""
        },
        "type": {
          "type": "file"
        }
      }
    },
    "requirements": [
      {
        "type": "DockerRequirement",
        "dockerPull": "mwendler/wget:latest",
        "dockerLoad": "",
        "dockerFile": "",
        "dockerImport": "",
        "dockerImageId": "",
        "dockerOutputDirectory": ""
      }
    ]
  }
}

Error Response

Code: 500 Internal Server Error

Sample Call

  curl 'http://localhost:9000/api/generic-backend/catalog/data/find/SentinelProviderDefault' \
  -H 'Accept: application/json, text/plain, */*' \
  --compressed \
  --insecure

Search In Catalog

This is a websocket providing search functionalities for the catalog. After opening the socket the client have to write a request in it. Then the server will send 0 to n responses in the socket.

URL: /api/generic-backend/catalog/{providerType}/search

METHOD: GET

Path Params:

providerType
- description: Not used but required…
- required: true
- type: “data” | “service” | “computing”

Message Body:

providerType
- description: Type of providers we are looking for
- required: true
- type: “data” | “service” | “computing”
terms
- description: terms of the query
- required: true
- type: string

{
  "terms": "satellite",
  "providerType": "data"
}

Success Response

There can be multiple responses in the socket

Content:

  searchResponse[]

searchResponse model

Sample Call

  import {webSocket, WebSocketSubject} from 'rxjs/webSocket';

const socket = webSocket<any>({
  url: "ws://localhost:9000/api/generic-backend/catalog/{providerType}/search",
});
socket.next({
  terms: "satellite images",
  providerType: "data",
});
this.socket.subscribe(response => console.log("response"))

Docs

Generic Backend

Functional overview

Technical overview

Api

Get All workflows

Success Response

Error Response

Sample Call

Get One Workflow

Success Response

Error Response

Sample Call

Get Workflow By User

Success Response

Error Response

Sample Call

Create Workflow

Success Response

Error Response

Sample Call

Update Workflow

Success Response

Error Response

Sample Call

Delete Workflow

Success Response

Error Response

Sample Call

Execute a Workflow

Success Response

Error Response

Get All Executions

Success Response

Error Response

Sample Call

Get One Execution

Success Response

Error Response

Sample Call

Get Executions Status Count

Success Response

Error Response

Sample Call

Get All Favorites

Success Response

Error Response

Sample Call

Add Favorite

Success Response

Error Response

Sample Call

Delete Favorite

Success Response

Error Response

Sample Call

Get Provider In Catalog By Id

Success Response

Error Response

Sample Call

Search In Catalog

Success Response

Sample Call