Skip to content

Harmonise toil and calrissian cookiecutter template #97

@spinto

Description

@spinto

It would be best if we can provide some level of harmonization between the Toil and Calrissian service templates, and thus in the values and configuration in the zoo helm chart.

This is because we have already 2 processing engines (Toil and Calrissian), and in the future we may have Argo (Calrissian version), Argo (native CWL version), and KNative , and thus the deployment may become complicated...

This should be also put in context with the integration with the workspace, which can be enabled or not...

Ideally, something like this should be in the values-template:

[...]
  #Definition of the cookiecutter template to use according to the processing engine
cookiecutter:
  {{- if eq (getenv "OAPIP_EXECUTION_ENGINE") "calrissian" }}
  templateUrl: https://github.com/EOEPCA/eoepca-proc-service-template.git
  # templateBranch: feature/python3.8
  {{- else if eq (getenv "OAPIP_EXECUTION_ENGINE") "toil" }}
  templateUrl: https://github.com/EOEPCA/eoepca-proc-service-template-wes.git
  templateBranch: main
  {{- end }}
[...]
  #Definition of the specific execution engine configuration
  inputs:
  {{- if eq (getenv "OAPIP_EXECUTION_ENGINE") "toil" }}
    WES_URL: {{ getenv "OAPIP_TOIL_WES_URL" }}
    WES_USER: {{ getenv "OAPIP_TOIL_WES_USER" }}
    WES_PASSWORD: {{ getenv "OAPIP_TOIL_WES_PASSWORD" }}
  {{- else if eq (getenv "OAPIP_EXECUTION_ENGINE") "calrissian" }}
    {{ getenv "NODE_SELECTOR_KEY" }}: "{{ getenv "NODE_SELECTOR_VALUE" }}"
  {{- end }}
[...]
  #Definition of the integration with the storage for stagein and/or stageout
  additionalInputs:
  {{- if eq (getenv "OAIP_USE_WORKSPACE") "true" }}
    WORKSPACE_SERVICE_URL: {{ getenv "WORKSPACE_SERVICE_URL" }}
  {{- else }}
    STAGEIN_AWS_SERVICEURL: {{ getenv "STAGEIN_S3_ENDPOINT" }}
    STAGEIN_AWS_ACCESS_KEY_ID: {{ getenv "STAGEIN_S3_ACCESS_KEY" }}
    STAGEIN_AWS_SECRET_ACCESS_KEY: {{ getenv "STAGEIN_S3_SECRET_KEY" }}
    STAGEIN_AWS_REGION: {{ getenv "STAGEIN_S3_REGION" }}
    STAGEOUT_AWS_SERVICEURL: {{ getenv "S3_ENDPOINT" }}
    STAGEOUT_AWS_ACCESS_KEY_ID: {{ getenv "S3_ACCESS_KEY" }}
    STAGEOUT_AWS_SECRET_ACCESS_KEY: {{ getenv "S3_SECRET_KEY" }}
    STAGEOUT_AWS_REGION: {{ getenv "S3_REGION" }}
    STAGEOUT_OUTPUT: eoepca
  {{- end }}
[...]

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions