Commit d8790d46 authored by Radu Carpa's avatar Radu Carpa
Browse files

daemons: rework the FTS renewal job 

The template still supports the old input, reverted to the behavior
from before the change done by Domenic. For everybody, except
escape, the change should be transparent, but old behavior is not
documented anymore. The old behavior is considered deprecated and
will be removed in a major release after announcing it to communities.

The new method of configuring the fts renewal job is much more
explicit. Requiring to manually mount secrets into the container
using the "secretMounts" (formally additionalSecrets), and passing
directly ENV variables required by whatever script is selected.
While this may seem as a downgrade in terms of usability, I would
argue it's much better this way. The old behavior had 2 major
problems:
- the user had to manually create, prior to running helm, secrets
  with very specific names. Which somebody could know only by
  looking at the source code of the helm chart. The work done by
  Domenic improved the situation, but made the following issue worse:
- the helm template is full of if/else which basically require
  per-vo specific configuration anyway. And the main way to know how
  to make that work was, obviously, to read the source code of both
  the helm chart and the container scripts.

This change decouples the "VO", from the script it uses to renew
certificates. This is important, because it should allow support
for multi-vo in the future and avoid duplicating the scripts (for
ex: the one from escape is basically a copy of the default script)
To support the new "script" argument, a small change will be needed
in the containers repository to pick the script based on this value
rather than using the RUCIO_VO env variable.

The commit also prepares the path for multi-vo configurations.
"vos" is now a list instead of being a string as before. And each
vo name+voms are exported via a separate env variable to the
containers. For reverse compatibility with existing scripts, the
first VO in the list is (also) exported via RUCIO_VO env var.

The commit also mounts the rucio configuration from the helm
"values.config" into the pod. While it's not required now, this may
allow, in the future, to change the behavior of the fts renewal
script based on rucio configuration. This could probably be used
for multi-vo, where each vo could have a specific configuration.
parent b0edc694

charts/rucio-daemons/Chart.yaml

+1 −2
Original line number Diff line number Diff line 
name: rucio-daemons

version: 1.30.4

version: 1.30.5

apiVersion: v1

description: A Helm chart to deploy daemons for Rucio

keywords:
@@ -11,4 +11,3 @@ sources: 
maintainers:

  - name: Rucio development team

    email: rucio-dev@cern.ch

charts/rucio-daemons/README.md

+26 −29
Original line number Diff line number Diff line
@@ -61,36 +61,33 @@ The conveyor needs a delegated X509 user proxy and the necessary CA so that it c 
        repository: rucio/fts-cron

        tag: latest

        pullPolicy: Always

      vo: "cms"

      voms: "cms:/cms/Role=production"

      gridPassphrase:

        required: false

        existingSecret:

          name: 'grid-passphrase'

          key: 'passphrase'

      servers: "https://fts3-devel.cern.ch:8446,https://fts3-pilot.cern.ch:8446"

      ftsCert:

        existingSecret:

          name: '' # e.g., fts-cert

          key: '' # e.g., usercert.pem

      ftsKey:

        existingSecret:

          name: '' # e.g., fts-key

          key: '' # e.g., userkey.pem

      longProxy: ''

      ftsLongProxy:

        existingSecret:

          name: '' # e.g., long-proxy

          key: '' # e.g., long.proxy



The possible VOs are:



- `atlas` expects a long proxy as input secret. It then creates a user proxy with the given VOMS extensions and with 96h lifetime and saves it as a cluster secret (`<releasename>-rucio-x509up`).

- `cms` expects a separate key and cert as input secrets. It then creates a user proxy with the given VOMS extensions and with 96h lifetime and delegates it to the given FTS servers. Then saves it as a cluster secret (`<releasename>-rucio-x509up`).

- `escape` expects a separate key and cert as input secrets, as well as a grid passphrase called. It then creates a user proxy with the given VOMS extensions and with 96h lifetime and delegates it to the given FTS servers. Then saves it as a cluster secret (`<releasename>-rucio-x509up`).

- `dteam` expects a long proxy like `atlas` and then creates, delegates and saves the user proxy like `cms`.

- `tutorial` expects a separate key and cert as input secrets like `cms` and then directly delegates to FTS. No proxy generation and `<releasename>-rucio-x509up` has to be manually created.

- Any other VO value will lead to the execution of the default script and expects a separate key and cert as input secrets. It then creates a user proxy with the given VOMS extensions and with 96h lifetime and delegates it to the given FTS servers. Then saves it as a cluster secret (`<releasename>-rucio-x509up`). Additionally a grid passphrase can be specified and saved in a dedicated secret.

      script: default

      vos:

        - vo: "cms"

          voms: "cms:/cms/Role=production"



Please check directly the scripts in the [fts-cron](https://github.com/rucio/containers/tree/master/fts-cron)

container to see their required input. For example, the "atlas" script requires

a proxy certificate (longproxy) to be mounted into the pod at the correct

location. And it will be used to generate a short proxy into the kubernetes 

secret with the name given in the `RUCIO_FTS_SECRETS` env variable. 

The configuration will be like that:



      script: atlas

      vos:

        - vo: "atlas"

          voms: "atlas:/atlas/Role=production"

        secretMounts:

          - secretFullName: release-longproxy

            mountPath: /opt/rucio/certs/long.proxy

            subPath: long.proxy

        additionalEnvs:

          - name: RUCIO_LONG_PROXY

            value: long.proxy

          - name: RUCIO_FTS_SECRETS

            value: release-rucio-x509up





### Reaper

charts/rucio-daemons/templates/renew-fts-cronjob.yaml

+48 −22
Original line number Diff line number Diff line
@@ -2,17 +2,29 @@ 
{{- if .Values.ftsRenewal.enabled }}

  serviceAccountName: {{ .Release.Name }}-rucio-edit

  volumes:

  - name: config-common

    secret:

      secretName: {{ template "rucio.fullname" . }}.config.common

  {{- range $collection := tuple .Values.secretMounts .Values.ftsRenewal.secretMounts .Values.additionalSecrets .Values.ftsRenewal.additionalSecrets }}

  {{- range $key, $val := $collection }}

  - name: {{ kindIs "int" $key | ternary (coalesce $val.volumeName $val.secretName $val.secretFullName) $key }}

    secret:

      secretName: {{ coalesce $val.secretFullName (printf "%s-%s" $.Release.Name $val.secretName) }}

  {{- end}}

  {{- end}}

  {{- if .Values.useDeprecatedImplicitSecrets }}

  {{- if or (eq .Values.ftsRenewal.vo "atlas") (eq .Values.ftsRenewal.vo "dteam") }}

  - name: longproxy

    secret:

      secretName: {{ if empty .Values.ftsRenewal.ftsLongProxy.existingSecret.name }} {{ .Release.Name }}-longproxy {{- else }} {{ .Values.ftsRenewal.ftsLongProxy.existingSecret.name }} {{ end }}

      secretName: {{ .Release.Name }}-longproxy

  {{- else }}

  - name: usercert

    secret:

      secretName: {{ if empty .Values.ftsRenewal.ftsCert.existingSecret.name }} {{ .Release.Name }}-fts-cert {{- else }} {{ .Values.ftsRenewal.ftsCert.existingSecret.name }} {{ end }}

      secretName: {{ .Release.Name }}-fts-cert

  - name: userkey

    secret:

      secretName: {{ if empty .Values.ftsRenewal.ftsKey.existingSecret.name }} {{ .Release.Name }}-fts-key {{- else }} {{ .Values.ftsRenewal.ftsKey.existingSecret.name }} {{ end }}

      secretName:  {{ .Release.Name }}-fts-key

  {{- end }}

  {{- end }}

  {{- range $key, $val := .Values.persistentVolumes }}

  - name: {{ $key }}
@@ -26,6 +38,10 @@ 
      resources:

{{ toYaml .Values.ftsRenewal.resources | indent 8 }}

      volumeMounts:

        - name: config-common

          mountPath: /opt/rucio/etc/conf.d/10_common.json

          subPath: common.json

  {{- if .Values.useDeprecatedImplicitSecrets }}

  {{- if or (eq .Values.ftsRenewal.vo "atlas") (eq .Values.ftsRenewal.vo "dteam") }}

        - name: longproxy

          mountPath: /opt/rucio/certs/
@@ -35,6 +51,16 @@ 
        - name: userkey

          mountPath: /opt/rucio/keys/

  {{- end }}

  {{- end }}

  {{- range $collection := tuple .Values.secretMounts .Values.ftsRenewal.secretMounts .Values.additionalSecrets .Values.ftsRenewal.additionalSecrets }}

  {{- range $key, $val := $collection }}

        - name: {{ kindIs "int" $key | ternary (coalesce $val.volumeName $val.secretName $val.secretFullName) $key }}

          mountPath: {{ $val.mountPath }}

          {{- if $val.subPath }}

          subPath: "{{ $val.subPath }}"

          {{- end }}

  {{- end }}

  {{- end }}

  {{- range $key, $val := .Values.persistentVolumes }}

        - name: {{ $key }}

          mountPath: {{ $val.mountPath }}
@@ -45,33 +71,33 @@ 
          value: "{{ $val1  }}"

        {{- end}}

        - name: RUCIO_VO

          value: {{ .Values.ftsRenewal.vo | quote }}

          value: {{ coalesce .Values.ftsRenewal.vo (first .Values.ftsRenewal.vos).vo | quote }}

        - name: RUCIO_FTS_VOMS

          value: {{ .Values.ftsRenewal.voms | quote }}

          value: {{ coalesce .Values.ftsRenewal.voms (first .Values.ftsRenewal.vos).voms | quote }}

  {{- if not .Values.ftsRenewal.vo}}

        - name: RUCIO_FTS_SCRIPT

          value: "{{ .Values.ftsRenewal.script }}"

        - name: RUCIO_FTS_VO_COUNT

          value: {{ len .Values.ftsRenewal.vos }}

        {{- range $key, $val := .Values.ftsRenewal.vos }}

        - name: RUCIO_FTS_VO_{{ $key }}

          value: {{ $val.vo }}

        - name: RUCIO_FTS_VOMS_{{ $key }}

          value: {{ $val.voms }}

        {{- end }}

  {{- end }}

        - name: RUCIO_FTS_SERVERS

          value: {{ .Values.ftsRenewal.servers | quote }}

  {{- if .Values.useDeprecatedImplicitSecrets }}

        - name: RUCIO_FTS_SECRETS

          value: "{{ .Release.Name }}-rucio-x509up"

  {{- if .Values.ftsRenewal.gridPassphrase.required }}

        - name: GRID_PASSPHRASE

          valueFrom:

            secretKeyRef:

              name: {{ .Values.ftsRenewal.gridPassphrase.existingSecret.name | quote }}

              key: {{ .Values.ftsRenewal.gridPassphrase.existingSecret.key | quote }}

  {{- end }}

  {{- if or (eq .Values.ftsRenewal.vo "atlas") (eq .Values.ftsRenewal.vo "dteam") }}

    {{- if .Values.ftsRenewal.longProxy }}

        - name: RUCIO_LONG_PROXY

          value: {{ .Values.ftsRenewal.longProxy | quote }}

    {{- else }}

        - name: RUCIO_LONG_PROXY

          value: {{ .Values.ftsRenewal.ftsLongProxy.existingSecret.key | default "long.proxy" | quote }}

  {{- end }}

  {{- else }}

        - name: USERCERT_NAME

          value: {{ .Values.ftsRenewal.ftsCert.existingSecret.key | default "usercert.pem" | quote }}

        - name: USERKEY_NAME

          value: {{ .Values.ftsRenewal.ftsKey.existingSecret.key | default "new_userkey.pem" | quote }}

  {{- end }}

  {{- with .Values.ftsRenewal.additionalEnvs }}

{{ toYaml . | indent 8 }}

  {{- end}}

  restartPolicy: OnFailure

{{- end }}

charts/rucio-daemons/values.yaml

+28 −21
Original line number Diff line number Diff line
@@ -398,27 +398,35 @@ ftsRenewal: 
    repository: rucio/fts-cron

    tag: latest

    pullPolicy: Always

  vo: "cms"

  voms: "cms:/cms/Role=production"

  gridPassphrase:

    required: false

    existingSecret:

      name: 'grid-passphrase'

      key: 'passphrase'

  servers: "https://fts3-devel.cern.ch:8446,https://cmsfts3.fnal.gov:8446,https://fts3.cern.ch:8446,https://lcgfts3.gridpp.rl.ac.uk:8446,https://fts3-pilot.cern.ch:8446"

  ftsCert:

    existingSecret:

      name: '' # e.g., fts-cert

      key: '' # e.g., usercert.pem

  ftsKey:

    existingSecret:

      name: '' # e.g., fts-key

      key: '' # e.g., userkey.pem

  longProxy: ''

  ftsLongProxy:

    existingSecret:

      name: '' # e.g., long-proxy

      key: '' # e.g., long.proxy

  script: 'default'  # one of: 'default', 'atlas', 'dteam', 'multi_vo', 'tutorial', 'escape'. The associated scripts can be found here: https://github.com/rucio/containers/tree/master/fts-cron

  vos:

    - vo: "cms"

      voms: "cms:/cms/Role=production"

  secretMounts: {}

    # - secretName: fts-cert

    #   mountPath: /opt/rucio/certs/usercert.pem

    #   subPath: usercert.pem

    # - secretName: fts-key

    #   mountPath: /opt/rucio/certs/new_userkey.pem

    #   subPath: new_userkey.pem

    # - secretName: longproxy

    #   mountPath: /opt/rucio/certs/long.proxy

    #   subPath: long.proxy

  additionalEnvs: {}

    # - name: RUCIO_FTS_SECRETS

    #   value: release-rucio-x509up

    # - name: USERCERT_NAME

    #   value: "usercert.pem"

    # - name: USERKEY_NAME

    #   value: "new_userkey.pem"

    # - name: RUCIO_LONG_PROXY

    #   value: long.proxy

    # - name: GRID_PASSPHRASE

    #   valueFrom:

    #     secretKeyRef:

    #       name:

    #       key:

  resources:

    limits:

      cpu: 500m
@@ -582,4 +590,3 @@ config: 
  # credentials:

    # gcs: "/opt/rucio/etc/google-cloud-storage-test.json"

    # signature_lifetime: "3600"