Unverified Commit 2aecc04d authored by Domenic Gosein's avatar Domenic Gosein Committed by GitHub
Browse files

enhancement: improve fts cron helm debugging Fix rucio/containers#215 (#131) 

* cherry pick changes from original PR

* change longProxy logic, remove quote from secret names and test locally

* cleanup

* cleanup again
parent 8a24ba24

charts/rucio-daemons/Chart.yaml

+1 −1
Original line number Diff line number Diff line 
name: rucio-daemons

version: 1.30.3

version: 1.30.4

apiVersion: v1

description: A Helm chart to deploy daemons for Rucio

keywords:

charts/rucio-daemons/README.md

+24 −7
Original line number Diff line number Diff line
@@ -52,28 +52,45 @@ Some of the daemons require certificates and CAs to work. They expect specific s 


### Conveyor



The conveyor needs a delegated X509 user proxy and the necessary CA so that it can submit jobs to FTS. For the CA you have to add a `<releasename>-rucio-ca-bundle` secret. For the user proxy a cronjob can be setup to either generate it from a long proxy or directly delegate the user proxy to FTS. The cronjob uses the [fts-cron](https://github.com/rucio/containers/tree/master/fts-cron) container which expects different input secrets and has a different behaviour depending on the selected VO. The cronjob run every 6 hours. An example configuration looks like this:

The conveyor needs a delegated X509 user proxy and the necessary CA so that it can submit jobs to FTS. For the CA you have to add a `<releasename>-rucio-ca-bundle` secret. For the user proxy a cronjob can be setup to either generate it from a long proxy or directly delegate the user proxy to FTS. The cronjob uses the [fts-cron](https://github.com/rucio/containers/tree/master/fts-cron) container which expects different input secrets and has a different behaviour depending on the selected VO. When enabled, the cronjob runs once upon installation and then every 6 hours. An example configuration looks like this:



    ftsRenewal:

      enabled: 1

      schedule: "12 */6 * * *"

      image:

        repository: rucio/fts-cron

        tag: latest

        pullPolicy: Always

      vo: "cms"

      voms: "cms:/cms/Role=production"

      gridPassphrase:

        required: false

        existingSecret:

          name: 'grid-passphrase'

          key: 'passphrase'

      servers: "https://fts3-devel.cern.ch:8446,https://fts3-pilot.cern.ch:8446"

      ftsCert:

        existingSecret:

          name: '' # e.g., fts-cert

          key: '' # e.g., usercert.pem

      ftsKey:

        existingSecret:

          name: '' # e.g., fts-key

          key: '' # e.g., userkey.pem

      longProxy: ''

      ftsLongProxy:

        existingSecret:

          name: '' # e.g., long-proxy

          key: '' # e.g., long.proxy



The possible VOs are:



- `atlas` expects a long proxy as input secret (`<releasename>-longproxy`). It then creates a user proxy with the given VOMS extensions and with 96h lifetime and saves it as a cluster secret (`<releasename>-rucio-x509up`).

- `cms` expects a separate key and cert as input secrets (`<releasename>-fts-key` and `<releasename>-fts-cert`). It then creates a user proxy with the given VOMS extensions and with 96h lifetime and delegates it to the given FTS servers. Then saves it as a cluster secret (`<releasename>-rucio-x509up`).

- `atlas` expects a long proxy as input secret. It then creates a user proxy with the given VOMS extensions and with 96h lifetime and saves it as a cluster secret (`<releasename>-rucio-x509up`).

- `cms` expects a separate key and cert as input secrets. It then creates a user proxy with the given VOMS extensions and with 96h lifetime and delegates it to the given FTS servers. Then saves it as a cluster secret (`<releasename>-rucio-x509up`).

- `escape` expects a separate key and cert as input secrets, as well as a grid passphrase called. It then creates a user proxy with the given VOMS extensions and with 96h lifetime and delegates it to the given FTS servers. Then saves it as a cluster secret (`<releasename>-rucio-x509up`).

- `dteam` expects a long proxy like `atlas` and then creates, delegates and saves the user proxy like `cms`.

- `tutorial` expects a separate key and cert as input secrets like `cms` and then directly delegates to FTS. No proxy generation and `<releasename>-rucio-x509up` has to be manually created.



When the release is first installed the pods won't start since the necessary `<releasename>-rucio-x509up` secret hasn't been created by the cronjob, yet. In that case you can run the job once manually:



    kubectl create job renew-manual-1 --from=cronjob/<releasename>-renew-fts-proxy

- Any other VO value will lead to the execution of the default script and expects a separate key and cert as input secrets. It then creates a user proxy with the given VOMS extensions and with 96h lifetime and delegates it to the given FTS servers. Then saves it as a cluster secret (`<releasename>-rucio-x509up`). Additionally a grid passphrase can be specified and saved in a dedicated secret.



### Reaper

charts/rucio-daemons/templates/renew-fts-cronjob.yaml

+20 −3
Original line number Diff line number Diff line
@@ -5,14 +5,14 @@ 
  {{- if or (eq .Values.ftsRenewal.vo "atlas") (eq .Values.ftsRenewal.vo "dteam") }}

  - name: longproxy

    secret:

      secretName: {{ .Release.Name }}-longproxy

      secretName: {{ if empty .Values.ftsRenewal.ftsLongProxy.existingSecret.name }} {{ .Release.Name }}-longproxy {{- else }} {{ .Values.ftsRenewal.ftsLongProxy.existingSecret.name }} {{ end }}

  {{- else }}

  - name: usercert

    secret:

      secretName: {{ .Release.Name }}-fts-cert

      secretName: {{ if empty .Values.ftsRenewal.ftsCert.existingSecret.name }} {{ .Release.Name }}-fts-cert {{- else }} {{ .Values.ftsRenewal.ftsCert.existingSecret.name }} {{ end }}

  - name: userkey

    secret:

      secretName: {{ .Release.Name }}-fts-key

      secretName: {{ if empty .Values.ftsRenewal.ftsKey.existingSecret.name }} {{ .Release.Name }}-fts-key {{- else }} {{ .Values.ftsRenewal.ftsKey.existingSecret.name }} {{ end }}

  {{- end }}

  {{- range $key, $val := .Values.persistentVolumes }}

  - name: {{ $key }}
@@ -52,9 +52,26 @@ 
          value: {{ .Values.ftsRenewal.servers | quote }}

        - name: RUCIO_FTS_SECRETS

          value: "{{ .Release.Name }}-rucio-x509up"

  {{- if .Values.ftsRenewal.gridPassphrase.required }}

        - name: GRID_PASSPHRASE

          valueFrom:

            secretKeyRef:

              name: {{ .Values.ftsRenewal.gridPassphrase.existingSecret.name | quote }}

              key: {{ .Values.ftsRenewal.gridPassphrase.existingSecret.key | quote }}

  {{- end }}

  {{- if or (eq .Values.ftsRenewal.vo "atlas") (eq .Values.ftsRenewal.vo "dteam") }}

    {{- if .Values.ftsRenewal.longProxy }}

        - name: RUCIO_LONG_PROXY

          value: {{ .Values.ftsRenewal.longProxy | quote }}

    {{- else }}

        - name: RUCIO_LONG_PROXY

          value: {{ .Values.ftsRenewal.ftsLongProxy.existingSecret.key | default "long.proxy" | quote }}

    {{- end }}

  {{- else }}

        - name: USERCERT_NAME

          value: {{ .Values.ftsRenewal.ftsCert.existingSecret.key | default "usercert.pem" | quote }}

        - name: USERKEY_NAME

          value: {{ .Values.ftsRenewal.ftsKey.existingSecret.key | default "new_userkey.pem" | quote }}

  {{- end }}

  restartPolicy: OnFailure

{{- end }}

charts/rucio-daemons/values.yaml

+18 −0
Original line number Diff line number Diff line
@@ -398,7 +398,25 @@ ftsRenewal: 
    pullPolicy: Always

  vo: "cms"

  voms: "cms:/cms/Role=production"

  gridPassphrase:

    required: false

    existingSecret:

      name: 'grid-passphrase'

      key: 'passphrase'

  servers: "https://fts3-devel.cern.ch:8446,https://cmsfts3.fnal.gov:8446,https://fts3.cern.ch:8446,https://lcgfts3.gridpp.rl.ac.uk:8446,https://fts3-pilot.cern.ch:8446"

  ftsCert:

    existingSecret:

      name: '' # e.g., fts-cert

      key: '' # e.g., usercert.pem

  ftsKey:

    existingSecret:

      name: '' # e.g., fts-key

      key: '' # e.g., userkey.pem

  longProxy: ''

  ftsLongProxy:

    existingSecret:

      name: '' # e.g., long-proxy

      key: '' # e.g., long.proxy

  resources:

    limits:

      cpu: 500m