Using Kustomize as a Post renderer for the AIS XNAT Helm Chart

Kustomize

Using a Helm Chart is a pretty awesome way to deploy Kubernetes infrastructure in a neatly packaged, release versioned way.
They can be updated from the upstream repo with a single line of code and for any customisations you want to add into the deployment you specify it in a values.yaml file.

Or at least that’s how it should work. As Helm is based on templates, sometimes a value is hardcoded into the template and you can’t change it in the values file.
Your only option would have been to download the git repo that the Helm chart is based on, edit the template file in question and run it locally.

The problem with this approach is that when a new Helm Chart is released, you have to download the chart again and then apply all of your updates.
This becomes cumbersome and negates the advantages of Helm.

Enter Kustomize. Kustomize can work in several ways but in this guide I will show you how to apply Kustomize as a post-renderer to update the template files to fit our environment.
This allows you to continue to use the Helm Charts from the repo AND customise the Helm Chart templates to allow successful deployment.

You can read more about the Kustomize project here:
https://kustomize.io/

Install Kustomize

Kustomize can be run as its own program using the kustomize build command or built into kubectl using kubectl kustomize. We are going to use the kustomize standalone binary.

Go here to install:
https://kubectl.docs.kubernetes.io/installation/kustomize/binaries/

Direct install:

curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh"  | bash

This downloads to whatever directory you are in for whatever Operating System you are using. Copy it to /usr/local/bin to use it system wide:

sudo cp kustomize /usr/local/bin

How Kustomize works

When using Kustomize as a post renderer, Kustomize inputs all of the Helm Charts configuration data for a particular Chart in conjunction with the values file you specify with your cluster specific details and then amends the templates and applies them on the fly afterwards. This is why it is called a post renderer.

Let’s break this down.

1. Helm template

In order to extract all of the Helm chart information, you can use the helm template command. In the case of our XNAT/AIS Helm chart, to extract all of this data into a file called all.yaml (can be any filename) you would run this command:

helm template xnat ais/xnat > all.yaml

You now have the complete configuration of your Helm Chart including all template files in one file - all.yaml.

2. `kustomization.yaml`

The next step is a kustomization.yaml file. This file must be called kustomization.yaml or Kustomize doesn’t work.
You create this and in it you specify your resources (inputs) - in our example, the resource will be all.yaml. The fantastic thing about Kustomize is you can add more resources in as well which combines with the Helm Chart to streamline deployment.

For instance, in my kustomization.yaml file I also specify a pv.yaml as another resource. This has information about creating Persistent Volumes for the XNAT deployment and creates the volumes with the deployment so I don’t have to apply this separately. You can do this for any resources you want to add to your deployment not included in the Helm chart.
Example using all.yaml and pv.yaml in the kustomization.yaml file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- all.yaml
- pv.yaml

The second part of the Kustomization.yaml file is where you specify the files that patch the templates you need to change.
You need to specify Filename and path, name of the original template, type and version. It should be pointed out there are a lot of other ways to use Kustomize - you can read about them in some of the included articles at the end of this guide.

Example:

patches:
- path: service-patch.yaml
  target:
    kind: Service
    name: xnat-xnat-web
    version: v1

In the above example, the file is service-patch.yaml and is in the same directory as kustomization.yaml, the name is xnat-xnat-web, the kind is Service and version is v1.
Now lets look at the original service.yaml file to get a better idea. It is located in charts/releases/xnat/charts/xnat-web/templates/service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: {{ include "xnat-web.fullname" . }}
  labels:
    {{- include "xnat-web.labels" . | nindent 4 }}
spec:
  type: {{ .Values.service.type }}
  #clusterIP: None
  ports:
    - port: {{ .Values.service.port }}
      targetPort: 8080
      protocol: TCP
      name: http
  selector:
    {{- include "xnat-web.selectorLabels" . | nindent 4 }}
  sessionAffinity: "ClientIP"
{{- if .Values.dicom_scp.recievers }}
---
apiVersion: v1
kind: Service
metadata:
  name: {{ include "xnat-web.fullname" . }}-dicom-scp
  labels:
    {{- include "xnat-web.labels" . | nindent 4 }}
  {{- with .Values.dicom_scp.annotations }}
  annotations:
    {{- toYaml . | nindent 4 }}
  {{- end }}
spec:
  type: {{ .Values.dicom_scp.serviceType | quote }}
  ports:
    {{- $serviceType := .Values.dicom_scp.serviceType }}
    {{- range .Values.dicom_scp.recievers }}
    - port: {{ .port }}
      targetPort: {{ .port }}
      {{- if and (eq $serviceType "NodePort") .nodePort }}
      nodePort: {{ .nodePort }}
      {{- end }}
      {{- if and (eq $serviceType "LoadBalancer") .loadBalancerIP }}
      loadBalancerIP: {{ .loadBalancerIP }}
      {{- end }}
    {{- end }}
  selector:
    {{- include "xnat-web.selectorLabels" . | nindent 4 }}
  sessionAffinity: "ClientIP"
{{- end }}

3. The Patch file

OK, so let’s have a look at our patch file and see what it is actually doing.

- op: remove
  path: "/spec/sessionAffinity"

Pretty simple really. - op: remove just removes whatever we tell it to in our service.yaml file. If we look through our file, we find spec and then under that we find sessionAffinity and then remove that.
In this case if we remove all the other code to simplify things you get this:

spec:
  sessionAffinity: "ClientIP"

As sessionAffinity is under spec by indentation it will remove the line:

sessionAffinity: "ClientIP"

In this particular case my AWS Cluster needs Service Type to be NodePort so this particular line causes the XNAT deployment to fail, hence the requirement to remove it.
OK so far so good. You can also use add and replace operations so let’s try an add command example as that is slightly more complicated.

Add and Replace commands example

OK continuing with our AWS NodePort example we will add a redirect from port 80 to 443 in the Ingress and replace the existing entry.
In order to do that we need to add a second host path to the charts/releases/xnat/charts/xnat-web/templates/ingress.yaml. Lets look at the original file:

{{- if .Values.ingress.enabled -}}
{{- $fullName := include "xnat-web.fullname" . -}}
{{- $svcPort := .Values.service.port -}}
apiVersion: networking.k8s.io/v1beta1
{{- end }}
kind: Ingress
metadata:
  name: {{ $fullName }}
  labels:
    {{- include "xnat-web.labels" . | nindent 4 }}
  {{- with .Values.ingress.annotations }}
  annotations:
    {{- toYaml . | nindent 4 }}
  {{- end }}
spec:
  {{- if .Values.ingress.tls }}
  tls:
    {{- range .Values.ingress.tls }}
    - hosts:
        {{- range .hosts }}
        - {{ . | quote }}
        {{- end }}
      secretName: {{ .secretName }}
    {{- end }}
  {{- end }}
  rules:
    {{- range .Values.ingress.hosts }}
    - host: {{ .host | quote }}
      http:
        paths:
          {{- range .paths }}
          - path: {{ .path }}
            backend:
              serviceName: {{ $fullName }}
              servicePort: {{ $svcPort }}
          {{- end }}
    {{- end }}
  {{- end }}

This is what we need in our values file to be reflected in the ingress.yaml file:

    hosts:
      - host: "xnat.example.com"
        paths: 
        - path: "/*"
          backend:
            serviceName: ssl-redirect
            servicePort: use-annotation
        - path: "/*"
          backend:
            serviceName: "xnat-xnat-web"
            servicePort: 80

And this is what we have at the moment in that file:

  rules:
    {{- range .Values.ingress.hosts }}
    - host: {{ .host | quote }}
      http:
        paths:
          {{- range .paths }}
          - path: {{ .path }}
            backend:
              serviceName: {{ $fullName }}
              servicePort: {{ $svcPort }}
          {{- end }}

As you can see, we are missing a second backend to allow the redirection from http to https.
In kustomization.yaml add the following:

- path: ingress-patch.yaml
  target:
    group: networking.k8s.io
    kind: Ingress
    name: xnat-xnat-web 
    version: v1beta1

# ingress-patch.yaml
#
- op: replace
  path: /spec/rules/0/http/paths/0/backend/serviceName
  value: 'ssl-redirect'
- op: replace
  path: /spec/rules/0/http/paths/0/backend/servicePort
  value: 'use-annotation'
- op: add
  path: /spec/rules/0/http/paths/-
  value: 
    path: '/*'
    backend: 
      serviceName: 'xnat-xnat-web'
      servicePort: 80

OK, so let’s break this down. The top command replaces this:

serviceName: {{ $fullName }}

In this path:

  rules:
      http:
        paths:
            backend:

With a hardcoded serviceName value:

serviceName: 'ssl-redirect'

I removed the extra lines to show you only the relevant section.

The second command replaces:

servicePort: {{ $svcPort }}

In the same path, with the hardcoded value:

servicePort: 'use-annotation'

Now for the add command.

- op: add
  path: /spec/rules/0/http/paths/-

This will add the values in normal yaml syntax here:

spec:
  rules:
      http:
        paths:
          -

NB. I have removed irrelevant lines to simplify the example. If there were already two sets of path directive, replacing or adding to the second one would require this path:

path: /spec/rules/1/http/paths/-

OK so the resultant transformation of the ingress.yaml file will change it to look like this:

spec:
  rules:
      http:
        paths: 
          backend:
            serviceName: ssl-redirect
            servicePort: use-annotation
        - path: '/*'
          backend:
            serviceName: 'xnat-xnat-web'
            servicePort: 80

Let’s look at our full kustomization.yaml file with resources and service and ingress patches.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- all.yaml
- pv.yaml
patches:
- path: service-patch.yaml
  target:
    kind: Service
    name: xnat-xnat-web
    version: v1
- path: ingress-patch.yaml
  target:
    group: networking.k8s.io
    kind: Ingress
    name: xnat-xnat-web 
    version: v1beta1

We are now ready to apply our kustomizations!

4. Bringing it all together

Create a new fle called whatever you like - and make it executable, in my case we will call it hook.sh.

vi hook.sh
chmod 755 hook.sh

#!/bin/bash
# hook.sh
#
cat <&0 > all.yaml
kustomize build && rm all.yaml

This takes the contents of all.yaml and kustomizes it using the kustomization.yaml file with the resources and patches I have previously described. Finally, it deletes all.yaml.
When you run kustomize build it will look for a file called kustomization.yaml to apply the transformations. As the kustomization.yaml file is in the same directory as hook.sh only the kustomize build command is needed, no further directive is required.

5. Deploy the Helm Chart with Kustomize post-renderer

OK to bring it all together and upgrade the XNAT AIS helm chart with your values file as values.yaml in the namespace xnat, run this command:

helm template xnat ais/xnat > all.yaml && \
  helm upgrade xnat ais/xnat -i -f values.yaml -nxnat --post-renderer=./hook.sh

In this case, you need to make sure that the following files are in the same directory:

values.yaml  
hook.sh  
kustomization.yaml  
ingress-patch.yaml
service-patch.yaml
pv.yaml

Feedback

Was this page helpful?

Glad to hear it! Please tell us how we can improve.

Sorry to hear that. Please tell us how we can improve.