Using Kubernetes behind a proxy

Using Kubernetes behind a proxy

Using Kubernetes behind a proxy can lead to all kind of problems. One problem that I encountered during my work with cloud native CI/CD and GtiOps tools like Argo CD or Tekton is that these frameworks need to access the internet (for example to poll git repositories for updates). Therefore they need to be configured to use the proxy.

Setting proxy variables using PodPresets

Configuring the proxies is done through the environment variables http_proxy, https_proxy and no_proxy (the later one used for addresses not the be used). Frameworks like Argo CD or Tekton can easily be installed by applying the manifest to the Kubernetes cluster as described here .
One solution to modify the deployments created would be to patch them after installation, or to download the manifest, add the environment entries and then do the apply.
An alternative solution is using PodPresets . PodPresets cen be used to set environment variables for every pod created in a namespace. Thus after a PodPreset is created in a namespace all Pods created after that will get these variables injected.
For proxy settings the PodPreset would look like this:

apiVersion: settings.k8s.io/v1alpha1
kind: PodPreset
metadata:
  name: http-proxy
  namespace: argocd
spec:
  env:
    - name: http_proxy
      value: "http://10.1.120.18:3128"
    - name: https_proxy
      value: "http://10.1.120.18:3128"
    - name: no_proxy
      value: "0.0.0.0,localhost,127.0.0.0/8,10.0.0.0/8,kubernetes.default.svc,argocd-dex-server,argocd-metrics,argocd-redis,argocd-repo-server,argocd-server,argocd-server-metrics"

Applying the example above will create the PodPreset in the namespace argocd which contain the appropriate proxy settings. Opening a shell in on of the pods confirms that after installation of Argo CD the variables are present:

$ kubectl exec -n argocd argocd-server-6fc7bfcf6-t5bsx -- env | grep proxy
http_proxy=http://10.1.120.18:3128
https_proxy=http://10.1.120.18:3128
no_proxy=0.0.0.0,localhost,127.0.0.0/8,10.0.0.0/8,kubernetes.default.svc,argocd-dex-server,argocd-metrics,argocd-redis,argocd-repo-server,argocd-server,argocd-server-metrics

Don’t forget no_proxy

After having set the proxy variables all http traffic is using the proxy, except traffic to one of the addresses specified in no_proxy. Since pods deployed as part of Argco CD or Tekton talk to each other using their services names, these names need to be added to the `no_proxy_ variable.
To get the list of addresses to exclude, the names of the services can be used:

$ kubectl get service -n argocd | cut -d' ' -f 1
NAME
argocd-dex-server
argocd-metrics
argocd-redis
argocd-repo-server
argocd-server
argocd-server-metrics

Advantages

Using a PodPreset to inject environment variables in every Pod in a namespace has the advantage that the existing manifests do not need to be modified prior to deployment or be patched afterwards. Thus new releases can be applied “as they are”.

Back to Home