Bruno Schaatsbergen website Mastodon PGP Key email A drawing of an astronaut in space The Netherlands

Proxying Your Way into a Private Kubernetes Cluster Using Terraform

in
writing
date
4/28/2024

Many organizations leverage private Kubernetes clusters for enhanced security and control. However, managing resources within these isolated environments, especially with Terraform running externally, can be challenging. This blog post explores how to utilize SOCKS5 proxies to bridge the gap and securely manage your private Kubernetes clusters using Terraform.

SOCKS5 Proxy to the Rescue

The SOCKS5 protocol enables TCP traffic forwarding, making it ideal for tunneling connections. Both the Kubernetes and Helm Terraform provider support SOCKS5, allowing you to channel requests to your private cluster.

We’ll establish a SOCKS5 proxy server using SSH and configure the Kubernetes Terraform provider to interact with the cluster through this secure tunnel. While the examples utilize Google Cloud, the concepts are adaptable to other cloud providers.

Creating a SOCKS5 Proxy Server

Terraform allows provisioning a VM and executing a cloud-init script during creation. This script will enable SSH tunneling:

#cloud-config

runcmd:
  - [ sed, -i, 's/PermitTunnel no/PermitTunnel yes/g', /etc/ssh/sshd_config ]
  - [ systemctl, restart, sshd ]

This script modifies the SSH daemon configuration to permit tunneling and restarts the service for the changes to take effect.

Establishing a Secure Tunnel

Google Cloud users can leverage the gcloud CLI to create a secure tunnel to the proxy server:

CLOUDSDK_PYTHON_SITEPACKAGES=1 gcloud compute ssh <instance-name> \
  --project=<project-name> \
  --zone=<instance-zone> \
  --tunnel-through-iap \
  --ssh-flag="-N -f -D 8888" \

This command establishes a secure tunnel to the proxy server on port 8888. The --tunnel-through-iap flag ensures a secure connection via Identity-Aware Proxy (IAP), eliminating internet exposure for your VM instances. The --ssh-flag allows passing additional SSH flags:

  • -N: Disables remote command execution.
  • -f: Runs the process in the background.
  • -D 8888: Creates a dynamic SOCKS5 proxy on port 8888.

To terminate the tunnel, use:

kill -9 $(shell lsof 8888 > /dev/null 2> /dev/null || :

Test the tunnel with:

HTTPS_PROXY=socks5://127.0.0.1:8888 kubectl cluster-info

Configuring the Kubernetes Terraform Provider

Set the proxy_url argument within the provider block to configure Terraform to utilize the SOCKS5 proxy:

provider "kubernetes" {
  ...
  proxy_url = "socks5://127.0.0.1:8888"
  ...
}

The proxy_url argument is the key takeaway here. It references the locally secured tunnel established through IAP. Configuring the Helm Terraform provider follows a similar approach.

Conclusion

By setting up a SOCKS5 proxy server and configuring your Terraform providers accordingly, you gain the ability to securely manage private Kubernetes clusters from any external environment.

/proxying-your-way-into-a-private-kubernetes-cluster-using-terraform