Proxying Your Way into a Private Kubernetes Cluster Using Terraform
Many organizations leverage private Kubernetes clusters for enhanced security and control. However, managing resources within these isolated environments, especially with Terraform running externally, can be challenging. This blog post explores how to utilize SOCKS5 proxies to bridge the gap and securely manage your private Kubernetes clusters using Terraform.
SOCKS5 Proxy to the Rescue
The SOCKS5 protocol enables TCP traffic forwarding, making it ideal for tunneling connections. Both the Kubernetes and Helm Terraform provider support SOCKS5, allowing you to channel requests to your private cluster.
We’ll establish a SOCKS5 proxy server using SSH and configure the Kubernetes Terraform provider to interact with the cluster through this secure tunnel. While the examples utilize Google Cloud, the concepts are adaptable to other cloud providers.
Creating a SOCKS5 Proxy Server
Terraform allows provisioning a VM and executing a cloud-init script during creation. This script will enable SSH tunneling:
#cloud-config
runcmd:
- [ sed, -i, 's/PermitTunnel no/PermitTunnel yes/g', /etc/ssh/sshd_config ]
- [ systemctl, restart, sshd ]
This script modifies the SSH daemon configuration to permit tunneling and restarts the service for the changes to take effect.
Establishing a Secure Tunnel
Google Cloud users can leverage the gcloud CLI to create a secure tunnel to the proxy server:
CLOUDSDK_PYTHON_SITEPACKAGES=1 gcloud compute ssh <instance-name> \
--project=<project-name> \
--zone=<instance-zone> \
--tunnel-through-iap \
--ssh-flag="-N -f -D 8888" \
This command establishes a secure tunnel to the proxy server on port 8888. The --tunnel-through-iap
flag ensures a secure connection via Identity-Aware Proxy (IAP), eliminating internet exposure for your VM instances. The --ssh-flag
allows passing additional SSH flags:
-N
: Disables remote command execution.-f
: Runs the process in the background.-D 8888
: Creates a dynamic SOCKS5 proxy on port 8888.
To terminate the tunnel, use:
kill -9 $(shell lsof 8888 > /dev/null 2> /dev/null || :
Test the tunnel with:
HTTPS_PROXY=socks5://127.0.0.1:8888 kubectl cluster-info
Configuring the Kubernetes Terraform Provider
Set the proxy_url
argument within the provider block to configure Terraform to utilize the SOCKS5 proxy:
provider "kubernetes" {
...
proxy_url = "socks5://127.0.0.1:8888"
...
}
The proxy_url
argument is the key takeaway here. It references the locally secured tunnel established through IAP. Configuring the Helm Terraform provider follows a similar approach.
Conclusion
By setting up a SOCKS5 proxy server and configuring your Terraform providers accordingly, you gain the ability to securely manage private Kubernetes clusters from any external environment.