Migrating out of GDC air-gapped in an emergency

Disclaimer. This is a personal thought exercise, written from public documentation on my own time. It is not affiliated with, endorsed by, or done on behalf of my employer, and nothing here represents my employer’s work, views, or internal systems.

Google Distributed Cloud (GDC) air-gapped is Google Cloud you run yourself, in a sealed facility, with no connection to Google or the internet. It is meant for workloads that cannot use a public network: classified systems, defense, or data that a regulator says must stay in the building. You get the Google Cloud APIs, running on hardware you control.

Underneath it is Kubernetes. Every resource GDC manages, a cluster, a database, a VM, a network, is a CRD under a *.gdc.goog API group. You create a Cluster and GDC’s controllers build it.

You can get the estate out. Export the *.gdc.goog objects to manifests and you have a full snapshot: every cluster, database, and network, the way GDC recorded them. That snapshot is only useful inside GDC. A GKE cluster in public Google Cloud has never seen those CRDs, so if you apply the export to one, the API server rejects every object.

None of this matters until the site is gone. Air-gapped means there is no warm standby and no copy in another region. If the facility is lost, the export is all you have, and there is nothing left that can apply it. I want a way to turn that export into running infrastructure in public Google Cloud when that happens.

This is break-glass recovery for a site that is already lost, not a resilience or failover feature. Whether a classified estate should run on public Google Cloud at all is a decision whose consequences live outside the tooling.

A B-2 Spirit stealth bomber in flight over open ocean
A B-2 Spirit stealth bomber, the aircraft built to carry the largest bunker busters. U.S. Air Force photo by Staff Sgt. Bennie J. Davis III (public domain).

the controller

If I were to build this, I would write a controller with kubebuilder, watching every *.gdc.goog kind and, for each object, creating the closest Config Connector resource. Config Connector then creates the real resource in a GCP project. A GDC Cluster becomes a ContainerCluster, a DBCluster becomes a Cloud SQL SQLInstance, and a HarborInstance, GDC’s container registry, becomes an Artifact Registry ArtifactRegistryRepository. You run the controller in a GKE cluster next to Config Connector, apply the export, and it rebuilds the estate on public infrastructure.

GDC has around 260 kinds, so writing one controller per kind does not scale. Discover the installed *.gdc.goog CRDs at startup and run one generic reconciler for all of them. Only the per-kind mapping is different.

Keep each mapping a pure function: a source object goes in, Config Connector objects come out, with no client calls and no writes. All side effects, the reads, the applies, the status updates, stay in the controller. A pure mapping is easy to test: you assert the exact resource that a kind produces, and the test fails when a GDC release changes a field. Because the controller is really just a mapper, Chainsaw is a good fit for testing it continuously: apply a GDC object and assert the Config Connector objects that come out. Register the mappings in a table, so adding a kind is one new file.

Make it deterministic. Derive every GCP name from the source object, so applying the same export twice is idempotent. When a name has to change to be valid, add a hash of the original, so two different sources never get the same name by accident. Allocate CIDR ranges from a fixed pool, keyed by the object, so the same export always produces the same addresses. The result must not depend on the order you apply things or the day you run it.

Some inputs are not in the GDC API at all: the GCP region to use, the GCE machine type for a GDC machine SKU, the CIDR pool, and the target VPC. Put those in a dictionary that you fill in beforehand. Make it fail early and clearly: the controller should refuse to start on a broken dictionary, and a translation that hits an unknown SKU should stop with the exact line to add. You do not want to edit this during an incident.

The most important output is what did not migrate. Do not skip a kind you cannot map; record it. Every source object gets a condition that lists the warnings, the manual steps, and the data that was left behind, such as bucket contents and secrets that only existed in the air gap. That way the person running the recovery can see what is done and what they still have to do by hand.

applying the export

Two things matter when you apply the export. First, namespaces: a manifest whose namespace does not exist is rejected before the controller sees it, so apply the namespaces first. Second, apply in two passes. kubectl apply only writes the spec; the API server strips the status into a separate subresource, and some GDC facts only exist in the status (a Cluster records its subnets in status.subnets). So apply the spec first and the status second:

kubectl apply --server-side -f export/
kubectl apply --server-side --subresource=status -f export/

Run it again until it is clean. Because the translation is deterministic, a second pass changes nothing that was already correct.

Not everything maps. GKE chooses its own Kubernetes version, a GDC load balancer pool has no GKE equivalent, and GPU partitioning has to be redone by hand. But an estate is mostly clusters, databases, networks, buckets, and service accounts, and those all migrate. When the site is gone, that is what you need back.