The GCP Config Connector (CC) is a Kubernetes add-on which allows you to create, change or delete cloud resources outside your cluster. It can deploy various GCP resources such as storage, databases, network and others representing them as a set of Kubernetes resources. It helps with a unified approach for deployment using Kubernetes to deploy the full stack for your application. The resources can be deployed by kubectl, helm or any CD (Continuous Deployment) tool you use in the organization.
When your Kubernetes cluster is in GCP GKE (Google Kubernetes Engine) then CC can be added to the cluster in a couple different ways. The first way is to enable CC as a GKE add-on and the second is a manual installation following the steps in the Google documentation.
A couple of notes about pros and cons of those different methods. From the first glance having the ACC as a GKE add-on looks like a great idea. You turn one switch in the cluster configuration and it is automatically installed and going to be managed and upgraded automatically. But soon after, when you use it more and more you realize that this way might have some drawbacks. The problem is that the ACC version depends on the GKE version and if you are not in the latest release of the rapid channel then some new features and bug fixes might not be available to you. The ACC releases are coming almost every two weeks, usually much more often than the GKE and your planned cluster upgrades. As a result the ACC version on your cluster can be 6 months old, lack some necessary features and suffer from annoying old bugs fixed in one of the newer versions.
Those reasons can make you rethink your CC deployment strategy and move it to the manual installation mode. How difficult is it? As a matter of fact it is quite easy and doesn’t mandate recreating all resources. All it requires is recreation of the Config Connector operator leaving all your CRD (Custom Resource Definition) and already deployed resources intact. Let me show you the process.
The first step is to disable the CC GKE add-on. You can do it either from the Web console or using the command line. Here is how I’ve disabled it for my sandbox cluster argocd-usc1:
Glebs-iMac:working$ gcloud container clusters update argocd-usc1 --zone=us-central1-a --update-addons ConfigConnector=DISABLED Default change: During creation of nodepools or autoscaling configuration changes for cluster versions greater than 1.24.1-gke.800 a default location policy is applied. For Spot and PVM it defaults to ANY, and for all other VM kinds a BALANCED policy is used. To change the default values use the `--location-policy` flag. Updating argocd-usc1...done. Updated [https://container.googleapis.com/v1/projects/gleb-sandbox-project/zones/us-central1-a/clusters/argocd-usc1]. To inspect the contents of your cluster, go to: https://console.cloud.google.com/kubernetes/workload_/gcloud/us-central1-a/argocd-usc1?project=gleb-sandbox-project
After the execution the cluster will be in “reconciling” mode until the update is applied and then you can see that the “configconnector-operator-system” namespace disappears from your cluster but the “cnrm-system” namespace and all the ACC resources are still there.
The next step is to find the proper version of the CC operator software and install it. You can find all the versions in the “gs://configconnector-operator” bucket. I had version 1.71 on my cluster installed as an add-on and found exactly the same version in the bucket.
Glebs-iMac:working$ gsutil ls gs://configconnector-operator | grep 1.71 gs://configconnector-operator/1.71.0/ Glebs-iMac:working$ gsutil ls gs://configconnector-operator/1.71.0/ gs://configconnector-operator/1.71.0/release-bundle.tar.gz Glebs-iMac:working$
Then I downloaded it, unpacked and installed it using kubectl.
Glebs-iMac:working$ gsutil cp gs://configconnector-operator/1.71.0/release-bundle.tar.gz . Copying gs://configconnector-operator/1.71.0/release-bundle.tar.gz... / [1 files][ 3.9 KiB/ 3.9 KiB] Operation completed over 1 objects/3.9 KiB. Glebs-iMac:working$ Glebs-iMac:working$ tar xfz release-bundle.tar.gz Glebs-iMac:working$ Glebs-iMac:working$ kubectl apply -f operator-system/configconnector-operator.yaml namespace/configconnector-operator-system created ... redacted output ...
If your CC configuration is correct then it will be picked up along with all your ACC resources. In my test cluster I had Config Connector configured in “namespaced” mode and a Cloud SQL database. After applying the new operation configuration I was able to continue managing it as nothing happened.
And what about the upgrades, that was probably the main reason to have it as a GKE add-on. How much effort do you need to upgrade the ACC on a cluster? It is even easier – you just download the new desired version, uncompress it and apply on top of the existing configuration.
Glebs-iMac:working$ gsutil cp gs://configconnector-operator/1.72.1/release-bundle.tar.gz . Copying gs://configconnector-operator/1.72.1/release-bundle.tar.gz... / [1 files][ 3.9 KiB/ 3.9 KiB] Operation completed over 1 objects/3.9 KiB. Glebs-iMac:working$ tar xfz release-bundle.tar.gz Glebs-iMac:working$ Glebs-iMac:working$ kubectl apply -f operator-system/configconnector-operator.yaml namespace/configconnector-operator-system configured ... redacted output ...
All the pods related to the CC will be automatically recreated applying the new versions of CRDs and operators.
NAME↑ PF READY RESTARTS STATUS CPU MEM %CPU/R %CPU/L %MEM/R %MEM/L IP NODE AGE │ │ cnrm-controller-manager-cbr88okgkgt6qmsm20o0-0 ● 2/2 0 Running 23 100 23 n/a 19 n/a 10.144.1.12 gke-argocd-usc1-argocd-4105fa4e-hxzm 45s │ │ cnrm-deletiondefender-0 ● 1/1 0 Running 15 96 6 n/a 9 9 10.144.3.6 gke-argocd-usc1-argocd-4105fa4e-quoe 37s │ │ cnrm-resource-stats-recorder-566cc997b4-6msd6 ● 2/2 0 Running 0 0 0 n/a 0 n/a 10.2.0.25 gke-argocd-usc1-argocd-4105fa4e-quoe 27s │ │ cnrm-webhook-manager-6ddfbbfc4-6j4jv ● 1/1 0 Running 0 0 0 n/a 0 0 10.144.0.9 gke-argocd-usc1-argocd-4105fa4e-7aq2 26s │ │ cnrm-webhook-manager-6ddfbbfc4-lwchw ● 1/1 0 Running 0 0 0 n/a 0 0 10.144.4.11 gke-argocd-usc1-argocd-4105fa4e-v9z1 38s
So, moving from a managed ACC (aka GKE add-on) is easy and painless. All your resources stay in place and work as usual. Of course you lose the ability to manage them for a short period of time when your operator is removed and the manual one is not installed yet. But, in my opinion, it is not a big price to pay. The upgrade is even less disruptive – you apply the new operator yaml file and everything is recreated on background updating the CC CRDs and operators.