Skip to content

Latest commit

 

History

History

k8s-sidecar

Using the Cloud SQL Auth Proxy on Kubernetes

The Cloud SQL Auth Proxy is the recommended way to connect to Cloud SQL, even when using private IP. This is because the proxy provides strong encryption and authentication using IAM, which help keep your database secure.

Configure your application with Secrets

In Kubernetes, Secrets are a secure way to pass configuration details to your application. Each Secret object can contain multiple key/value pairs that can be pass to your application in multiple ways. When connecting to a database, you can create a Secret with details such as your database name, user, and password which can be injected into your application as env vars.

  1. Create a secret with information needed to access your database:
    kubectl create secret generic <YOUR-DB-SECRET> \
        --from-literal=username=<YOUR-DATABASE-USER> \
        --from-literal=password=<YOUR-DATABASE-PASSWORD> \
        --from-literal=database=<YOUR-DATABASE-NAME>
  2. Next, configure your application's container to mount the secrets as env vars:

    proxy_with_workload_identity.yaml

       env:
        - name: DB_USER
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: username
        - name: DB_PASS
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: password
        - name: DB_NAME
          valueFrom:
            secretKeyRef:
              name: <YOUR-DB-SECRET>
              key: database
  3. Finally, configure your application to use these values. In the example above, the values will be in the env vars DB_USER, DB_PASS, and DB_NAME.

Setting up a service account

The first step to running the Cloud SQL Auth Proxy in Kubernetes is creating a service account to represent your application. It is recommended that you create a service account unique to each application, instead of using the same service account everywhere. This model is more secure since it allows your to limit permissions on a per-application basis.

The service account for your application needs to meet the following criteria:

  1. Belong to a project with the Cloud SQL Admin API enabled
  2. Has been granted the Cloud SQL Client IAM role (or equivalent) for the project containing the instance you want to connect to
  3. If connecting using private IP, you must use a VPC-native GKE cluster, in the same VPC as your Cloud SQL instance

Providing the service account to the proxy

Next, you need to configure Kubernetes to provide the service account to the Cloud SQL Auth Proxy. There are two recommended ways to do this.

Workload Identity

If you are using Google Kubernetes Engine, the preferred method is to use GKE's Workload Identity feature. This method allows you to bind a Kubernetes Service Account (KSA) to a Google Service Account (GSA). The GSA will then be accessible to applications using the matching KSA.

  1. Enable Workload Identity for your cluster

  2. Enable Workload Identity for your node pool

  3. Create a KSA for your application kubectl apply -f service-account.yaml:

    service-account.yaml

    apiVersion: v1
    kind: ServiceAccount
    metadata:
      name: <YOUR-KSA-NAME> # TODO(developer): replace these values
  4. Enable the IAM binding between your <YOUR-GSA-NAME> and <YOUR-KSA-NAME>:

    gcloud iam service-accounts add-iam-policy-binding \
      --role roles/iam.workloadIdentityUser \
      --member "serviceAccount:<YOUR-GCP-PROJECT>.svc.id.goog[<YOUR-K8S-NAMESPACE>/<YOUR-KSA-NAME>]" \
      <YOUR-GSA-NAME>@<YOUR-GCP-PROJECT>.iam.gserviceaccount.com
  5. Add an annotation to <YOUR-KSA-NAME> to complete the binding:

    kubectl annotate serviceaccount \
       <YOUR-KSA-NAME> \
       iam.gke.io/gcp-service-account=<YOUR-GSA-NAME>@<YOUR-GCP-PROJECT>.iam.gserviceaccount.com
  6. Finally, make sure to specify the service account for the k8s pod spec:

    proxy_with_workload_identity.yaml

    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: <YOUR-DEPLOYMENT-NAME>
    spec:
      selector:
        matchLabels:
          app: <YOUR-APPLICATION-NAME>
      template:
        metadata:
          labels:
            app: <YOUR-APPLICATION-NAME>
        spec:
          serviceAccountName: <YOUR-KSA-NAME>

Service account key file

Alternatively, if your can't use Workload Identity, the recommended pattern is to mount a service account key file into the Cloud SQL Auth Proxy pod and use the --credentials-file flag.

  1. Create a credential file for your service account key:

    gcloud iam service-accounts keys create ~/key.json \
      --iam-account <YOUR-SA-NAME>@project-id.iam.gserviceaccount.com
  2. Turn your service account key into a k8s Secret:

    kubectl create secret generic <YOUR-SA-SECRET> \
    --from-file=service_account.json=~/key.json
  3. Mount the secret as a volume under thespec: for your k8s object:

    proxy_with_sa_key.yaml

    volumes:
    - name: <YOUR-SA-SECRET-VOLUME>
      secret:
        secretName: <YOUR-SA-SECRET>
  4. Follow the instructions in the next section to access the volume from the proxy's pod.

Run the Cloud SQL Auth Proxy as a sidecar

We recommend running the proxy in a "sidecar" pattern (as an additional container sharing a pod with your application). We recommend this over running as a separate service for several reasons:

  • Prevents your SQL traffic from being exposed locally - the proxy provides encryption on outgoing connections, but you should limit exposure for incoming connections
  • Prevents a single point of failure - each application's access to your database is independent from the others, making it more resilient.
  • Limits access to the proxy, allowing you to use IAM permissions per application rather than exposing the database to the entire cluster
  • Allows you to scope resource requests more accurately - because the proxy consumes resources linearly to usage, this pattern allows you to more accurately scope and request resources to match your applications as it scales
  1. Add the Cloud SQL Auth Proxy to the pod configuration under containers:

    proxy_with_workload-identity.yaml

    - name: cloud-sql-proxy
      # It is recommended to use the latest version of the Cloud SQL Auth Proxy
      # Make sure to update on a regular schedule!
      image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.12.0  # make sure to use the latest version
      args:
        # If connecting from a VPC-native GKE cluster, you can use the
        # following flag to have the proxy connect over private IP
        # - "--private-ip"
    
        # Replace DB_PORT with the port the proxy should listen on
        - "--port=<DB_PORT>"
        - "<INSTANCE_CONNECTION_NAME>"
      securityContext:
        # The default Cloud SQL Auth Proxy image runs as the
        # "nonroot" user and group (uid: 65532) by default.
        runAsNonRoot: true
      # Resource configuration depends on an application's requirements. You
      # should adjust the following values based on what your application
      # needs. For details, see https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
      resources:
        requests:
          # The proxy's memory use scales linearly with the number of active
          # connections. Fewer open connections will use less memory. Adjust
          # this value based on your application's requirements.
          memory: "2Gi"
          # The proxy's CPU use scales linearly with the amount of IO between
          # the database and the application. Adjust this value based on your
          # application's requirements.
          cpu:    "1"

If you are using a service account key, specify your secret volume and add the --credentials-file flag to the command:

proxy_with_sa_key.yaml yaml # This flag specifies where the service account key can be found - "--credentials-file=/secrets/service_account.json" securityContext: # The default Cloud SQL Auth Proxy image runs as the # "nonroot" user and group (uid: 65532) by default. runAsNonRoot: true volumeMounts: - name: <YOUR-SA-SECRET-VOLUME> mountPath: /secrets/ readOnly: true

  1. Finally, configure your application to connect via 127.0.0.1 on whichever <DB_PORT> you specified in the command section.

Connecting without the Cloud SQL Auth Proxy

While not as secure, it is possible to connect from a VPC-native GKE cluster to a Cloud SQL instance on the same VPC using private IP without the proxy.

  1. Create a secret with your instance's private IP address:

    kubectl create secret generic <YOUR-PRIVATE-IP-SECRET> \
        --from-literal=db_host=<YOUR-PRIVATE-IP-ADDRESS>
  2. Next make sure you add the secret to your application's container:

    no_proxy_private_ip.yaml

    - name: DB_HOST
      valueFrom:
        secretKeyRef:
          name: <YOUR-PRIVATE-IP-SECRET>
          key: db_host
  3. Finally, configure your application to connect using the IP address from the DB_HOST env var. You will need to use the correct port for your db-engine (MySQL: 3306, Postgres: 5432, SQLServer: 1433).