Use the Capsule Operator for easily implementing, managing, and maintaining multitenancy and access control in Kubernetes.
-
Helm 3 is required when installing the Capsule Operator chart. Follow Helm’s official steps for installing helm on your particular operating system.
-
A Kubernetes cluster 1.16+ with following Admission Controllers enabled:
- PodNodeSelector
- LimitRanger
- ResourceQuota
- MutatingAdmissionWebhook
- ValidatingAdmissionWebhook
-
A
kubeconfig
file accessing the Kubernetes cluster with cluster admin permissions.
In the following sections you see actions which are required when you are upgrading to a specific version.
Introduces a new methode to manage all capsule CRDs and their lifecycle. We are no longer relying on the native CRD hook with the Helm Chart. The hook only allows to manage CRDs on install and uninstall but we can't deliver updates to the CRDs.
When you newly install the chart we recommend to set crds.install
to true
. This will manage the CRDs with the Helm Chart. This behavior is the new default.
The following Values have changed key or Value:
- All values from previous releases under
webhooks
have moved towebhooks.hooks
. mutatingWebhooksTimeoutSeconds
has moved towebhooks.mutatingWebhooksTimeoutSeconds
validatingWebhooksTimeoutSeconds
has moved towebhooks.validatingWebhooksTimeoutSeconds
The Capsule Operator requires it's CRDs to be installed before the operator itself. Since the Helm CRD lifecycle has limitations, we recommend to install the CRDs separately. Our chart supports the installation of crds via a dedicated Release. The Capsule Operator Chart can be used to instantly deploy the Capsule Operator on your Kubernetes cluster.
-
Add this repository:
$ helm repo add projectcapsule https://projectcapsule.github.io/charts
-
Install Capsule:
$ helm install capsule projectcapsule/capsule --version 0.7.0 -n capsule-system --create-namespace or $ helm install capsule oci://ghcr.io/projectcapsule/charts/capsule --version 0.7.0 -n capsule-system --create-namespace
-
Show the status:
$ helm status capsule -n capsule-system
-
Upgrade the Chart
$ helm upgrade capsule projectcapsule/capsule -n capsule-system or $ helm upgrade capsule oci://ghcr.io/projectcapsule/charts/capsule --version 0.4.7
-
Uninstall the Chart
$ helm uninstall capsule -n capsule-system
There are two methods for specifying overrides of values during chart installation: --values
and --set
.
The --values
option is the preferred method because it allows you to keep your overrides in a YAML file, rather than specifying them all on the command line. Create a copy of the YAML file values.yaml
and add your overrides to it.
Specify your overrides file when you install the chart:
$ helm install capsule capsule-helm-chart --values myvalues.yaml -n capsule-system
The values in your overrides file myvalues.yaml
will override their counterparts in the chart's values.yaml file. Any values in values.yaml
that weren’t overridden will keep their defaults.
If you only need to make minor customizations, you can specify them on the command line by using the --set
option. For example:
$ helm install capsule capsule-helm-chart --set manager.options.forceTenantPrefix=false -n capsule-system
Here the values you can override:
Key | Type | Default | Description |
---|---|---|---|
crds.annnotations | object | {} |
Extra Annotations for CRDs |
crds.exclusive | bool | false |
Only install the CRDs, no other primitives |
crds.install | bool | true |
Install the CustomResourceDefinitions (This also manages the lifecycle of the CRDs for update operations) |
crds.labels | object | {} |
Extra Labels for CRDs |
Key | Type | Default | Description |
---|---|---|---|
affinity | object | {} |
Set affinity rules for the Capsule pod |
certManager.generateCertificates | bool | false |
Specifies whether capsule webhooks certificates should be generated using cert-manager |
customAnnotations | object | {} |
Additional annotations which will be added to all resources created by Capsule helm chart |
customLabels | object | {} |
Additional labels which will be added to all resources created by Capsule helm chart |
imagePullSecrets | list | [] |
Configuration for imagePullSecrets so that you can use a private images registry. |
jobs.affinity | object | {} |
Set affinity rules |
jobs.annotations | object | {"helm.sh/hook-delete-policy":"before-hook-creation,hook-succeeded"} |
Annotations to add to the certgen job. |
jobs.image.pullPolicy | string | "IfNotPresent" |
Set the image pull policy of the helm chart job |
jobs.image.registry | string | "docker.io" |
Set the image repository of the helm chart job |
jobs.image.repository | string | "clastix/kubectl" |
Set the image repository of the helm chart job |
jobs.image.tag | string | "" |
Set the image tag of the helm chart job |
jobs.nodeSelector | object | {} |
Set the node selector |
jobs.podSecurityContext | object | {"seccompProfile":{"type":"RuntimeDefault"}} |
Security context for the job pods. |
jobs.priorityClassName | string | "" |
Set a pod priorityClassName |
jobs.resources | object | {} |
Job resources |
jobs.restartPolicy | string | "Never" |
Set the restartPolicy |
jobs.securityContext | object | {"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":true,"runAsGroup":1002,"runAsNonRoot":true,"runAsUser":1002} |
Security context for the job containers. |
jobs.tolerations | list | [] |
Set list of tolerations |
jobs.topologySpreadConstraints | list | [] |
Set Topology Spread Constraints |
jobs.ttlSecondsAfterFinished | int | 60 |
Sets the ttl in seconds after a finished certgen job is deleted. Set to -1 to never delete. |
nodeSelector | object | {} |
Set the node selector for the Capsule pod |
podAnnotations | object | {} |
Annotations to add to the capsule pod. |
podSecurityContext | object | {"runAsGroup":1002,"runAsNonRoot":true,"runAsUser":1002,"seccompProfile":{"type":"RuntimeDefault"}} |
Set the securityContext for the Capsule pod |
priorityClassName | string | "" |
Set the priority class name of the Capsule pod |
proxy.enabled | bool | false |
Enable Installation of Capsule Proxy |
replicaCount | int | 1 |
Set the replica count for capsule pod |
securityContext | object | {"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"readOnlyRootFilesystem":true} |
Set the securityContext for the Capsule container |
serviceAccount.annotations | object | {} |
Annotations to add to the service account. |
serviceAccount.create | bool | true |
Specifies whether a service account should be created. |
serviceAccount.name | string | "" |
The name of the service account to use. If not set and serviceAccount.create=true , a name is generated using the fullname template |
tls.create | bool | true |
When cert-manager is disabled, Capsule will generate the TLS certificate for webhook and CRDs conversion. |
tls.enableController | bool | true |
Start the Capsule controller that injects the CA into mutating and validating webhooks, and CRD as well. |
tls.name | string | "" |
Override name of the Capsule TLS Secret name when externally managed. |
tolerations | list | [] |
Set list of tolerations for the Capsule pod |
topologySpreadConstraints | list | [] |
Set topology spread constraints for the Capsule pod |
Key | Type | Default | Description |
---|---|---|---|
manager.hostNetwork | bool | false |
Specifies if the container should be started in hostNetwork mode. Required for use in some managed kubernetes clusters (such as AWS EKS) with custom CNI (such as calico), because control-plane managed by AWS cannot communicate with pods' IP CIDR and admission webhooks are not working |
manager.image.pullPolicy | string | "IfNotPresent" |
Set the image pull policy. |
manager.image.registry | string | "ghcr.io" |
Set the image registry of capsule. |
manager.image.repository | string | "projectcapsule/capsule" |
Set the image repository of capsule. |
manager.image.tag | string | "" |
Overrides the image tag whose default is the chart appVersion. |
manager.kind | string | "Deployment" |
Set the controller deployment mode as Deployment or DaemonSet . |
manager.livenessProbe | object | {"httpGet":{"path":"/healthz","port":10080}} |
Configure the liveness probe using Deployment probe spec |
manager.options.capsuleConfiguration | string | "default" |
Change the default name of the capsule configuration name |
manager.options.capsuleUserGroups | list | ["projectcapsule.dev"] |
Override the Capsule user groups |
manager.options.forceTenantPrefix | bool | false |
Boolean, enforces the Tenant owner, during Namespace creation, to name it using the selected Tenant name as prefix, separated by a dash |
manager.options.generateCertificates | bool | true |
Specifies whether capsule webhooks certificates should be generated by capsule operator |
manager.options.logLevel | string | "4" |
Set the log verbosity of the capsule with a value from 1 to 10 |
manager.options.nodeMetadata | object | {"forbiddenAnnotations":{"denied":[],"deniedRegex":""},"forbiddenLabels":{"denied":[],"deniedRegex":""}} |
Allows to set the forbidden metadata for the worker nodes that could be patched by a Tenant |
manager.options.protectedNamespaceRegex | string | "" |
If specified, disallows creation of namespaces matching the passed regexp |
manager.rbac.create | bool | true |
Specifies whether RBAC resources should be created. |
manager.rbac.existingClusterRoles | list | [] |
Specifies further cluster roles to be added to the Capsule manager service account. |
manager.rbac.existingRoles | list | [] |
Specifies further cluster roles to be added to the Capsule manager service account. |
manager.readinessProbe | object | {"httpGet":{"path":"/readyz","port":10080}} |
Configure the readiness probe using Deployment probe spec |
manager.resources | object | {} |
Set the resource requests/limits for the Capsule manager container |
manager.webhookPort | int | 9443 |
Set an alternative to the default container port. Useful for use in some kubernetes clusters (such as GKE Private) with aggregator routing turned on, because pod ports have to be opened manually on the firewall side |
Key | Type | Default | Description |
---|---|---|---|
serviceMonitor.annotations | object | {} |
Assign additional Annotations |
serviceMonitor.enabled | bool | false |
Enable ServiceMonitor |
serviceMonitor.endpoint.interval | string | "15s" |
Set the scrape interval for the endpoint of the serviceMonitor |
serviceMonitor.endpoint.metricRelabelings | list | [] |
Set metricRelabelings for the endpoint of the serviceMonitor |
serviceMonitor.endpoint.relabelings | list | [] |
Set relabelings for the endpoint of the serviceMonitor |
serviceMonitor.endpoint.scrapeTimeout | string | "" |
Set the scrape timeout for the endpoint of the serviceMonitor |
serviceMonitor.labels | object | {} |
Assign additional labels according to Prometheus' serviceMonitorSelector matching labels |
serviceMonitor.matchLabels | object | {} |
Change matching labels |
serviceMonitor.namespace | string | "" |
Install the ServiceMonitor into a different Namespace, as the monitoring stack one (default: the release one) |
serviceMonitor.targetLabels | list | [] |
Set targetLabels for the serviceMonitor |
Key | Type | Default | Description |
---|---|---|---|
webhooks.exclusive | bool | false |
When crds.exclusive is true the webhooks will be installed |
webhooks.hooks.cordoning.failurePolicy | string | "Fail" |
|
webhooks.hooks.cordoning.namespaceSelector.matchExpressions[0].key | string | "capsule.clastix.io/tenant" |
|
webhooks.hooks.cordoning.namespaceSelector.matchExpressions[0].operator | string | "Exists" |
|
webhooks.hooks.defaults.ingress.failurePolicy | string | "Fail" |
|
webhooks.hooks.defaults.ingress.namespaceSelector.matchExpressions[0].key | string | "capsule.clastix.io/tenant" |
|
webhooks.hooks.defaults.ingress.namespaceSelector.matchExpressions[0].operator | string | "Exists" |
|
webhooks.hooks.defaults.pods.failurePolicy | string | "Fail" |
|
webhooks.hooks.defaults.pods.namespaceSelector.matchExpressions[0].key | string | "capsule.clastix.io/tenant" |
|
webhooks.hooks.defaults.pods.namespaceSelector.matchExpressions[0].operator | string | "Exists" |
|
webhooks.hooks.defaults.pvc.failurePolicy | string | "Fail" |
|
webhooks.hooks.defaults.pvc.namespaceSelector.matchExpressions[0].key | string | "capsule.clastix.io/tenant" |
|
webhooks.hooks.defaults.pvc.namespaceSelector.matchExpressions[0].operator | string | "Exists" |
|
webhooks.hooks.ingresses.failurePolicy | string | "Fail" |
|
webhooks.hooks.ingresses.namespaceSelector.matchExpressions[0].key | string | "capsule.clastix.io/tenant" |
|
webhooks.hooks.ingresses.namespaceSelector.matchExpressions[0].operator | string | "Exists" |
|
webhooks.hooks.namespaceOwnerReference.failurePolicy | string | "Fail" |
|
webhooks.hooks.namespaces.failurePolicy | string | "Fail" |
|
webhooks.hooks.networkpolicies.failurePolicy | string | "Fail" |
|
webhooks.hooks.networkpolicies.namespaceSelector.matchExpressions[0].key | string | "capsule.clastix.io/tenant" |
|
webhooks.hooks.networkpolicies.namespaceSelector.matchExpressions[0].operator | string | "Exists" |
|
webhooks.hooks.nodes.failurePolicy | string | "Fail" |
|
webhooks.hooks.persistentvolumeclaims.failurePolicy | string | "Fail" |
|
webhooks.hooks.persistentvolumeclaims.namespaceSelector.matchExpressions[0].key | string | "capsule.clastix.io/tenant" |
|
webhooks.hooks.persistentvolumeclaims.namespaceSelector.matchExpressions[0].operator | string | "Exists" |
|
webhooks.hooks.pods.failurePolicy | string | "Fail" |
|
webhooks.hooks.pods.namespaceSelector.matchExpressions[0].key | string | "capsule.clastix.io/tenant" |
|
webhooks.hooks.pods.namespaceSelector.matchExpressions[0].operator | string | "Exists" |
|
webhooks.hooks.services.failurePolicy | string | "Fail" |
|
webhooks.hooks.services.namespaceSelector.matchExpressions[0].key | string | "capsule.clastix.io/tenant" |
|
webhooks.hooks.services.namespaceSelector.matchExpressions[0].operator | string | "Exists" |
|
webhooks.hooks.tenantResourceObjects.failurePolicy | string | "Fail" |
|
webhooks.hooks.tenants.failurePolicy | string | "Fail" |
|
webhooks.mutatingWebhooksTimeoutSeconds | int | 30 |
Timeout in seconds for mutating webhooks |
webhooks.service.caBundle | string | "" |
CABundle for the webhook service |
webhooks.service.name | string | "" |
Custom service name for the webhook service |
webhooks.service.namespace | string | "" |
Custom service namespace for the webhook service |
webhooks.service.port | string | nil |
Custom service port for the webhook service |
webhooks.service.url | string | "" |
The URL where the capsule webhook services are running (Overwrites cluster scoped service definition) |
webhooks.validatingWebhooksTimeoutSeconds | int | 30 |
Timeout in seconds for validating webhooks |
This Helm Chart creates the following Kubernetes resources in the release namespace:
- Capsule Namespace
- Capsule Operator Deployment
- Capsule Service
- CA Secret
- Certificate Secret
- Tenant Custom Resource Definition
- CapsuleConfiguration Custom Resource Definition
- MutatingWebHookConfiguration
- ValidatingWebHookConfiguration
- RBAC Cluster Roles
- Metrics Service
And optionally, depending on the values set:
- Capsule ServiceAccount
- Capsule Service Monitor
- PodSecurityPolicy
- RBAC ClusterRole and RoleBinding for pod security policy
- RBAC Role and Rolebinding for metrics scrape
Capsule, as many other add-ons, defines its own set of Custom Resource Definitions (CRDs). Helm3 removed the old CRDs installation method for a more simple methodology. In the Helm Chart, there is now a special directory called crds
to hold the CRDs. These CRDs are not templated, but will be installed by default when running a helm install
for the chart. If the CRDs already exist (for example, you already executed helm install
), it will be skipped with a warning. When you wish to skip the CRDs installation, and do not see the warning, you can pass the --skip-crds
flag to the helm install
command.
You can enable the generation of certificates using cert-manager
as follows.
helm upgrade --install capsule projectcapsule/capsule --namespace capsule-system --create-namespace \
--set "certManager.generateCertificates=true" \
--set "tls.create=false" \
--set "tls.enableController=false"
With the usage of tls.enableController=false
value, you're delegating the injection of the Validating and Mutating Webhooks' CA to cert-manager
.
Since Helm3 doesn't allow to template CRDs, you have to patch manually the Custom Resource Definition tenants.capsule.clastix.io
adding the proper annotation (YMMV).
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.5.0
cert-manager.io/inject-ca-from: capsule-system/capsule-webhook-cert
creationTimestamp: "2022-07-22T08:32:51Z"
generation: 45
name: tenants.capsule.clastix.io
resourceVersion: "9832"
uid: 61e287df-319b-476d-88d5-bdb8dc14d4a6
See Capsule tutorial for more information about how to use Capsule.