Ambassador Edge Stack leverages Kubernetes Custom Resource Definitions to provide a friendly self-service api for configuring Ambassador Edge Stack. Over time, these Custom Resources have evolved and Ambassador Edge Stack supports multiple api versions. However, Kubernetes only supports a single
storage version which means the different versions have to be converted into the
storage version when applying or fetching resources from the kube-api-server. The APIExt server is a Kubernetes Conversion Webhook providing support for this conversion. To learn more about Custom Resource Definitions,
storage versions and conversion webhooks see Versions in CustomResourceDefinitions.
The kube-api-server requires a Conversion Webhook server to communicate using
https so by default the APIExt conversion webhook service manages its own CA Certificate and ensures the CustomResourceDefinition is properly patched so that the kube-api-server can properly convert the CustomResources.
Here are the main components of the APIExt Conversion Webhook Service:
|An HTTP handler that converts Custom Resource to requested version.
|Internal CA Cert cache that generates Server Certificates
|Controller watching CA Cert Secret and providing CertificateAuthority the CACert
|Controller patching getambassador.io CRDs to ensure CABundle matches CACert
|Runnable watching CA Cert Secret and ensuring it is always valid and non-expired
By default, Ambassador Edge Stack installs the APIExt Server in the
emissary-system namespace and the default manifests provided are configured to use this namespace. If you wish to modify this behavior then you must ensure the following objects are updated properly:
Deploymentfor the APIExt server
- RBAC needs to be adjusted for the
ServiceAccountwhich is bound to the
Rolefor the apiext server
- Update EdgeStack
Deploymentso that the new waitForAPIExt init container is looking at the correct namespace
By default, Ambassador Edge Stack will only watch CRD's with the label
app.kubernetes.io/part-of=emissary-apiext which is set on the default crd install manifest. This ensures that the CRD Manager only patches the
getambassador.io CRD's that leverage the APIExt Server as a Conversion Webhook. If you need to modify these the APIExt Server has the
--crd-label-selector flag that can be modified in the APIExt Server Deployment.
For example, if you want to modify or add additional selectors then you can modify the
--crd-label-selector passed to the Deployment:
Alternatively, if you do NOT want to set any label selectors you can remove it:
By default, the APIExt server manages its own CA Certificate and patches the CRD with the CABundle but some organizations may already have their own policies and tools that they want to use to manage the CA Certficate and/or CRD Patching. The APIExt server provides the ability to disable the
CRDPatchController and the
CACertManager. This can be done by adding the following Environment Variables to the APIExt Deployment.
DISABLE_CRD_MANAGEMENT- If this env var exists then the CRD Patching will be disabled and it will be up to the user to ensure the CA Bundle is correctly patched.
DIABLE_CA_MANAGEMENT- If this env var exists then the server will not generate or renew the root CA Certificate stored in
emissary-systemnamespace. This will be up to the user to manage.
Many organizations already leverage CertManager for generating Certificates for applications or leverage it for ACME support with Ambassador Edge Stack. It is a good alternative that allows the CA Certificate and CRD patching to be handled externally from the APIExt service.
Here are the steps for setting it up:
- Disable CACertController and CRDPatchController on APIExt deployment
- Annotate each CustomResourceDefinition
The following YAML instructs CertManager to generate a self-signed root CA Certificate and inject it into the
emissary-system/emissary-ingress-webhook-ca Secret. The APIExt server will watch that secret, load it and use it to generate Server certificates used for incoming connections.
Cert Manager can be taught to inject the CABundle from the CA Secret by adding an Annotation to each CustomResourceDefinition. All CustomResourceDefinitions (Mappping, Host, Filter, etc...) will need to have the following annotation
cert-manager.io/inject-ca-from: emissary-system/emissary-ingress-webhook-ca added before applying or patched if already installed in a cluster.
Here is an example of the CustomResourceDefinition for