Docsright arrowEdge Stackright arrowAPIExt Conversion Webhook Service

7 min • read

APIExt Conversion Webhook Service

Ambassador Edge Stack leverages Kubernetes Custom Resource Definitions to provide a friendly self-service api for configuring Ambassador Edge Stack. Over time, these Custom Resources have evolved and Ambassador Edge Stack supports multiple api versions. However, Kubernetes only supports a single storage version which means the different versions have to be converted into the storage version when applying or fetching resources from the kube-api-server. The APIExt server is a Kubernetes Conversion Webhook providing support for this conversion. To learn more about Custom Resource Definitions, storage versions and conversion webhooks see Versions in CustomResourceDefinitions.

The kube-api-server requires a Conversion Webhook server to communicate using https so by default the APIExt conversion webhook service manages its own CA Certificate and ensures the CustomResourceDefinition is properly patched so that the kube-api-server can properly convert the CustomResources.

Here are the main components of the APIExt Conversion Webhook Service:

EntityDescriptionRequiredLeader Election
ConversionWebhookHandlerAn HTTP handler that converts Custom Resource to requested version.YesNo
CertificateAuthorityInternal CA Cert cache that generates Server CertificatesYesNo
CACertControllerController watching CA Cert Secret and providing CertificateAuthority the CACertYesNo
CRDPatchControllerController patching CRDs to ensure CABundle matches CACertNoYes
CACertManagerRunnable watching CA Cert Secret and ensuring it is always valid and non-expiredNoYes

Modifying default namespace

By default, Ambassador Edge Stack installs the APIExt Server in the emissary-system namespace and the default manifests provided are configured to use this namespace. If you wish to modify this behavior then you must ensure the following objects are updated properly:

  1. The Service and Deployment for the APIExt server
  2. RBAC needs to be adjusted for the ServiceAccount which is bound to the ClusterRole and Role for the apiext server
  3. Update EdgeStack Deployment so that the new waitForAPIExt init container is looking at the correct namespace

Modifying default CRD labels

By default, Ambassador Edge Stack will only watch CRD's with the label which is set on the default crd install manifest. This ensures that the CRD Manager only patches the CRD's that leverage the APIExt Server as a Conversion Webhook. If you need to modify these the APIExt Server has the --crd-label-selector flag that can be modified in the APIExt Server Deployment.

For example, if you want to modify or add additional selectors then you can modify the --crd-label-selector passed to the Deployment:

Alternatively, if you do NOT want to set any label selectors you can remove it:

Disable CA Certification and CRD Patching

By default, the APIExt server manages its own CA Certificate and patches the CRD with the CABundle but some organizations may already have their own policies and tools that they want to use to manage the CA Certficate and/or CRD Patching. The APIExt server provides the ability to disable the CRDPatchController and the CACertManager. This can be done by adding the following Environment Variables to the APIExt Deployment.

  • DISABLE_CRD_MANAGEMENT - If this env var exists then the CRD Patching will be disabled and it will be up to the user to ensure the CA Bundle is correctly patched.
  • DIABLE_CA_MANAGEMENT - If this env var exists then the server will not generate or renew the root CA Certificate stored in emissary-ingress-webhook-ca in the emissary-system namespace. This will be up to the user to manage.

Using CertManager to manage CA Certificate

Many organizations already leverage CertManager for generating Certificates for applications or leverage it for ACME support with Ambassador Edge Stack. It is a good alternative that allows the CA Certificate and CRD patching to be handled externally from the APIExt service.

Here are the steps for setting it up:

  • Disable CACertController and CRDPatchController on APIExt deployment
  • Setup Issuer and Certificate
  • Annotate each CustomResourceDefinition

Setup Issuer and Certificate

The following YAML instructs CertManager to generate a self-signed root CA Certificate and inject it into the emissary-system/emissary-ingress-webhook-ca Secret. The APIExt server will watch that secret, load it and use it to generate Server certificates used for incoming connections.

Annotate each CustomResourceDefinition

Cert Manager can be taught to inject the CABundle from the CA Secret by adding an Annotation to each CustomResourceDefinition. All CustomResourceDefinitions (Mappping, Host, Filter, etc...) will need to have the following annotation emissary-system/emissary-ingress-webhook-ca added before applying or patched if already installed in a cluster.

Here is an example of the CustomResourceDefinition for annotated: