Emissary-ingress Release Notes

Previously, after reconfiguration, routes could briefly be misconfigured due to a shift in the routes list. This has now been patched. Thanks to Joe Andaverde for contributing this bugfix!

This upgrades Emissary-ingress to be built on Envoy v1.27.2 which provides security, performance and feature enhancements. You can read more about them here: Envoy Proxy 1.27.2 Release Notes

Upgrading to the latest release of Golang as part of our general dependency upgrade process.

Upgraded the base image that Emissary-ingress is built on to ensure we have the latest dependencies.

Upgraded to Python 3.11 to ensure we stay updated on our dependenceies and to take advantage of the performance benefits introduced in 3.11.

By default, Emissary-ingress will return an UNAVAILABLE code when a request using gRPC is rate limited. The RateLimitService resource now exposes a new grpc.use_resource_exhausted_code field that when set to true, Emissary-ingress will return a RESOURCE_EXHAUSTED gRPC code instead. Thanks to Jerome Froelich for contributing this feature!

Envoy runtime fields that were provided to mitigate the recent HTTP/2 rapid reset vulnerability can now be configured via the Module resource so the configuration will persist between restarts. This configuration is added to the Envoy bootstrap config, so restarting Emissary is necessary after changing these fields for the configuration to take effect.

APIExt would previously allow for TLS 1.0 connections. We have updated it to now only use a minimum TLS version of 1.3 to resolve security concerns.

- Update default image to Emissary-ingress v3.9.0.

The APIExt server provides CRD conversion between the stored version v2 and the version watched for by Emissary-ingress v3alpha1. Since this component is required to operate Emissary-ingress, we have introduced an init container that will ensure it is available before starting. This will help address some of the intermittent issues seen during install and upgrades.

This release includes security patches to the current Envoy proxy version to address CVE 2023-44487 and includes a fix to determine if a client is making too many requests with premature resets. The connection is disconnected if more than 50 percent of resets are considered premature. Another fix is also included which exposes a runtime setting to control the limit on the number of HTTP requests processed from a single connection in a single I/O cycle to mitigate CPU starvation.

Upgrading to the latest release of Golang as part of our general dependency upgrade process. This update resolves CVE-2023-39323 and CVE-2023-39325.

Upgrading to the latest release of Golang as part of our general dependency upgrade process. This includes security fixes for CVE-2023-39318, CVE-2023-39319.

As of v2.2.2, if two mappings were associated with different Hosts through host mappingSelector labels but share the same prefix, the labels were not taken into account which would cause one Mapping to be correctly routed but the other not. This change fixes this issue so that Mappings sharing the same prefix but associated with different Hosts will be correctly routed.

In previous versions, if multiple Headers/QueryParameters were used in a v3alpha1 mapping, these values would duplicate and cause all the Headers/QueryParameters to have the same value. This is no longer the case and the expected values for unique Headers/QueryParameters will apply. This issue was only present in v3alpha1 Mappings. For users who may have this issue, please be sure to re-apply any v3alpha1 Mappings in order to update the stored v2 Mapping and resolve the issue.

When the Ambassador agent is being used, it will no longer attempt to collect and report Envoy metrics. In previous versions, Emissary-ingress would always create an Envoy stats sink for the agent as long as the AMBASSADOR_GRPC_METRICS_SINK environment variable was provided. This environment variable was hardcoded on the release manifests and has now been removed and an Envoy stats sink for the agent is no longer created.

This upgrades Emissary-ingress to be built on Envoy v1.26.4 which includes a fixes for CVE-2023-35942, CVE-2023-35943, CVE-2023-35944.

- Update default image to Emissary-ingress v3.7.2.

This upgrades Emissary-ingress to be built on Envoy v1.26.3 which includes a fix for CVE-2023-35945.

This upgrades Emissary-ingress to be built on Envoy v1.26.1 which provides security, performance and feature enhancements. You can read more about them here: Envoy Proxy 1.26.1 Release Notes

This upgrades Emissary-ingress to be built on Envoy v1.25.4 which provides security, performance and feature enhancements. You can read more about them here: Envoy Proxy 1.25.4 Release Notes

- Update default image to Emissary-ingress v3.6.0.
- Add support for setting nodeSelector, tolerations and affinity on the Ambassador Agent. Thanks to Philip Panyukov.
- Use autoscaling API version based on Kubernetes version. Thanks to Elvind Valderhaug.
- Upgrade KubernetesEndpointResolver & ConsulResolver apiVersions to getambassador.io/v3alpha1

This upgrades Emissary-ingress to be built on Envoy v1.24.5. This update includes various security patches including CVE-2023-27487, CVE-2023-27491, CVE-2023-27492, CVE-2023-27493, CVE-2023-27488, and CVE-2023-27496. It also contains the dependency update for c-ares which was patched on top.

Upgrading to the latest release of Golang as part of our general dependency upgrade process. This includes security fixes for CVE-2023-24537, CVE-2023-24538, CVE-2023-24534, CVE-2023-24536.

Fix regression where the Module resource fails validation when setting the ambassador_id after upgrading to getambassador.io/v3alpha1.

Thanks to Pier.

Upgraded to the latest release of Golang as part of our general dependency upgrade process. This includes security fixes for CVE-2022-41725, CVE-2022-41723.

In Envoy 1.24, experimental support for a native OpenTelemetry tracing driver was introduced that allows exporting spans in the otlp format. Many observability platforms accept that format and is the recommended replacement for the LightStep driver. Emissary-ingress now supports setting the TracingService.spec.driver=opentelemetry to export traces in the otlp format.

Thanks to Paul for helping us get this tested and over the finish line!

The /ready endpoint used by Emissary-ingress was using the Envoy admin port (8001 by default).This generates a problem during config reloads with large configs as the admin thread is blocking so the /ready endpoint can be very slow to answer (in the order of several seconds, even more).

Emissary-ingress will now use a specific envoy listener that can answer /ready calls from an Envoy worker thread so the endpoint is always fast and it does not suffer from single threaded admin thread slowness on config reloads and other slow endpoints handled by the admin thread.

Configure the listener port using AMBASSADOR_READY_PORT and enable access log using AMBASSADOR_READY_LOG environment variables.

When wanting to expose traffic to clients on ports other than 80/443, users will set a port in the Host.hostname (eg.Host.hostname=example.com:8500). The config generated allowed matching on the :authority header. This worked in v1.Y series due to the way Emissary-ingress was generating Envoy configuration under a single wild-card virtual_host and matching on :authority.

In v2.Y/v3.Y+, the way Emissary-ingress generates Envoy configuration changed to address memory pressure and improve route lookup speed in Envoy. However, when including a port in the hostname, an incorrect configuration was generated with an sni match including the port. This caused incoming request to never match causing a 404 Not Found.This has been fixed and the correct envoy configuration is being generated which restores existing behavior.

Previously, specifying backend ports by name in Ingress was not supported and would result in defaulting to port 80. This allows Emissary-ingress to now resolve port names for backend services. If the port number cannot be resolved by the name (e.g named port in the Service doesn't exist) then it will continue to default back to port 80.

Thanks to Anton Ustyuzhanin!.

The emissary-apiext server is a Kubernetes Conversion Webhook that converts between the Emissary-ingress CRD versions. On startup, it ensures that a self-signed cert is available so that K8s API Server can talk to the conversion webhook (*TLS is required by K8s*). We have introduced a startupProbe to ensure that emissary-apiext server has enough time to configure the webhooks before running liveness and readiness probes. This is to ensure slow first-time startup doesn't cause K8s to needlessly restart the pod.

Upgraded to Python 3.10 as part of continued investment in keeping dependencies updated.

Upgraded base image to alpine-3.17 as part of continued investment in keeping dependencies updated.

Here are a list of changes to the helm chart:

- Update default image to Emissary-ingress v3.5.0.
- Add support for configuring startupProbes on the emissary-ingress deployment.
- Allow setting pod and container security settings on the Ambassador Agent.
- Added deprecation notice in the values.yaml file for podSecurityPolicy value because support has been removed in Kubernetes 1.25.

This upgrades Emissary-ingress to be built on Envoy v1.24.2. This update addresses the folowing notable items:

- Updates boringssl to address High CVE-2023-0286
- Updates c-ares dependency to address issue with cname wildcard dns resolution for upstream clusters

Users that are using Emissary-ingress with Certificate Revocation List and allow external users to provide input should upgrade to ensure they are not vulnerable to CVE-2023-0286.

This upgrades Emissary-ingress to be built on Envoy v1.24.1. Two notable changes were introduced:

First, the team at LightStep and the Envoy Maintainers have decided to no longer support the native LightStep tracing driver in favor of using the Open Telemetry driver. The code for the native Enovy LightStep driver has been removed from the Envoy code base. This means Emissary-ingress will no longer support LightStep in the TracingService. The recommended upgrade path is to leverage a supported Tracing driver such as Zipkin and use the Open Telemetry Collector to collect and forward Observabity data to LightStep. A guide for this can be found here: Distributed Tracing with Open Telemetry and LightStep.

Second, a bug was fixed in Envoy 1.24 that changes how the upstream clusters distributed tracing span is named. Prior to Envoy 1.24 it would always set the span name to the cluster.name. The expected behavior from Envoy was that if provided an alt_stat_name then use it else fallback to cluster.name.

Support for the getambassador.io/v1 apiVersion has been re-introduced, in order to facilitate smoother migrations from Emissary-ingress 1.y. Previously, in order to make migrations possible, an "unserved" v1 version was declared to Kubernetes, but was unsupported by Emissary-ingress. That unserved v1 could cause an excess of errors to be logged by the Kubernetes Nodes (regardless of whether the installation was migrated from 1.y or was a fresh 2.y install); fully supporting v1 again should resolve these errors.

It is now possible to configure active healhchecking for upstreams within a Mapping. If the upstream fails its configured health check then Envoy will mark the upstream as unhealthy and no longer send traffic to that upstream. Single pods within a group can be marked as unhealthy. The healthy pods will continue to receive traffic normally while the unhealthy pods will not receive any traffic until they recover by passing the health check.

The healthcheck server's bind address, bind port and IP family can now be configured using environment variables:

AMBASSADOR_HEALTHCHECK_BIND_ADDRESS: The address to bind the healthcheck server to.

AMBASSADOR_HEALTHCHECK_BIND_PORT: The port to bind the healthcheck server to.

AMBASSADOR_HEALTHCHECK_IP_FAMILY: The IP family to use for the healthcheck server.

This allows the healthcheck server to be configured to use IPv6-only k8s environments. (Thanks to Dmitry Golushko!).

Previously, the Agent used for communicating with Ambassador Cloud was bundled into Emissary-ingress. This tied it to the same release schedule as Emissary-ingress and made it difficult to iterate on its feature set. It has now been extracted into its own repository and has its own release process and schedule.

Updated Golang to latest 1.19.4 patch release which contained two CVEs: CVE-2022-41720, CVE-2022-41717. CVE-2022-41720 only affects Windows and Emissary-ingress only ships on Linux. CVE-2022-41717 affects HTTP/2 servers that are exposed to external clients. Emissary-ingress does not expose any of these golang servers to external clients directly.

Updated Golang to 1.19.2 to address the CVEs: CVE-2022-2879, CVE-2022-2880, CVE-2022-41715.

By default Emissary-ingress adds routes for http to https redirection. When an AuthService is applied in v2.Y of Emissary-ingress, Envoy would skip the ext_authz call for non-tls http request and would perform the https redirect. In Envoy 1.20+ the behavior has changed where Envoy will always call the ext_authz filter and must be disabled on a per route basis. This new behavior change introduced a regression in v3.0 of Emissary-ingress when it was upgraded to Envoy 1.22. The http to https redirection no longer works when an AuthService was applied. This fix restores the previous behavior by disabling the ext_authz call on the https redirect routes.

When an AuthService is applied in v2.Y of Emissary-ingress, Envoy would skip the ext_authz call for all redirect routes and would perform the redirect. In Envoy 1.20+ the behavior has changed where Envoy will always call the ext_authz filter so it must be disabled on a per route basis. This new behavior change introduced a regression in v3.0 of Emissary-ingress when it was upgraded to Envoy 1.22. The host_redirect would call an AuthService prior to redirect if applied. This fix restores the previous behavior by disabling the ext_authz call on the host_redirect routes.

Updated Golang to 1.19.1 to address the CVEs: CVE-2022-27664, CVE-2022-32190.

Previously the Host resource could only use secrets that are in the namespace as the Host. The tlsSecret field in the Host has a new subfield namespace that will allow the use of secrets from different namespaces.

By default, when Envoy is unable to communicate with the configured RateLimitService then it will allow traffic through. The RateLimitService resource now exposes the failure_mode_deny option. Set failure_mode_deny: true, then Envoy will deny traffic when it is unable to communicate to the RateLimitService returning a 500.

Set AMBASSADOR_EDS_BYPASS to true to bypass EDS handling of endpoints and have endpoints be inserted to clusters manually. This can help resolve with 503 UH caused by certification rotation relating to a delay between EDS + CDS. The default is false.

The AMBASSADOR_RECONFIG_MAX_DELAY env var can be optionally set to batch changes for the specified non-negative window period in seconds before doing an Envoy reconfiguration. Default is "1" if not set.

It is now possible to set custom_tags in the TracingService. Trace tags can be set based on literal values, environment variables, or request headers. The existing tag_headers field is now deperacated. If both tag_headers and custom_tags are set then tag_headers will be ignored. (Thanks to Paul!)

Changes to label matching will change how Hosts are associated with Mappings and how Listeners are associated with Hosts. There was a bug with label selectors that was causing resources that configure a selector to be incorrectly associated with more resources than intended. If any single label from the selector was matched then the resources would be associated. Now it has been updated to correctly only associate these resources if all labels required by their selector are present. This brings the mappingSelector/selector fields in-line with how label selectors are used in Kubernetes. To avoid unexpected behavior after the upgrade, add all labels that Hosts/Listeners have in their mappingSelector/selector to Mappings/Hosts you want to associate with them. You can opt-out of the new behavior by setting the environment variable DISABLE_STRICT_LABEL_SELECTORS to "true" (default: "false"). (Thanks to Filip Herceg and Joe Andaverde!).

The envoy version included in Emissary-ingress has been upgraded from 1.22 to that latest release of 1.23.0. This provides Emissary-ingress with the latest security patches, performances enhancments,and features offered by the envoy proxy.

Distinct services with names that are the same in the first forty characters will no longer be incorrectly mapped to the same cluster.

Previously, setting the stats_name for the TracingService, RateLimitService or the AuthService would have no affect because it was not being properly passed to the Envoy cluster config. This has been fixed and the alt_stats_name field in the cluster config is now set correctly. (Thanks to Paul!)

If a Host or TLSContext contained a hostname with a : when using the diagnostics endpoints ambassador/v0/diagd then an error would be thrown due to the parsing logic not being able to handle the extra colon. This has been fixed and Emissary-ingress will not throw an error when parsing envoy metrics for the diagnostics user interface.

Emissary-ingress 2.0.0 introduced a bug where a TCPMapping that uses SNI, instead of using the hostname glob in the TCPMapping, uses the hostname glob in the Host that the TLS termination configuration comes from.

Emissary-ingress 2.0.0 introduced a bug where a TCPMapping that terminates TLS must have a corresponding Host that it can take the TLS configuration from. This was semi-intentional, but didn't make much sense. You can now use a TLSContext without a Hostas in Emissary-ingress 1.y releases, or a Host with or without a TLSContext as in prior 2.y releases.

Prior releases of Emissary-ingress had the arbitrary limitation that a TCPMapping cannot be used on the same port that HTTP is served on, even if TLS+SNI would make this possible. Emissary-ingress now allows TCPMappings to be used on the same Listener port as HTTP Hosts, as long as that Listener terminates TLS.

The agent is now able to parse api contracts using swagger 2, and to convert them to OpenAPI 3, making them available for use in the dev portal.

Adds a new command to the agent directive service to manage secrets. This allows a third party product to manage CRDs that depend upon a secret.

Add additional pprof endpoints to allow for profiling Emissary-ingress: - CPU profiles (/debug/pprof/profile) - tracing (/debug/pprof/trace) - command line running (/debug/pprof/cmdline) - program counters (/debug/pprof/symbol)

In the standard published .yaml files, the Module resource enables serving remote client requests to the :8877/ambassador/v0/diag/ endpoint. The associated Helm chart release also now enables it by default.

A regression was introduced in 2.3.0 causing the agent to miss some of the metrics coming from emissary ingress before sending them to Ambassador cloud. This issue has been resolved to ensure that all the nodes composing the emissary ingress cluster are reporting properly.

Updated Golang to 1.17.12 to address the CVEs: CVE-2022-23806, CVE-2022-28327, CVE-2022-24675, CVE-2022-24921, CVE-2022-23772.

Updated Curl to 7.80.0-r2 to address the CVEs: CVE-2022-32207, CVE-2022-27782, CVE-2022-27781, CVE-2022-27780.

Updated openSSL-dev to 1.1.1q-r0 to address CVE-2022-2097.

Updated ncurses to 1.1.1q-r0 to address CVE-2022-29458

Emissary-ingress has been upgraded to the latest Envoy 1.22 patch release which provides security, performance and feature enhancements. You can read more about them here: Envoy Proxy 1.22.0 Release Notes This is a major jump in Envoy versions from the current of 1.17 in EdgeStack 2.X. Most of the changes are under the hood and allow Emissary-ingress to adopt new features in the future. However, one major change that will effect users is the removal of V2 xDS Transport Protocol support. See below for additional information.

Envoy removed support for V2 xDS Transport Protocol which means Emissary-ingress now only supports the Envoy V3 xDS Transport Protocol. User should first upgrade to Emissary-ingress 2.3 prior to ensure that the AuthServices, LogServices and RatelimitServices are working properly by setting protocol_version: "v3". If protocol_version is not specified in 3.Y, the default value of v2 will cause an error to be posted and a static response will be returned. Therefore, you must set it to protocol_version: v3. If upgrading from a previous version, you will want to set it to v3 and ensure it is working before upgrading to Emissary-ingress 3.Y. The default value for protocol_version remains v2 in the getambassador.io/v3alpha1 CRD specifications to avoid making breaking changes outside of a CRD version change. Future versions of CRD's will deprecate it.

With the upgrade to Envoy, Emissary-ingress is now able to provide downstream support for HTTP/3. HTTP/3 is built on the QUIC protocol which communicates using the UDP network protocol. QUIC requires TLS v1.3 be used when communicating between client and server.

When using the zipkin driver for the TracingService. The collector_endpoint_version no longer accepts `HTTP_JSON_V1` due to the Envoy upgrade. The new default value is `HTTP_JSON`.

Updated Golang to latest 1.19.4 patch release which contained two CVEs: CVE-2022-41720, CVE-2022-41717. CVE-2022-41720 only affects Windows and Emissary-ingress only ships on Linux. CVE-2022-41717 affects HTTP/2 servers that are exposed to external clients. Emissary-ingress does not expose any of these golang servers to external clients directly.

If a Host or TLSContext contained a hostname with a : then when using the diagnostics endpoints ambassador/v0/diagd then an error would be thrown due to the parsing logic not being able to handle the extra colon. This has been fixed and Emissary-ingress will not throw an error when parsing envoy metrics for the diagnostics user interface.

Bump Go from 1.17.12 to 1.19.2. This is to keep the Go version current.

If a Host or TLSContext contained a hostname with a : then when using the diagnostics endpoints ambassador/v0/diagd then an error would be thrown due to the parsing logic not being able to handle the extra colon. This has been fixed and Emissary-ingress will not throw an error when parsing envoy metrics for the diagnostics user interface.

The synthetic AuthService didn't correctly handle AmbassadorID, which was fixed in version 3.1 of Emissary-ingress.The fix has been backported to make sure the AuthService is handled correctly during upgrades.

Previously the Host resource could only use secrets that are in the namespace as the Host. The tlsSecret field in the Host has a new subfield namespace that will allow the use of secrets from different namespaces.

Set `AMBASSADOR_EDS_BYPASS` to `true` to bypass EDS handling of endpoints and have endpoints be inserted to clusters manually. This can help resolve with `503 UH` caused by certification rotation relating to a delay between EDS + CDS. The default is `false`.

Previously, setting the stats_name for the TracingService, RateLimitService or the AuthService would have no affect because it was not being properly passed to the Envoy cluster config. This has been fixed and the alt_stats_name field in the cluster config is now set correctly. (Thanks to Paul!)

The AMBASSADOR_RECONFIG_MAX_DELAY env var can be optionally set to batch changes for the specified non-negative window period in seconds before doing an Envoy reconfiguration. Default is "1" if not set.

Emissary-ingress 2.0.0 introduced a bug where a TCPMapping that uses SNI, instead of using the hostname glob in the TCPMapping, uses the hostname glob in the Host that the TLS termination configuration comes from.

Emissary-ingress 2.0.0 introduced a bug where a TCPMapping that terminates TLS must have a corresponding Host that it can take the TLS configuration from. This was semi-intentional, but didn't make much sense. You can now use a TLSContext without a Hostas in Emissary-ingress 1.y releases, or a Host with or without a TLSContext as in prior 2.y releases.

Prior releases of Emissary-ingress had the arbitrary limitation that a TCPMapping cannot be used on the same port that HTTP is served on, even if TLS+SNI would make this possible. Emissary-ingress now allows TCPMappings to be used on the same Listener port as HTTP Hosts, as long as that Listener terminates TLS.

A regression was introduced in 2.3.0 causing the agent to miss some of the metrics coming from emissary ingress before sending them to Ambassador cloud. This issue has been resolved to ensure that all the nodes composing the emissary ingress cluster are reporting properly.

Updated Golang to 1.17.12 to address the CVEs: CVE-2022-23806, CVE-2022-28327, CVE-2022-24675, CVE-2022-24921, CVE-2022-23772.

Updated Curl to 7.80.0-r2 to address the CVEs: CVE-2022-32207, CVE-2022-27782, CVE-2022-27781, CVE-2022-27780.

Updated openSSL-dev to 1.1.1q-r0 to address CVE-2022-2097.

Updated ncurses to 1.1.1q-r0 to address CVE-2022-29458

A regression was introduced in 2.3.0 that leaked zipkin default config fields into the configuration for the other drivers (lightstep, etc...). This caused Emissary-ingress to crash on startup. This issue has been resolved to ensure that the defaults are only applied when driver is zipkin

We have backported patches from the Envoy 1.19.5 security update to Emissary-ingress's 1.17-based Envoy, addressing CVE-2022-29224 and CVE-2022-29225. Emissary-ingress is not affected by CVE-2022-29226, CVE-2022-29227, or CVE-2022-29228; as it does not support internal redirects, and does not use Envoy's built-in OAuth2 filter.

Completely remove gdbm, pip, smtplib, and sqlite packages, as they are unused.

When CORS is specified (either in a Mapping or in the Ambassador Module), CORS processing will happen before authentication. This corrects a problem where XHR to authenticated endpoints would fail.

In 2.x releases of Emissary-ingress when there are multiple Mappings that have the same metadata.name across multiple namespaces, their old config would not properly be removed from the cache when their config was updated. This resulted in an inability to update configuration for groups of Mappings that share the same name until the Emissary-ingress pods restarted.

It is now possible for a TracingService to specify collector_endpoint_version: HTTP_JSON_V1 when using xDS v3 to configure Envoy (which has been the default since Emissary-ingress 1.14.0). The HTTP_JSON_V1 value configures Envoy to speak to Zipkin using Zipkin's old API-v1, while the HTTP_JSON value configures Envoy to speak to Zipkin using Zipkin's new API-v2. In previous versions of Emissary-ingress it was only possible to use HTTP_JSON_V1 when explicitly setting the AMBASSADOR_ENVOY_API_VERSION=V2 environment variable to force use of xDS v2 to configure Envoy.

It is now possible to set propagation_modes in the TracingService config when using lightstep as the driver. (Thanks to Paul!)

Emissary-ingress now supports Envoy's Certificate Revocation lists. This allows users to specify a list of certificates that Emissary-ingress should reject even if the certificate itself is otherwise valid.

Previously, a LogService would always have Emissary-ingress communicate with the external log service using the envoy.service.accesslog.v2.AccessLogService API. It is now possible for the LogService to specify protocol_version: v3 to use the newer envoy.service.accesslog.v3.AccessLogService API instead. This functionality is not available if you set the AMBASSADOR_ENVOY_API_VERSION=V2 environment variable.

A future release of Emissary-ingress will remove support for the now deprecated v2 transport protocol in AuthServices. Instructions for migrating existing AuthServices from v2 to v3 can be found on the AuthService page. This change only impacts gRPC AuthServices. HTTP AuthServices are unaffected by this change.

You may now choose to enable TLS Secret validation by setting the AMBASSADOR_FORCE_SECRET_VALIDATION=true environment variable. The default configuration does not enforce secret validation.

Kubernetes Secrets that should contain an EC (Elliptic Curve) TLS Private Key are now properly validated.

Support for the Envoy V2 API is deprecated as of Emissary-ingress v2.1, and will be removed in Emissary-ingress v3.0. The AMBASSADOR_ENVOY_API_VERSION environment variable will be removed at the same time. Only the Envoy V3 API will be supported (this has been the default since Emissary-ingress v1.14.0).

Upgraded Envoy to address security vulnerabilities CVE-2021-43824, CVE-2021-43825, CVE-2021-43826, CVE-2022-21654, and CVE-2022-21655.

The Ambassador Agent now correctly supports requests to cancel a rollout.

Support for the Envoy V2 API is deprecated as of Emissary-ingress v2.1, and will be removed in Emissary-ingress v3.0. The AMBASSADOR_ENVOY_API_VERSION environment variable will be removed at the same time. Only the Envoy V3 API will be supported (this has been the default since Emissary-ingress v1.14.0).

Emissary-ingress will now watch for ConfigMap or Secret resources specified by the AGENT_CONFIG_RESOURCE_NAME environment variable in order to allow all components (and not only the Ambassador Agent) to authenticate requests to Ambassador Cloud.

Emissary-ingress has updated Alpine to 3.15, and Python and Go dependencies to their latest compatible versions, to incorporate numerous security patches.

Emissary-ingress now supports the metric ambassador_log_level{label="debug"} which will be set to 1 if debug logging is enabled for the running Emissary instance, or to 0 if not. This can help to be sure that a running production instance was not actually left doing debugging logging, for example. (Thanks to Fabrice!)

Emissary-ingress is now leveraging a new Envoy Proxy patch that allows Envoy to accept escaped '%' characters in its configuration. This means that error_response_overrides and other custom user content can now contain '%' symbols escaped as '%%'.

Support for streaming Envoy metrics about the clusters to Ambassador Cloud.

The Ambassador agent now receives commands to manipulate Rollouts (pause, continue, and abort are currently supported) via directives and executes them in the cluster. A report is sent to Ambassador Cloud including the command ID, whether it ran successfully, and an error message in case there was any.

Kubernetes Secrets that should contain TLS certificates are now validated before being accepted for configuration. A Secret that contains an invalid TLS certificate will be logged as an invalid resource.

Support for the Envoy V2 API is deprecated as of Emissary-ingress v2.1, and will be removed in Emissary-ingress v3.0. The AMBASSADOR_ENVOY_API_VERSION environment variable will be removed at the same time. Only the Envoy V3 API will be supported (this has been the default since Emissary-ingress v1.14.0).

Docker BuildKit is enabled for all Emissary builds. Additionally, the Go build cache is fully enabled when building images, speeding up repeated builds.

Emissary-ingress 2.1.0 generated invalid Envoy configuration for getambassador.io/v2 Mappings that set spec.cors.origins to a string rather than a list of strings; this has been fixed, and these Mappings should once again function correctly.

Changes to the weight of Mapping in a canary group will now always be correctly managed during reconfiguration; such changes could have been missed in earlier releases.

A Mapping that is not part of a canary group, but that has a weight less than 100, will be correctly configured to receive all traffic as if the weight were 100.

Using rewrite: "" in a Mapping is correctly handled to mean "do not rewrite the path at all".

Any Mapping that uses the host_redirect field is now properly discovered and used. Thanks to Gabriel Féron for contributing this bugfix!

Mappings with DNS wildcard hostname will now be correctly matched with Hosts. Previously, the case where both the Host and the Mapping use DNS wildcards for their hostnames could sometimes not correctly match when they should have.

If the ambassador Module sets a global default for add_request_headers, add_response_headers, remove_request_headers, or remove_response_headers, it is often desirable to be able to turn off that setting locally for a specific Mapping. For several releases this has not been possible for Mappings that are native Kubernetes resources (as opposed to annotations), as an empty value ("mask the global default") was erroneously considered to be equivalent to unset ("inherit the global default"). This is now fixed.

It is now possible to set a Mapping spec.error_response_overrides body.text_format to an empty string or body.json_format to an empty dict. Previously, this was possible for annotations but not for native Kubernetes resources.

Resources that exist as getambassador.io/config annotations rather than as native Kubernetes resources are now validated and internally converted to v3alpha1 and, the same as native Kubernetes resources.

Resource validation errors are now reported more consistently; it was the case that in some situations a validation error would not be reported.

Support for the Envoy V2 API is deprecated as of Emissary-ingress v2.1, and will be removed in Emissary-ingress v3.0. The AMBASSADOR_ENVOY_API_VERSION environment variable will be removed at the same time. Only the Envoy V3 API will be supported (this has been the default since Emissary-ingress v1.14.0).

Emissary-ingress supports getambassador.io/v2 CRDs, to simplify migration from Emissary-ingress 1.X. Note: it is important to read the migration documentation before starting migration.

The incremental reconfiguration cache could miss some updates when multiple Mappings had the same prefix ("canary"ing multiple Mappings together). This has been corrected, so that all such updates correctly take effect.

When using Kubernetes Secrets to store ACME private keys (as the Edge Stack ACME client does), an error would always be logged about the Secret not being present, even though it was present, and everything was working correctly. This error is no longer logged.

When using gzip compression, upstream services will no longer receive compressed data. This bug was introduced in 1.14.0. The fix restores the default behavior of not sending compressed data to upstream services.

Update to busybox 1.34.1 to resolve CVE-2021-28831, CVE-2021-42378, CVE-2021-42379, CVE-2021-42380, CVE-2021-42381, CVE-2021-42382, CVE-2021-42383, CVE-2021-42384, CVE-2021-42385, and CVE-2021-42386.

Update Python dependencies to resolve CVE-2020-28493 (jinja2), CVE-2021-28363 (urllib3), and CVE-2021-33503 (urllib3).

Previous built images included some Python packages used only for test. These have now been removed, resolving CVE-2020-29651.

It is now possible to set the circuit_breakers for AuthServices, exactly the same as for Mappings and TCPMappings. This makes it possible to configure your AuthService to be able to handle more than 1024 concurrent requests.

Any token delimited by '%' is now validated agains a whitelist of valid Envoy command operators. Any mapping containing an error_response_overrides section with invalid command operators will be discarded.

The Host CRD now correctly supports the mappingSelector element, as documented. As a transition aid, selector is a synonym for mappingSelector; a future version of Emissary-ingress will remove the selector element.

We're pleased to introduce Emissary-ingress 2.0.4 for general availability! The 2.X family introduces a number of changes to allow Emissary-ingress to more gracefully handle larger installations, reduce global configuration to better handle multitenant or multiorganizational installations, reduce memory footprint, and improve performance. We welcome feedback!! Join us on Slack and let us know what you think.

The x.getambassador.io/v3alpha1 API version has become the getambassador.io/v3alpha1 API version. The Ambassador- prefixes from x.getambassador.io/v3alpha1 resources have been removed for ease of migration. Note that getambassador.io/v3alpha1 is the only supported API version for 2.0.4 — full support for getambassador.io/v2 will arrive soon in a later 2.X version.

The getambassador.io/v3alpha1 API version and the published chart and manifests have been updated to support Kubernetes 1.22. Thanks to Mohit Sharma for contributions to this feature!

You can now set dns_type between strict_dns and logical_dns in a Mapping to configure the Service Discovery Type.

You can now set respect_dns_ttl to true to force the DNS refresh rate for a Mapping to be set to the record's TTL obtained from DNS resolution.

You can now set buffer_limit_bytes in the ambassador Module to to change the size of the upstream read and write buffers. The default is 1MiB.

The release now shows its actual released version number, rather than the internal development version number.

Large configurations no longer cause Emissary-ingress to be unable to communicate with Ambassador Cloud.

The l7Depth element of the Listener CRD is properly supported.

We're pleased to introduce Emissary-ingress 2.0.3 as a developer preview. The 2.X family introduces a number of changes to allow Emissary-ingress to more gracefully handle larger installations, reduce global configuration to better handle multitenant or multiorganizational installations, reduce memory footprint, and improve performance. We welcome feedback!! Join us on Slack and let us know what you think.

The environment variable AES_LOG_LEVEL now also sets the log level for the diagd logger.

You can now set dns_type in the AmbassadorMapping to configure how Envoy will use the DNS for the service.

It is no longer necessary to set DOCKER_BUILDKIT=0 when building Emissary. A future change will fully support BuildKit.

We're pleased to introduce Emissary-ingress 2.0.2 as a developer preview. The 2.X family introduces a number of changes to allow Emissary-ingress to more gracefully handle larger installations, reduce global configuration to better handle multitenant or multiorganizational installations, reduce memory footprint, and improve performance. We welcome feedback!! Join us on Slack and let us know what you think.

Upgraded envoy to 1.17.4 to address security vulnerabilities CVE-2021-32777, CVE-2021-32778, CVE-2021-32779, and CVE-2021-32781.

You can now set allow_chunked_length in the Ambassador Module to configure the same value in Envoy.

Envoy-configuration snapshots get saved (as ambex-#.json) in /ambassador/snapshots. The number of snapshots is controlled by the AMBASSADOR_AMBEX_SNAPSHOT_COUNT environment variable; set it to 0 to disable. The default is 30.

We're pleased to introduce Emissary-ingress 2.0.1 as a developer preview. The 2.X family introduces a number of changes to allow Emissary-ingress to more gracefully handle larger installations, reduce global configuration to better handle multitenant or multiorganizational installations, reduce memory footprint, and improve performance. We welcome feedback!! Join us on Slack and let us know what you think.

Ambassador Agent reports sidecar process information and AmbassadorMapping OpenAPI documentation to Ambassador Cloud to provide more visibility into services and clusters.

The optional stats_prefix element of the AmbassadorListener CRD now determines the prefix of HTTP statistics emitted for a specific AmbassadorListener.

The optional stats_name element of AmbassadorMapping, AmbassadorTCPMapping, AuthService, LogService, RateLimitService, and TracingService now sets the name under which cluster statistics will be logged. The default is the service, with non-alphanumeric characters replaced by underscores.

We have updated to k8s.io/klog/v2 to track upstream and to quiet unnecessary log output.

Logs now include subsecond time resolutions, rather than just seconds.

Set AMBASSADOR_AMBEX_NO_RATELIMIT to true to completely disable ratelimiting Envoy reconfiguration under memory pressure. This can help performance with the endpoint or Consul resolvers, but could make OOMkills more likely with large configurations. The default is false, meaning that the rate limiter is active.

We're pleased to introduce Emissary-ingress 2.0.0 as a developer preview. The 2.X family introduces a number of changes to allow Emissary-ingress to more gracefully handle larger installations, reduce global configuration to better handle multitenant or multiorganizational installations, reduce memory footprint, and improve performance. We welcome feedback!! Join us on Slack and let us know what you think.

Emissary-ingress 2.0.0 introduces API version x.getambassador.io/v3alpha1 for configuration changes that are not backwards compatible with the 1.X family. API versions getambassador.io/v0, getambassador.io/v1, and getambassador.io/v2 are deprecated. Further details are available in the Major Changes in 2.X document.

The new AmbassadorListener CRD defines where and how to listen for requests from the network, and which AmbassadorHost definitions should be used to process those requests. Note that the AmbassadorListener CRD is mandatory and consolidates all port configuration; see the AmbassadorListener documentation for more details.

Where AmbassadorMapping's host field is either an exact match or (with host_regex set) a regex, the new hostname element is always a DNS glob. Use hostname instead of host for best results.

The behavior of the Ambassador module prune_unreachable_routes field is now automatic, which should reduce Envoy memory requirements for installations with many AmbassadorHosts

Each AmbassadorHost can specify its requestPolicy.insecure.action independently of any other AmbassadorHost, allowing for HTTP routing as flexible as HTTPS routing.

Emissary-ingress 2.0.0 fixes a regression in detecting the Ambassador Kubernetes service that could cause the wrong IP or hostname to be used in Ingress statuses -- thanks, Noah Fontes!

Emissary-ingress 2.0.0 fixes a bug where mTLS could use the wrong configuration when SNI and the :authority header didn't match

The TLSContext redirect_cleartext_from and AmbassadorHost requestPolicy.insecure.additionalPort elements are no longer supported. Use a AmbassadorListener for this functionality instead.

The use_proxy_protocol element of the Ambassador Module is no longer supported, as it is now part of the AmbassadorListener resource (and can be set per-AmbassadorListener rather than globally).

An AmbassadorMapping will only be matched with an AmbassadorHost if the AmbassadorMapping's host or the AmbassadorHost's selector (or both) are explicitly set, and match. This change can significantly improve Emissary-ingress's memory footprint when many AmbassadorHosts are involved. Further details are available in the Major Changes in 2.X document.

An AmbassadorHost or Ingress resource is now required when terminating TLS -- simply creating a TLSContext is not sufficient. Further details are available in the AmbassadorHost CRD documentation.

By default, Emissary-ingress will configure Envoy using the V3 Envoy API. This change is mostly transparent to users, but note that Envoy V3 does not support unsafe regular expressions or, e.g., Zipkin's V1 collector protocol. Further details are available in the Major Changes in 2.X document.

The tls module and the tls field in the Ambassador module are no longer supported. Please use TLSContext resources instead.

The environment variable AMBASSADOR_FAST_RECONFIGURE is now set by default, enabling the higher-performance implementation of the code that Emissary-ingress uses to generate and validate Envoy configurations.

Service Preview and the AGENT_SERVICE environment variable are no longer supported. The Telepresence product replaces this functionality.

The edgectl CLI tool has been deprecated; please use the emissary-ingress helm chart instead.

You can now set respect_dns_ttl in Ambassador Mappings. When true it configures that upstream's refresh rate to be set to resource record’s TTL

You can now set dns_type in Ambassador Mappings to use Envoy's logical_dns resolution instead of the default strict_dns.

You can now set buffer_limit_bytes in the ambassador Module to to change the size of the upstream read and write buffers. The default is 1MiB.

Upgraded Envoy to 1.17.4 to address security vulnerabilities CVE-2021-32777, CVE-2021-32778, CVE-2021-32779, and CVE-2021-32781.

Update from Envoy 1.15 to 1.17.3

You can now set allow_chunked_length in the Ambassador Module to configure the same value in Envoy.

AMBASSADOR_ENVOY_API_VERSION now defaults to V3

Logs now include subsecond time resolutions, rather than just seconds.

Fixed a regression when specifying a comma separated string for cors.origins on the Mapping resource. ([#3609](https://github.com/emissary-ingress/emissary/issues/3609))

Envoy-configuration snapshots get saved (as ambex-#.json) in /ambassador/snapshots. The number of snapshots is controlled by the AMBASSADOR_AMBEX_SNAPSHOT_COUNT environment variable; set it to 0 to disable. The default is 30.

Set AMBASSADOR_AMBEX_NO_RATELIMIT to true to completely disable ratelimiting Envoy reconfiguration under memory pressure. This can help performance with the endpoint or Consul resolvers, but could make OOMkills more likely with large configurations. The default is false, meaning that the rate limiter is active.

Configuring multiple TCPMappings with the same ports (but different hosts) no longer generates invalid Envoy configuration.

Ambassador Agent now accurately reports up-to-date Endpoint information to Ambassador Cloud

Ambassador Agent reports ConfigMaps and Deployments to Ambassador Cloud to provide a better Argo Rollouts experience. See [Argo+Ambassador documentation](https://www.getambassador.io/docs/argo) for more info.

Add AMBASSADOR_JSON_LOGGING to enable JSON for most of the Ambassador control plane. Some (but few) logs from gunicorn and the Kubernetes client-go package still log text.

Fixed a bug where the Consul resolver would not actually use Consul endpoints with TCPMappings.

Ambassador now calculates its own memory usage in a way that is more similar to how the kernel OOMKiller tracks memory.

Fixed a regression where Ambassador snapshot data was logged at the INFO label when using AMBASSADOR_LEGACY_MODE=true.

Fix a regression from 1.8.0 that prevented ambassador Module config keys proper_case and preserve_external_request_id from working correctly.

Fixed a regression in detecting the Ambassador Kubernetes service that could cause the wrong IP or hostname to be used in Ingress statuses (thanks, [Noah Fontes](https://github.com/impl)!

Incorporate the Envoy 1.15.5 security update by adding the reject_requests_with_escaped_slashes option to the Ambassador module.