If present, the
Module defines system-wide configuration for Ambassador Edge Stack. You may very well not need this resource. To use the
Module to configure Ambassador Edge Stack, it MUST be named
ambassador, otherwise it will be ignored. To create multiple
Modules in the same Kubernetes namespace, you will need to apply them as annotations with separate
ambassador_ids: you will not be able to use multiple CRDs.
The defaults in the
There are many config field items that can be configured on the
Module. They are listed below with examples and grouped by category.
allow_chunked_length: truetells Envoy to allow requests or responses with both
By default, messages with both
Content-Transfer-Encoding are rejected. If
true, Ambassador Edge Stack will remove the
Content-Length header and process the message. See the Envoy documentation for more details.
envoy_log_pathdefines the path of Envoy's access log. By default this is standard output.
envoy_log_typedefines the type of access log Envoy will use. Currently, only
envoy_log_formatdefines the Envoy access log line format.
These logs can be formatted using Envoy operators to display specific information about an incoming request. The example below will show only the protocol and duration of a request:
envoy_validation_timeoutdefines the timeout, in seconds, for validating a new Envoy configuration.
The default is 10; a value of 0 disables Envoy configuration validation. Most installations will not need to use this setting.
would allow 30 seconds to validate the generated Envoy configuration.
error_response_overridespermits changing the status code and body text for 4XX and 5XX response codes.
By default, Ambassador Edge Stack will pass through error responses without modification, and errors generated locally will use Envoy's default response body, if any.
See using error response overrides for usage details. For example, this configuration:
would explicitly modify the body of 404s to say "File not found".
Two attributes allow providing information about the client's TLS certificate to upstream certificates:
forward_client_cert_details: truewill tell Envoy to add the
X-Forwarded-Client-Certto upstream requests.
set_current_client_cert_detailswill tell Envoy what information to include in the
Ambassador Edge Stack will not forward information about a certificate that it cannot validate.
server_nameallows overriding the server name that Envoy sends with responses to clients.
By default, Envoy uses a server name of
suppress_envoy_headers: truewill prevent Ambassador Edge Stack from emitting certain additional headers to HTTP requests and responses.
For the exact set of headers covered by this config, see the Envoy documentation
ambassador_idallows using multiple instances of Ambassador Edge Stack in the same cluster.
We recommend not setting
ambassador_id if you are running only one instance of Ambassador Edge Stack in your cluster. For more information, see the Running and Deployment documentation.
If used, the
ambassador_id value must be an array, for example:
defaultsprovides a dictionary of default values that will be applied to various Ambassador Edge Stack resources.
Module Defaults for more information.
enable_grpc_http11_bridge: truewill enable the gRPC-HTTP/1.1 bridge.
enable_grpc_web: truewill enable the gRPC-Web bridge.
The gRPC-HTTP/1.1 bridge can translate HTTP/1.1 calls with
Content-Type: application/grpc into gRPC calls: Ambassador Edge Stack will perform buffering and translation as necessary. For more details on the translation process, see the Envoy gRPC HTTP/1.1 bridge documentation.
Likewise, gRPC-Web is a JSON and HTTP-based protocol that allows browser-based clients to take advantage of gRPC protocols. The gRPC-Web specification requires a server-side proxy to translate between gRPC-Web requests and gRPC backend services, and Ambassador Edge Stack can fill this role when the gRPC-Web bridge is enabled. For more details on the translation process, see the Envoy gRPC HTTP/1.1 bridge documentation; for more details on gRPC-Web itself, see the gRPC-Web client GitHub repo.
grpc_statsallows enabling telemetry for gRPC calls using Envoy's gRPC Statistics Filter.
envoy_cluster_grpc_upstream_<stats>- only when
Please note that
<service> will only be present if
all_methods is set or the service and the method are present under
all_methods is false or the method is not on the list, the available metrics will be in the format
all_methods: If set to true, emit stats for all service/method names. If set to false, emit stats for all service/message types to the same stats without including the service/method in the name. This option is only safe if all clients are trusted. If this option is enabled with untrusted clients, the clients could cause unbounded growth in the number of stats in Envoy, using unbounded memory and potentially slowing down stats pipelines.
services: If set, specifies an allow list of service/methods that will have individual stats emitted for them. Any call that does not match the allow list will be counted in a stat with no method specifier (generic metric).
upstream_stats: If true, the filter will gather a histogram for the request time of the upstream.
proper_case: trueforces headers to have their "proper" case as shown in RFC7230.
header_case_overridesallows forcing certain headers to have specific casing.
RFC7230 specifies that HTTP header names are case-insensitive, but always shows and refers to headers as starting with a capital letter, continuing in lowercase, then repeating the single capital letter after each non-alpha character. This has become an established convention when working with HTTP:
Internally, Envoy typically uses all lowercase for header names. This is fully compliant with RFC7230, but some services and clients may require headers to follow the stricter casing rules implied by RFC7230 section headers: in that situation, setting
proper_case: true will tell Envoy to force all headers to use the casing above.
Alternately, it is also possible - although less common - for services or clients to require some other specific casing for specific headers.
header_case_overrides specifies an array of header names: if a case-insensitive match for a header is found in the list, the matching header will be replaced with the one in the list. For example, the following configuration will force headers that match
X-EXPERIMENTAL to use that exact casing, regardless of the original case used in flight:
If the upstream service responds with
x-my-header: 1, Ambassador Edge Stack will return
X-MY-Header: 1 to the client. Similarly, if the client includes
x-ExperiMENTAL: yes in its request, the request to the upstream service will include
X-EXPERIMENTAL: yes. Other headers will not be altered; Ambassador Edge Stack will use its default lowercase header.
Please see the Envoy documentation for more information. Note that in general, we recommend updating clients and services rather than relying on
add_linkerd_headers: truewill force Ambassador Edge Stack to include the
l5d-dst-overrideheader for Linkerd.
When using older Linkerd installations, requests going to an upstream service may need to include the
l5d-dst-override header to ensure that Linkerd will route them correctly. Setting
add_linkerd_headers does this automatically. See the Mapping documentation for more details.
max_request_headers_kbsets the maximum allowed request header size in kilobytes. If not set, the default is 60 KB.
See Envoy documentation for more information.
preserve_external_request_id: truewill preserve any
X-Request-Idheader presented by the client. The default is
false, in which case Envoy will always generate a new
strip_matching_host_port: truewill tell Ambassador Edge Stack to strip the port from the host/authority header before processing and routing the request. This only applies if the port matches the underlying Envoy listener port.
By default, any port is preserved.
admin_portspecifies the port where Ambassador Edge Stack's Envoy will listen for low-level admin requests. The default is 8001; it should almost never need changing.
lua_scriptsallows defining a custom Lua script to run on every request.
This is useful for simple use cases that mutate requests or responses, for example to add a custom header:
For more details on the Lua API, see the Envoy Lua filter documentation.
Some caveats around the embedded scripts:
- They run in-process, so any bugs in your Lua script can break every request.
- They're run on every request/response to every URL.
- They're inlined in the Ambassador Edge Stack YAML; as such, we do not recommend using Lua scripts for long, complex logic.
If you need more flexible and configurable options, Ambassador Edge Stack supports a pluggable Filter system.
merge_slashes: truewill cause Ambassador Edge Stack to merge adjacent slashes in incoming paths when doing route matching and request filtering: for example, a request for
//foo///barwould be matched to a
By default, the Envoy that ships with Ambassador Edge Stack uses a defailt of 1MiB soft limit for an upstream service's read and write buffer limits. This setting allows you to configure that buffer limit. See the Envoy docs for more information.
use_ambassador_namespace_for_service_resolution: truetells Ambassador Edge Stack to assume that unqualified services are in the same namespace as Ambassador Edge Stack
By default, when Ambassador Edge Stack sees a service name without a namespace, it assumes that the namespace is the same as the resource referring to the service. For example, for this
Ambassador Edge Stack would look for a Service named
upstream in namespace
Mapping would look for a Service named
foo in the namespace in which Ambassador Edge Stack is installed instead.
diagnosticscontrols access to the diagnostics interface.
By default, Ambassador Edge Stack creates a
Mapping that allows access to the diagnostic interface at
/ambassador/v0/diag from anywhere in the cluster. To disable the
Mapping entirely, set
With diagnostics disabled,
/ambassador/v0/diag will respond with 404; however, the service itself is still running, and
/ambassador/v0/diag/ is reachable from inside the Ambassador Edge Stack Pod at
https://localhost:8877. You can use Kubernetes port forwarding to set up remote access to the diagnostics page temporarily:
Alternately, to leave the
Mapping intact but restrict access to only the local Pod, set
See Protecting Access to the Diagnostics Interface for more information.
enable_ipv4determines whether IPv4 DNS lookups are enabled. The default is
enable_ipv6determines whether IPv6 DNS lookups are enabled. The default is
If both IPv4 and IPv6 are enabled, Ambassador Edge Stack will prefer IPv6. This can have strange effects if Ambassador Edge Stack receives
AAAA records from a DNS lookup, but the underlying network of the pod doesn't actually support IPv6 traffic. For this reason, the default is IPv4 only.
Mapping can override both
enable_ipv6, but if either is not stated explicitly in a
Mapping, the values here are used. Most Ambassador Edge Stack installations will probably be able to avoid overriding these settings in Mappings.
enable_http10: truewill enable handling incoming HTTP/1.0 and HTTP/0.9 requests. The default is
corssets the default CORS configuration for all mappings in the cluster. See the CORS syntax.
ip_denydefine HTTP source IP address ranges to allow or deny.
The default is to allow all traffic.
ip_allow is specified, any traffic not matching a range to allow will be denied. If
ip_deny is specified, any traffic not matching a range to deny will be allowed. A list of ranges to allow and a separate list to deny may not both be specified.
Both take a list of IP address ranges with a keyword specifying how to interpret the address, for example:
peer specifies that the match should happen using the IP address of the other end of the network connection carrying the request:
X-Forwarded-For and the
PROXY protocol are both ignored. Here, our example specifies that connections originating from the Ambassador Edge Stack pod itself should always be allowed.
remote specifies that the match should happen using the IP address of the HTTP client, taking into account
X-Forwarded-For and the
PROXY protocol if they are allowed (if they are not allowed, or not present, the peer address will be used instead). This permits matches to behave correctly when, for example, Ambassador Edge Stack is behind a layer 7 load balancer. Here, our example specifies that HTTP clients from the IP address range
184.108.40.206 will be allowed.
You may specify as many ranges for each kind of keyword as desired.
reject_requests_with_escaped_slashes: truewill tell Ambassador Edge Stack to reject requests containing escaped slashes.
When set to
true, Ambassador Edge Stack will reject any client requests that contain escaped slashes (
%5c) in their URI path by returning HTTP 400. By default, Ambassador Edge Stack will forward these requests unmodified.
Envoy and Ambassador Edge Stack behavior
Internally, Envoy treats escaped and unescaped slashes distinctly for matching purposes. This means that an Ambassador Edge Stack mapping for path
/httpbin/statuswill not be matched by a request for
On the other hand, when using Ambassador Edge Stack, escaped slashes will be treated like unescaped slashes when applying FilterPolicies. For example, a request to
/httpbin%2fstatus/200will be matched against a FilterPolicy for
Security Concern Example
With Ambassador Edge Stack, this can become a security concern when combined with
bypass_authin the following scenario:
bypass_authset to true. The intention here is to apply no FilterPolicies under this prefix, by default.
/prefix/secure/without setting bypass_auth to true. The intention here is to selectively apply a FilterPolicy to this longer prefix.
Have an upstream service that receives both
/prefix/secure/traffic (from the Mappings above), but the upstream service treats escaped and unescaped slashes equivalently.
In this scenario, when a client makes a request to
/prefix%2fsecure/secret.txt, the underlying Envoy configuration will not match the routing rule for
/prefix/secure/, but will instead match the routing rule for
bypass_authset to true. Ambassador Edge Stack FilterPolicies will not be enforced in this case, and the upstream service will receive a request that it treats equivalently to
/prefix/secure/secret.txt, potentially leaking information that was assumed protected by an Ambassador Edge Stack FilterPolicy.
One way to avoid this particular scenario is to avoid using
bypass_authand instead use a FilterPolicy that applies no filters when no authorization behavior is desired.
The other way to avoid this scenario is to reject client requests with escaped slashes altogether to eliminate this class of path confusion security concerns. This is recommended when there is no known, legitimate reason to accept client requests that contain escaped slashes. This is especially true if it is not known whether upstream services will treat escaped and unescaped slashes equivalently.
This document is not intended to provide an exhaustive set of scenarios where path confusion can lead to security concerns. As part of good security practice it is recommended to audit end-to-end request flow and the behavior of each component’s escaped path handling to determine the best configuration for your use case.
Envoy treats escaped and unescaped slashes distinctly for matching purposes. Matching is the underlying behavior used by Ambassador Edge Stack Mappings.
Ambassador Edge Stack treats escaped and unescaped slashes equivalently when selecting FilterPolicies. FilterPolicies are applied by Ambassador Edge Stack after Envoy has performed route matching.
Finally, whether upstream services treat escaped and unescaped slashes equivalently is entirely dependent on the upstream service, and therefore dependent on your use case. Configuration intended to implement security policies will require audit with respect to escaped slashes. By setting reject_requests_with_escaped_slashes, this class of security concern can largely be eliminated.
use_remote_address: falsetells Ambassador Edge Stack that it cannot trust the remote address of incoming connections, and must instead rely exclusively on the
true (the default), Ambassador Edge Stack will append its own IP address to the
X-Forwarded-For header so that upstream services of Ambassador Edge Stack can get the full set of IP addresses that have propagated a request. You may also need to set
externalTrafficPolicy: Local on your
LoadBalancer to propagate the original source IP address.
See the Envoy documentation on the
X-Forwarded-For header and the Kubernetes documentation on preserving the client source IP for more details.
xff_num_trusted_hopssets how many L7 proxies ahead of Ambassador Edge Stack should be trusted.
The value of
xff_num_trusted_hops indicates the number of trusted proxies in front of Ambassador Edge Stack. The default setting is 0 which tells Envoy to use the immediate downstream connection's IP address as the trusted client address. The trusted client address is used to populate the
remote_address field used for rate limiting and can affect which IP address Envoy will set as
xff_num_trusted_hops behavior is determined by the value of
use_remote_address (which is true by default).
use_remote_addressis false and
xff_num_trusted_hopsis set to a value N that is greater than zero, the trusted client address is the (N+1)th address from the right end of XFF. (If the XFF contains fewer than N+1 addresses, Envoy falls back to using the immediate downstream connection’s source address as a trusted client address.)
use_remote_addressis true and
xff_num_trusted_hopsis set to a value N that is greater than zero, the trusted client address is the Nth address from the right end of XFF. (If the XFF contains fewer than N addresses, Envoy falls back to using the immediate downstream connection’s source address as a trusted client address.)
Refer to Envoy's documentation for some detailed examples of this interaction.
listener_idle_timeout_mssets the idle timeout for incoming connections.
If set, this specifies the length of time (in milliseconds) that an incoming connection is allowed to be idle before being dropped. This can useful if you have proxies and/or firewalls in front of Ambassador Edge Stack and need to control how Ambassador Edge Stack initiates closing an idle TCP connection.
If not set, the default is no timeout, meaning that incoming connections may remain idle forever.
Please see the Envoy documentation on HTTP protocol options for more information.
keepalivesets the global TCP keepalive settings.
Ambassador Edge Stack will use these settings for all
Mappings unless overridden in a
Mapping's configuration. Without
keepalive, Ambassador Edge Stack follows the operating system defaults.
For example, the following configuration:
would enable keepalives every two seconds (
interval), starting after two seconds of idleness (
time), with the connection being dropped if 100 keepalives are sent with no response (
probes). For more information, see the Envoy keepalive documentation.
cluster_idle_timeout_mssets the default idle timeout for upstream connections (by default, one hour).
If set, this specifies the timeout (in milliseconds) after which an idle connection upstream is closed. The idle timeout can be completely disabled by setting
cluster_idle_timeout_ms: 0, which risks idle upstream connections never getting closed.
If not set, the default idle timeout is one hour.
You can override this setting with
idle_timeout_ms on a
cluster_max_connection_lifetime_mssets the default maximum lifetime of an upstream connection.
If set, this specifies the maximum amount of time (in milliseconds) after which an upstream connection is drained and closed, regardless of whether it is idle or not. Connection recreation incurs additional overhead when processing requests. The overhead tends to be nominal for plaintext (HTTP) connections within the same cluster, but may be more significant for secure HTTPS connections or upstreams with high latency. For this reason, it is generally recommended to set this value to at least 10000 ms to minimize the amortized cost of connection recreation while providing a reasonable bound for connection lifetime.
If not set (or set to zero), then upstream connections may remain open for arbitrarily long.
You can override this setting with
cluster_max_connection_lifetime_ms on a
cluster_request_timeout_mssets the default end-to-end timeout for a single request.
If set, this specifies the default end-to-end timeout for every request.
If not set, the default is three seconds.
You can override this setting with
timeout_ms on a
/ambassador/v0/check_readyis automatically mapped
/ambassador/v0/check_aliveis automatically mapped
By default, Ambassador Edge Stack creates
Mappings that support readiness and liveness checks at
/ambassador/v0/check_alive. To disable the readiness
Mapping entirely, set
Likewise, to disable the liveness
Mapping entirely, set
A disabled probe endpoint will respond with 404; however, the service is still running, and will be accessible on localhost port 8877 from inside the Ambassador Edge Stack Pod.
You can change these to route requests to some other service. For example, to have the readiness probe map to the
quote application's health check:
The liveness and readiness probes both support
rewrite, with the same meanings as for Mappings.
This lets you add resilience to your services in case of request failures by performing automatic retries.
circuit_breakerssets the global circuit breaking configuration defaults
You can override the circuit breaker settings for individual
Mappings. By default, Ambassador Edge Stack does not configure any circuit breakers. For more information, see the circuit breaking reference.
default_labelssets default domains and labels to apply to every request.
For more on how to use the default labels, , see the Rate Limit reference.
load_balancersets the default load balancing type and policy
For example, to set the default load balancer to
If not set, the default is to use round-robin load balancing. For more information, see the load balancer reference.