DataDog APM Tracing

In this tutorial, we'll configure Ambassador to initiate a trace on some sample requests, and use DataDog APM to visualize them.

Before You Get Started

This tutorial assumes you have already followed the Ambassador Getting Started guide. If you haven't done that already, you should do that now.

After completing the Getting Started guide you will have a Kubernetes cluster running Ambassador and the Quote of the Moment service. Let's walk through adding tracing to this setup.

1. Configure the DataDog agent

You will need to configure the DataDog agent so that it uses a host-port and accepts non-local APM traffic, you can follow the DataDog documentation on how to do this.

2. Configure Envoy JSON logging

DataDog APM can correlate traces with logs if you propogate the current span and trace IDs with your logs.

When using JSON logging with Envoy, Ambassador will automatically append the dd.trace_id and dd.span_id properties to all logs so that correlation works:

---
apiVersion: getambassador.io/v1
kind: Module
name: ambassador
config:
  envoy_log_type: json

3. Configure the TracingService

Next configure a TracingService that will write your traces using the DataDog tracing driver, as you want to write traces to your host-local DataDog agent you can use the ${HOST_IP} interpolation to get the host IP address from the Ambassador containers environment.

---
apiVersion: getambassador.io/v1
kind: TracingService
metadata:
  name: tracing
spec:
  service: "${HOST_IP}:8126"
  driver: datadog
  config:
    service_name: test

4. Generate some requests

Use curl to generate a few requests to an existing Ambassador mapping. You may need to perform many requests since only a subset of random requests are sampled and instrumented with traces.

$ curl $AMBASSADOR_IP/httpbin/ip

5. Test traces

Once you have made some requests you should be able to view your traces within a few minutes in the DataDog UI. If you would like more information on DataDog APM to learn about it's features and benefits you can view the documentation.

More

For more details about configuring the external tracing service, read the documentation on external tracing.