NGINX, HA Proxy and the Evolution of L7, Proxies, and Microservices

In a microservice architecture, services communicate with each other through L7 protocols such as gRPC and HTTP. Since the network is not reliable (and services can go down!), managing L7 communications is critical for reliability and scale.

Smart RPC

The first efforts at managing L7 came around 2010 in the form of smart RPC libraries. The team at Twitter created Finagle (https://twitter.github.io/finagle/), the team at Netflix created Hystrix (https://github.com/Netflix/Hystrix), and Google introduced gRPC (https://grpc.io/). The library concept wasn’t perfect though, because it was difficult to port and maintain the libraries in multiple languages. This problem became more difficult as polyglot architectures became more fashionable.

Smart Proxies

In 2013, AirBnB announced SmartStack (https://medium.com/airbnb-engineering/smartstack-service-discovery-in-the-cloud-4b8a080de619), which combined HAProxy and Apache Zookeeper. Quickly adopted by other companies such as Yelp, SmartStack was the spiritual ancestor of the modern day service mesh. SmartStack was designed as a sidecar, and deployed adjacent to each service. All service egress traffic was routed through SmartStack, which introduced client-side load balancing and resiliency patterns.

2016

2016 was a major year for proxies and service meshes. In early 2016, Buoyant announced Linkerd, which implemented Finagle as a sidecar proxy. This model enabled non-JVM users the ability to use Finagle as the core RPC protocol, adding resilience and observability to a microservice application. Linkerd helped popularize the service mesh concept.

In September 2016, Lyft announced Envo (https://envoyproxy.io/)y. Envoy, written in C++, provided rich L7 management capabilities (resilience, observability). Designed with microservices in mind, Envoy has a tiny memory footprint, broad protocol support (e.g., gRPC and HTTP/2), and zero downtime reloads.

The incumbents respond

NGINX and HAProxy weren’t going to take the challenge from Envoy Proxy lying down. NGINX released NGINX Plus R13 (https://www.nginx.com/blog/nginx-plus-r13-released/) less than a year after Envoy was announced, adding a runtime API for dynamic configuration and traffic shadowing. HAProxy released 1.8 (https://www.haproxy.com/blog/whats-new-haproxy-1-8/) soon thereafter, adding support for hitless reloads (finally!), HTTP/2, and a runtime API.

The proxy landscape today

Envoy Proxy is now a full Cloud Native Computing Foundation (https://www.cncf.io/) project, with a broad and diverse community. Of the big three proxies, Envoy is the only project that does not have a dominant commercial vendor. (We’ve written how this was one of the drivers (https://blog.getambassador.io/envoy-vs-nginx-vs-haproxy-why-the-open-source-ambassador-api-gateway-chose-envoy-23826aed79ef) for us to adopt Envoy in Ambassador.)

Envoy pioneered the use of dynamic APIs for management, and an ecosystem of additional open source projects that use Envoy has evolved. These projects generally function as so-called control planes (https://blog.getambassador.io/the-importance-of-control-planes-with-service-meshes-and-front-proxies-665f90c80b3d) to manage Envoy. Projects that use Envoy proxy include Consul Connect (https://www.consul.io/intro/getting-started/connect.html), Istio (https://istio.io/), and Ambassador Labs.

Summary

• Managing L7 is critical to modern cloud-native applications
• HAProxy, NGINX, and Envoy Proxy are evolving to meet these new requirements
• With neutral governance and the fastest growing community, Envoy Proxy looks to be the new standard for L7 proxies
• Most users don’t use Envoy directly; they use a control plane like Ambassador (https://www.getambassador.io/).

Learn More

Flynn (https://www.twitter.com/_flynn) gave a talk on this subject at DevOps Days Boston this year. You can check out the slides below.

If you have any questions or feedback on this topic, we’d love to hear it. Feel free to drop a line in the comments, join our Slack channel, or follow Flynn (@_flynn (https://www.twitter.com/_flynn)) on Twitter.

If you’d like to learn more about Envoy, check out our resources page: Envoy Proxy 101: What it is, and why it matters?. Learn more about Edge Stack