Python & Kubernetes: Rapidly Developing Python Microservices
Build a cloud development environment with Telepresence & Python
Kubernetes has become the de-facto standard for running cloud applications. With Kubernetes, users can deploy and scale containerized applications at any scale: from one service to thousands of services. The power of Kubernetes is not free — the learning curve is particularly steep, especially for application developers. Knowing what to do is just half the battle, then you have to choose the best tools to do the job. So how do Python developers create a development workflow on Kubernetes that is fast and effective?
There are two unique challenges with creating productive development workflows on Kubernetes.
- Most development workflows are optimized for local development, and Kubernetes applications are designed to be native to the cloud.
- Most Kubernetes applications either start off or evolve into a microservices architecture. Thus, your development environment becomes more complex as every microservice adds additional dependencies to test code. And in turn, these services quickly become too resource-intensive and exceed the limits of your local machine.
In this tutorial, we’ll walk through how to set up a realistic development environment for Kubernetes. Typically, we’d have to wait for a container build, push to registry and deploy to see the impact of our change. Instead, we’ll use Telepresence and see the results of our change instantly.
Telepresence is an open source project that lets you run your microservice locally, while creating a bi-directional network connection to your Kubernetes cluster. This approach enables the microservice running locally to communicate to other microservices running in the cluster, and vice versa. Since you’re running the microservice locally, you’re able to benefit from any workflow or tool that you run locally.
Step 1: Deploy a Sample Microservices Application
For our example, we’ll make code changes to a Go service running between a resource-intensive Java service and a large datastore. We’ll start by deploying a sample microservice application consisting of 3 services:
VeryLargeJavaServiceA memory-intensive service written in Java that generates the front-end graphics and web pages for our application
DataProcessingServiceA Python service that manages requests for information between the two services.
VeryLargeDataStoreA large datastore service that contains the sample data for our Edgey Corp store.
Note: We’ve called these VeryLarge services to emphasize the fact that your local environment may not have enough CPU and RAM, or you may just not want to pay for all that extra overhead for every developer.
In this architecture diagram, you’ll notice that requests from users are routed through an ingress controller to our services. For simplicity’s sake, we’ll skip the step of deploying an ingress controller in this tutorial. If you’re ready to use Telepresence in your own setup and need a simple way to set up an ingress controller, we recommend checking out the Ambassador Edge Stack which can be easily configured with the K8s Initializer.
Let’s deploy the sample application to your Kubernetes cluster:
kubectl apply -f https://raw.githubusercontent.com/datawire/edgey-corp-python/master/k8s-config/edgey-corp-web-app-no-mapping.yaml
Step 2: Set up your local Python development environment
We’ll need a local development environment so that we can edit the
DataProcessingService service. As you can see in the architecture diagram above, the
DataProcessingService is dependent on both the
VeryLargeJavaService and the
VeryLargeDataStore, so in order to make a change to this service, we’ll have to interact with these other services as well. Let’s get started!
1. Clone the repository for this application from GitHub.
git clone https://github.com/datawire/edgey-corp-python.git
2. Install the application dependencies with
pip (you may need to type
pip3 if you have Python 3 installed)
cd edgey-corp-python/DataProcessingService/pip install flask requests
3. Run the application (you may need to type
python3 if you have Python 3 installed):
4. Test the application. In another terminal window, we’ll send a request to the service, which should return
Step 3: Make code changes with Telepresence
To test a code change with Kubernetes, you typically need to build a container image, push the image to a repository, and deploy the Kubernetes cluster. This takes minutes.
Telepresence is an open source, Cloud-Native Computing Foundation project that solves exactly this problem. By creating a bidirectional network connection between your local development environment and the Kubernetes cluster, Telepresence enables fast, local development.
1. Download Telepresence (~60MB):
# Mac OS Xsudo curl -fL https://app.getambassador.io/download/tel2/darwin/amd64/latest/telepresence -o /usr/local/bin/telepresence# Linuxsudo curl -fL https://app.getambassador.io/download/tel2/linux/amd64/latest/telepresence -o /usr/local/bin/telepresence
2. Make the binary executable
sudo chmod a+x /usr/local/bin/telepresence
3. Test Telepresence by connecting to the remote Kubernetes cluster
4. Send a request to the Kubernetes API server:
curl -ik https://kubernetes.default.svc.cluster.localHTTP/1.1 401 UnauthorizedCache-Control: no-cache, privateContent-Type: application/jsonWww-Authenticate: Basic realm="kubernetes-master"Date: Tue, 09 Feb 2021 23:21:51 GMT
Congratulations! You’ve successfully configured Telepresence. Telepresence is intercepting the request you’re making to the Kubernetes API server, and routing over its direct connection to the cluster instead of over the Internet.
Step 4: Set up an Intercept
An intercept is a routing rule for Telepresence. We can create an intercept that will route traffic intended for the
DataProcessingService in the cluster and route all the traffic to the local version of the
DataProcessingServicerunning on port 3000.
1. Create the intercept
telepresence intercept dataprocessingservice --port 3000
2. Access the application directly with Telepresence. In your browser, go to http://verylargejavaservice:8080. Again, Telepresence is intercepting requests from your browser and routing them directly to the Kubernetes cluster.
3. Now, let’s make a code change. Open
edgey-corp-python/DataProcessingService/app.py and change
orange. Save the file.
4. Reload the page in your browser, and see how the color has changed from blue to orange.
That’s it! With Telepresence we saw how quickly we can go from editing a local service to seeing how these changes will look when deployed with the larger application. When you compare it to our original process of building and deploying a container after every change, it’s very easy to see how much time you can save especially as we make more complex changes or run even larger services.
Learn More about Telepresence
Typically, developers at organizations adopting Kubernetes face challenges slow feedback loops from inefficient local development environments. Today, we’ve learned how to use Telepresence to set up fast, efficient development environments for Kubernetes and get back to the instant feedback loops you had with your legacy applications.
If you want to learn more about Telepresence, check out the following resources: