Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2020 - Virtual, 4 Sep 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Developing the Kubernetes Python Client - Scott Lee, Facebook

Description

Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Developing the Kubernetes Python Client - Scott Lee, Facebook

The Kubernetes Python client allows users to programmatically interact with clusters using the Python language. With Python being one of the most popular programming languages, the Python client provides a way for those unfamiliar with Golang to still reap the benefits of a code interface to Kubernetes. This session will cover the repositories and resources that are used to generate the Python client and dive into examples of how the client is used. This talk will also cover how to get involved as a contributor to the Python client by covering processes and resources. Note: Brendan Burns gave a talk titled "!go, Interacting with and Extending Kubernetes in a Polyglot World" but it covered Kubernetes clients generally while this talk is specifically about the Kubernetes Python client and dives deeper into the nuances of the client and how to get involved as a contributor.

https://sched.co/ZetU

A

Hey everyone, my name is scott and in this presentation, developing the kubernetes python client I'll be talking about how the python client is developed a little bit about myself. I'm a systems engineer and the kubernetes python client is one of the areas in kubernetes I contribute to.

A

In my free time, I've been involved with the python client for about a year now, since last summer, and I found it was a great way to start contributing to kubernetes at the time I wasn't comfortable with goaling, so it was a good way to still contribute in a technical capacity.

A

The expectation for the viewers of this talk is to understand the primitives of kubernetes, so you should know about and understand, pods deployments and so forth and have at least some experience interacting with the kubernetes cluster through cube ctl.

A

Also, since the talk is about the kubernetes python client, a basic understanding of python would be helpful in this talk I'll briefly, go over some background about kubernetes, discuss the available clients and why they're useful then I'll go through setup for examples as well as demo, some examples, and, lastly, I'll cover some github repos for the python kubernetes client, as well as how you can contribute.

A

As you may already know, kubernetes has a distributed architecture. While there are many components, the two components that are important for this talk are first, the api server, which is a part of the master node, whether you're, interacting with the cluster with cube, ctl or another method such as curl commands. All calls go through the api server.

A

Second, the cubelet, which is found on every worker, node all communications to a node, go through the cubelet.

A

Kubernetes has officially supported clients ranging from go and python to javascript and haskell. Officially supported clients are found in the kubernetes client repo are generated using open api and are supported by the kubernetes community.

A

There are also several unofficially supported clients for languages, including closure, lisp, perl, php, ruby, rust and scala. The cool thing is, you can generate your own client, whether that's for learning purposes or just because you want to add features that are specific for your use cases I'll talk more about how that is done later. In this talk, let's take a step back and talk about what clients are and why you would use them.

A

Clients allow you to programmatically interact with cluster. While you can string together a whole bunch of cube ctl commands by writing a bash script. This can easily become inflexible.

A

One thing you can do with the client is to watch for events and then take an action. For instance, if the amount of space being used by persistent volumes in your cluster reaches a certain size or limit, you can send an alert to a slack channel or someone tries to create a pod with an image version that is known to have a vulnerability.

A

You can have the cluster, destroy it automatically.

A

With python being one of the most popular programming languages, the python client provides a way for those who are unfamiliar with goaling to still reap the benefits of an interface to kubernetes to get started. All you have to do is run pip, install kubernetes.

A

Once you have the kubernetes python package installed, you'll need a cluster. There are several options you can take. You can use a managed cluster, like gke from google cloud platform or eks from amazon web services, the downside being it costs money. Unless you have credits, you can also run a self-managed cluster using something like cube adm, but this also costs money. Unless you have credits and ends up being more work, another option which is free is to run a local cluster on your computer.

A

Two options are minicube, which uses virtualbox and kind which uses docker I'll, be using kind to show some examples of how to use the python client. I won't go through the setup for kind, but if you're using a mac, you can use the brew package manager and run brew, install kind, then you can run kind create cluster as shown on the screen, and it will create a kubernetes cluster.

A

The first example is the equivalent of running cube. Ctl git pods dash dash all dash namespaces. It lists all paws across all name spaces. In a cluster the config.loadcubeconfig line loads, the cubeconfig file found in your dot cube directory.

A

Then it uses the core v1 api to list pods for all namespaces. Just as the method name says, this returns a list of objects that have properties like status or metadata.

A

When I run the script you can see for every pod in my cluster it will output the pod's ip address, the namespace it exists in and the name of the pod.

A

The thing to note here is that this script was called from outside of the cluster. When I read the script it made calls to the kind cluster and the kind cluster returned data. This is important because in the next example the script is called from within the cluster.

A

This script is almost exactly the same as a previous script. It lists pods across all name spaces in the cluster. The difference is the config.load in cluster config line. This script needs to be called from a container running within a kubernetes cluster.

A

So, instead of executing the file and getting a result, I would need to create a docker file and a pod manifest to run the script within the cluster.

A

This is the docker file I will be using. It uses the python alpine image copies the script, installs the kubernetes package and runs a script.

A

Here I'm applying the manifest and then I'll check the.

A

A

You can see it's listing all the pods within the cluster, which is pretty much what the last example did, the difference being it's now running within the cluster.

A

This grip creates a deployment. It opens a file called nginx deployment.yaml and uses the contents to create a deployment in the default namespace using the apps v1 api, it's equivalent of running cube ctl create dash f, nginx deployment.yaml.

A

This is a yaml file that will be used. It launches three replicas of pods running nginx.

A

Once I run the script, it will create the deployment.

A

If I check the deployments, you'll see that the deployment has been created.

A

And you can see the three replicas of the deployment within the listing of the pods. In this last example, the script uses the core v1 api to add: a label with key foo and value bar to a node.

A

Before I run the script, I want to show you that there's no labels with kia foo on any of the nodes in the cluster.

A

Now I'll run the script which will apply the labels and if I get the labels on the nodes again, you can see there's a custom label with key of foo and value of bar. If you want to see more examples, you can find them in the kubernetes client python, repo, there's an examples. Folder that contains the code of examples I covered and several others there's also a docs directory under the kubernetes slash docs folder within the python repo.

A

For example, the apps v1 api lists the available methods that you can call with the python client and most methods like the create namespace deployment method, have examples, parameters and return types, there's also generated reference documentation on the kubernetes.I o website.

A

In fact, we can find the same api for the create namespace deployment method by going to deployment v1 apps write operations and create here, you can see that the http request path, apis, apps, v1, namespaces, namespace parameter deployments matches what we saw earlier.

A

So if we go back to the top, see that create namespace deployments also has a path of apis, apps, v1, namespaces, namespace, parameter and.

A

Deployments, there's also lots of other useful information in this api documentation. So please take a look earlier. I mentioned that you can generate your own kubernetes client. This is possible through the kubernetes client gen repo, which uses the open api generator and custom generator scripts that are shared among all the client libraries.

A

The readme does a good job describing how to use the scripts and you'll find the scripts in the open api folder, for example. Here's the script that generates the python client.

A

Python.Shell next I'll talk about the repositories specific to the python client. Python client is one of the few official clients that is split into two repos python and python base.

A

The python repo is basically the output of the generator script python.shell from the gen repo that I showed you earlier, of course, there's more to it. But if you explore the repo you'll see that a lot of the files are generated.

A

For example, if we go to kubernetes client and then api client you'll see that there's a comment at the top that specifies that this file is generated.

A

Watch out for these generated files, especially if you plan on contributing a change, there's no point in trying to change a file that was generated as it will just get overwritten later in general. It's always a good idea to open an issue before making a pr, so you can describe what change you want to make and why and more experienced contributors can give you feedback.

A

The python based repo contains hand-tuned pieces of code that are not in the standard restful api. This code is not auto-generated by the scripts in the generic code in the python base, repo is added to the python repo using git sub modules.

A

Here you see config hack stream and watch.

A

If we go back to the python repo you'll see that there's the same reference files, config, dynamic stream and watch.

A

For builds and tests, the client repos use circle. Ci you'll typically see a tool called prow being used for other parts of the kubernetes ecosystem.

A

The reason that the client repos don't use prow is most likely because the client projects were started before prow was available, but circle ci continues to serve the client repository. Well, so there's no reason to move off of it.

A

The release versioning also works a bit differently. So I'll explain that here, python client, major release versions correspond to kubernetes minor releases, so python client, 10 maps to kubernetes 1.14 and python client 11 maps to kubernetes 1.15.

A

Like kubernetes, though, the python client supports the last two releases of kubernetes, so python client 11 will support kubernetes 1.15 as well as versions 1.14 and 1.13.

A

If you're interested in contributing to the python, client or any of the other officially supported clients, we'd love your help, some of the types of work open source contributors can do include documentation, issue, triage releases, bug, fixes feature development and increasing test coverage for the python client. Specifically, some cool features we'd like to see are type hinting with mypi and creating more release automation.

A

We have bi-weekly meetings on monday mornings for more information visit. This lock channel kubernetes dash client and that's the end of my presentation. I hope I've been able to shed some light in how the python client works. I look forward to seeing people contributing in the future.