Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2019 (Barcelona), 22 May 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Writing kubectl Plugins for Everyone: Develop, Package & Distribute - Ahmet Alp Balkan

Description

Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io

Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects

Writing kubectl Plugins for Everyone: Develop, Package & Distribute - Ahmet Alp Balkan, Google & Maciej Szulik, Red Hat

Many users install extra tools for tasks they can't do with kubectl. Did you know you can write and distribute your own kubectl subcommands? kubectl had notion of plugins since 2016, but after listening to the community feedback, SIG CLI revisited the idea, and new and simpler plugin mechanism is now available in kubectl. The new mechanism allows anyone to write and distribute their own subcommands to kubectl. This opens the door for community contributions to make kubectl better without having to propose a feature to the kubectl core. During this session, Maciej (lead of SIG CLI) and Ahmet (developer of Krew plugin manager) will explain: * how kubectl plugin mechanism works under the covers * how does a good plugin look like, best practices & available libraries * hands-on demo of developing a plugin * how to package plugins for multiple OS/distros and make them discoverable

https://sched.co/MPa2

A

Welcome, my name is ma che and I'm working for Red, Hat and I'll be talking for you about, keep CTL plugins, and this is my friend Ahmed I'm.

B

A developer advocate at Google cloud: okay,.

A

So, as you probably heard earlier, keynote communities is about extensibility. We have multiple options for extending kubernetes, to name a few CRTs. Allow you to extend API API extensions. Do exactly the same thing you can extend scheduler, you can write controllers to support the CR DS or the API extensions.

A

There are Network plugins and finally, there are cube CTL plugins. So if you touch the cube CTO plugins about a year ago, or so, you probably will be surprised that everything has changed. We literally scratch the entire old mechanism because it was very inefficient. It put a lot of burden onto the authors as well as onto the users, and we decided that the mechanism is wrong.

A

We spent a bunch of time looking at the available mechanism in different options and we went with the gate, plugins solution approach. So what is an extension for cube, CTL? Well in the short way, it's just a binary, a binary that enhances the current kubernetes cube CTL functionality, and let me quickly ask you two questions who which one of you used: cube: CTO plugins, either the old ones or the new ones? Oh cool.

A

That's pretty nice, how many of you actually spend some time and wrote their own plugin? Oh, that's pretty cool, ok, so more or less you probably know what we will be talking about. So, like I, said cube. Ctl plugins is about enhanced the capabilities. The built-in functionality of cube CTO.

A

This will support, keep CTO. On top of the official commands, you can build your own, which will enhance your custom workflows. There are just for you, but maybe you are brave enough or you want to share with the world and publish the plugins, and we will be talking about that in a in a couple more minutes or eventually, you feel that the current flow that you need to implement will be better.

A

If you do it your own way, it will feel more natural for yourself to have it implemented differently than what we decided in the cube CTO world. So, let's state a problem I want to see what the permissions I have currently for working with the coop community's resources. These days there are commands that allow you to do this. You can get that information easily from kubernetes using cube CTO, but it is very cumbersome if you're working a lot with the permissions with the are back.

A

You can easily create your own plugin, like the one in the slide. Somebody created a nice graphical slide that easily shows which the resources and what actions I can work with, and what are the permissions that I have to work with those resources?

A

If you find it useful, just look for I can rename access metrics, it's pretty cool, so other reasons why you want to work on your own plugins is the official commands take a lot of time to get the proofed implemented released? There's a long process of releasing new commands. It has to go through 60 Li approval process. It has to follow strict guidelines for for cube CTO when you're working on the plugins you're not constrained by any of these, and you can release little on a daily basis.

A

If your feels oh, and if you have users, you can fix their box and release instantly with the fixes with keeps ETL built-in plugin built-in commands, we have much more rigorous schedule. We are releasing every quarter. It takes time to release a plug to release a command and then graduate it through several stages so that we can fully support it and promote it for users to to use eventually.

A

Well, when you are creating a standalone, binary, / plugin, you might end up creating some funky names like arc hill or whatnot, whereas if you build it a plugin, the flow of invoking the commands will feel much more natural when you will be invoking keep CTL kill. The thing that you want to do so working with plugins makes the flow more natural.

A

And finally, if you heard the keynote from last year, cube Khan and Seattle, melanie Sabula from Airbnb, mentioned that they are encapsulating. They are wrapping cubes ETL to provide a consistent workflows for the engineers, and the reason for that is particularly is ok, I'm, not saying it's wrong, but the thing is they are wrapping cube CTO to ensure that all the engineers have a consistent way of interacting with the platform.

A

Why not, instead of building bash scripts, encapsulate that logic and put it into a cube, CTO plugin that way the engineers were feel feel more natural, interacting with the kubernetes platform.

B

So, let's take a look at how to actually go ahead and develop some cue CTL plugins. The development plugins actually is not a big deal. You can probably go home and develop a plug-in yourself. So the first thing that you need to do is actually write. Some code and I could be in any language of your choosing. You can write Q, CTL, plugins and bash. You can probably write them in Fortran if you wanted to do that. So as long as you name, your executable q, CTL foo, you're.

B

Writing a sub comment to QC gel named foo, and if you place this executable in your path, environment, variable you'll be able to invoke this plugin through QC GL. So.

B

How does the plugins work for me from the cube details perspective if you type Q CTL food to your command line, keep CTL is making this exact. The system called called exec, V and exactly system call is replacing the Q CTL process, with the plug-in executable that you have provided. So when the system call is involved, the environment, variables and standard descriptors, such as the standard house and their input, and all these things- and these descriptors is all the environment.

B

Variables are inherited by your plugin, so whatever Q CTL has invoked with your plugin is going to have access to those and your plugin actually determines the exit code of the Q CTL command line.

B

So right now we're going to have a short demo of how to develop a sample plugin. We also have at this URL. If you go to github.com / Rene's / sample CLI plug-in you'll, be able to find a simple plugin implementation that you can go home and clone the repository and play with it yourself.

B

So here I'm going to switch to my command line. So, let's try to write a fun plugin all right who here likes bash? Hey, that's good! So here I'm gonna go ahead and write a file called cube, CTL colorful. So what we're trying to do is create a bash script that is in vocable that directly executes and what I'm doing here is taking the arguments passed. The cube, CTL and I'm piping them to a program called locale right.

B

The next thing I'm going to do is I'm going to make this file executable, Q, CTL, colorful and then I'm going to extend my path environment variable with the current directory that I'm on so that keeps ETL can find that right. So when I run this, if I go and type QCT I'll plug in list grab, colorful you can see. Qcl is finding that plugin that is residing in my path. So let's go ahead and invoke this cube.

B

Ctl colorful and, as I said before, the arguments that I pass here are actually going to get passed, so the actual Q CTL. So if I do get nodes, hopefully we're gonna get a colorful list of nodes. It's the Wi-Fi permits, awesome, yep.

B

Yeah is it always giving the same color there's not sure. So, let's go back. Okay,.

A

So once you're done with your actual plug-in, what are the important things that you should be taking care of?

A

You do want to be consistent with gives you deal even though I said that, when you're working on a plug-in, you don't have to it's very good to be consistent and the reason for that is keeps ETL users or cubic aural users are familiar with certain standards certain patterns. If you follow those standards and patterns, you will get more users easier to consume your plugin, rather than if you come up with something completely different.

A

Packaging and distribution and updates are another topics that we'll be covering in a bit. So let's expand the consistency topics so, for example, plugin flag, naming command, naming all that it would be good to support the same functionality as kubernetes QTL does so. For that reason, we created something that is called CLI runtime. A CLI runtime is a library for all the plug or command out.

A

There's that you can consume it, whether you want you're thinking about writing a cube, CTL plugin, but not necessarily, it contains a set of polymorphic helpers, which is literally for lots of weird and funky stuff, but above all, the most important thing that we have. There are a consistent way for providing and consuming flags for reading conflicts as well as for printing.

A

So if you're thinking, how am I going to be printing objects in the same way that keeps you TL get or describe or other comments are, this is already done for you just use this library. It has all the libraries. So if you're familiar with cube, CTL get parts Oh yum, you can have the same functionality for free when you will use this particular library.

A

Naming naming is another important thing, I've mentioned before consistency with what cube does that also includes working with the verb plus action pattern that we set up in the entire cube CTL as well? What is important in case when you are producing your plugin is the uniqueness of the name I'm, pretty sure that each and every single one of you can come up with a login name for logging into your console, but log in is so ubiquitous that I'm pretty sure that it will be a hundred two hundred plugins named plug log in.

A

Why not just going specifically if I'm thinking about logging into as your name it as your log in if I'm thinking about openshift logging, let's call it open shift log in if I'm talking about different provider, let's call it provider log in this will make your plugin stand out. It will be searchable, it will be fine with the login name, but the fact that it will be unique will make you stand out and it will not conflict with the others.

A

Finally, for for the plug-in naming a little bit of nasty details, so, as Ahmed mentioned before for keep CTL plugins, all the binary has to be named cube, CTL and whatever comes after. The dash is the name of your command you're, probably asking what about commands with multiple names? What about commands with dashes and all that keeps ETL by default, will match the plug-in with the longest name? So, if we're thinking about, if we create a cube, CTL foo keep CTL foo.

A

The cube CTL for invocation is what it will be passed to if we invoke that same binder that same assuming, we have that plug-in installed cube city of foo and we invoke it as cube. Ctl foo bar the bar is actually the first argument being passed to keep CTL foo, but on the other hand, if we had a binary that will be named cube, CTL foo bar the foo bar will be the one match over the cube, CVL foo the question about the under the and the name.

A

We solved it because we reserved the for splitting the name. The sub commands names. So if you want to have hyphens in your names, you're going to use underscore in the binary name- and there are actually many many more intricacies into naming the binaries- that people came up throughout the entire process when we were creating the mechanism for plugins I highly, encourage you. If you're interested in to read the cap, which is number 24 for plugins. It contains additional details for how we are resolving resolving the names and transforming those into actual plug-in and vocation.

B

So, let's talk about the plugin management, so right now you see TL. If you go home and try to use plugins as a user, it does not give you a way to install plugins. So what you would do if you're trying to trace history, keep CTL plugins you found on the Internet. Is you either find like a zip file that the developer has created? You download the zip file?

B

You extract the files to somewhere on your path or you're, probably finding like your homebrew package for Mac or maybe like an EPS or DNF package for Linux distributions.

B

So if you wanted to install plugins yourself today, that's kind of a hassle and similarly once you install those plugins, especially the ones that you took out of the zip files and put put onto somewhere in your computer, you can't really update them right and when it comes, when the time comes to actually clean those stuff up like to remove plugins, you have to go figure out what you installed first, so that you can actually remove all the files that you actually installed as part of the plugin right and as a developer.

B

Hopefully, you know you'll go home and try to develop some plugins after this talk. If you wanted to advertise your plugin like there is no central place that people can go and see a list of all the plugins that are available, and if you wanted to actually write a plug-in that lets say you used go to write a plugin, so naturally your plugin is probably going to work on Mac, Linux and Windows. So should you like figure out how to do the package management for Windows or Mac and Linux separately?

B

So these were the problems that we observed. That would eventually cause Lord roadblocks in the adoption of Q CTL plugins right. We wanted this experience to be smooth, so we had to do something so meet the crew project. Crew is aq, CTL, plugin manager. It is developed at Google. It was developed by Luke Burchard. He was my intern in internal last summer, so we've been working on crew project over a year. Now, almost and crew simplifies the plug-in management both for QC tell users as well as Q CTL plugin developers.

B

As of a few months ago, we have incorporated crew in six CLI as a sub project, so I want to show you how you can go home and install crew and try to use crew as a cue GTL user, so here I'm going to switch to my command line back again, a rather I'm gonna switch to my browser first. So, as I mentioned before, we have this repository on github Corey's sakes crew.

B

If you want here and if you scroll down a little bit, we have this small install script that installs crew for you you're like why? Don't you install crew with a package manager? Instead, the answer is, as we're gonna see soon. Crew actually is a plugin itself, so the plugin manager is managing other plugins. So let's go back and what I already I, already installed crew so like I, didn't want to go through the conference Wi-Fi.

B

So if you type Q CTL crew, you're gonna see that crew itself is a plug-in and then it has all these commands available right. So, let's try to see what we can do with it. If I type, Q, CTL crew search, you're gonna see a list of plugins that are available on crew today and we're gonna come to that about where we find these plugins. So let's choose one of them: I'll choose the first one, the XS matrix and then let's try to install it huge ETL crew, install access matrix.

B

So when I run this command, what crew just did? Is it parse the plugins manifest which we'll talk about later? It saw the download URL it downloaded the plug-in from the internet and it extract the plugin the way the author of this plug-in wanted it to me, and then it finally has created a symbolic link somewhere in my path, so that I can invoke the plugin. So if I go ahead and do QCT our plugin list, you're gonna see that I have the Q CTL excess metrics plug-in here with the underscore much I described earlier.

B

So if I type Q CTL excise matrix. Hopefully this plugin is gonna run and it's gonna show me what can I do on my cluster right now? Let's see yeah. So it looks like these are the api groups that I have access to, and these are the like operations that I can do on these API groups. So this is how easy it is to find the plug-in, install it and start using it.

B

So if I wanted to go ahead and get rid of this plug-in now, I could just type QC gel crew remove excess matrix and then it will be gone. So this is crew in a nutshell, as a user.

B

So, let's talk about a crew project a little bit more crew is not trying to be a fully fledged package manager. What it's trying to do is get a binary, that's available on the Internet to your computer and do it sim link in a way that you're the QC GL can find it so that you can invoke it? Actually, it doesn't do anything complicated right now, it's basically just pulling the latest version. That's available.

B

It's not trying to get into the difficult problems of actually writing a package manager and, as I showed earlier, we have a centralized index of plugins that we go and we have all the plugins that are available to crew there. So this is great for discoverability, like if you're developing a plugging yourself, you can just add your plugin there and everyone will be able to find it.

B

And similarly, as a user, you have one place to discover plugins, but it's kind of difficult to manage the centralized index, because how the way it omits plugins there, which plugins are eligible for it- and you know we're trying to figure out a way to break this apart and have these centralized plug-in repos so that maybe Red Hat will have their own plug-in index and you'll be able to tap in there and get the plugins offered by Red Hat and maybe, like your company internally, will develop plugins that are available only to your company.

B

Another thing crew does, is you write a plugin manifest once and you can have different installation methods for Mac, Linux and windows? So as a plugin developer, you don't have to worry about like all these different package managers in different operating systems, so you might be wondering okay, let's say I, do all the plug-in. How do I use it with crew? Well, first, you need to make your plug-in package. Let's say this is a tar wall or a zip file available publicly somewhere on the Internet, and then you have to write a plugin manifest.

B

This is a file that I'm going to show soon, and then you see if the manifest is working correctly when you install it locally and then you just make a pull request of the crew index repository, which I'm also going to show soon. So let's go ahead and try to package our plugin I'm gonna go back to my terminal, and here I have the plugin that I created earlier, which is named differently.

B

I drop the QC GL prefix, because I don't have to call it that way, which why I'm gonna do that I'm gonna show them in a bit. So let's take a look at what's inside this file.

B

Okay, so that's just a plug-in that I showed earlier nothing that interesting. So, let's get add colorful Sh and then you get commits first import and then get push. Origin master.

C

Alright so looks.

B

Like I, just publish the plug-in and let's do brows and it seems like I- have a plugin on the internet right now. So what I want to do with this is I want to make a first version right and then I do get tagged, V 0.1, let's go with that and then, if I do get to push tags, I'm, gonna upload this tag to the Internet right.

B

Okay, so looks like I, have a release: let's go ahead and find it yep. So what I do here is I choose the tarball or the zip doesn't matter, I copy its address, because that was that's where my plugin is right, and then I mentioned this baguette manifest file earlier, and let's talk about that a little bit. So this manifest file is something that we're trying to finalize crew is in its early versions and we're trying to figure out. What's the best way to do this, and this manifest file describes.

B

Where is my plugin, which platforms I support, and how does it installed on these platforms? If I come down here, you're gonna see that I have this platforms under the spec and I'm. Saying match the oysters that are Darwin, which is Mac OS I am going to the URI part and I am just putting the URL here and I need to actually find the shout 256 hash, which I'm not going to go through that right now.

B

This basically ensures that there's no tempering while the file is downloaded from the internet and once I do that I just take this Yama file and I go to the crew index repository.

B

So if you go to the crib index repository today, you're gonna find that we have a directory called plugins down here and let's click on that. So this is where all the plugins live, and if you go to your browser, let's see, do you think I can just drag and drop and make a pull request. Let's see, let's try that.

A

Better try yeah.

C

Alright so looks.

B

Like I can make a pull request, adds new plugin, color or full Napoleon alright. So if I just go ahead and propose this change to the CRU index, repository what's gonna happen is that I'm gonna go and say: I've read the naming guide and I validate the manifest, which is a total lie, and then I'm gonna go ahead and create a yep I just created a pull request.

B

So if much a approves that I will have a plugin that is available to all of you when you go home, you'll, be able to say QC, GL crew install colorful. So this is how you package and ship a plug-in with its crew.

B

So this is the plugin manifest that I was mentioning earlier, so the name part of the manifest is basically describing. What is your executables name? And if you look at the platform spec that I mentioned earlier, you're gonna see that if you have certain binaries that are like say, Mac or Windows and you compile for a certain architecture, you can match all mesh to all these things with just label selectors that you're already familiar with in kubernetes, and we have this file section which basically describes I.

B

Have this archive file and I want to copy particular files out of it to these destinations? And finally, the bin field actually specifies the entry point of your plugin like this is the binary that you should call. Then crew crew goes ahead and creates a symlink with the name that is specify all the way up there and that's the name of your plug-in executable when it's invoked.

A

Okay, now, since we have so many of you in this room, I'm very happy to see all of you, we have a call to action to all of us, I'm, pretty sure that each and every single one of you has an idea for at least one plugin, if not more and can be wrappers around the current existing commands, because you find them limiting or they are not fulfilling. Your role and I would like to see each and every single one of you writing at least one plug-in. Why it's important?

A

If we make more plugins there will be more people interested in using it and more people interested in actually expanding the ecosystem that we currently have I.

A

Need to pick what I was going to say more so I'm pretty sure that also since kubernetes is been in the wild for four years or so I'm, pretty sure that you already have your custom workflows that I was talking about earlier today, you probably build some kind of a tolling. If you feel that there will be users that will also benefit the same functionality that you developed, locally, publish it, make it available to other users: let's consume our.

A

Let's make our lives easier by helping one another by producing the buyer by producing the plugins, sharing the plugins and making our worlds better.

A

And actually this is more or less final. Note is crew, and the plugins mechanism is part of the six CLI projects. I'm one of the six CLI leads Ahmed is very active in the six year line. If you want to learn more about the 6 July itself or what we do where we were, if you have problems developing plugins, we would like to hear that come find us either meet myself or Ahmed or other folks from the six Eli. We will be having a intro section to later today at 2:00.

A

I can't remember the rule name, but it was somewhere somewhere here. If I remember correctly, f1 may be generally to 6 July introductions I'll be doing a coat tour as well, which should help you also to find yourself in the queue CDL code base if you're interested in develop in in helping us with this. That will be pretty cool as well and, lastly,.

B

You know again, these stuff are really new, we're as a community we're trying to set the standards for it like where we want to. We want your opinion in terms of what should be a playin like what should be an ideal name for a plug-in light. Look like or you know, is cru excuse our experience good. Should we make the cue CTL plug-in mechanism and cue CTL better.

B

So if you go home and try with and try some plugins, maybe install crew, try to install a few plugins, maybe try to develop one and send us feedback. That would be great.

A