Kubernetes Kubernetes Contributor Summit - NA (Detroit) 2022, 2 Dec 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: KCSNA22 - Seeding kubectl bind getting over operators and CRDs as a community

Description

Speaker: Stefan Schimanski

We have CRDs and operators since 2016. Where are we heading as community with them? Are they solving the problems we have today?

We believe: CRDs and the patterns around them block innovation and collaboration in the ecosystem. The problem landscape is not anymore that of 2016. CRDs were never made to connect services of multiple clusters, or to consume APIs of 3rdparty service providers. Kubernetes does not provide the service consumption model of today. The ecosystem has learned to work around this, with lots of toil and complexity.

"kubectl bind" is a vision for a world where SaaS can natively be consumed, without all the complexity of running operators and the immense cost of building them:

$ kubectl krew install bind
$ kubectl bind service https://mangodb.com

BOOM – we can create resources. Vendor neutral. Native. Nocode. Nopods. Nooperators.

A

So welcome to our talk. We want to present something we have worked on for a couple of weeks now. Basically conference driven development as everybody loves it and um it started out in August and I realized something which was kind of embarrassing, because I was in the middle of this construction team there, which built the bridge in kubernetes. We do not really support software as a service, so my usual example is I. Have a cube cluster and I need a mySQL database. How can I declare a mySQL database ads into kubernetes?

A

It was hard at the time before CID is working, but even today it's hot, like you can find a MySQL operator and install that somehow with Helm and hope it works.

A

You cannot easily get a mySQL database declaratively in kubernetes from any cloud provider. So if I want to have an RDS from from Amazon, for example, it exists, I can click through the Amazon API and create an RDS instance and somehow get a service into my cluster. That is all complicated and the question was: can we make that simpler, much easier and supports us in a native way? So this is what Som it's a Twitter thread from from August.

A

So no stars lots of discussion around there, there's ack and similar things for Azure, so they are operator kits which put cloud apis into Cube. They exist. But my claim here in the talk is we shouldn't build those things. We should have something better and yeah, basically surrender six years building operators, everybody is involved in those operators.

A

They are complex right, that's thousand lines of codes of code for simple things like offering and crd for cloud API, so six years of building operators and maybe not the best use of time and installing operators is one thing- ham install an operator every time. I do that I. Ask myself is this maintained? Does it work in a year and I really don't want to maintain this thing? So I have to upgrade, maybe once in a while, so I built something around. Maybe git Ops base to update game charts and we are building this.

A

We as an ecosystem basically build something.

A

A SAS API should be native without all the glue around, without thousands, tens of thousands of go codes, building operators and running them in my cluster to get a mySQL database and as a declare disclaimer here, I'm not talking about operators in the in the real sense of operating like when, when core has announced at City operator, the first one and permissions I think this was about operating like running at City instances, which is complicated, as everybody knows, to run them in in the cluster, and this is totally valid.

A

That's a valid use case for operators, but we're building operators for not to running stuff in the cluster, but to consuming stuff, and this is basically what we get so um the result of this the threat in Twitter. And then we gave a talk in Hamburg as a continuous about that.

A

um We want to build something which makes consumption of services Cube native, and the result of that is if we build such a thing and it's about Cube apis which we get into the cluster for those services. We have a yeah, basically a service provider Persona in the story, and this is mostly missing in kubernetes. Nowadays we don't have that like there is this admin of a cluster as the owner of the cluster and there's a user of a cluster who is not admin.

A

This exists, but there's no service provider, maybe the the developers of operators. They are something like an um persona for service providers, so everything should be should be secure, of course, and group native, of course, so um here's the future. um This comes from the talk uh from September I, think yeah, I think we're just just give it two moments.

B

So how hi everyone so as what Stefan already mentioned, things should be way simpler and you, as a user, should be only you know, care about how things should look like from user perspective. We have been using cubectl for quite some time now and what we should do is, for example, Cube City I'll bind and then you bind into a URL, and that thing will just hypothetically start the uh the browser for you in order to authenticate and authorize.

B

So you choose, for example, to go with GitHub, and then it asks you, of course about some information, the same oidc style that everybody see and know, and afterwards uh you get what really matters here. You are interested of getting this service, that's called mongodb, for example, up and running we give you or the service provider Persona in a way gives you the information that it only cares about which is or the user.

B

That cares about what kind of tier you're running in the region, backup and some other of course, other stuff that you as a user and as a consumer, would care about.

B

Then you bind into this API or into the service and afterwards, uh for example, the service provider of course needs some assets that needs to be installed in your cluster, such as secret or yeah.

B

um Whatever that the the service provider needs in order to complete the service that they are providing, then you as a user, for example, you would just accept these claims and you have the transparency. You know exactly what the service provider is installing in your cluster. It's not just you go into an operator. You install whatever crds, whatever Services, whatever kubernetes objects in there. No, you actually Vigilant and the high level of transparency would be reflected here and then what happens?

B

That's it. You actually got the uh the crds got the things that would get your application or your service up and running, and it's there.

B

What is really important is if we take a look at the um the the the the claim or the The Binding that we did. Those are the things that we saw in the browser. That's what matters to us, the most as a service operator as a person who would like to use that and yeah off you go that uh mongod mongodb instance or CR. Is there now what's super important? Is the pots some would say if I create the crd? I might be seeing something a pod, an operator, something that runs in my cluster?

B

That decides how my service looks like no, nothing. You only see the API server, the control plane, the things that everybody knows about kubernetes, the normal ones, nothing, there no operator, no Shenanigans, no magic. Just what do you know and you control that?

B

Let's take a look about the CR here? Is we have a mongodb that runs somewhere? We know it's running, hence the phase over there.

B

So it gives you also the size it gives you, the the region and all the information that you already see it like saw before and you approved, and then, if we take a look, how this thing looks like CR, it's a custom, resource or custom resource definition that you vigilantly approve to use and it has all what it matters for you, and if we take a look at the surface, for example, you get a service that you already saw and you already approved, and then you get something very interesting, which is the bindings the bindings decide what you have actually bound to what is the face for it.

B

We ran into this mongodb um resource and it gives you all the informations about it. When you did it some phase and some extra information that maybe the service provider get to decide, that's it you have it I give it back.

A

So sorry, you have one so just to stress the points again so there's nothing running in this cluster, where we bound to so it's a service which is provided by somebody else, a SARS, basically, a software as a service, but kubernetes native.

A

And this was basically it was a ren in Twitter and we talked about um yeah. This could be done. Probably we showed this demonstration and let's build this thing, there's kubecon coming and we have built it. So that's Cube point: um qbind is a project, it's open, so you can go to GitHub and check check it out. Try it out contribute. Of course, um it's vendor neutral, that's important. There is something running on a consumer cluster. We call it a connector, a connector which is a small agent which maintains the crds.

A

So you saw a CID installed so very important. It's not about an installer, it's not an installer. What we have built here, it's something it's a binding which has persistence so it lives on this cluster. We have seen there it's still bound to this API. When the service provider updates the CID. This agent will put it down the updates and update your CID.

A

If the service provider needs a different claim, he will tell you like accepting something like creating another secret or something. So this thing is living it's a long, long term connection between a cluster and a service provider. That's binding: that's what we call binding and coupon implements that and you see here at the bottom. So basically it's a cube00 plugin, so you can use Cruise.

A

So you have to install crew first and you have to add the index because we are not in the public index yet and you say just who install bind and then you can say: Cube cuddle bind.

A

It's here two point: um we have a slack channel for discussion, it's open when you're neutral, we love collaboration. So if you're interested in that please join it basically implements those two things and I had a chat earlier today. Already um about this topic, this is the bind command. We have seen that already in the first. uh In the first part of some presentation, you bind to a URL, the URL implements some protocol for binding, it's a binding provider and eventually you get the CID.

A

But actually what is happening is there are two parts there is this whole authentication flow like the web, UI thing where you can click around to authenticate. You click on the the resources you want, maybe other stuff so depending on the service provider. All of that is totally open, so it's not set in stone. You can customize that what we have here is an example implementation.

A

So we will see that in a second where you can select the CID from the service provider, but everything on the authentication part and the web UI part is totally up to the service provider after the screen command. What comes back is basically in API service, binding request, it's a generic or it's an object which defines which declares what to bind you can pipe it into a file, for example, and then there's a second part which you can also manually execute, so you can use it for automation, for example.

A

So you can put something like that into your Argo CD or something like that. So so your iPad is really the development story. It's super nice to just be able to click through some UI and find the right resources and be done. Few clicks super easy. Nothing to look up, nothing to remember, of course, in some automation setup where you want to do it automatically. You also need something- and that's that's basically.

A

The second part here, but two cuddle bind API service, sends a request and some remote Cube config to get this out to the service provider, and then you are done so you get the CID and everything as we have seen before.

A

So this way you will find when you go to the repository, so there's a Q Credit bind command.

A

There is a connector which is the agent, so the coupon bind will automatically install that and maintain that and update it. The connector is running, the connector is basically pulling cids and it's making sure the objects you create, like you, create a mongodb object locally. Is this a send to the service provider?

A

So there's a sinking process happening first provider and when there's an update of the status at the service provider site, this is synced back into the consumer cluster. So it's both ways: syncing up the objects and the spec shrinking down the status. Basically and the other thing. What is oil change repository is this example backend, but as I said, it implements a protocol. It has some example UI it has integration with oidc, so you can connect to GitHub or something like that for authentication.

A

But it's really just an example: the protocols matter everything else can be customized and should be customized.

A

That's a bigger picture how this example plugin works. So on the left side, this is black terminal session. Here, that's the consumer cluster. We have seen previously the connector as an agent. The cube cutter bind operate by the user on the right side. You see the service provider, the example backend it's running on a cube cluster on any Cube cluster. Basically, you can use an open shift or kubernetes you can e-cluster whatever you like, run the example back end and it will Implement those apis.

A

It's a protocol we have defined for binding kubernetes has namespaces for isolation. It's not great. It's, of course. It's isolation. You can expect from namespaces, but you can turn a kubernetes cluster into a service provider that way and very important.

A

You can use a standard operator. You can use so I played with the mongodb operator and just implemented that so the mongodb API implemented at the back end via an open source operator from the community. This works, I will show it on Thursday. I have a longer talk about that. I will show that as a demo, how you can do that with cross plane in between so it's pretty nice, you get a mongodb without any line of code. You can become a mongodb service operator without writing code, which is pretty awesome.

A

um Yes, isolation. As I said it's it's okay-ish. Every cluster gets a namespace a cluster namespace. So if you have many clusters here for each GF and name space at the end and if you create a namespace here with a mongodb object, so in the example we were in the default namespace, so we created the mongodb on the left side, then the corresponding namespace is created on the other side, on the right side, of course, with some prefixing to make it unique. But all of that again this is not a protocol which matters.

A

This is the example implementation. So if you want a different implementation, bigger scale, many clusters totally possible. What really matters and Cube bind is a protocol.

A

So if you go bigger- and you want to have really thousands of clusters connected like a real software as a service, imagine mongodb or any other database as a service company wants to use that they need smallish kubernetes like workspaces, so kcps workspaces, which are like Cube clusters, but they are much much cheaper, much much smaller in memory, consumption and CPU consumption, but they give much higher isolation. So if you want that kcp is a natural backend. If you want to really scale that up.

A

But if you are in an environment like you as a IT service provider for other teams, like you provide databases, mongodb databases for your development teams, maybe you don't care about that. You! It's totally fine. If you have an namespace isolation, all right, so just the real thing um colleague of us sergius played with cockroachdb again. This is a it's a random operator for a database taken from the community and that's quickly that.

B

Actually works, which.

A

Actually works- and this is also background for all of the motivation for all of that- um not sure about the experience with operators by by you. um My experience is often not so good quality of operators. They often don't work, and you really have to have lots of knowledge to understand. Why that's why we want this. The separation of personas, so EHC basically, is a cui, very simple. There's a crdb cluster CID coming from this operator, you can bind to it. It Happened, One, command, CID exists.

A

And you can create a cockroachdb object which Segways will do in a second, so the S9 on the right side at the bottom. That's the provider, cluster service provider, cluster, where the backend is running, so the upper part is the consumer cluster.

A

So here is an example cockroachdb everything here is real: it's not a fake, as in the previous part of the presentation. So let's apply as an example. It's a at the bottom. You see something happening here. So there was a namespace created.

A

And the operator in the background is doing something, so you can see here. There's instances of cockroach supports, I, think he's showing right yeah, he does supports, are created totally invisible from the consumer. Consumer does not see anything of that and if you go back and you check the CR, you will see the S status like conditions. Everything is synced down and again the pitch is installed. This plugin called two credit bind against a URL of a service provider, enter click through some UI bind done.

A

All right does it end with API services, so I'm from the API side, so I've been involved in cids, so much I love apis. That's why I built that, but binding, of course, is a much more generic concept. So not, coincidentally, the command is built or yeah. Basically spec. In this way, Jupiter bind is generic than this API service so think about. There can be other things again. The Binding is something which is like an installer, but it has a life cycle, like the things are bound to this cluster and updated by the service provider.

A

Think about examples which might be similar nature connections, so any kind of service binding there are. They have been work about service, binding operator. People will know that it could also be something like that totally integrated in Cupertino binds to get a bind will do all the flow at the top about Authentication.

A

Eventually it will cause a cube cutter binds and then the type as a sub command. So you can provide another sub command for the support in the beginning and then provide the sub command to do the actual binding. So you could bind something like a network service, so other ideas about something like an agent, an agent service. So some service providers, like I, don't know providing as a service built by by a polar signal, for example, there's an even agent to to send profiles to their their service. This is also a binding.

A

Why do you install something into the cluster it sends? The data to the service provider could also build be built in this way, so be creative. There are many more kinds of Bindings that would or could make sense all right. That's more so we only have uh 25 minutes now. um There's another talk on Thursday and we have a room on Friday 11, where we can just uh yeah go into more details, discuss get feedback, think where this could go. It's an open source project.

A

You have seen we have built the bases, but we want to see where this could go. So if you are interested in that you're, a service provider or you're just interested in the consumer side. In the experience talk to us, there are those two locations where you could do that, all right. So there's. If you want to re-watch that or share it, um we shared the first part of the talk here at this address.

A

Q cattle bind IO um Cube bind at this at this lack of cube, so you can just join us in the channel. There's a repository. We love code collaborations. This is not something by one service provider. It should be a generic thing. If this is interesting for users like binding in one command in a few clicks, um this could be the basis all right, and this is what you are finding in the repository again. So if you want to play with that, it's there document in the readme everything's either here.

A

So it's pretty fresh and new. So don't expect that everything works. It's not production ready for sure, but it's much more than just a prototype. It's it's the basis. Is there all right so far? That's the presentation part in a sec questions. So that's it back um love! This I think it's a great space I'm curious to know whether you think that a project like service catalog would sound very similar to this you're not going to allow induction.

A

If you think that was just being sure now I'm right, why somebody for successful service yeah, so one thing which we I mean the ux so to be super simple, and it should be possible for the user of a cluster who is who can just call the command and get a service? He doesn't have to ask anybody. That's basically the there's a core idea here, so the users can use the technology if they like it, we don't need to actually get an API server, something which is really complex and services cannot gets that right.

A

It was really deep in the system. It was awkward to operate and you had to convince your your admins in the company, for example, to install that or so it was pretty complex.

A

um There's one more thing: everybody knows companies who are wrapping Cloud apis. Today there are some companies, some startups which are part of the business, basically is to take cloud apis and build Cube apis out of that, they can offer that right. So, even if the service provider like mongodb, is not interested in that, somebody else can do that. The tools are there and can offer the service even internally internally in the company, your I.T Department the database department.

A

They can use this tool to offer a service, so the user is pretty free to consume and the service providers if they opt in they use this thing and implement it. It's cool, but there's a second approach where some other party does developing or even that works, so the Bahia I think is lower than the service catalog, it's much more Federated, let's say I think you are first.

B

So I'm trying to power back off.

A

Yes, for cids, you need that for sure, so you must be admin in the sense, so we don't solve the crg problem not with sets.

A

A

So the authentication you have seen is against a service provider site, it's not about the local cluster.

A

It's not using the whole system like the service provider is not or your cluster is not protected via airbag, so you don't depend on airbags. That is the service provider cannot do bad things. We don't do that, like the agent is a tricky as an agent which is open source and it follows a protocol. This implements the things we have seen at the very beginning. You remember when, when modes said, yes, there was this prompt, yes or no.

A

Basically, you told so the agent the connector to implement this protocol, which includes that only the things which the user opted in are really done right. It's a pool model, it's coming from the consumer cluster talking to the service provider. That's why it's safe without any additional airbag power, because airbag is not powerful enough for that. That's that's also, of course, important.

B

Probably for my original credential.

A

My identity, that's a story about how you can give an API like this community operator thing or maybe Atlas SS, offering how you give that in a secure way to engineers in this deal with cosplaying and stuff in the in the bigger talk, I think you will see something which is an answer here.

A

Basically, it's about hiding the actual API, which has the power of your credentials and turning that into something simple and secure and controlled for your use case.

B

Yeah, just just to add one thing: I mean: don't forget the thing that you saw on the consumer cluster So, eventually you're the admin cluster, and you got this mongodb uh CR right, you could hypothetically, create some service account, give it some information like where you cannot update this mongodb mongodb CR, and you can forward that service account to your colleague or to different people.

B

So you can decide, for example, how others can communicate on resources on your cluster, based on normal rbac from kubernetes itself right, you can say that I create this I bind into the mongodb thingy I am the admin now I would like others to use the CR on my cluster I just create a CA for it. You're allowed to get you're allowed to update, maybe something but not delete, for example. So you decide how people would interact with that by the end of the day. The important thing is once it's bound: it's there.

B

Then it's up to you how you distribute it.

A

And it also means maybe that's confusing, because the example is too so simple, like um you authenticate as Mo in GitHub right, it's not doesn't mean that at the end of CID, which is important, basically operates under you, the source provider can provide something like a service account for this cluster and there can be quota assigned and everything. So it's just simple here and this demo that you go so GitHub and it's running under your account. It can be something, of course, all right. James.

A

A

Yeah we haven't several the details here: I guess yeah probably I mean if the the permission claim is to get access, it would be synced up. Yes, um if the pollution claim needs less power, whatever this means for a secret I'm, not sure what it can mean. We can talk about that. So solution claims it's a sketchy at the moment, so we have to talk about which makes sense which are secure. Those should be minimal. Of course, as should be a I mean, that's a the rule of thumb, right right.

A

Yes, so there's no access to the local cluster very important and no aggregation or anything like that. It's really a poor model.

B

Combined, it didn't see and ask him anything like. Are there a subset of name spaces.

A

So yeah I mean this is again this permission claim thing: we have to design that the details about that um yeah restriction to a name and the namespace makes sense like in the example I think we only had a name, not a namespace or something that's what your main, meaning right. Yeah, it's too simplified. That's a demo um yeah. It must be again minimal. So we have to provide everything which is necessary to make it minimal. The power service provider needs to express the need, but in a minimal sense. Yes,.

A

All right, so if there are no further question um you have seen the repository, you saw the the two um yeah chances to to talk to us, so we have to talk on Thursday um Thursday at 11, I, think and Friday. Let's open floor, I guess the route with 50 people I heard. So if you have more questions, um you want to talk about API and stuff. That's the place, so please join us.

A

B