Kubernetes KubeBuilder Controller Runtime and Controller Tools, 14 Jan 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes KubeBuilder Meeting 20210114

Description

KubeBuilder Meeting for 2021/01/14. See https://sigs.k8s.io/kubebuilder for more details.

A

All right, hello uh and welcome to the first q builder, controller, runtime and controller tools. Meeting of 2021. uh today is thursday january 14th. uh As a reminder, this is being recorded, so don't say anything you don't want recorded for all posterity uploaded to youtube.

A

uh All right so looks like we have a pretty full agenda today. uh So let's get started with adirio.

B

Hi, well, um I have a couple points in agenda, but at it's pretty busy busy, I will try to be pretty fast so that we can go through all the points and the first one was um we. We are using uh six couple, a six kubernetes based jump parsing for the configuration files and there are a couple special things about this uh implementation of yum.

B

It's not parsing jam directly, it's first it's generating ideas on the json string and then it's using the json string to create again a interface, a go interface and from that go interfaces then creating the the yam representation.

B

uh So I was wondering if we really needed the the slavery of or we could switch back to, the one that we were using previously. There are some advantages and drawbacks from both of the approaches. So I would like to know what the rest of the members think about it uh just to number them.

B

If we use the one from kubernetes, we are doing this three steps process to to parse and to ma to marcel and to on marshall, both uh and a and we use json tax and marshal json uh methods.

B

If we switch back to the other one, we would have to use marcel yaml instead and jump tanks instead, but we would only have one step to to marshal the the configuration files, I'm being leaning towards both directions depending on I've been changing my mind. So I don't know what do you think about this.

C

I'll try men with my two cents. um I think if we- I don't know if we do this in q building right now, but I've seen multiple times in the past where using the non-sigs case, I o yaml.

C

If we're ever going to be marshalling your unmarshaling enamel, that's destined for the api server, uh then it seems like we should stick with sigs kate, zayo yaml, because I've seen like unexpected marshalling unmarshalling things happening. If you don't, I don't know if we do that or if we plan on doing that. um I kind of have this hunch that we might based on the fact that we scaffold a bunch of yaml that ends up going to the api server.

C

So unless there's like a pretty good reason, not to use that, I think my vote would be to stick with sigs kate's yaml.

B

Yes, just to to point one thing: I'm referring to the library we are using to store the config files, that's not going to the ap server in any way, but yeah. I get your point.

C

Would we, I guess the question then is like, would we use different like would we have both sigs gates, I avo and goyamble in this, in our go mod? If we were, you know, depending on what we're marshaling.

B

I don't think so I would have to check, but I don't think so because I think that the other parsing would be done in the scaffolded projects, not in cuba builder itself.

B

But I would have to check it, I'm not 100 percent sure.

C

Yeah, I think you're right, I think like if you look at q builder right now. I think we just you know, have strings that just get written directly to disk as yaml, but my concern would be if, if we ever marshal or unmarshal those strings, would we kind of shoot ourselves in the foot by like just by default, using the one that we're already using and then have these issues where things marshal are unmarshalled unexpectedly, based on what kubernetes expects.

B

Yeah my main concern to switching back to the other one was that for the next plugins phase, we will probably have to deal with non-go based plugins, and that will mean passing the configuration file or the configuration information from a go to another language, and we will probably use json to for that.

B

So that was my main concern.

D

I mean the basic issue: is that anything other than the 6k or thing may inhibit different behavior, which is especially bad if we in any way end up embedding anything, that's an upstream api because we may end up just not muscling it correctly or like upstream expected, and if I understood correctly, the only advantage would have been performance.

D

But I think this is only run interactively on developer machines and I think, having two passes of martial and marshalling internally is just not an issue in that context.

A

I I think, the main from my experience, the main point um where you want to use the like that the underlying ammo implementation, um instead of the six kate's yaml wrapper, is, if you need the like, if you need the structure preserving part of it so like in in um controller tools, there's one point where we use the underlying yml library in order to do transformations on the gamel without, like you know, keeping comments and stuff like that without, like actually unmarshalling but like that's the main, though that's the main point that I've seen for for using that.

A

um So if we ever decide to do that, it might be worth revisiting. But if we're not doing that, I don't know if.

E

A

Yeah, I don't know if it's worth it.

B

Okay, I will update the the issue after the meeting. The second point I wanted to to mention once we are doing a pretty- I don't know how to define it.

B

We are handling two variables two flag variables a bit, especially when we do a when we run a command to try to to resolve the plugins that will be used for for that command. Those are project version and plugins.

B

And the way that it's resolving the plugins, it may not be the best one.

B

For example, it doesn't allow to to set a different set of plugins for a certain command if we have a set q builder in it with some plugins, every command will have to use those plugins and it doesn't allow to temporarily change it to to only for a single command or things like that, and it also doesn't allow to resolve the project version ourselves when, when it's defined, for example, if if I'm using only the version 3 alpha plugin, I know that the config file is going to be a version 3 alpha, because it's the only one supported.

B

So I wouldn't need to to set that flag and to solve that. I I uploaded an empty, an enhancement proposal. I don't know if you have had time to to review it, but I would appreciate some feedback on on the new algorithm on the new approach. I think it will be solve those problems that I mentioned and will be a bit more efficient.

B

In that sense,.

F

Right and that's, I think, uh a fair thing to question. Originally, uh we wanted that actually to be the case where you set a plug-in on init and then all other commands would use that plug-in.

F

But, for example, if you had uh some plug-in that could initialize and create an api for a project, but then you had another plug-in that you wanted to use to create a web hook. You couldn't do that using the current scheme and I think we want to allow that going forward um just because we we provide the option or we we don't require every plug-in to implement every single uh sub-command interface.

F

So by implication we don't require an init plug-in to implement all of the the different types of the sub-commands.

F

So I think that's a fair proposal and we can probably have discussions about what does this mean semantically in that proposal? Pr.

B

B

I don't know if there is any other opinion on that topic. We can keep the the discussion in that pull request. If you want- and we will come past to the next one, because there are a lot of points today,.

B

Okay, so the last one, the last point I had was I was uh revisiting all the tests and checks that we were doing in cube builder. They are pretty uh a bit tangled and unmissed.

B

um The first thing is that there are things that are being done in in pro and other things that are being done in in github actions. For example, github actions allow us to.

B

Use a test for macbook for for mac and it also allow us to to chain test so that they are not running all until certain conditions have succeeded. So I was wondering if we wanted to migrate everything to github actions or keep brow or maybe migrate. Everything too pro, I don't know I feel like proud, doesn't allow to do everything that github actions does and that's a bit of a limiting factor. So I'm a bit leaning to make everything in github actions and unforget about bro. In that sense,.

G

One of the reasons for prowl to allow I'm sorry.

D

Go ahead, please.

G

I was just wondering if one of the reasons for prowl is to allow folks to run the commands individually rerun a test or something like that, because I don't think you can do that in github. If that's not a need, then we can proceed, but I thought that was one of the uses of it.

A

If you structure your your github actions separately, there's like a re-run, individual action um in and you have actions.

A

I think I think part of it might be like I have from what I've seen. There's like there's, there's three kind of three reasons: we're using prow. The first is that uh it's, it was what the rest it's, what the rest of the uh kubernetes project uses by and large.

A

um I think I only have two actually and the the second one is that like uh it gives us for, for like certain things, it gives us the ability to um like, especially for unit tests. It gives us the ability to like gate on who can actually run the tests, because you know nominally running or technically running the tests. The unit test is running arbitrary code that people upload and so uh prow has the ability to like currently check, if someone's in the in the org or otherwise require okay to test versus github actions.

A

Just runs whatever someone uploads um and so technically someone could upload something some sort of malicious code. um I think, which is why we currently have a split between github actions and prow um versus github actions, as you mentioned, does allow us to run our osx tests um so technically we're already running those the technically untrusted code um and has maybe a nicer interface in some places.

A

Did I anyone else have any points there that I missed.

D

Yeah, there are two more advantages for pro: one is re-testing, because if with github action, you have a workflow where at some point in time someone creates a pull request, it gets green tests, then something gets merged to the base branch say master that conflicts. With this pull request, then nothing will rerun the github actions and that pull will merge. The that was created sooner but merge data, and you end up with a broken branch.

D

That's one thing and another one. I know how relevant that is with power you can integrate with test grid and that in turn is very useful to get some kind of trends for flaky tests and debug them and yeah have some longer-term view on what tests flag more often or if tests, flag and stuff like that.

A

Oh yeah, the the first one you mentioned is is super important right that one that one slipped my mind. I think github made progress on fixing that recently, but I don't remember where I saw that so I take that with a grain of salt, but like yeah that first one especially is very important um and and the the test grid is also nice, but like making sure we run the test against the merged, the merge commit, that's actually going to be merged is important.

C

I don't really have a huge opinion, one way or the other, but another plus for github actions is that the code that's used to define the test is stored in the same repository as the code, which um dealing with prio sometimes gets a little bit annoying because you have to make a couple of commits and merges and multiple repositories before you can un unblock yourself from something that's failing.

C

I haven't seen that too often with these repos, so maybe that's not a huge deal, but probably a minor plus in the github actions. Side of the column.

A

Yeah, I think the I think the the re-run on fail is is is like or we sorry not re-running phil rerun on merge commit changing before merge is like the blocker. I think we need to leave them in place for that.

B

Okay, well, that was my three point, so I don't know who's next.

A

We should, uh we should maybe add a note to like our dev documentation, uh just in case people like have this thought in the future, um because it will probably occur again uh just as for why the why things are the way they are.

A

H

All right, I guess I'm up uh so last friday I submitted pr to add some testing clients um in the past. I've brought up the idea of this reactive client, so you had said you know reactor to oversimplify reactors bad, um and so I added these uh air injector and spy clients that are kind of more special purpose, single purpose, clients, uh testing clients to inject errors or to spy on api calls um and then uh alvaro's feedback was I like. I like this. I, like reactive, client, better.

H

um I really he likes reactive client better. So I thought it would just be good to open a dialogue on uh what we want to do. uh Do we want to include reactive client? Do we want to include the other two, both neither yeah? That's it. I would.

G

Not mind a reactive client I had this. I started one on the sdk for myself because I was doing testing and it made testing easier for certain error cases.

G

D

Because david mentioned that you were as you as in sully are opposed to the reactive client and they because there was some discussion very long ago, so my question for sully would be. Are you opposed to the general idea of having a client that allows injecting arbitrary, behavior or just the way client go implements? This.

A

uh Yeah, so, okay, so I think my number one complaint is that, like the actual way client go implements, this is like terrible right like I I I I have like it. uh Nothing's typed, like everything's type assertions uh like the resource name, is just like an arbitrary string that sometimes changes out from underneath you, depending on how the underlying api machinery is feeling that day like it. It's not.

A

It's really not a good experience, especially in the edge cases where it's very opaque, or at least this is my experience like writing reactor tests in kk.

A

um So if we're gonna have a reactor all right, if we're gonna have something that's like a reactive client like we should have a controller runtime style interface for it, um as opposed to a client, go style interface, um and we should make sure that, like the weird corner cases are like not weird. So, like you know things like uh what happens if you return like false, this is unhandled and it falls off the end of the handler chain, like there's really weird behavior in client. Go that doesn't really make a ton of sense.

A

um There's like parameters that, like you, think they do one thing and do something else so like we. We just need to make sure that, like if we're going to have a reactive client like we need to make sure there is a solid, easy path um and you know maybe have like easy helpers for like the simple cases like you know, inject error, or whatever um separately like I I you know. I originally was like very very much like.

A

I would like to push people towards using m test instead, but, like I think, we've gotten enough feedback from everybody at this point that, like uh the reactive client, can is useful in certain situations that, like I'm, no longer hard opposed to a reactive client but like if we're gonna have one.

A

We should have one that you know. Works works well and doesn't have all sorts of weird holes in it.

H

Okay, so the implementation I have actually just uses the client go implementation of reactors by um can kind of it's kind of an adapter to to the to the api. It does like a round trip through converting things to the test actions and then converting those back to um controller runtime style calls.

H

um Would you I don't know I hesitate to like re-implement reactors um in our own way, but maybe maybe that it sounds like. Maybe that's what you would would prefer.

H

um Maybe it wouldn't be so bad. uh The api to um error to the air injector would actually be. I think, pretty similar to uh what I would want to create for a reactive client. That's you know not inspired by client go or not using clanko's reactors.

H

I don't know how I don't know how much you've looked at it, but maybe that's, maybe just some get getting some concrete feedback on the design. Api of a reactive client would be useful.

A

Yeah, I can um try to take a look at the. I haven't gotten a chance to look at the design, yet I'm just kind of scheming over skimming over it now um yeah, the the the direction of like how inject error, works and stuff is definitely, um I think, like if it's maybe fine. If it ends up wrapping the client go one but like a yeah, we need something like error like inject error um and and b like or we need.

A

We need an interface, that's more like inject error and and b like we just need to make sure uh that, like behavior is well documented uh in terms of like what happens. If you take certain, do certain things.

H

A

Like you mentioned.

H

The what happens if you only return false and fall off the end of the chain.

A

Yeah, yeah uh and stuff like that um there are, there are cases where, like in the in the like the, for instance, in the reactive client from client, go where it will helpfully uh attempt to apply a label selector for you, but that's not really documented anywhere.

A

So, if you don't put the labels on your objects, it will, it would like filter them all out and be like there's no objects supported, which you know maybe is nice but like.

A

We need to make sure it's documented and stuff like that.

H

Yeah, okay, so it sounds like we're staring in a an all of the above uh resolution. To this um does that anybody disagree with that.

H

All right cool, if that's the case, then I'll just like keep going and get take everybody's feedback and polish up those clients, because you know they're, not they're, not done, there's kind of a in a uh please give me feedback state. So um all right thanks! I think that's it. If there's nothing else,.

I

One of the things that we've talked about before I think uh is that using m test you end up using the actual cube api server. So validations are you're, not writing your own validations uh in the reactor and then now you've been implemented. Like you know, the pod api server validation to test your tester controller to handle like all the different ways I can handle it. Is there something we could do to use the cube api server, validations.

I

Here as well, so that you don't have to re-implement that in your reactor.

A

uh I think that would involve pulling in kubernetes kubernetes, and that is not a thing we want to do.

D

Yeah, I also think that if you're writing tests, sometimes you're probably going to use an object that wouldn't be accepted by the api server, because all the fields the api server requires might be completely irrelevant for whatever you're currently testing. So I'm not even sure if it makes sense. In this context,.

A

Yeah, I I kind of think if you, if you are like, if you are using reactive client like you are probably you are hopefully in a situation where, like there is a real uh reason that you just cannot use m tests like and like if you needed that you would use end test, um and I also kind of think that's how we should document the reactive client like prefer m test.

A

um You know uh you can even inject error like there. You can wrap the m test.

A

You can wrap the normal client to inject errors for m tests if you want but like if you end up getting into a position where, for some sort of performance reason or for um you know, there's like a really uh like interesting sequences of events that you need to trigger that are hard to get your trigger in a realistic environment like here's, here's here's this and you can use it, but, like you know, we don't recommend just generally using this. For these reasons,.

H

I mean I have other thoughts about that I mean but yeah we don't need that's fine. We can have that as a community recommendation. That wouldn't be my recommendation, but that's okay.

A

Yeah I mean we can. We can maybe discuss this at the end of the meeting a little bit. This is this has kind of been our been the. I guess I should probably say this. This has been. This has been my kind of advice since um basically, as long as we've had m test, um but we let's yeah, we can I'm happy to discuss that at the end of the meeting again but like and hear your thoughts, but um that could get into a long discussion. So.

A

All right thanks, um I think next up is me, but I'm gonna move myself to the bottom, because I have a demo, um so I'm gonna look. We can do that last and we'll get through. I think the other points are probably a little bit quicker. um Joe or jenny. Are you ready.

E

Yeah, um ours is real quick. So, a couple weeks ago we had mentioned that we had we've been working on generating an apply client um that we'd love to get um uh real feedback from the community.

E

On on that, um we're gonna have probably in less than a week um uh something that that people can do a hands-on experiment with so basically and the ability to build a cube builder that will also generate um applied type bindings um in at least one one or two ways: um either the structure the bit or the builder functions, or both that we mentioned before um the only question we had was: how should we communicate that to this group so that people are interested in trying it out? um We could see it. Is there a?

E

Is there a mailing list or something that we should use or slack or all of the.

E

A

uh Yeah, we we have a there's a mailing list um and- and you can ping slack, um I think different people probably monitor different ones.

E

Okay, um yeah we'll we'll do a little blast once we have something in place, um we'll probably have like a read me: everybody can fall through um just for everybody on this line. If you, if you use, apply we'd, really love you to try it out. um Basically, what we're gonna do is we're gonna have like two or three weeks um where um it'll be available. You can experiment with it.

E

If you have like an existing crd, um that you use coupe builder with it'd, be great, if you just try running it, see if it does something reasonable for you. um We're gonna have like a questionnaire at the end. That asks um like certain preferences that you have about this, and so it could really impact um some design decisions. It'll also be used to inform sig api machinery on on which, which style of generated bindings we like best for apply um so yeah um I'll, send out a blast in hopefully also the week. Thanks.

A

uh All right, um the next one is camilla.

J

Hello, um my one will be first. I think um I raised this before um we have just to give you a little bit of contacts when we discover the project using uh kobe builder, uh one of the we use the the proxy which is this image, and it is not she hurtless.

J

So we have the fix, but we need to be great. uh Do a huge change? Push a new image for the google cloud.

J

Few people say that she would like to use their upstring for don't need anymore, this menu work. Others say that's not because it's in quite so, could you have a you know a downside and in my opinion the cons is like how we can uh ensure the supportability of cheese image.

J

If we don't don't have that anyway, I'm just raising that, for we reach our consensus and ask a help or build gender positive image from someone in the google that has access and permissions.

F

Yeah, I think vince's concern with quay was that quay is down a reasonable amount, whereas gcr and docker hub are not um so he preferred to not use quay. For that reason, otherwise I yeah, I think it's reasonable to build and push our own. I guess uh or just pull and tag and push to gcr.

F

I don't actually think we need to build anything ourselves.

A

Yeah, the the previous iterations of the image have just been um from what I understand just you know, uh pull and tag um I uh as as, like I yeah, I I'm sorry, but I've I've been a little bit.

A

I've been super busy kind of recently and I haven't like I haven't had time to work on this, but I I think, like I think we should have a a cloud build config that does the pulling tag instead of requiring an individual because, like uh that's, that's like it's not great right like I, you know from a like semi selfish perspective.

A

You know I don't yeah if I'm on vacation, I don't want to have to go like push an image and from like a more and then like, reflecting that to a more project stability perspective like we shouldn't have it, you know it. It's not great that, like we have the case of like oh, we need to go get you know a googler to push this image or whatever, like that's, not a good situation. I think um so I I I think I mentioned this before but like.

A

I would really like to see us just have like a short cloud bill config for that, and then we can just do it like we do for the um for the the like bundles, that we publish and like the snapshots and stuff like that, where we just have uh cloudville config sitting on its own branch and then every time we wanna, we need to push a new version. We just do a commit to that branch and we have cloudbelt looking at that branch and it just builds a new copy.

A

F

Write something up right after this meeting and make a pr to the tools, branch.

A

Okay, cool yeah yeah it should I like. I imagine it would be it's probably a pretty short cloud but config, because it should literally just be. You know, pull that pull the existing one tag it as the new and then tell cloud build that it needs to be pushed to yeah, replace yeah. Just let me know when you've done that and I'll enable cloud build I'll. Make sure that that cloud valve is enabled.

B

Thanks just just a comment about that: uh should we move that part to the other repo to the kubernetes cover builder release.

B

Tools, I don't know if you, if you what I mean the brand, is.

H

B

Sitting in kubernetes, quick builder, instead of ins, a q builder release tools and we may want to move it to the release tools. Repository.

A

uh Yeah we could um honestly I'm kind of ambivalent. On that, like it's, it seems like an okay proposal. I guess.

J

Thank you folks,.

A

uh All right, uh if no one has any more comments on that, um I think eric you have the next.

F

Next, one yep just a quick announcement. um We, I think, are ready to cut uh v080 control runtime. um This is going along with what we talked about before the holiday, where we are going to try and cut controller on time versions.

F

Whenever a new cube version is released, just to keep uh more closely aligned with upstream, um the 070 release was really big and I think it it definitely included uh one coupe bump, but it's uh it was in. I guess a little behind upstream, so uh going forward. We're gonna, try and release uh more frequently to capture new cube uh versions.

F

So yeah this uh this next version 0.80 is not going to be uh super feature full, except uh for the coupe 120 buck, and that should be happening today.

F

A

All right- uh and I think we have one more thing before me: um oh server side apply any plans to include rapper and controller runtime. uh Yes, uh there, there there are plans, uh talk to talk to joe and jenny, uh who just appeared briefly.

A

uh They are working on server side, apply related um things, uh so we'll we'll be probably adopting a similar, basically a similar, a similar thing. You know using the same kind of generated, structs and stuff um generated from controller tools, uh but you know just working with our client instead of their client.

A

Does that answer your question.

K

I guess so. Thank you, I'm sorry. I just uh this question a bit a little bit later than other folks, and there are some plans to use since I applied to do some better testing in openshift.

K

K

So it can tell around time in a short amount of time. I will definitely talk with uh jamie and jeep bets about this, but yeah. If anything else, I would be eager to try it out help with anything.

A

Cool all right, uh I don't know if joe is still here but joe. If you are there's a there's, a extra volunteer, um all right, uh okay. So the last thing is for me, so I have been working on something kind of on and off for a while, um and it has recently I've recently like gotten it to a point where it is demo-able.

A

um So uh I have been working on an idl for kubernetes. um So basically, the idea is, instead of writing, um go to define your your types. um You'd write, this idl um and kind of the benefits are. uh It has like first-class concepts for like kubernetes concepts.

A

um So, instead of like you, know, writing structs and having to like manually embed, metadata and kind, you just write like kind blah and and then that gets generated for you. um Hopefully the idea is, it also should make it easier for uh non-go people to interact um to so that we can better generate.

A

um We can like you know, people don't have to run, learn, go and and install the go tooling. If they're say you know, writing something in like python or whatever, um and then you know maybe make it uh easier for upstream as well to like generate well-typed and validated things that we can then um embed in our crds and maybe address or hopefully address some of the issues that we've had with the existing go tooling.

A

So hopefully, I'm gonna, I'm gonna, give a quick demo here and then also, uh hopefully, eventually I love to get everyone's feedback on it and um should be a cap happening soon for upstream as well, and then uh I have some not quite finished internals of controller tools that are refactored to like rebase on this as an intermediate step. So hopefully this will be the uh new internals of controller tools as well.

A

um All right, let me share my screen, an application window. I think it's this one.

A

Yeah is that one cool all right cool? Can you all see my screen.

J

A

Awesome all right, um so, basically, uh let me, let's take a look at um I've. I've actually converted all of core v1, but it's a little bit intense, so I figured I'd, show you kind of a little bit of a snapshot, a subset that shows kind of the important parts. um So everything is grouped under uh a group version in this. You can have multiple group versions per file if you want to instead say organize your code by, like the kind so have all the kinds next have all of one kind.

A

Next to each other, or you know you have maybe a small api and splitting it up between files is, is, is unwieldy um and then we have documentation, has a special syntax.

A

uh This prevents weird spacing mistakes um and accidentally, including comments and uh other other metadata in the documentation which happens sometimes in kubernetes, um and then uh the the naming is oriented towards like what it ends up. Looking like in the json this for generating go, this gets converted into go forms.

A

um As you can see, we have first class concepts of optional and kind of the different types of of like lists and mappings that we have in kubernetes, so lists versus lists maps same thing for like defaults, we have oops first class concepts of defaults, my syntax highlighting broke for some reason. Very briefly, um yeah you can see. We have all the usual suspects so like um wrapper types and structs and volumes uh first class concepts of enums.

A

um Among other things. This should allow us to publish api documentation with like actual descriptions for each of the enum variants, um and also to like more easily have validation for enum fields. Instead of forcing people to write down like write down the go, constants and then write down, slash plus enum separately um and then finally, we also have unions, which is, I know, been a feature that a lot of people are asking for. So this is, like um you know, the kind of kubernetes standard, one ofs.

A

So in this case we have a union with two variants: host path and empty dir. This is just an example of volume source, obviously in the real volume source.

A

There's more, but you know I got a little lazy and want to type out the whole thing um and then finally, we have kind of markers, a more structured form of markers as an extension point, so there's some built-in ones for like common metadata um and then you'll be able to define your own and use them like this markers are strongly typed and you have to import the definitions that you have the things that you want to use um so no more like.

A

Oh, I accidentally misspelled this marker and controller tools um just like ignored it, because it didn't know if I actually meant a different one, because you have to import them and specifically type them. uh You know uh the the new parser will be able to go. Oh, I don't know what this is. You didn't import the definition, so I'm gonna error out, which is hopefully a better user experience.

A

um Any questions so far.

A

Was I I'm I'm assuming I moved a little fast there just because we're running a little short in time right I'll? Take that as a no comments questions for now, um so the architecture of the parser um and this will eventually be bundled into a single helpful command- was kind of like modeled.

A

After the way, proto does things so um we have a kind of uh oops that is the wrong directory hold on I cd up to too many directories there you go um all right, so we have an intermediate form um that is like serialized to disk that can be consumed immediately or like we can save around. This is intended for consumption by tooling. So, like various parts of controller tools. um Theoretically, you could also like you.

A

Can you could cache this or keep it on disk around, so that you don't have to like recompile every time or distribute it or whatever so like, if we say, for example, wanted to try compiling our example and let's hope that I didn't anger, the demo gods and accidentally change something late last night.

A

uh And then- and this is just gonna spit out- the like text- form of the the intermediate that it's represented- uh the intermediate data that it's generating well all right, so we've generated um some intermediate parks form. This is actually been serialized to all around.ckdl, which we can see is a lovely blob of compiled data. It's actually it's in proto form, so like any tooling, can import the proto definitions and make use of it, um and then we can feed that into.

A

A

This is just like a hacked up form of the same machinery that we've used from controller tools, um and I have evidently screwed something up.

A

I have anchored the demo gods, let's see, let's we can uh can quickly see if I can figure out what I screwed up, but uh otherwise it's not a demo. Unless you screw something up, it's definitely not a demo. Unless I screw something up, what did I screw up.

A

A

It's not liking, my oh, that's why you know it would help if I like actually specified the right syntax there we go.

A

So you can see, we've got like an actual uh crd that we've generated here, um uh as one might have expected, uh with, like api version kind metadata, the spec that we described all of our documentation um there's uh for we have um we even for like that union. We had we generated like the very weird, complicated blob of open api. That's required to get a tagged union to work in kubernetes structural schema without it complaining um yeah.

A

So that is. That is what I have so far. um I also have not quite ported from my initial prototype, the equivalent of like a go format for this um and a tool and also a tool to help you migrate over from go definitions. You give it go definitions, it spits out the intermediate representation and then there's a separate tool that I haven't quite finished. Importing from my initial rust prototype um that spits back out from intermediate representation, the equivalent textual idl form uh just does anyone have any questions? Comments, concerns.

C

Got one um so: what's what's your vision for, like this long term, would would we instead of scaffolding like go files for the types? Would we scaffold this instead and then generate go and crds.

A

Yeah that would that would be the idea. um The the plan for upstream as well is is um hopefully there's so. I plan on having it kept for this within the next couple weeks um and showing it off to api machinery, uh and then hopefully the idea is upstream, adopts this as well and and generates the go types as well um but yeah.

A

The idea is, we we'd scaffold this out um and you'd be able to write this instead um and we've made we've maintained, compatibility and controller tools, so we'd probably maintain the ability to pull in go at least for a while, um but like the internal tooling would all be around the intermediate like the compiled form so like we just compiled the go to like the compiled form.

A

A

Yeah, so theoretically you could you could go to to from ckdl to java, um so uh kubernetes the project is definitely going to maintain a generator for go. um I know I've talked to some folks upstream and there's like concerns over whether or not we want to generate other things off of like the eventual open api or versus the ckdl. I like generating off of the ckdl, because um you kind of get a more constrained problem set and so like you can generate like better you can.

A

You can make better decisions about what you generate, especially in more strongly typed languages, um so, like you can generate like you, can generate like rust, rust, enums for unions for instance, or or stuff like that, um but yeah you. You could definitely like. I one of one of the things I wanted to enable is people consuming this and do an interesting thing with things with it, generating better api documentation, for instance, um is like a thing that we could do as well um yeah I have so.

A

I have my initial stuff I'll drop a link in the document, but I have my the initial work. If anyone wants to like go and play around with the syntax there's the um two compilers that you saw below or here um a couple of examples, um a kind of formal-ish grammar and some vim syntax, highlighting if you're working in vim so um yeah try it out um file. Definitely like you can file issues on this repository.

A

um If you uh um like, encounter anything uh that you think is breaking or doesn't work for you or whatever um yeah.

B

Cool, let me know.

I

We considered how you would potentially ship the kcdl with a given controller.

A

I

What like, what do you mean like how yeah sorry, that's probably like a really weird question coming out of nowhere, uh I guess what I was considering is uh controller. A is written in rust. Controller b is written in go controller, a wants to use controller b, but it would have to like would have to then generate types for itself right um and how we could you know if controller b is installed in a cluster? How could they generate those types um before I would?

I

I would assume that would be from the open api spec that is served by discovery, um so I guess I was asking if, if we're potentially also thinking about serving in some way the kcdl for for this, so you can easily generate your type based on that.

A

I don't think we have any immediate plans for doing something like that. um It's definitely not in the mvp um yeah. This is this is like there are upsides and downsides about generating everything based on the ckdl.

A

um This is one of them, on the other hand, generating based on the open api from the crd is a little bit wonky because you don't have any type names in it and you lose a whole lot of structured information. So, like that's, that's like maybe a longer term discussion.

A

um My my thoughts, kind of short term would would be just like today. You know we effectively say, distribute the go types and instead we'd say distribute the kdl or the ckvl um and then like the the parser can import either via from kdl or ckdl directly, um and and so you could, you could generate your rust, uh your rust bindings. That way, um it's a little bit closer to our traditional method, but, like it's also doesn't require. You know any any changes to the api server or whatever right.

I

Yeah just just wondering if it's been brought up or if, if the question had been asked, I guess so thanks.

A

uh All right, I think, unless anybody has any other quick questions or comments, um I think we are over time for today. So uh with that I'll see y'all in a couple weeks.

F

Great demo, thanks a lot, everybody.

A