Kubernetes API Expression Working Group, 4 Aug 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: WG API Expression Bi-Weekly Meeting for 20200804

Description

WG API Expression Bi-Weekly Meeting for 20200804

A

All right uh welcome to working group, api expression, it's the 4th of august 2020, and we have uh a few points on the agenda. um Antonio, is this one from you? Are you updated on progress made.

B

Can you hear me yeah, yes,.

A

B

Yes, but it's just the status that we usually do.

A

B

A

um Okay, you guys have a design proposal.

C

So this is for um supporting, um apply in a type safe way through the go client. um So this is how do you? How would you define a manifest or an applied configuration and go? You can't just use the structs that we have today because of zero values.

C

You can't know you know if somebody were to construct a deployment using the typical ghost trucks and the resulting deployment might have replicas set to its zero value, and you don't really know if they had intended to set that or if it's just because the zero value automatically gets set for that field, so we need some way to construct exactly a um applied configuration from go.

C

So what we did is, at the top of this document, there's a there's, a yamas example which um can't be constructed from ghost trucks, but we should be able to construct um and then what we've done is we've gone through um just the design space for this? So what are the different ways that you could imagine doing this? Could you construct some new kind of ghost trucks that did it? Could you use some kind of builder pattern where you call functions and that constructs the object for you?

C

Could you you know, do a couple of the other alternatives that we listed? Those are the main two ones and what we're really looking for is um feedback on. If we people agree that these are viable options, um if they agree with the cosmetics, like the agronomics of these, do do these look like idiomatic things.

C

um What can we do to improve them? Stuff like that?

C

So if you're going to use ghost trucks, then obviously you need to have a way to emit anything completely, so you and probably end up with a lot more indirection or pointers or something in this trucks. If you use builders, you have something like you can see on the screen right now, where um there's a couple different ways we could imagine doing this, but where you basically call functions, some of them, which may you know, return the next object to be constructed or return the object.

C

You just set a field on to do to do the construction, so I think the questions that we had were um what should be the next steps, we'll obviously give people a chance to look at this, give some feedback um we'd like to we'd like to focus on getting a prototype going as soon as possible. So if anybody has any thoughts, we would like to incorporate those as soon as possible so that we can um start prototyping, basically today or tomorrow, along one of these lines, which doesn't mean we can't switch tracks.

C

It just means that we'd like to pick one to start working on and then once we agree on it, we were curious. What next steps are? Should we be updating the existing cap? Should we be creating a new cap? What would be.

A

Appropriate all right, I looked and sounds really great. You put a lot of work in it. I.

A

Yeah, I I I I it's sad that girl makes us do this.

C

I don't know some of those things could be moused over, so you can see they're inspect that we're then providing this back. It doesn't have to be that way. You could either call us back and have it automatically construct one from you and then you would avoid having to do that and you could just do spec up and open close parenthesis dot and then field names or you can have like a new spec. So there's a couple different ways that we could do this with the builders. You actually.

B

C

The full flexibility, um which is one of the nice things about the builders you can kind of you can kind of extend them in different ways right. um So, um of course, the comp you know what you got to do with the builders, presumably is either you're building an instruction under the hood or you're populating the goatripes plus some kind of field masks so that you're able to track exactly what you're actually applied configuration should contain.

A

D

A

I I don't know, do you have, I guess we should post it on slack as well and ask for more feedback if it's not already.

C

Yeah, I think one thing that I could really get help with um just from this meeting specifically, is: where else should I post this? How should I get all the appropriate people involved, um so post on slack sounds like one thing.

B

Yeah, this is extremely close to what sig api machinery does in general. I feel like they've, been responsible to help a client in general, and a lot of people in the api machinery group have an opinion about this. I think so yeah, because maybe even talk about it next week during the meeting.

A

B

I think I think that makes sense.

B

And then uh next week we're gonna have something maybe an actual proposal to make.

A

Yeah, that's what I wanted to ask. This is a document for like a draft now I I think some of the options can already be filtered out, but at what point, should it be a cap and be discussed in on github.

C

We'd like to get there pretty quick we'd like to get them in like a week or two um so at least start opening a cap in like a week or two once we have a pro time. One thing antoine that you made me think of that I think is really important, is just because we're building these builders for apply doesn't mean they're specific, to apply right, you're building objects, and so, if we add this functionality, it might be, it might be that it shouldn't.

C

I don't know like this is a question for groups but um of people to answer, but should you only be able to build manifest with this or you should be able to build any object with this? And if so, then it does definitely kind of overlap with api machinery.

C

Yeah yeah, okay, okay, so we'll take it to the at-bat machinery, sig and um yeah. Please everybody! Look it over.

A

Yeah, and so far our only concern is is go as well. I guess, like the other clients, have different language features. We shouldn't what did it do thanks.

C

Yeah all right yeah, I think I think go. We might want to phrase that carefully, but we'll be setting an example when we when he decides what to do with go, but other languages can do this in very different ways and leopard language features.

B

Okay, I have, um I have a few- not ideals but suggestions to make on this document, but I think we'd be able to move forward quickly on that. So that sounds good right. Cool.

A

um All right, then naming for tomster tombstones. I think it's also you.

C

Yeah, I won't say as much on this one but um we've I think we mentioned this in a previous meeting. What we're doing here is just adding again the a way to very directly say that you want to feel to be gone and retain an opinion about that field, even though it's absent, um so you could say, um there's a lot of things you can express in this.

C

You could have a controller say that if you're deaf recruiting an annotation that it goes and takes ownership of that annotation tombstones it and then, if anybody tries to add it again, they get a conflict and there's a lot of simpler examples. So I think the main question is: there's only a couple lines up near the top, where we show an example of what this might look like.

C

um Yes right there, so the two highlighted ones we're not sure what to name this. We do know that, if you're doing a field, it probably needs to be a map with a single key in value, and if it's an associative map or list, um then you have the second example where it's like you put this extra field.

C

um Next to the key fields um that says that doesn't exist, but I don't know what we should call those keys and values, um and so I'm really would get a lot of value if people could just think about that and think of any names, there's a what to call it section there, um where we listed out a bunch that we don't like, but we couldn't think of anything better.

A

I have two questions. Is this a I, I suppose that's really valid? Yaml like you can use a dollar uh symbol in a key yeah yeah, I'm pretty sure that's valid, and and for this one like um or or this one as well like, that would be an invalid field, so that would make manifesto written for apply incompatible with being used for anything else is that.

C

True yeah, this that's right, that's one of the big implications. This is, we don't know a way to add this and still retain full, like, like you, wouldn't be able to take like the open api, spec and still validate the dock. um You would have to strip out the tombstone first to do that, um and one of the reasons why we couldn't think of a way to do that is when you have a field and its value is just whatever primitive type. It is there's really no value that you can pick.

C

That would be able to tell you the tombstone something so we have to if we want to put the tombstones in line like this, so they're in the dock, in a convenient way, we have to introduce something: that's not valid.

A

That that also includes incompatibility with, like editor syntax order, completion that kind of stuff.

C

Yeah it would, it would.

D

um We submit a second manifest that has the fields to be deleted in the same request.

C

D

Sorry, if this has already been discussed, but could we send a second manifest that has fields to be deleted and the same request.

C

um We did think about putting it somewhere on the side somewhere um it. It ends up being kind of difficult agronomically to make sure that you've got the right thing, because you have to either like construct a second manifest and make sure that it's paired with the first one, which is fairly burdensome to write correctly or you have to like write field paths which are fairly burdensome to get right and getting.

C

If we did do that, which is an alternative, then we'd have to get all the safety nets up for that, which could be quite a bit of work.

C

So yeah, it is an alternative. um If somebody wants to advocate, I would certainly be willing to discuss it, but this is the one that we were. We were going to try and champion.

D

B

Right, I yeah just to finish on this topic. I um I feel like this is going to be more controversial than the other topic, so changing the type for hostname, for example, is uh it's hard for me to accept uh creating new fields? Maybe, but changing the type is kind of com.

B

I it's hard for me to be okay with it for now, and uh the other thing is that, especially for the containers thing it's difficult, it's possible to both have does not exist and values, and I'm not completely sure how we should deal with that yet like what, if you have the item assertion and the image, for example,.

C

It's I would consider that a like an invalid document and try and give you a good validation error. What's wrong with it.

B

Because I I could see how okay, I could see how we could actually build a schema where we have the item assertion in literally every single list item like we could have. This thing exist right and maybe host name. We would have the ability to say: okay, hostname can be a string, but it can also be a struct that has exactly one field named field assertion that can have only one value being does not exist.

B

um So I just I, I would like to be able to have a schema that describes what we're trying to do.

B

I see your point see what I'm saying I I'd like not to cheat the schema.

C

Yeah, I think that makes sense. um I think open api spec is capable that you would just have a different schema for the manifest than you would for the main items potentially.

B

Yeah and I could see how the container could have a schema being either you have name and item assertion, or you have name and the typical actual schema. uh We would be using some sort of one-off, but um at least we could even say we want either the item assertion only the item assertion all the other things.

C

Right yeah, you could do that with like uh some kind of union or enumen open api to make it so it's one or the other yeah. We don't.

B

Have a good way to do that in uh in um in go extract, but if again, if we're using the actual client that we are planning on implementing, we can have to be okay as soon as you set the. I don't want this item, then it's going to reset everything else yeah, um but you need a smart diagram, but okay yeah. It needs to be at least a little smarter right, yeah.

E

C

Oh, I was going to say: there's there's um some interplay between these two designs, because whatever we do for the go client builders, if we're going to do tombstones in a particular way, it's going to have to support it in that way. Right. So that's kind of one of the reasons why I've been trying to get these designs to be available at the same time, because I don't want to paint us into a corner with one design option that affects the other, without being able to consider the interaction between the two left and.

B

To be honest, I know you want to move forward on the other one, but I feel like this. One is a pre-requisite for the other one like it really is more important than uh the client.

C

Yeah, I think, with the client if we can agree on a direction um to prototype, that's good enough for now and then hopefully, in the meantime, we can get some real progress on this yeah.

A

I I think it's partially, with what jacob already asked. Did you four for the declarative fee venture we had the discussion about having a field. That's only, and that's it's right only and not persistent. Like did you look at an option where you you specify um field paths that get deleted.

C

We did we considered that and the the main downside that we saw to trying to provide field paths is that they're they can be fairly burdensome to construct um yeah and it's hard to know.

C

If you got it right and it's pretty frustrating if you got it wrong and you're sending these things, and you can't figure out what's wrong with it with this, it's really obvious where to put the thing and then, when you read the document, it's really clear what's going on, so if you can see all the different applied documents, it's really clear like why what's going on with the interaction is, if it's off to the side somewhere, it becomes a little more of an expert problem to try and figure out. What's going on.

A

True, it's also challenging. I saw people having with uh customized that they didn't know why something happened. They had to write, find the right place for it. Okay, yeah, that's.

C

A cool yeah, so we're really worried about this economics, so we think from like a usability perspective, if you're just writing yaml in your editor, this is pretty close to as good as we can before. We couldn't find a better way for that case. We do recognize that, like it does create some problems, especially around the type representation right like it's, not it's, no longer just a strict subset of the schemas that you do with normal objects. It's now like got this extra thing.

A

Yeah, I I think my biggest concern is the compatibility with a existing open api schema and editors and stuff, because uh I I the people I I when customers I watch they. They rely on that because I know nothing about the old types it just press ctrl space and if we can keep that, I I don't know enough about open eye open api schema, but that would be beautiful.

A

C

Yeah, let me look into that a little more I'll add a section about in the dock. That's really helpful.

A

Great then we reached our progress update. It should be here.

A

All right, we should make ssa great for controllers. uh I I saw jenny's here. Do you want to say something about that.

F

um Yeah, I still need to get consensus on this, but it's not ready to be decided on. um I think, are we focusing on the client or is this? Are we trying to get this in.

B

First, hey jeremy, if you want to, if you want to focus on the client, maybe I can take this one.

B

That would be great.

F

B

Yeah all right, okay, cool. I I work on the cap.

A

All right, yeah, star swiping, is, is moving slowly. It's uh the most fun work. You can do copy pasting stuff and not making any mistakes while doing that.

G

I'm I'm staying on it and trying to get as much time as I can book as hard these days. um But it's moving.

A

Forward yeah, we had the api clients just now.

A

I I don't remember if I I don't know, if the person who had this or mentioned this last time added a comment here, what is it yeah? We had a discussion about this.

A

Okay, no, no new updates.

B

Yeah, I I keep thinking about it a little bit, but I need to have some real time to focus on that.

B

I can see how I don't want to go into the details right now, but I can see even how doing cup cuddle deep, coop, cuddle, deep and then uh actually applying is going to be different, and maybe you want to apply only the thing that you did and so I'm trying to see how the general part means- and I feel like when you apply you only care about some fields, not all of them like most likely you don't care about the updates made to the status, so the resource version is not the best way to have this um optimistic locking for these.

B

So I'm curious if we could implement some sort of optimistic logging on some specific fields, the fields that you care about. um So I'm I'm basically trying to understand that it's not very clear yet, but uh yeah.

A

Yeah, I don't know if we can ping him again, but it would we're probably not hurt to have some actual people at work. This way uh describe how they do it or they want to do it. I mean I mean we had the issue raised, but.

D

I uh I joined last week and spoke about this. I think uh I'm getting.

A

D

Comment on it still, but I think the problem statement is actually laid out pretty well. It's I have the exact same use case. An asynchronous plan approve and then apply a workflow, and we just want to make sure that what we apply is actually what we approved previously.

B

Okay, make sense to me.

A

Yep sounds good um mike said he's not here today. I don't think there was any more progress on the document.

A

um Same for topology, I guess did we do anything.

B

No, I've made a pl to improve the documentation and it got merged, but- um and this is kind of mentioning- this is actually probably related to these.

B

I've we've had a problem internally at google. While someone was mentioning that they didn't know, they could use these type and list map keys for clds, and so I looked at the documentation and how the person could have misunderstood that, which was very obvious, because we were not saying it explicitly, and so I tried to improve the documentation to make it clear that uh the um the open api tags can and are interpreted by uh server side apply. So we have done some progress. I don't know if it's too early.

B

A

Yeah, I I wanted to use the list, map keys and list type stuff recently in a crd, and I couldn't because our cluster is too old because we're in gk and we we didn't.

G

Get all all our new stuff yet yeah, I'm waiting for it, and I I think I only knew that I could because I'm here now so we can work on the talks.

A

um All right, we have directly converts here, this structural schema to smd schema, um and just it's still pending approval is that correct.

B

A

Was there something new on the manually updating image feeds through sub resources.

B

I think it's because the tests are flaky and so they're not merging anything right now.

A

Hello github thanks.

A

uh Tests are green now, as it seems.

B

Oh, maybe we need the no, we don't need the milestone, it's just not open. Yet.

A

Okay, so we need review from daniel or something.

B

It's just that they're not uh they're, not leaving code freeze, because the test suites.

D

B

All right, so we just need to maybe, but we probably need to help fixing the and then we'll be able to merge that all right.

A

For the api service, we need expert help. That's new.

G

I didn't see that last time.

A

Yeah and um joe's assigned to empty pot select, it doesn't mean empty. I said I haven't.

E

Progressed on that one: okay,.

C

uh Let me circle back on that. That's a good reminder all right.

A

Oh, that's that's under this item.

A

Okay um is julian. Here, no.

B

Yeah, we basically need to do a test as we we discussed about that last week and I don't think we've made progress, but we need to make a test um about these.

A

All right um and we have to still fill out this. uh I I added a few things to this question. Thingy, um I'm waiting for um uh lucky. I I don't know um to tell us when he has time and we look at it together. Yeah do you want me to do anything about it? um You can you can look at what I wrote um and I think if a few questions are or like addressed to to both of us like every one of us has to say something. I try to say some general things.

A

If you see anything.

B

A

Incorrect, but it's not super important, so keep it on your on your queue where it is. If you are busy sure all right backlog, I don't think there's anything new is there.

A

I think we're good to be honest, okay, yeah. uh What one thing I I saw while I was searching for service center plan github, we have a few issues um that don't have our attack. So if you see anything that seems related feel free to tag it with our github tags, we get everything together.

B

I um isn't this done by the uh the bug scrub.

A

Partially yeah yeah, um but not always, I thought I I thought we have to edit label ourselves and some and we only get cc'd and assigned to the yeah. Well.

B

Typically, look at it and yeah. If I look at it, I try and remember to set the type.

A

Yeah I yeah I, I don't think we should go through all the tabs. I have opened up, at least if you.

B

A

Yeah I'll, uh if I, if I find something that we should look at I'll post it like or add it to to the agenda or doc and mention people, I just wanted to see how how other projects have problems and if we're causing anything all right, then I think we're we're good awesome.

C

C

Fourth, great thanks: everybody and um jenny will be doing a lot of the work on the go client um so be sure to circle her in. um If you want to talk about that.

G

A

Thanks thanks, everyone then have a good remaining morning for you.

B

A

G