Istio Technical Oversight Committee, 22 Jan 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Technical Oversight Committee 2021/01/22

Description

Istio's Technical Oversight Committee for January 22nd, 2021.
Topics:
- Follow-up on untested platforms
- 2021 Roadmap guidance
- Doc Working Group updates
- CRD Status API Graduation
- Policies & Telemetry renamed to Extensions & Telemetry
- VM Beta Graduation

A

To look at the author of the test and ask them to fix it well right now there are issues which are release. Blockers opened up against the owners of the pages.

B

That but yeah, I think they may not. If they didn't write the test, they may not know how to fix the test. It may actually be over with the test case.

A

I guess I I we we can, I think all of them had somebody at least tagged in the issues. I know I've created them all.

C

But I don't know if they all have.

A

I don't know if they all have owners and they probably aren't and you're, probably right. They probably haven't.

B

A

With the person who wrote the test.

B

And you know they are tagged. You know how it goes. You know like how many people can keep up with the flood of information, so um you know I swear like how do you want to handle this? I mean if, if um one thing I've seen in the past, is that it's really good for for any github issue to make sure that there's one owner so there's no sort of confusion about who owns it and then actually reach out to the owner and get them to act that they own it and fix it.

D

The problem which eric is mentioning is we don't even have an owner yeah.

B

So so so I'm saying that we needed you know do a little bit of owner. You know make sure we have the right owners for these issues.

D

Okay, we can review that in the.

E

That sounds great. Thank you all right.

C

D

So how do you want to handle next week if we don't get enough owners for 75 tests, and next week is the last week by toc? The week should be done? What should we do for the 2-0 which are not claimed.

B

I don't, I think, historically, have we actually had 75 tests.

D

We did not, uh but this time I think there were a couple of missing tests and jacob did a good job by adding those in the template, so they may have increased the number of p0s.

B

Yeah- and I just I I I would say that as long as we're in this land of manual testing increasing the number of p0s is just going to make it kind of non-deterministic behavior, which p zeros actually get tested, um it'd be better if we could just focus on the p zeros that are really truly p. Zeros.

B

B

uh Other people in the tlc I'm doing the complaining talking here. What are your thoughts.

F

The same thoughts yeah I mean.

B

It's hard reality yeah.

F

I mean you're you're reading from a a well used and often cited uh concern.

F

Is it, is it just me, or did the number of p0s go.

B

Up, yes, it went up, I think, that's what we're hearing so.

F

B

That, let's try to go the other direction and can somebody stop the recording? So I don't have to like go on the record saying we're gonna reduce.

B

F

um What is the reason the number of p0s went up? Is it because we added critical features.

B

Yeah I mean so part of the way of reducing that set. Is we just make a call? You know the word group, please make a call on whether they think anything going in in the release is likely to upset.

B

You know the the functionality that is detailed in the page, and if it's not, we just say okay, we don't really need to test it. It's probably gonna be okay, it's a p1. In that case, and of course you know what happens with p1s.

D

So josh, one thing which we have discussed in the past is the concept stage, most of the time that don't change, but we still keep that as p0. um Would that be a risk for anyone if we move that out from q031 for this release, I.

B

Would ask the working group leads if anyone thinks it's a risk and if no one says it's a risk, I'd bump it down to p1.

C

Okay, yeah, given.

D

C

Stability release, so probably the risk are very minimal.

C

Louis sorry,.

F

No, no, I I think, that's roughly what I was going to say only more eloquently.

C

G

C

Other you guys.

H

Have worked together for a while.

B

uh Okay, not clean 75 all right, so I think any anything else on this one or should I move on to the next one.

D

Please move on to the next one.

B

Okay, let's do the oh, hey, looks like a familiar link, uh pick three test cases to automate.

E

So two of the working group did that, let's start and we.

D

Have 15 test cases which needs help.

B

I think this is this is a great approach instead of boiling the ocean. Let's just try to make a little bit of progress. Every release, so working group leads. Please try to get three done.

B

uh And good point john: why do we have? uh uh Why are we, including.

B

P zeros um that are automated here are we, including p zeroes that are all made in this list.

B

It's not 88, because some have already been automated yeah. How do I, how do I tell how do I filter those out.

I

You already filtered them out, but that list is not 88 rows.

B

It is 54. you're, right, yeah.

D

So john, the one which are automated: how can we get the updates on whether or not they're working um the because they're not getting updated in the sheet.

I

uh I'm not sure what you mean is the whole point is isn't the whole point of automation that we don't have to worry about manually testing it anymore.

A

The automation ones that the automated ones that are failing today have github issues on in the github column yeah. So there should be, I believe, six column or six six of the automated ones that have a github issue associated with it.

A

That's what I've been doing to try to identify the automated ones that are failing today, okay,.

B

So those should be fixed. um Okay,.

C

Okay, so that goes the number I guess 6-0 instead of 88. yeah.

B

More palatable, but still too high.

H

F

So our our convergence rate, if we're doing what what would we say, three per working group right.

J

B

It feels like a 30-year mortgage payment or something like that. 106.

F

uh It's at least 15-year.

B

K

F

uh So we would get 18 a release right.

D

Not all of the test cases can be automated, so what we.

J

D

F

Sure, but we would still make good progress on what 60 right, yeah.

F

And what is our growth rate.

L

Yeah, it's a good.

F

Question do we know how many we had in one dot.

D

Oh, I have to calculate that I do not have those.

B

Yeah, it sounds like wouldn't be too hard to calculate, but no one knows that off the top of their head. Okay,.

B

Okay, uh what did I do? Okay, sorry uh ais from last meeting what to do with untested platforms like alicloud? ah Yes, josh needs to do this.

F

This was the dog's question.

D

Here um I know you took volunteer job of reaching out to them, but my question is: what are you planning to do in terms and the banner which we have added? What should we do with that?.

F

Right so there were some suggested verbiage changes for the banner right to make it less terrifying.

C

B

What what is the banner can, I is that here.

C

It's one of the platform I think maybe or something if you go there, you'll see it yeah.

B

Yeah, that's a terrifying banner.

M

uh uh Yeah so so I had actually changed this to master based on uh feedback from marriage. uh I'm just waiting on the uh cherry pick to be approved for.

B

M

B

Jacob, if you wrote this banner, this looks like something I would write.

A

Somebody's attention: yeah yeah, if you go to preliminary.skill.io, you'd, see the the new one. Okay.

D

Awesome. Thank you. Jacob.

F

Yeah, let's look.

B

Is there a dot between preliminary and istio or just.

G

B

There you go: okay, hey it's! It's! Yes, equally, accurate, less terrifying, good, uh actually, uh quicksandy check. Anyone have any concerns with this new banner.

F

um The only question is whether we want to still indicate that it was vendor provided.

B

Content that would help understand that would help. If anyone looks at us and say why isn't it been tested that would answer that question.

L

The one question I have about this- and I know yes, it's not as scary, but a lot. A lot of these haven't been tested since, like 1.3, if you say, hasn't been tested since 1.9, would people assume it was tested in 1.8 or 1.7.

F

Well, I don't it's, this phraseology doesn't say since right, it just says has not been tested on whatever the latest for the.

B

Yeah, you could add a you could add in may and may have not been tested for prior releases too.

F

Yeah, that's probably overkill.

C

Okay and also there's the last updated, which is two years ago for this page, so you can tell.

B

All right, let's I'm going to capture some notes here, uh all right, so we are saying what do you want to substitute? How do you want oh, come on.

B

What do you want to change this to.

F

So so there was, there was a question right if we wanted to blame the vendor right effectively,.

B

um So this vendor provided vendor maintained document.

F

In some cases, some of these environments were written by people who didn't work at the vendor. Do I have that right.

B

C

C

um I would think it's most people from the.

F

F

In the case of the alibaba one uh yeah, but there were jacob. What do you remember what the list of them were.

M

um I mean I could get the authors, but I don't know uh like where they were from. I couldn't find any association uh with those authors. You know, okay,.

F

Let's just assume they're from the vendor and they can always yell about it later. Do.

B

You want vendor maintained, then.

F

uh Vendor provided or vendor maintained, which one well if it, if it's two years out of date, it's not maintained just.

F

M

F

And then you know that provides an incentive for the vendor to come back in and clear the flag right. Yeah sounds great. I like that and or provide automation if they don't have to do that.

B

Okay, I think we are moving on so sorry context here, uh iso 2021 roadmap theme, guidance, uh niraj will take the lead on this. Niraj are you here I haven't heard you speak up so you'll all pile on we're missing busy.

B

What was that mitch said? We're missing mirage and dan okay? Well, I think mirage is, is busy leading the effort in the 2021 road map.

D

I needed a family emergency. I had a discussion with him earlier this morning.

B

Okay, all right uh anything, we should consider here right now.

B

We had a review on this recently jimmy.

B

F

I thought it was presented in the work boot leads meeting.

F

D

F

D

The working group leads are working on updating the sheet and there are presentation dates on which they will be presenting soon.

B

And then I think, the other than you know this plan we're also asking for the kind of the a road map for each feature for the promotion.

B

Yeah sounds good. All right looks great.

D

um The first step, the toc members are working on the bees and neither just taking the lead just for everyone's.

B

Okay, anything sweater that you want us to do other than just take a look and say it looks good.

D

Not yet let the working this doesn't. Thank you. Okay,.

B

Sounds good uh doc working group updates who added this one? Who wants to talk.

C

Oh yeah, so I'll get started, um so you guys all know frank um has been working on the documentation. Will group lead for probably a little bit over two years uh and in january he submitted a pr that he wants to step down from the documentation will complete so that pr is already been merged.

C

um So now we have, uh we have been discussing this um scenario um on the steering committee and what we kind of decided, because of lacking of one person willing to step in for frank's job as documentation worker fleet we're going to try to establish a auditory review team so far, I believe we have six members from steering staffed for that review team and we certainly would love to have some of you willing to help um be part of the review team as well. So I guess this is a call for action.

C

If you guys are interested, um please let us know here or offline, you know, would love to. Have you guys part of the review team.

B

Do people uh so, okay that contact thing? How do they reach out to the steering committee? I guess I just no so.

F

Oh yeah, how do you want them? Dude.

C

F

We'll have a group who are content reviewers, who can provide editorial review right. That does not mean that they're necessarily going to be able to provide a technical review right.

F

So right, that's that onus still resides on the working groups, and the working group leads to make sure that the content is technically accurate, right and consistent.

F

um So this group is to make sure that the docs uh follow. You know, content standards right and are consistent in other ways right the things that frank was doing before right, yeah.

G

I guess I would I'm sorry go ahead finish.

F

G

F

Things that you know, uh riggs and adam were doing before right, where they're not the subject material expert, but they are editorial experts right. They can provide guidance about how to write good docs.

F

um We don't have professional documenters on the project anymore, so we have to backfill for that function in some way, and this is the best that we can come up with, which is we have a pool of reviewers who are going to do their best to play dog writers in their spare time.

B

What happens to the doc working group.

F

Great question so there's still a question about whether there's a a remaining function, right, which is someone who is responsible for docs in all the ways except content right. The infrastructural aspects of docs, any tooling linting.

F

And maybe a managerial function to make sure that.

F

Content is being produced and maintained, not not that they're responsible for the content, but that you know they're doing all the infrastructural things around the content and possibly participating, maybe somewhat in pad hurting, and I was going to talk to you a little bit about uh help with this process, because now we have a more diffuse process that will need a good organizational muscle, um something that I don't claim to be good at.

F

But I am happy to be a content reviewer. I volunteered to do that. um So there's a question right to define the charter for the docs working group and leads so that they could run it and and what is that charter.

F

And this is the phase of the meeting where it is entirely open to public comment and suggestion about what they think what people think might be an effective docs working group charter and who might be interested in such a trigger.

F

But first maybe we should talk about the pool yeah before we talk about the charter.

C

Yeah, so one thing I would want to add to this is um now we have technical review and auditory reviewer, so there may be a little bit more approval process involved, especially at the initial stage, because it used to be frank, is kind of doing both technical and editorial as well.

C

So once you have his blast on the approval of the pr, the pr would just merge now you would have somebody from the maintainer, a worker plead from a giving worker first and then somebody from the auditory review team and because of the editorial review team is also new. So initially they feel more comfortable to have actually two people review. First um from the editorial review team.

B

uh So I still have a very basic question: how does somebody join the editorial review team.

C

So I volunteer to set up the github uh for the auditory review team, so they can just contact me offline unless we want to set up an approval process for that which I don't think we want you at the moment.

A

And I guess my question is: is this going to change the current behavior because right now I think it's frank myself and I think nate are probably the people who are approving the prs that are going in right now?

A

Okay, so if we're gonna change that process right, do the three of us become members of this review committee.

C

Yeah, so that's what I was mentioning. So if you guys are willing to continue to be editorial review, which frank clearly said he doesn't want to be, you are willing to sign. You are open to sign up with me. You know to be part of the review team um else. Whoever who else is on the review team would do the auditory review and then we still want the technical side, whether a maintainer or a working lead from the work group to do the technical review as well.

A

Okay, I mean that makes sense. Have somebody from the from the from the real work group review it as well as some editor sort of person exactly.

F

Yeah, the technical review should happen first and and the content review would happen that editorial review would happen. Second, I think that that would be normal right. um We don't have to run in that order, but it probably better if we did.

F

You probably also need right- and this may be within that pool- or maybe some other function, which is a super reviewer right, which is someone who's, just capable of performing, who is entrusted to make good decisions in an expeditious fashion. If we need a review to go through quickly, um but that that would be an exceptional thing and not a normal course of business thing.

B

Do we have anyone identified for that role yet.

F

No, I mean we we we have, I guess you know eric and nate who have been doing this, um but that's not like. I don't want them to like it. It should be an exception right like okay, like we need this to unlock the release, and so this pr has to go out today, kind of a thing and for whatever reason we already have that function, though right there and the toc leads, can perform that function.

C

Oh yeah, the first smart force merge.

F

Right we can, we can do that. So I guess we don't need that. I guess we can just say that the toc can do that in exceptional cases, but if people want to volunteer for it, that's fine, sorry. What was that john.

I

Sorry just side conversation about how we're going to actually do that enforcement because I don't think github can say you need docs working group and never. For example, yeah.

F

I don't like it you're right, it is convention.

C

Yeah, but we still want to make sure the people who are in the pool is that have engaged to do the final approval.

I

Yeah, that makes sense. That's what frank would often do. He would take a look at this pr and then he would approve it only after that happened. So.

C

Yeah, except for smaller one, you know he would just approve it.

J

Can I ask a related question: um do we do we have any guidance on on where to put non-critical, docs and wiki? We discuss a lot of things, things that are not meant for most users, but we still won't document it and maybe with a lower bar.

N

That that has been wiki right. The the only the only challenge has been with vicky is a virgin awareness. We have uh people have landed on wiki pages, which were specific so now several people have started like putting version specific information in the wiki. As long as we do that, I think that's an easier and a quicker thing.

J

Works for me, I just want to make sure that we have. We are on the same page.

N

Well, I mean the tlc, I I don't know the tools he thinks.

C

Yeah, I think wiki is great. I would appreciate some discoverability of the wiki. Maybe someone somebody created a useful wiki. They can socialize at least in the workforce meeting. So people knows because I do know that john wrote a really good wiki page and I didn't know it until kyle stoney wrote a blog about it, and I happened to read his blog to learn that how do you troubleshooting istio?

C

So these are the things you know. I'd rather know it. In the first place.

J

I know you like this: how about twitter.

C

Twitter could work too depends on how much you want to be discovered by the user if.

O

We wait long enough. Carl will tweet about them for better.

J

Okay, from now on, all the dogs will be tweets.

O

Louie you're, muted.

B

Should we move on or anything sorry.

F

I'm just gonna say: there's, there's, there's always an istio.io. If we wanted to have a separate docs pipeline that published to that for things that were just truly internal, um we could do that. For the moment. I think mandar's suggestion in the wiki is more than adequate.

B

um I I just want to capture, can I check point mandarin's suggestion uh mandar? What do I say here.

N

That wiki is okay, but uh make sure that, if make sure that the version is mentioned.

B

N

Because wiki itself is unversioned.

B

Yeah, okay and then um uh eric what were you gonna say.

A

Sorry, I interrupted you no problem, um I was just gonna say. There's still, and I think we mentioned this earlier right is there's still some. If you will documentation specific things like the scripts that we run at release, time to generate the new release and populate, you know istio.io from what was in preliminary.istio.io, etc.

A

So I don't know how that is necessarily handled if there isn't a work group per se.

F

You know I I think there needs to be a work group and a work group lead yeah for that function. Okay,.

C

Yeah, so the notes um that josh captured early dark global shifts to more of an infrastructure focus right. Okay,.

B

Okay and then somebody else uh try to speak. Who did we clobber.

J

I'm trying a quick follow-up story, uh so one question I have is a lot of the docs are all for or kind of experimental in his official docs. Is there any way we could move that stuff to the wiki or whatever? We do.

B

That's interesting: what do you guys think about that, like if you're experimental be on a wiki? Don't even don't even take any? It's not better!

B

That's actually our documented standard, oh I'm in favor of it for for experimental for alpha. So I think.

O

Experimental feels.

B

Like a no-brainer alpha, I could see us having a discussion on.

O

On the community page, it states that we do not provide user facing documentation for experimental features, yeah, so, okay, so experimental.

B

Let's say we key first, um what do we do about alpha.

L

I'd argue two different ways here. First of all, we want users to start using alpha, but the other thing is: um there's no there's no requirements for documentation tests or anything like that with alpha. So there's no guarantees that it actually works, but yeah.

C

Actually, your alpha checklist has documentation in there because I went through one last release.

L

I thought that we didn't require that until um beta.

J

They can use it using the wiki, but at least it will be a clear distinction between what is stable, reliable and uh you know we can focus basically on on on the stable stuff, and I don't know.

B

Do do we have concern, so I think it it's. The current state is that people can add a documentation to istio.io for alpha features. Do we actually want to say no? We would really prefer that to go into wiki.

F

That's super hard right. I mean yeah like with alpha apis right right, which are very clearly intended to get user feedback.

O

Right if we put them in the same place as we put documentation for experimental features, it will be more difficult for users to discern the difference between the two experimental features. We don't want, users adopting and alpha we do want them adopted. That's a good way of looking at it.

J

But I would rather have user confused between experimental alphas and between production and uh yeah.

B

But then we won't get adoption so yeah, that's the trade-off. You know, adoption versus uh you know greater adoption, but greater risk that people can take a alpha feature out of production uh without awareness of the risks.

F

Yeah I I would rather, that we in the docs, where something is alpha, give people or make discoverable more precisely. What alpha means in that context um and look at the the documentation user experience around that rather than.

F

Hiding things that drastically.

O

We do currently use an alpha banner that sort of explains that it's there for feedback. I can give an example in the chat here.

F

Yeah, no, I I've seen it.

F

F

Yeah, I mean that's clearly appropriate.

C

F

You know if there's a question about that that not showing up everywhere it should right. That's that's something else we could talk about, but.

F

If users choose not to read that well, what do you want to.

B

Do move on to the next one.

C

F

Well, I guess the question is: if there is this more defined role for the docs working group lead, we need candidates, so if, if people would like to self-nominate or nominate.

F

D

Is there a possibility of advertising the role as well? If nobody from here is interested.

F

Yeah, I just view this as one one avenue of advertising.

D

What about discuss, dot, io or twitter? If you have.

F

Not twitter uh discuss yes, the work group leads probably can also bring this up in their meetings. um Usually this has been the most fertile ground for asking for nominations, but I'm like obviously not everybody shows up all the time he included. So um let's just start that process and if people are interested.

D

Okay um is there, is there um can frank help with one pager of what may he may be doing outside the charter of docs, so that we are not missing anything and uh it's a discovery later so that way, at least if we can write what he was doing as a work in the docs working group lead. That may be helpful to jot down where we need help.

F

Yeah, no, I think we should start a process of refining the charter right and and frank can be a part of that.

C

Okay, yeah. We never had a dark horror group chatted so who is going to own that character? Who is willing to drop the job.

F

C

F

Be willing to do it as part of his handoff if he would win.

G

F

Okay, all right, then, that that's on the toc, if, if it's not on prank,.

B

Yeah, I you know, I I think that if somebody so one one way of handling, it is um whoever volunteers, for it can create a charter right. Otherwise it might be bound to a charter uh yeah either way.

F

Yeah I mean ultimately, the tfc has to review it and approve it.

B

Yeah, but if the toc creates a charter, somebody else might look at the chart and say yes, I want to sign up for this. It's more concrete right. Yeah. Does anybody in the tfc want to write a charter for the doc working groups.

F

I don't feel qualified to write it, but.

J

F

To write it um eric are, you: are you willing to uh like help me write it even if I'm not implying that you're volunteering for this job.

A

That's fine! I can do that too. Oh that's, great! Wonderful!.

H

H

B

Status: okay, going one's going twice any anything else on this one and I'd love to close it out.

C

I don't remember: what's the decision for the experimental.

B

uh Experimental features in terms of where the documentation should land.

C

Yeah I mean: are we going to move all the experimental to the wiki? Oh, is that going to be defined in the docs chapter?

C

B

I think I think we said that new experimental content should go to the wiki. I don't think we address what we should with existing experimental content and I would just say I would be fine just saying you know let it stay there and either graduate it to alpha or or evict it over time.

C

Okay, that's fine yeah.

G

B

Okay, fine just had to say graduation, um so I think that mandar and mitch do you want to. I think that you're you're um you're, saying you're you're, going to reduce the scope of the api to get a subset of the api approved more quickly. Is that right? How would you want to introduce this.

O

Utl, I think maybe they should introduce it.

B

So that would be center louis. We had a random discussion on this, but uh you know that that wasn't from the toc but mitch. I guess I don't understand the intention of of bringing this up to the toc now. So if you could.

O

I did not put this on the agenda. This is not my idea.

N

It was, uh it was already on the agenda as a um as discus api or something like that, and I just converted that, to kind of more specific, we can choose to skip it right now, if that's that's more appropriate and we could move on to the next one.

B

Okay, uh then it's just if you want to discuss it now, if there's anything you know you'd like us to review just let me know, otherwise we can discuss it in some future week: okay, okay, okay, great.

B

um Okay, next one doug read uh rename policies and company working group to extensions and telemetry working group makes sense. Now that makes a lot of sense.

H

uh Well, I'm in favor of it because I wrote the doc, um so we've done this in everything, except for name officially, I mean our slack channels the way we refer to each other, but it's still causing confusion when we look at goals and ownership of tests and things. So it's just asking for toc approval to officially rename policies and telemetry to extensions and telemetry.

B

Should just be extensions, and then you know, policy and plundering can both be extensions.

N

uh No, we, I think we should call out telemetry, specifically yeah, because it's uh yeah people.

F

Are asking, I think so too? Okay, should it be just just, should it be telemetry and extension.

N

Let's see, I I think it, it should be whatever words we are already using in in some of the other docs, so that we don't change to change those again.

H

And constantly we agree, that's why we're trying to get rid of policy from the name here?

H

F

Policy keeps coming back to bite us. um I approve. I approve.

P

F

If you want to call yourself telemetry and extension, you're also welcome to do that.

H

F

Thank you. That's all.

F

uh Do we have quorum to approve? I think we do.

C

We have four.

A

B

Okay: okay, okay, uh john dm graduation.

I

Yep uh we're looking to potentially mark the sysbeta for 1.9. um I will full disclosure we do not have every single checkbox on here marked off. um The reason is that we don't currently have the infrastructure to do like the long-running qualification tests for vms, yet we're working on it, but we don't have that yet same with the automated documentation testing.

I

I will note that multi-cluster also just went to beta and they don't have the release qualification testing either they don't that's. That would be our two big gaps there. It's just the the automated testing, so we of course manually tested the load and uh the documentation, but we don't have the automation yet.

N

So, john, what what about the? uh What about the doc yeah the documentation stuff that was identified yesterday in the review is that is that considered? Was that considered non-blocking.

I

Oh yeah, sorry I I forgot about that yeah, we did this affordability review and we there was more documentation requesting uh requested. So we're going to work on that, and I guess what I was looking for was like conditional approval. Then, like once we had those docs that were we're good to go. I put this on here before we did the reviews, because the reviews yesterday so I kind of forgot about that but yeah we basically we're missing documentation on like operations like how to debug it. We have all the all the information is there.

I

We just don't have the documentation on how to actually do it and like architectural diagrams to understand. What's actually going on. um That was the result of the supportability review.

I

So our plan is by one nine. We would certainly have those docs by one nine. We will almost certainly not have the automated testing in place, though so.

F

um What would it take to get the automated testing? What is necessary.

I

Yeah, I think, there's two parts: one is the documentation testing. um I think we can probably do like it's it's hard to get a real vm and a test in our pre-submit um and possibly not even something we want to do because it's slow and expensive, but we could do like run it in a docker container and it would probably work for the docs test. um Actually, I know it would work because that's how I do all my development, but we just haven't done that for the automated testing. We everybody like the load test.

I

We just need to come up with some way that we can automatically provision vms, set them up, connect them to the the cluster. It's more just about automating um that stuff, and that would be you know, with real vms. So.

F

Like real vms, as opposed to something like cubeverse,.

I

uh Like in our in our integration tests in the main istio, we actually don't run vms, we just run them as pods with every single feature disabled. So, like we turn off dns, we turn off. We don't match them with the service, et cetera, and so we have like fake vms, essentially.

F

But we're still getting some of the effects of kubernetes networking because we're in the pod right.

I

If we do, we do get some. I think it's, it's proven to be a pretty close approximation, though, and it's captured a lot of issues with real vms, so yeah.

B

So, john, I I think I'm trying to summarize the exceptions you're asking for so one is uh some doc issues have been identified. You would fix those before marking beta the other one. Is stability tests don't currently exercise vms? I know that people on our team are actually working on that now, but those tests would happen sometime in q1. They will not. They wouldn't block the release, so they'll fast follow um they'll. Just search is your release, I mean um and then.

F

Justin those tests would actually be able to be put into open source, or do we think that those would be able to? I think the.

B

Good question: my understanding is that we put them into open source. Okay. Is that right.

I

Yeah, I think so I mean I'm sure they will only run on gcp, probably because it's kind of hard to make a generic cloud provision vm set up. But that's that's kind of already.

B

What we do so, I think earlier you made a statement that we would manually do the stability testing. That's.

B

I think that without the automation that manual test would be laborious to set up, so we wouldn't do it, for you know patch releases stuff like that, so there is.

I

I completely agree: that's probably not something to do.

J

Okay, uh josh, I think I think we also discussed many times having the stability clusters running, for I mean that we are doing for release to be running continuously, uh also to part of the upgrade the work that we are doing, because we want to verify the upgrades so uh presumably we'll have vms that we set up we tested continuously, including for upgrades and stuff, so patrick. This is not in any manual testing. Basically just the initial setup.

B

Right so, okay, so I guess what I mean is that you know they're kind of like three things: there's a a one-time pin of the ass manual setup. We want to we'll do that just for the initial beta graduation, but we can't stay there for long.

B

The second thing is we have our our current stability test, those will start exercising vms, but they're still manually, triggered and interpreting and as far as I'm concerned, that's really the bar for beta uh and then there's a third thing of like automatically triggering these tests on new releases and doing some sort of pass fail automatic determination, and we don't have that for anything right now. So I wouldn't call that a blocker.

O

Josh I I'd wait. I'd say that I agree that it's about time, vm moves to beta and that you know these are small enough things that it should move forward. However, I do want to point out that this is a different standard than we applied just last week to the ux promotion to beta request. We have.

F

Yeah we have not agreed on this.

O

By the way yeah, so we can yeah.

I

O

We'll tell the.

B

I

Between that is that his contention here is around testing, whereas the contention on the status api is in the implementation. I haven't.

O

Heard anything last week it was about testing this week. It's about implementation and the conversations moved on, but last week. The reason that we chose not to promote the beta was because it was not yet enabled in our automated release qualification, test pipeline yeah.

B

O

B

That this this the first, this is where we should be focusing the discussion. Is this? Okay, I'm not sure it.

Q

Is so I I think the there is a difference here right between a an optional feature that does not affect anything else and something that is on forever. So I I think, there's there's a little bit of also equivalence there.

Q

The question here is for vms, where you know it's it's still in beta and it doesn't affect any existing um environments.

Q

Is it okay? If we go out without having automated load testing.

C

Right, I would much prefer some testing in dark, even with darker containers, so that you know people follow. The instructions can actually get it working.

B

Was that, but it would you block it because I think you know one it would be holding it to a much higher bar than other features and then two this one is perfectly hard to do against vms.

C

You're right but multi-cluster, we have automated testing right every single page of multi-cluster. We have automated testing, that's cool.

B

Is it okay, so lynn, are you saying you would block beta on this or or is it a strong preference.

C

I would definitely strongly prefer, especially when seven kind of told me he couldn't get vm working with 1.8, so you know we've been having this for offer for a long time. If we make it better, I think the expectations people should be able to photo the doc. That's working.

C

And I'm not saying you have to do it on the real vm, I mean if you can do a docker container, it's better than nothing.

I

Yeah, I think one thing is was that adding the docs testing is probably easier than adding the stability test really possible.

N

I

If we, I think so, I haven't actually tried it, but.

N

So actually, john, what like so I for stability test. Yes, I think automation is important, but we do have long running multiple clusters and adding like whatever five vms to that test, while the real automation is in progress is actually a fine compromise to get some mileage on on those tests of like number of days and uh whatever else like I I I don't think that automation should just completely block like testing is, is more important.

I

And go manually add like five vms and connect them, but those vms are manual, but the it's connected to the automated ones. Is that what you're saying right correct? That's a good point. I I think we could do that as well.

J

And we'll also close upgradeability best, because.

F

Okay, so it sounds like there's a a level of mileage. We can get to right between some of the dock tests, but maybe docker container only and some of the stability tests we're not getting all the way there, but we're we're getting very close.

B

Yes, I I haven't heard anyone say that the automated dog testing is actually a blocker, I'm hearing strong preference that would put in the fast follow category, but in terms of adding it to the stability tests. I that might really be a blocker.

Q

So I think that the concern with the docs testing is that doc, like setting up vms, is very complicated. So it's it's more likely than normal that the docs will not work.

Q

Then then say it's a blocker. You know it can be a blocker.

C

All right just get upgraded. You want now.

B

Okay, uh so that's one decision uh before I move away from that. You know john, what do.

I

You think uh yeah, that's fine. I also wanted to ask, because all these things are documentation and testing. If we take longer than 1.9 to do this, can we still promote it? um However long it takes for us to complete these, because there should be? No, you know code changes.

F

um I think I think we've been okay with the the post post-release promotion.

F

If there were no code changes.

Q

Or or if there's only bug fixes that went into a password.

D

Are we allowed to promotion tax releases.

Q

I don't think, there's anything. Have we done that before.

I

It doesn't even need to be in a patch release. We can promote right. We just operate a random day yeah. I was assuming.

Q

A bug to fix, but yeah.

F

Yeah, the only question is right here, like the release. Notes are the mechanism for announcing the status change today right, we would need to figure out some other microscope.

B

No, I think we can. We can just be transparent, which is that you know uh vm support in the initial 190 release wasn't considered ready for beta, but give people advanced notice that we may be able to promote the beta. You know midstream, so stay tuned.

F

So just stay that and say that in the release note yeah, if that is a possibility, all right.

Q

Q

I think we should also also announce it to beta in 110 for people who don't read one patch appointment. Yes,.

O

Yeah well, this is a big enough feature that a blog post seems like it would be warranted as well.

Q

I think so actually.

K

Had a couple of.

Q

Quick questions on um on the beta graduation, john, the so are. We are all the apis moving to beta as part of beta graduation. uh That just has not.

I

Been planned, okay.

B

All right um all right: we need to checkpoint a few things. First of all, the highlighted, uh yellow text does toc actually agree with this statement.

B

I agree with that yet flynn, willy.

B

Then we still need to so. Actually maybe we've already answered the disability test. I think we're saying that stability tests are also a blocker, but it's suit, but but as soon as it's satisfied you know we could potentially um promote to beta in a one-night patch release. Yes,.

B

C

B

John thoughts, that's that's fine for me.

C

Okay, yeah: it's a really short release to promote things. One night.

B

All right: okay, great going one's going twice.

F

B

Can either these been done quickly? Okay, not the first one. Can we do jacob's quickly.

M

um It's fine to talk about it next week um it unless people uh see this is urgent. um We've discussed similar things in the past.

B

I I would I would be. I would approve this right now. um If does anyone in the toc disagree with an immediate approval, if so we'll just discuss next week, um I.

F

Am playing with google groups being a mechanism.

B

Okay, so we can discuss next week then, but.

G

Yeah, okay sounds good.

B

Okay: okay, great thanks.

D