Istio Technical Oversight Committee, 9 Apr 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Technical Oversight Committee 2021/04/09

Description

Istio's Technical Oversight Committee for April 9th, 2021.
Topics:
- Moving TOC meetings to Monday morning instead of Friday morning, and an hour earlier.
- Inbound Forwarding for 1.10: Approval to add upgrade pre-check command.
- istioctl analyze warnings on alpha/experimental: Approval to add whitelist of expected alpha/experimental usage, warn on the rest.
- 1.10 release pushed to 5/18

A

It'll be starting at the second floor. There you go, we are now live.

B

All right uh so, let's get started so the weekly routine. Here I guess, raise managers. That's done right for 110, so we should just take.

C

This yes, please, by the way I would like to call out a few of the old managers are helping. So thanks a lot in in reviewing the process and helping us out.

B

Yeah, that's excellent, that's good to know. uh We really need to start sending some swag once things become better in qs, I guess for people who are doing many jobs, uh let's see so the weekly routine here feature freeze was april. Sixth, we have done the branch cut and our free feature frozen. That's good! uh Anything to add there.

C

No, that was just fyi.

B

Okay, excellent and then the first community testing day is april. 13Th, tuesday.

C

Correct um we have a couple of automation which needs to be done, but separate than that uh there is no sign up since last week and uh the automation required the ones which actually requires testing at 34, because other than that they are automated. So 34 test cases which are still p zeros requires the ownership and the testing is starting tuesday.

B

Okay, so this is a call out, I guess for all the attendees and your buddies who work in sto, let's sign up uh and crack these 34 manual tests, so I'm guessing there's no way to automate these. That's why they will remain manual forever or is that just lower priority.

C

So there is a caveat right out of 34: 12 can be automated. The rest are the ones which either needs consideration, or they can cannot be automated.

D

B

All right in the last community testing, I'm guessing, we don't do the live ones anymore. We just have the asynchronous ones. Is that correct? That is correct. We still.

C

Do the asynchronous.

B

All right, so what I would suggest srita is see till monday. If you have more participation, if not, you can focus on slack and make, and during the working group leads meeting bring this up. Hopefully we'll get enough participation.

C

Okay, great we'll do thank you.

B

Anything to add here, other folks: okay, it's pretty routine stuff uh release health for 110, so there are three release blockers. Do you want to talk about what they are.

C

uh We can talk sam. Do you want to talk about that? I think we have a good handle, so I just want you to give fyi so sam. Why don't you go ahead and get some.

E

Yeah so uh three release blockers, um I think, as of like the last hour, that should actually be two release blockers and ten p0's, but um I think it's premature to discuss the specifics right now in toc, but I've reached out to the owners individually and um I I think that these are release blockers. We have paths to resolution for so.

B

uh Okay, yeah all right, so it looks like the way you said it. You removed one of the blocker to be a p0 okay,.

E

Yeah, that's good.

B

All right, so let us know again if you're short, stuffed and somebody needs nudging. It looks like sam and the other release managers have this handled well, so we'll keep moving next release, istio 111, all right, so the timelines are 10th august. What's that, I think we discussed it last time. Do you need an explicit approval? Srita or this is again an fi?

B

Are you on newt.

C

Sorry, we have an approval already. We hadn't agreed upon time.

B

All right, I'm just gonna, say this is approved so next time we don't have to talk about.

F

B

Oh, nobody wants to look at that.

B

Nobody wants to know what I was doing.

B

All right docs for achievements, uh there's nothing written here, but I just want to open the floor up. uh Are there any achievements somebody wants to highlight? We should do that, and this is not restricted just to the toc members.

B

Oh, can you keep moving then.

G

B

G

Want to say I thank you to everybody working on um security related fixes and cds.

G

I know there's a lot of work being going on in the community related to that without naming names, naming people naming cvs or anything like that. I do want to acknowledge that.

B

Yes, yes, I mean I'm part of product security, working group louis and there's lots and lots of effort going on uh so for folks who are uh on this call, it just means we are gonna, make our security better, it's not where it is, but it takes a village to make it better and that's what we're going through.

B

B

All right keep moving, doesn't.

H

B

There are any docks for offline review. ah Shruta did you have something nope all right uh jacob? I think this has been here for a while. I don't think is this ready now.

B

B

He's here, but he's not responding, so I'm gonna suggest we remove this because it's just been sitting here for a while with no activity and then bring it back once it's ready all right. If nothing else, we'll move on to today's agenda items.

B

Does that sound good all right? So, first one. uh Hopefully this is quick, uh I'm proposing to move toc to uh the monday time time slot at 9 am pst. The two reasons for it. First is I wanted to move the time earlier so that folks, like me and other from different time zones, can attend in a reasonable hour. uh I had discussed it with other toc members and we had rough approval.

B

The issue was on friday. There were other meetings like steering where which was already on: 9 am pst and it looked like monday was kind of empty for the community calendar, so we chose monday.

B

So I just want to open the floor up here. If anyone has reservations, uh let us know uh otherwise. We are gonna, skip the next one on friday and then start a new series from monday, following from 4 19.

B

Louis, I think we didn't explicitly hear back from you.

G

Oh yeah, I was on vacation um yeah. I think this is fine. I think you know we'll have a couple of more missed days, because u.s kind of national holidays tend to fall on mondays right when there are three day weekends. So I think you know we'll probably that will cause us to have three or four more non-scheduled toc meetings a year, maybe a bit more than that, like uh I don't remember how many it's somewhere in the four to six range right, so we'll lose one and a half a quarter.

G

Interesting point.

B

Yeah it looked like even on fridays, we were going to lose because of the new google policy of every few fridays being off. Is that correct, yeah.

G

I mean that that that you know that that's this year, it's not durable, but yeah. I mean we can always revise this anyway so and if we need to schedule uh one-off toc meetings to compensate, we can do that too. So I think it's fine.

I

I

Is is this only tlc voting on the on the times or others.

G

I think feedback is open for everybody. um We didn't actually just vote. That was just more, um but certainly the toc members need to be available for the meeting.

G

So there's you know, there's that.

G

I

If you have some concern or points.

I

The only concern I have is that a lot of people join from the pacific time and beginning of week as the first stop at the like. The first thing is is not ideal, but if, if we cannot make anything work, then oh yeah, why don't we run the.

D

B

Okay, I am not a wizard here. How is this then.

B

Do you want to do that other toc members trying to understand what's the next steps here or do you want to just continue with, let's try it out and see what happens and change it.

J

Oh looks like you can call from google meat yeah.

G

I'm trying to figure out how to do it.

A

The people that are not coming now you're not going to get them in the polls, so it's skewed towards people that can make today.

J

Yeah, which are 24 people, so that's a good number. I.

A

Say we just try it for try it in like something on the um on the schedule to revisit. I don't know in four weeks or something.

B

All right, so I'm gonna, say approved uh is that, okay with everyone.

G

Approved for experiment: how about that.

B

Yeah, that's perfect.

C

Josh is not here as one of the tlc members. Do you think we should wait for him or I think.

B

We just talked with him yeah.

G

I can check with josh okay.

J

I think he proposed this by.

F

B

Yeah, I'm just gonna put like at least four meetings or something like that and see what happens.

B

Okay, cool, so can someone with calendaring powers, probably from google, do the changes.

G

uh I think that's you sven right.

A

But I don't own the I don't own. The working group calendar, I'm not an administrator.

C

I can I can do that.

J

Yeah, thank you shiva. I think I lost all my power when I changed jobs.

C

So can I then um just move it from next week to 4 19? If you all agree right.

E

C

B

All right, thank you thanks all right, john, the next one is your agenda item, so why don't you take it away.

K

um Yeah, so we've discussed this a couple times in the past. This is basically about the changes to the inbound forwarding with the local host cluster and matching kubernetes.

E

K

We got the feedback previously that we want to do this, but we are a bit worried about the impact it may have on upgrade and the feedback from qsc was yes, sorry, if that's too small, we should have some command that you know before every upgrade users run the upgrade check and we'll encode, all of or as much of like upgrade notices um in code rather than I expect users to read them, and the first would be checking for this. um So we have done that.

K

um You can kind of see the output here, it's a bit small, but it will show you all of your pods. That will be impacted. So if you run this before you upgrade, then assuming all goes well. You should be notified that anything's going to change and you can go fix those pods or if it was just a mistake and they're not actually broken, then ignore it. So that's pretty much it. I wanted to make sure that this is what we wanted and, if there's anything else we want to do before.

K

We enable this for one time.

B

So john, this is great, so we incorporated this as part of the upgrade check right. So this is good at this to ctl command.

B

Where do we document this workflow, or is this the first time when the release notes come out, we'll say hey before you upgrade run this command.

K

Yeah, so this is the first time, so what I'm planning is one for this specific change, I'm going to have a blog post, explaining it, and then that thing it will specifically say like yourself, you detect your impact and run pre-check.

K

um Two we'll mention it in the upgrade notes like run the pre-check command, and I think we can also put on the install and upgrade um documentation it's actually. It can be used for both. So it's meant to be not just upgrade check. We changed the name um since that picture, but it's a check for pre-install and pre-upgrade.

K

um So I was going to also update the main documentation to say like before, used to cut install run the check tool, and it should give you the green light to go out and install same with upgrades.

J

Okay, hey john, I think this is really good.

J

One question I have is: what's the thought process for you to have a dedicated command was: have it part of the analyzer command, because I do remember we kind of tell people you know if you need to upgrade issue, go ahead and run the analyzer command and see what only you got right. If we have newer version of api that may not be compatible with your environment, we would want you. So I assume this is an additional command and what's your thought, process of not having it as part of the analyzer.

K

Yeah, so I it's a good question um yeah, like you may note, this is using the analyzer stuff under the hood, both under the.

E

K

And the output, like the exact same output as the analyzer gives, I think we probably can consolidate them a bit.

D

So, in particular, the reason that we don't have this running as part of the analyzers today uh it's twofold one. It reads: data that the analyzer framework does not have access to analyzers are by definition config only they do not have runtime access to envoy, etc, and this particular operation does require that so things like ci cd would not be able to use this. The second is: we are trying to create a set of analysis like operations that are particular to upgrade.

D

So if you're running istio, one nine you're running istio, one nines analyzers, and so those analyzers are not going to include any problems with an upgrade to 110. The idea would be that you download istio cuddle 110 and run upgrade pre-check, which runs only commands that are relevant to your upgrade. Does that make sense.

J

Yeah, that makes sense, except I guess I was under the impression when you upgrade you download the new istiocado, exactly like you said, but run the analyzer on the new isrocado.

D

Yeah running all of analysis will will give you a lot more information than is necessarily relevant to your upgrades and also will not give you this particular check that john has written. um So that's why we have a particular command. It's just the subset of analyzers that are relevant to upgrade along with a few other runtime checks.

K

Yeah so mitch, I would say that, like today, you're absolutely right, the analyzer doesn't have access to this info um and in the continuous running analyzer. I certainly think it never should like. It should never execute the pods. But there's.

E

K

That the easter cuddle analyze command can't also include this right. It seems like I mean we have to think about the details, but we could probably combine them. You can add a flag, that's like dash dash include upgrade checks or whatever we want, but uh ultimately I'm sure we can find a way to make it reasonably be uh just one command.

D

Yeah, I'm very hesitant to have analyzers you do this amount of deep access into the cluster. This is meant to be a lightweight command. It's also. It needs to be able to ingest files only- and I don't see how that would be feasible for this. This.

K

Works well, it.

D

Doesn't need to be.

K

You need to have an option to run files on this right. We already have the and by default, do access the cluster. All right be pretty cool.

L

K

They don't have the cluster access right access.

H

Right can, I add, was one small comment here on this topic um in in practice. uh Yeah. I understand that how it is today, but uh there is a need to have more fine granularity of uh permissions and access control, starting with ability for users to to run analyze with access to an a space only and then with file only which is very important for csd, but also what what john mentioned with cluster admin or with mesh admin. So we need to to to kind of refine a bit uh the analyzer permission model.

H

So all three models are supported because uh in particular, where you have access to namespace only.

D

So right now that is supported. uh Analyzers can be run against a particular namespace only and no matter what model you're running them under they are read only so they don't do any port forwarding any pod exec. If you have read permissions to the kubernetes api server or to a file system that has your configs in it, you can run all the analyzers.

H

You know I meant you have for namespace. You should be able to also execute reports and what deeper analyze. If, if necessary, I mean it's too limiting to only have api server access. Think.

B

What the source of confusion here is, I feel like like there are too many sub commands or commands, and sometimes it's not clear when to run for a user, because I think there is a osteoctl verifying stall. Maybe that's going to get deprecated, I don't know, uh then this new one and then istio ctl analyze. I understand mitch your point that there are security boundaries where some commands you know fit very well.

B

So the ux here which is important is do you want to have commands with options where the security boundary gets expanded. Or do you want to have different commands all.

D

Together, this command is actually not the final form right. We're not using upgrade check we've combined it with the uh pre-check command, so that it's.

K

D

K

New yeah, but we've gone from having a set of commands to six commands, we're still like check analyze, verify install uh upgrade, install like there's a ton of commands. I I even couldn't figure it out, so I kind of agree. It's complex.

G

E

G

There's probably some desire to consolidate or rationalize the ux across that set of commands, but, but I think we're all in agreement that we we want a very clear definition of a command that is a pre-upgrade or pre-install as a use case, and it's perfectly fine for that to be well a command that is named differently than analyze analyzes, an incredibly generic name anyway. So we might want to think about that. But if the question here is, you know, should the toc approve this command? I think the answer is yes.

G

I think the question then going forward to the u.s working group is what should the rationalized set of commands and options? Those commands should be to make sure that those varying use cases are well understood by your users.

B

Louis, I I'm not totally sure if the answer to the first question is yes, especially if this is not an experimental command. Okay, is this an experimental one like? I don't want to give something to users and then take it away the next release? This is experimental.

B

Okay and uh istio city experimental is like experimental in other pieces of stu, so we can take it away and break things without uh without.

D

Backwards compatibility, that's correct. As a matter of fact, this command represents a backwards compatibility break with the previous pre-check command, which was also experimental.

G

Right, assuming that you had automated that right.

D

Had automated the pre-check.

G

I automated the call of it right.

K

Yeah not just the call but automated like parsing, the output, which is not standardized.

L

B

B

So and then I agree with louis actually I'm fine approving this uh with the caveat that we will revisit and understand. What's the gambit of commands look like and can we give a clearer instruction to the users.

G

Okay, yeah, I mean, I don't think we have an image correct me if I'm wrong as yet put enough rigor into trying to say formalize the output so that it could be consumed by automation, right that hasn't been a primary goal. Yet for a lot of this.

D

For all of the analyzers, that is a that is one of the primary goals. They have structured that.

G

Do we have the the feedback to establish that we've, but the automation people are building on top they're, providing us feedback saying that we feel like this is stable and is doing what they want.

D

We receive a lot of positive feedback around analyzers. I don't know that we have dug into that to understand how much of that is an automated consumption, how much of that is manual running yeah.

B

I so from my interaction with customers and also proud. You know for using the sto or github issues I think, most of the time it's running it getting the exit code printing it. I have not seen a lot of automation around consuming the json output or any other format about it. Okay,.

D

Now I want to be clear: we do consume our own structured output as well in the use of the analyte analysis messages in the status uh which we are hoping we'll move to beta in 111.

G

Okay, all right.

D

B

So what's the decision here is when you want to weigh in be heading in the direction that you like.

A

Sorry coming off mute um yeah, I I I'm fine with this. I don't have two strong opinions. Obviously you know we should try to consolidate commands as much as we can, but like the most important.

E

A

Is getting this out for 110, so people can actually see the see the errors and prevent them.

D

Yeah one thing to be aware of too: this is reporting on any use of alpha or deprecated. um What is it annotations in your install, which we do use by default in istio install? So one of the reasons for keeping this experimental is that we know we need to clean up our existing alpha and experimental et cetera features by default. This will so we're going to get some users who see this and are a little bit concerned, as they should be. Frankly,.

K

Mitch I I did not include that in the bureau.

K

D

So that was in my pr, yes and then I removed it.

D

K

Because it, I didn't think it was useful because forever pod, it reports, errors that are bogus, and I want users to see the real errors. So I figured we would wait until we actually made pretty useful um and not too noisy.

D

Okay, uh that is something we've gone back and forth on a number of times now, whether we want this enabled or like how we move forward, and the most recent decision that we had was that the best way to move forward is to set it up as experimental, where it's not going to be bothering users that are using our normal commands uh and then iterate and improve from there.

K

I I yeah I mean this is going to the weeds, but I don't see how it's useful to report an error. When there is none, I mean we, we literally inject these annotations onto every pod, so then to go around and warn them that we injected an annotation is kind of. Not it just doesn't make sense to me so, but we're probably going too far into the weeds here.

B

Yeah, so for me what I have heard and then feel free to pitch in if you still have reservations about this, looks like the experimental upgrade pre-check command is approved. uh We want to make sure for 110. We are highlighting this in all the forums and documentation, blogs, release, notes we have and then the last thing is uh after 110 we want to have the ux working group. Look at the various htoc till commands come up with a better ux with a consolidated workflow.

J

No, this is great yeah. I I agree with this and john thank you, for you know having a program away for user to detect this, and I, like also you remove the the bonus warnings. I'd rather not see them. So thank you.

G

So, john, just just to be clear in the behavior um when we provide the the warning is telling users that the change that the they are going to be affected by a change in default behavior and that they should correct right. That is what we're doing. It is not that we're trying to do anything more refined than that.

K

uh Yeah, so it will detect where this behavior will change and then they can go to that link for the details and get you know explanation of what's going on. Okay, all right.

B

So yeah good, sir.

A

Oh, I was just gonna, it's getting probably too much from the weeds, but I john on the actual error message. I don't know if the what's in here is the current one, but ideally there would be a like. Don't just report the error, but tell the user what to do so.

A

We should say something like you know. If this is okay, you don't need to do anything. If you need this to be exposed. Do this.

K

Yeah yeah the error. I changed the air a bit. I don't know if it fully does that, because it was already getting kind of long, but it gives you, I think, a summary and then the intent, or at least my expectation is that they would then go to the ist code, which uh I don't think you can see down here at the bottom.

K

It says like if there's warnings that says go to this link to find details and then on that page, we'll have the full estio.io doc that can be large and have diagrams and big yaml examples and whatnot to explain. What's due yeah.

A

I would it would be good if we could put that link in the actual message rather than having them have an indirection of like look at the.

I

A

Copy the code go, look at some other link right, the more we can do it. You know.

E

A

This and put it and it'll tell you how to fix it, the easier it is for users, so.

J

Yeah I almost yeah.

G

Yeah mitch, random suggestion is, maybe we could go and convert the ist codes into a brutally short link.

D

So that's just a flag on the command.

D

You can have it output, documentation, links the problem with with that flag is that it makes the output exceedingly long, because the url is quite long as well, so it's off by default, but it's available yeah.

G

I'm wondering if we could find a shortly convention that might make it tolerable.

D

uh When you're dealing with the terminal and we're talking like you know, 240 characters or less that's, uh not a lot of urls are going to fit there, especially not when you need to have the all of the ist codes, etc.

D

I mean if we wanted to do urls only we could do that, but then there's no error message.

A

I think louie was suggesting we have like a shortened url.

G

That's something very awesome: if you could get the ist code to be almost the same length as the url, then that might be a worthwhile trade. The isd code would become a url.

G

It might be a trick anyway. Reads: we can move on.

J

Yeah I want to make one comment before we move, uh I mean I certainly agree. We should look into the links. um I almost feel the warning is a little bit too weak. I think it's like an action.

J

They actually have to do something correct me if I'm wrong john, if they take the walling uh and not do anything, their application will be broken.

K

uh Well, it may be broken, maybe they just expose the port on their service and they don't actually care um and no one actually calls it. Okay, so.

J

If yeah, their message is not the.

K

Actual error message in the in the final command.

I

B

I

Though, if it is called or not right, like we, I mean we, we have metrics, we we know it, we don't have to guess wow.

B

We won't have access to it in most of the environments.

G

Yeah, so certainly if the plot, the port number was like 80 or 443 right, there's a very, very high probability, it's going to be called, but I think users can make that assessment right as long as the documentation tells them um they can always try it and if it breaks, then you know they were warned, ah but they prefer not. They should go, read the documentation. I think.

J

Yeah, we're just thinking change that one into action, because we really need them to assess this and do something um to evaluate whether their app going to be broken or not.

B

Here is my suggestion, lynn, uh so john, why don't you link the pr here and lynn and anyone else in the toc? Otherwise we can go and comment on it and see if the error messages or the warning is suitable. I think that can be done on the pr side instead of deciding here how about that? Yes, yeah all right.

D

Well, I, since we're on the topic, I'd like to raise the issue of this alpha analyzer and ask for some kind of gridlock breakage. This is something that we added to one nine um and because of it, the verbosity in analyze, we decided to roll it back. So we didn't want these analysis messages going out. The analyze command is not experimental, and so there was some concern that we're we're telling users we're a little bit more noisy than we should be there.

D

uh The decision, then, was to add it to 110 disabled by default, which we've done well. We did, and now it sounds like it's been pulled back from 110 as well, um who should be deciding how we move forward on this feature and how do we ensure, because we're kind of stuck on it at this point.

G

So this is the the you're using an alpha feature and you should or an experimental feature, and you should probably.

D

Stop yes, this was part of the the initiative to up or improve upgrades in istio, one nine.

M

My opinion is that we should never flag as a warning something that istio injected itself, so that means anything on the side. Car probably should be allow listed.

D

I totally agree with you ed it's. I think I don't think it's a question about that, um because I think we have agreement there that we shouldn't be. You know, warning users about things that are set by default. The question is more: how to move forward. Do we enable the command, knowing that we're using some alpha features by default? Or do we wait to build this command until we are comfortable that we're not using any alpha features by.

L

Default because that'll be fine, that'll be a really long time. Is there a way to have.

G

An option to have the roll-up the instance the separate levels of detail right so right if you're, using an alpha, annotation or an alpha label or some other funky edge legacy stuff. The default behavior, if we think it's noisy, is that we tell you, you have three of these things and then run with this flag.

D

I don't think it's noisy and the the concern around noise is about how many uh alerts you're going to get as much as that you're going to get alerts on things that we set for you by default. You're, going to be warned against things that we've done rather than things that the user has done so.

B

I I think the path forward, then, is reasonably straightforward here, right for things that we have to do to make istio functional, we have to allow them or whitelist them like ed, was saying until we don't no longer depend on them, and even after that, if we find that the verbosity of the output is high, we can do some level of collapsing and then expanding. We are more detailed, cli arts right.

D

So that sounds mirage like introducing sort of a third tier of life cycle. There's, like you know, exp or fourth, fifth, experimental alpha, beta, ga or stable, and then there's also this sort of not yet beta, but also we're not going to complain about it.

D

We can call it special, we can hide it and basically, we wouldn't report on those particular things. Sadly, that's where we are. Might it not be better just to say if we're we're using these by default and have been using them by default for a long time they are defacto beta.

B

Unless we plan to remove them because.

D

I know there are few.

B

Things that we were discussing that accidentally happened to be used by sto, but we will remove them once better apis land like, for example, telemetry v2, using the onward filters right. There might be better ways to get rid of it, so we might never make onward filters data.

A

All right, I I don't think we should mark something data just because we use it. I don't think that's the right way, yeah. I think the right path is but like, let's allow list the things that we're using and that actually is a nice checklist for us of things. We need to resolve right in the future. Absolutely all the stuff that we're we're sort of cheating.

D

Okay, that I think that sounds reasonable and I think it's feasible from a technology perspective. Can we document that decision here so that we don't run the risk it's out of 11.

L

B

So make an explicit allow list of alpha experimental features used by sdo.

B

One on rest and then basically, if the output is verbose, we can figure out other ways of organizing it.

J

Yeah michi, thank you for raising this. I think this is very useful for our user, because there's just so many warnings today, it's it's hard to process.

B

All right, this is good, uh any other topics we have for discussion today.

J

So savannah you want to bring up the message you sent.

A

Oh sure, oh sorry, I think, john uh well. I guess that everyone, so should we make it an error instead of a warning. So john uh john in the chat said that basically, the actual.

M

A

Warning here is the pod is listening on that port and there's a service pointing at it, which means like. I thought it was just there's a pod listening and you know it's listening on localhost, but so what? If there's no service, but this is actually the ones where there is a service. So that's not.

E

Just a warning: that's.

A

An error like it's gonna break, so we should make it an error.

B

Sounds good to me, I mean this is not an error when it is.

J

I I think we generate too many warnings and people tends to ignore warnings. So if it's an error, I'd rather be arrow.

F

All right consider it done. I sent the pr to make it an error. Thank you that.

L

Was quick? Do you see your review.

B

uh I I would say john, when you start writing, release, notes and stuff like that, for this feel free to include us to get early feedback. If you know this is a breaking change already, we want to make sure it's front and center.

K

Yep I plan to make a lot of noise about this. um I do have a pr for a blog already and then we'll probably link to it or copy it in the release, notes or some something like that. So it's in.

E

A lot of places.

K

Sounds good awesome and we should market it not as we're breaking everyone, but we're making a great usability and security improvement.

K

We're talking a lot about the negative here, but we should present it in the blog as as positive. I think.

G

Yes, people catch their unintended consequences.

L

B

K

uh We technically don't have it as cve. I was thinking as a way to give more visibility. It might be useful to do a. What do we call a security advisory.

L

K

B

We can talk about it, john in the product security working room.

B

Let's do that first and then bring it, but yeah lots of noise. I think I'm in agreement with that.

C

B

uh Are there any other topics or can we get 16 minutes back.

A

All right short, I had put something on the top about pushing the release back week. I don't know if that's still valid.

C

uh I would like josh to be here. It was discussed in the pswg group, so um I'm still not here how to bring these things up. So I would like to wait. I think we.

G

Have proved it, it was raised on the toc channel in slack.

G

And it was approved there. I think so.

H

Yeah you're good to go.

C

Okay, so then it's fi for everyone here that we are pushing the date by a week.

K

B

Okay, can we just write it down? What are we pushing to.

C

B

B

Is that it 418.

J

Yep, it's may right. Oh it's april, I thought it's me yeah me. Five.

A

Okay, hey may 18.

J

Yeah we have one more mask.

B

Yeah got it perfect.

B

Believe me, working on ipv6 is not easy on dual stack. That listener has been hounding me the whole day.

B

That's why it keeps coming back in my face: vlog anyways.

B

Yeah, let's see anything else, folks, if not uh we'll meet on the monday and we'll have a whole week without seeing each other. So that's good! I guess.

C

Happy friday enjoy this, everyone.

L

All right all right, thanks, bye,.