Kubernetes SIG Node, 2 May 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Kubernetes SIG Node 20230502

Description

SIG Node weekly meeting. Agenda and notes: https://docs.google.com/document/d/1Ne57gvidMEWXR70OxxnRkYquAoMpt56o75oZtg-OeBg/edit#heading=h.adoto8roitwq

GMT20230502-170521_Recording_2560x1284.mp4

A

Hello, hello, it's uh May, 2nd 2023 uh signal weekly meeting. Welcome everybody um I want to I Really, um restart this tradition of looking at active PRS. um We ended 127 with 200 active PRS, and now we grew uh 40 more so now we're at 241 um and uh yeah trend. Is uh we see more and more PR's coming in because floodgates are open. uh So if you have energy uh to review or proof, please do so um and movie merge in the beginning of cycle uh safer.

A

We will be because we more likely will find some bugs anyway, if you're interested what was happening- click these links. uh They will lead you to the specific queries for this week uh with that I want to switch to planning and Reacher um Raven uh agreed to get a uh to do a retro Raven I made a co-host. If you want to share your screen, you can do that.

B

A

B

You can find the share button.

B

Okay, uh can you all see the uh see my screen and I was like? Can you all hear me? Yes,.

A

B

Okay, cool um sure, let me fix my audio okay.

B

All right, so here is a table of uh uh the Caps that we tracked for 127 and uh also whether they have merged. So during 127 we have a 19 uh caps tracked and out of them, we have 13 emerged uh I used to track the status uh in age of the cap issued to determine whether they have been emerged or not uh yeah.

B

So uh let me just to quickly go through uh the merge ones, so uh kublitz, parallel image pools that it's merged and uh uh uh support memory, Qs with secret V2, is merged, uh and this is a big one in place.

B

uh Pod vertigo scaling it's merged, uh Dynamic resource education and then uh support for user namespaces and then uh extend pod resources to include resources for Dra, and also uh we have a couple of caps graduating to Beta in 127, uh open, Telemetry, tracing and uh event tit plaque, uh yeah and also we have a a couple of caps graduating to stable uh downward API uh sitcom, uh topology manager, uh jrpc probe and uh configurable gray, Square Reader, probes, yeah and here are some history data you can see.

B

These are the trends the number of Track Captain merged capped in previous cycos and in 127 we have a a slightly more tabs tracked compared to 26 and 25, and more caps emerged yeah. So that is these are all the uh statistics for 127 cycle.

B

uh With that we can uh discuss uh things that went well things that uh could have gone better and I also pasted, since that could have gone better from 126 ritual. So we can discuss them uh afterwards uh and see if we have improved on any of those so yeah. We can uh uh start by start with things that went well.

B

I guess I will mention that uh in place called vertigo scaling is finally in.

C

D

E

Okay: okay, good good progress on graduations.

B

Also from statistics we have more uh caps attract emerged. I will see that as uh saying that goes well,.

A

F

I think overall.

A

Planning went uh way better this time.

A

We had like all the right comments in the right places and uh um we did everything on time. I think uh maybe a couple things went through exception, but it wasn't uh complicated.

B

All right um anything else.

B

Okay, uh if not, we can uh go to the things uh could have gone better part.

A

Yeah I think uh immersed in place very late, so we found so many regressions and so many discussions like because we fixed about whether we don't fix a bug. Foreign.

A

Bugs that needed to be fixed.

B

Also sugar, you mentioned that we um from the last temperature we needed Mark's help for cab planning uh to apply milestones for caps, uh I, guess that is still true for Windows 7.. We need someone to apply the masternodes.

A

No, we, we could have two more chairs, so now we can. We have more people on our stones.

C

Pretty good this time, do you apply milestone I'm, so practicing applied. My also sometimes the overdue, even.

A

Foreign needs to go into what went well.

E

And I think maybe worth mentioning that we we thought it would be better to not take sidecars, because we got one big change in with in place right.

E

C

That's good path right, so we made yeah yeah good part exactly well, we cautiously not to take a additional complexity, because we single is not ready, especially previous. A lot already introduced a lot of new changes, complexity and the incoming one also is complicated.

A

One thing: I I think uh we tracked all the bugs for in place upgrade but I don't see, um I mean I, don't see a lot of work happening right now to fix it. Beyond Windows, support, I, think I mean windows thanks.

A

G

A podcast, open or we'll have some Windows support for that.

A

Yeah that that is true, but uh we discovered that we need a major refactoring after this cap to improve how we do in place upgrade and I don't see much of a work happening, so it may stay, as is.

C

But at the other hand, I do think what we did a good job to quickly make decision not use because Windows to block that progress right. So so that's why we at least count merge that change a little bit earlier. Otherwise we will last minutes to merge and because I, basically, the the white implants upgrade, cannot merge before so many times. We always have the new feature block that led to PR that change.

C

So so this time we basically said: okay, if it's not the fundamental, so we could uh a layer down scope a little bit for that, for example, for Windows, then next release can fix Windows anyways Alpha, so we can detect the problem not last minute. It has a problem. No.

A

Yeah I want also to mention that the Brooks and the long Cooper twice uh during this release, uh one with some storage improvements and one with In-Place upgrade. Now we have press admits for in place for Standalone kublet, but we don't have a regular tests, so maybe that can be improved as well.

B

Actually, you imagine twice one for uh want to buy and place the other time modes for some.

A

Storage cap I, don't remember which one.

A

Why all bathing coming from me like three items and all of them named with my name, who.

G

C

So, like high requirement high standard s personal fail, we did great this time compared to previous release. So all this thing was good thing.

G

Okay, so I I'd like to like point out I, don't know it's, maybe it's like usual practice, but anyway we should think uh if we can change it. So the most of reviews and approvals, actually I was done like in in the last last 10 days before the like, before the code freeze, so I don't know if it makes sense to maybe like spread the the caps and implementation like through the cycle more like smoothly to like, because it's like it happens like every every release.

G

So if we can think about it, if we can do something about it, that would be probably the the secret benefit from that.

C

So I agree with you: yeah.

G

I'm, just thinking like if we have caps that are like already kind of ready, can we just like when we uh like discuss the planning? Can we kind of schedule them a bit earlier in the cycle and make him make them like reviewed and maybe even merged uh earlier? That would kind of spread the load.

E

I think one thing we can do is write like once we're done with planning we'll have a list right and then we can say: okay, we, as the reviewer approval Community, will try to focus on these two caps this week right and that way like every week. When we come back, we see. Oh, we can check. Did we make good progress? Did we review? Do we upload? Did we approve? What we said we were gonna approve. uh Is something blocking so that way?

E

Okay, we take two every week and hopefully you know we make progress over time rather than the last 10 days. As we pointed out.

A

Yeah, but you know another only about caps. uh We also had the regulator.

H

A

I remember last few days: Paco picks like three or four merge conflicts and uh flakes I fixed two, and then somebody else fixed the couple so yeah there is. There is problem with many caps moving uh in the same time around this time,.

C

Actually, a personal feel this time. Cap review actually is better than three wheels. Actually is the implementation code review actually uh last a couple days and the block right because we keep finding new problem, but the camera actually at least the I personally, a little bit of spread, not the last minute. I did a poor job, I apologize, but this time I really on purpose to to process those cap a little bit earlier. So but that's still at the end, still there's the Dynamics changes right.

C

The people so kind of actually a little bit easier but cold, because there's many things, it's not the uh self-contained by signal and also not the self-content, the bad review or approval and the author. There are like test infrastructure pre-submit, all those kind of things, so that's kind of either more Dynamics changes there. So the code actually at the end, I even don't know which one I should take a look. I I, I I I, admit to look at the last minutes code that I only unlocking the minimum time. I have to admit yeah.

C

So I, don't know how to change that. One for code review.

G

Well, I would propose to like the like earlier stage of release cycle, to concentrate on the caps and implementations that didn't pass uh like they like didn't go into the previous release, but but were planned for it so that they, they probably are like more ready to be reviewed and approved at the early stages, because the the rest of the people are busy like with their caps and implementations and they they can be like not ready yet, but those people who kind of missed deadline last time. So they they may be more and more ready.

I

I can I can add one. One thing, I think also went, went well in this cycle, which is uh I, think the testing caught much more issues before uh like after merge compared to uh prior, when we were kind of releasing things not seeing things actually in production, so like, for example, for we made a couple changes in pod life cycle due to kind of some job controller cap, um and we found kind of two issues with it after we merged it, but one was found would be a testing and one another.

I

One, fine, like earlier report very early on, we were able to get those fixed before uh you know. 127 was released so um I think that's improved a lot.

I

C

So you suggest us, because we do have some cap uh submit the merged for previous release, used to suggest us, but let's implementation, maybe not finally go through and there's the pr waiting. So we should focus on code review for those, so I'd emerge the couple before.

G

Yeah yeah, a good example is sidecars, so it's missed previous deadlines, so we can.

C

G

C

Yes, yeah I just want to yeah make sure. Let me.

B

Implementation PRS that have like a grade upon design. First.

C

We can help with spreading the code of implementation review as early as possible. Yeah.

E

Yeah and I think we kind of did that with saying that: okay, we're gonna, try and get in place in right. So sidecar is now the.

H

E

Next thing that we try to get in- and there are six items here which will be at the top of the list for 128.

C

Yeah but anyway,.

C

We systematically follow up this suggestion. We did sometimes the team Board app yeah thanks.

A

Do you want to review what uh we taught in 1.6 Etc to see if things improved and then uh switch to applying.

B

Yeah, so uh these are the uh things that could have gone better from 126 retro, uh so I think we've talked about the first one uh Native Mark. We in 126 we needed someone to help for uh care planning to apply the milestones and we did a better job this time and also uh last time, I think from America or no. We didn't enforce uh soft freeze, uh I I, don't know if that's still the case for 127.

A

Yeah we didn't enforce it either. I think in place went way past the soft fees, if you would Define one foreign.

B

Yeah, and also uh also from 126, we had a more test of failures uh and uh well I. Think David just mentioned that in 127 we have desolator spotted those test. Affiliates actually helped us uh uh catch issues, instead of catching them. After the release.

A

Yeah I think it's improved here. Yeah.

B

uh And also last time, uh dra in the in place were two large uh caps and uh people had people um uh had a hard time to find bandwidth uh to review the the cabs.

B

You know this time we got in place in.

A

Yeah I think we noticed Boston to cities, yeah complaining about it in 126, and we found time 1.7.

B

From Derek last time, uh struggle on budget for external, facing changes uh doing better with internal bid things.

B

B

People reached out discussed caps hard to keep all details of all caps, uh expand the list of people in domains to make a Proverbs on caps. Yeah I think we. Now we have more uh chairs, I, think down just Dimension I think we are doing a better job on this front as well. In 127.

J

A

I think it's so positive right, so we still have a lot of things to do better, but uh so many things out of 1.6.

B

Yep, um okay, so I guess uh with that we can conclude 127 retro thanks everyone for the discussion.

A

Now, do you want to take over.

E

Yeah, let me see if my sharing is still gonna work. Sergey, usually I lose audio. When I do that, can you give me, can you give me host.

A

You should have it. Oh.

E

Okay, all right: okay, foreign.

A

Yeah now we want to switch into 128 preparing. So typically we have a table and we look at what uh we have in this table and see if we can assign people to that, and uh somebody will be working review and approving things do.

E

You see, do you see my screen? Yes, use the machine. Okay, awesome, so I think what I'll do. First is I'll just pick items from uh Reuben's uh list that didn't make it and the first one is sidecars and we'll start inserting those at the top here or move them here. I just made a copy of what was there from last time. Let's see if sidecar is already in this list, I see Keystone but I don't see the rename to sidecars.

A

Yeah I don't see it.

E

A

Just create a new.

E

Yeah, we'll just create a new item there yeah at the top.

E

All right so sidecars uh priority.

E

Bye, sorry, all right, the stage Alpha we have authoris Sergey, okay, uh who else is working on it? If you can add other folks names.

A

Yes, okay I will let it do it offline. um Okay, okay, no worries, okay, so from approvers and reviewers perspective uh we have Team uh for Tim Hawkins for API and Derek is listed as main approval from signode.

A

E

I can do that yeah.

E

Okay, thank you, but this one is good. uh Let's take a look at the next one, so there is node memory, swaps, support, uh Hershel or Ryan. Do you guys want to talk to it.

A

Last time we wanted to do alpha 2, uh because we only have like a little bit of resources. uh These days, I understand that more people want to participate, so we can even go to Beta and there is an open, PR uh movie.

E

Yeah so I think, like harshal and like we have additional folks from red hat, that they have time to work on it and they have I think they have a cap update. So.

F

D

E

That they are planning to work on it for sure yeah.

D

We're proposing a beta, 1 and beta 2 um for the features.

E

E

Er on this one, okay.

J

Oh yeah, no yeah, it's.

E

Fine yeah, we can both do it yeah all right, yeah.

E

All right: okay, uh let's go to the next one; uh fine-grained supplemental groups, control.

E

So we probably want to make progress and I know Sergey. This is kind of dependent on the on agreeing.

J

E

The CRI API as well right, I think it's your IPM.

A

Remote storage.

A

So I don't think we need to agree on that. We need to agree on defaults and how to migrate. People.

E

Okay, all right I forget the name of the author, but I think we can populate that on side.

E

uh Sergey you were already looking at it. Can I put you down as uh reviewers. Oh.

A

Please thank you.

E

I lied myself and I know: Tim was covering the approval, so maybe we can continue that.

E

Okay uh part conditions around readiness.

E

I think this is deep saitama if I remember correctly, you see on the call.

E

You can check with them if he plans to work on it.

E

Then there's the uh app armor support, um not sure do you know Sergey where it got stuck.

A

It goes stuck on cap for writing, so.

A

Cap- and uh it has couple comments, so somebody needs to go through the comments. uh Get everything written down. um One question we had is uh how to re-admit our ports. If, let's say apartment was disabled on a node like how does it uh affect workload, do we need to re-admit or support, so we need to just run as usual, but, like then, don't have a parameter on this no ports. So this kind of questions that we need to answer besides, that I think everything else is approved and API is there. Everything is there.

E

Is Tim still driving it.

C

Team is not he.

E

C

To find the uh I think he finds some owner, but the company, the owner, I, didn't see the available.

C

I think I've been chasing this one, um but the problem is I, never find the owner new owner is available, I think he he mentioned to me. He find a new another new owner, so we.

E

C

C

Did he agree to supervise and the review and all those kind of other work, but they want to have the new owner? So that's why he tries to sign the class okay reviewer. You can put a team of player there, okay and approve, where you can put me yeah. Okay,.

E

A

So if anybody wants to do an exercise of GN feature, uh please join this party. It shouldn't be too hard.

E

All right, uh what's the last okay graduate the cubelet Pod resources, end point uh Swati: uh do you know about that? One I.

K

I think I can help here. Basically, the last thing which missed was the proper graduation. There is a PR which just moves the feature gate to GA and locks.

K

Do it the reason why it missed because the last fix was measured last day, so the the re, the last last change to actually graduate to GA should just miss the train because of that so long story, short I, think the work to do that is minimal and by minimal I mean merged at PR, which is already approved and looks good to me when I I hold that for because I wanted to have this conversation and update the cap to replace 27 to 28.

K

I. Think that's literally what I mean I can remove the old right now. If the Sig agrees to that and it's merged and then I need to fix the captain to reflect the updates, whatever you folks believe, it's best.

E

Sounds good thanks, Francesco's I put you down like I put swathi as uh reviewer if Kevin can approve or if he can't I can I can help there.

E

K

E

You yeah so I think we got all of that done uh now, we'll just review the previous list and see if something needs to move forward or it can be punted so I. Think first, one here is the c groups: memory qos uh David: do you have any additional work planned here? Do you think we can do anything at extra or we need to sit on beta for a release to get feedback.

I

um I think there's some some testing and uh benchmarking that we were discussing I, don't know you're going to call.

L

L

Benchmarking, uh we'll do some benchmarking tests and try to see uh what can we do to push it to uh GA? Eventually, okay,.

E

Okay, yeah all right I'll just say it stays beta and will add Alpha.

L

It's an alpha right now. It needs to graduate to Beta.

E

uh Do we want to Target moving to Beta, or we want to get the testing and benchmarks first before we attempt to Beta.

L

uh We'll do the testing and benchmarks first.

F

I

F

I

Maybe we can, we can aim for paid like, depending on the outcome of the benchmarks and.

E

Testing and so forth, yeah, we can say it's going for beta next and yeah, we'll see how often like that yeah. Let's.

C

Try try better, but.

A

C

Can stay on our fan later if we decide not good.

K

Exactly yeah, okay,.

E

All right, uh cubelet plugin model based of dra owners, Marlo and.

E

C

Yeah we'll keep working on it.

E

Okay, all right.

E

So the goal is still to get a kept, merge right.

C

E

All right, uh juven anything remaining on the image pull or we are done, I think right. Yeah.

B

Your word John uh in this um cycle, I'll, probably just to add some more tests and graduate to Beta, are yeah, so should be quite straightforward. There.

E

Yeah, okay, all right.

E

The next one is extend the power resources API to include dra uh Francesco. uh You want.

J

To talk to this yeah I believe this got merged last release. If I understand correctly.

E

Do we want to move to Beta or it stays Alpha.

J

I am not sure, I think the owners did not express that they want to move it to Beta. Yet you just graduated to Alpha. So probably the.

E

Answer is no okay, I'll just say no here, and then we can remove it later unless something this, if do we expect to add any additional tests or anything.

J

I am not aware: okay.

E

All right, okay, so we covered this one already: uh okay, I'll just copy this data, the owner up there from here uh topology manager, GA, is done right, yeah,.

J

That's when that one is done as well right.

E

Then cubelet power resources GA, we just talked about it.

E

A downward API each page is beta since I think Sasha finished this.

A

E

Yeah, this is done in place resources what's next here, uh do we want to add tests, windows.

A

C

Yeah and as I think the when they come back to say uh actually Windows face and also all those test tests that we want to yeah. We could change it, but the sofa looks like stay on Alpha for.

I

Yeah I know there's a kind of list of issues we've as we were reviewing and so forth. There's like a list of issues that are tagged within place. That I think we wanted to address before beta.

E

It makes sense yeah we'll just uh continue on the bug, fixes and test it yeah all right. So the next one is split, standard out standard uh log stream. I know we don't hear back from the author. We need to check again and see if they are interested in taking it forward.

A

Think of an implementation was like PR was.

E

There yeah the pr was there and I think the pr was very big. Some comments were left, I'm, not sure. If they were addressed or not so I guess we can check again or maybe see if someone else is willing to take over the pr.

E

E

Ad uh I think this is merged right. The status host type is part field, so yeah to be done. Yeah more granular, probes, Mike Brown, the.

M

The codes there we address the performance questions. This is more of a do. We want to do this by itself, or should we do it in a work group I think the surgery was putting together for a bunch of changes to the probes.

E

Okay, but I think this is a new net new feature anyways right, so if we want to pursue it, uh uh do you have the bandwidth to work on it, I guess as a question.

G

M

The code goes right.

L

E

Okay, all right so I'll just say yes and then we'll begin yeah, so Danielle you were listed as a reviewer. Are you still? Okay with that.

K

That was good to me.

E

Okay, awesome thanks.

E

Anyone Don, you want to be an approver or.

C

Okay, yeah, okay,.

A

We might want to do. Antonio is a program from Sig networking because we had collability issue recently. It was open, sockets.

A

E

All right, uh so this one is folded into sidecars qos resources uh um Sasha, and are you on the call you guys are gonna continue working on it right, I know like.

H

Absolutely yeah absolutely yeah. We really want to get that in yep.

G

Can can we move it up like because it's like it's? It's means that it lights out.

E

Yeah my attention.

G

Would be added to yeah.

E

Okay, I'll move it up a bit here, maybe yeah.

D

E

Yeah I, like Tim Hawkins here as well, because I know he was doing.

D

J

E

It's the next one part German uh termination, grace period GA. This is done right, Ryan item I, remember, yeah, cheer, PC probes is done.

E

uh Cri starts uh David or Peter. You guys Wanna Give, an update.

I

Yeah for this one I think uh we want to continue with some more work on the runtime side and then I think focus on testing uh setting up test jobs. For this um sorry.

A

Sorry, you also do windows, part of it.

I

Yes, yes, there's also some folks working out from the windows side, so making sure we have test coverage there as well.

F

F

Yeah I'd like to keep pushing for beta, but uh because it's reliant on support on.

D

The run times, it's not you know we're not a shoe-in for beta at this round, but I think it would be good to, like you know, say we're working on it towards that.

E

Sounds good thanks, so CRI image pull with progress notification uh is anyone on the call uh talked with this one. I know there was interest we didn't hear from the author uh during 127.

H

I'll open Galaxy about it.

E

Okay, all right thanks thanks Sasha.

E

Okay, so this one we covered.

E

Add node local plug-in support, I'm assuming same status.

E

Do we feel we have enough I wanted to raise? Actually there are a couple more I know at the bottom of the list. So one is the username space support, so I know this is being worked on because I attended a call with cigarth, along with Giuseppe and Rodrigo on this topic, so I'll just move it here.

E

So basically, the call was about figuring out how username spaces intersect, with uh with the Pod security policies and uh Rodrigo and Giuseppe, are planning to work on the state. Full support.

E

Anyone wants to be a reviewer on this one.

A

uh You can put me yeah and Michael also.

C

We also need a sick, also.

E

Yeah say: go outside I, think Jordan and David eats yeah. One of them should be able to oh.

C

Yeah but the children I know this. This quota will be super busy.

C

E

Yeah I I I'll go to one of them because they said they'll review it so either Jordan or David cool thanks.

E

And one more that came up and Mike Brown: this is our ensure secret pool images. What do you think wanna take a.

M

Stability Paco's still here he is volunteering to do the storage request um the course the code, the coach there, except for the storage, but I, would like to get in front of cigno to talk about an alternative authentication for images. I think we need to move key ring support um from kubelet down into the Container runtimes, so that we're not passing Secrets see you know off across the CRI wire. Instead, I think we should be passing cash.

M

um You know, policies and, and or you know, the user that we're supposed to be pulling for the key ring um that makes more sense right. I can do a detailed mock-up for it.

E

Yeah I think Mike. If you can, if you can uh do an update.

E

uh Just everyone remember where we were at when we last left it all right, yeah.

M

I mean this will work and it will improve the security and it has. It is still the oldest feature. We you.

N

M

Issue we've had in kubernetes, but maybe there's maybe we need to step back um unless unless you want to do it in two steps.

E

Yeah I think I mean I I want to I want to see if we can make at least some minimal progress here right like better than the status queue and see.

M

Yeah, let me get a hold of Paco and and see. If he's still volunteering to do the storage part I think he had done done some work on it. Okay,.

E

Sounds good yeah all right uh should we go through through these, maybe uh node local I, don't know we'll have to check. Add ex execution, node Affinity manager to support. Does anyone know what that one's about.

E

Container Notifier.

A

If this is active,.

E

Okay, uh fine grain, cubelet, API authorization.

E

uh Maybe circuit the two of us can make a pass at these and then, if anything is active, we can bring it back next week.

A

Yeah I want to shout out a couple: things that have been worked on is versus Dynamic, uh not resizing. uh So cap.

J

A

It's new one: um okay, 953, um it's about resizing uh CPU and memory requests and limits uh dynamically for the node. Okay,.

E

Okay, I'll just add it here.

C

This Dynamic recessing any difference from the less doing, or this is the um special purpose. One.

H

It's about it's about the situation when you have uh ISO memory added to remove it or CPU goes online offline, so theoretically doable, but it might open kind of forms in some scenarios.

N

A

Okay, so it's a beacon, like it's small in uh semantic size but uh on the implementation side is huge, so design will be. A discussion will take a while. So let's see if it will make 1.8.

E

Okay- maybe we just go with this just to at least first get the agreement on a cap right and then take it from there.

E

Who's the author on this one Sasha, is at your team.

H

No, it's some somebody I think from India from some field operators: okay, all right, I, I, just looked at this camera, okay,.

E

Do you want to be a reviewer on this one Sasha.

H

Yeah, why not? Okay! Thank you.

E

A

There's another one: uh it's about um uh intelligent sleep as a post start uh pre-stop hook.

A

Please stop hook. Intelligence nip as a built-in action for pre-stock hook.

E

E

We will have a link to the cap.

E

This is net new as well: okay, yeah.

A

And I know there is interest I remember already. Two people asked for environment variables from file um this one and another one. Is um it's not formulated as a cap? Yet so there is no link. There are just issues on KK and another. One was uh uh back off, uh backup timeout to be configurable.

A

I can pin people who are interested if they want to.

E

Yeah, okay, I think maybe yeah.

A

E

If you can do that, that'll be helpful and then we'll see if it is actually uh bandwidth and capacity to take on more absolutely. Thank you all right, I think. That's a good first pass like folks. Please uh update with comments here uh if something changes with regards to your ability to work on it during 128, so we can change things accordingly. We'll make passes over this again before you finalize it.

J

A

You, okay, uh we had another item, but Lucy agreed to move it forward. We still have five minutes.

N

Yeah, let's just skip it for now, it's uh so that everyone can go, and we can talk about it next week. It's not particularly urgent anyway,.

A

To look at the pr introducing new events, it will be great. Please please.

E

N

A

Thank you. Everybody.

E

Thanks bye now, foreign.