Kubernetes SIG Auth, 4 Jan 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Sig-Auth Bi-Weekly Meeting for 20230104

Description

Sig-Auth Bi-weekly Meeting for 20230104

A

All right, uh hello, everyone: this is the January 4th uh meeting of Sig auth, happy New Years. It is now 2023. wow, um let's get started Mo are you around today, yep I'm, here, uh I, hope, you're, feeling, okay,.

B

C

Right, just I just need like a one week nap and then I'll be good.

A

Yeah, well, all right so any plan, so any plans for our forwarding conditional requests. My little attitude reviews, so I am some okay, I I guess I will answer this chance. It is my camera I am after implementing this um I.

A

Was a little bit.

A

I I was not actually uh I'm, still not sold on how to do. Caching for this um in a way, that's reasonable, I think that uh so the general idea of this was like if we did want to have additional headers added by like an auth proxy or something such as um you know, like IP, based context added to user info for use and authorization.

A

um uh This would allow additional headers to be transferred through to the authenticator um and like storage. In, like the authentication context, um the motivation for This was um gcp has a feature called context. Aware access where you can do your context, you can do authorization based on you know.

A

uh Yeah IP ranges, I think is the easiest easiest example, but they can also be like on like the VPC or virtual Network um request originated that it could be like you know, um even like something like a trusted trusted device or something, um and we needed a way to store additional information in the authentication context to perform uh authorization rules on it.

A

uh The challenge that I encountered while um while um trying to implement this is how it interacts with the authentication, proxy right and the authentication cache in a way and I was trying to figure out if there is a way to make the authentication sketch. Still it's.

D

A

Efficient while allowed in this future, so that is kind of why I paused it and have not been pushing on it um I I guess I would like to hear. Does this sound like something that is useful, why'd you bring it up? Is it because it's rotten and it's about to be closed or what.

C

Yeah, so I I wanted to make sure that you know if it, since the body was harassing it that um if you know, if you decided you didn't need it, then you know we could just close it and with some explanation of why you decided you didn't need it or, and it was the the state of it was unclear like what you just described, was sort of unknown. To me, um how.

A

Many comments that on the bug and let it uh get club or let it be let it remain closed, uh but thank you for bringing this up. Maybe um if it's uh gets, maybe there will be a time in the future where we can prioritize it again and work through some of those problems.

C

Yeah, that's fair, um I had a question for you. This does it so I I assume the desire to have this is for one reason or other. You don't want to use an authentication proxy to do this, since the authentication proxy would be able to have the full Network details and just embed them into the um extra field.

E

A

Yeah, that is a good point um that.

D

A

Is a very good point: I think that would um I think that's valid.

A

um The I I think whether or not the authentication proxy approach is used, which maybe that makes the most sense here or this approach is used. They're still, uh challenges with making sure that, like the um captioning is efficient, but yes, I I, see your points that maybe authenticated proxy would.

D

A

Right approach, um without other issues, um with uh implementing something like this.

C

F

C

The bar is kind of high asking people to write. Windows is kind of High.

A

um Yeah but um the bar is high I. It is unclear to me uh whether the um requirement here is Niche, so I think if the you know, if the requirement is Niche, then the bar should probably be high right uh or I. Guess it's fine. If the bars are not the Martian probably be High.

A

um Is this a something that like is this a capability that anybody else thinks that they would make use of?

A

um Just maybe Azure have similar capabilities that aren't supported in AKs.

G

I, don't think we would other than a proxy and I guess: I'm, not Azure, um red hat I. Don't think we would probably.

C

I mean I could certainly ask Anish. Do you have anything on top of your mind that comes up to where we could maybe leverage something like this.

B

um No I don't think so.

A

Okay, so I'm going to leave the tab open, I'm, going to explain the problems uh with the cache and also add.

A

A

Also awesome proxy. It is like to be.

A

C

This and then I'll post.

C

A

Foreign subject, access.

E

Review and uh you can skip that that actually got approved and merged already so.

A

A

Okay, um who wants to take this one.

C

um I think I'm here and I'll see you I see Tim on the call too. uh The gist of this was to be a folk agreed with the assessment um that we should remove, that uh plug-in. uh I forget this okay uh Timmy. Maybe you can remind me if it's already deprecated uh I, don't remember, um but we I think the feeling was that we have built far better approaches to doing what this thing was doing, and it's not so it's got gaps and probably should not be used.

C

I'm trying to remember where the retroactive thing came into being. Do folks, remember why that was suggested. Was that just to document what it was trying to do and then have a better explanation of why we don't think we need it anymore?.

E

um I, don't remember the retroactive cup. Normally we do that when there was all like Perma beta thing that is wanting to be GA and we use the retractive cap just to capture test stuff and upgrade stuff and make sure it's actually GA if I thought we'd mark it deprecated, but I just looked and I don't see any code mentioned, but maybe we mentioned it in release. Notes.

E

I I think we can unequivocally say: people should not be using it because, if you're using it to protect yourself, it doesn't effectively protect you.

E

um So, at the very least, putting a note in like wait, you use it like printing out something saying you probably don't want to be using this, um that that seems like a no-brainer uh I, don't know that it has actually cost us any maintenance effort. I think it just sits there, um so it might be worth doing a sweep for like any references in public GitHub to see if we can find any evidence of use, um and maybe what we've done in the past is reach out like file issues.

E

If we find use of deprecated things, you did that for the service account token deprecation, um reach out and say like do you know that this doesn't work the way you think it does, um maybe maybe a sweep of that if we can't find any used or we do find use and we reach out and they say: oh thanks, we'll switch or something, then that would be a stronger case for announcing deprecated and removing if we do find use- and they say no, we love it and here's. Why, then, maybe we can reconsider it?

E

Try to understand if it's a reasonable thing to keep them.

C

Looking at the kubernetes website, Docs there's no indication that there's anything wrong with this plugin. It just looks like a nice first class plug-in right beside like service account.

B

The first step: we should definitely update that that page that I linked in chat and say this is deprecated, do not use this.

E

um So yeah updating the code, updating the docs scanning for use. Those are some concrete things. uh Maybe we could put that in an issue in tags and security and Market has help wanted um for those concrete actions.

C

E

I think we have.

C

An issue right, so we maybe we just put the next steps in there.

E

ah There it is yep open in July.

A

uh Let me drop it into the agenda.

E

Oh it's already. There cool.

E

I'll make some I'll update the description of that to call out these action items and market help. One.

C

Yeah but I from the gist of this conversation, I, don't see any uh one immediately saying. um Do I really want to keep this? Basically, it's it's more of. Let's explore to make sure that our belief that it can be safely removed is correct, and if so, let's do our very graceful dance of slowly removing things.

C

um Okay, uh are we ready to go to the next thing.

C

Take that as a yes, um uh so this came up during one of the uh the triage meetings.

C

I I, what I was looking for was.

C

What do folks believe should happen if you're in your log mode is blocking strict, but then the file that you were writing to gets messed with in a way that you're no longer writing to the file, but because the file descriptor is still valid, there's no errors but you're, just not actually writing logs.

A

A

You're writing logs just not linked to the file system anywhere. I, don't know.

E

Did we roll our own log handling or are we using a library, oh.

C

We use login the library.

A

Or so blog rotate or like the recommendations. Generally, you just use, like a log rotate for the ones that, like the log files that we write um but yeah I'm, pretty sure this is just right. The audit policy just right or uh I, don't know. I just went to a file foreign.

A

This is like the standard Behavior. If you read a file and you write to it and something else deletes it.

A

um The file descriptor remains valid.

C

Right um and I think if you, if the log mode wasn't blocking strict I, would feel pretty confident saying just I mean I, don't know that happens, but it just. It feels really awkward that you know you set it up in a way like I, I, guess the question becomes: whose responsibility is it to make sure that the file remains valid? Is that Within the API server's responsibility, or is it the operator to make sure that Nobody messes with the host and thus cannot mess it in the file.

B

We're also not doing things like forcing files synchronization after every write, um like I. Think if you really really want blocking strict mode, probably send it probably use the webhook implementation and implement the semantics. There feels like the better approach.

B

um That said, maybe there's some usefulness to uh Tim's suggestion to periodically reopen the file.

B

But I'm a little inclined to say that this is like an operator problem. Yeah.

A

Me too I'm trying to think if there's a valid use case where it this is, this would be used. I, don't actually know how like system, if system D whatever do this.

B

I think my main concern would be if there was some bug in the like log rotation, where, like the rotation kind of didn't complete successfully.

B

How would you detect that.

C

What what are the traditional steps for a rotation? Is it like move the existing file and then create a new file at the old location, or is it or is it like copy the existing file and then truncate the existing file.

D

Yeah, both both modes are used, like usually like you, move the existing file and then send a signal to the process uh which causes it to reopen the log files.

C

I, don't think we support the the signal thing right. Yes,.

A

That's not what um log rotate does so Analytics.

D

It's got a copy truncate uh option, so you like copy the old file and truncate the uh the existing one and new rights go to the like the previous existing file.

A

Press rotate, okay, so yeah okay, so you can also send signals. Okay,.

D

Yeah, but that this really depends on like what what the definition of audited is right, like um and in production, I think like. We don't really use log files uh in the sense like do you just forward it to a log auditing service?

D

Is that true for most clouds or like production deployments,.

B

I think some deployments use the file as a uh like a local buffer in case there's a network uh segmentation or something like that.

G

It's Not Unusual to use the file right if you're running, on-prem and you're not sure or they don't have a hosted logging environment. It's pretty common to say the next best thing is to write this on disk.

C

But scraping discs can be totally automated to later on right, so you don't lose.

G

It yeah it's a common way: people choose to integrate, I, don't I, don't have really strong opinions about this like it doesn't seem like a high priority bug to me.

C

I I think the question more: is it a bug or is it just the way the system is supposed to behave.

A

I would say it's probably not a bug. um I would maybe call it an enhancement.

D

um Is there any way to detect lost audit records like if I'm looking at the like a sequence of what it records can I like see? Oh there's a gap in the sequence numbers here and I know that some records were lost.

E

Randomly generated I, don't think it's a sequence.

A

You could have a running hash.

E

um Okay, I don't have that.

E

Yeah the threat model of someone who has access to the disk like relatively privileged things on the disk, like the audit log, but doesn't have access to things on the disk that could let them mess with the running API server.

E

It was a little weird um like someone else mentioned, like we don't fsync after every right either. So like there's a maybe a variety of things like does, the file still exist. Are we f-syncing?

E

um Is there a way to detect gaps in the audit log, like there's sort of a few things around this? That maybe could be done? Some of them would take a performance at some of them. Like uh you know, incrementing sequence, on the events, a single server emits, maybe wouldn't be a performance hit, um but I kind of agree. It's a sort of low-ish priority enhancement request like reliably ensure we get all events in the file and sort of bundle, those things up into that.

B

A

A

B

A

I, don't think there was any I guess it's between the way, more evidence back like I, don't think! There's like I didn't hear um anybody. Thinking saying that we should not do this so I'm just gonna put that clock.

E

I think backlog is probably fine if it's important to someone enough to pull it out of the backlog. I wouldn't object, but yeah I, think someone who wants to see it done would probably need to grab it.

A

um Anything else that we should add here.

E

uh I'll update the description and call out the fsync.

A

um Yeah and detecting gaps, I guess the uh to up level. It's just more uh look at reliability of file based on it sinks.

A

um All right are we done with that topic, I think so.

A

F

uh Yeah so I'm, looking at picking up the cap 3257 again and there's a two halves to the functionality. One is the API server pieces about accepting trust anchor sets or whatever. We call them now. Think trust member sets okay and then there's the bit about cubelet, actually being able to project them and I'm wondering should they be 1pr, one Mega PR or to still quite large vrs.

A

I like to still quite large PRS.

A

Yeah I think uh re-reviewing changes is a lot easier, um the smaller the Pras.

A

um It's a lot easier to miss stuff that changed in the PRS, so yeah I would even like if you can break up the um API stuff, like maybe uh admissions separate from uh the API Group um and registry stuff.

F

Is it okay to merge? So, basically, what I wanted to avoid was a situation where there was code valid at head. That would accept the object, but then like not apply, validation or things like that.

A

um I think that is not uh I, don't know what do you think Jordan.

E

um All the API level changes, including validation, should be grouped. The implementation that like makes use of that and actuates stuff based on that, can be stacked in a separate PR, but so in an API PR. We have the fields, the definitions, validation,.

D

Yeah, that's basically it yeah.

F

That's all that's in the first PR, it's just a correct kind of stuff um should I include so there's also API changes for the cubelet piece, like changing the definition like adding a new projected volume, type. um Okay,.

A

Yeah um and I would say the admission changes can be separated from the API validation, defaulting Etc as well. Even.

F

A

Is also the API server okay, uh so that's like four or seven years. We.

C

Have a lot of buffer when it's Alpha right because, like we're not We're, not gonna, get it all done in One release. Probably so you have to accept at some point, you'll draw a line and say this is the amount of functionality we implemented in the first attempt, yeah.

E

You just want the like: you want a particular API and the validation associated with it to be coherent in one PR, so yeah.

F

I think that's what I have maybe there's I can extract the admission piece to make the pr slightly smaller I. Can't it's been like a month or two since I've looked about it so and holidays.

C

Yeah, the the we just have so much code gen from the deep copies and everything.

F

C

Like when yeah, when you act, like I I, do appreciate when folks put the the generated stuff in its own commit. So that way you can look like you should like I like to review it, but I just like to review it separately from like the parts that you hand wrote uh just because it tends to be easier to figure out what happened.

E

Also uh get um just uh released a feature that makes working with stack branches way easier. So if you have like one local branch, uh you can push it to like three or four different remote branches and update three or four different remote PRS. All.

F

Of those which is that would be great yeah. What.

C

Is this called that branches update.

E

Ref, so you can have a branch that has like four reps on it and then rebase and squash and then anyway add.

A

It to my reading list, yeah foreign.

A

E

And actually uh I'm I have a API review um question that I try to do every other week, um and so this is on the list for tomorrow's. uh If you want I can invite and if we want to do any hype, Android stuff tomorrow, that might be good. Okay, true.

F

A

Exec auth plugin uh in 127. Nick is on the call. I would really like this to get done in 127 and I'm not going to start focusing on it a little bit more um yeah, I guess.

C

Well, yeah I'm I'll be available to take some some work for uh in 127.

A

C

Are you still on leave? Where are you going no I'm back, okay, I! Don't like the message, people that are on me.

A

Yeah, so the One technical thing that I wanted to discuss, while I had about Jordan in the room is the changes to the um uh upgrades to use uh the HTTP client rather than the dialer or a weird dialer thing.

A

um What do you want to see to allow that to happen, because it turns out that it makes the client's exact boss plug-in it's far far easier to implement great.

C

Do you want to link those and.

A

And give context Mike yeah, okay, I will yeah.

C

I was having a really hard time understanding how a round Tripper could give you a base round Tripper when it had two bases, and it selectively picked one based on the context of the or based not based on the type of request.

A

um So the problem that we have today that this attempts to address is that we have uh when we do an upgrade from Speedy for Speedy, um and maybe this is true for websites I can't remember, but definitely for Speedy. We uh create a request, we dial a connection and we write the raw request to the connection, and um then we parse the response.

A

The problem with this custom code is that it lives independently of, like you know, the transport cache configuration um ideally what we would want to do with this client exec as per spec in the kept is we want to. We can implement this fairly straight fairly easily by just replacing so like the base transport um that obviously doesn't work for our Speedy upgrades.

A

um It seems possible- and you know fairly straightforward to remove this. uh um You know dialer based implementation of upgrade requests and switched to just the HTTP transport, which allows us to implement clinics like auth plug-in just by replacing the the lowest transport in the transport cache.

A

That's the motivation for these changes, but it would be nice to land these before beginning the client exact auth.

D

A

Work, otherwise we are going to make that dialer upgrade stuff like much more complicated, um and it's already kind of Fairly complicated time is.

D

It possible more complicated, yes,.

C

Mike as a possibly different approach like I, think what you're describing is uh I just want to change like a piece of the round Tripper and thus implement it. That way, and in order to be able to do that, the Speedy stuff needs to behave nicely with that happening.

C

um I think what I had originally tried to do was effectively try to use the HTTP reverse proxy stuff. So, instead of like, instead of changing the client I effectively changed the server and then uh extra hooks into that that transport.

D

A

That I think is compatible with what I want, but if that's still, that reverse proxy stuff doesn't work with our custom dialing, because our custom filing doesn't use a transport.

C

So I I what I? What I was saying is that you would have all of your custom. Dialing logic stay where it is, and it would make a request, it's just that it makes a request to a different server right. So it's it's! Instead of trying to change the semantics of how like the server stuff works in the transport, we just literally changed the request to be a different place, all together right. So, instead of like going to API server Dot aks.com, you go to like 127.0.0.1 literally that's like the server you're going to.

A

Yeah um I I think that is the option where we make the dial stuff a bit more complicated.

C

A

I I think that would be valid if we are so afraid to touch this stuff um that we can't like do some simplification but I think in general. um Removing that would be beneficial if we can do it carefully and correctly.

C

Yeah I mean yeah I generally agree with the notion of trying to come up with some way to simplify this stuff, because it is incredibly difficult to reason about. Even if you know what it's supposed to be doing, um I I, guess I think I would ask like how would we? How would we validate that we didn't break it? Yeah.

A

That's exactly what I was uh leading with. uh Is this like an upgrade test uh for um Cube control exec? Is it like a down downgrade test for compatible backwards compatible? What are we, um what is acceptable here.

E

How many of the tests I I vaguely remember that we actually had pretty decent test coverage of a lot of different scenarios, but a lot of it was at the unit test level and I'm, not sure how much of that relied on the implementation. It had to be ripped out or changed from this changed.

A

um We have, uh we have e to e-tests I, don't know whether they are running in the. um um Let me see the ete test. D I, don't know whether they are running in um any version ski Suites, but the the ete tests are fairly extensive.

A

um The change here uh was that we stopped I removed the go proxy library because it was actually kind of busted, um but, like the ews here are extremely extensive. um They are do execs through um a student proxies um I think that we have a. We don't have anything sex through a um uh socks proxy, but we could add one, and we should also make these run. Inversions key Suites.

E

Yeah the fact that this appears to only be changing the client is promising changing the client and the server at the same time is sketchy um I I, like the idea of simplifying it a lot.

E

um It might be worth looking through the history of the speedy roundtripper and um like seeing when some of the options and things like the Ping period, And like I I'm, just looking through the file and seeing some of the options and wondering um if we have test coverage all of those things so like Ping period, keep alive. I, know exact, like long-lived, exec, um sort of terminating after a minute or some some short time out. The Ping period might have been added for that and we might not have ede coverage of that.

E

um The proxier uh HTTP proxy and socks proxy, um so just making sure that we're exercising all of the sort of knobs and levers that the old one had um and then probably uh sitting down and doing a review like in person. So we can talk through each each thing. That's changing and um just make sure we're catching all the balls. We're throwing.

B

C

What were your thoughts on like so we we have the ability to un unwrap transports that you can kind of do at the quote level.

C

The the part that scared me most about this one I think was that, like you have to make some assumption about, like which transport you're gonna go back because you have, but you have two distinct transports now you have a HTTP two one and an HTTP one with upgrade one and.

A

I fixed that that is not the case anymore. Yeah.

E

I was looking through it and I didn't see a bifurcated.

C

Okay, so just ignore me, then that doesn't matter anymore: it.

E

Looks like a pure rap now: okay,.

A

um Yeah so check it again. I can't remember exactly where um but I registered.

C

Is used rather than um so since David is on the call hey David. Do you want to like go rewrite the upgrade handling on a aggregated server so that it doesn't also have that horrendous mess and.

E

One one major proxy transport wrapper rewrite per release. Please: okay,.

G

I'm willing to obey the one at a time, but I would look at the technique and see if we could simplify it, uh although um I will admit to being a little gun shy after yeah to end all cves.

C

Now David, it was only a 9.8, you had Point left, you got it, you gotta see if you can make it yeah.

G

uh So so I'm I'm open to it, but I don't think I want to go for it. At the same time,.

A

A

Okay, um yeah I will uh follow up uh yeah with whoever's interested um and then yeah uh once this is in, like y'all all to talk with or I guess, even before that I'll sync up with Nick and um see what the next steps are for. The actual implementation of the meeting parts.

A

And we have till March.

C

um I I assume we're all still pretty happy with the idea of using like a proxy based approach right like no.

D

C

Has since changed their mind and wanted something else, I I I've recently seen like the scheduler folks were like looking at like awesome, plugins and stuff I just wanted to make sure that no one else was like nah I want to make a proxy.

A

I, like proxy I'm, still happy with proxy.

E

Yeah I still like the idea of it. The I I haven't followed the discussion closely, but the after walking some folks through, like the client, go exec plug-in thing. The only question um would be like invocation of the thing and are we opening ourselves up to like consuming Cube config? That makes stuff get exact.

E

um I know there were some concerns around that, but the exact plug-in. So if this is another mechanism for like spawning a process or yeah doing a thing, um it's just worth thinking about how we protect that or inform people of it or how we let people manage that in their environments,.

C

Yeah I mean so I think in Alpha and stuff we were going to require like an nvar or something to be said before it would be used, but otherwise it uses the same API as clango credential plugins, it's. It is a classical credential, plugin yeah.

C

You could say we are making them more powerful.

E

But previously previously, like the clan go, credential plugin was limited in what it could observe and now we're sending the requests through it. So.

A

Yeah yeah, but you could I think that there's that aspect of it but there's also the aspect of um you- have a cube, config file that can run arbitrary code. That can you know demonize, and at that point, does it even matter if the request is passing through it or not. I think this is. There is a valid security concern here that might not be as addressed as.

B

A

And I think it probably applies to both the Old and the new client exec plugins. You know yeah.

C

Like my long-term thoughts around this is um I know, six CLI is working on like QRC support. I I would like to be able to add like a list of trusted plugins into that QRC file, and once you have that, then it'll just refuse to invoke anything that isn't one of those and I think at that point. Your attack vectors, really small, like just.

D

C

Bash in there and you're kind of good, um so that was kind of my thoughts, long term and trying to lock this down, and it basically has to move out of the cubeconfig file into a separate place where the user controls it and it has to be a file that the user doesn't like. Let other people set and generally you probably wouldn't other people mess with your QRC file. Probably.

A

Yeah, it's both QRC path to Binary Sim links in between it's fairly complicated, but.

A

I think there's a lot of exposure there. We should. We should look into it. I think.

A

All right um does anybody have anything else on this topic.

A

um If not, let's move on to our CI stuff did.

E

uh Was there anything else for 1 27 that folks were um planning to work on that? We wanted to mention here for visibility or in case people wanted to be involved.

E

um The who am I API I would like to see us promote debata.

E

I can link the cap.

E

That was very self-contained uh and.

E

It's hard to imagine it evolving much beyond the send me my user stanza Yeah Tim, my.

B

I just wanted to call out the web hook. Exclusions is something I'm looking for looking at in 127 I think this Falls more under API Machinery, but kind of off the Json um uh I. Have the cap open, I'll drop enough notes, um I'm uh going to write up an alternative cap to the one. That's currently open um that looks at using cell predicates instead, um but I think some some sort of approach in this space is something that I'm hoping to get to Alpha in 127.

G

When you write up that cap, can you be sure to describe which so variant you plan to use and why we actually have two? Now we have uh self cell variant for crd validation, and then we have a cell variant for for admission and even though it's the same cell library, all the bindings there's a very different set of Bindings that can be used and.

A

Is it the like macros.

G

So there's different variables that can enter it. So so, like you, access an object in in crd validation, you can get the self and the new self and it's a self-induced self. That is like part of a schema because you're defining a scheme in the same spot for admission there is the old, the new and then a bound policy object and how those things all intersect is a challenge.

E

Yeah basically gets the fields that would be in an admission review as variables. The cell expression can okay and the additional params variable. If you configured parameters in your policy and.

B

Maybe also the namespace object. That's that's.

E

A field in the vision review.

B

E

It is uh the namespace.

G

Name, what's his name now there is talk about whether we need to allow access to that object and then there's the secondary Apple check. Yeah.

E

So that one's still progressing through Alpha there's those are the kinds of questions we're going to be asking this release there yeah.

G

But I also think that I would ask very similar questions on this enhancement as well. Yeah.

B

Yeah that makes sense I'll make sure to cover that in the cap. um Just very briefly, like I'm, imagining this having access to the full admission review and nothing else.

B

To keep it as simple as possible and might be.

A

All right and yeah, the other thing was, we did drop um webhook configuration.

A

Dynamic admission, uh Cube config um support for trust, bundles, and maybe we will uh reincorporate that, but I'm I would guess it's unlikely to get that see that implemented in one 127, but maybe the camera will be, um uh the design will be redefined.

B

Mo, do you know if anyone's going to be driving multiple authorization web hooks? Let's release.

C

um I can reach out to the room and see, if he's feeling up to it. When I talked to him at UConn.

C

um He seemed optimistic because uh um when I, when I described to him like what I wanted from the cell stuff in there, he felt a lot better, because I I think he didn't immediately grok that the schema was effectively static. So you didn't need to do a bunch of answers, um but I can reach out to him and see.

F

Foreign ing to maybe introduce a cap, but probably not try to get it landed for 127. That's the Duel of the trust anchors. Instead of doing trust anchors into pods putting certificates into pods.

C

Wait are you wanting to um at least get it marked implementable, or is it just like you want to have it open in this release?.

F

I mean I like I'd like to get it fully done and implemented now right, but well realistically, uh probably just start. The discussion.

A

Okay, all right uh CF, flicks,.

A

A

A

uh Namespace controller, it doesn't work and container any GCE cause.

A

498 testing there's 26 today. That is interesting.

A

A

Exit status, one wow.

A

And then these are.

A

Okay, these don't look great.

A

Okay, uh these look a lot worse than usual.

A

Let's see how test grid looks.

A

A

A

Does this indicate that our tests are fine in our test, Suites and failing in others, so maybe there's something disruptive going on.

A

In the other test rates, I will uh I'll grab.

A

These two in file tests like bugs because they look or actually now there are a number of egregious failures.

E

It looks like there are four specific jobs that are failing, like hundreds of tests, so the proto-ip Alias, slow and local jobs are just totally red. Okay,.

A

Yeah, alias slow and local, okay, um I guess, I'll search for uh Vibes um for those test Suites and if they aren't there, oh, maybe a file, an issue. Okay, maybe ask Sig testing yeah, but ours look stable as usual. Nice work not ready, not fighting tests. Everyone, um cool, I, think that uh concludes it. For this meeting, thank you for joining us. uh Happy, New Years. Everyone I, will see you all in two weeks.