Kubernetes Kube-Proxy Special Interest Group, 19 Jan 2022

Previous Meeting

⏯

youtube image

►

From YouTube: Kube-Proxy Bi-Weekly Meeting (APAC) for 20220119

Description

Kube-Proxy Bi-Weekly Meeting (APAC) for 20220119

A

Hi everyone uh welcome to the cupping apac meeting. Today is 19th jan and here we are uh going through kpng.

A

Just a reminder that this meeting is under the cncf code of conduct cool, um so looking at the agenda, here's a quick update sort of um first of all, you folks able to see my screen right.

A

Yeah yeah cool, uh so so we'll probably have to go to the sig network uh meeting and uh just add an update around what all uh is happening in the coming apec working room- uh and I was thinking mostly around uh the ci, mostly around the ci.

B

Like you know,.

A

I think there have been a couple of test cases that have been fixed or uh some are in progress. I'm not about that. So maybe you can help me here, so I I'll just go through what what I have in mind and then you folks can add so something around the ci uh uh something around uh the ipvs stuff that uh that we can all have been working on. uh Sorry that hanuman has been working on. uh uh There have been some ip tables enhancement that we have been working on.

A

uh Apart from that, uh you know, 1.18 update, uh so this is sort of, and obviously like user space update uh user space port. So this is sort of what I had in mind. uh Do you folks have anything entirely.

C

um I think yeah, the this looks good, I think, but uh maybe you can um hanman. You are working on that diff state or something I pissed. I heard some abstraction you're doing, I'm not sure if.

D

Yeah I mean basically yeah. There was last sunday. You know there was a long meeting. I think a couple of us got together just to give you that update you guys were not there. uh Basically, uh it is again a consequence of couple of test cases failing right in the ci, especially around session affinity and all for the ipvs, but I think for iptable.

D

I think those test cases are passing right. Vic. That is not a issue I think, but I think session affinity some of them are failing is what per has told, but it's not always it's taking. Basically, some of them yeah, okay, okay, but at least in ipvs case yeah. Still the code is missing, so I was supposed to add it. So that's why they were failing, but I raised a review in spite of that they were failing, and then I mean I did analysis.

D

I think last saturday, sunday so couple of those six seven test cases, so they have a test case related to switching the session affinity on and off at runtime. Basically, they create a service with session affinity on and then they will, uh you know, switch it off so that kind of test case. So then, when I saw you know, the kind of discussion went into somewhere else, because um right now we are using service events in the ipvs back end.

D

There is something called service events, a thin layer between actually a backend code and a cupping server which mikhail had put together for those back ends, who want a granular events if that back end is granular in nature. The best example is ipvs kind of a back ends right when I say granular.

D

Basically, you can, I mean sorry atomically, you can. You know, insert the end points uh you know and you can atomically delete them. uh All sort of atomic nature is supported uh in comparison to. If you take ip table right, that is having a altogether a different nature. So service events when me and mikhail had discussed long back, so he had put together that layer in between, uh but that was not sufficient. I mean it was not covering other scenarios like session affinity or even going forward.

D

There will be something like internal traffic policy external traffic policy, so the discussion started with that on sunday. Just to give you the quick update and that's when you know mikhail was.

D

I mean that discussion went on with michael j and everybody, so mikhail has put together a very generic full state, diffing logic. It is not exactly what I wanted, but he what he said is there is something called full state and filter filtered state. Just to tell you guys.

D

So basically, whenever you know the cupping server gives a full state to the back ends right, how they should handle, and there is an option, also called filtered state- or I mean name, might not be so proper here. It is called filter reset. If you look at the code, so that will give you the the atomic updates. Basically say you add service, a b c right in a sequence, so full state gives you a a b abc like cumulative.

D

You know uh data, that's called full state right, so, whereas filter reset, if you see it, will give you a b c in that, if you create a it will give you a if you create service b, it will give you service speed so like that, that's a filter reset so mikhail whatever he has done last two three days under the context of uh you know this discussion and as a part of the discussion right, so he has put a diff diff logic for the full state kind of back-end.

D

So at least that's what I understand from his code, but the discussion I started was more for an atomic calls. So still you know you guys hold on don't get confused. Where is that going? So I am also parallelly working on enhancing the service events, at least so mikhail. Is here, hey mikhail, so so I'm just telling them. So that's the activity which is going on and neha just to quickly answer you as a part of that code.

D

Changes right, which michael had raised two days back, wherein he provided a very generic diffing logic for the full state kind of back-ends. He used a generics. Basically, so generics are very recently introduced in golang and it is used in I mean it has come out as a part of 1.18 beta and very latest build of go and that's what mikhail has used and which made us upgrade to 1.18. That's the context.

C

Okay, okay, yeah.

D

Our michael is here yeah.

C

Hi hi mikhail here, hello, yeah.

D

C

Are using this full state-based logic and building that div state and and we will use that in ib tables also.

D

uh Right now, mikhail yeah mikhail, is there.

E

C

Has done it for.

D

Nft, maybe vivek can answer. Probably he hasn't gone through. Maybe you can say about it right now. We are using that filter reset uh in the ip table. So I mean earlier. I was looking at service events for that atomic changes, but then, like last and the last meeting was saying right- I mean there are some issues with that.

D

So right now I am not proceeding in that direction. So right now I think the other one how it is in cube proxy, but only thing is, it won't be a full state but the even based using filter, reset yeah.

A

Okay, well, uh just before we go ahead, uh I I forgot that uh you know we've got ashraf here, so I mean, uh since this is his first meeting, I mean feel free to increase yourself.

E

Thank you raj. I will do.

E

So shall I process myself.

F

E

Yeah so uh hey, my name is ashraf, I'm a senior devops engineer at sempress technology and I'm interested at kubernetes, and I wanna help you guys and learn from from it. I want. I want to learn how how to do it, I'm not very professional. This is my first open source project and I hope I'll be helpful.

A

Well, nice, to have you here so so. This is the cube proxy uh next generation sort of a working group, uh and this is the apec friendly meeting. What we're trying to do over here is uh sort of redesign uh cube proxy and you know uh decouple it from the current logic that it has so feel free to say hi to us on the slack channel. We are sick network, a png on kubernetes, slack and you'll find some resources on what we're doing here uh cool.

A

So uh you know thanks for that uh sort of overview around full state, and I was kind of going through it. So the way I understand it is.

A

Just correct me, if I'm wrong, so full state uh sort of uh back end, that's like only uh as of now that's nft.

A

uh So it's it kind of gets all the events uh from kubernetes and then uh the back end will read sort of a diff uh and then write uh nft rules depending that are different, like like in layman's terms like this is how I'm picturing it uh like because for iptables, what we've been doing is like uh use.

A

The sync function- and you know, through the set service, delete service function, kind of uh update the service change, tracker data structure, things like that and then write iptables rules, so full state kind of gives another type of a backend right like.

G

It's another way to do it right and it's it's less even basic. It's actually really taking the whole state uh state of unknown world for blood and then produces all the objects you want for your backend and then you will have a div for every object but of your subsystem, not of so you're not trying to propagate the events like set service or setting point or other ip port you're, not trying to propagate that.

G

To changes to your subsystem, you actually compute the whole state of your system again and from there you know exactly what you need to do to sync your system with a new uh wanted state of this subsystem.

A

All right regard it and and uh in this example around uh the users, like the user space example wherein the client itself uses something like I add, ib port and things like that. This is more often like a granular event base, because the client cupping client is sort of sending uh uh the events to the backend right.

A

G

Another, uh it's still computing a diff of a capping state not of a subsystem state. So I think it's harder to think events from a generic system or for something. That's not your subsystem to your subsystem than to just compute the whole state of your subsystem and then know what to change in your subsystem.

A

Got it so the entire subsystem changes are what I've done in nft as of now so are we trying to move uh iptables implementation towards nft.

G

No, I think I've disrupted enough people with that. It's already confusing enough. It's smells like an ft, is a demonstration for that.

A

G

But how would you map that to ip tables, for instance, that you have one one diff store for your chains and one gift store for your ip sets, for instance, and when you get a new state you're just going to compute? All of that, and you will have your your you had if you need to apply so which tables has have been impacted by the change and which which ap cents have been impacted by the challenges in the state in the keeping state.

G

But you do not need to think about. Okay, it's an update to a service or it's a new ip part. So I need to get in this chain and add these rules.

G

You don't think like that when you use register, you really just say: okay, I'm just going to write everything I want, and it's going to give me what what has changed.

D

Yeah mikhail actually like, uh if you remember like until last week until we had the discussion with lars, I was trying to do that instead of going through all the rules and then pushing just get the changes and just add the iptable rules just for that. So in that case I think uh that new uh event based service events event based change- was what I was using and it was actually working.

D

I mean I didn't complete the code, but I did for service and end points etc, but still it was like about 80 percent done, but then, like lars was.

C

D

It would not be useful right, I mean because he says like uh when you do even say. For example, you just add iptables minus a and add a rule. Basically, uh the kernel actually does uh pull everything up and then insert this rule and then push it, which seems to be more. You know less performance when the user space does it.

G

It's faster to update all your or your change. I think it's because when you add something, epithelius takes the chain and then does kind of have to to to rebuild the chain code.

G

I'm not one hundred percent sure, but I feel like it's because of the changes to and fte and on those bpf based things, but you actually have to get all your chains to compile them to the new uh table or bpf rules, I'm not sure. What's going on behind it, I think that's why you need to get the whole thing.

G

Do the change complete it back and send it? So if you it's true that if you actually have the whole thing to replace, you don't need to uh to get it from the kernel before completing it and sending it sending it. So maybe that's uh that's why it's uh it's! It does have a performance impact.

D

Yeah so I mean right now: I'm not going in that direction. I've just stopped it there and maybe I'm just trying to use the ip set there. So maybe that's it.

F

So there are many sets in your.

G

Proxy for now, right, yeah.

F

It's not that right tables, at least yeah.

G

Yes, there are potatoes, of course, so in legacy proxy, so maybe you can. You can use the disk storages for your ip sets. It would be working.

D

D

D

You have any idea whether ipvs has a similar limitations, or you know drawbacks, I mean say in iptable. Yes as when lars enlightened us that okay, it is not efficient to just push the delta. Rather, you read the full thing from kernel and just add what you are right now uh modifying and push it so that it is more efficient efficiently done in user space than kernel. So a similar situation does it exist in ipvs. From your experience, have you commented.

G

I don't know what the plan does in the background.

G

I would assume that math will have will have a pet method and a direct method, but maybe it's still more more cisco's to do that than to just replace the ipsec. It probably depends on the size. I'd say.

D

No, I was not talking of ip set like we have that ipvs adm minus ln that table is there right, so that it supports autonomously.

H

G

I don't know yes, I really don't know correctly.

G

H

G

In the kernel we can optimize a lot more because it's much closer to to what we want to express, but I I have no certainty on the technical side here.

D

Okay, okay, because yeah, because, as vivek said, he has stopped working in that direction right. So, but if you see from earlier, you know we are working on the very atomic calls to the ipvs.

D

So, but if you see the cube proxy, it is very similar to the ip table. You know, so they every time they rebuild the whole ip table content and keep pushing. I don't know whether it is because they ported the code of iptable and simply you know, reuse a lot of code and just did ipvs. Maybe.

G

That, I think, there's a lot of cotton past because of the an architecture of the legacy proxy kind of okay.

D

Sure yeah, maybe I will just continue, maybe when, when we start measuring the performance or something maybe yeah, we can look at it. I mean yes.

G

I think yeah we should avoid to to try premature, optimization.

H

C

So I mean you mentioned some session affinity also bugs in that ipvs. So you are fixing those bugs or or what's the plan for like getting those fixed.

D

uh No, no, it's not about bug so currently ci. If you see for ipvs a couple of session affinity, test cases are failing. One one reason was the code itself was not there. So that's why I raised one code review.

D

You know to add and delete the session affinity for service ips.

D

In spite of that, when I saw some more test, cases are failing. That is. There are two types of test cases which are failing in spite of my code being there one is. They are dynamically, switching on switching of the session affinity right, so that code also is knotted there. So that's what I have to add second is uh related to timeout. Actually, so you have this session affinity for a certain amount of time right so that will expire so looks like with ipvs.

D

It is not stable, I mean maybe mikhail might know, and even when I saw mikhail write, e2e test cases, they have mentioned certain bugs in ipvs about timeouts being buggy actually. So these are the reasons they are still failing, which we may have to. You know analyze further like why those test cases are failing. Actually. So I will be analyzing them in coming days. After I add all these scenarios, like you, know, dynamically, switching on switching of session affinity uh and all those cases yeah.

C

G

That's a pretty good demonstration of how hard it can be to propagate changes.

D

Right so yeah that's about couple of ipvs test cases I analyzed, but still there are other categories of test cases, also uh a non. You know, session affinity, related and uh non-traffic policy related, which we have to look. uh I have to analyze actually 85 or something yeah 15 are failing totally.

C

Okay, okay, yeah, because uh we will like a plan for looking at that, maybe because because ip tables are mostly done, I think only very few are left.

D

I somehow feel some of the test cases are. I mean I don't know why it is not failing, because we don't have contract and ready state still in there still.

A

I'm not seeing any.

D

C

I also have the same doubt because uh I I was seeing contract failures uh like but uh this week, or maybe I'm not seeing those failures. I am not sure on that and ready states a couple of weeks only we are not seeing that uh failure.

C

I I think so that is, we need to check why it is not. Do you try it locally, also running this test. I will try that maybe.

D

No, the code itself is not there, so even if there.

C

Is a test case.

D

It should fail.

C

D

There is no doubt about that.

C

D

C

D

Roughly, how many test cases are there in contract like they, whether they have good coverage I mean, even though code is not there, they are passing means it's a mystery right, yeah yeah, I mean at least when I saw last when per told contract. I had seen few failing, but I mean now it seems it's not failing, but I don't know why yeah but ready state there wasn't even a single one that I could see.

C

Ready state, like uh you imagine, I opened first issue, so it was on ready states only and uh there and and and after some time we are not seeing that error issue, not sure.

D

Oh yeah, you raised the issue.

C

D

From the test case, only right, yes,.

C

That was my first issue.

D

C

Yeah not sure need to check on that magic because I also want to understand, like you means we updated that proto buff right from uh to google, something I don't remember the after. That I mean I just uh posted like there are only six failures out of some 30 or 40 or 50. So so like what happened in that change,.

D

Yeah but at least these won't pass by itself. Yeah like there were many cases which were failing because, um like in between the kpng cluster itself, goes down because of that protobuf. So that's why many of them passed.

C

uh Okay: okay, many.

D

Of them are valid, but yeah still, I think few more should be there. It should fail.

C

We are because me and anusha are also working on one of the issue. Multiport issue.

D

C

We are stuck means there. There is definitely a code change for that in endpoints map I mean the way we are using endpoints mapping. Cupping is different than what it is in proxy.

F

C

Because we don't see that map definition correct in the copy.

D

C

D

That change, can you please tell me about the issue as well as maybe not maybe.

C

I will sync with you uh after this meeting, because jay wants seems that it, the issue, might be in that server model. So I need to understand that with you or arya, anyone who maybe I'll connect with you after this meeting on that issue as well.

F

C

A

Well, uh just just a just a uh interrupt, but uh I just wanted to know if you're, okay, with this list of things uh from the apec working group, you know telling a sig network around that, like the the work that we do, I mean whatever we've been doing around ci. You know ip vs ip tables uh things of that sort. So if does this look? Okay to you, folks.

C

Yeah, I think it looks good yeah what up what up yeah.

A

Okay um uh yeah: do you want to continue around the ipvs ip tables test failures, or should we move on to uh hacking around 1.18.

D

Yeah, I think we can move on yeah.

A

Sure neha, like do you, want to share your screen or.

C

Yeah before sharing screen, I think uh I think nikhil is already here. So I we see the changes of a bit are already there in copic uh means. I have seen that pr merged and we see 1.18 beta changes in the cupid. So mikhail I mean do we I mean, should we go with 1.8, but it's not available. Also.

G

The uh the profile and there's a bitter one, that's available, and it wasn't too hard to install at least for me.

G

So yes, I would say: let's go. I asked if, if I were, if I should postpone a bit but jay was like now, let's go so it was okay. Second, validation. Let's go.

C

Yeah, because I mean we will need to download that also right, but it's not available now.

C

It is okay, yes,.

G

And you can actually uh do a go, install something and it will. uh It will build the program to update your your go installation how to download the go in your home to do.

C

G

Precisely it wasn't very hard. I just had to to run that and uh and just update my path to the new renewgo 118 beta.

C

Only 1.18, I think.

H

No, no yeah beta one.

G

If I just want to to open, uh go around the top, it's.

C

Okay, let me do this. Let me install this beta1.

D

Yeah beta1 is where you have generics now. So that's what mikhail has used two days back here.

G

With generics, I could finally do it.

H

Yeah yeah right here.

A

C

But I see the code changes are already there. I mean what other changes will need, I'm not sure for that, because because in docker file I see the changes are there and in code.mod also uh because yeah means I'm.

C

The only change I think is in now github action file. That's it because other changes are already there.

A

Yeah yep, I think uh we only have to update this ci and I think yeah yeah.

C

Ci file I'll.

C

A

Yeah, the docker file also has been updated right.

C

Yeah, yes, yes,.

C

So only thing is we'll have to update our local go version and try and test this right build once yes,.

D

Mikhail, I think you worked sunday night, I believe right sunday night, we had a meeting. I think you didn't sleep overnight. You did this change.

G

G

I'm trying to fit camping in the schedule. It's a bite.

A

I'm still yet to catch up. On the you know, the code base around diff store and uh golang genetics so.

G

Yeah you want to to do that, though,.

A

Surely I don't mind, we have some time.

G

Yeah, so it's in client, the most important thing, client store, 2.

G

And there you want to start.

G

So I I I tried to roll a spot example test, so the first one.

G

So it's easy to to match the output with the card, and it gives her a pretty straightforward. Straightforward example:.

B

G

A

This is the test. You want me to look at right.

G

Yes, you just have to go up. If you want to see the card.

G

So the generic part here is between brackets the string with new blue, faster.

G

So buffer star is one helper function to build a store that will contain the bytes buffer that you can use so every leaf of a store. The thing that you you get with star.get is going to be by its buffer, so you can just use your regular writer functions like like if print fprintf.

G

So you write all the things you want and then you call down when you call down you get the d from the previous state and it allows you to print it. So there's a private printed function, but just responsible for doing the output. We are matching below.

G

So, for the first first round of the star, I just get a and print loa inside this. This leaf this entry, if you prefer so it means that after dot and print d, I should see that I've created a with the content load.

E

G

So that's what we see we have put just a line below here then I do another cycle that I simulate have no changes. So actually I generate exactly the same thing, so I can reset to say: okay, I'm going to write a new set of data and this new set of data will be the same as before, because it's get a and right and the right load and call down on printf, and so here I should see that there have been no changes since we produce the previous state.

G

So that's why it's it's printing same.

G

Then I do a third third event. Let's call it an event where I get a new thing. That's b and I write lob in and I still have my a with hello a inside. So all I see is the d that I've created b, but a is still the same.

G

Then I get another event: that's kind of torture in the store where I I have two actions where b is removed and a is updated to ia. Instead of hello, a.

G

So that's what we see in the div and so.

A

A

So so this is basically how so this would require for, like I'm thinking in terms of nfd back and like wherein it would require the internet.

G

Let's go inside just going nft and chain buffers.

G

It's a nfl now, so it's it's been renamed.

G

So that's our presentation of one and f table. That's taking advantage of this of the store.

G

So to pay the store, I just have a familiar name: the change inside it and the maps inside it. So your ip table chains and your uip sets it's kind of this. The same id.

G

And then I I've just put some helper functions to call for the interface and on both stars. uh When I do when I want to call it on the nf table, that's really just just that.

B

G

And uh can it start just uh helper functions to to say, okay, the store maps will be for the kind nf table kind of object, sky map and chains for the chain object in nft. So it's really nft objects that I'm representing here and I'm representing them. Just as the code. I need to define them within nft.

G

So really, really simple! So if you want to see how it's used just go to nft.go, it's a bit uh a bit bigger, but not an actual nft instance. Yes, yeah, it's! Okay, just didn't see the mouse move.

G

So what happens in the callback? It's just that uh I'm initializing and I want to be sure that I will be calling the reset of every table the table for v6 and v4 at the end of the callback. So it's it's clearing the start at the end playing the state and and then it's, uh I think, not that hard to understand.

G

I just go through every service and uh every service will have its own chain and on its own maps,.

G

Every every endpoint will be in a map.

G

And I'm basically just just using right, for instance, if you see for every endpoint iep, I just have them into the map. So I just get my my line. 198.

G

Just before just computing, the ips that I want, but it's all that's all it's doing and and then it's really working uh on vna7 it should be remained.

I

You never mess up the port as well. It might cause the spark.

G

Yeah sorry, there was a question: that's.

I

Like you, you don't map up the combination. Is it a cross product address and ports.

G

uh No because I have a dispatcher tournament, all right for the multiport stuff yeah, it's a multi-part game after I did all those things initially yeah. It raked a little bit of havoc yeah.

G

So what I do is for the endpoint map. I just uh just write the map elements, so I I uh what you can see at line 202 is that just the definition of the map with with ap family destination address and then the elements uh there's this uh different thing on on star items that allows you to call things at the end of your rendering when you, when you call undefeated.

G

And then I just write the entries of the map.

A

All right, so this is basically iterating through this channel and then of service endpoints, getting the service endpoint and then.

G

The channel maintains the whole state, so I just write all the definitions. I need in the.

F

G

At the end of all these, that's a bit more technical with some burgundy and things like that. You can still scroll down because you need to to get after the loop.

G

I just run the different ring, so it's just going to basically close close all the mouse with you with a curly bracket and then I create the dispatch chains, both shooting chains, and then I say I'm done and when I'm done, I just go to nft and write all the all. The updates that I need to write.

G

So that's what you can see.

A

G

Rendering table in 374 yes under your mouse, so this one is going to to to write to nft so to the pipe uh the updates that has to be to be sent. So if you go to this function under the tables which is below, if you want, but if you can click.

G

Yeah, so this is where we use the.

G

Red star, so that's a little trick in the left servers because I may want to have a full receipt if there's an update issue somewhere, but the nominal case is not fully sync, so just ignore that- and this is where I use the diff.

G

So the first thing is, I do is empty everything. That's been deleted, so that's what you see line 532 yeah. So it goes through the deleted items and just flush them. So every reference is cleared.

G

Then, for everything that's been changed, I also flush them because they are going to be written.

D

G

Yeah and then I I do, I take all the changes and and just dump the definition of everything. That's been changed.

G

Every table every map every chain, every map.

H

A

Yeah looks uh like at least from the floor. It looks pretty neat as compared to like the user space port, uh but yeah.

G

I think there'll be.

G

And you you just see which listeners have been deleted and then you can just just call clues to the listeners and everything that's been changed will ever have nothing to do or if you have been you have been created, you need to start them.

G

That can be something like that.

G

So, now that we are over the example which you can go back to to register clients.

G

And just maybe go through the api itself, the api. So, let's get to yes different. I think it's pretty good way to start.

G

So that's a generic one, so you have a key that must be ordered and you have a leaf that must fulfill the leaf interface. That's on line nine! Yes, so you want a function to reset the leaf content and another one to get the ash of the leaf content.

G

The arch can be tricked to be addressed and integral you increment. Well, things can be done like that.

G

And then using the b3 to implement the functions. That get item is going to give you uh an item. So the item is the key and the under value and allows you to add those functions. We've seen just before.

A

G

And if you are just interested in the leaf, there's a shortcut function, that's called get and it gives you relief directly, not not the item, it's just basically giving the value calling it item and returning its value.

G

uh Home deferred is just a way to run. The third on every every item of of a tree, then, is what I said. It's going to compute the ashes of every node that has been touched.

G

List is going to give you everything that has been touched in your current run. So it's the whole data set, regardless of if it's been updated or not deleted, is going to give you what has been removed and change is going to give you only the items that have been changed.

A

So basically, people started the b3 data structure and trying to calculate the hash and checking if it's touched.

G

I use the b3 and then I I add my own logic of of of the thing objects.

A

G

A

Yeah this, I think this puts things into perspective like how it internally works.

G

Yes, it's not supposed to be very complex. The reset may be interesting. Only a lot of of internal states to to make sure that things that have been deleted are marked uh are removed, things that were uh and just uh put every item in the tree in a state where it says. Okay, I have been deleted. It's kind of that.

B

B

What would be the.

D

Optimal size of a node I mean because we are doing the hash on the node. So what would be the maximum size for it to work optimally?.

G

If your changes are big, it's probably harder for the system to under that after, but I don't think it does have any impact on that, because you still have to ask the whole thing you your route so.

D

Okay, I was just thinking because uh say you have a big string there somewhere and then, when you have to calculate a hash on that, that might be a little little bit slower than if you have a smaller length data. So I mean I don't know, I'm just thinking.

E

Upper limit to that.

G

D

G

I

By two you're, just going to have to hash twice as much too exactly so so the hashing is is really quick right. As long as it's tough in memory, which I assume I mean yeah, I think I mean.

G

And I'm using xxh, which is uh actually like, 10 gigs, a sequence.

I

Or something like that, I think what we really need to be afraid of at some point, and it's not that it's not the code before, like a certain comment when we get when you run with a hundred thousand services and a hundred thousand and fight that the whole way that all information known everywhere will not sort of really work. Well, we need to have another scheme, then that you don't have to know about all the service to endpoints resolutions everywhere.

I

You need to know the connections that set that, but the resolution to find the right endpoint for you, maybe that should be distributed and so on. But that's I think those problems are still years ahead of us, but that's uh that's your work. Yeah.

G

But it's easy to do.

B

It's really easy.

I

To do I mean you just divide the uh the service addresses and say the this part of the address sets for services is served by these servers and so on. So it's it should be quite simple to do, but no I like what you've done. I agree.

I

Helpful for you yeah, but if you haven't then maybe a little later, we can talk about the multiport. So uh we have some problems there right off, yeah.

G

Is going to give at least things that are much easier to to change uh and to adapt to those changes.

I

Yeah this I mean we see this on every release that changing something I mean: internal policy, external policy and all these things. That will, of course, then map on what you think the the set of endpoints is for this internal.

B

I

So you put this code together in one day.

B

Yeah, that's one of my feelings.

G

That's why I have to to be careful with people. It happened to me too many times like.

G

I always remember a time when I was working for a customer three days a week, so I was there on two days at your office and when I came back in the morning and a developer was asking me like, I have this problem: do you know what it can be, and I I was just like saying hello to everybody and and came to him just ah yeah.

G

I think it's there and you just found that and that's good and then I I I continued to the next uh next one and then this project manager you've been a bit rude with him. He's been struggling with that for three days.

G

A

G

A

G

A

Great, and and thanks thanks for like walking around like you, know the entire piece of lipstick thanks thanks for that.

G

We haven't done the whole thing, but just just have a look at, uh for instance, buffer leaf, which is quite easy or json leaf. If you want, if you want to change it, so you can see how to implement your own leaves if you, if you need them.

G

So that's your purpose by the investor and then there's the structure of the leaf. It does have get and set as method. So you can just get a value and set the value.

G

Reset is just putting vmt value in value and and the hash is just using json to do so. As long as you have an object that can be represented as json.

G

To have a tree after whatever.

I

You need it's serious at first to jason and then hash it yeah all right: okay, okay,.

G

I

Lines of code, exactly what you said.

I

That people use so many do wrong when they hash something they don't think about the uh the padding and then you turn on the bug and put the bag patterns in the non-new spaces of the structs and people passion you need somewhere. So.

G

That's why it's good to have version in the leaf, so you think about it once and then yeah.

I

Absolutely and yeah I haven't checked like this. I assume that it's uh it's a really good hash that changed a lot for just a small change right.

G

Xxx is really fast, it's just not uh it's not crypto safe. If you, I think the fastest hash that's crypto saved today is number three. I don't know.

I

G

Speed and spread yes, it's it's safe on on the spread. uh It's just that it's uh it it can be. I think there are some collisions that are a bit too easy to to get for security purposes, but this is not our problem. No passwords, yeah.

B

Cool and it looks really good.

I

So that's why it was fast to.

G

Do this, you know not a lot to do.

I

I have it up. I said I was going to look at it. I liked the idea with the right right right done and be able to have that I'm just looking at it was another type of problem, probably a different solution. You know when you have a well active replicas and you want to get the changes over to the replicas that are on different machines, sort of different ways of doing that. This is, of course, one way, would serialize and then replay it on the other side,.

G

Yes, let's use full file. Well, that's actually what capping is doing.

I

Yeah yeah, exactly that's like it's real, I mean I don't I mean I don't I don't really like the uh cash db. I mean the uh uh the old c c. Caching function there and I've done so many functions myself when you end up, you know, do relative pointers and faces and then just send those blocks around, but that's just a c plus for us to be great.

G

And that's the last thing in item.go, which is uh something where you get when you get items.

G

This is the b3 things you can get the key and the value with big functions, of course, and you have a change created, updated deleted functions that are allowing you to know the state of the item.

G

Everything is quite trivial and and there's a different thing, but just adding to the default functions, something that can be run and when you run the affair- and you call when you call around the third, it's just going through this, this array of functions and call them and then say: okay, it's empty.

G

That's it quite easy.

A

Yeah I mean this looks pretty neat I mean I think we can add some readme on this and uh you know like add some dust.

G

Yes, and the first thing you, you already have some documentation, because there are example functions uh for indie, for instance, it's just what I've I've shown.

I

Yeah around this, have you don't know for you, michael? Have you guys used like c plus plus templates, or something like that? I mean sometimes when you generics can get you scratch your hands, sometimes there's so much auto conversions going on.

I

It was 20 years.

G

B

I

Of course, I did not use that obvious.

I

And yeah, but it's slow to compile oh yeah yeah. Of course I mean first, it's gonna have to do the static uh generation of code and then compile the code. Yeah.

G

I

G

You have another example function. If you want to have a look at it,.

G

That's where I've shown I actually had it for 4j, for example, as, for example, any store file like which is just like. Let's pretend we are reading a file and just putting every line in the store to compute the diff.

A

G

So that's another way to have a look at it, so all those things should be in the goddark automatically since our example functions.

C

A

Yeah, I think I'll I'll basically take that action item on me and then add some readme around this.

G

So, yes, you should I'm not very good at writing readme files because of what I've said before.

B

A

Well, uh thanks thanks for getting this done, I think we have five minutes uh ahead of time, so I think.

I

Just um the multi port!

I

So what for when? I saw right? The end point contains the ports right. So when there's a new endpoint coming in look at all ports, that's in there and add them to the the correct address port pair or and then when removing an endpoint do the same thing.

G

uh Do the multiple uh I mean we are not like uh part.

I

Ranges: no, no! No! No! No! No, not port, ranger, that's even worse right! Oh that's the same when you have tc tcp, port 14 or maybe udp port 82., I mean basically, when you have many ports specified in the service and then because the port range maps to the same pod. So if you specify your wordpress so that that's one port definition right, then, then any part that supports that needs to have all the portrait ranges, whereas the difference specified can be that deployment.

I

One maps to port specification, one deployment, two maps to port specification, two and the point three maps to both and they're all mapping over the service right. So so all all pods do not cover all the ports from the service.

I

hmm Yes, exactly.

I

Wow I've seen some k. I actually can think of some real cases, but I mean the test. Does it.

I

Yeah, so um I was put like this. I was really surprised when I read the test. It's like who does this and then sort of okay?

I

That's how it's specified, but uh that's what I don't think capping does handle right. Now we can push this. I mean I can send you the the link to the test and you can look at it and we can discuss on friday instead of doing it now or I mean or next week.

I

Yes, we can do that. There is an issue for it, of course, but I think it requires some some thought. uh If you look it from a pure low balances perspective, it's fine right, because it's going to be there I mean you have the whatever you want to call it.

I

Try, try topple poor, I mean pro to port an address or four top left include the address type. That is the front for the load, balancer right and then it's gonna select the endpoint. So you already have the different.

I

You only need a different sort of front matches and then it's just that the lists of endpoints can be different depending on which port it is I'll. Send you the.

G

Yep, okay, so we'll have a look at that later.

A

uh uh Hanuman or uh wake you folks have anything else to discuss.

H

No, I guess that's it. Nothing much yeah.

A

Cool, so I think we are at the end.

C

Of the meeting.

D

A

Thanks everyone for joining see, y'all, yes,.

D

Thank you. Everybody.

D