Istio Networking Working Group, 26 Sep 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Istio Networking WG meeting - 2019-09-26

Description

- Endpoints slice
- Istio failover
- Config distribution status

A

A

Wonder if the recording is working, look like it.

B

Says it's recording yeah.

A

Now, it's all good, all right, I think we can start hello. Everybody! Welcome to this. Your networking working group meeting. We have like always very interesting topics in the agenda.

A

The first one is discussing a proposal to achieve well I kissed your failover for it yes like and I will probably pass it to too young to discuss his proposal and debate a bit on that cuz. There been some discussions already and second John has a very interesting proposal about again about TDs and it's called end point end point slides from what I read a bit about and discuss with John. It's actually matching the first proposal, so maybe we can merge these two and give adequate coverage and also Mitch later we'll discuss about config distributions.

A

So I see there are three topics: I proposed like 20 minutes for each of them, but or maybe less like- let's say 15 minutes for each of them. Let's try to be on time and please add your names to the attendees. I still see that not everybody's there and Jana. Please feel free to present.

C

Which, because I think it's important contest about my spice before you discuss the second one since I've some implications, I, don't.

A

D

A

Yeah, the two are slightly related, so yeah.

D

But this one is simple and provides context for the second, so yeah. This one probably won't take too long yeah. So basically kubernetes added this new API in the 1/16 release as an alpha called endpoint slice and the long term goal is. This will replace the endpoint object. Basically, what it is is endpoint with a few more fields added and it's designed to be extensible. So it's actually not even part of the core API group anymore, because it's designed for things such as east yo to utilize them in our own ways.

D

So they added a couple things like apology, which has first-class support for a locality, and we can also add arbitrary metadata. We went there, they had different address types, so we could probably replace our current resolution where we have. You know some host names that are resolved in DNS and then some IP addresses there are some things like missing fields that we have right now, like the weight, the l7 protocol and export to, but I think a lot of those we could.

E

Replace just by annotations or labels and if I can provide some context. The reason kubernetes active this API.

C

Is because endpoints what say you're using before it wouldn't scale to larger number of hosts? We have exactly the same problem where I said be centrally, doesn't scale to large number. Of course, if we get to a thousand two thousand ten thousand endpoints editing formation, expansion, for example, becomes almost impossible and the size becomes too large to to set in Protoss and and so forth. So that's how an kubernetes is going to scale and because I was actually inspired from history API a formal requirement.

C

Instead of duplicating the API, we can just adopt API and gradually switched over to this, which is most scalable and second sighs. All.

D

Our deployments so the reason.

E

D

Is because before there's one endpoint to one service now you can have many endpoint slices to one service, and so, if you change one end point, you just have ten update the one endpoint slice I pick that has maybe a hundred endpoints rather than this huge endpoint object, has like ten thousand and points which is for scale but.

A

This is basically a step in the right direction for what we called incremental IDs right, like the ability to receive one endpoint dealt at a time, and here we will receive a slice which is the best.

C

F

Is Charlotte incremental.

C

It is charting the endpoints in multiples. My husband means there's not going to be incremental in any way, but just anyways. That's that's scalability I do so far. There is no other alternative to Z seen from in from any point of view, because what for, if you go to ten thousand, what we have doesn't work yeah.

D

So this side dock was just having a higher level overview of how we can embrace this new API. The first step is just support in pilot. This is we need to do this because otherwise, once people start using endpoint slice, there is no more endpoints created and so pilot will not work at all. So this just we have to do this. It's probably not that hard, and this isn't kubernetes 116. So hopefully we could get it in one for my four, but it is an alpha right now.

D

So I don't think it's really using this, but by bayi to 1.5 for sure we need support for this work with.

A

Envoy, so what what does it change you know are like? Yes,.

D

Yeah, absolutely nothing. It's just a different input from kubernetes. We get an endpoint slice instead of an endpoint, but we still can generate the same config. One.

C

Things that we did for our user see if they used kubernetes 116 for Safari to work so being open and second, they will be able to use mesh expansion or other features using endpoint slice and no longer have to use semi sentences. You will use a first class benefits API and, as a side effect automatically, it will be integrated, never be visible. To so, prometheus would be able to scrap it, and so.

D

Forth yeah- that was my next point, is that once we get support, we can also just complete the replace service entry with descent point slice. It serves the same functionality, but it simplifies a lot of things. We don't have our own C or D, that we have to maintain teach users about document etc like cost. Instead, we get first class community support, especially things like Prometheus will work out of the box and they're really like almost the same guys. So it doesn't really make sense for us immediately.

C

Being defined us one year from now, when we 169 is a minimum support division. So when people who is 1/16 can already start using it, and people who use older veterans were continuously sent you of course, education, for it will be one year kind of.

E

D

One thing this also adds is a new ready field for each end, point which I believe was the only thing that motivated the synthetic service entry ceremonious.

F

We can might we.

D

Replace the synthetic service entry with this as well so Reggie, yes,.

A

I'm sure this can replace the synthetic service entry I'm, not sure how it can replace the external service entries, because please.

C

Note is not just about the existing and.

A

Basically, the updates from like, let's say, college, to pilot right and all the processing code in Pilot and I. Wonder if there is a similar like effort in DDS, so that we download we don't download the full list of endpoints. That means.

D

That Bob is an exporter station yeah here I give some examples of service entries and their equivalent in endpoint slice. It's kinda hard to see on one screen, but if you look at them they're effectively the same thing, so here's one for DNS service entry, here's what it would look like as an endpoint slice space is the same thing.

D

E

D

This one is really it's completed, image that require some changes yeah. So so what is actually right now the address types, only sports IP, but it's still alpha and I'm, pretty sure we can them to allow a hostname. So we can do it. The DNS stuff, as well I, talked to some of the people on the community side. It seemed open to that, but one floor would go weak, yeah.

E

One for me we'll just go.

D

With support and office more in the future, so the next thing is kubernetes now has scale of endpoints and we don't because we still send the full set of n points to on what. If you don't think, that's a big deal, you can look at some big numbers that I put on this table.

D

Basically, if you're going from a thousand and points to ten thousand you're gonna send three three hundred thirty two billion endpoints, it's just basic and squared problems, so we should come up with solution in the near term to do some sort of incremental or batch TDS. One potential solution for this is what yan is going to present.

D

If you could have some sort of aggregate cluster that has so many sub clusters, so we have like her HTTP bin cluster and then that would be composed of a few different sub clusters which have their own independent endpoint sets. So that way, if an endpoint gets updated, we don't have to send all 10,000 endpoints. We can just send one chunk. Another one is if this was just. We had a first cost support for this in EDS. I. Think there's been some efforts around that, like the Delta x, TF yeah, but.

C

But it doesn't solve the problem because it is that sort of proof. One incremental update illusion to one extra endpoint is a initial, send it still the full list and it's still probably larger than the G RPC buffer size and yeah. That's what does work so it's it's! It's a palliative, but is not going to solve the problem.

D

G

So it's pretty much! No, actually, how you use a delta X is you can send a small set after n points first and send as you incrementally it. That's basically how you implement that in peril, but.

C

The original said: I mean initial. When you connect a new sends the first, the first connect you still need to send the entire set. No, no.

G

You can you can send us, like small slice of that and do add more in the like subsequent incremental exodus that basically doesn't make any difference pick between the aggregate caster right well,.

C

It's not a lot, we don't have it supported right now, so we.

G

Don't have the incremental XDS it's not fully implemented so, but that, basically, you can implement her not to send in that way. Okay,.

C

Let's discuss it only discusses the second topic because that's where we discuss options, and so, but this is one part of the slide heading plus the slices is one implementation option to address the scalability issues that we have. That's and yes, maybe that's another option with incremental. So.

A

Reason when you're saying it's implemented, but not fully implemented in envoy.

G

Mentholated Spain has implemented, but I think you try using the review process on that yeah.

H

There are two layers of the old how wise kind of the wheel at my side, so your watch only see the sliced differences, but actually between my and pilot, you will see a huge message that contains money and point, and the next step is to allow that the transportation between pilot and a way that this.

C

Is not implemented, so we can still make a choice to go with with a cluster slices or we go into its incremental I. Think, since it's not yet committed.

A

There was a bit of echo sorry so I'd like to hear also from other people on the line, their thoughts related to this particular proposal.

I

Endpoint slice support for the kubernetes adapter, that's fine, I, don't think there are any state to replace service' tree I mean our deprecated service, intricacy I mean maybe pod VM expansion and so on. That's great service entry represents an actual service. It's an object. That's actually declaring this is the service with so-and-so parameters and properties, and endpoint slice represents a group of endpoints that belongs to one or more services, so overloading that API to represent the services, as shown in the examples here, he's pretty wrong, especially because how you can add additional external entries.

I

That's that's not the, but.

F

We can definitely like either you.

I

Need be, and if there are an actual use cases that vapor people come and say you have a country with 10,000 endpoints and you would need to have some sort of like that Unicode you can always create an additional like you know, referential point sets or whatnot treaties, but I don't think. We've come across the case that somebody has come in and said: I have 10,000 points in one single service entry and adding and removing it is creating an issue so.

A

Because service entry itself, it's a merge of all thing already, so it's able to merge endpoints, so it in this case it would merge the endpoints slices right, but it doesn't necessarily mean that the end floor, endpoint slice, the first speeds or only gets rid of the need for a service entry, yeah.

I

But as such this proposal, I mean we should definitely support. The endpoints lies on kubernetes, because but I think, even if we support it, I think it's pointless until we update pilot to like not do the incremental ad as the town. Why? Because that pain point is in terms of the differential, should have been higher.

D

Pointless because it's neither you either have end points or end points lice. So if anyone on their Buster I guess you have even mean.

I

We will not reap the benefit of this thing on.

A

I

A

Not fully but I think we will reduce some of the pilot, CPU and processing and blasts. We will do it like incremental in terms of effort and people who are working on it, because it's a different set of people working in pilot and galley, which is the one working on DDS right, so that would need to change also in LA Bradley, to create cosmetics, 37 points with slices instead of actual services and end points in kubernetes.

A

Let's not forget we're trying to keep a certain abstraction in pilot, an abstraction from kubernetes as well right, so that would likely go more into golly than in right.

C

Now, right now we are not using the end points from guys, so it is not something that we have important in any production system. We.

A

Are not using, and one of the main reasons we are not using was exactly this. One was because we would get the entire list eventful. Anything I think that pausing goes sort of an O n log n, but.

C

I, don't see I, don't forget to the scope tool to switch to MCP, to two endpoints in one for and then try no for one for we need to move to two endpoints less, so we don't break support for one six I. Don't.

A

Like I, don't see, I don't see the benefit, then.

D

It's completely broken, they can't use these. Do so, we'll give them the option. You either need to change your entire cluster or you can use ezo and that's not really an option. We want to make people decide on right. All.

F

J

I

Start using this, then.

D

We have this blue. This.

I

Is just an internal stuff that we have to anyway add and is in yeah after I? Reverse it, because yet another way of reading endpoints come kubernetes, that's it I mean if they wicked one and add one. That's this part of us maintaining the interfaces with kubernetes left leg way. We do it by the other systems, though, but that whether we use this and galley and all the other things in the sky. Oh that's a second, that's a separate discussion thing, but yes, we still need to add support for this. Normally.

A

About we're duplicating effort here right.

A

Dependencies on kubernetes were just like changing that code and then we'll have to meet the equivalent in golly all.

I

I

Yes, that is that writing that I mean not that much of an effort in terms of reading from this API versus, like you know, just adding it up with small API thing, but the actual streaming stuff is where big.

C

Abilities can be the same because even if we need a link cardigan an incremental support at this level and for slicing or other things, so it's kind of let's first start with. You know the simplest solution. We can write I.

A

Will take this discussion offline because we already allocated the 20 minutes to this topic and since I, let me check first, if other people then have more to say on this. My.

B

Only comment was it sounds like we're extending the dependency on kubernetes in pilot, which should be an anti-pattern.

A

Right right, so that's.

C

A good point, but just like service engine other things, it's not necessarily the spec for for slice is not necessarily, it is exclusive. I mean it can be used in any environment. It's a protocol. So that's what we're discussing within CP I mean it is sent over MCP. It can be translated from any source. So harsh.

A

Cuz then I, don't think anybody agrees with the idea of removing service country and using the slice. So III think there is like a misunderstanding here and got one with: why don't you America boat service entry? What.

D

But we can merge endpoint slices and we can get the exact.

A

Same functionality.

D

With endpoint slice, a service entry, yeah right.

A

So this is, this is basically the endpoints which is right now implemented in two places. It's in kubernetes plugin running directly in Pallet, and it's also in gully right and what Shannon is saying is that we keep updating that code in pilot instead of getting rid of it and going with the galley and MCP, and the mCP can send the service entry with the merger ball slices of endpoints, which are the kubernetes slices.

C

Doctrine is good to go again with water, but balance is using an avoid duplication by.

D

Using I, don't think your goal is, as Easter is to add a duplicate API of a coronaries one right. It's no api's, like virtual service, because you can't express the same thing in kubernetes, but now they have an API. That's almost identical a service entry is literally inspired by service entry, so I don't think it makes sense to have a API. That's essentially the same thing as the endpoint slice and just make your users in there and get another API. That's subtly different and doesn't work with.

C

For meteors and other concepts and how insightful than it is tea water, because you can still use exactly the same protobuf in any environment that doesn't use profanity, so we're not losing much about. If the.

A

Ep is : point slice, I, don't see how this can mean the same thing as a service.

K

Remember the service has two parts right: it has the collection of endpoints part right associated with a name, and then it has the. How do I talk to it part right. The reason why they're doing endpoint slicing kubernetes is because of change. Velocity right within large groups of endpoints is a scaling limitation. We're actually subject to the same scaling limitation right, particularly when endpoints have labels are duplicate. Labels on every endpoint, fully denormalized is very expensive for us and is only going to become more so if we try to model.

K

You know the reason why they're doing in point slice is to support a communities cluster that has thousands of nodes and hundreds of thousands of pots. We have the same physical limit, so it exists for a good reason and we share the reason so no and as decoupling are allowing for a functional decoupling of the. How do I talk to a part of service from the one endpoints does. It have part is probably a good compositional change. Now we can model it differently to kubernetes. We have an abstraction layer where we do that.

K

There will be a question that what's the point, so we have to consider that very carefully, but I don't want to I, don't want to just say we're only doing things because kubernetes did an API change. They did it for a good reason, and it's not just because of that.

C

K

And if you look at almost any service discovery system that scales they have the same thing console has the same thing: Google Cloud has the same thing. When I looked in other clouds, they have the same pattern for the same scaling region.

B

Agree with excuse.

F

B

I, don't disagree with the using core resources from kubernetes if they serve our purposes. That seems fine. I just want to make sure we're not avoiding the desirability of the way pilot gets configured is via MCP and not in another way.

K

For something like.

B

These endpoint slices over MCP that's great make.

F

Sure that we're.

B

Not like adding support for pilot to get additional configuration directly from kubernetes I. Think because he's working on some things that actually push that forward faster yeah. So.

A

So is there a proto definition for the kubernetes? How will we integrate that with MCP? We.

K

Will have to create one as a product.

A

That looks exactly like the service entry, then no.

K

Because it doesn't have the how do I talk to get part? Yeah, remember, service and research, I.

A

Think it's a very nice concept to have in an API.

K

Yeah we have to have that part right. Soku granny's has both endpoint slice and it still has service. So.

A

K

Each we exclusive for.

A

That purpose, so why it's already a merger bow thing right ever.

K

Since technically it's multiple right, but there's expense in that merging today, because we flatten labels down onto endpoints so which.

C

Ip has a set of labels in the new API. Is a set of label Cincinnati's to five days of contact, understand.

A

Now you use at least we agree- we've discussed many times in the past about changing to service century because it doesn't match well, this incremental of endpoints need so basically what the proposal is actually to revamp the service entry to Matt slice.

A

K

That's what's happening right. We want to support the that there's the the usability thing of like. Don't make me update two things if I'm doing something really simple right, which is why we allow for inline endpoints in service entry right, but we need to support a model where that can be decoupled for scaling reasons. All.

A

Right right, so I think put this way. It's very different than we're getting rid of the service entry and the replacing it with an points like that's not.

C

K

Well at scale suggesting that people do a mortgage aggregated thing so.

A

If somebody wants to model an external service, they might want to think of that in terms of a service with then maybe a single end point, and not necessarily in terms of an end point slice right.

K

I mean this is like you know: envoy has the same model as well right, they have clusters within line points right or you know, direct resolution of its own end points or it does it delegates to enact an end points sirness, which basically provides it what it needs. This is the same logical composition, remodel. You could be there how.

F

The inline thing, or the other or.

K

The disaggregated thing, but for systems that need to scale up, we would have a recommendation that they don't do the inline play.

K

Right if you have a high rate of turn of endpoints right updating service entry, this is probably a really bad idea.

C

Suspect that someone is using same place twice, committees and all the services are committed if I hate. This will actually immediately centuries'.

L

C

To see it so it's a bit more interoperable.

K

That's probably my mother listens the same debate. They have to kind of deal with the same transformation internally as well, so we.

I

Had 20 minutes right and yeah yeah, okay,.

A

Sorry so I will have to continue the discussion of line, maybe with some comments in the design document and now, let's move on to the next topic, so young, please feel free to share your document and present your ideas.

M

So we are going to solve the problem that no dependency between the services from the from internal and external services. They come from different service chiefs at discovery mechanisms so for internally Eusebius and for its terminal. We use DNS resolution and.

M

Now we solve because for EBS, so if we use EDS only, then we can only send the IP address directly without putting the command the host name in the ideas response. So we need to extend service entry to support that.

M

So what we are going to do is that we stand the service entry to have some load balancing hosts, which contains post mod internal host name and extraneous name and provide some priorities to denote the properly full of oil fell over sequences and, for example, in this example. So they the traffic, will those to eternal for Baca come first in there. If this host or fail, then took the traffic will go to phobar calm, and even even we aggregate this one.

M

So user can still apply the destination row to the host to to the hosting or not AME. Host, like user, can specify details, contacts to the internal format account and to foe, become with different, like HTTP options or TRS context.

M

And that's there to there that so some some type some some alternatives like using like CD s instead of easy audience. That's because city s Cydia! So when you say CD s response and then the clusters will be recreated in fractions, were like lots of simple cost and doing the s real.

M

Why am I not like, for example, when the pilot, it's very far from the oil, the tiara, which is not it's not ridden boat because it responds to following so envoy, receives IP address and the IP address may not be reachable by the owned by down boy and that's one, like maybe ds2 DTS resolution, so that's first, that's the implementation may be similar to Africa cluster, but the conscious that way. India's receives the response there. So the ideas routes response will not be applied immediately because it need to do some resolution. Yeah.

C

It's good to separate the discussion on API if we need to make any changes aggregate in such a limitation.

I

Of which that's probably what they use new schemes that drives this thing, so the reduced.

M

I

Not really use case and then go down into the India. What might be needed? Okay,.

M

So the use case is that so we have a customer that have warmers has internal services and external services, and they won't say they they want to. When the internal services are down, the traffic goes to it so knows for back, fall over to external services. That's a deuced case and.

C

And in general, the problem that we are trying to solve is that we have services that are implemented in local clusters and can be resolved, the TDs and load-balanced and so forth. But the exact same service can be implemented by one or more DNS. Entries do not have what we do not have. Access to psychiatric assistance, I need to go to to DNS resolution, and besides fallback, we also have the concept of ro curricular balancing.

C

We also has a concept of where, when we may want to apply pretty much all the rules that we apply for for everything else, woody network so.

A

Here the basic problem is that the ideas response doesn't allow a mix of IP addresses and hosts that.

G

One is that feature is requested a couple of times in a void, Committee and it just not just someone is not implemented yet doesn't have the bandwidth to do that. We are we, okay with you to add that works for sure, okay,.

A

So we wait, we definitely have more than one entity asking for the feature right implement what.

I

Is the other thing to notice that today you can just ship a strict DNS cluster with IPS and the DNS name? And you know the IP addresses will just be a no op for DNS resolution and the DNS names will just get resolved as usual, and that is a solution compared to which requires more 18. Just it call it request with a small fix and pilot such that I could be able to define hey for this entry with the same host as the kubernetes service, name and I can add a bunch of other endpoints.

I

Then we should be able to merge them that merging all.

F

I

Existent pilot yet, but we can actually we have to do it anyway, and we can add that merging logic. And if you had to add that merging logic, then it becomes a very simple thing of shipping. A simple, strict, DNS clusters by our serious, which has the for eyepiece and the you know the three accord minion fall back, but.

C

That will not scale and very well, because if we put it in CD s is a serious response in line. We need to send full CD s responses and also it doesn't scale, because what we discussed earlier was that this is not smooth 0. No, that's.

I

C

That is where I I.

I

Talk to you can I talk to all of the other Walmart guys. This is not a large scale scenario, nor is this one where they have thousands of endpoints of that thought. This is all core stuff, that's happening in the Walmart, like the stores and so on, where they have few pods that is running the service is on coupon. It is plus one or two VMs actually supporting the same thing on VMs, and they simply want to be able to pay low from the pod into the vault. The pods are unavailable.

I

They do not have a scenario where this is running across 10,000 plus pods, and they have like 5,000, plus DM n points and so on, but.

C

I

Always the case, because if you do it by scale and you create a more complex solution, which is not a common case- and it is always, then it goes in a completely different path, and until we hit that scale thing we shouldn't like over, complicate a solution that gets into more much please. But if we don't require a change because the command key. This is a change where we just don't even need that scale. Then why I think there.

A

Is like we can achieve this a bit faster by making region.

I

Small fix and pilot to allow actually merging service entry. They should be able to satisfy this use case today with zero API changes, zero changes in Hanoi and any, but so it's DNS entries in ideas we can actually like achieve. You can start shipping that same cluster as in EBS cluster, rather than a strictly honest cluster, and that becomes if.

D

We do this. This is strictly worse than resolving the DNS response right, because now we have to send the full cluster and any time a single endpoint changes. We have to drop all the connections, recreate the cluster.

I

Endpoints changing to the extent that, when, when every time an endpoint changes, if it's a pod, that's coming up and going down sure yes, we can just send that cluster on. We can fix that seem serious to actually just send back series alone right, there's, nothing that prevents them actually.

A

When we change the cluster, what's.

I

That mean what happens in the add in the moment, endpoint in a cluster as a connection.

C

Responds with additional endpoint and then we're just on the endpoint. Oh sorry, Steph Nick, necessary codex page.

A

C

A

I

No changing ever just remove an endpoint if.

C

It's removed from the endpoint is: how do you and I be.

I

C

Respond that says.

I

Remove this endpoint you sent for each classified.

C

Opeth, only for the process are changed if one cluster is changing, we sent only say n points for that particular class parent and the discussion we had before about slicing it is still applies because we want to slice the EDS responses run into high.

I

But then we can do anything even an envoy today, right leg, even with by changing small changes in our wire to make sure that when the CTS clusters endpoints are almost the same with only a new endpoint being added or removed, we can just explain finish it on work to basically not drop connections.

C

I

You just wanted to do this whole shard, business and so on, and so I'm in it's like. Yes, that sure that's also the equally case. Yet she make the CDA stickiness thing to have this incremental property. Then we don't necessarily read yes, but the point is that none of them would require another cognitive effort for people to understand what this new type of resolution is not have to, like. You know, figure out what, if I, declare sub such here and there and how do I actually reconcile them and adding it.

I

Another type of this whole thing and it spirals on its own, set up a complexities worst case really wanted. This way, you can always go uses on rifle to API, to create the special clusters or our customer for those for those services where they want this hybrid kind of load, balancer, which still also eliminates the need to like complicate api with with this kind of a one-off scenario. So.

A

I have a question: if we go with Sri Rams proposal. Does this one preclude us from later implementing Yance proposal.

C

A

The problem are, we have to agree that Yuans proposal is fairly involved. Right, like it requires G. You.

D

Know, I don't think we've concluded what is yall's proposal. We have a bunch of different ideas.

K

The assumption is that the client has done DNS. Resolution of the host is trying to talk to. Yes, what is original destination network is the whole like.

I

It should oh, that's actually even better than it can actually add. So. For these cases we can just simply chain the entire thing into an original destination thing. I.

K

I

It won't work because here's the thing, though he what UK wants is that he wants like he wants to pay low to the VM. Only if all the pods in the cluster I.

L

Know that's only because we capture the bat.

K

If you talk us to the web right, you would have gone to the endpoint.

C

Balancing and boys come out of the way, so we don't have any emotion.

L

K

External service.

C

K

Something that they had programmed in both they visited on emerge right. What would TNS have resolved.

C

Those loser costantino right.

K

They want quite an Dina's, alias.

L

Was external IP is not forcing turn right and they have knows you talk to isn't aware of the external, also.

I

Said he's actually thinking of introducing another in aggregate alias for these kind of services that basically says that, like I, don't want to even rely on like tell people to use the chaos I'll just use, uber calm, which will either is all to be MF, curious or the K. It has internal service I.

D

Strongly agree that we should not have any anything to the service.

I

Yeah, this is not something.

D

We need for the proposal of aggregate cluster. We could still urge them with the usual, not renewing so right now today, if you dim to service centuries, one with DNS and that's not DNS highly actually will send the hostname in the EDS response, like it completely breaks on. Where rejects it, we can change that.

I

But most amazing penis clusters or we could do it so it does. The aggregate cluster.

G

A

You please speak up cause. We missed you yeah.

G

Well, I mean like adding dear supporting media's, is it's totally fine? I mean it doesn't have to be like strict TNS and in ninety nine point in the city, s I.

C

Think the primary question is: what is the best solution? Long term I mean shouldn't. We have EDS extended to support this DNS, which is proposal one. What what sridham is proposing, which is the extend seediest to get rid of EDS completely and.

I

Be like under strictly known as clusters, if the performance the company shoe but like not the the thinking here is our large-scale extreme shown and that people will just imitate the few, but for more people who are using standard service entries with DNS they're, not complaining about, like you, know, connections being terminated or so on.

I

Fine they're, using external services, Google, APIs and so on by just applying them all DNS.

C

Changes we have design for future and for large communities effects. The Train of people are using it. Small numbers doesn't mean we can so.

L

We would want to be able to Lego endpoints.

K

Tomorrow, how they were resolved right, because that works with all the baseball bouncing.

G

K

Other features for.

G

Different, let us context here, settings in a.

N

G

G

Labels into endpoint.

N

See less contact selection on.

G

N

Glass that we could have some and no I.

G

Don't like the inconsistency of like by putting it like different, like tier settings, into aggregate cluster and have put in per level section. But.

C

It's more certain in the cluster is a lot of options that allow you to say that that particular set of n points has a number of ways to connect to that. You.

K

See if the general time of 0 is that they're going to combine features two endpoints using the label selection on the effluence right. So if I have multiple T, less contacts with a label selection yeah, all right, then right.

C

But what is the problem of thrusters many points is another.

L

Way of expressing the anger concept.

C

L

It's higher I, just don't have names yeah.

I

M

I

M

We need to like to express HTTP option at other options.

K

M

K

So design, I guess the question for you is are other things you know unjust. Tell it like. Is this pattern gonna be for everything every capability to cluster, including protocol selection and other features, and not just like just 40 laps and.

C

What's the implication of because again, we could discuss that is better to have the endpoints kind of sliced and diced and have different labels and the top level versus putting in each endpoint a sleepers yeah.

A

Is called failover right, so the youngins proposal with the aggregate and the cluster allows you to set a certain priority here and I'm, not sure if what Shriram proposed allows the same.

A

I

A locality based failover, so the idea is that you could declare a service entry way. You specify the locality for your internal parts and a separate locality for your BM, based like in the DNS endpoint, and then you just defined l, think that says like hello from the porch, the VMS, when they all fail and that's it and that way it is it works. Today. People are doing the same locality based like no pail overs and what about the road balancing today and so that exact same principle applies from the ok.

A

Thanks for clarifying, so that would not be any like an area of concern. Then yeah.

I

The column 3 o'clock creates a top level cluster which in turn has two sub clusters, like a smaller VM classroom.

F

I

There will be total of three clusters. That means to be declared and no money sucking about three clusters. We have a bunch of properties on clusters from protocols pls context to subsets to in a what not and we now have to start resolving. All the you know the interplay between all of these properties across three different clusters as to like you know what happens when you define, but not in the other, and, for example, if I, just a bunch of and together.

A

And you have like longer term goes here, especially to align this with the endpoint slice right. I get this. On the other hand, we need to find some practical solutions for a so so urgent problem right.

G

I

So you know you can robert pants at the walmart folks like they wanted this functionality for they extending the mesh, but the solution is dial-up much more simpler than going with aggregate cluster. Unless it is a real, like you know what it called showstopper fail issue which does not simply talking to him even.

C

If I am certain, we go into with the solution again, we see to have a design that works longer-term and satisfies scalability and integration features in.

I

The longer term can't we do what like design, assisting that support and adenosine EDS, at which point that we can offload a whole bunch of serious stuff out of into EDS and we'd, get best of both worlds right, I.

I

E

I

As well, vmn.

C

G

So we already do a lot of like n points group and like a subset lo, the banner and like the things going on to that. So like overriding some black castor level, setting by like having selecting some subset of endpoints, you can do that like in in my as well.

C

So the conclusion would be to add some DNS entries is a hideous response and it would be fine if 1s or what, because I.

I

Mean, oh, that I guess depends on the resolution type, but I think one just be like sakina's. We just resolved it, get a bunch of IP and.

D

I think the EDS response actually already has a resolver name in it. I don't know.

G

People request to have like EDS response: half the DNS risk with his over name or do whatever they expect. Dns resolving and again I.

C

Don't know how long it was they got in getting one for I.

M

Don't know I ever talked about.

C

F

With Harvey.

M

But he's not, it doesn't support that to just use he guest respond years.

F

What decision word so.

M

He says because yeah because he things that that point is concept equals to aggregate I.

M

I

About this, in this meeting, he said like animes about a difficult class does either so that what he.

K

Listen you're the one who made the argument that we should align with the existing envoy pattern of relying on labeling right because that's going to use to select other features onto these endpoints. So if we are beholden to that requirement, then the only way to achieve that effect. It is d LS in EDS.

C

You can still video make money and have separate classes, interaction.

L

Between class sizes, those.

C

L

Things that is.

C

L

This aggregation happens, I mean you types of locality, I'll be won't work with that. No the existing or the wrong way features. We already allocated you.

C

Think to blasters, you put them together and you treat it as a single cluster, but you still send to different and you may still have some different properties across the level they.

G

Still have to locality, and so.

C

You just much you think, the cross that one cross that will merge them. Yeah they're equivalent, inter you sends pds larger ideas with more label, so you sent slices which cause the business slice, and then you put them together in Revit the only thing sliced our endpoints with labels yeah, but you can slice it further. If you choose to that's kind of the nice and I think that's. Why surely like at.

K

Some fundamental level you have to get down to an end point having labels, because that's how you mind yeah things, we do what's fair specification, but.

C

The reason why you don't have to have this kind of right.

K

That's an API change as far as always concern.

C

L

Yeah posture I say we need one.

K

Cluster source they will yeah for and therefore, by definition, they merge posture. They have people represent the same concept, any single resource type, yeah, possibly, but otherwise that have wild ABI problems. You know so.

O

F

K

Go argue with Harvey what I'm saying.

F

K

Basically, that only one of those solutions is aligned with the existing one. What about well features.

G

K

G

Well, it's all the balance between their comics things, you put in the city, s and ideas. Read it since, like aggregating starts you you can basically having the caster Fantas classes and having angry. If you said.

K

That, like that Sheila's contact selection, is really only implemented based on label match against endpoints. Yes, then, we'll just gonna end up with a bunch of duplication around here. Less contact selection for two clusters within the aggregate yeah, which seems totally I.

G

Don't know they selection, the selection part is how do you specify, based on the level, is specified insidious? No.

K

I, don't understand what the criteria are or the thing it matches against and in aggregate cluster right we have effectively to cluster specifications ready three, not two. Three, each of.

I

Which will have to have a TLS.

K

The attack selector.

P

M

Top labeled Casa, it's actually just a concept cluster, you don't. We may like a real connection to to load-balancing, delegate and all abandons into the inner cluster.

K

Ring and don't get me wrong by the way. I actually think if I were to choose I would think I will get cluster with which teacher and emergence is actually probably a better option. As long as you could select. As long as the aggregate cluster could select the child clusters with more flexibility, because it probably scales better.

C

And as the same elements that kubernetes went to for four slices, where were you know, less traffic on the wireless in.

K

Arguments that see less like we wanna do like see less contact selection inheritance from the a cluster, so we have to normalize it down into every leaf cluster right. You have all this redundancy. What's.

D

Possible to costumes, if we go with the DNS and EDS, that's a good short-term fix for us.

K

D

K

Term well might.

F

G

Sanny we can music.

D

G

Yes, that's what I'm saying like short-term good ideas, India s and then increment everybody.

A

Happy she Ramos Oh yep.

P

P

If the, if you can like persuade top, is Harvey, say: okay, we do it. Yes, with.

M

J

A

So here's the problem we went way over time with both discussions. The first one was supposed to be quick and it was like 25 minutes and now we are going on to the third the item and we think, if they're in the room and meet you up, how would you like to proceed? You wanna, like next.

O

Week so it was fairly simple and so I think what I propose is that you look at the document offline if you're interested at a very high level, the 30-second pitch is I'm, proposing we begin hashing, config versions and pilot using those hashes as nonsense to XTS and leverage that data to help our customers understand when a config change has propagated out to the proxies. The details are all in the document and I think relatively boring, but I do address comments as they come in. If there are enough comments, I'd be happy to come back.

O

You come back not physically, but virtually come back in a week and talk about them again, if not I'm, already kind of into the implementation phase, so I can just move forward. If there are yes.

I

Let's do it because this has been now minds were long and maybe wanted to use some semantic base of like something in the version that actually allows us to track whether a function propagated all the way to data plane or not. The problem is that the non certify stand. Are they specific to in an 80s thing? Is the one single non for the entire chunk alright unknown spoiling yet one for sleep, theater, yeah, yeah,.

O

It's a different knots per resource, but the base of your nose will be stills are hashed yeah. We.

E

Will use the the.

F

Hashes, the prefix.

H

I

H

O

I

Can continue with.

O

A UUID for the suffix, the.

I

Problem here is this right, so it's like, if even if it's, let's sail LD s, it's a composition of multiple resources. So if it's a question of like to what granularity do you want to track the the update propagation? If you want to track the propagation of this virtual service, all the way down to the South, that is an onsen in RDS and sometimes possibly in LDS as well.

O

The designs for that by hashing the entirety of the config, your concerns about I'm, happy to come.

I

Doing the hashing the entirety of the well I guess: hash is a decent way to do that. Yes, I'm sure, because the point is that you can, instead of doing the hashing the entire context, which is not going to be of that much use, basically because dependent Duty eventual consistency. Some pilots may have all, and some pilots may not have all, and each one is going to have a different cache and from an end user.

I

The hash they will not see instead well, but there's no solution for, is because pointers create the concept of epochs, which is like you know at this I poke. This is all the configuration that this pilot has and we could have a linear ordering across all the pilots, in terms of not necessarily all the eventually much that the end user should be able to see, lay okay, this on. Why has only confirmation from epoch, one.

A

Okay, sorry is everybody still there.

A

Beyond who started showing us all kinds of things, so I hope now it's everything good, all right, good, perfect, I.

I

Guess the the phones I have this uh box, where we can actually tell them. You said this: is these envoys are an epoch, one which very one only has these configuration, then that would be great rather than had an effect. This idea of like need to have a central depository of what et focus and some form of linearization of meeting from one followed to.

C

I

And so in that point that point and then when you do this hash thing, then what is it that you're gonna return as a response to end user when they ask like, has this conflict change manic so.

O

Basically, you'll be able to use the hash tag or the the tree to answer.

I

F

O

Version each proxy is on for a particular config. The the user interface is I want to be able to run similar to like a cute. Cuddly is do cuddle, wait for this virtual service.

C

The basically will go to one which pilot tasks, a pilot say give me you have a configuration for this hash and, if still has it, it's.

I

Okay, you could get multiple hashes right, because if I wait for a virtual service, where three pilots have the same virtual service, but other conflicts are different, they would have three different hashes yeah.

O

The user actually doesn't specify the hash. They only specify the object, the virtual service or other object that they're interested in pilot then retrieves, the hashes from the nonce mapping on the on the connection, looks it up in its own table.

E

And returns the version that.

C

Each proxy has.

O

C

Would be because we replace push one takes is very fast: it pushes you have the previous hashes and his gun.

O

So you cannot ask for.

C

It anymore you.

O

Know the tree actually can store, has historical hashes, and then we can control how long that history is the quantity tree. So.

A

I have a proposal this like this, is going very detailed discussions. I'm sure not everybody had the chance part should be written on this under proposal, and there is one thing which I may have missed, which is the utility. What is this good for and actually found it in the document right so any feature? Mystic power.

C

Supposed to debugging yes/no.

A

Mugging, it's actually. It helps with the status with K native identifying when mister is ready to receive trophy.

O

A

And they have been complaining a lot in the past that they they don't know when Easter is ready and Michi. This proposal will help with that. Is that correct? That is.

D

Correct? Yes, not.

O

D

It's also normal users or tests even.

E

C

Including my favorite events, sending events.