GitLab Pods Group, 25 Oct 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Group Pods - Router POC demo

Description

Walk through an early implementation of a stateless router.

A

Cool this is Camille myself and Camille is going to show us some router action for parts and by us I mean myself right now and then everyone who's listening.

B

Okay, so I've been uh trying to model something that would resemble a stateless router right now. It's part of the GitHub Workhorse. uh The idea of that is like if a request, that is, that request that is received that is unknown like how it should be assigned to a given pot. The router will try to learn uh a place for this request. I mean they're, not the target part of that request. Yeah.

A

um And so the core problem here is that we'll have many pods and organizations will be on different parts and so will be groups and projects. And so, when a request comes in, we have to figure out where it needs to go.

B

Correct so um basically, it's mostly Works in a context of the browser I mean you can see like seamlessly sweet between both that you are observing on some pages, not all of them. For example, if you open this new project, which is like underpod 1, it will actually register these new project effectively.

B

I mean new group is located on the Pod one, and even if you change like to the put zero, it will kind of reject that request, because it knows that this is on pod one because it already registered through Workhorse the the place of this group.

B

um The same kind of goes like to any other routable entity like any other project or group that is like has a strong location.

B

um It is time to a given pot, so it seems like I kind of got to this project.

B

Everything seems to work fine and I'm, not even like doing anything special, let's say I kind of go now to the send the URL to the new group. Previously, we would have to switch a cookie uh to be able to interact with the new group, because it's like 100.1 right now. It doesn't really matter because router I already knows that the new group is on the put one so I, just kind of type new group and, as you can see like this port location, got automatically resolved because it was learned before that.

B

The new group is um in the code. One and basically, I can now seamlessly interact with all aspects of that. Even with all of these crazy requests that are not.

B

um uh Coked properly, as you can see in this network tab, let's take a look at xhr, uh I'm, not sure. If we're gonna see anything useful, uh that's it's actually. A graphql graphql request is like the security right, non-scoped requests, so right now, I'm kind of doing the hack.

B

The hack is really that if you see an IP graphql request, I'm kind of trying to deduce the the old location for this request based on the referral because referral actually encodes a new group, it's hack, it's not really like a Target solution, but it kind of allows to make the majority of the browser interactions to work. Just fine, because this is like this is like rotable entity So based on the.

B

If I request, graphql based on the referral I can simply use that to find if this is routable- and this is what I get like uh that IP graphql was redirected to the bot one because of the referral.

A

Oh, that's pretty cool, but.

B

It's hack, it's high because it works on the very specific circumstances that browser session puts like a proper referral but, as the result is kind of like works out of the box with all of the and my graphql action cable, because if you execute them in the context of the browser page, it's going to be properly rooted um and you can actually interact like similar theme between different projects and kind of switch between them without having to change a spot specification, especially like all of these dashboard Pages.

B

You have to switch port to see like different data because they are not scoped, but products Cooks. They are scoped right now, so I can kind of seamlessly switch between different ports, and this is kind of reflected in that um how learning process looks like so learning process right now, it's implemented as two different methods. One method is like a special pre-flight request that is being executed by router.

B

This pre-flight requests specifies a method and path info that you want to access right. Let's say flight .js, which is like our uh group on a port0, and it kind of gives you an information put zero prior to here. This is your table entity, but, for example, if you do new group, this rotable entity is on the port one. This is information that is like sent by the Workhorse by the router when it wants to learn to which port to redirect that particular request.

B

This is one way some some endpoints, some controllers, simply are forced to use a specific both like an admin section, it's forced to use port0, always because most of the pages just make sense only in report zero. What one thing to to take a note, what I'm kind of showing right now is that.

B

Put one interactions are blocked to make a right to cluster-wide tables, so basically, what you kind of see here and do it here. You only do in locally in the context of the port one, but you are forbidden to make cluster white changes in the output one. So things like admin, user profile, they are scope to them, I mean they are forced to the port 0, because only pod zero can make a changes on those pages not put one.

B

So this kind of like tries to model async rights approach uh with like this white listing three different sections um that may affect uh these interactions and I kind of break this cluster. Why right rule of the port one so.

B

This is how it works, but this is like one of the ways, but the second way is like um application can learn. Mapping basically doing this pre-flight request using what's learn, but can also learn uh uh request based on the return headers. So, basically, when request is being processed, rice can send back easy. Guitar pop name request path to indicate that this root table path should always be directed to their port I'm kind of keeping track how things are being learned now, but this how this mapping looks like basically coming 451 map code.

B

It was actually learned using a pre-flight request, and this is like rotable entity like this endpoint that is located on the port0. It doesn't really understand that, like rotable and DT is like coming for four five one, but this is something that's like we may improve later, but technically. If I now put like comic.

B

451 adapt code, it's not gonna send a pre-flight request because it already knows how to access, for example, here commit 451 block code. This is indicator that it was found in the pot in the mapping Json on the router side, so it was redirected to the port0.

B

It also looks that okay, we also request refs Masters luxury, which is like a Graphics of that is like commit 451 lab coat. So it's not exact match, but this is like fully rotable to deposit zero, because this, how we, how we make a decision- and it kind of sees like another request Manifesto to Json- was kind of deduced based on the referral, because we uh accessed that the same goes to epigraphql.

B

Now, let's take a look like this is like the request: search, open search, dot, XML. This is a normal table request. We don't know where to send it. So like a pre-flight process we can for because, like we, don't know how to map it right now.

B

uh What goes now, if we quickly do commit 4.1, so call me to 451 it's like learning, so we kind of send pre-flight request for this endpoint and we learn that this is spot zero as it came from from race and this kind of force like catched and stored in the market. This mapping is right now, like as a text file, because I can manually, amend this file. Yes and it's gonna be like reloaded dynamically.

B

So, for example, I can force commit 451, because I noticed that this is broken to be using pod one and it's kind of gonna force it to redirect to the put one. So uh it's more like I guess like a table thing, but right now, it's like a fixer Json, easier inspect um how these mappings are resolved and like uh what different requests are being observed, for example, rpb4 Pro, X 24.

B

This is pretty like a rotable entity about a pre-flight process, doesn't understand yet uh how to mark that into a table. So technically I can now fix that manually and solve that like saying happy before hours, our 10 months. Maybe this is the wrong one. Okay, maybe maybe we can just get the issue, so this is right now redirected to the put zero it's correct because, like this is our cookie, how it sets so it's using what is cooking I can actually kind of force it or not.

B

I suppose ah maybe it should be on put yes so Pro X 24 is like it's only about one not put zero, so I kind of can force it dynamically to to change the mapping and kind of I use that method to understand different endpoints being observed by the application, because it kind of gives me like an idea. What is like my next Target to PreFlight to resolve- and this is like one of the cases like Pro X 24.

B

It's probably like my next Target to solve, like how I can figure out that this path is basically put one always and kind of fit on that information back to uh to to router.

B

B

A

B

A

B

Mean if I understand this correctly.

A

This mapping.json file is essentially your cache and you have it as a text file, because you can manipulate it right now. But if we like at some point, this would be stored in in a way that makes sense. You know to keep the cash warm, but anything that goes in here means that you have some learned information about what the router is supposed.

B

A

Do- and that means you don't have to guess the next time and it's just more efficient- is that the high level advantage of this approach.

B

Yes, like I, can manually inspect what is happening. It's much easier than looking for locks and I can also quickly comment that yes and change and I also kind of get an indicator. What is the source of that information, because I would kind of assume that, like raw table entities by definition should be as a mirror and like very shortlist, whereas uh I mean this is not rotable? It's actually. Probably you don't know what is that? Yet, because it's not marked but kind of gives like some idea about different endpoints observed and that's the idea.

B

uh Probably one amendments to to that is really like that, and this is.

B

um A prefix of that so technically I could also clean up that as well, but yeah. This is I I guess this is more like that uh easy way to to test like and try to classify these different requests, but basically like we have a stone of difference, requests just like going through GitHub that are not scoped and manifestation I, don't know what is that forum, for example- and this is I guess like a simple way to learn.

B

Actually, it is explore projects, so this is something that should be scoped to the organization. I see.

B

My new first Jason uh item um I, don't know. Maybe this is.

B

Now maybe it's like a default URL, it's interesting handpoint! Oh.

A

Yeah yeah cool- and you mentioned that your photograph graphql you're, using the referral field in the header and that that's a bit of a hack. So what would be the alternative at some point for graphql? We set the manual header field or how would that work? Probably something like.

B

That okay, I I, will do some something like that, where we kind of scope graphql in a four Euro table entity- and we kind of I guess like this approach- really requires that, like we go through each of this land endpoint and make a decision whether this is fixed or it should be modified to be rotable and like if it's should be modified to be rotable like what would be the scope, because this has created like the wrong thing right. Yes, we would really want to have something like organizat projects.

B

You know something right here, because there's a screen like notable yes, but maybe the surprise, Slash new, uh it's something to do.

B

Tensorflow like if you do price, that's new Slash new, you kind of have to first choose an organization in which it creates. So so maybe it's not really like a kind of a tiny, existing endpoint, but really redirecting this to the correct one later. So maybe this is like 3.0 fixed, but maybe this project uh Slash new effectively writer with rx2 organizations, GitHub project Slash.

A

B

And- and maybe this is like how how it's implemented, that we we keep existing endpoint States has kind of automatically redirect you to your organization's code. One.

A

Yeah and then for new new entities, new organizations, it would just not have to redirect at all anymore. At some point, foreign.

A

Because we have to keep the old yeah I, don't know, I I, don't know, but I I think.

B

This is like, ultimately, what really what kind of decision we need to make. um We kind of need to probably figure out all of the different observe endpoints and classify them to some.

A

B

And uh there has to be some systematic way for doing that.

B

um And it's not really that hard, I guess but I guess the hard thing is really.

B

There is just plenty of them, yes,.

A

I can see that. Well, it's great progress. I mean this is one of the key things that we needed to figure out is how to root requests and I. Think this is a pretty good start.

B

It's still far from being useful, but maybe it's enough for us to start basically testing application without having to switch manually.

B

So this this may be like a good objective, because maybe this is how we can now go to the next level and try to uh go through these different workflows. And maybe maybe this is right now enough for me to run graphql a QA tests against, like these workloads, Source routers, transplanted pots and figure out if there are some like if in General application behaves properly.

B

Yes, so so this may be, like my my next Target to kind of have try to run Black Box tests that we have and see what breaks and see like where I have to fix. Mappings to to make it like pass most of time.

A

B

Because this is I think also gonna show where cluster-wide modification will break on the Pod one, because I think what I want to want to do like I want to run as many things against Port one, but change only some things to be to have to be run against Port zero. The ones that require is cluster-wide centers. So this would be I think my target to validate using rights is actually like a valid strategy.

B

A

B

Seems to work, it seems to be like a high strategy. Yeah.

A

Yeah I guess this is this: is the progress that we're looking for right, just exploring some of these strategies and figuring out what sticks and where the problems are and go from there I think, at least for me, it's easier to do this more concretely, rather than reasoning about it abstractly. But now you can actually see you know, what's maybe problematic,.

A

B

I'll stop recording thank.

A