Layer5 Meshery Build & Release, 24 Feb 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Meshery Build & Release Meeting (February 24th, 2022)

Description

Meshery Development Meeting - February 24th 2022

Agenda -
Failing cypress tests
Meshery Test Plan (spreadsheet)
Incorporation of the GitHub Actions
Adding applications into e2e tests.
Propagation of consolidation of label-evented workflows.

Join the community at https://layer5.io/community

Find Meshery on:
GitHub: https://github.com/meshery
Twitter: https://twitter.com/mesheryio
LinkedIn: https://www.linkedin.com/showcase/meshery/
Docker Hub: https://hub.docker.com/u/layer5/

A

Hello: everyone, I'm audible, you are okay, so welcome to today's call and uh if you haven't already written your name in the meeting minutes uh I'll share the link in the link in here. So you can.

A

I'm also entering my name right now.

A

uh So the first topic was about uh adding applications into our end-to-end integration tests. I have pinged augustus, although he is not available right now, so I'll push this a little downwards I'll talk about it. So if anyone else is interested, you can also pick this task up. It's pretty quick to accomplish and nothing. You know too complicated here. So before I move forwards, uh mario, I would like to talk about the failing cyprus tests.

A

So initially I thought that it was some versioning issue which it could have been easily, but then I realized that uh before I make assumptions, are you also installing uh the is your control plane before deploying all those patterns or you're directly using the patterns.

B

uh Good question: well, I thought I was um but to be honest, I think that.

C

What gets installed is just a like: the measuring steel, adapter.

B

I I I yeah I was in that I might have been in the wrong assumption that running measuring is.

C

Still like, like as the cic, is doing also like handled installation of uh steel, I think that either it's not either it's not doing it or it's or it's um doing it for the wrong version.

A

So yeah actually um you'll have to install this view, because when you install this view, the issue control plane actually goes ahead and registers those crds with the cube api server. So, although we have the schemas and we can ask ubi server to install something like convoy filter in in your specific case, but the thing is kubernetes won't recognize it because it's the control plane is not up. So if sq is not installed, there is no point in things like onward, filter and virtual service, because they are pointless because the control plane is not running.

A

So I think you should uh go and install some different pattern. Maybe if you can install two patterns at once. uh First, you install the istio uh normal in history, installation pattern which you can find there in the service mesh patterns repo and after that is installed. You can then go ahead and install the uh install the other things, or there is a minimal book info, something by that name uh that and that pattern actually installs the book info app and before installing booking.

A

For it make sure these your control pin is running and all those things. So you can just uh deploy that uh pattern, and that would be good enough test. uh According to that.

C

Okay, yeah. I I personally agree with that. uh I also think that it it's a good opportunity for um like. Can someone pop up the like the samples, the pattern samples?

C

The thing is that well uh part of the the issue that I got solved like the idea was to, um and how can I uh explain it eh? Well, basically, just a so we want to test that the samples we have uh in that repo.

C

You know that they're working as we continue making changes right, and so um I don't know if it's uh negotiable, like is there a way we can kind of like update those samples or- or I don't know um like idea- is that the samples work right, so the ci cd, uh as for prior discussions like we'll, eventually want to add more and more uh different patterns that get updated.

C

Aha, so maybe I I might have missed some kind of readme in there that talked about the prerequisites.

A

um Yeah, actually uh is there.

C

A

Yeah, actually there are prerequisites I mean um recently. I talked about, I added a feature about importing patterns and there actually talked about that in our service. Mesh catalog we'll have two things that what you can configure in a pattern and what are the prerequisites so, for example, this is your filter pattern.

A

Nowhere does it say that it has a prerequisite, but um it's like um I mean it's um it's something like of a common knowledge that, if you're installing something like onward filters, you you have your data player control, plane setup, and that is why you're dropping these filters and if you don't have that already set up, then it kind of doesn't make you know sense. So it's not explicit that the prerequisite for this pattern is that you have a you know, installation of, uh for example.

A

Here also, you can see the workload selector label is version v1 and app api. So basically it assumes that you have you know your deployments running with these particular labels. So, of course, almost all patents have some prerequisites and when we will upload them in the service mesh catalog we'll make sure that we'll tell them what the prerequisite is. You have to install this particular thing and, and the that thing will make sense in that.

D

A

Context only so this thing that you are trying to install it tries to install onward filter, but the thing is that, as there is no control plane, the kubernetes does not understand it. Although we understand it, that's where there is no problem on the mystery side,.

D

We have the schema.

A

The keyword is under the same goes with the book info pattern, because here also, as you can see, you have things like virtual service, dot, stu and all, although, um if I'm not mistaken, you are using. You are deploying this book info pattern in the slash api, slash application, slash, deploy, endpoint right.

C

uh I'd say yes, um uh just right now, I can't I need to take a look, but one is istio. Is the parent deploy and the other one is the book info? I just can't recall. I think that the istio filter pattern is on the pattern. Route yeah and the bookkeeper pattern is on the app um application.

B

C

B

Actually actually just just a question so but but the error message seems to.

C

Point that, like that that see so so there's a crd, that's not available correct.

A

Which error message? The filter pattern, one right.

C

Okay, but but then again what I'm seeing is that, at least by the message itself, it looks like istio. Is there or not is this? Is this.

B

Assumption incorrect.

A

Yeah, it still won't be there. Actually, uh the message that you're getting the error that you're getting is from kubernetes itself. It's saying you're saying that I don't recognize this particular resources, uh because if you- because since this theory isn't installed, it has not registered its own custom resources with the kubernetes.

A

So yes, that assumption is actually uh wrong. We cannot assume that is. Theo is already there the work of adapt. When you install an adapter it doesn't. You know, install services.

D

For you, it's just.

A

There you have okay, yeah.

C

Okay, one question: so why does it say that it's not so no matches for kind envoy, filter in and then it says version network dot? Is your io b1 alpha 3.

A

Yeah exactly so kubernetes has this concept of g keys, so it goes into that particular when you specify your kind and things like that, so it goes into, uh for example, in our when this particular pattern is evaluated.

A

This thing has been evaluated so, though we actually when we are uh trying, when we try to deploy this thing, uh our adapter, what it does is, of course we are at the end going to talk to kubernetes, so it creates uh uh basically a kubernetes manifest, and in that manifest you see things like kind and all so uh there you have specified thing like networking.istio something, so that is the your group and version. So what kubernetes does is it goes into that group and version, and first it tries to find that.

A

uh Is there any this particular resource in this group version? So, of course, that group version itself doesn't exist. So that's why it's uh uh giving the error that in networking.sku, whatever that version is whatever that group is. There is nothing of this kind in that group version, so that gvk is not available with kubernetes itself. So that's why I actually tested it after deploying it before deploying, and I I got the you know, the issue will resolve.

A

If you uh have your, if you have your basically because that error is from kubernetes, we cannot do anything about it.

C

Okay, uh can I share something, so I think you know I agree with your resolution like I think you did like a really good. uh What how can I say it like good uh investigation and just a sec, I'm trying to share the screen.

C

I just found something that, since uh some time back related to this problem, can you guys see my screen right now. Yeah.

A

C

Okay, thank you. Thank you. uh So um the error looks very similar to this one.

C

So there's a closed issue in istio, not sure if you guys like uh got a chance to see it and so issue seems to be closed already since uh some time back, but it's it seems to be like some anti-profile uh like a specific scenario right and the issue is like I don't know it's just too similar.

A

Yeah, actually, this header is a very common error, so it's like so.

C

We would see the same error in different circumstances, even if.

A

It's not there yeah.

C

Okay got it so then it really doesn't give like. Okay, okay, so okay, I got it so then. What should we do so that so that this stops happening.

A

C

Say that we have to choose another pattern.

A

Yeah either we uh you know.

C

But then one question: if, if we do that, if we do that, we wouldn't be able to test integration with istio like in.

A

An end-to-end perspective, yeah, okay, so what I can do um and because we have the pattern import feature right now, it is pretty easy to do. I can have a pattern where, for example, uh let me share my screen.

C

Sure sure go ahead, I'll stop sharing! Thank you yeah. I just saw uh leonard after me saying, like the okay, I still think cool.

A

So, for example, uh I can create a small so this, instead of directly using this istio filter pattern. What we can do is that uh I can write a small uh uh another pattern where basically, what I'll do is I'll first import, this minimal book info app or maybe I'll, uh basically I'll import. This seo filter pattern, uh where I'm at first installing the http installation. So after the.

D

Htm installation is complete.

A

I will add, like here, I've added book info I'll, create another pattern and basically upload it here: okay,.

D

Under the samples.

A

And then you can use that and that pattern would be exhaustive of most of the things that we would want to test so that you can just use that pattern, that that would import.

D

A

And everything and that pattern would.

C

A

That everything is installed so.

C

I get you one question um so a pattern I don't know: if is it viable to make up our sample patterns like uh independent in the way that they either install dependencies that they need, but I'm not sure if, if we run, let's say we have pattern one and pattern two and they both will install istio. Would that cause an issue in a like would measure it fail out, because you know oh, this is already there or would it just say? Oh that's there. That's, let's keep going.

A

Okay, so you're saying that a pattern that uh always works by figuring out what are its dependencies right.

C

Yeah I mean how the clarity is our our current, a engine in the sense that um well yeah it. It.

A

C

Go ahead. Sorry.

A

I think it's more upon the order of that pattern, so whoever writes a pattern uh that person is aware of uh what variables this particular pattern can expose and what are the prerequisites, any particular assumption to be made, is uh you know pretty short-sighted, so tomorrow we can have a complicated.

D

A

And then it would get uh chaotic about to figure out what do we need to install first and and the second, so it has to be explicitly. You know someone has to.

D

A

Say this that uh and that's what I propose that when you'll upload these things in service mesh catalog, we'll actually tell you that this pattern is self-sufficient or, and this pattern is actually.

C

Okay, so we'll just document out uh what dependencies are in each pattern, it's just that these patterns, like like we're, still building them out. So that's this is the reason like uh this is happening now, but when we document this more uh it'd be simpler for any user right to know.

A

So the idea is that patterns should be atomic units of things that do one thing and they do it good. So, for example, sure.

C

A

This is a minimal booking for uh e dot yaml pattern, so this pattern will install your sql control play and it will install your book info app, and this is what it does. It does not make any assumptions, but there are some.

C

D

Who make some assumptions.

A

But they that the one thing- basically they do one. For example. This is your filter pattern. Its job is to uh basically apply the apply. The these filters apply a.

D

E

A

And this uh generic histo filter uh onto.

E

Your camera so.

A

D

A

uh And it it, of course it assumes that you have your uh control pane already installed.

D

Okay, one thing we can do.

A

Is that, basically, in each pattern, where we do these things, we can, you know, have uh an issue installation again, but you know that would make everything complicated and patents would you know.

F

A

Of noise would start to accumulate on a pattern so.

F

A

Be even better, the better option would be to actually.

C

A

It uh somewhere that.

C

Yeah or maybe it's easily, you know, I agree with you because sorry yeah, oh sorry, yeah so you're right because we want to make measuring smart by, but how smart we want to make it that it will, like some users will like that behavior but others might not. So we we try. We need to keep like try to be as unbiased as possible um yeah. So so we could talk about different scenarios and depending on which scenarios we're talking about or use cases, some decisions might seem more rational than others.

A

Yeah so I'll give you one use case when that would fail. For example, let's say that we have a pattern file and in that pattern file we have a service which is very generic, or let's say that, and for example, here your type is very, you know, onward filter. Is you so you know that you need seo uh what, if tomorrow you'll have a type of resource that is available both in traffic and in linkedin, which one would you install in this case, because the same type is available in different service meshes?

A

So we cannot make assumption that this type is associated with that service mesh and this type is associated. The way mischief is designed. It does not make any assumptions which makes it very easy.

C

But but when we say type ammo envoy filter is still so that that does make an assumption that it's for istio right.

A

But yeah. But in this case but in.

C

This case yeah but yeah, but generally speaking generally.

A

If we would make something yeah, we would go.

C

A

Onward, filter istio: this is our core component internally. What we are doing is we just convert this thing into convoy, filter, the actual onward filter. We.

D

Are just very thin.

A

Abstraction we just take this, so basically, here, as you can see, type is very explicit, but in a lot of cases it won't be and in future, if when we, you know, build higher order, components and all uh we can really not be sure about uh what thing that so there are traits which can be applied on multiple workloads and basically the idea is that we don't make any assumptions it's up to the user.

A

uh So that's why we don't want to enforce any kind of you know you have to install this and install that basically, a better way would be that whenever someone writes a patent, what they can do is they can comment out on on their yaml that hey this needs that assumption a very basic thing to do, or if we upload it, we can tell the users that uh up until this point, no one pointed an eye to this, because you know things were like very obvious to at least to us people who are using service messages.

A

C

Course, you're using.

A

Onward filter, so you need an seo so that kind of thing, but to you.

C

So experience people like really don't don't bump into these kind of problems right, that's correct! Exactly yeah one question: do we like the current message like? Is it descriptive enough? uh Let's think about we're a user like we probably not that familiar? um Would that be the most common case like.

A

Maybe 50. is the one that that this version is not that that one right, aha.

C

Yeah would the user know what what to do next.

A

uh Actually, yes, uh the reason is because uh we are not that's not our error. That's kubernetes error! First of all, so if you.

C

Thoroughly come yeah.

A

So user, if user gets into that error, we are sure that okay, so there is some problem with the community, for example, in this case, I just looked at the error and there were only two possibilities. There was some versioning problem or the resource is not available, so I think uh to have this kind of transparency, where we just take that kubernetes and throw it on to the user is much better than to you know. Try.

C

To map it to our own domain.

A

And just yeah exactly because we cannot be exhaustively sure that what are all the errors in kubernetes right so.

D

A

To have this kind of you know we so that whenever user tells us that hey, I cannot deploy this pattern. This is the error. That's coming. We can. We can look at the.

D

A

And in that type of we have the mesquite error, so it actually says that okay, so while.

D

They're deploying.

A

This ohm component, they had this error and then we can figure out that okay. So this the kubernetes.

D

A

It actually so we can, you know, go from there, so I think.

F

C

Yeah, it makes sense yeah it's just that you know when the user installs measuring and you know there is a flag to choose a specific like adapter right. So.

F

C

Just to make sure when we and when we when measuring, receives that okay uh istio, okay, that flag it'll um it'll use that adapter it'll grab it from wherever okay, it it'll have it. But then this doesn't install any version of ys2 at all. Yeah.

A

C

On the cluster, that's right, it doesn't that's it.

A

Yeah, no, it doesn't. It does not install this, because how do we we can have like 10 adapters and users can have all the 10 adapters running? uh So we are not going to install 10 service meshes onto your class.

C

Oh yeah, no totally not, but then, but then what? How would how what was the preferred way of installing it.

D

C

Measuring user, like is.

D

There a way in.

C

From azure interface to to go in and and from let's say, the measure ui to install.

A

C

A

Yeah, so in mesh ops v1, you would see that that configuration page is that page you'll go to that page and it has actually you can configure. Basically, there is a list of all of these service meshes there. So if you have your seo adapter running, it will connect to that. Is the adapter and uh then wait a minute I'll deploy machine.

C

Yeah, could you show like a flow? Let's say a user just starts. They just install ui measuring mesh reservoir, everything and then they just want to use measure. You know they're not familiar with uh istio, but but they need it.

C

Maybe they need it, but they they just you, know they're just playing around, so maybe we can introduce like a bigger or wider integration test or end-to-end by exercising that flow and then making sure at least that if, if measure, if there's some flow, that installs measure like istio, that measuring makes it still getting installed, uh then at least we can make sure that it's installing it correctly because after that running that that specific pattern that depends on the steel, at least that's my my opinion.

C

You know like, um of course we're not like doing extensive things you know and but we're basing that the user has a certain minimum amount of experience or not knowledge of what needs to be there. You know if, if we add documentation around that, that's good, of course, but at least when we're talking about a specific uh pattern file when I left a while back, it was because I read uh a list saying that the word pattern was being tossed around.

C

In our conversation, it was just so funny for me because you know he's absolutely right. You know we're still like kind of like uh you know, defining that go ahead. Sorry.

A

Yeah so uh here in your life cycle, actually not configuration you'll see, uh for example, if I click on seo. Actually my cluster has basically I didn't start my cluster, so here actually you'll see things like um yeah in your wait. A second.

D

So I'll chime in to say that we do have integration tests that.

D

That will build the meshri adapter for istia.

D

Stand up a kubernetes environment, deploymentry server along with that freshly built istio, adapter or any any of these adapters, and then it will invoke just the basic workflow of provisioning of istio, with just its control plane components and we almost almost universally have that same set of integration tests embedded into each adapter repository today we don't go, but that's actually part of today's discussion is to take those to and to extend those workflows, and so someone like raj and pranav and hersh and well like all of us, on the call.

D

But like some of these other guys, um I was I'm hopeful that they'll be able to pounce on the framework. That's there and start to add on additional use cases like okay. Well, so if the workflow and this framework is present for doing a measuring deployment into a kubernetes environment and then doing the first step of like provisioning, the provisioning, the service mesh and it's basically empty, okay, good, that's where we are now. If, if I'm pranav and I'm I'm looking at it- and maybe it's the istio adapter, so I'm saying is there a red?

D

Is there a handy sample application here? Oh yeah, it's the bookinfo sample app um and we have operations to provision that what are the next couple of uh jobs or whether it's a job. The next couple of steps in that workflow that can be added to just invoke mesher's next operation of deploying the sample app onto the mesh good like that would be successful. That would be a whole.

D

Another series of tests that are rather just having that single step that single next test would be um would be incrementally helpful and would verify that it wasn't just that we stood up istio and actually istio doesn't actually work and we no no. It works because we deployed a sample app on it, so it works and that's a whole. That's a big fat check mark the next step being something like.

D

Oh, we have a handy service mesh performance, github action that can then be incorporated as a next step, where great so we've got kubernetes and mesherie. We've got an instance of the service mesh stood up now. We've got a sample app deployed now.

D

If we take this service mesh performance action, that already has some of the hard stuff dealt with, um and we say well we'd like for you to generate low use, meshry to generate load against one of the endpoint like the sample app and then have it, evaluate it doesn't really matter what the performance is. The point was just not only do we deploy the mesh now when we deployed an app on it, but we also sent a fair bit of requests towards it and it's responding back so clearly, everything is working. Okay, great excellent, perfect.

D

Those big those are two big fat check marks on the integration test. You know, like the list of things that we need to automate it'll, be really helpful beyond that yeah you can get really expressive in any of the the patterns to say. Well, we want you to generate the test or generate all that load.

D

It doesn't have to be a lot, but just generate a little bit of load um before we do um so in between deploying the sample app. Let's actually apply a little different configura, some specific configuration to the mesh- maybe maybe let's intentionally drop 50 of the packets or like, let's intentionally divert some of them over there or like. Let's deny them unless they have a certain token of a certain form. Some.

C

D

And- and that would be some it- wouldn't it's not like we're trying to like the point of doing that, isn't to rerun all of istio's tests for istio just to verify that istio is doing what they're supposed to do. The point would be more like no, no mescheri is capable of not not just internationally yeah it at all levels at all levels of specificity, and so we're not going to hit them all they'll be pointless for us to go hit them all and yeah. It doesn't really.

B

D

If we kind of touch the end of a few of them, it's like okay, there's um super high confidence that if you redirected, 25 or 75 or like like all those tests, you know istio can deal with. But this uh could we just make it to the end and did things flow, then we're pretty safe to assume everything was correct.

D

There's just a couple of key points to like to check you know and, and that's what that's actually where we are, that this output here is showing is showing some of those individual adapter tests and whether or not they're being successful up to the point just before we would provision a sample app. So that's actually a call to arms on. This call is like a call for all of us to go.

D

um Like it's, actually it's it's one of those things where, when one of us writes a small little workflow that deploys a sample app that we basically want to take that and go add it to the other adapters.

D

Maybe we use a different sample some sample apps on this one or that one, but.

A

Yeah, so to answer mario's original question: how does a use how if you have a running adapter and a user, wants to install service mesh? How can you do it? It's pretty out of the box, so here you can on my dashboard. You can see that I'm connected to my menu cluster and also I'm successfully able to ping my adapter.

A

uh If I click on istio now this used to be the uh only way you could install istio. uh This is what we call uh mesh ops v1 as compared to mesh v2, which is uh using patterns. Basically, so here you specify your namespace, uh although because seo makes it uh compulsory to install your service message to your system, so we override this anyways, so you specify your namespace and you click on add and um and basically this installs your service. So it's pretty. It's like one click thing to install yourself.

C

So then it just it just invokes like the like most default. uh Measuring, I'm sorry, sto install script from github right.

A

uh Not from github, actually we uh the way we install uh basically yeah. There is.

D

A help default.

A

Installation yeah through helm, it installs.

D

Rigorous gateway, ingress.

A

Gateway and uh basically it's not a default installation, it's I think it's called demo installation. uh They.

D

Have different profiles.

A

So it's the demo profile so because this used to be not so com. Basically, you cannot customize your the way you want to install it. uh Basically, you know you you are. This is kind of basically gives you uh an ability to just click on this plus button and install it. But what if you want to choose a different profile, and also for you see now I got the uh notification. That is your service message installed successfully.

A

Although this thing is pretty, as you could see that I clicked on that button, then I had a conversation and then, after a point, we had this thing. So our one of our next things in our dot seven road map is to have a notification center. uh The way, basically, at what stage we are at while evaluating this pattern, we need to continuously pump that data onto the ui, so that user is in touch of what's happening so yeah to my original point that um here.

C

As you can see,.

A

C

Just click in there install and just wait for that to return like a good response and or see that message and it's now, it's installed yeah and now running a pattern file that depends on istio.

C

It would work cool, okay, one question: uh okay: first, I'd like I want to say that lee is totally uh like nailed, a right on spot with confusion that the adapter chips it it makes the user or any dumb user like me, think that istio is installed when it's just interfacing with an adapter, and that doesn't mean that it's actually installed, and so, for example, do you have um do you have console installed right now? No right.

A

uh The adapter or the service mesh.

C

Okay, uh you mean okay, um so.

A

These chips, just to when you, when you click on these ships, these chips are basically the request goes to messy, so messy server creates a grpc call to the adapter, and it asks if.

D

The adapter is.

A

Up so this, these chips are just as you can see, local. uh If I mean you can see localhost one zero, zero, zero. Two so.

D

A

Like it's a running server, so if you were, if we were pinging a control plane, I don't.

D

I mean you would expect this.

A

To be not, you know something like this, so these.

C

A

Are just adapters that your adapter is ready and it's available to perform operations.

C

Yeah, I ask you this because I noticed we have a bug in one of our other tests, so there's an e2 test in cypress folder. So there's this setting spec it checks for service mesh uh api. It's what's the page, the settings page. So if you go there um and if you click on, let's say you select um the second one. I mean sorry, sorry, I'm still reading this out. So basically it's about uh chip, adapter location, okay! Is there a place for the service meshes okay, yeah service mesh? Please uh-huh, okay, tap service. Okay!

C

We're here! Can you select the one that is for uh um this console.

A

uh I could, but uh basically I won't be able to connect, because I'm not running the other.

C

Right now, okay, but we are hard, because the test is hard coding that it's uh the import uh 1002 is that correct? Is that for console.

C

So it is, but the problem in the test is that it's not really asserting the response. It's just waiting for the response to return, so you would get a 500 or something right. What would you get in here.

A

If I, if I'm not running the adapter, and if I try to connect.

C

Let's say you have the adapter.

A

C

A

Let me start the adapter, then um starting it's taking some time, because uh my I'm not being a gendered, but.

C

Do you know the answer like let's say the adapter is running with this? Would this return uh 200 or some other error.

A

If your adapter is uh running, then you'll see a second chip. It's 200.

C

Okay, but so then, as I'm trying to say so, the test is okay. It's just missing the like the the risk. It's not looking at the response. Just waiting for the post uh post api system, adapter manage.

A

So you're you're in your test, you're clicking on these chips. Basically.

C

um Well, it's clicking on the submit, which one is that so.

A

It's selecting one.

C

A

The adapters have a default port, so if you already have, uh for example, in my case, I'm already running my adapter on this localhost one zero, zero, zero. Two, so that's why it auto detects it. But if you, if you're running it somewhere else, let's say some.

D

Different domain.

A

Whatever address you specified.

C

Yeah no yeah, I think that the cicd installs it in that same place. You know this other test was written way back, so I'm just trying to understand the difference between the adapter and the actual mesh. But uh okay. I think this is good enough. I'll just write on an issue to to add that instruction, so so we're waiting for that post. But you know the test is not adding value, it's the ping and submit console.

C

But it's not it's not inspecting the the response, so it could even be returning a 500 and we wouldn't even notice so and but anyways yeah or if you would you would you guys want me to show up that or totally fine omitting that.

D

Wait to have it, you were saying to have a test that confirmed.

C

We have a test, we have a test, it's just missing one instruction and I think you know uh it's. I think it's relevant because we're we have some tests, but I think that the the communities, or at least the contributors- you know it's like a black box or does anyone disagree with that.

D

Yeah, no won't wait um so raj. I was just kidding about not a word about not being able to access figma because you had to go through that pain. So, um by the way pranav is it the you don't have a different is: is your heart male address google account backed like? Can you edit, google docs with.

F

No actually, I for google docs, I use a gmail account. Only so shall.

B

D

F

D

Great, I was just I just share your embarrassment with that uh hotmail account I just uh just met, um so I'm totally totally messing with you. Mario was saying um mario, I heard black box and yeah.

C

Yeah, so I think I think that with these tests, so the ones we're looking at so these are based on each adapter right right. So, okay, so I think that's. uh uh It looks like that's very broad by itself, but when we talk about uh measuring ui measuring server and the intune tests we have that are with cyprus.

C

I think that's where maybe it's a wrong impression, but I think that uh it'd be good to uh share like a bit more of uh how those are scripted and what they're checking and just clarify what they're not supposed to check so that we're all in the same page, at least for now. uh What's what's the actual scope like what are the objectives of those tests? I don't know.

D

C

D

That's it yeah. um That is that's actually kind of on the today's agenda. um If, if we were to bring up the this yeah.

C

Let's go with the agenda.

D

Yeah, no, no yeah, but actually we'll just we'll like roll right into what you just said, which is in I mean there may be multiple aspects: multiple faucets. What you just said, one is: if anyone is able to take the time to integrate the output of the cypress tests, if we're able to programmatically retrieve those and add them to the test results in the docs.

D

That would be beautiful because you can because then, like the question, that a lot of people just want to in general you'd want to know you'd want to say hey, so I'm working with this service mesh. So I want to take a look at the tests that are done with this with the representative, adapter.

D

Okay, let me take a look at those tests and and so you'd, whether those were run uh in in this workflow or that workflow or using cyprus, or that tool like from a user's perspective like while we want all that detail, and we we want to know that when that failed test was to run by cyprus and as it was run based on this, and we we can have all that linkage in that detail. What's some what's what.

C

Disability right.

D

Yeah yeah what someone else is looking for sort of the high level summary that says: well, I'm running kubernetes, 122, I'm using console and so and I'm also running it in azure and so is how is mesh redoing is a is measuring compatible with that. That's in part why the integration tests page is actually named compatibility matrix like we're not quite there yet, but the goal is to roll that up into a summary that says: here's measuring status of compatibility across the various versions of the meshes and the various tests that we run like.

D

Does it provision a control plan? Does it can visual? You know, provision sample apps, um the other aspect to what you are saying I think can in part be facilitated. If, if you go into that spreadsheet, that's linked in the second topic, this is meshrey's test plan.

D

It is um has a lot of tests and it also is missing tons of tests. um But then that's fine of the tests that are defined. Well, there's there's! You know you can test measure ctl, you can test measure ui, you can test just the adapters and their grpc interfaces like you know, there's various vectors from which you can invoke a test and have things run. Hirsch has been um through some of the work that he's been doing on performance testing he's invoking.

D

He is that's sort of, like top top level, he's sort of invoking the full kit and caboodle, because, because he's running performance tests against endpoints that have been deployed onto a service mesh deployed by meshrene, all that so in in this spreadsheet, there's individual tests that are specified if we're able to identify which of these are covered by cypress or by a workflow run. That's separate from again we, the goal here is to we have now automation frameworks in place that have been running for a bit and are do are doing fairly.

D

Well, it's we're in a position now, where we can go um just add additional, like the framework in the system is all there. We just need to add some additional tests, and this is a great way of to what you had said. Mario is to like identify how the what functions are in measuring, which ones are covered by a test and which ones aren't you know if we can link in like including the link to the cypress definition like the test case definition.

D

um Great, like you know, that's excellent as well, because, ultimately, eventually, like it's kind of an interesting thing, when we first start you and start with a blank spreadsheet, then you add some test cases for measuring ctl. Then you add some from sri server then for the other components, then you get better at defining what they're supposed to be like. Then you get better at saying.

D

Well that should work on this os, this os that cloud that cloud, this kubernetes version and all that and then you go through and you start to you try to automate some of them. So we we have automation, frameworks in place for most of the components, the adapters, the okay and then finally, you get into like and then and then we're coming back here and we're kind of documenting the fact we've got at least covered eventually it'd be great.

D

If we could hit delete on this spreadsheet and what we're ultimately looking at is basically the same content, but in the docs itself that are just like the systems and the frameworks are just auto documenting, what's covered with the.

C

D

And we're kind of there like okay, we already have some auto documenting for um the tests that are being run against the adapters, and so that provides a like, like the hard part, is oftentimes getting the framework in place and then from there you can kind of run wild and automate the rest of the tests yeah, you know so great hersh yeah. We should uh hershey's ma. I didn't see hershey's message.

D

um Mario, is that are you good with? What I just said is that yeah.

C

I I think it makes sense uh I just um like they were like we discussed about uh what approach is best um and I did propose like uh making those tests so they're commented out, but uh so I can create a task and then work on that, and the idea is to like my suggestion is that we can cover a like it's kind of confusing, so we can cover a bit of the life cycle management. uh If, if we want to to do that right, you can just.

C

I can just make the tests, uh whichever tests that need that needed that do the deployment just whenever they need uh istio.

C

They could have um just make sure to go into that page and click that button like um install it on on the cluster, which seems the simplest way and also we can also kind of like test a bit of integration of okay when a service mesh gets installed in that way, uh not sure if that's already tested anywhere, it might be written on them to do that. No okay, good and then awesome yeah.

D

Well, it's kind of it's kind of both it's sort of like in some respects it's redundant of what the tests that are running for the adapters, but, on the other hand, it's like.

C

B

We're integrating.

C

That yeah that server it uses that pro okay yeah good. So then that's totally.

D

It's okay, that they.

C

Overlap and then you know once that's done, of course, um we could also do that through an end point just so it's fast, um okay, uh just keep it things fast and then, uh since those tests are like based on the same template, yeah just well basically yeah. I think I know what to do so that that solves the problem without uh adding like extra, uh like the other ideas, were good, like uh I didn't get like running other pattern, etc.

C

But I don't know at least that issue that was attempted to be resolved is like to to actually test those patterns. You know, and I think this approach would solve like issues, and I think that that, like really pretty like the clarification from she's, like that's great, uh um so yeah I'll, just follow with that a create some issue. For that, and um or I don't know- maybe we open the other issue- I don't know but yeah. I think it's all for now. Thank you.

D

C

D

So I'll I'll try to follow up as well to make sure that some of the things.

C

I did ask a question like okay, this specific scenario: do we care right now, or would we want to capture a like how a user would just see that error? Like um again, I like to reiterate um that it wasn't that easy to identify. You know I I I didn't solve that for myself, and so so, how can we capture that for a future?

C

uh Enhancement of of that, like, of course, will not do extensive things, but I I do know that mesherie has a capability of doing some dry run and something it's just that probably we're not using that for that specific deploy. Workflow like what do you think lee.

D

I uh I'm gonna, can we just.

C

Can we just check at least that okay measure is not installed? Let's just output that like it's, not, I don't think that's hard to to check. Is it.

D

Nope um nope and uh uh prana pranav's work on the notification center is gonna help with some of this explain to people that, like hey. Actually, the pattern failed, not because the pattern is bad, but because there's a missing dependency potentially and then um other work would other work eventually will make it.

B

D

Obvious upfront that, like hey, you shouldn't even attempt to deploy this thing because there's a missing dependency and here's what that dependency is like yeah.

C

D

C

Do we consider that enhancement in the scope of the notification center itself, or or does that, deploy? Let's say the deploy application deploy pattern like? Is that something else like? uh Would we use that solution in there as well? Like I'm, just curious.

D

Yeah um yeah, the nice thing is that the most of them will benefit from the same logic, the same underlying logic, saying.

C

I got it okay, so then, that that's why we're just focusing on the on the like the usual path? Okay, like just uh let's just add the step to get that installed and just make sure it works like okay. I think it totally makes sense. Thank you, yeah yeah,.

D

C

D

I'm rehearsed you've got.

D

E

So I'll keep it I'll keep it short mainly because I probably do not have the energy today. I have to keep up uh right. So this is what I this is the run that I did for yesterday's meeting. Unfortunately, we did not have time to cover this, so I added- and I sent you that I was getting this particular issue. However, this still showed uh they still ran that particular test, and then that did show me that the test runs successfully.

E

I'm not too sure how exactly this works. uh Or is this an error which panics and sort of uh exits that part or exit smash area like shuts that process down or something I'm not totally aware of it? But given it is running a test and it's completing successfully, I'm assuming it's not doing that so.

E

Yeah, so so, uh actually, uh I had also taken up a helm issue. I think you know that, so I thought that before I contact switch there, I decided to just give uh the scheduled case and the cases which run uh direct, which runs in a scheduled way, or let's say this: what this manual does, although I run it manually, but what this does is that it runs four tests consecutively, one after each other. So that somewhat gives me a proxy to how scheduled tests in general will happen right.

E

Any certain configuration can just pop up anywhere anytime and then just we need to make sure that uh we can handle that. uh So I was working on that and uh I was working on that yesterday and now I'm at this stage where all these four particular uh tests are succeeding right uh so either one if you like. You just see uh sorry.

C

To interrupt one question: why are they called manual? Is it because they're trigger manually.

E

uh Yes, yes, yes, okay,.

C

Got it they take.

E

It manually and then there's this scheduled part which runs on a schedule, so I will.

D

E

Currently, this is configured just for the master branch and I'm working on a separate branch right. I'll, probably configure that as well later. But uh but given these manual right so a manual, you can see that I'm currently using one worker right this one. I'm there's a machine I created in tokyo and equinix, because tokyo is closer closer to india and dallas is away from india. So I'm not creating any more machines and data because that's laggy so now.

E

Because of that, it runs one by one, so when this runner gets free of that one particular run, it picks up the second second workflow. Then it picks up. The third workflow then picks up the fourth workflow. However, if we like and that's what I think- because I noticed this in the other case as well- that if, if we supply two machines to it, then it will probably parallel parallelize them and run them uh together.

E

Right so and it will share the load across the two machines and so on and so forth, for uh if you increase the number of machines, obviously up till four, because five dozen exams for four tests uh right.

E

uh So I have been able to do that. So there were a couple of caveats that I had to solve that and those I have been able to do so. uh I think this would certainly assist with what uh jin is doing himself. So I mainly he focuses on the workflow part and then the scheduled part. Maybe it is the uh hardware side and the whatever os permissions we're getting into, and whatever small caveats they're there about cleaning up that particular instance.

E

After we're running I'm trying to handle those uh right now and right now, I I do think that this ford checks is a good sign. So what my idea would be to sort of set up scheduled tests on that particular branch that I'm working on, and uh I hopefully I like, I hope, to wake up to uh males of successful- runs tomorrow right. So that's that's what the idea is and then I'll probably context switch to that helm issue.

E

uh Right so this this is it this. Is it from my side? No, nothing more.

D

Quick confirmation, the clusters that it's still a single node cluster, that you've yes.

E

D

E

Actually, yes, I I had a question. I had a question on that so when we're speaking about, let's say multi-node clusters and we're speaking about, let's say running these particular tests right. So ultimately we aim to sort of run a few tests on that machine. Or are we thinking, let's say uh our entire service mesh service, mesh sort of consists of uh containers on multiple hardware or that sort of thing.

D

Oh um yeah, it would be that the service mesh itself is operating cluster, wide and yeah and so across nodes, so that so that yeah, if there was a three node cluster, we install a mesh um and then install and and the namespace spans nodes in the cluster that the tests themselves are the the the I think, the vast majority of the test itself.

D

Probably the workflow probably doesn't change much it's it's mostly just the initial part of ensuring that three individual nodes that the kubernetes cluster is initialized with multiple nodes and actually the rest of the still just be one kubernetes context. One one cube config.

D

Left to the default behavior of kubernetes, um when you, when we deploy a sample, app it'll, get spread out across the the cluster across the nodes and yeah. I know it's more yeah like it's more just the equinix metal. You know provisioning of an os and you know the dynamically feeding. I don't know if it would be dynamic or ideally it would be dynamic.

D

We dynamically feed um the workflow, the automation, the automation, not necessarily the github workflow, because I know you've got automation within equinix metal outside of that, but to the extent that you grab three nodes, kubernetes the os gets deployed, kubernetes gets initialized within a multi-node single cluster and then the rest of the test is the same from there.

E

Right right got it, so I think I'll.

E

I will uh do that with I mean, but I think this is this probably is a good start to at least uh handle sequential tests or tests running up one after each other and kind of hand, and for like you could say, uh the minimum possible cleanup that we need to do in order to be able to successfully run tests at regular intervals. So I think we are one step uh closer to that, like. I think we have that minimum thing now. Maybe we do some more optimizations on.

D

This that's awesome, yeah, the tear is: are these uh manually invoked tests, the environment that they're running in? I think you just answered this question of mine recently, so I apologize, but are they being torn down or to what degree are they being torn down today in between tests.

E

uh Not a lot uh not a lot, so it's ultimately the hardware and the users etc. Those things remain same. I do uninstall the mini cube cluster and I do sort of tear down my uh the service mesh part, but the service mesh package is mainly because I was getting one small issue there, so I just tried doing that, but other than that everything remains constant in that environment.

D

Good actually, in some respects, that's an even better answer, because because then one mesh can't say: oh well, you can't compare these because when you provisioned, you were using that that specific hardware over there and now you're using a different one from this type like so that yeah.

C

That's great so then it's same hardware and it just um so. The mini cube cluster is torn down and raised like from scratch between one test and another.

E

uh Correct see right now, right now what I am doing. I am not tearing down the entire hardware. I am not uh sort of forcing uh any reprovisioning or anything of that sort, uh but I think eventually, I think the way jin explained in his particular flow. The way he's probably going to do that he's probably going to use terraform to provision servers on demand when something is run.

E

So I think that in a way would be that flow would mean that we we are provisioning servers at the start of the flow means we are clearing down the entire cluster at the end of the flow. So I.

D

Think that means.

E

In the final flow, this this would be the idea, however, right now uh it's just one after the other and I'm ensuring that uh all these steps are done.

C

Yeah, I think it makes sense, because uh yeah, of course, eventually we'll want to do that cleanup because uh you know there's a concept of, um and you know by continuously, using that same, uh let's say uh you know, probation things. You know we could eventually see some degradation, but mainly because of like whatever data. That's still there, probably not in this case, but um you know, usually uh you know that's like the importance of cleaning up but yeah. I think this is a good good uh progress.

C

um Maybe I'm just too well um like I'm familiar with all this uh performance part so for fortio and work. Two. So are those two. What are they like uh looks like configuration, but um anything more specific than that.

E

Right, I think uh right, so it's still. These are service meshes for you. I think it's that I'm not too sure I'll just.

C

I think it's like the performance uh tool used right or not.

E

Yes, yes, I think that's.

C

C

All right looks good and what what what endpoint is being triggered or what's the actual test consists of.

E

What what the actual test consists of is actually a perf run like uh you just do a measuring ctl puff uh I'll. You could see it in the code. I don't think it should be different from the master per se, but there are some changes. Ultimately it's perfect for play, and then you give configurations. Then you give a uh give all these so mesh name, load generator.

C

So it's basically the is it the health endpoint of each uh service mesh.

E

uh I don't know, I don't know the specifics of mesh measures.

C

Oh, but that's defined in a specific configuration file, then correct.

E

Right so the configuration file right so here what we are doing is we are using something called as load test. uh This is what we supply. This is what we are supplying right, so load generator is for sure right and then we are uh basically running this uh on onto uh right. So there's another one, there's something called soap test as well, so these are the two uh things that are available to me in order to test right now, I'm using both.

C

Excellent looks good. Thank you I'll read more about all of this or interesting things.

E

uh Cool I mean.

D

Yeah um hirsch, that's it that's exciting yeah! I can. I don't think that those tests are lying to you that they probably are. um They probably are completing um I have to. We have to look a little bit more um there. I we need to empower you to be able to verify that as well to see the specific output of those tests as they get captured in measuring cloud, and so uh so that's good.

D

So, by the way we yeah it's been too long since we've had a service mesh performance meeting with shin and you and the others to like to help take those next steps to kind of help get us all on the same page.

D

So soon, um maybe it's next thursday. I'm not have to look, but it's really. It's really good. There's um isovent was just reaching out there they're wanting to participate. um Yeah I mean I.

C

Can't wait for us to start publishing. There were two more items in the agenda right and, but I think we're out of time.

A

So I think we can conclude now we have four minutes after so we'll. uh If there was one agenda item for per now, we can maybe continue that in the dev call or in some other call. So it was good to see all of you uh see you see all of you in the next call bye, bye.

C