Kubernetes Event-Driven Autoscaling Standup, 2 Apr 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2020-04-02 KEDA Standup

Description

Notes: https://hackmd.io/cEi_FerdQvyTvB1i-5U0kg

A

B

Is it is this? Is this zoom called secure because I'm here.

A

There's some new vulnerability that people are really about I heard.

B

That the zoom team has now just posted that they've stopped all new features and just focusing on security for the next two weeks now and it's.

A

Never the blog post, you want to be reading.

A

Alright, we'll go through feel free to come in folks if you want to just drop in your own, but.

A

A

Let's go ahead and we can just start with the agenda, so we only have these every two weeks now. So this first one is actually really cool. Tom I'll. Let you kind of share this or actually no we'll go around really quick before we go to the agenda and we can just do a quick, just a quick kind of less intro I think we've all been on the call before, but just a quick kind of recap. If there's any items you do want to cover in this call that maybe aren't already on the agenda.

A

So I'll start I'm Jeff for those of you who may be watching the recording, I guess you've never been to a stand-up I, don't have any big updates. The only one is. We do have a Microsoft blog scheduled to go out on Monday that we finally got kind of like blocked away. I need to still write the content for it, because there's a blog that we did post but I'm gonna link to the blog that we did post it took a little bit longer to get the blog set up.

A

So that's the only item that I have is that I'll be helping amplify that blog on the Microsoft open source blog on Monday other than that I've spent a lot of my time doing: nan kata things, unfortunately, so I don't have.

B

A

B

The blog blog going to be about amplifying the fact that Kara is part of CNCs is exactly.

A

That yep and then it will link to the blog post that Tom might show in a second that we posted on our brand-new site. So honored you can go next, though I don't know if you have any updates from your so.

B

I did the same thing: it's been with all the just trying to adopt to the new normal, and this is so meetings going on and just like just for Tom and Nick we're also you can imagine like in Azure. We are dealing with a lot of e in a good way, a lot of increased usage, because of so so there's lots of lots of lots of stress at different levels.

C

Here so lost found that we filed a request with the CNCs to do a website revamp and as of fighting yesterday, that's now merged.

C

It's beautiful yeah.

A

C

As part of that effort, we also now have a blog and my follow-up action from the previous standard was to move mine to Google Docs, so it could be reviewed, and now it's basically, this I use the day off. When did I. Add it Tuesday, so 31st of March, yep I, think that's. Okay and that's. You guys think that we should move it back in the in the past. No.

A

I think that's great I think that's really good and yeah. I I have an extra item too, because when I saw this blog I, probably just need to share out I, don't know what the best way to share it out is but credentials to like buffer, so that if any maintain errs want to post or schedule a tweet off of the k2 account that they can I'm more than happy to do it. If people ping me and they're like hey, do it, but I also help try to amplify that too. So.

A

But this is great Tom honestly, the doing the work with the CNCs design team, I've.

C

Been doing it I just filed it, a quest that that.

A

Alone, you knew it was an option. I didn't even know that we had an option to say: can we get design resource but then getting the blog out? So that was a great one. Cn.

B

Cf actually has sources to help out with a log like website, design and stuff. Like that, I will post a link.

C

In the chat and it's a gator, playful called CN CF service desk and there it lists whether it gets two of the projects and I'm actually now looking because Jeff mentioned sharing secrets and one of the tools they have is large boss, so.

D

I agree should.

C

Of course, to have that for the theme and the yeah.

A

That that's a great idea that that's a even better way than me sending passwords and hoping that everyone keeps them secure and.

C

Another one they added this week was make I, don't know if you guys know snake.

D

C

That image scanning- so maybe that's also one we can add. Oh.

B

C

We're bored from day we.

B

Have worked with them in the add your functions context right, alright,.

A

Yes, because they have similar functions, capabilities to scan deployments very nice they're, a great company sweet.

A

So Mia do you have anything you want to share today. It's great to see you on the call yeah.

E

Hey guys, it's the first time I'm joining the community call I'm working in Microsoft as a CSA. We have couple of POC is going on with a couple of large customers and I'm having some issues with care, especially for job scaling, so I thought that will join this call and get your advice on the issues, and it is a blocker for us, and so that's recent joining it's.

A

Great awesome, I just add it to the agenda, we'll grab it. We might get there pretty soon, but we'll pop to it before we sign off for sure, even if you're a little bit over yeah. Thank you thanks for joining so Mia Ahmed. Anything from your side, I saw you've been.

F

A

On a few PRS and such no.

F

Nothing really on my side right. They call to be ours when the time yeah, it's been busy great.

A

G

Hey hi Jeff Hey from the cadence I'd, actually right now, I'm doing is to use the kid all to scale the function when we do our project right for the coordinate is the app services that what I'm trying to test out other than that I. Don't have anything specific for the kid. Oh great.

A

Sounds good, zip, eunuch, I know, you've been busy and I even see a few agenda items here. I, don't know if you want to since you're the last attendee that I need to check I, don't know if you even just want to jump into some of your updates. We can start some discussion or or yes.

D

Yes, we can, we can jump right into the agenda great.

A

And Daniel this was Daniel's item right, whoo, I, don't see on the call right now the path to commit our maintainer I can kind of flag what he was interested in. We can have a short discussion and then we'll move to v2 shortly. Did anyone else besides Daniel put this one: I've got the cameras over here. That's why I keep looking over here? Okay, I think that was Daniel. So there was a question Daniel. If you know he's been pretty involved from astronomer day. Oh he.

A

He added this agenda item, which was Tom, actually did a great job, giving some thought into a doc here around governance and in the doc we talked about project maintainer x' and how a maintainer must remain active and stay responsive and that we can add new maintainer x' by the maintainer voting and I. Think Daniels question was yeah, he's really just willing to help, but I think it's a good question to answer. I just want to get everyone's quick thoughts. Quickly is. If someone is interested in being a maintainer.

A

What's the process they should follow and do we have any thoughts around how many maintainer x' we want to have it once I, don't know if we should say like hey, create a github issue and say that you're interested in helping as a maintainer and maybe kind of give a reason of why?

A

Maybe we only want to have I mean I I, don't know what the list is right now, I would guess four or five five like I, don't know if we want to shoot for five or less so I'll just stop there and see if, if anyone has any thoughts on on on what what we should do, if folks are interested in volunteering to be a maintainer.

B

From the number perspective just curious, if CN CF has kind some kind of are other projects.

B

A

I, don't know I'm, not sure on that one I don't know if anyone else like I, don't know. If I looked at like even a big project like helm, how many, how many maintain errs they have at one time.

C

I think it's typically kind of small, so my suggestion would actually be around six or so I guess.

D

The odd number of vendors- yes.

B

D

Good, it's probably good yeah supporting.

B

Actually I was going to say yeah to break tire to break for voting. Okay.

A

Well, we don't have to spend too much time on it today. Maybe I can take an action to create a github issue around this for more discussion, but it sounds like we. We want to try to keep an odd number. I. Think five is probably great. Seven might be a little bit much, but at least folks who are interested can create an issue, and then we can kind of chat about it on a on a by need basis.

A

So at least then we kind of have the answer to someone who's like hey, I'm, interested in being a maintainer. We could say: look we kind of want to target five blah blah blah. If you're interested create an issue, we can talk about it in a stand-up. So okay, sweet I, mean add. That is an extra time really quick, Jeff whoa, my keys were on the wrong spot. You have to create issue for maintainer.

A

My keys are so clicking all right, v2.

C

A

Sorry go for it. I presume.

C

You laugh at the, but was it the government's documentation yeah.

A

My initial thinking was to create an issue that we would kind of closed on this discussion and then we could at it, but I could go straight to the PR for the governance, one in at least like we could say we we at least you know we want to have it be an odd number and then maybe I'll go straight for the PR we might as well we'll go for the PR. We can have discussion on the PR and decide from there. So, okay.

C

A

Sweet okay, 2.0, zip your neck I think this is uh this: is your your time to shine.

D

All right, all right so I was thinking about about like possibilities that we can extend. Takeda I both like the features we can add. So, as you remember, we were talking with the committed guys about duck typing concept. So I was looking at it and I have found another solution. So basically the motivation behind it is that at the moment we can scale just appointments and jobs and we, like I, was trying to find a way how to scale just like more more resources like different ones, so stateful sets and some custom resources.

D

So the first attempt is the duck time. Basically, the typing is is let's say, because go code doesn't have like generics and all this stuff so duck typing concept is about it. You have like a shape of an object and then you are trying to, and if you have another resource you are you, you are trying to guess if that resource fulfill all the requirements by the objects. So let's say we will have like some kada scalable thing, whatever like the duck type, and it specifies okay.

D

If your resource want to supposed to scale, it should have, for example, in the spec field, it should have replicas and it should have other fields. So this way we can define the duck tag and then, if anybody has like some resource and he like basically fulfill all the needs, so just like the spec in the spec together and pick us and the other fields necessary that are they defined in like type this way you can handle these resources basically generally.

D

This has some analogies and some methodologies, because I was at how the screening is happening in the kubernetes space. So basically you can have. You can have multiple fields. You can have like that, so that great, because you can have the replicas hidden somewhere inside your spec in another field, so the ductile thing wasn't.

D

The best I was like the best approach, because the other other thing is that in kubernetes, if you want to handle resources, you need to specify the are Peck, which is basically the roles and permissions for for your operator for quickly the operator to be able to do, let's say, handle these resources. So, for example, if we at the moment want to skill deployments, we need to add permission group.

D

If we want to scale some resource that at the moment, we don't have the resource, what was the name, it was the kind was the net. Was the API version, cetera it's a little bit difficult, so we need, to, let's say, do a hex with some annotations etc. So then I have found out that there is basically, though, in kubernetes, there is like a skill, some resource, which is basically a standard way how to scale resources.

D

So at the moment deployment has implemented this custom resource state for set has disappear, sub resource and if you have any your custom resource and implement, discourage it basically a end point which is basically handling the scaling. You can. You can scale any resource you want, so I move this direction and even if you have the Arctic, the permissions etc are much more easier because because you can say in our big file that you want to access like skill, some resource in every every resource, inter-cluster, you can put like the asterisk over there.

D

So you can have an asterisk slash scale. So, even like the permission thing is it's pretty much solved. So this is the this: is the proposal, basically that we will. We can modify kala a little base that, if you'll be able to basically to be able to scale any resource that has this skill assembly sort defined, which is the other other issue linked may be in the yet. A 703 m.

A

D

A

With a piano is, is it linked to than what I of 703 right here yeah.

D

Yeah, so basically what we have to do. We will have the scaled object, as it is the right now and in the scale target ref field. We will have like this free, like 300. All right parameter is basically the name which is at the moment it's just a deployment name, but we can put like the API version and current in there. This would be optional. So if it is not specified, it will be like just a deployment.

D

So if you have your custom resource, for example, art go out, which was the requested in in some in some issue before you just specify in the API version, 2 API version of the art world and the kind to be back it should be, it should be it.

D

So it's it's like a minor change in the in the spec and it could be level. There is one one concern like handling the secrets, because at the moment we can basically get the secrets from the containers into the problem and so into into Ott's skilled objects. Spec we have the container name property which could be like fruit.

D

So with this approach we can do the same, but it will work just for deployment state for sets. It won't work for the custom resources because the custom resource could have the container hidden somewhere. So so we can.

E

D

Way so, basically we can. We can support like the handling the secrets, as we do it now, just for deployments estate for sets, which are the standard comparative subjects and for a custom resource. If user wants to specify the secret, he will is to use the trigger authentication custom resource as we have it now or we can or we can drop the support for the secrets in the deployments for v2 and just use the secure authentication. It's it's up to our decision. I'm, not sure what are the preferences over here great.

A

Very that made it sense any any thoughts from folks on the call, questions or alignment agreement concern in terms of.

C

Consistency, I would say, regardless of what you're scaling it should be. The same experience yeah.

D

But I suppose that, like the deployments, would be like the 90 percents of of any like objects that we are going to scale so I'm, not sure. If users are like like familiar with with the concept we have right now, so they tends to expect the secrets in the deployment I don't know. So it's yeah.

F

I think that's a good question. I'm, not sure how how many people use that, as opposed to just specifying trigger a notification option grab it a secret from the deployment simplifies the scale object that you need to deploy. If you can assume that everything matches like the secret I'm, listening and deployment or the same secrets that I need skill on at least that's how other functions used to work. So that's why that was added initially. But it's a good point.

F

Good insignia is raising I, don't really know how useful that is for people or how easily it is yeah.

C

The reason why I was saying that is that I.

D

Real suit I could add the support for further further for the secrets into containers and we can drop it. I know it's I term opinion dis I got your point on, but we should have like a unified way. You know, I, don't know.

A

Yeah, my only thought like I do and it might not be as big of a deal anymore, but my main like even when we were going to 1-0. My main concern was for someone just getting started with kada. The ability for them to get started very quickly with is minimal concepts as possible.

A

I think is beneficial, and so, if they are going down like what we think is the 80 percent happy path if they can just be like hey I, just have to like I'm, already mounting a secret on I'm already doing these things all I have to do is add this one more CRD for scaled object. That sounds nice to me. So that's why I kind of like the getting started?

A

Hello, world, inline secrets, but I, don't know I, don't know how painful it would be if we're just like look just do scaled object, plus trigger authentication and then it's consistent for every type, even if you're in the 20%, not the 80%. So that's my only thought is that like would it be too much concepts for people just getting started if we made them wrestle around trigger authentication from the get-go? Let's.

C

Keep it around then, because it's less breaking as well.

D

Yeah we did maybe we can. We can think about, like other name of the object property, because container name is like somehow miss Lily, because people could expect that. Basically we are scaling that container and it's not you know it should be different I'm from Mike. We don't do. It should be like our no environment source container or something like that, because we are not scaling that container. We are just grabbing the grabbing, the secrets and environment variables combat container.

D

So if we are going to add that into the new spec, we should probably find other.

A

D

A

That's great, though, honestly great investigation, I, think I think where you landed I like this idea. This scaled objects that I'm looking at here the flexibility to do this looks really awesome, but.

C

Doesn't that involve the security risk, meaning defy the scale container at XYZ, but I want to get the secrets from container ABC? Where am I understanding it wrong?.

D

No, like the to point, is different because, if, like the container like the container name like or the quedagh is using it for because in deployment, you can have multiple containers in one appointment.

E

D

If you scale the deployment, you always scale all like containers, you know you can't scale just what.

C

You know just have a look inside okay, I thought.

D

C

D

I was just thinking of moving in incrementing a property, but I wasn't able to find a like proper name. Okay,.

C

D

So so this is the sister. This is the one. Basically, the wine program proposal and Ravin proposal is: is basically the skilled object or like steering jobs. So we can, we can use the we can use the same same approach so basically use the skilled objectors, custom resource and just reference the the job directly in the in the skill target ref. So we will have just one one field so, instead of typing, you know deployment and crime. Whatever you just put the put the job in there, we can do that or we can.

D

We can introduce a new you custom resource for for scaring the jobs because I'm not I'm, not expenditure under on the jobs or under scurrying, but I. Suppose there were some requests that the job requires slightly a little bit different options than the standard employment. So maybe.

C

D

C

This issue, because I saw different expectations based on what they're trying to do, because with the skill, that's why I introduced two different names: let's call it scale deployment or staple set, or whatever it's typically a demon constantly running and as the metric goes up. Let's say they want to add more of those instances well with a job.

C

They actually expect that if there is a new message, it will start the new job, who will only process that one message, but we have only one concept of the scale object, so actually we're mixing two scaling approaches into one cr-v yeah. That's why I would prefer to split them so that it's clear what the scaling of no kids that make sense at all yeah.

A

I see what you're saying yes, sir Ted, if I'm understanding kind of what, what what's being discussed right now is we could do what I'm showing on the screen right now and just having scaled object, and we could let you say the kind is job and the API version is whatever the API version for job is, and we could then start doing exactly what we do for jobs today, which is like hey for every cue message.

A

You create a job, and you have like this buffer of jobs like we could just shove it in which is more or less what we're doing today, but I think Tom's concern, which is a good one. Is those two behaviors between scaling a job and scaling a deployment specifically are very different, like a deployment is looking at the cow and trying to feed in how many instances are needed. The job is really like: creating queues, rosser's.

D

A

There every single message, and so the concern is, if we cram them into the same CRD, is that more confusing to people, because now they have to kind of like, even in our Docs we've kind of made, the approach that we break them apart and that we have you know, scaling jobs and scaling deployments, because it's easier to reason.

A

That way should that be reflected in the CR, D and and I think what that would turn into because I still like this flexible version, I, don't know if that means like stateful set or agro deployments like do those have to conform to either scale deployment or scale job or now are we in a spot where we might actually have multiple kinds, I think it's a fair concern, and maybe I don't know if we'll be able to close it on on this one or if we want to just keep having super poses and github.

A

But it's it's good discussion. I know. If anyone else has any thoughts, yeah.

D

Sorry, just for one more note, so if we introduce like a separate customer resource for the job, we can like put a different. You know spec in the in the spec field. If there are some, you know needs for for different behavior different. You know options properties for just for dog jobs, I'm, not sure if there are any like any requests for this, but this could be like they should be taken aback.

C

The reason why I added that that issue is that we had a couple of support issues where people were confused about the outcome, while it was actually working how it should, but it was not really based on their the type they were using and.

B

I think Mia's here to talk about maybe one more issue: more jobs, so yep.

A

I do wonder because I might like I, don't know if anyone's thought about the stateful set one and what even that means, when you scale a stateful set but I've assumed scaling. A stateful set would be more or less the same kind of logic of scaling, a deployment where you're.

D

A

In metrics and deciding so maybe we don't call it scale deployment and skilled job, but maybe it's like scaled metrics and scaled messages or something where it's like. One is doing something per event and the other one is doing a metric thing, and maybe then it makes more sense that, like okay, you know the stateful set thing as part of the metric scale. I, don't know what the right names are, but I.

D

Would keep like the skilled object folder for the generic one, basically for the one between right now, so we can call over all the CR CR or custom resources. It's as if I discuss every source should be there and then scale job or something like that or skills later for yeah I see okay, because.

A

Because is because you handle.

D

All like the state for set deployments or even like a custom resources you handle the same way. Basically, you are getting metrics and skill data. That's it! So that's my point. Yeah.

C

What I was thinking was him using the same names like the open application model, where a job is a task basically and deployments and stateful sets, are demons which are constantly running and now that we're talking about this I'm wondering, if maybe the more flexible approach which is not maintained by us where they bring their own research. Why don't we separate that? But maybe with because it's fully the same as a deployment today, but not.

D

Yes, it is, it's not basically.

E

D

Of it, so you have one custom resource, find a skilled object or it could be like a skilled resource or whatever, and you handle them the same way. You just basically scale them. How.

C

Can you do a recom.

D

Do we call it scale, object the scale, research all right, I would.

A

Keep the scale jitters, I, don't know sure yeah.

D

A

I'm, either way like skilled object, has a nice like it's kind of a carryover from from v1. It's not necessarily the best best noun, but it works, but I do I. Actually, my I personally know I'll pause in case. Anyone else has thoughts. I do like kind of this proposal that we're chewing around right now, which is we keep scaled objects which scales deployments.

A

It could scale all this cool stuff, but we introduced a new kind may be called scaled task or scaled job or skilled worker, which is specific for for doing these job type, wear clothes which for for at least now the only one I'm aware that it would scale would be jobs, but maybe something will pop up in the future that would fit that model as well. That could fit into that CRD kind. Well,.

C

Like a natural function, isn't that good? We use the same approach, I.

A

C

A

A q3 good, no hay native K native is closer to that one, but yeah function. Does that, like the scale, deployment or scaled object.

C

Approach native it's closely related: will that mean that they want to have resources as well.

A

I, don't know if you need, because K native has its own autoscaler today yeah, but somebody today.

D

Was because of me basically, when I was thinking about its unified approach on scaling resources with the skills of resource I was talking to some guys in the infinitive community and they kind of liked it. So they were thinking of using it for for the eventing, for some, like parts of the event, think so skill skill, some some things over there, so they kind of kind of liked it. So that still is the way why the guy, like commented on this great.

C

Great, but doesn't that mean that it needs to work with jobs as well? Then no.

D

No, no! It's basically, basically, if, if we define the skills objects that it is able to scale any custom resource with the skill sub resource, this is the thing so.

A

D

They give they implement the skills and resource on the resources they want to scale they could they could easily plug in a skill resources. My.

C

Question is more in terms of how they want to scale so.

A

I think it would be more a scale deployment because they have a dish yeah.

D

A

D

Not a lot of jobs, it's about objects already on that scale. Okay,.

C

Okay, I think we have two very nice features: yeah great discussion.

A

Okay, anyone else want to chime in on any of these v2 things. We've been talking about or more before, we move on to the next item and and get to some.

C

Of the you know, maybe.

A

C

More, if that's okay go for so we've seen. A lot of confusion is the big word, but there's no consistent way on how we name a configuration, certainly if it's contributed by new people. So that's why and I didn't add the issue, but yeah we want to propose to basically go through all the scalars and use the same name for the same things and also the same approach like some call. It address some call it connection while they are just the same I see and some people want to have a host and board separated.

C

So we would like to see if that's interesting as well, where, if the connection.

A

C

A

C

No connection is typically the full host with the port and someone to have the host and the board separately. For example,.

F

G

F

Was no somebody.

C

Asking so we could use the approach and actually that's how we added it now. I think is that if the full address is specified use that if the host and the board are specified separately, use that as a fallback, if not there, when we throw an exception, because because we basically rely on the information being available on the container, so we can't really enforce it.

A

Okay yep, so it's kind of recap the potentially creating some I would imagine as part of this, we would create like this is our recommended naming convention.

D

A

Metadata of scalars, and as part of that it might even be refactoring ones that have already been merged in once we land on that that right pattern, yeah.

C

So we need to define the I think we need a scale at requirements document, let's say which lists the names that are available now we need to go through all the scalars and all align, all of them and the beauty of space over a major change. We can just change them without all the backwards compatibility I see. That's.

A

C

A

Makes sense any thoughts on this one.

B

I'm sort of a little bit I just feel that it should be yeah I mean you can have a recommendation, but but every is like: what's because everybody who's it, who wants to write their own scaler and because, because every event source might have a way, a different way of calling the same kind of car, and so we should be more, it should be more closer to what the event source traditionally calls it on what it uses, rather than what we think we should do with respect to kada right that just my thought right.

B

So if somebody is using Kafka or somebody is using, then whatever the same concept is called in Kafka, it should be natural for should be more closer to what it is called in that event. Suppose that that's just my that's just one thought. Instead of trying to standardize it across all the event sources, because, because, if somebody's trying to scale something is in Kafka or something you know, scale use event table or whatever these be the nation they would assume I assumed they would already know what does what these different properties are.

B

With respect to that scalar or that particular event scores what they're called and and and if it is called something else, although it is standardized across to Kyra, so it might be more confusing. That's just my two cents.

F

I sort of agree, at least that was initially thought that the name should match the scalar. So, for example, if it's house it might be a bit weird, but it's broker. You wouldn't know what that is, um but.

G

F

Because somebody might be using like rather than keel and Kafka, and then they have things looking different between them. So I don't really know what the right answer. Yeah.

C

Maybe a better example is what, if I use a rabbit and Redis and then maybe.

D

C

Prometheus and all the same things of call differently, but an option might be that we have the standard ones and we can also have let's say aliases, where you can. Let's say with: for example, you can use connection or you can also use host.

C

Kafka name is so that we have the standard way and I'm, but if people want to use much more natural to that scanner, they can also do that. Yeah.

D

C

Think that's what that make sense or not.

B

C

B

A good compromise I'm just wondering just like just to be very how much of an issue is it really is it like, because I would imagine, yeah I think I think if we feel that for a particular scalar, if we feel that the those properties are not sort of the right first properties for somebody who actually works with that particularly rent source, then we should definitely give feedback and say: hey.

B

Can we change it because this is more natural, but but I don't know if it's so much of a problem that they are not standard across different scalars I, don't know if it is that just my two cents, a good.

C

Example is connection Springs, some call it connection strings. Some connection, yeah.

A

I was looking because I do I was actually what I was clicking through is I tried to go through all the Asscher ones, because I was like oh I, wonder if we did that, but at least from the ones I clicked through briefly, it looks like all the azure ones use connection.

A

So at least there's consistency.

D

There, because that is that's.

A

Before I give up.

B

So I feel it's fair if they're, if they're a couple of things like that which there there are where you where, where it would not be scalar specific right or the events or specific something like connection connection, is a great example, and for that we can give suggestions, and we can say that hey better but I think um but I think just going overboard with this is probably not needed, and probably even detrimental so.

C

Let's do this so we'll just have a look at what's around, what's confusing and then see if it's worth Chinchin things great I, think that's make sense, yep and now that sorry one thing pops in my head is the configuration one is now still flat. Were we going to make it more structured for v2 or what was the plan there again yeah.

A

It was brought up specifically in the concepts of azure monitor. This is what it ended up. Looking like an azure monitor so I know initially, mal wanted to do like like these would be in a sub list, and these would be in a sub one and I think they used to there be like three nested. This works today, I, don't know if we've heard much other noise from other scalar, so I'm a at the point where I don't know.

A

If I have enough reason to say we should go through and make it more structured, but I'm not opposed to it. I just don't know if we have justification for the work well, good, yeah cuz, the only one I've heard it from is Hatcher, monitor and I could see it like it. There is a lot of metadata here, but I think most of our other ones are a lot better. So I think.

C

That might be the.

A

Only scalar that would actually leverage it I think they.

C

Could all benefit from it, but in the new question is, is it worth it like? The the huawei cloud i1 can benefit, I should monitor, can benefit yeah.

A

Cloud watch would be another one.

C

Would need to check them all, but yeah. That's okay! I was just wondering what what we decided that I I.

A

Think it's a valid one I just I, don't know if I would bump it to the we need to make sure this is like I. Don't think I would block v2 until somebody goes and does that work that way it amendment would.

C

Know, that's not the breaking one I was going to say any version, the trinkets, because then you can change them easily no later on. But that's, oh sure, it's an addition. So that's not breaking.

A

C

A

You're good, all right so I think we're we're on track and we might jump to job scaling for Sowmya in a second just to make sure he has time within the hour. But this is a good one. Move stand up to an earlier time, so it bumped slightly with daylight savings I believe, but it is, it is kind of later than most other stand up. Side. I, don't know who created this year. If anyone has any proposals but I'm interested in discussion on this one yeah, it might.

D

Be basically it's for people from Europe, so because at the moment there is like almost 8 p.m. so I understand that this this thing is always tricky. You know, because all the time zones and regions so I was thinking about at least moving it like 60 minutes or 90 minutes earlier. It would be like it would you like help me a lot, for example like personally I'm, not sure about the guys, but would.

C

It help me as well I think, okay great, it would help my wife.

B

I think that I think I'm, okay with 30 minutes at least I'm okay with 9:30.

G

B

Think a mother, you can just not sleep that night.

A

The only one I'm aware there's the service workgroup, which right now is primarily focused on event, grid or I'm. Sorry cloud events, so I had to join a bunch of those, but that is an hour before this one, but I I, don't know how many of you end up having to go to that or how much crossover we suspect, but that'd be the only risk if we move it just half hour or an hour earlier. Is that we'd now be overlapping that stand up but I'm, okay with it I'm?

A

Okay, if we overlap one to one and maybe we have to adjust things later on, if that becomes a conflict, any concerns with that one. So should we do, should we go 60 minutes earlier, like 9:00 a.m. Seattle time or just 30, honor it and Ahmed I'm I'm up regardless, because I've got a four year old, but I I don't want to speak for everyone on the call. So.

B

The thing is that, while just to be a while, while we are in the this corona virus induced thing, then definitely I can do whatever time, at least personally, but once we go back to office, then I I can't do in line because.

C

A song that will happen anymore.

A

B

Will drop off after you drop off and I'm just I will not be able to come. There's no problem, I. Think right. So I can't come at 8:30 order, so I think the earliest I can go is 9:30. Of course we can do that reverse. Also, if it helps evenings is no problem, so it's.

D

B

It's mornings, for you guys so I, don't know how morning works for you guys morning.

C

Me you guys is now evening, for you guys, is our middle of the night. It's.

D

Minus minus 9 hours.

C

B

The only block of time I can do is 8 8, o clock to 9:00 9:30 in the mornings 7.

A

F

A

It worth doing half hours if your neck and Tom does that make enough of an impact, or is that not? And we can, we might have to pause this one. So we get to the other ones, but I gotta.

B

A

I do get the one on this farm.

B

A

Life goes back to normal 8 8 to 9:30 is a trickier time. I know for many, but maybe honor adjust. Maybe we just have to do these without on roads, sometimes yeah.

B

That's right! That's absolutely fine and.

C

We could give you old wood, that's fine.

A

Ok, I'm trying to figure out what the best way to land on this one is because I do want to close on it. Maybe it's over email, so maybe we'll just finalize it over email in it, I'd be interested.

B

In meeting a few and.

A

B

Already do a half another. It's ok! Firstly,.

A

I guess like want to be better than email, yeah.

B

A

Are we cool with that? If we, if we keep this going, but we maybe you sell it offline? Ok, thanks good good discussion, though I am very aware of this. Ok Tom I'm gonna skip these for now, if that's cool, so we can give Sonia some time. Cuz he's been very patient.

D

A

Mia I'll, let you go ahead and if you want to share some of what you're seeing and we'll see where we can help, or at least provide some next steps, yeah sure thanks.

E

If yeah I'm, so I'm working with the two different opportunities, primarily to liberate scared scaling the jobs, so it's that the customer has 15,000 messages into a queue and they need to process all of these messages in parallel and each message will take up to three hours from two minute.

E

So we are planning to use cater for that scenario in Cuban artists and the issues that we are facing is that and the scaling so well in even in my testing I saw that if I push the messages into the queue and if I, if I sell, if I send like 200 messages but I'm, seeing that Kara is creating like 350 different jobs and parts, it's not really. The number so what's happening is in customer scenario. They are creating an auto scaler employ in the Kuban and dis cluster.

E

So whenever it sees more jobs and parts, it unnecessarily spinning up more VMS. So it's a it's a concern for them too, in terms of the cost. So what they're looking for is that we need to get exactly the same number of jobs and parts to be created in parallel with the help of kada, and so that way they can manage either using virtual cubelet or using the node autoscaler, and also they would like to see.

E

Can we also control the number of jobs can be created like, for example, we have max replicas account for deployment type scalar objects, but cops is there a way that we can control the number of parallel jobs as well? So these are the two issues.

E

I was able to reproduce the same issue on my end and I saw there is a tuple to be request like the job is already in open, State, so I don't know if it is hitting directly on, but it's similar we are seeing that behavior see even my first testing I did with the parallelism, s1 and Q length s1, so I saw that each jobs are creating sequentially, then to create in parallel. I tried queue length like, for example, 100, then I'm, seeing like a weird experience.

E

I want to get your expertise, sedition and thoughts on that. How what would the, but is the best way to configure the scalar object to get this parallelism with jobs, yeah.

A

And if I understand right, just just to see if I'm hearing like the you're dropping, let's say a thousand messages in a queue and what you want to have happen is have some job spin up. But you want to be able to cap the number of jobs that will spin up to like 20 and what you're seeing right now is you drop the messages on the queue and it's just going crazy, so you're getting like 350 jobs, which is causing like the cluster autoscaler to kick in and a bunch of other things.

A

Is that a fair recap of what you're, seeing and you're in your testing yeah.

E

That is one issue and from other customers requirement perspective. They want to create it like parallel jobs like if I have like a 1000 messages into the queue they need to spin up a 1000 Jeff parallel to process these matches, it's so that they can, because each job is taking up to 3 hours. So that is one of the reason they thought that go with the cloud-based, so they can process it in parallel and they did it Taizo bats there. Everything is good, but they would like to see if they can leverage cuban.

E

It is for the similar scenario. Yeah.

A

In in the I guess for this, what the second problem I think and maybe others on the call got it the 1000 numbers in that case is that will that always be static? Like will always be? You know, always run up to a thousand jobs in peril? Are you saying that number might change based on how many items are in the queue, and you really just want it to always spin up and number of jobs in parallel, yeah.

E

It readies, and so it's a multi learning solution, some some customers when they create a job there will be maximum fifteen thousand messages and at minimum there will be 1000 messages for each job processing scenario. So the number of messages coming to the queue will vary customer to customer. So minimum is 1000 messages and maximum. It can go up to 15,000 messages.

E

So for that 15,000 miss a scenario they have to create 15,000 jobs in parallel to process. All this messages, with the help of Kara I, see yeah.

B

I'm a bit so I'm, just I, think just trying to get clarity here. It looks like in like I'm just trying to figure out somewhere which which problem, because they almost seem these two problems, seem at odds to each other, I. Think at one point you said that you need that we are scaling out too much and the other and I think on the other side, you're saying that we are not scaling out enough, so it's they're not being processed in parallel. So.

B

Do you want to solve so which one is it is it like? They are not scaling enough or we're scaling too much. Just I think we are trying to sort of get clarity on what exactly or you want both and you want to control it like yeah.

E

B

You want both the scenarios you don't want, but you just want that knobs to be able to control that hey in this case here. I, don't want it to scale out and in this case I do want it to paralyze. Is that the question yeah.

E

I'm, the first scenario is that the main requirement is that they need to process this one in parallel. So that means, if there are 1,000 messages, they want to see only 1,000 jobs to be created. Okay, so, and what are you seeing? What are they seeing so, for example, in my testing, when I send like a 200 messages, basically it in like a three hundred and forty seven jobs so care, I spinning up okay, so we are expecting like a 200 jobs to be seen with the help of the scaler got.

B

It so you're saying that when you have n number of messages you are seeing in plus some X jobs and not some exactly n jobs, that's.

E

B

A

Don't know if anyone here has experienced with from from the outset, I think that this should be possible through flagging. Some of these knobs and I think there's also some weird behavior in the open PR, where, like queue length, the number you set for queue length actually really impacts the number of jobs that get spun up in the metadata, so, in short, I believe the behavior you're, describing in that scenario it should be doing what you're expecting, but if it's not that, maybe there's something wrong, I don't know.

A

If anyone has any thoughts where you're like.

F

Oh actually, no because I see if I saw Phillip ers talking about this, but I haven't I'm not too familiar with the job scaling, I, think Steve, I sure guys now his last name was thrown into edit I. Think. The best course of action is for someone to get take a look at it and see why I can't make it they got. One I can test the job scaling and see.

F

A

F

A blending thing a look at the existing PRS for job scaling, so maybe.

A

Before the start go for it's a benign yeah.

D

I was about to say to say basically, if you, if you can create issues for both both the issues that have it'll be great, so we can discuss both yeah.

B

So so so I'm gonna do here that just created shoes for those in the github repo yeah.

D

B

You haven't already and generally, if you have a if you can give some kind of a sample app on a deeper than that yeah.

D

And so, and these things could be like basically taken in account for v2 and for the scale jobs right now, because I think that the the job scaling is still a bit alpha. From my point of view, yeah.

E

Yeah sure no yeah I saw like a two issues in the in the open pull request that a tuple to various similar to this particular issue. What we are seeing in the pull request, it is still open. So any idea that when it may merge- because this is what very time constraint to the customers of you may need to go with a different route. If, if it take time to stabilize the job scaler.

A

C

Just American landscape, that's like so so an option.

A

C

A

Outputting an issue here: this is the one you're talking about this. Oh, it's, a different pull request, I'm, trying to figure out the issues you mentioned yeah you might just have to flag us. It's it's hard to say: I, don't know! I haven't looked at these portal quest, so I can't give a definite answer, as Amin mentioned, he's planning to look at it. But if there's other hiccups here it could delay these.

A

But if you can at least even add I'd, be really curious, like what are you doing when you're repro Inge, so you're dropping X number of messages in a queue. This is the scaled object that you're deploying with like every single value that you're using the secrets. Obviously, even if you add that to an existing issue, that would be fine, but if you can do that, it can help us accelerate in general. The next scheduled release for kada is two weeks from this week.

A

So best-case scenario that would be the earliest should be in the docker hub, but it's possible that it gets merged and you could use like the master branch to start pio saying even earlier. Ok got it yep thanks great.

C

Okay, we're at it I'll go for it. Seven only thing on the 14 note this week, yeah.

A

A two-week to it, the next oh okay,.

C

Because somebody.

A

Was asking and.

C

I was saying it will be this week. Oh.

A

B

A

That's right, Android is that weird you're marking as well yeah.

B

All day is there: is there any sort of big changes or anything that is blocking.

C

There was a new way to configure ready, yeah.

G

C

Was something else that people wanted? Which I cannot remember, but.

B

If it's not blocking- or it's not a big thing, then we just stick to that, but a bigger task on the helm.

C

Chart from where people say, why do we have to wait in.

A

That one I think people can just if they just do pull requests with the chart. It they'll push right away like we don't have to wait for releases on how I'm sure yeah yeah.

C

But the question is: do we ask people to always do that and have a lot of increases or the wave more frequently do that I'm.

A

At any thought, so now I'm cool. If people just update the existing package and like I not going to approve a PR if it looks like a breaking change and now back, you need to do a new version, but if people just want to do a live, update of the helm chart and instantly release it, I'm cool with that I mean.

C

I, don't know if you have any concerns.

A

C

Then I would suggest to add it to the yeah. I'll. Add a PR that late sure.

A

C

Says read my steps yeah.

A

And people don't have to, but if they want it, then I'm more than happy. If they're like I, here's the updated chart. Okay and.

C

Just so you know, I will reintroduce the readme for the ham charts when I find time so that all the options are listed so that people know what they can tweak and what it's for certainly for the helm hub. But they know what's up great.

A

Okay, we're at the hour there was one item we didn't get to I'll. Let you make the call on this one if you want to squeeze it in CLA bar on docks. It.

D

A

D

The boat is still there in the in the docks. Repo. Oh, is this the Microsoft.

A

Open-Source pot yeah.

D

A

You I'll flag them to make sure they pull that off and.

C

There's still a whole heap of Microsoft people in the organization. Do we keep them or Microsoft.

A

Or like the open-source compliant bunch of people like.

C

A

If their owners or maintain errs, then we should put, then we should read them out if they're just members I, don't think we gave him any permission. It was mostly when we were private when we were a private org. We had to manually at everyone, but I, don't think they have permissions to do anything. So we can, if we want I'm fine, if we pull them out, I find if we leave them.

A

Okay, great another full hour, anything else before we sign off anyone wants to make sure we get in while we're well. We are here in person very good I guess one would think. Sorry before you.

C

Go shall we, you know, start sharing the blog post to do wait for open source. One I think.

A

Go for it now and we'll just amplify it again on Monday. So if you want to send me the text for the tweet or you want to do a little password dance with me right now and I'll, give you credentials to the Twitter account but I'm happy. If we start promoting and making.

C

A

All right: okay, thanks everyone we'll talk again in two weeks. If you need me before that, if you ping me on slack on the kata channel in the kubernetes group, I can help out especially run like the scaled job thing so Mia, either on teams front slack. Let me know if we need some help making that progress.

A

Thank you great. Thanks to ten, everyone will see in two weeks, like you guys.

B

Lie to you about how you do.