Kubernetes SIG Scalability, 2 Mar 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 2017-03-02 Kubernetes SIG Scaling - Weekly Meeting

Description

2017-03-02 Kubernetes SIG Scaling - Weekly Meeting

A

ah So this is the Thursday marks. Second scalability sig meeting public recording there we go I guess we can get on out.

B

Of here, where are those two targets.

A

B

A

B

Of stars like I, also thanks to looking it up for it. Oh.

A

We've got we kind of got a mishmash of them and they changed the update and we're behind the update policy like where you're supposed to store them. So we have put them on a on a different youtube channel there for a while and we're supposed to repost them to someplace new it. We just haven't done so.

B

Someone put it into the boats, yes,.

A

All of this is this is open it into the notes. Right now. I need to get the recordings.

C

Uploaded to the new location.

A

Yes, yes, indeed, we'll work on that thanks.

D

So closely as a channel to know.

A

I just want to get rid of that. Well, the old channel blue notes somewhere, but the problem is, is we have a mix of videos from before and after and we sneak it through and we.

A

E

A

You know I need to work on that. No okay, the the main thing I wanted to cover today is and I was just about to post. The link here is.

A

Your out stick here, but let me also.

A

So I just I just suck a link to a presentation in the notes. Can you look at the notes? No I can't share.

F

A

F

A

Is there a free to give a a cig scale update 380 minute this morning, so I just put together two notes and I was something we can get some input. There's a couple of spots. I was a couple of places that we should probably have an update around anyway and I can get the updates properly.

A

So, for example, I got I think I have the right link to the performance. The performance run comparison stuff that you showed us because I think that's one of the one of the interesting bits here recently here. If you can, can you share that presence can share the link there? Okay,.

F

What's going on we're.

B

Just looking at a slicer starts like Golden.

F

Rose stepping into stuff these are different. You is that.

A

I'm on the wrong track here: America, but.

B

Anyone actually believed that one thing for 70 days with you know.

E

How it works, nobody else, absolutely not mom,.

B

It wasn't only girl, I were really quiet.

G

In there can you get closer to the mic.

H

Send-Off down on that, it is off until they.

A

H

A

Out alter your speaker off kids. Sorry, you all right cool! We want to do in a shirt, though sure what okay sure that rose. It is intermediate.

G

No but I mean on it. The topic of your.

I

Life, we are going to announce some believe it. This way.

I

We can't hear you, what is your name so? Can you give me now yeah barely.

E

I

So basically, we are going to announce 5000 notes in some limited skulls like what like it won't be. It won't be like crazy. We support 5000 notes, but it will be phrased in a way that we can around 5,000 note, and we have some tests like we have it like eggs or testing.

I

We have said that are quite basically pressing. This cluster and I are like satisfying our solos.

I

Ok, it still means that, like there are a bunch of things that may potentially not schedule like, for example, like services for service propagation may be all open curly, but we are going to say something from Allah.

G

So something is just.

I

G

A blog post certain there will be above branch.

A

So so the topic came up. I attended the the lasting team meeting where this came up and what I told the PM meeting was that we believe that if I help with control, plane, engineering and care the 5000 totally possible, but we were presently carrying it for the ones that fix release as a specific goal. So I think it's easy to get a little bit tied up between the eye from a marketing perspective for the project.

A

That totally agrees, you know saying: look we can run five thousand if you're, careful and you know what you're doing five thousand those totals doable I mean this one's a statement we were, we were making around 1000 of stuff back when the project was hundred notes.

A

A

So with what's your, what's your suggestion, I mean we can just have, we can just make that commentary at the meeting that I think you know, project status is a little different than then marketing output. Almost patience for you.

I

This is a good question like a like. Basically I know that we will announce something like Aparna, who is like our p.m. will announce something like pretty much. What like I think it's phrased, that's like five thousands notes, some folks testing, her education costs or something like that like so, which obviously doesn't mean like fully. We supported.

I

You know whether what you do about this like it's, not that, like there's, no performance targets, person great so have you run five thousand.

A

I

A

So I think I mean the state when I made it to PM group is aligned with yours and it's the same and I would make in public as well, which is, we believe, a 5000 s. Buster's buildable look care because we're not running see I for the project at five thousand.

I

Anyway, we are running like like. We are not running it continuously, but we are running five thousand elastics, not that we're on it like what well.

A

Enough is running it in FBI, as part of the community project is different than one of the community project members running an occasional test at that size, razy, but pacific.

I

A

J

I

Like we will never be running five thousand notes after as a community, some you need to like continue like it's just too expensive, I.

G

Just think, there's a difference between bragging about how performance has improved versus saying that now, as if Cuba nineties 16, you can totally do elements, but it.

I

Was looking up that weekend felt value whatever you do it will be? Is we are not safe? Yeah.

E

I think, as long I think we can say we run tests at this scale so that the you know we know for certain endpoints and services. That fix is still not in so like in this release cycle. If you had, you know a hundred services per machine and you load that cluster up you're gonna, wash it and blown right like wow like it will.

I

Be super slow about it it. It won't like blow to a thought for up well.

E

Well and all depends to find how you blow it up, because what happens is you're going to load individual boxes down there going to be the resource reservation for iptables. You know smashing the machine, so it's going to be weird. So, let's consistent.

A

With my payment, if you know what you're doing five thousand does is very doable you, wouldn't we it's not like you need to begin. Your user is going to crank up a five minute cluster anyway. So in some level it's kind of a silly discussion, but it's a.

I

Sub sorry I think that I'm looking into this slide and tedera said like no performance targets upgrades from 125 to enter, it doesn't.

K

I

About like that, 5,000 notices like so.

G

I think I went so it sorta phrase that like where we are always working to improve performance, but we didn't sent a stake in the ground, specifically 16 right, we're always trying to make sure that things go up into the right, but yeah I think.

E

I think it's okay in the community BTW could say that sandbox tests, because we in this is questions for the broader community, say that sandbox tests have been than 5,000. Then they were looking green yep. Okay, I.

A

Will I will work.

G

On that, the sensitivity here right is just you don't want this picked up in terms of marketing and saying now, Cooper days officially supports five notes. Well,.

A

G

A

Reverse we have the problem with the reverse, which is that the community target was excessively conservative and he got picked up the other way, and that was also bad. So, when.

E

I was two years ago, so that's.

A

I'm just trying to trying to keep from repeating that mistake again vocal, both ways right, yeah I'm, an extra son to the bonnet, because I got beat up on this so much in public. No, the google folks did.

A

Alright, any other any other off a big concerns.

E

um Alright, your upgrade tests are all green. Now, after you had a couple of problems, wojtek who's up to the hole I'm.

I

Not sure about, like you, mean, operate, that's 43 or upgrade this in general.

E

Ok, test for a t3 and okay about the anyone's really.

I

Not really, I think they are not green or not fully green. The HDD b31 are pretty much agree. I think that's like the problem that we are observing. There are like there are some refresh rare slicer, but they're like no one like really has kindly looking into them deeper, but I looked and the first glance they are just like sites for regular test. Not the not realize it's jency giving free, but.

F

G

Means Pacific links to the right test. Great thing, I've been seeing some all green runs, it's the rollback stuff, it seems to be linking, but the upgrade stuff seems to be working. Ok,.

I

A

I

Clarifies like Aaron's like there are basically two two things. It's like like upgrades with upgrade control back in the 1.5 branch, which seems to be pretty stable along.

L

I

And upgrade and upgrade and roll back in later, which is like 1.6, which contains a bunch of different up. Drake, said that are slightly by because they are slightly not because, like it's SED, be three others. So.

E

We I think that we.

I

Should mostly look into 1.5 until we get like Papa, be like a break testing.

A

Ok um I mean let me try to boil the boil down an acceptable summary for the community being so I cd34 like fresh cluster work is in good shape, for one dot, 5216 upgrade is in good shape or rollbacks. We think we're getting caught with engine we're getting caught with general into the general purpose flakes, but generally everybody feels like it's. It's going to be in good shape.

A

I

Sounds good to me, ok, are you.

G

Just that dumb question: if you stand up a new cluster with 16, you default MCD, three right. Yes,.

I

G

Was that the same in 15 I could put it no ok, so I? Would we call that out that new clusters are going to default, mp3 I will call it out.

H

A

Let's say wind up five to wind up six upgrades in good shape with a 5-1 top 6 215 downgrades rollbacks.

E

That's not that's not a thing for anybody, but Google. No one does that. So just no one has the rollback support. So, let's just I think is a community-wide. They you can. You can articulate what the Google support is, but make sure your premise end with Google.

A

Okay, I'm always nervous about speaking for other people in these kind of community meetings, but I'll do my best on that's wounded, I'm, sure now, I'm sure, google, plus the disagreement free. Something won't be shy about speaking up in the middle of the update. So.

D

What you mean by a little as I, need it is off of it. Could we do it as a process.

E

That at city folks use for rope a snapshot before they try to do the upgrade. If it fails, they restore from snapshot. They run that way. They do not do a day of rollback data migration right because it's not supported so the the folks. You know why tech worked with the other guys from core OS to do a migration backwards, but that's not supported by anybody except for on Google's infrastructure right really so pretty.

I

Much yet so this this is raw like it's like kind of costume for us like it. It's everything is open sourced in this area, so everyone can potentially use it, but it seems not like only. We are really interested in having tough.

B

Which kind of surprising? Because, like we did no one.

I

Cares about cutting robeks, maybe if we do that, like if Google do like upgrades like, but upgrade the bunch of fastest indicate like in rowland or not wait, work well.

E

There is no depends, like people have outage windows for stuff like this, because 16 the thing that people are going to need to pay attention to is 16 is like a complete cluster outage window, because it's more than just a TD modifications. There's the secret hierarchy, changes that will force you to upgrade and offload. Your good looks like there's a bunch of ongoing stuff outside of the scope of this thing that is occurring in this cycle that will force.

E

You know, operators if they have a running cluster, to have a planned outage window at the 100. So.

I

To be honest, like in Google for GK classes, we will be upgrading to a CD three independent appraisals.

I

Like new clusters will be started with a CD be free, but their existing ones will be like. We will be upgrading independently to the series from from abrading two entrances. That was my regular is basically that's one or one will affect the other.

G

Side effects how the API server communicates with rat CD again a little bit out of it. Do you have like one way of talking and evening three versus be too I.

K

Must understand.

G

The question: if I have an API server, that's used to talking to a CD too, and you swap std-free underneath of it is it does it need to change what it talks? Whatever call you say.

I

Maybe you need to change the flag, an API server, okay, so.

E

You could well I think the question is like if you do an upgrade of your of your environment. You're already talking be too, and you have be to back in that you upgrade to a CD three you're, fine right. If you want to you, want the API server to run in b3 client mode, then you have to do an offline migration, yeah.

G

Wait, those are the I agree. Those are two distinct things when I was trying to clarify which it sounds like both of those will be done.

D

Automatically within.

G

Google, yes, okay, I.

G

Had a question that's unrelated to what we have on the slides, but I was curious where we're at in terms of decoupling kid mark from google I know there have been worked to make it a lot more provider, independent and I, wasn't sure, with the status of that worthless. Young.

B

As well, but it's mostly done like it's like possible to implement your own food into cement. Other things like you need to do some more to chart active mark on may WH or something, but we could read up oh yeah, but a clown flowing.

J

Move on from the cubic startup scripts, so it's also kind of valid document. In the comments on how to define this interface and you pretty much for a club toilet to start complying, the perimeter to read I like it define hold function. That is like starting up master instance and using once and then they literally as a search engine.

K

Is it is my gps not do I.

G

Create I think this is super awesome. I've been super excited about this for what it's worth and I want, I think you should brag about it, but I'm hesitant.

L

G

About if we don't actually have a working demonstrable case, this being used someplace outside of Google idea? Why we're.

L

Doing the bandwidth.

G

To test right but like if I saw to mark running on cops or something like chaos, that would be awesome right. Do you think that's feasible thing? Well,.

I

We can't do it guys, but it's not about cults. It's more about like running on AWS, for example, because, like venting out of the class setting up cube mark like machine, is completely different than we like backing up tomorrow. I was.

G

Calling you okay specifically because it seems like they're deprecating, if not outright, removing the cluster kuba for the AWS provider, so I think offices be now accepted. Well,.

I

What but like so it's also like perfume mark. We are not using Cuba for GE, it's like ready, tater skins, so you need to tell us also dedicated to the.

B

Static laughter database laughter, then you peel around the start to merge start doing. Buddy.

I

B

I

Original faster, you can do whatever it was like I Steven now like it's like kind of independent, but you need to have a dedicated piece of code that will create this, like cube, mark master vm, and it's like we are not reusing, Cuba or anything like that for digital educated children. If you want to run it'll, for example, on AWS, you also need to run a dedicated. Why than six bacon 14 p should.

A

I should I make a statement about the fact, because there's interest in having folks that want to do large to mark based on AWS, we out their participation or set.

A

I'm trying I'm trying to get the update for the larger community boiled into something that isn't quite so. Youth is short and interesting. I guess.

G

All right, coming, I think that's fair. If it sounds like what what I heard you guys say was yeah, we can totally run coop mark and other places. You just need to work with these things and we'd be happy to show you what those are. You can point you to the documents you can reach out to us for questions. Yeah.

B

It is pretty simple: I guy is literally creating machines. Releasing the creating machine vision, machine SSI done stp; okay, the thing that you would expect I.

A

I'll just obvious a large key mark made cluster destiny. Ws, sir, are of interest for a goal. Utricle I.

E

Think I now in my new, in with my new hat on, which is no longer redhead, you know your new. How does go under red? Does that mean you had a no longer head would definitely have interested in pursuing this. The amount of time I will be able to devote to. It is a question work, but I think there's absolutely desire to stand up to the ADM, install the AWS and ran kumar against it.

E

L

All right, hello, Rona I, have a question regarding q mas. This is shirley from redhead, so we have a laundry list of stuff that we have to change for rest of the cloud providers. You are talking about two functions: sham yeah.

J

I'm actually just sharing the link on the side, so so here you're, not you need to do boo thing is define these interface functions. So there is a skeleton utility finder you need to override and there are four functions in it to create the instant along with the persistent disk and supply gets a great Winston and the others to believe the instant. Apart from that, you need to define a function which SVP and one for epithet and the exact specifications I'm actually sharing the link to that file.

J

Now, and apart from this, the other thing you need to do is to be able to define your configuration file for it up device or greater area. So, basically, and not a very big, it should be really difficult because there is a conflict. 5 4, GC, orderly and you will be able to like one of us have equivalent of all.

J

L

So I started looking at the code last week, so there are like to change. We have to make I guess one is related to the ssh and ICP that you are talking about I mean. Is there any reason that it's very much specifically because there could be done yeah.

J

There's no specific relation to gk, so I mean not a good start of this Q month after he leaves for basic operations. You need a consultation, and you also need to be able to entice to that machine back people. So they have a wrong stuff sounds good for him and for pce when that.

B

Package, oh that no no sickies and torrential activity cloud compute FSH, which I've done all fixed before it allows vc ssa machinist. So this is the site had kind of a whopper page which of the reason why we like.

J

Okay, I just share a link to the file.

B

We are getting his out, so would be to disconnect I wanted to talk about the SLO that I sent the list, but I guess we'll good. In a week.

A

We'll do that had we had, we agreed that one of the low was what I wanted to put it. I can either leave the general update just saying: there's a new SLO, we're thinking about where I can actually start saying it. There's this one too. It's.

B

Not well defined, like I, think like if we are to be here that this is kind of it. The way we want to go then I guess you can announce. Oh I.

G

Think we didn't designing worker in your SOS and would welcome feedback from the community if they have particular user story, hello, I won't say we picked.

A

A specific ass LOL, I would say this- is the discussion we're having about the kitchen again.

L

A

L

A

Everyone thank.

J

A

C

C

That's because they doesn't mean I need to the meeting.