Cloud Foundry Cloud Foundry Summit EU 2019 - The Hague, 18 Sep 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: "Cube" your Enthusiasm: Bringing CF and Kubernetes Together with... Julz Skupnjak & Mario Nitchev

Description

"Cube" your Enthusiasm: Bringing Cloud Foundry and Kubernetes Together with Project Eirini - Julz Skupnjak, IBM & Mario Nitchev, SAP

Is container scheduling commoditised yet? The Eirini project is bringing pluggable container scheduling to the Cloud Foundry Application Runtime. In other words: the `cf push` developer experience we all love, with your choice of container orchestrator - including Kubernetes - under the covers. This session introduces the Eirini project, talks through the reasons behind it, updates the audience on the latest progress (passing CATs! available in beta! ready to try out!) and tells you what it means for your developers (not much) and your operators (potentially, quite a lot!). The speakers will also demo getting up and running with Eirini on Kubernetes using a single Helm chart. After this the audience will be able to try out the `cf push` developer experience, going from code to first-class Deployments, Pods and Services in your existing Kubernetes cluster, all in a single command!

For more info: https://www.cloudfoundry.org/

A

So welcome to de rainey update talk. My name is Jules or also known as her jewels and I'm here with my colleague, Mary Anitra from sa P, and, let's start okay. So we'll talk about free things. Today, we'll talk about the irony and white letters, we will talk about an irony which are the updates that we did since the last update talk at the last summit and we will talk about the arena verse, which is about making the whole platform more kubernetes native. So, let's start with the y Rini.

A

So that doesn't.

A

All right, so why rainy? Because humans are lazy, let's think a bit about this a little bit more like technology and machines that we use on daily basis are mostly invented by human laziness. So we used to wash our clothes by hand in the past, and then we invented this nice machine. If it pops up I would do it by hand.

A

Which is called the washing machine and all you need to do you need to put your clothes in there, you press a button and you get clean clothes. The same appears for dishes right. You have this, there's dishwashers where just put your dishes in you press a button. You get my dishes. We used to dug holes in the past some constructions and roadworks, and we have these big diggers that are helping us a lot by doing this work, and we are even too lazy to cook for ourselves.

A

So we have fast-food restaurants, which was not the best idea. Probably, but we love it right and we used to go by foot and at some point we rented cars. And now we at the point where we even go to self-driving cars, so we don't have to steer ourselves the cars anymore.

A

So what I want to say is that we abstract things away from us right. So it's really about abstraction and this laziness and this abstraction is not a bad thing. It is really worth it because it's really time-saving and if you can't save time, you obviously have more time to focus on other important things like Netflix in gaming, making music like I, do, if you mean electronic music by the way you should check out my soundcloud.

A

No serious, we can focus on more important things like like we human problems right. We can spend time on fixing this, and this is what we should do with software. We should focus on our product and solving problems and not on details.

A

So remember back like last year who attended the summits and the irony talks. We talked about this big elephant in the room, this big scary elephant in the room which was kubernetes.

A

This elephant is not scary anymore, but it's still a big elephant. It comes with a lot of context, comes with deployment, stateful said services, helm and all that stuff right.

A

So if we want to deploy our application on top of this platform kubernetes with all that context, so how do we do that? So first, you need to create your container, which contains the application. For that you need to write docker files. You need to know about the docker CLI, you need to know your runtime and dependencies and how to package them, and then you just got the container you're not done. You need to get this thing somehow on top of kubernetes.

A

So for that you need a registry. Where can push your image too, and so that kubernetes can pull it actually, and you need some deployment description to tell kubernetes say how many instances of the app it should run. Should it be a front-end service, a back in service? And, finally, you need to know about the tooling to interact with kubernetes right. So now think about all the steps you will need to do over and over again every time you push an app, and this is not what we want now.

A

There was a tweet from Casey Hightower early this year and he says that kubernetes is for people building platforms and, if you're developer, building your own platform like cloud foundry, then kubernetes is for you. So what we want to do again is we don't want to interact with kubernetes directly because it's too complex we want to abstract away from it. You want this abstraction. Read developers want to push their apps easily and operators with kubernetes gets their benefit so boundary.

A

Is this abstraction on top of kubernetes right and it reenie provides this abstraction, and that's why it's important and with other projects like the sea of quarks projects, you can even get the hotel foundry control plane on the same cluster, which is really great.

A

So till the last update to give you a heads up, we were already able to CF push our application to kubernetes. We were able to CF scale the application, also to get the locks for your application right and to restart an app or just an instance if you like, and finally also to get the route for the application actually displayed on the browser or just curl it. So this is all really basic functionality of Cloud Foundry, but this is not what pounder is all about. It's not just about pushing an app to the cloud.

A

It's way more, that's why we didn't stop to work on irony. We continued and Mary will give you the updates.

B

Right so, during the last few months, leading to the summit's we've been making a lot of changes to every knee to make it more ready for an alpha release and I'm. Not gonna, give you a whole big, boring list of things, but we just picked out a few that we thought were important, starting with security. I think we can all agree that security is hard and as a platform that makes developers and operators lives, more easy, I think Cloud Foundry should make security easier. So what have we done on a Rini to make that happen?

B

Well, first, we secured the connections that I mean he makes and it's apps. First we use TLS for a ringing and for the know we actually require Club controller to have a set of client certificates, so use MPLS to actually talk to a weenie.

B

Next, if you actually remember the last last product update, we had a little demo, which was basically got the applications, docker URL image, URL and just using the terminal on my machine. We were able to pull it, but if you think about that, that's terrible because that's where the application user applications images live, and you don't want just anybody pulling them.

B

So what we there there is, of course, we use yeah Wes for the docker registry, so it's secure over the wire and we actually made the container runtime on kubernetes authenticate the bit service using a user and password just actually revisit this demo. So what we did was I only have an app it's a door and what we did was I got the image just by describing the pot I'm gonna grab for it and, if actually copy this again and do a dr.

B

pol doctor is going to tell me that I don't have any credentials for that. So what we can do now is actually we have a doctor, login I'm, just going to enter the password and now, if we try that again, we can see that it works eventually.

B

So, okay, we have a secure, ish registry.

B

We also lock down all the connections that the application is made so before you can actually reach other apps and even other parts using the part of the network that kubernetes uses, which is not something that we want. So we lock down everything. No application can reach another application using just the kubernetes network and they can't reach the kubernetes master either and for that matter anything else deployed on that cluster, including the CF control plane, except, of course, for things like go, router and Doppler, so actual routing and logging works.

B

Of course they have access to the internet. So if your application happens to be exposed to the Internet one, other application can of course reach your application. We also secured staging and let's actually see why staging was insecure in the first place. So what did we do? We first downloaded the application bits from cloud controller using Atiawa certificate mounted on the continue. Next, we used those bits to actually build the droplets, and this is running the user code.

B

Next we used or just those droplets and we use another TLS certificate uploaded back to the blobster. So why is this insecure? Well, you have the build step mum on the same container as the other two steps and those other two steps require some secrets. So it's way better if we just separate everything out into separate containers, so each container is one single step which only uses the things that it needs to do its job properly.

B

So I think we all agree that security is hard and again it we all committed a password or two to github before, but it's sometimes not your fault. It's sometimes there's a security vulnerability and one of the packages, one of the dependencies or even the OS itself, and at that point you have nothing else. You can do it's a update and that's why really fast patches are so important. So how do we do them in ireenie?

B

Well, if you're an operator- and you already have your installation but I've been in CF, all you need to do is edit your values file for for helm and do a helm upgrade. What that will do is change the annotations bit service and and I vini, which will cause them to start so after I.

B

Really answer started on new applications will be annotated with the new route fast version and when bit services were started, it would download the new route fess and serve any new images or old images, also using that, but that's for new apps, but what about the old apps that are already deployed? Well for them? We have a real, fast, batcher job, which basically does the same thing.

B

It updates the annotation on all existing apps, the new fast version causing them to restart and as they were starting, they will pull down the new image from bit service, which only has the changes in the new route bus. So, let's actually run through this together. So I already have my deployment again and I'm going to edit the values file for it, and here we can see the route of reservation. It's version 81. Now we've decided that version.

B

81 has some security vulnerability and we want to change to version 84, okay before I actually do the helm upgrade. Let's see the cube the annotation on that pod, just to make sure it's 81, it is, and so we have something to compare I'm going to SSH to it, and I'm gonna do a shot, 56 song on this specific file. Now this is the file that actually runs the droplet and I know between these versions. It won't change sha. So for this, this is our shot. I'm gonna copy it.

B

Okay, I'm gonna exit the container, and so gonna echo it just so we have it okay, so we entered in our values file. Let's actually do the helm, upgrade and notice that we're using the same values file. So when, when helm does its thing eventually, we're gonna see that bit service and I mean er starting.

B

It's it's gonna take a while, because bit service needs to download the route, then you reverse, and that can be up to a gigabytes, but we can also notice, while waiting, that we also have a pod named Rufus batcher version 8400, which is the thing that's actually going to update the application when bit service is done. So after a few seconds or more than a few seconds, we're gonna see bit service restart.

B

You're on the first batch professors are big.

B

And even though there we go okay, so now it's running, we can see that the root of a spatula is completed and an application is restarted. So now it also needs to pour down. Then you would first, but while it's doing that, we can cube still describe again for root of us and see that we have the new version 80 for now. Let's just start it, let's go back and do a shower 256, some on lifecycle launch, and this now has a different sha and if you don't believe me, they have it in history.

B

So that's how we're the first batch in works so.

B

Yes, that was when you're done so, of course, security is important, but what else should a platform like Cloud Foundry do one? Second, it should also scale so we've done some initial scalability tests for ireenie and I'm gonna share with you some results. So our first scenario was shown of small. It was a hill deployed ireenie with cloud foundry. What a cluster of about ten nodes and we can push about 200 apps, and then we can. We saw a few issues which we'll get into later now.

B

These are apps that are pushed in batches of sizes, about 10 to 50. So parallel pushed next, we had a slightly bigger deployment. It was a Bosch deployed laughs only this time around what about deployed they're really using the community both for these for every this cluster had about 21 nodes and it can reach about 1,000 apps before seeing the same problems, and our final scenario was done by the member of the community.

B

It was again a cluster scenario deployed with helm with absolutely massive cluster of 250 nodes and naked push, 2,500, absent parallel and then up to about 3000 pushed one by one. So what issues that we see? Well, first, we saw issues with whelming, so when your application workloads become greater routing had became a lot slower, so any new routes got delayed.

B

Basically, they got delayed when emitting the routes, so you can reach them with this. We have also very small but very annoying bug, which gave a 502 bad gateway. Whenever you do, you register routes, because there is a small gap between actually we're starting, the or killing the application instance and the registering routes- and these are things that we fixed for now. We also fix problems metrics, which were actually related to the kubernetes metric server, which needs to be scaled up, as your application workloads become further and a similar problem.

B

We found with logging, so you also need to scale dr. as well. Of course, this is not everything, it's we're still working on it and we still see problems with staging upgrading decrees, master and worker nodes and also bit service being, and there are a lot of a lot of pressure doing these larger workloads. There is, like general unavailability about it. So, of course, this is not everything we've done. You can see the large list of things like CPU quotas memory in this limits. If your app goes over limit, it gets killed.

B

You have custom built back URLs and all these great features that we've done out of the over the past few months. So that's what we did, but what's actually next for everything all right.

A

Let's talk about the universe, so we have kubernetes and we know it's a great scheduler and we also have cloud foundry, which is the great platform that this awesome user experience right and we need brought like Cloud Foundry as a platform on top of kubernetes, and now we are working on making the whole platform more native, more kubernetes native. So let's take a look at a really verse, so we got more projects looking into making. The hope that were native so copy is working on customer resource definitions to make the API no native.

A

We have the networking team working on a steno integration to make the networking more common. It is native. And finally, there is the irony, a project that is creating a framework to build extensions around Rini to make basically everything more native, so that the whole platform is becoming more native.

A

Next is, which is a big part for the future, for the next milestone is to have the migration from Diego to Irina working seriously.

A

There are a lot of discussions going on, so this will be a big part of the future and another big partner we will work on is so probably most of you know that arena today comes with ham charts that you can deploy together with the SCF on top of equipment kubernetes. But now we also want to support Bosch for the bot environments, of course, so to integrate there smoothly, but also because of the evolution of the foundry and with the crog's project and the CF operator, so that you can easily deploy with it.

A

And if you want to try, you really go to our github repository, which is Cloud. Foundry incubator, slash arena release. We started to release more frequently like every week, usually Mondays and yeah. Try it out, deploy it on your kubernetes cluster, and if you have problems any questions or you just want to complain about a rainy or how good it is.

A

We give office hours like every first Monday in a month. You know zoom call and yeah feel free to show up and ask questions chat with us. The zoom call you can get it. If you go to the Cloud Foundry selection, we always post a select zoom link into the channel before the call actually starts so feel free to join. And, finally, it's not just us working on arena. Of course there are many more of us. So here are the names and if you are up for any questions, we are of course around.

A

We are here, come to us ask whatever you want to ask and thank you for showing up.