Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2018 (Copenhagen), 14 Nov 2015

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Keynote: Serverless, Not So FaaS - Kelsey Hightower, Kubernetes Community Member, Google

Description

Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18)

Keynote: Serverless, Not So FaaS - Kelsey Hightower, Kubernetes Community Member, Google

About Kelsey
Kelsey Hightower has worn every hat possible throughout his career in tech, and enjoys leadership roles focused on making things happen and shipping software. Kelsey is a strong open source advocate focused on building simple tools that make people smile. When he is not slinging Go code, you can catch him giving technical workshops covering everything from programming to system administration.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

Alright serverless: that's where I come in to your data center and still your servers.

A

Alright, so this talk is really about kind of my own personal kind of journey tours this service thing right, I'm coming from this world of kubernetes Coronas is everything run everything in it? And here comes this service thing and for me it's not so fast slow down. So how do I or how do we typically think about service? So when I talk to people, I said what is service?

A

It's like functions and apparently you must write them all in JavaScript for some reason, and the other thing is this events thing alright and this typically, if you listen to people, they will tell you something happens in the cloud and it calls their function. So we've taken this event-driven architecture with all this promise and we scoped it all the way down to this limited view of the world and for me, a person that wants to know how things work or see bigger potential and how things could work.

A

This is a very narrow view on some of these particular principles. Why are we constraining them so early, it's too early to lock them down and when I look at this functions represent the? How and then, when I look at the events, they represent the win and I'm asking people. Why are you doing serverless ever asked that question like? Why am I doing the surplice thing that get confused like there's no other option and the data?

A

The whole point of this is that you're doing something I hope that makes money, or you also want a lot of money and soon you'll have to pay them back, but if you're producing data, this is your data. You own this data and when this data changes, maybe you're in the manufacturing business, you're, shipping, real goods to people and something will change in your system and ideally one way. It's a program against that change is to emit an event, and then some logic can receive that event and do something how the whole world works right.

A

This is a thing we've been doing and how humans operate, but yet we want to constrain it to a platform or two. So there's a lot of potential here. If we think a little bit more broadly about what this is and people say, data is important. Why is data so important?

A

Well, data is what we collect in the store, and then it becomes information once we analyze it, but, more importantly, it becomes knowledge once we understand it and if you don't understand that piece, you're, just kind of toiling away, people that do really good with this stuff they get to this part. Okay. This is the point. Yes, this quote is by me yesterday, it's it sounded really good I'm, like that's going on the slide, then Twitter.

A

So when you talk to people about service, they always give you this use case. I, don't know how this became the canonical use case. You upload something to a fowl bucket. The goal here is I want to take a piece of text and translate it from one language to another, use some ml, API, okay and then once it hits the object store in the event will be emitted and it will call your function and then you will call another API and then you will return. This is beautiful. This is like the dream come true.

A

You can destroy all your code. You could do no code, but that's a little extreme. We can do this server this thing, so this is how it's sold. Anyone see the problem with this any issues with this, this whole cloud thing this whole ability to distribute your compute run anywhere that you want yet we're going to follow it all back into one system.

A

This is not necessary. So what's the problem here, let's throw a little wrench in this, so I work at Google cloud, I'm, not advertising! That's my arm! Prim, okay,.

A

Let's flip this a little bit we're going to translate a text file three from English to Danish, but I have to do it from my own premise: I'm in Google, Cloud and somehow I need the thing in s3 to call my function in GCP.

A

You all have the solution in your head. If you like that, you're screwed did.

A

So one thing we need is a common way to talk about these events that are emitted when your data changes so there's a project in the works to define a spec around this, and some of you is like. Oh, we don't need a spec for this, so it's not quite yet done it's in draft mode, and hopefully it will end up in the CNC F, and the goal here is to define a few things number one.

A

The producer owns the type of the event we're not going to try to standardize every event that can be emitted from every system. That is a fool's errand, but we want to do, though, as may be standardized the envelope and what we capture that event.

A

So a content type what's in the body and then we need to have some decision and one of those decisions so far is maybe we can use HTTP to transport this between different systems and the last thing, whether you like it or not, Jason could be used as a format that you could read these things and deserialize. There's one place, as this has worked. Think about networking you own. The data we encapsulate it in a well-known format and we can distribute it anywhere in the world.

A

You grab it pop the header get the data out of it and you can go on so we want to define this of the work of this project. Cloud events is an attempt to standardize at least the envelope and the transport of these events. So we want to democratize these events. They should be able to fluro through any system that you want to use. So then you can start to think about a producer using this envelope and either sending it directly to the consumer.

A

My cloud function, or maybe we can build a platform around that where think about. If you were building into some shipping and logistics instead of you calling, let's say the Fed Ex or shippers api's, they just call you and tell you exactly what's going on with the things that you care about, so this is kind of that dream of service and I've actually built some things in this kind of model, and it does feel nicer I'm, not calling 5,000 api's. They call me when there's something to do so.

A

I can see value in this, so what we want, though, is not just a broker. It comes up a lot. You should just use Kafka. You should just use a message queue. Why do those fall short? They have no semantics around how to route something to another end point somewhere else and remember we're server lists we're not just talking about servers, the physical machines, we're also talking about the code where you start up, bind to a port or initialize a connection or bind to a queue. That's all gone.

A

The only thing you have is the ability to receive some data or an event and process it and go away. So we cannot go on out and connect to a queue. We need something to reach back to us, so we need a new thing, so this event gateway lucky. For us, one of these exists.

A

It's open source and I. Don't know if you know anything about the servers back home folks, but they are actually making this service thing. Usable I made the mistake of thinking that it's easy to create a function in all cloud providers is actually really really painful that people are actually using terraform to deploy a function. I thought there was supposed to be no servers.

A

The config to deploy your function is bigger than the function something's, not right. So we have this any function, any event, any provider. So now, let's try to solve this problem objects door and when I started this project between s3 and Google cloud or Amazon and Google Cloud, my all we're all wall Gardens. It's all about locking us evil cloud providers lock in lock and lock in lock in. We actually don't spend time in a room doing this. Actually just so you guys know.

A

So what we want is the client to send something to s3 and it turns out. Amazon is actually pretty nice. They have a notification system. You can register any endpoint. You want to receive the event you do have to send it to a lambda function. This is the first piece I need because they are the originator of that event.

A

So I just need them to redirect it to where I say we have this new again event gateway I can tell them to push it there and once it lands in the event gateway, I can decide who gets the event. Next, it could be on pram. It could be my laptop, it could be a function. I'm gonna do blasphemy. Here it could be a container.

A

Yes, you can send HDPE packets to a container and it will respond. Don't tell anyone else and then I can do this Translate thing and then I can also do a pub sub model and give multiple functions. The exact same event and let them respond. I can do a canary roll out all the things we've learned. Well, when other models we can actually do. We have this broker in the middle. You have to do more intelligent rings, but I don't have that much time. I'm gonna get to the last piece.

A

I can't come to kukai without the demo. Now this is the silliest thing. I've ever done.

A

Everything that I'm using pretty much doesn't have a version number right now: okay, but we're gonna, try it anyway, so we're gonna switch in my laptop and the goal is this: is the data remember the task, take our input file and translate it into what language so I, don't know if I'm going to do this right. So if you do speak the language, let me know if I offend, you just talk to me later about it.

A

So this is the goal. So the nice thing is I. Have this a bit broker it's an open source project? Where do you run the event broker?

A

Anyone where, unfortunately you're going to need some servers again, but it's ok only for this part, so we need an event broker and we need it to be somewhere and there it is, and it's nicely done it actually scales really well and each of these event, brokers will be gateways, throw their configuration in at CD and then they can actually traffic to each function that you do so. The next thing we need to do is register our functions, so they have a nice API.

A

Actually so what we do is we say: hey, I, have a function and all we want is HTTP endpoint. So if you've never used a service platform before I'm going to show you really quickly so here's a service platform, Google, Cloud functions and I have this trigger. It notice really tiny to read, but that URL there is how if I send some packets to it. It's going to call my function on demand so anything that you have an HIV endpoint. You can register it as a function, so here it is. We want to add it.

A

So you will see me running some tools and I built this command line tool just for this demo, so you won't find it online. So if you, after this, you look for an event CTL, it doesn't exist. Okay, so we want to say, functions, create F and we'll give this function. So we create the function.

A

So nice thing about this: now you can do late binding, so the function is just registered and now we can give us a scription slightly like you would do with the message broker, but we just want to tell the a the event gateway what we care about. So we're just going to look at our subscription really quickly, and this is an s3, so it just says: there's going to be an event that shows up at the broker or the event gateway at this path notice.

A

The semantics here, all your routing layers, authentication layers, all those things work, so you don't have to go and redo everything you can kind of drop this in and keep moving on and we're just going to register this. So coop CTR, coop CTL just up there. So we'll do this description and we're going to create this description. So we're gonna. Do this binding! Now the nice thing about this, you can actually do updates to your functions because you have this indirection going on so now we have this description.

A

So now, let's get to the task at hand. We want to translate that file.

A

This other thing people don't talk about in serve this world is authentication how the hell is this function when it gets an event going to have permissions to actually read the file in s3 I've got the notification, there's no guarantee that I'm going to actually send the data with the event and most of it and driven architectures you're not going to put the data in the actual event. So how do you do this? It also turns out, says most cloud providers use open standards like old, El Cid.

A

You can actually have Amazon trust Google as the authentication provider, just like you do when you log in to log in with your gmail account or a log in with your I, won't say the other one, because we can't be like them, but, oh all right. So if we can have one cloud provider trust another, then we don't have to go and pass secrets around to every function that we want to access a resource. On the other side, we can map identity between multiple cloud providers, I learned that last night and it totally works.

A

So when this function gets this event, if everything works, it will actually be able to read the s3 bucket process. The data put it back all right, so we're at the final piece and I'm hoping this works and look at this. My marketing team is like what are you doing.

A

This is Amazon.

A

Let it bake all right, we can do it all right, so here I've said Amazon. If there's anything that shows up in that bucket I want you to send the event to that URL. So that is the IP address of the broker running inside of my cluster and that's the path for that event. When the event shows up.

A

Of course, the broker knows nothing about this Amazon specific event, but what it will do is it will wrap it in the cloud event structure and pass it along to my app and then I use the normal libraries that I would do if I were running in lambda in any other cloud provider to process the event straightforward. So let's do it upload here find the file input text hit open, upload.

A

Now service is supposed to be magic, so you're supposed to do nothing else. That's it. You upload something. A chain reaction goes off and it always works. That's what they say all right now, I, don't speak Danish, so you have to tell me if it's correct, let's see.