Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2018 (Copenhagen), 14 Nov 2015

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Securing Serverless Functions via Kubernetes Objects - Sebastien Goasguen, Bitnami

Description

Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18).

Securing Serverless Functions via Kubernetes Objects - Sebastien Goasguen, Bitnami (Advanced Skill Level)

Serverless is fast becoming a new application architecture paradigm. As glue code that links cloud services together it is tempting to forget about the security of functions being deployed. In this talk we will discuss the very important security considerations that we still need to keep in mind with serverless functions. Using kubeless, the kubernetes native serverless solution and focusing on AWS EKS deployment we will discuss how Pod identity, Pod security context, Network Policies, AWS IAM profiles and API gateways can all be used in concert to create a secure serverless application.
Demonstrations will highlight how Kubernetes brings critical value-add to a serverless solution and will re-inforce the need for a strong security mindset for cloud-native applications.

About Sebastien Goasguen
Seb has been working on Kubernetes since June 2014, he is the author/co-author of the O'Reilly Docker cookbook and the Kubernetes cookbook. With many years of experience teaching computer science he stills enjoys teaching people Kubernetes and getting them off the ground and on their container journey. Currently he works at Bitnami where he oversees all things Kubernetes, he prototyped Kubeless, the Kubernetes native serverless framework and other community tools like kompose and Cabin. In 2008 he was selected for the first round of Astronaut selection by ESA but sadly is still stuck on earth.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

Join us for KubeCon + CloudNativeCon in San Diego November 18 - 21. Learn more at https://bit.ly/2XTN3ho. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

Okay, good afternoon everybody a couple more hours of Cuba in Europe, so thanks for sticking till till the end, usually you know the sum of the last talks in the afternoon of a conference are always you know less attended. So thanks a lot for for staying I think we have some exciting content. Austin did an amazing job earlier this morning the demo was really really cool, so my name is sebastian quest again.

A

I'm the kubernetes lead that at bit, nummy I've been in Cuba latest since 2014-15 did things like Campos cabin, the web app came machine work a lot on I worked with the Linux Foundation on certification course O'reilly and so on at admit, nummy. You know we. We work a lot on packaging apps, so you may know us from the marketplace on Azure or AWS Google. We see you, you see a lot of applications that are packaged by us, of course with generate containers, and you know when community study we got very involved with the charts.

A

So we also do build charts and then participate in the community upstream to build those those applications, so we're all about apps, apps apps for any platform and, of course, when we start thinking server less, it makes a lot of sense for us to get involved. So today, I'm going to talk to you about securing server less functions. So it's a little.

A

You know a little bit of an advanced topic, but I didn't want to cover the basics again, so I really wanted to dig a little bit deeper into some of the things that communities gives us out of the box and those things help us secure those those functions so I'm going to go through some of those topics and then joining me will be UNS. Angle. Key is right. Here is waiting for his container to download.

A

So, let's cross fingers, Uranus works at s, AP hybrids, which is the e-commerce platform for 4s AP and as a contributor to 2q blast the project that I'm going to to mention and this morning, if you, if you followed the demo from from Austin, you saw that there was one of the functions. Was a cublas function running in the ASAP cloud platform, which was calling the ASAP machine learning system? Okay, so we work very closely with with s AP and they contribute to to cublas.

A

So I thought it would be nice to have you NS come here and then explain to us how they actually use East Joe. So it's not an easy o talk. It's a talk on security, but happens that the the use East Joe okay. So what type of applications are we are we talking about here and yeah? The slides are pretty simple: I I didn't concentrate on making very fancy slides, you know I. Can you can you can tell it's marked down so what type of apps are we? You know we're thinking about here.

A

So of course, micro services, containers underneath running in running in Cuba natives, but then we're going to have events, and that's where you know it resonates with what the the working group is working on. It resonates with the cloud events. Specifications because you know I believe that future apps will be made of you know small glue, the functions that will be triggered by events and when we mean events, you know it could be, of course, events in something like Kafka or Nats, or you know one of the more regular Enterprise pub/sub but also cloud events.

A

So events coming from cloud provider services, so machine learning, storage, streaming, services, databases, you name it. So all these cloud services are going to emit events and when we're going to start writing applications that those applications are going to be a set of functions triggered by those events, not really new. You know we've done messaging based applications for for some time, but I think that now we we're going to have more and more events and the ease of deploying functions is going to kind of rejuvenate that type of application paradigm.

A

So the whole idea is to compose many different services and trigger them on events could be in the cloud could be on Prem and, of course, here I'm saying aw as lambda clone.

A

That's you know, meaning that when we look at some of the solutions that Austin mentioned this morning, they are yeah they're clones quote of lambda, okay, so systems that allow you to deploy functions- and this is not a talk about Cuba as I'm, a very bad sales guy, but I'm going to use this system to illustrate some of the security features that you get out of the box with kubernetes to help you secure functions. Okay, so you know where I'm going with this is that apps are going to be.

A

You know made of functions triggered by events, those functions they are deployed in your cumulative cluster and, of course, underneath there they're managed just like regular services. So the rest you'll see some specifics about about cublas, but they are really here to illustrate. You know what you need to watch out for so digging into cuba's now it's really a cubist extension, and we see more and more of this.

A

You know at this conference and even in the community, there is more emphasis on using custom resource definitions to extend the community system, and the idea is that queue. Bilities is really becoming a platform on which to build additional systems. So when we think about you know how do you deploy your functions? How are you going to deploy that glue? Well, that glue is going to be deployed on Kuban. It is an extension of the community system, so it's all based on CRD.

A

You have a controller and in the end you deploy, you know you have deployments and services and so on. Cublas also has server. Let's plug in that's a shout-out to Austin this morning. I think we can do better than his event gateway, but don't don't tell him ok, but he has yes a great. He has a great system so the two starting into a security aspect.

A

You know if we look at the original architecture of cublas, which was really a meant as a puck at the beginning was quite simple: let's extend the community's API with a CR D and that CR D is going to. Basically, you know, allow us to have a function, object, kind function and then, when you create a function, so you have a manifest for a function. You have your cublas controller. So your controller sees that object and then creates a deployment that deployment then launches a pod, and in that pod you inject a function.

A

Ok, that's actually quite easy, going to say easy! That's actually quite easy to do is just a single controller, er D definition, and if you, if you stay at this in terms of server less basically, you know 2018 CGI, that's a relatively simple system. Now, I really think that the trick comes when you have to trigger those functions. When you have events, okay, so that's going to be really the the value-add of of those systems, so we we spend quite a bit of time recently trying to I mean not trying we achieved it.

A

We solve that problem. We cracked it. We react technique, we architected cublas and I'm not going to show it to you like a lot of details, but now we actually have event triggers that are defined as new CRD objects. So, if you want to trigger on an event in Nats, we have we define an adze trigger. If you want to trigger on an event in Kafka, you have a casket trigger. If you want to trigger on a sqs message, you have an sqs trigger object. Yeah you get, you get the idea, you get the drift.

A

So now all your cloud services out there kinases, you know Google storage and so on, you're going to be able to define a mapping between an event and a function using a cumulative CRD. So I think this is really really interesting and it opens you know, a ton of a ton of doors. So that's the overall architecture. But what up you know? What about the the security features and we have people, you know they're, saying hey: we want to use Cubase but we're on openshift. We have big security, you know requirements and so on.

A

So we need to really think about the security of that system. So I I, you know at a high level, I said two types of security, internal security and then external, so internal I mean authentication authorization who can create functions and manage them, delete them, modify them and so on.

A

How can you limit what a potentially compromised function can do and then things like auditing and logging and so on? I'm? Not going to talk about auditing and logging because you get that out of the box with kubernetes is just a set up and then external security, so you've deployed your function. You have your authentication, Arabic and so on external security I mean who can authenticate. Who can talk to that function? Endpoint, okay. So how are you going to enable this?

A

Usually it's some type of anticipation at the ingress layer, okay and then the I think that the next problem is really. What can the function do in terms of talking to a cloud service and to be able to talk to a cloud service? You need to have an identity. You need to have an identity and authentication profile, a yam profile. Something like this. So when you start thinking about this now, your function needs to leverage pod identity, functionalities. Okay, there was lots of bullets.

A

If you prefer a picture, it looks like this: okay, external security who can call it the ingress X, and then you know, what's the identity of the function, to talk to other services and name need to be able to consume those those events, for example, and then internal I'm, going to really talk about this authentication solved out of the box, because your system, if you base it on Kuban it is a function, is now a custom object. So you have authentication out-of-the-box. You use the authentication mechanism of the Kuban if it's a PI server.

A

Okay, so here you know, you see the example of the the function object right now: API version cublas, iuv, 1, beta 1, the kind is function and then, of course you have the metadata. You have everything that comes in communities, so authentication solved and I. Think that's really the way you should design that system. If you have to build your own function, API server, I think it's! You know, it's not the right way to do it. So here we leverage the authentication system of kubernetes and, of course, out of the box.

A

You also get our back. Ok, it's just another object. So if you, if you create a communities cluster with our back enabled- and then you see here at the top- is a cube CTL get functions in a specific, with a specific contacts which is just a different user. That's not allowed to do too anything you see, it says: hey cannot list functions so typically, that's because the our back row, the user making the call, doesn't have the our back role to be able to do the function. Then you see that I create a role.

A

I create a role binding and now I do a get function same call and now it says no resources found. So you know nothing magic here, but that's out of the box because you know cube less or any other system actually like nuclear or I, haven't, checked, FN or or with things like this by basing their function system on top of queue bilities they get all of that out of the box. And then, if we start thinking about the the isolation of those functions, then it's great because you know it's a Cuban.

A

It is native object, so you have namespace isolation, so get function in the default. Namespace get function on all namespaces. Suddenly you see, another function show up okay, so you get namespace oscillation and suddenly. It starts becoming super interesting to have namespaces relation and also to have support for labels. Okay- and this is just standard communities I'm guessing some of you know this, but you see at the top I'm creating a function.

A

So that's the typical cube, less realized, Cube last function deployed and at the CLI I can define some labels, so you could write a function as a function manifest, but you can also use this here I and then we have this. You know I call that convenience to be able to define the labels. So now you have a function. Object, that's going to be created, that's going to have labels, we propagate those labels to the pod, that's going to serve that function, and what does that mean?

A

It means that suddenly you're going to be able to use network policies to isolate your functions between well each other or other things. Okay, so the fact that you're Cuban, it is native authentication, are back namespace, isolation, labeled support for label, which then opens the door to to network policy. Here you see a network policies that says hey all the parts that have the tag label fubar.

A

You know you allow ingress from pods that have access equal, true, okay, so you can stop incoming traffic to your to your function and then you know it's just an example: I'm going to spare you the demo, because it just works: okay, honestly, okay, so authentication our authorization, isolation thanks to network policies out of the box. Now we start you know wanting to isolate. You know what a potentially compromised function could do on the on the underlying system and at the end of the day, it's a pod.

A

So it's just pot security out of the box again, so you know, if you look at the pod, that's that's running your function. There is a security context. Okay, by default, we we actually backed it into into into cublas, and unrest did this so we said you know: FS group run as user 1000, so that out-of-the-box without changing anything, it's going to run on open shift, because open shift has a little bit tighter security constraint out of the box. Okay. So by default you get you get this.

A

You get some security context, but then you also have the container. That's the runtime. Okay, the function runtime so that function runtime has been built. Also, and that's just an example for the from the the go runtime. We just announced golang support user 1,000. So that's really the the basis. So the the server that's exposing your function doesn't run as root.

A

That's really the the minimum thing that that you need to you need to do: okay, I'm, not saying that that's the best security, but at a minimum you, you know you need to do this, so here your honor you're now trying to limit what a potential compromised function could do and then, of course you know if somebody managed to get out of that of that container and reach your host. You know you would like to be able to to limit the blast radius.

A

So that's where things like Kotak containers come in or you know, I haven't had time to to play with it, but the newly announced G visor from Google. If you're running kubernetes on hosts that have G visors installed, then suddenly you know each of your pods are going to be really well isolated from from each other. So it's just it's just beautiful, okay and I. Think that's it's a general tenancy with.

A

Well, it's not even a tenancy. So it's a general thing with technologies that we see convergence of things. Okay, everything I'm talking about here is just baked into Cuba niches and then the the availability of you know things like like G Weiser, okay, but it tells you why you end up with extremely secure functions now by default pods when they start they're going to have a default service account- and you know some of you may remember at the beginning in Cuba noticed there was no Arabic.

A

So when you were running with a default service account, you were route from the capabilities, API server, that was, that was great. Okay. Now it's much more, you know much more restricted, so the default service account, you know, cannot do much and the other day I was writing a function. That was calling the API server to get secrets and I was like. Oh, can it get secret ah nice nice security feature out of the box? Okay I was I was actually restricted to do.

A

You know things on the API server, so that means that what you would like to do is be able to run functions, maybe with a specific service, account typical, typical thing in Ncube Anytus. So this week we did a demo with the folks at sis deck. We did a nice integration with their system. Falco Falco is the open source security.

A

It's not a security scanning system, but basically you can define a set of security rules and it's going to monitor all the system calls that are happening in your in your hosts and if it detects an event that you know matches a specific role, then you know it's going to trigger an action.

A

So the integration that we did is that we are running Falco on a gke cluster and then we have some rules to detect Bitcoin miners and when the Bitcoin miner is detected, it sends an image to nuts and then the Nats event triggers the queue blast function that kills the pod okay. So we have an automated Bitcoin miner killer, but to be able to do this, we needed to be able to kill pods and a default service.

A

Account cannot kill pods, okay, so we ended up creating a service account and then using that service account for the the underlying function. So how can you do this and kind of customize? This cublas has a config map. That's its cluster wide right now. Ideally, we would try to build that as a multi-tenant kind of configuration so that every user may have a custom config for their underlying function. Right now, it's cluster wide, so you edit that configuration and then you can actually configure the deployment manifest. That's going to be used for the function.

A

So if you want a function to use service, icon Falco, you just specify that in the config map and every function, that's going to start is now going to use that service account so you've, just locked it down even further.

A

Does that does that make sense so far? Is it interesting or boring? No interesting? So that's internal security. I! Think it's a you know it's mostly out of the box cumulative stuff that you know you need to be aware of, and then turn on and understand how to configure. So now, external security. That's you know it's a little bit more interesting from the actual faz solution.

A

You know that that's not a Cuba native feature per se, but when you need to think about it from the the QoS DF and you know all those all those solutions, so mostly it's an authentication issue who can call your function? Endpoint, okay and then what can your function? Do? Okay, so in cublas you can do cublas trigger HTTP create okay, so you can create HTTP triggers how am I going to call my function and that's really the basic. You know if it's exposed by the HTTP, you want to call it over HTTP.

A

You want to expose that function outside of your cluster, so with cublas trigger create you, we generate cube, let's generate an ingress object. Okay, so you have. You know a set of options there to turn on TLS, specify the you know, secrets and and so on.

A

Okay, so more or less in the Cubist CLI, you have a convenience wrapper to create an ingress object and, of course that means that now, if you're using an engine, X ingress controller create a password HTTP password, you create a secret to stick that HTTP password and now, when you define your trigger cube, let's trigger create you specify basic auth secret, you make a reference to accumulative secret. That's the beautiful thing about the system.

A

Ok, everything falls in place, so you reference your community secret and now you actually have a basic auth endpoint to your function from outside your cluster. Okay.

A

This is just to property of the ingress controller engine, X ingress yeah, and by the way you see, if you do a cube control get CRD in a cluster where you have installed, the cublas you'll see that there is a CRD for HTTP triggers.

A

So you know all our triggers are defined as your the objects. That's what I was mentioning. Of course nginx. You know it's one thing, and that just gives you basic auth. Maybe you want something a little bit more interesting than basic auth and we just added you know some support for for Kong.

A

So Kong is basically an api api gateway, but you can program it very nicely and you can get JWT token to access an HTTP endpoint. You can get. You know, api keys and and so on. So if you use the the newly announced kong ingress controller, suddenly it opens the door to you know a lot of ways to access your your function, endpoints, Joe, Joe, Beda the other day- was asking on Twitter what to talk about as it is Friday thank God. It's Cuban. It is Gigi.

A

Okay and I suggested that it talks about security and a security configuration and so on, because I want to 20k to my create this cluster with my github account but anyway, and they have gangway I've had time to play with it. But it's a utility to be able to authenticate to create this with different system and I think they have a plug-in with off zeros off zero.

A

That's very interesting because I think if they were to put gangway in their ingress, we could secure an HTTP endpoint. You know cube less using off 0 and the server less community, the users of server less. They are very, very keen on off to zero as a as a way to manage security to their to their api's. So it would be a perfect combination to get you know in the zero authentication set up there, yeah and and yesterday I was finishing.

A

The slides and I realized that in the abstract, I said, I was going to use eks, but I I didn't get access to it. Yes, so yeah there is no gas content in there. I apologize.

A

So now, let's move on to external service certification. That's that's where it gets interesting, because those functions that are glue between cloud services, those functions. They need to be able to do things with the cloud. So typically that means that they need to have access to some type of service account to talk to cloud storage or talk to kinases. You know cloud ml, you know you name it. So how do you give pod function, an identity so that it can?

A

It can do things if you are on on AWS, you can use a system called cube. I am cube, I am ok, and with this you can annotate a part and that part will get access to the role specified in the annotation. Ok, so that's that's great. That's very helpful.

A

In cue bless. It gives us a little bit of a problem because now we need to add some convenience in the in the CLI to be able to provide an annotation and then different functions may have different annotation.

A

So we need to think about how to enable that you know nicely, but it's totally doable and on GCP on GCP, it's a little bit easier because you can just reference a service account with an environmental variable so create a service account in your Google Cloud console download that service account in you know JSON file, and then you need to just make it available as Google application, credential, okay and then, and then you get it and in in the cublas CLI.

A

You see that you can specify an environmental variable and you can also specify a secret and then will automatically you know mount everything so that the function has accessed that that's secret and it's defined as an elemental variable. So you do this, and now you can. You can talk to two things I'm going to. Let did you the Joe container, download, yeah, okay, cool, so I. Don't need to keep talking.

A

Okay, so I'm not going to do this demo, but that's just to give you an idea where we're going with with cublas specifically to talk to cloud services. That's just a prototype there of a mini Python controller that monitors ingress objects and when it sees an ingress objects, it makes a call to the AWS, API gateway and registers the route in an API gateway. Okay, why? Why did I do this is because you know we package this as a little pod running in the cluster and it monitors.

A

You know what you're doing with your your cublas system and then suddenly you get the API gateway the program automatically and if you look at the call I'm making to the API gate where it says authorization type, none if I want I can just say authorization type, AWS, yam or kognito, and that now I fall back in the AWS land and can actually give an identity to or secure that function. End point so we have, you know, lots and lots of of potential here. So with that I'd like to to.

A

Let us show us how is actually leveraging another system that converges with communities which is, of course East Joe that helps us secure, even more function, environments.

B

B

Can you hear me now awesome? Well, that's the wrong screen. So let's try to move all the stuff over here we go the first one oops, sorry, there's something you should not! Oh.

C

Wait, a second is to use an open platform to connect, manage and secure micro-services sq+ is based on kubernetes deployment. This view can be applied easily to cublas. During this demo. I would like to show you how you can easily expose secure q+ functions using JWT tool, as well as arabic, rules to secure the communication between services deployed to the SQ service measure.

C

First of all, the demo architecture it's based on at CDU. We have an product service and the Commons functions of a product function at the Commons function, and all this is display y and web UI to make it a bit more interesting, I build a small beer shop. mmm All this is a exposed wire and is to you ingress controller. While the history of ingress controller, it is working like every normal ingress. You know about were the addition that we have to have an annotation to use the sto ingress class.

C

The functions are both pretty easy, so we have a product service or the products. I was able to get all the products out of the H CD as well, or if we provide a path, that's able to get one. The comments function is looking similar, so I won't show it again.

C

In issue ia enable neutral TLS by an ailing mute LTL, as you make sure, the connection between the invoice inside the service measure so between between the sidecars is encrypted, as well as server and client are authenticated. Using the spiffy protocol. I won't go into any details here, as it is known, not an SEO talk, but with this system your you can make sure there is no man in the middle between services talking to each other.

C

First thing I like to do is, as the beer shop is already exposed, I like to disable the comments, frank functions from people who are not authenticated.

C

So if you haven't looked on to the details page, you see comments to lis to the selected beer and.

C

To apply and JW to add JWT authentication to to a service, you simply have to write an authentication policy policy in this case very restricting the comment servers and allowing only connections from people authenticated by the by this google service account. So let's apply this.

C

C

Will already get in JW teto?

C

So if we refresh the page, there are no no comments shown. So, let's quickly log in with our JW teto and now the comments are shown again because the authentication was successful.

C

In the next step, I would like to restrict the access to particular services using our make rules, so the Arabic rules are done in the Geo mixer. So we are sending the request context of each incoming request to the service to the mixer, where the artistry Arabic engine is evaluating the the Arabic policy and then allows or denies daily the access.

C

That's set to enable Arabic, we have to first define the request context. This is pretty much the configuration from the ISTE or documentation, so I won't go into any detail and the second step we have to define an Arabic handler and or configure the Arabic handler. And finally, we have to map that every breakfast inside the default namespace will be redirected to the Arabic handler. Using the request context we defined earlier.

C

So, and if we applied this, the entire all the connections are denied because we have not defined any any deferral to allow access to the web UI and the product service to do the so we have to add a service role which is allowing all methods so can't post, update, lead and so on on the product, pur shop and comment service for demo purpose I have excluded the HCD cluster we will enable it later and what we have to do, for we have to map the service role binding to the east geo system namespace.

C

This is the place where the issue ingress is located and for internal connections on to the default namespace so quickly apply this.

C

If we refresh the page now, you can see the page is loaded. Fine, the only thing which is not loaded is the the pro are the products and the reason is the service server got in 500 error because he is not allowed to connect to the HDD database. So what we have to do now. We have to enable connections to the etcd database and after a few seconds it should work.

C

Here we go. Last but not least, the comments are still not shown because we still have to add the JWT toll. The our big rule is just validating that con that the connections are from the east, you ingress and not from somewhere else.

C

That said, we ran at the end of the short demo. I. Think issue is super good, fit to cublas, as it is enabling a lot of security options which are not there by default. There are still some some limitations so to enable mutual TLS you we we have to employ. You have to implement a short workaround to disable the liveness probe. The reason is is to you is not allowing any liveness probes from cuban eaters.

C

This will be fixed soon, either on qibla side as well as on this. Your side and the cublas CLI is not fully functional with mutual TLS. As the cublas CLI is using the kubernetes proxy ur to connect to the to the function, which is not part of the sto mesh, but I guess we will have a solution there soon.

C

D

A

Thank you thanks us, that's great. He had to stream down a little bit the demo and then the the mutual TLS was a little bit complicated. That's because we need to exclude the else checks, ports and the metrics / rap /, matrix route from the East Joe set up very.

D

Mutual TLS is enabled here: Anita's itself cannot access any ports and as well as you cannot get through the proxy endpoint API servers providing to you to the to the services so and the solution will be to take yeah have two ports at the end, great.

A

Thanks, so that's coming in that's coming in cue, bless and I will give us mutual TLS via East Joe, which means that now we have the full power of service mesh and we can deploy functions within a service mesh. Those functions will be triggered on events and will be able to to call other cloud services. So the conclusion you know queue bilities provides you with all the API objects to secure functions. A lot of this is out of the box. You just need to know it's there and then configure it properly.

A

Keep an eye on a so keep an eye on cube. Let's look at the way that we're handling events with new triggers with announced NAT support, which is really exciting, so you're going to see Kenny's is super support soon and that's even more exciting, and then you know Flo us on Twitter and thank you very much.