Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2017 (Austin), 15 Dec 2017

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Using Custom Resources to Provide Cloud Native API Management - Frank B Greco Jr

Description

Using Custom Resources to Provide Cloud Native API Management - Frank B Greco Jr, Northwestern Mutual

API management is an essential component for all production services. Northwestern Mutual uses it to secure 100s of microservices deployed to our Kubernetes clusters every day! Learning from our API management journey over the past few years, we found many ways to innovate in this space. Using Custom Resource Definitions as a catalyst, we created an open source project called Kanali, a Kubernetes native API management solution. In this talk, we will take you through our API management journey that led up to Kanali and then discuss how to use Kanali to secure your Kubernetes workloads. We will also look at how Kanali integrates with open source developer tooling such as Opentracing, Jaeger, and Grafana.

About Frank Basil Greco
Hi I’m Frank! I’m an extremely passionate tech engineer, developer, and architect from Milwaukee. My current passions lie in highly available and scalable infrastructure, containerization, serverless architecture, automation, artificial intelligence, web development, API management and algorithm theory. I am also very passionate about open source software and contribute regularly. I love working in a fun, collaborative and startup-like atmosphere with passionate individuals. For fun you’ll usually find me developing new custom skills for Alexa or integrating the latest IoT with my home.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.

A

Thank you guys for coming. My name is Frank. Greco I am a cloud native engineer at Northwestern Mutual in Milwaukee Wisconsin. If you haven't done so, you can check out our booth, we're not a vendor, but we are showing off how we do kubernetes at scale talk about Canali and maybe attract some talent along the way.

A

So Northwestern Mutual built a nice tower opened earlier this year.

A

All right, I do I, do 27th floor, it's 32, so ITV gets the top floors. So I'm here to talk about Canali, which is an open source, API gateway, kubernetes native cloud native. The plan is to blow through a bunch of slides, do a demo and then leave plenty of time for questions, because I'm sure you guys will have plenty, and so the first thing I want to talk about is API. Is why API is why an api gateway, etc?

A

Brush up on some of the terms if you haven't dealt with them a while or introduced them? If you haven't so we'll start with a simple example, so let's say if a Google home and you want to talk to a nest, what you'll do is you'll actually go through Google home will call the nest services and then the nest services will talk to the nest for you.

A

But what's going to happen, is it's going to go through a little piece of software called an API gateway and that's going to make sure that you're not maybe calling the API make sure you know it rate limits. It maybe does some transformation. You get all these things that come with what's called API management, so things like API, API, key authorization and a JD Beatty authorization, you know rate, limiting transformation, monitoring, routing tracing, etc. And it's this piece of software.

A

It's one piece of software: it's a gateway and it's the entry point to all of your API s, and then traffic goes from the Gateway to your back-end services. So some common examples of those in the industry might be Apogee. The azure API gateway, mule soft Kong, the AWS API gateway. So if pictures help you associate vendors to solutions. Think of these when think of an API gateway- and this is what Canalis helps to solve and we'll talk about it more in a little bit. So that's what an API gateway is I'm.

A

So now, I'm gonna take you through a little bit of history on the API management at Northwestern Mutual and how we've evolved from where we were two years ago on kubernetes 9, to where we are today with Canali in production.

A

So here we have two VP C's. This is a setup that we have for our Northwestern Mutual website. So here we'll have a VP C on the left and a VP C on the right and we'll peer them and we'll have our we'll have our micro, apps or the apps that are running our UI in one VPC and then we'll have our kubernetes cluster in another. And so here, whatever our cluster and the way we started out is we gave every kubernetes service and ELB me expose it by a node port.

A

And then this blob of squares represents the vendor that we were using and all traffic from the left, V PC would go through our API management solution and it is outside of the context of kubernetes is a separate vendor and then traffic would get routed to the appropriate, ELB and then he'll be would route traffic to the service on prom woes.

A

It got pricey over time as we scaled a number of services, each service needed its own EOB, and so we needed a better way, and so we did what's lots of people in the community did?

A

Is we had an ingress controller and then we threw one alb on top of it and then service would go from the API management vendor solution through the EOB onto the ingress controller and then based on context route and some other some other items traffic would get, would land on whatever kubernetes service you wanted, and so then you got rid of the price. Now this may look calling all well and good, but what it's not showing you is the mess that may or may not be like the vendor solution.

A

So what happens is when you have a cloud native tech stack? You want the tools that service, your application to be as cloud native as possible, and so, if you're, using a vendor as an API gateway, that's outside of the context of kubernetes. You lose a lot of insight that you can get from having something running inside of the context of kubernetes and then because it's maybe not a cloud native lucien, it's different and hard to automate.

A

You may need like two or three different types of databases to store the information and it's hard to integrate into your into your pipeline in your CI CD model, because now you may be forced to configure stuff through the UI when you want to do them as code and so there's a big mess of stuff, and it didn't really fit well with the cloud native approach that we wanted to take with our environments.

A

And so we thought that we could create a an open source solution for this, and if we were going to create an open source solution to solve this problem, we have some design goals and have very high-level design goals. So, when I had to be innovative, so we needed to be solving a. We need to be solving a problem not only in the context of Northwestern Mutual, but also solve a problem in the context of larger open-source community, so that we can contribute it back and that others could benefit from it.

A

I need it to be lightweight, so it needed to be just minimal resources minimal. You know amount of pods if it's in kubernetes running and just stupid, simple to deploy very easy barrier to entry performance centric. So this probably goes without saying it's middleware components, so it needs to be very performant, robust, API key management at the heart of any. You know, API management tool. This is a big piece of it, so we needed to have a robust, kubernetes native a way to handle it. You did have an extensible plug-in framework.

A

So again, you'll also see this with API management vendor solutions. It's maybe not a good idea to hard-code every single possible thing that you might want to do into an API gateway, but create a way where other people can add their own functionality. So if they want to do their own complex transformation for their business use case, for example, they can do so.

A

It need to be kubernetes native. So kubernetes is all about its metadata, and when you have your Gateway in the context of kubernetes, you can do some pretty cool stuff with canary routing and also things so we'll look at in a second. But you know custom resource definitions, you're able to create kubernetes native specifications, and you get things like storage for free divers tooling. So the gateway is just one piece of it, but you can't have a full API management solution without tools for tracing analytics and monitoring, automatic API, key generation, etc.

A

So it needed to have this whole ecosystem of tooling around it and needed to be easy to automate, and this is one thing that we did not get with our vendor solution that we were using before and by easy to automate it not just that it need to be easy to automate it needed to fall in line with the same patterns that we were using for our existing cloud native automation, so that spawned Canali logos very new and there's stickers at the sticker table in our booth so feel free to pick them up.

A

We have a lot of them, so what is canale Canali is a lightweight kubernetes native API gateway that, together with robust to network policies or, together with network policies, provide a robust, open source solution to kubernetes ingress API management and API security. So I'll talk a little bit break this down a little bit as we go through the next couple of slides, but first I want to go back to the picture that we looked at before and look at how with Canali this picture changed.

A

So, basically, traffic would all go through the EOB and now we'd be running our API management solution, which would also do our ingress force and traffic would land on one of the pods and it would get routed to the service, so the complexity reduced tremendously, resulting in a lot less infrastructure needed and then yeah. So so now now, let's look at some of the differences between a gateway and a service mesh. So a lot of you have probably want to talks about a service mash talks, so SCO envoi link your D etc.

A

So we have Canali at the top, and then we have our three application pods on the bottom, and this is what we want to happen. We want everything to go through Canali, our API management layer and traffic to land on a pod problem is that this is also passable by default, and this is not desirable, because service service on the left could talk to service on the right and not get managed, not get verified. You know with the API key, etc, but kubernetes provides a native construct for us to fix this Nats network policies.

A

So we control these lines in here and say: hey only traffic is allowed to come from Canali and so it'll make this um it'll it'll make this this traffic not possible, and so with this we're able to take we're able to completely get around the service mesh and make our clusters a lot more lightweight and just have the gateway model.

A

Alright, so I mentioned Canali is lightweight and when you think of an API management solution rate limiting and quota come into their some people think as quotas, a subset of rate limiting, but just take that and/or. Take that as a whoever you think about it, but we needed a lightweight way to basically enforce this across multiple pods. So, if you're using a vendor as an API management solution or basically just like you know this monolith application, it can, you know, check its database. Get an information back then perhaps see the requests.

A

But one thing: Canali boasts is zero network overhead on any requests. So when you Dan incoming requests, the only network traffic that happens is the actual proxy to the backend application and back and so basically Canali implements a ventually, consistent method to do base to do rate limiting. And so what will happen is we'll have an incoming request here on the right. It'll hit, one of the Canali pods. That pod will talk to at CD, / g RPC and then via G RPC.

A

It will let the other Canali pods know that has had a traffic point that it's able to update it sufficient in memory store, and we have eventually consistent rate limiting. So this set CD is different than the actual kubernetes at CD, or at least that's the recommended approach. So Canali is open source open source. It's it's a very, very new project. It's like two months, two months old, we've gotten some good traction on it so far and it will continue to grow.

A

But now, let's dive into how Canali works from like a user side of it, and this gets into more of the custom resource definition side canalis a couple weeks away from a two dado release, and so the specs will be two title: specs that we'll be looking at so just to give a brief about custom resource definitions not going to spend a lot of time on this.

A

But, as you probably know, keynote mentioned it this morning, but kubernetes is a very API driven application and there's three parts to every API: there's a group name, a version name and a resource name and so like. If you look at the changelog, you know things might be moving from one version of the next, a resource may be graduating to a different version or they may be moved around to different groups. I think one date did some of that and Canali does the same thing.

A

So Canali has a group name, canale Tadeo versions and then resource names, and it has four major resources which we'll look at, but this is basically how it extends the extends the API. So the major spec is the API proxy spec, and these three examples provide examples. A lot of functionality, not all the functionality, but the main thing to note is the different type of backends that you can do so.

A

Canali allows you to proxy two services inside the cluster, allows you to practice services outside the cluster and allows you to proxy to mock targets which define mock responses that can be returned, and this is key for or the most interesting here's the middle one is the external endpoint. So it's a large organization will have a lot of our services running inside of kubernetes. But what about all the services that are running? Maybe on the legacy applications you're still going to need some api gateway solution to do those right, but with?

A

If you think about it, you can still proxy traffic to legacy applications, but still have part of that as a cloud native. So you can take your API management piece and make that cloud native, and it has a lot of benefits because you get things like storage for free. You get cloud native configuration or kubernetes native configuration. You get a nice pipeline around permissions as code which I look at it a second, but that's something that that is cool and will be new in dado.

A

But the other thing you'll see here is and we'll talk about plugins some more in a little bit, but you'll notice a plug-in section, and this says what plugins will get executed in every response or on every request, and one of the big things for us is we needed a way to have version controlled plugins, and that was something that was a foreign concept to different products that we had looked at so here we're saying that we're going to use an API key plug-in on every request.

A

We're gonna use this version of this plug-in and then we're gonna pass in some arbitrary configuration to this plug-in that the plug-in will need to do it's logic and it's a list of plugins. You can create plugins. You can add them to your proxy and it will run in a synchronous manner. So this is the API proxy spec, we'll see a demo this in a little bit.

A

But next is the API key bindings back so Canali files in our back like approach, so we have API keys which are abstract from which are abstract it from proxies, and then you have API proxies and then you need something that basically binds a set of keys to approximate the API proxy spec does, and it basically says here's a list of keys that have access to this binding and this example were giving Bob's API key access.

A

We're saying that whoever uses this binding, Bob's API key is able to call this proxy 100 times a second and by default they have access to everything. Let them let them call everything except. If you come in on the slash balance sub path, then we're only gonna give you permission to use the to the get HTTP method. This is just a simple example of how you can configure how you can configure permissions but notice that it's completely abstract from the from the API proxy.

A

Another thing that we're doing in v2 here on the right with the API key spec is we needed a way to enforce or not enforce, but to allow for non-breaking API key rotation, and so we follow. We bump back and forth a few ideas, I github and ultimately we landed on this one. But say you have an active API keen.

A

You need to rotate that you'll basically create another version of a revision of that API key and make it active and then you'll keep the old one active until teams are able to update their projects, at which point you'll make it inactive and then eventually removed from the spec. But this allows non-breaking API key rotation, any API keys back then there's a mock targets back which I'm not gonna, spend a lot of time on. But if you do use the mock target, back-end option just way to configure a mock response.

A

Canalis a go program, so this just shows how you can configure go. If any of you are go developers, it uses Cobra and viper so standard configuration. You can do off of that I'm, so plugins, so Canal uses the go pack or the the go plugins package for plugins, and the only requirement is that it implements the plug-in interface and you can do whatever you want within that with within these functions that will get called in the lifecycle request and it will execute you conversion these.

A

So, for example, the API key plugin is stored in the separate repository that is dynamically brought in when Canali starts and and the logic executes and then will bring in the version that we want. And so, if someone else wanted to design their own plugin, they would create a project implement the plug-in interface. They would change the configuration and then dynamically load that plug-in at runtime and then or yeah every one time, and then it would execute their logic, so decoupled extensible plug-in framework where they can do whatever they want.

A

There's a yeoman template which, which does some of it so tooling and we'll see more of this in the demo, but the first tooling is you need a way to automatically create API keys, and so we do that with Canali cuddle, which is a CLI tool for Canali, and we use this extensively at Northwestern.

A

Mutual and developers seem to like it seems to make their lives easier, but it's just a binary that we distribute and allows them to create API keys and then they're able to add it to the cluster and because API keys me nothing until they're actually bound we're able to let developers create the API keys and then go to the service team and say: hey here's, my API key at its here add it to your service and that models worked for us. But it's also in permissions is code too, and we'll see a demo of this.

A

Next we use graph Anna and in flexi b4 for analytics and monitoring, and we've found a completely configurable. The default dashboard comes with like response time. Total requests, error rate stuff like that, and then we use open tracing Jaeger comes out of the box with it, but the main part is open tracing, so it works right out of the box with open tracing. So it'll play nicely in that in that life cycle, and then it's Yeager's the default client for that all right. So let's do a demo, that's.

A

How am I doing on time doing REO all right, so I have I've, went to the Canali github page and I've executed the QuickStart steps here so I've started, mini coop I have so I've flown the project. I've started mini coop, I've, spun up Canali as well as Jaeger and influx and Griffin, etc, and I basically have this just as just to make sure it's working here. I have.

A

All right so I have a deployment. That just is that big enough for everyone. I have a deployment. Just a simple service. Just returns just returns a simple message when you hit it and then I have an extremely simple version of an API proxy. Now this is a V once back, it's very similar, some subtle differences, but basically I'm coming in on slash coop con and that basically, with this configuration, the only thing it's doing is ingress for me.

A

So if I go here to my cluster and hopefully everything's still running awesome for do, coop cuddle apply I'm just going to apply this and then try to curl the service. I'll get a response. I'm just using postal use for a future example, but like I, have my ingress working so now, let's go ahead and basically add some API key management to this so I'm going to go and create an API key.

A

So I'm gonna use Canali, cuddle and I'm going to create an API key I already have this command saved somewhere? Alright. So when I create an API key, I'm going to call it Frank's API key when I give it the name I'm going to give it the location as a public key and I'm gonna save it to this file.

A

Alright, so here's my API key that I'm gonna use the actual API key spec has been saved here. So I can open this up in my editor and there's the API key that it created for me. I can go ahead and add this to the cluster.

A

All right so now I've went ahead and added this API key to my cluster. Now I can go back to my specification and I can basically say so now that I have my key I'm gonna go ahead and enable the the API key plug-in on this proxy and I'm gonna create a binding which basically is gonna, bind Frank's API key to this proxy I'm by default, I'm gonna, let it call everything over the get method.

A

But if you come in on the Foo sub path, you can only you can only call it via the post method. So basically it works by finding the highest priority rule. So if there's no that's, if the incoming request isn't overwritten by a sub path, then it's gonna fall back to whatever the default rule is I'm gonna go ahead and apply this.

A

All right, so that's applied now I'm going to go ahead and try to try to curl.

A

This service and it's gonna, say that API key is not found and requesting. This is enforced by the actual by the API key plug-in I'm going to go back to where I can figure. The key, which is right here and I'm, going to add a header and.

A

We call it API key, then I set it equal to the value, and then it's gonna give me my my response now, if I change this and let's say so, I was using the post method if I use this, as the get method will see that API key is an opera and why so, if you see my subpath, its flash coupons, /qu confer bar now slash coupe con is the incoming path, but we're overriding the target to slash now here, we're saying anything that starts with /foo we're only gonna give post access to and the increment request matched this sub path.

A

So it knows that it only has access to post and not get, and so that's why that's why the traffic or the API key was not authorized.

A

So now we're going to we're going to we're going to curl this service a whole bunch of times, and it doesn't matter if it's doesn't matter for airs out I want to show well, actually I want to show you a ger, so maybe it does.

B

Api key equals this and then we'll set the method equal to post. Okay,.

A

So I'll have this go a couple times now, as this is going we'll go over to Jaeger and just by default out of the box. You'll you'll get your tracing with it. So if you went to any tracing talks, specifically your own Jaeger, there great talks, but here we see the overall span. We see the actual plug-in on request, lifecycle hook happening and we see that Frank's API key is the API key being used because the backend service also instruments open tracing.

A

We can see the extra requests hitting the back end, which is this right here, etc. So the talk this talk is not about open tracing, but it does come with open tracing out of the box and if I look at my metrics in graph Anna I can see some metrics start to come across total requests, average proxy response, time, etc. Response time, percentiles so see it's a very brief demo.

A

So a future work, so v2 at O is like one or two weeks away, but everything's tracked by the normal, the normal open source flow. Everything is tracked by issues etc so give the demo. So there is a tutorial, a tutorial that Canali. Oh, this is where the documentation lives now it'll probably be ported ported to something else in the future, but for right now. This is where it lives.

A

All right, so five minutes left so I'm gonna ask the first question. So what about Sto so earlier? I hope I kind of justified this a little bit. But Canali is not a service smash, it is an api gateway, and so because it's not a service mesh, it does separate itself apart, but then also it's not an API management tool Canalis.

A

What are the only ones that I know of outside of Kong? That is actually an actual API management tool.

A

Viseo does have plans in the future to add API management to their product or to the project, and when that's done, we will see what comes about it. But I know it's more for vendors to be able to hook in their specific API management to it to fall in line with that life cycle. So Canali is an api gateway now to service mesh. It's a very lightweight one and everything is fully included. So you get the API key management. You get the API proxy. You get the binding everything is out of the box.

A

You get storage for free because it's kubernetes, because the customer resource definition and it when, when you start to think about SEO, is versus what Canali is. The differences are very different, so so yeah so adapt. Let me answer whatever questions you have and we'll start yes, I'm correct. So it's so it's not a super smashed and lightweight meaning. It's it's just one pod. It is the Canali pod and that's it.

B

A

I'm, so HTTP is the only protocol that it supports right now. There's future work to add other protocols, but we just want to get it out the door and currently the only support station or HTTP um might support HTTP two as well, because I believe by default, the native go HTTP library supports HTTP, I.

A

Don't I don't believe so now.

A

We have it in the back. There support what oh, so there is no official JD Beauty plug-in. Yet there will be that one that comes, but that's the thing with the extensible framework is the frameworks there and you can build whatever you want. So they're like the JD Beatty would be a plug-in that would that would live in its own repoed. That would be included in.

A

So the beauty has the beauties with open tracing. Is you don't it doesn't matter how many pieces are in the puzzle as long as you play nice and handle handle the context as it comes in and then pass the context along when you're done with it, whatever downstream may?

A

If someone else wants to do something with it, they can but Canali plays nice and that flow will. It will accept and and use, and then it will also you know, start if not or impasse along so.

B

A

So I say one pod, but it's one like deployment. So it's it's one. So in our deployment we run it as a daemon set, but you could run it as a deployment as well. But it's it's like one like there's. Only one container yeah correct correct, but it's it's it's a it's a replicated pot in the back yep.

A

Correct so API so Canali at its best is an API management solution, but API management or the API gateway is also a proxy. The thing that I didn't show is Canali does do it does canary routing as well and dynamic back-end service routing. So you do get some of that as well, but yeah.

A

Yeah we so so we've been running Canali in production for a few months now we I think there's been times during our batch job, we'll be running like 300 TPS and it's I, don't I, don't know if we've hit it with a lot of low pass, 300 TPS, but it's it's been running in production at times that 300 TPS.

A

So it's kubernetes native. So when you're inside of the context of kubernetes, you could do very powerful things, but then also by using custom resource definitions, you get storage for free. The way you can figure, it is kubernetes native everything about. It is like it's like coop, that that kubernetes native cloud native of format, and so like, like with the AWS api gateway, like that's abstracted from kubernetes, and so you lose the context.

A

But then you also have to worry about how that's managed behind the scenes and then, if it's and then like in a cloud native context to be portable, you'd have to revisit your your solution.

A

So so we're running v1 in production right now, v2 will introduce the sed G RPC and- and so we don't have v2 in production. Yet, but yes, it will be another one.

A

The reason why it's not recommended to use the same sed as kubernetes is because if you have access to the sed that kubernetes uses you have root access to the cluster, and so it's it's always recommen, even with other solutions that use extra separate entities like calico, they recommend separate FTP, so there's so Canalis Greek for canal and so, like all of these projects, o Canali open for kubernetes open-source projects, I'll fall like the nautical theme and they find a word and they turn into Greek.

A

So Canalis Greek for canal, and it's kind of like that canal owns the cluster but correct. So there's probably opportunity to also have them write and see and then pour them to go and whatnot, but yeah and the way that'll work is they'll, be in v2 they'll, be like a requirements like diamo file or whatnot, where you say like here's, the plugin, here's, the version canal you use like in an it container to load them in and start the Canali pod and you'll be good.

B

A

To make the wet yeah so right now it uses in flux out of the box, but there's no reason why I couldn't use like Prometheus or something else right now. It's not pluggable, but it yeah it. It should be in the future.

A

A

Yes, with CPUs, so it out load we'll see each instance of a Canali pod to use like half a CPU and.

A

Probably like a gig of memory, it's 'men. It gets tweaked as we find different ways to improve on it. But that's that's! That's! That's the v1 production numbers.

A

Yes, so um so Canali does do SSL termination. If I look I believe it was in the spec, you were able to specify an SSL object and it basically you give it the secret name and it's a type kubernetes that I owed to slash TLS. It enforces the certain the key and, if see a dot served as present it'll be bi-directional, so so believe we're out of time. Thank you guys very much. I can answer questions after.