Cloud Native Computing Foundation KubeCon + CloudNativeCon North America 2021, 29 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Untangling the Multi-Cloud Identity and Access Problem With SPIFFE Tornjak - B Lum & M Sabath

Description

Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Untangling the Multi-Cloud Identity and Access Problem With SPIFFE Tornjak - Brandon Lum & Mariusz Sabath, IBM

When an organization moves to a multi-cloud environment, one of the first questions a developer will ask is “How do I access my S3 bucket in AWS from my GCP cluster?” (or any other permutations thereof cloud services/providers). This is an unsurprising request. However, the solutions to these problems today are surprisingly inadequate, especially when security and compliance are considered. This problem stems from cloud providers/services each having their own notion of workload identity and schema, which makes federation difficult. This talk proposes a shift in the perspective of workload identity from being “platform specific” to “organization wide” using SPIFFE/SPIRE and the new SPIFFE Tornjak project to provide a consistent and secure organization-wide management plane for workload identity and access across multiple clouds. After all, user identities are managed on the organization level (e.g. LDAP, etc.), why should our handling of workload identities be any different?

A

Good to be here back in person, um so today we're going to talk about a couple things we're going to be talking about um workflow identity, we're going to be talking about multi-cloud and, more specifically, we're going to be talking about the multi-cloud access problem and how we are going to solve this with workload, identity and technology such as spiffy, spire and tanya.

A

So before we go ahead, a little quick introduction, my name is brandon and I'm here with my colleague marush and we are both from ibm research and today we are going to start by telling you a bit more about workload: identity, uh how it works in a cloud provider. We're then going to bring this into multi-cloud context and we're going to talk about solutions that work today and some of their limitations.

A

And then we are going to introduce this notion of a universal or organization-wide identity, of which we have a really cool demo for you.

A

So we have a lot of content to dive into so let's go ahead, um so let's first introduce the concept of vocal identity by looking at uh the ana, the anatomy of a vocal identity within a context of a single cloud, so every cloud provider kind of handles uh local identity in a similar way.

A

So each of them have has an identity provider that comes in the form of a root of trust or certificate authority. This certificate authority, then issues identities um through the form of usually some kind of infrastructure. Service, such as you know, for aws example, would be the aws metadata service.

A

So we then have a workload, that's running on, let's say kubernetes or openshift, uh this container that wants to access a service such as a database. uh What they have to do is then they have to talk to the service to obtain a workload, identity that comes in the form of usually a x, 509 certificate or jw key token.

A

They then present this to the identity and access management on the top of the cloud, and because you know the cloud is well integrated with itself. It's able to authenticate the workbook identity and provide access seamlessly.

A

So now, let's talk about this in terms of multi-cloud. In this scenario we have an organization example where we have three different clouds. We have ibm cloud, we have s0 and aws and blue green and yellow respectively.

A

So we notice that, if a workload kind of lives within its own silo, we don't really have that much of a problem and, in fact, within a single cloud im identity access management is pretty much a solved problem.

A

However, the difficulty comes in when we all do multi-color access when we want to access a resource from one cloud from a workload in another example of this is ion access and aws s3 bucket from a workload in ibm cloud.

A

So let's talk about how this is done today right, so the first method is using long live credentials. uh This is probably something that most have had done before myself included, and what this involves is um you know, for example, if I want to access azure cosmos, eb as a resource for ibm cloud, what I would do is I'll go up to the azure service say: please create an api key for me, or some long live credential and myself as an administrator, would take this api key, which is just a bunch of um bytes.

A

I'm gonna create a kubernetes secret. I'm gonna mount it into my workload and my workload will then be able to access the service. So this technique is is great because it's very simple: it's really easy to get started. uh Unfortunately, it comes with several caveats, so one of which is, we are using a long life credential. That means that it's very difficult to manage the risk. uh If there's a there's, an incident that happens it's difficult to measure the blast radius.

A

um The second part of this is that, as we saw, we kind of had to trust an administrator or component such as myself in order to handle these highly privileged credentials, and therefore we are putting a lot of trust in a single admin on a single component which, in compliance cases, can lead to a lot of complications.

A

So the next method of doing this is through cloud federation. The idea behind federation is fairly simple. It means that we want to teach an im system for one cloud: how to interpret and how to authenticate identities from another right. This is usually done by protocols such as oidc or taking the trust, bundles or certificate authorities, and then moving them to the next cloud, and this is great right, because we are still using the short-lived credentials from the cloud. We don't need administrators or any intermediaries that come, and you know, handle these credentials.

A

Awesome talks over, unfortunately, that this is not the case, so there are several problems with federation today. So the first is that federation support kind of varies within within the clouds. If I want to access a gcp resource from aws with google cloud's identity pool service- and this can be done fairly easily, however, if you want to do the opposite, this is a fairly involved process and this blog post linked here actually describes. You know what you need to do. It's you need to install a bunch of things. You know kubernetes, cluster and so on.

A

um So besides that, um secondly, each cloud does workload identities or defines workflow identity slightly differently. So every cloud provider has its own identity scheme. It may contain a bunch of attributes. It may contain policy, ids security group ids, and things like that.

A

The issue with this is that some of these fields may not have meaning in other clouds, for example, aws security group ids when we see it from in ibm cloud perspective or zero perspective, don't really have any meaning and what this results in is. It becomes difficult to manage identities or to create meaningful access policies across clouds within your organization and, lastly, comes the problem of scale.

A

So we are doing cloud to cloud federation, so we can see that, for example, just with three individual clouds, if we federate all of them, we end up having to maintain six different trust relationships, and this is not even considering you know what happens if one cloud provider decides to add something into its identity scheme or decides to change some attributes of its identity.

A

So to be able to solve this problem, we don't have to look too far to draw inspiration.

A

In fact, if we ask ourselves, if we look at user identities and ask ourselves how do we, how do we address the problem of user identities and the answer to that is done organization-wide or globally, when we think about a user john, for example, we don't think about john as john aws john ibm cloud, john gcp.

A

We think about john from development, he's a funny guy. You know he he has access to these bunch of clouds right um and so the question is: why don't we treat our identities the same way right? Applications and services are, after all, organizational constructs right. Why are we married to this idea that workflow identity has to be tied to the platform?

A

So how do we do that with workloads? So to do this, um we we say we want to have a universal or organization-wide workload identity. So what this means is that first, I need to define an identity scheme for my entire organization, so one identity scheme. In this example we take from the spiffy book.

A

We can see it has the name of the organization uh it has a region, so maybe, if you use compliance or compliance controls, whether it's dev staging prod and also the department or the project in which the workload belongs to another part of creating an organization, a wide vocal identity. Is you want to really minimize the points of federation?

A

So you have a single point of federation, so this is great because it helps reduce risk of misconfiguration, because operators only need to deal with one scheme and they only have to deal with one federation point so now that we won this, how do we go about doing it and we're going to introduce this concept of a0 trust workload, identity, management platform?

A

I know xero trust is like a very overused word um in this case when we say zero, try to just mean that everything's authenticated and tested um throughout.

A

um So what do we need out of this platform in order to make universal vocal identity work? So, first of all, we need to be able to define what the identity scheme is. We need a way to enforce identity scheme through our organization.

A

It's also as important that we make sure that workloads get the correct credentials and the workloads with the correct credentials really have been attested, and we can prove that it is, they are who they say and last but not least because after all, we want to alleviate the the management problem. We want to have a single point of federation to all the cloud services and how we're going to do this is we're going to use spiffy, spire and tonyank.

A

So all right I'll give you a bit more there. You go awesome. So let's meet the technology, so so these three projects are all on. Cncf projects under the spiffy umbrella uh won't have time to go too much in the detail, but so spiffy first off is um the secure production identity framework for everyone.

A

It is kind of like the identity spec, it defines how workflows can securely obtain their identities. uh We have spire, that is the implementation and implementation of spiffy. In this case it provides the zero trust associations of the workloads and the infrastructure, as well as, together with the oidc discovery. Service provides a final federation, and, last but not least, we have tonya, which is a control plane, slash ui for spire and together with the kubernetes workflow registrar, it provides a way to define uh the universal workload, identity and enforce it.

A

So our claim is that if you install a trust, folklore identity management platform- and you can't exercise this notion of organization wide or universal workload, identity, you're going to get all the great stuff that we talked about and in in in addition to that, you're going to be able to centrally manage, as well as audit all your workload identities.

A

So with that, I'm going to pass it on to marius. That is going to show who is going to show you a really cool demo.

B

Okay, thank you brendon for the great.

B

So, let's take a look at our hypothetical scenario.

B

Of several kubernetes clusters and they are deployed in different clouds. We have microsoft here, amazon, ibm cloud, then we deploy workloads that need to access the s3 storage bucket. So we will demonstrate that using universal workload, identity. This task is very simple and very secure.

B

So first we need to stand up the identity management service with spire and thornyak to manage all the organization, workloads and all the identities in the cluster.

B

Then we deploy spire agents as demon sets with one agent per each node in every cluster, then every igen agent is attested with the aspire server for at the station. We are using cloud provider specific plugins or at the station that is based on kubernetes platform.

B

This solution is giving us zero trust at the station for underlying infrastructure.

B

Next, we use the kubernetes workload registrar that automates the process of creating universal workload, identities for all the workloads in the organization.

B

All the agent, attested workload identities can be easily viewed and managed from one location, which is the thorny interface.

B

Once we have, the universal identity schema using the open id to federate these identities uh across. We can use this uh for uh federating across all the all the clouds. Now we can create and manage policies uh that control access for the all organizational workloads for uh to access to storage, for example.

B

So in our quick demo, we will show you how to enable the identity provider in amazon, how to set up the policy and role for guarding the access to s3 storage, buckets and then we'll demonstrate the workload running in ibm cloud that is requesting the universal spf id for itself from the agent and then using this identity to request access to the storage.

B

So here is the example of the universal workload identity that we use for our organization is composed of the trust domain that represents the organization, the name of the region, cluster namespace service account and pad name.

B

So here's just the example just for fun. We use the open shift in spacex that is running in east u.s, east region. The cluster name is tsi. Cube01 default, namespace and elon musk service account. So now we can define policies that would guard access to the s3 storage.

B

So, in this example, we want the region to be must be in u.s location. We can use any kubernetes cluster, however, the namespace must be mission and service account. Elon, musk and the pod must start with the mars mission name.

B

Okay, so let's do a quick demo.

B

Good okay, so, let's start with the tourniquet interface. So if you look at the navigation bar, we have a management for clusters, agents we can list and manage the entries. There is a section for uh server information and the dashboard tourniac dashboard. So let me just go to the clusters, and here we go. We manage six different clusters and they are deployed in different locations, different clouds. So, for example, we have here amazon.

B

This is azure and there are a few in ibm cloud. Some of them are openshift and others are just a native kubernetes. Okay, if we click here on tornack info, you can see that this spire server is defined with node, attesters and right here we have one for kubernetes, there's one for amazon and azure. Okay, if we click here on the dashboard oops. Sorry, when you click here on dashboard, we can view here, agents, workloads and all the clusters.

B

So let's go to one of the clusters: we're going to use the tsi cube01, which is in ibm cloud right here. Details- and here we go. We have three nodes with three different agents. They are all are tested. You can see here, let's look at the entries, so there is one entry here. This is the workload registrar that automates the entries for all the workloads that would be created here.

B

Okay, so before I show the demo on the how we create the workloads and get the access, let me just show you quickly what we've done on amazon site. So here is our mars bucket. It's called mars spire and there is a special file called mars txt, which is the sensitive file that we want to protect. So this is what we want to govern.

B

Okay, so next we go to identity management for amazon and right here, I'm going to just copy this guy.

B

Bigger, okay, okay, thank you! Okay, so we're going to create identity provider. So let me just click on provider: open id connect type, I'm going to use the url of the spy server, the thumbprint okay. Here we go, it works. We define the audience, let's say my s3!

B

I guess that's it at the provider. Okay done so we just created ourselves our sales identity provider. So let me look all of them. You see that all of them- okay, this is the guy spacex o3. Okay, so we've done this already earlier. So let me show you how we define policies so there's one xo5 that we would be using further. So, let's click on the roles.

B

Okay, there's mars mission.

B

Okay, what roles do they basically tie the policies with the identity providers? So let me show you how the policy looks like come on all right, so there is a policy that defines read access to the mars spire bucket, the one that I showed you earlier and let's look at the trust relationship for this identity provider.

B

Here are the conditions, so the conditions that we defined for this experiment are following: it has to be open shift, spacex organization.

B

The region must be us, so it has to begin with the us prefix and the cluster name mars. I'm sorry mission, namespace, elon, musk service account and mars mission, the pod name, okay, so we're done here now, let's go to the to the cluster, so this is the blue cluster. This is the ibm cluster. So from here let me show you there's nothing running yet all right.

B

So, let's create.

B

Okay, mars demo; okay, so what we just did we created a service, account elon, musk and then deployed a mars mission on this account. Okay, so let me all right: it's running right here: okay, let's now get inside.

B

All right and we have just quick script because I don't want to talk and type at the same time. So let me just show this demo script, so the first operation we're going to retrieve the jwd token, which is basically the identity of this container.

B

So let's do that, and here is the identity in a text format. Let's take a look at edit, so we have open shift space x, that's the domain! That's the organization region is running in u.s, south cluster name, tsi, cube01, namespace mission, elon musk as a service account and the pad name. Okay. So in theory that should comply with the policy that we created earlier and this chunk over here. That's the gwd token. So let's retrieve this guy into a file.

B

All right it's done now. We can invoke the call to a aws to copy the file from this bucket. So let's do this. As you can see, we are using the royal mars mission. We pass the ident, the jwd token that was captured a minute ago and we're just simply asking for a file and copy it locally. So, let's see if it was delivered, it's done. Let's just look: what's inside and here you go, you have a recipe for the concentrated dark matter from the rick and morty series.

B

Okay, so before I go, I would like to show you quickly that we could now go back to our super cool tornack.

B

Okay, here we go. Where was it the clusters tsikubo one?

B

Okay- and here you go- this is new identity that was created. You can see, you can easily see it from tarnick and so are the other ones. Okay, back to the presentation.

B

Okay, so in our demo, you could see how easy it is to get identities in one place from all the multiple clusters. Multiple different clouds and one policy allows access enforcement for several dynamically created workloads, in fact, in our multi-cluster organization. If you invoke the same calls in any of the clusters, you should get exactly the same results in theory, assuming that policy complies with the identity of the workload okay, so we also have a demo that shows the same uh concept, but this time using vault and for retrieving secrets from vault.

B

So in interest of time, I suggest going to the recording that will be listed at the end of the presentation. So just to summarize, the universal zero trust workload. Identity runs on everything that supports kubernetes.

B

It strengthens the security by executing cloud provider, specific attestation, as well as the platform at station, and it can support different identity mechanisms like iam key cloak. Open standards like open id using consistent universal identity schema so brandon back to you very cool, so.

A

Just a little little bit before we end so um the project that you saw the ui is tonya. This is something that marish and I started working on pretty recently uh we're both maintaining it, but we're also looking for more maintainers.

A

um They are not ibm as well help the governance, um it's fairly, it's still in um development. So there's a lot to do. That's it's going really fast.

A

There's a lot of cool things to that we're going to work on such as you know, integrating with the uh the policies as well as aggregating logs, um we're open to a lot of feedback. Tell us what you think um as well as if you liked what you saw in the demo the marriage gave. We have help charts to kind of like help. You redeploy the whole thing as well, as we have a bunch of youtube videos that kind of show a little bit more than what we have time to show today.

A

So, if not um yeah, let's start sharpening our cloud native cattle, um and I think there is one uh talk that that's after after this around talking a little bit more about how to define these 50 ids. If.

B

A

Thank you very much.