Cloud Native Computing Foundation SigstoreCon NA 2022, 28 Oct 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Verifiable Build Environments in the Cloud: Powered by Sigstore and Enclaves - Fabian Kammel

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Verifiable Build Environments in the Cloud: Powered by Sigstore and Enclaves - Fabian Kammel, Edgeless Systems

Confidential computing is a breakthrough security technology. With it data can be kept encrypted during processing. Tools in the confidential computing space utilize these new concepts to provide fully-encrypted, high security environments, but as everyone in security knows: you are only as strong as your weakest link. Supply Chain Security is one of our industries weakest links. This talk will provide a deep drive of how Sigstore can help confidential (and other high security) products maintain a high level of security, keep their trusted compute base minimal, all the while preserving a high engineering velocity. To that end we will sketch out an architecture to build and sign in the cloud without malicious actors being able to steal signing keys or tamper with build processes. We will also show a live working demo of how such a system could be realized.

A

Cool thanks for the introduction um yeah. My name is Fabian I'm coming from edgeless systems, and I would like to talk to you about a verifiable built environments in the cloud and yeah to motivate this and give you a bit of an overview. We want to basically answer the following three questions in the session. So first off, why do we need Better Built environments? In the first place, then?

A

Second, we want to explore if confidential Computing could be a solution to tackle either one or all of the problems we are currently facing and how a confidential build environment could actually look like, like an architecture sketch draft a first point to basically start thinking about a solution.

A

Okay, first, a bit about me. So I have worked five years in automotive, Enterprise building, Cloud native Key, Management Solutions, and this year I've joined editor systems as a senior security engineer, and this is the second talk this year. I will give to share my confidential Computing knowledge with the kubernetes community, because I personally wasn't aware of where confidential Computing is previously, and it's really starting to take off and get us some real world benefits.

A

I can also take this off right. All right. uh We ageless Building open source tools for the confidential Computing space, so, on the left hand side we have basically our client libraries ego as an SDK for building software for enclaves and I will go into a bit more detail and further slides, and we also have marble run as a service mesh for enclaves and we have agile sdb as a confidential database and on the right hand, side.

A

We have constellation as a confidential kubernetes distribution and confidential Computing in general wants to reduce the trusted compute space like reduce a trust as little parties as possible in your stack and building these tools out in the open, using GitHub actions is kind of a bit wacky right when we measure the whole stack and make sure that everything is confidential but at the same time we're kind of clueless about what really happens in our built environment. So this is only one of the challenges like some companies might might face when they want to adapt.

A

Sas CI CD Solutions yeah, the first one is basically a quote from us, so I, don't trust the provider. I have zero trust policies, as we have for networking like zero trust is a good thing. You don't want to trust unnecessary parties. This would just increase your vulnerability uh surface. Basically and then.

A

Second, a lot of companies I've talked to are worried about intellectual property, so they don't want to give away their code to some unknown untrusted party and take the chance of leaking the code, which is basically all of their business, is relying upon that being Secret.

A

So, third one we have supply chain attacks and I. Don't think I need to talk to you about why this is important, um but it would be nice if we could know like. Can we verify if there was any code injected during the build process and code injected really going down to the lowest level like what's the firmware manipulated, for example, because this is something we can't measure today and then. Lastly, we have data privacy concerns I come from Germany, so EU regulations are very strong about, like not having data or code leave.

A

The country, so it might be a bit difficult to trust some I, don't know cicd solution that is hosted in the US and get your data over there.

A

Okay, so confidential Computing um up until now, I just talked about this stuff like who has actually heard of confidential computing, all right a few and who has used like a confidential VM or an enclave before and worked with that impressive. Usually, there are no hands for that question all right. So what is confidential? Computing first wide adoption basically started with with Intel sgx or their so-called enclaves, SJ extending for software guard extension and what this basically provided us was three defining properties. So first one was a process based isolation.

A

The process was isolated from the rest of the system and at the same time, the process was also runtime memory encrypted. So even if some other process could read the memory of the process break the isolation, it was still encrypted with a key specific to that process and they would just read garbage.

A

And thirdly, a remote attestation- and this is really important. I think for the whole verifiable build environment, so keep an eye on that. One remote attestation allows us basically to measure everything that is running inside that Enclave inside that isolated area and then sign it with a trust being inside the CPU inside the hardware. So the CPU attached to the fact that only the code and data I attended to run in that Enclave is actually running there.

A

So this is how this looks in the end, so our app or whatever talks to the to the hardware and instructed to create an enclave, and then we can load code and data. If we want, we can load it encrypted into that enclave and only ever decrypt it inside that secure environment with a key only being available inside the enclave, and all of this is enforced by the hardware. So I do not need to trust my operating system, the hypervisor all of those are outside of this boundary.

A

So this is process based without the operating system, which is great for a tech service. But it's horrible. Like horrible for developer experience right, you need to touch your code. You need to recompile it. So it actually works in that isolated environment where there's nothing but the code and the data you put in there.

A

So the tech second technology that arose was then confidential, VMS, which basically widened the surface. It's not just process isolation. You have a whole virtual machine that is isolated from the rest of the things that are running on the same CPU um and the most usable form of confidential VMS is currently from AMD. So it's AMD, Sev SNP, that's a mouthful like it's a secure, encrypted, virtualization and secure nested pages. So it gives you the same guarantees.

A

We discussed before, like it's an isolated virtual machine with memory encryption at runtime, and you can again query the CPU to measure basically over the whole virtual machine down to the firmware level bootloader in it and so on and get an attestation. So a signed thing of all the software that is running there.

A

This is realized by the VM directly talking to the CPU. There is a so-called secure processor, an SP that is running inside the CPU, and you can basically ask it for those things like create a new confidential machine measure it and provide me that attestation statement.

A

So how does it look like a remote party, for example an administrator or a devops engineer, or someone wants to verify the state of the virtual machine and they ask the virtual machine for this attestation report. It in turn asks the AMD CPU, please verify or provide an attestation about myself.

A

Anything can provide that back to the remote party it's signed by AMD, so you can just download the root CA from AMD and verify that attestation Report with it inside you will find a list of hashes which basically corresponds to all the different layers of software that are running inside the virtual machine. So the firmware is measured, the bootloader is measured, the OS is measured and everything else you want to include okay.

A

So where can I use these today, like Google and Azure, both have confidential Computing offerings with slight differences in implementation and AWS has Nitra enclaves with a big difference in implementation. I won't get into that. They kind of wrote their own thing, both Google and Azure, mostly relying on AMD and also Intel, will bring TDX, hopefully at some point in time to also provide confidential VM support.

A

Okay, so how can we use that new knowledge, those new Computing Primitives, to tackle the challenges that we talked about earlier? So we have the zero trust policy challenge to tackle this. We really want to establish a route of trust inside the CI CD system.

A

We have supply chain attacks, we want to mitigate and we can do this by verifying the build system's Integrity down to the lowest levels and we have those data, privacy and IP concerns. So we can control the access to sensitive data by having all of it runtime encrypted and not being accessible even by the provider of the solution.

A

All right, so, how might this look like? So we have five entities or parties involved. We have the Enterprise, on the one hand, side who has the repository with sensitive code and data, and we have a user like an administrator, wants to configure or integrate with that third party sales provider to use the cicd offering there on the SAS provider side, we have two confidential environments. The one is the trusted build service, which is basically our root of trust in that environment which can carry out and orchestrate all the builds in the CI CD system.

A

And then we have built environments, you're, probably familiar with like GitHub actions, where you can just request Ubuntu latest as a built environment.

A

Both of these should be confidential, Primitives, either an enclave or a CVM, and then thirdly, we have record and or provenance storage yeah to do just store the metadata and do the verification and so on.

A

Okay, how would we use such a system? So the user initially connects to The Trusted build service which was set up by this hat provider. They can put in the The Trusted build service, binary and run it, and the user can basically log in and for the first time and lock down that system, as you would with any any server software like when you set up gitlab, for example, you set a root password and log out everyone else something similar going on here, but in addition to that, you also want to verify this attestation statement.

A

So it's really like the firmware. Okay as the OS okay, it's the trusted build service. Okay- and this is all software I expect to run there.

A

We could verify this against the white list, but this is kind of static, so what we're doing with with constellation also with our product, is we sign and then publish those meta information to record so that customers can dynamically request? The newest trusted measurements that we put out there, which way you prefer is basically based on your risk appetite right. You can say no I only want these two bit environments or just the build services, or you say, I want to have all that updates.

A

Update all right, then. Second, we need to authorize the access, like some simple scenario like a web hook, as you know, from tecton, probably or other build tools could work here where you basically allow the enter prize repository to communicate if an event is happening. That should trigger a build okay now for the interesting part. So when the build is triggered, the repository can transfer over the code and the metadata that basically defines our CI CD process and The Trusted build service can use this to request a built environment.

A

So if my corresponding GitHub action says Ubuntu latest I can request that I want to have a Ubuntu latest built environment, but in addition, The Trusted with service can now also verify those measurements. Like am I really certain. This is a secure environment. Before I put in my code and data and again same same thing, we could verify against the whitelist or we can ask record. Are there new measurements? I should trust.

A

Okay. Afterwards, we can just forward code and data to the build environment. Have our artifact produced installed somewhere.

A

The nice thing about the setup is that the build service, which I think is also called build service and salsa, is really the man in the middle that can control or at least observe all the information like which code commit was used to build, which commands were triggered to build the binary, but they can also store these measurement information like which firmware was used to host the OS to actually run my build stack in the end, and we can remember all of this information put it in the provenance and then publish it in the end foreign there's.

A

A lot of detailed information like and most customers might not be able to reproduce and verify everything. That's quite a huge leap to get to that point, but even if we don't verify everything from the beginning, we have all the information there and have it in record or some provenance store. So we can verify it later or use it later. If an incident is happening.

A

Okay and then we also have the decision, whether to use cvms or enclaves as the built environment. Right so remember, Enclave is just a process isolated. You would basically have to rewrite your whole build stack like if you're, using GCC or whatever you need to recompile it and make it work in an enclave. That's a huge step to take. You could start with a CVM, just lift and shift it works out of the box, but Additionally you can verify like which code is running at the lowest levels.

A

All right advantages: yeah, we have the hardware verified root of trust. I was riding that horse a lot that helps with the zero trust policies, especially for us writing software. For that confidential space we have the hardware verified Integrity of the built environment, which helps with supply chain attacks, and we often always encrypted bit environment that is not accessible for the SAS provider. So I can provide my my sensitive data and use it to build. My my artifacts in the end.

A

All right, it's a quick recap: I think that six-door salsa and confidential Computing could really come together here and bring enhanced privacy and security properties to the CI CD offerings. We have today we're a small startup with 10 Engineers, so we're busy building the tools we have already, but if you're interested in getting in touch, maybe building this together as an open source, Community I'd be more than happy than helping with the confidential parts of the system.

A

Let's explore this idea. I think this has some potential and again it can enable a lot of different scenarios from lift and shift to really high security options for your build environments, so I get them out of your data centers and into yeah the SARS offerings. We love so much that enable us to write software fast again. These are the tools you find them on GitHub. They really help you to get up and started with confidential, and this is everything I had prepared. So if you have some questions around those ideas, there's your chance.

A

All right, everyone still sleepy from lunch, I think all right. If there are no questions, then just find me afterwards, I'll be here all week. um You can find me on on Twitter LinkedIn, just give me a message and we can meet up anytime. All right thanks.

A