Kubernetes WG KMS, 8 Mar 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: WG KMS Bi-Weekly Meeting for 20220308

Description

WG KMS Bi-Weekly Meeting for 20220308

A

Hey everyone: this is kms meeting number nine for march 8 2022.. uh So, let's see I think christoph has things he's been working on, which has things he's been working on being real? Also things you want to talk about stuff, I think we can start with you kristoff. If you want to talk about what you've been looking at.

B

Yes, so uh we discussed um a lot of topics um about a potential reference implementation for kms and how it could look like. So I had a couple of days so two days to work on it, so I used, as I mentioned, the google think library, to make this uh the layout keys. So we have now a kind of an intermediate which I call just check and the root key that comes from the kms on hms.

B

So um this works now more or less. The only thing that is still open is um how to store the keg. So as of now, I um prepended the dex to the encrypted data, which is kind of fine, but for the keck we didn't decide where to store it um so rita.

B

Had the question um where we would like to start, but I didn't remember what the answer was that we gave to her like a month ago and yes, I also uh worked on the pixel 11, um the api, that's called cryptokey and try to break down the um the wrapper around the c library into something sane, because it's it's very bare. So I try to split up and look and have a token logic and the cryptologic on top of it.

B

uh It's still working progress and then kind of then also partially trying to glue it back into what thing expects for um for a working key and yeah. That's currently what I'm doing.

A

Just trying to like scan your code trying to get a sense of where things are.

C

Just uh back to like where the local cac is stored, um I I did write it down in the doc actually one last time after we talked because I figured we will forget, um I think when we said it's in the dock somewhere in the middle, it says uh we need to store a lot local keck in scd and once decrypted it can be stored in memory to be used for encryption, decryption of deck.

B

Okay cool, so currently I just expect an io writer to be passed in and then it will justify something down.

A

So so, when okay, so local tech is the key encryption key that is encrypted using these, the real kms's um key encryption key right, but it's it basically the remote kms does one encryption call for this key encryption, key this location key and then all the other encryption calls are handled by the locals and kristoff. The question is: where do I put this local key encryption key in an encrypted state right? Where do I store it in its encrypted form, right right.

C

A

C

Go ahead, go ahead.

B

No, no, no, I just basically, I have just an eye of writer that I expect and whatever it is, it will be written into it and but I'm not particularly sure if I completely describe the logic, because, for example, what I wanted to do is to attach to the key encrypt to the location, encryption key and like a counter, how many times it is used and then automatically make a rotation.

B

uh So the rotation is already there and also the counter is there. But the question is where, for example, west or the the the things beyond the key right, so it would be a whole whole object. I just defaulted now to gob the the golang globe, go binary format, which is pretty straightforward, but I guess we would like to move to protobuf, but again it. I was just writing down in two days hoping to get as much.

B

As much progress as possible, so I can show something: okay. So so if I.

A

Understand correctly, when uh so the kuru, the kubernetes api server, when it does the grpc encryption call, it passes you a data, encryption key and you will encrypt the data encryption key that was passed in using this local keck and respond with this structure as bites as a bite array for storage purposes. Right.

B

Yeah, so this is also a thing that I need to to continue to to also the kubernetes pass is also a thing that I need to kind of uh work more on it.

B

Okay, so I can take the basic model that we talked about, wrote it down and started to to work on the k, pixie side, and the next step would go to the to the community side.

D

Yeah, I think so as part of the request that we get from the api server to the kms plugin. The plain text is actually the deck right. So today we already returned the encrypted deck uh back to the api server. So in addition to that, at least with the approach that I'm doing, I'm also returning the encoded kick encrypted. Kick so that so there are two things we are returning the encrypted deck and then the encrypted kick and basically uh using the protobuf. We store all of this in hcd and then for a decrypt call.

D

When, in the decrypt request, we will send this encrypted deck, which is the cipher text, and then we will also send the encrypted kick that we have stored in hcd.

D

um At this point, the kms plug-in what it would do is it will look up its cache if it's maintaining it in cache, it'll see if there's already a um encrypted encoded check and then what is the corresponding value for it, similar to how api server does if there is, it will just use that decrypt the deck and then send the response back.

D

But if it's not available, it will make an external kms call to say: hey, can you decrypt this uh encrypted check for me and then once it does, it will add it to the cache and then it will send it back to the api server.

A

Yes, so the idea would be is in memory this. This kms plug-in might have many local techs that it has cached, but it will only have one local keck that it's using to do rights with and that one it maintains an in-memory counter. It does not need to send the counter to the api server right. It's just going to keep it in memory. If it gets restarted. It's fine I'll just make a new keck and restart its counter.

A

That's totally cool, and it doesn't need to synchronize its counter with any other instance of the kms plug-in, because the other one should never try to use its key as a write key. They should only use the keys that they generate because they want to have the counter themselves right.

A

So I think the counter implementation is pretty easy, which is basically just increment the in-memory counter for your right, your right, keck, and once it reaches some arbitrary number, like a thousand some smallish number that seems reasonable, but better than one um you just you just throw it away and you're like cool. I'm done with that one. I'm gonna make a new one and you know there had there.

A

You want to balance that number with the fact that the other instances of the kms plug-in when they encounter that encrypted key will have to make remote calls. But, uh honestly, even with a thousand, I think we would basically sort of compress the amount of remote calls to almost nothing.

B

So currently I set it to 2 million, so it's still still feasible, but this is a really nice idea to not care about the current time. When there's no counter, we just create a new cactus, really nice.

C

Well, the balance.

B

C

Between, like the size of the cash and the number of times, you have to decrypt or right, yeah, yeah.

B

So so there is a kind of a cash for for the decks, um but but something well, I wouldn't say cash. It's just a growing map and and one of the one of the nexus would be to make make it a a rule. Cache yes,.

C

Well for the decks and the local cac.

A

C

A

It's a there's a bit of a like.

A

I would probably expect us to like like like this thing. I would expect to probably make a nice diagram describing where all the keys are and like I like that, you named everything. I think that will be very important when trying to explain this to anyone else, because they'll be like what are you talking about, because we, because the keck and local keck, all those are just a little bit ambiguous. We like need to basically say api server.

C

A

C

Only a little, you said it's a little ambiguous.

E

C

To explain to people that we there's a keck and a deck is already like like wait. Why do we have two, and now I was like wait? We have three.

A

Yeah, if we want to be cute, we can have different emojis for every single actor and type of thing, so you can be like the carrot. That's this. I don't know if people will appreciate that or find it more annoying.

A

I know I I find emojis to make me happy, so I try to put as many emojis as possible, but okay, so I think I think we're making progress. I wanted to ask anish: do you happen to have um somewhere that I could get to uh the definition of the proto? The schema anywhere yeah.

D

I have it in a bunch.

A

Yeah I'd be curious to see it um just just so we can kind of conceptualize it a little.

B

And I will update the the naming so that cac will be the local cac and the root key will be. There really kick.

C

And then 2.2 million.

A

The 2 million I mean, I know it's still fine, but it makes me a little nervous, but I.

C

Mean your cash, I mean I don't even can that cd, I guess scd can still load it.

A

Wait, what do you mean.

C

Well, because we're storing the local keck in std right, if, if, if we're, really doing two million, that's a lot on on that.

A

I think said that he was doing a two million rights per local check right, which is still, I think, well within.

B

I see yeah, so I I calculated it so that was a taylor polynomial um about the birthday problem and do you end up with nearly impossible, so it was as good as the encryption itself, the probability to uh what a um basically a double hit and when you have um 2. 2 to the power of 21.. So this is uh sort of probability that you will have and non used twice would would be. The probability would be really really low.

B

um So certainly you know this magical number where cryptographers say or there's more atoms than in the universe or something so yeah. So it's.

A

Okay, yeah uh so yeah, so the more rights you allow with the local keck, the better performance you get out of your plug-ins because they don't have to do this handshaking as much right they. Basically you know the the remote kms is being used as this synchronization agent effectively like. If the other instances see your key, they basically have to ask the leader, because they're kind of like the followers like hey leader, tell me how to keep doing my job, because some other follower basically made a decision.

C

Yeah, sorry, I misunderstood, I thought you meant, like two million local tech.

A

No two million right yeah.

C

That makes more sense: okay,.

A

uh Okay, anish did you wanna, walk us through the program, real quick.

D

Yeah, so I've made very small modifications right now. So if we look at the decrypt request so I've um before we had the version and the cipher. uh So in addition to that now, I'm adding a uid which is for the observability purposes that we talked about and then the bytes key is basically the encoded kick that we are storing in lcd.

D

The naming can be better but you'll see, but in terms of the decrypt response, the kms plug-in the kms plug-in will still return the key that it used to decrypt um just for observability purposes, but we're not going to be doing anything with it.

D

But if we decide to do some kind of validation in the future for auto rotation, we can do that. uh So that's for the decrypt request and the decrypt response and for the encrypt request. We only because it's a whole new encryption. We only have the new string uid field uh for the observability and in the in the encrypt response.

D

So this is the one that I was referring to so bytes cipher is basically your encrypted deck, which is what is being returned today and then there is another thing called bytes key, which is the encoder encrypted returned from the kms plugin.

A

Yeah cool, so the question I wanted to ask this group is so did this seems uh if we ignore anything related to the whole? uh I guess this, it might not matter for this. uh I don't. uh I know we had conversations about gcm and cbc. I guess I don't know if they even surface in this api, but the question I have for this group is: would we want an api like this?

A

That's very sort of specific to the the idea of this key being sort of this, this uh encrypted key being kind of passed around or would we want something a little bit more generic, that's like sort of like metadata, like some like key value pair, where the the kms plug-in can basically send arbitrary, contextual data back from encrypt and decrypt.

A

That way, it's not like. You know this is a set of bytes, so it could literally be any encoded structure that the kms wanted it's more of. Do we want to facilitate or encourage a nicer structure than just like a set of bites and I'll give you back the bikes that you originally gave me, but you're responsible for all the decoding and do whatever it is um just thinking.

D

Yeah, that might be useful. I I mean I don't know in which scenario they might want it, but I actually having that right from the start would definitely be something. That's useful, like a key, is probably one thing that they have, but apart from that, if they want to have some other metadata, like you said like which keyword they're talking to or anything which could be useful for them for maintaining the state, then I I think that'll be really useful.

D

But the only question is at some point: do we want to limit how what the size of the metadata can be or can plugins abuse that and just send a really huge metadata and those kind of restrictions? Is that something that need to worry about.

A

I mean, I don't think we would conceptually try to limit this field right. So I'm not sure we conceptually we want to limit the metadata field either like like the kms plug-in, is effectively a root authority on your environment. Right, it's it's it's. It might not be seeing the raw unencrypted data, but if it ever encountered the encrypted data it could decrypt all of it if it wanted to right. So it is a pretty privileged actor within your system.

A

uh So I don't know if I'm concerned about it, storing too much or anything like that. I mean you know we could we could arbitrarily pick a limit and say hey. It needs to be like, under uh some amount of like I think, xcd has a default uh limit of like 1.5 megabytes, yeah yeah, something.

D

Like that yeah, I think it's one point: five or two yeah.

A

Yeah, it's it's relatively small right, so we could arbitrarily enforce that hey! You can't use like more than half a megabyte because.

C

A

Only like, like you're like really starting to like use up the storage space, that's available for the actual data, so we could do that uh yeah, but yeah. I was thinking along the lines of like you know if you had a plug-in that wanted to, even if it didn't need all the state itself, if we just wanted to make it super trivial to like an operator that hey like. I am like this plug-in at this version using this key vault and this configuration I'm just shoving it in as just metadata.

A

So that way like, if you happen to find this encrypted data, you have like a really good idea what it is so that way, if you want, if you have capabilities of getting the keys like you know, everything is there um just just just to basically encourage ease of sort of data, migration or data recovery. Where you have access to the keys and you don't you don't need to like somehow know.

A

Oh, this is like a json structure, that's embedded within the protobuf, and if I decode it this way, this is what the keys are, and um just this is a thought. I'm just trying to think of how how we can like, like the negative of what I'm saying, is if we sort of give plugins a free-for-all. Well, they can kind of do anything and there's no standardization right.

A

So, whereas this one is kind of saying like well, there has to be a key thingy, I mean, but you can still shove anything you want in the key thing is still kind of opaque to us.

A

Just this is more of an early thought. I just wanted to try to see what folks have in mind.

D

Yeah, I think the standardization is a good point like because there is it everything. The data is going to look different for every different provider, uh but also accessing that data is not something that's going to happen, that commonly right, like at least the way we're trying to do it. It's not that someone's going to get that data from fcd, try and decrypt it and look at it. So that's not going to happen so having it kms plug-in.

D

Specifics should still be fine, uh but I do like that idea like just having traditional metadata, and I think that also my this discussion might come in handy when we talk about the latest, because I think we were talking about how the user can see the storage migration is done right and then you mentioned that we will expose that to the user. So if this is something that we expose, then maybe users have access to it.

A

Yeah, um I guess before we move on to that thread christoph. Did you have anything further that you wanted to talk about? I was going to mention. I know we had brought this repo up a long time ago. uh This was my little sort of toy implementation of a kms thing.

A

If you, if you, if you wanted to use it as a easy way to borrow, you know like uh sorry from something if you wanted to borrow the uh the shim layer right that basically bridges the grpc api uh right so like there's, there's a bunch of like abstraction layers already in there that you could just kind of borrow and stick in the your your code, so you don't have to write it all from scratch. If you, if you didn't want to like this already, has the grpc bits implemented and it'd be very.

C

Easy for you to.

A

Strip out the bit that like says, I will execute a command and get a key encryption. Key too, I will do my fancy key thingy.

B

That's really cool this. This was something I wanted to pin you about, but now that you mentioned okay cool. I will do it.

A

Yeah, so it should be pretty pretty easy uh all right um anything else on this topic before we do other stuff.

A

A

A niche did you have something you want to talk about, I know rita has something I just wanted to make sure that we sort of cover.

D

uh No, I think so the proto is here and then I also have some of the changes in the branch. Now I'm trying to make changes in the provider right. So that's why I'm saying I can show a whole e to e demo of that, but also like when we talked about the proxy.

D

I think we did talk about the general idea of what I am implementing, where we are sending the encrypted get back and then using that, and I think all of us agree that's what we want to do so that's good um and then, in terms of the topics here I think we've pretty much covered all of it, and I know we did talk about option to choose encryption protocol in the last cigar meeting. So maybe we can just summarize that and before we go to the next topic,.

A

Okay, so one quick thing so we may want to allow.

A

A

Where is that? Did we record in here somewhere that we wanted to.

A

Right here this is so the.

A

If I remember from that discussion from sigath, it was basically you say: yes tcm, it's fine, I didn't really get any.

C

Last time we said, should we default to gcm right so did what was the conclusion.

A

My understanding was yes, defaulted, gcm and it seems like a good idea to be explicit and include that within the storage, so that you that you were doing that. That's the cipher that you're using uh and we don't need to expose configuration about this right now we can. You can defer that to if it's necessary, um but basically go ahead and build the plumbing to allow such a configuration to be easily added like.

C

Don't hard code, it yeah.

A

Don't hardcode it again right.

C

Again, but don't hard code it but don't expose it where it's easily configurable either.

C

C

That's yeah. I think that was pretty much what we were all talking about last time right as.

B

Of now I just defaulted to asg sim with a key size of 128 bits. um This should be kind of.

A

B

So that's within.

A

The kms plug-in right- yes, yes, so that one is a little different right, that is, uh that is different from the internals of the api server right like I think those are separate.

A

I I I think I would lean towards just do the same thing like you know,.

C

A

It explicit that you are doing what you were doing. Don't need to wire up config about stuff, yet.

A

Finish, were you trying to say something.

D

A

Okay, all right, so that covers what we talked about last time.

A

No, I I know I thought we had something. Oh.

C

A

I might be that might just be one gym, thoughts from places together.

D

I don't think we have it in the dark. I think. Last time we talked about how to do rotation like without the user having to go in, so it probably came off from the hot reload of encryption country, because.

C

D

We were talking about the rotation where the user has to go and update the encryption config and rely on the order today.

D

um So I think that is probably why the latest topic came and then I think I was also talking with rita last week um about the latest stuff and I think there might be a different way to do it. But first we can start off with that discussion and then um I can provide some update on what we were talking about.

A

Oh yeah, so, what's on your mind,.

C

um I mean initial uh thought was. That was a good idea, because then users don't have to think about it. um But for some reason I guess like I was kind of thinking back to like in production. A lot of companies would want to know exactly what happened when and having something as late as is hard to track, um because at least in some of the kms's like when you say I want to create a new latest or whatever. That's just literally whatever was the auto-generated uh key right.

C

So it's it's not something that the user explicitly provide and thinking back on how you do roll back and roll forth like how I don't know how that will impact something like that, because latest is not predictable, but I agree if, if we can track it in scd, that kind of helps, um but again I'm not sure how that can be visible outside of scd. If I'm an operator and I'm I don't have access to that, how would I even know which version I'm like that particular key is on.

A

Yeah all right, so uh I I think so there's a few things there right like so I I suspect that this this idea of using latest versus pinning to a particular key or whatever, I think, ends up being some kind of configuration to the kms plugin like our reference implementation right. So I think we would give you the choice as your configuration right, it's more in my head. What I want to see, if we can make technically possible, is the capability of using something like natives, not enforcing that. You followed that as.

C

A

Pattern- and I I think right now, like I I know we have good proof that you could pin to a particular version and do slow migrations over time.

C

A

You want, I think we have good support for that today and we would have even better support with the hot wheeler.

A

I just want to make it possible to do something like latest, because I know platforms like cube, adm and openshift have a much harder time changing those files on disk and require much more orchestration, and I think they would be better served with config, that's effectively static and then you're using you're, using your key kms, the real remote kms to orchestrate the change.

A

um So I I do agree. We have to come up with some way of making this observable within the platform in some reasonable way, um and I'm not saying I have a good answer for that. Yet I'm still trying to think through that.

C

A

Does that make sense, uh yeah.

C

A

C

I I get exactly why you're asking this and I see the value of it and I think it's super nice for like an operator not having to worry about or keeping track of, uh the key version even right, but but then there's like when things go wrong. I want to go back to it. How would I even know which version I was using or the the plugin was using, and that makes it really difficult. I think.

A

Yeah, I think this is where initially mentioned. You know that you know if you had extra metadata within like if first you have to have extra metadata somewhere for the plug-in to tell you this information and that's you know, step one you're actually storing this somewhere and then the other step would be like okay. I am now storing metadata about what the plugin knew about its state of the world. When it did this encryption for me once you have that, then you can be like all right.

A

Well, I don't want to use that cd ctl to figure out what that was. I want yeah.

C

It's better or not to mention you won't, have access right in a lot of cases. You will not have access to. Yes,.

C

I I- and I think I mean honestly, I I as a user- I will just I will love latest right like it just makes my life easier, um but I also know there will be companies where they're like I can update in my kms following some path per process right and then how I update.

C

How I introduce that to my cluster is a whole different process, whole different sets of users and they should not be influencing each other, whereas this latest thing is it's so beautiful and simple and automated that it assumes by whenever I update here in my kms, I get that for free over here and that's okay, and that might not be the case for a lot of companies.

A

I guess so I I still don't have a good grasp of the cloud kms's and their api capabilities could could a user like say I wanted say. I wanted basically something like latest, but I wanted it in a more controlled manner. So I wanted to say: hey kms plugin. I want you to use what I currently call the right key as your key right and basically when I want a rotation to happen, I create a new key in the kms and once I'm I'm ready for the rotation to occur. I tell my cloud kms hey.

A

This is now the right key like this. Is the label or whatever the other one is no longer the right key.

A

Is that something that would be reasonable, because what I'm saying is we don't have to literally pin it to the idea of the most new key. It could be that there is a way to express.

A

This is the key I want use and then also then express a change of which key is going to be used. So that way you get kind of the best of both worlds right, which is that you can say just use the right key, that's the one I want to use and you just allow a configuration change in cloud kms to explicitly be an opt-in, saying, yeah. I created this key and I did oh and I realized oh, that was the wrong time. I just went and deleted it doesn't matter.

A

Then I go ahead and make a new key and say: okay, now I realize I've done all the right processes got the right key size, whatever I'll go ahead and mark this, as the key like, is that expressable within the apis, or am I just making it.

C

E

C

D

If I understand correctly you're saying when there are multiple versions, you want to mark one as the right key and then want to get that right key. Whenever you call encrypt.

A

Yeah I I want um so there has to be some. Certainly there will be some uh uh delay right like I don't expect the kms plugin to be literally sitting there constantly pulling the remote and saying.

C

A

The thing change and I'm not expecting that what I'm more saying is, can I express a static configuration in my kms plugin config that can be dynamically interpreted within the cloud tms. So that way I can use the cloud kms to orchestrate the change and not the encryption config. That's a file right. Like basically can I use the thing that already has the rest api dynamically instead of trying to like.

A

Basically, you know if you imagine that you're you're a cloud provider- and you want to offer the encryption configuration to your users, because you just decide to do that- you're effectively, going to end up building a rest api on top of the encryption configuration right, it's basically just going to be a shim layer and I'm saying, can we make it so that the shim layer is unnecessary and can be orchestrated basically in the apis?

A

That already exist, because I think that's what we would like to be able to express- and I think that would handle both the safety net- that we want, which is stuff, doesn't just change, because somebody made a new key without like explicitly saying, I want to use the key or a new version of a particular key, but at the same time you can express your intent in that system. Somehow.

C

I mean the closest one I can think of is kind of like image, tags versus image, digest right.

E

D

Thing like you can provide.

C

The digest, if you have it and if you want to be very explicit or you could just say, use the latest and then even though we tell people don't use the latest tag.

C

A

Does the tag concept exist within cloud canvas? Can you tag a particular version so.

C

I think aka azure keyboard, I'm just looking at the docs, I think azure keyboard. If you tell it to you like, I think, there's a way you can just say use the latest.

C

um I could be wrong, but I think so uh so if to your point, if an ap, if the kms api allows for that, then you could. You know in you could just say latest instead of like the the whole hash.

D

Yeah, so if you don't provide anything- and you just request for the key keyword- will just use the latest, whatever is the latest version available today.

E

C

That kms has to provide. That is, is your point right right right.

A

So, but is there a middle ground between, I don't provide a specific marker, so it uses latest or I provide a specific revision of a key yeah.

C

You can understand, is there anything it doesn't work.

A

Right right right, but is there a middle ground between I pinned to a particular version or I just used later? Is there a middle ground that lets me express? I named this particular version ubar and I want to use fubar, and I want to be able to basically change which one is toolbar.

D

Yeah, as far as I know, there is not I mean so these for these particular keys that you create. You can define like tags on it like you can just tag it with some custom fields, saying like uh created by a niche, or something like that.

D

So similarly, here you can say use for right and then the kms plugins would need to query the version that is using this particular tag, so they have to list all the versions and then filter on the ones that has this tag and then just use the one that matches the tag, but there is no native way to do it like this is just with all the decorators that's available. We can do it, but yeah. There is no way to pin the particular version as the latest for now.

A

I think whatever is.

D

A

Fine though right like, if so, it would be, basically it would be.

A

You would be sort of semi abusing the tag api to do this, but I think it's sort of fine, because the idea would be is that you're you're setting up a contract, an api contract between the kms plugin implementation and how you configure things in key vault in this case as a key role right, and I think that would be very valuable because it lets you do what the orchestration that you want right. You can very specifically say.

A

I know this is the version being used because it's literally written as a version being used right now, because it's got the tag and if there's more than one with the tag you can be like you know, I I don't know what the definition of the kms plugin would be. It probably would start airing saying or it might just use the last one it used or whatever I don't know what state it would go into.

A

But you know it could certainly try to do some safe fallback like use the older key like by time temporarily, whichever one is the older one.

C

So you're actually not suggesting using the latest in, uh but rather like a way to identify the latest and have the plugin smart enough to pick up the the latest.

A

I I I want to make it easy to express the concept of this. Is this key at this? uh At some revision is currently the right key right.

C

A

To be able to easily change data within my key vault to change that expression without.

C

A

C

To touch the configuration file, I yeah that would be great.

A

Yeah and I think, that's possible if, if we I mean so, I think we have the right pieces, at least at the kms layers like the remote kms. Now it's I think, a matter of trying to figure out or can we can plumb it all back down to kubernetes in a reasonable way.

C

uh But wouldn't this be very, very plug-in specific right I mean I would imagine this would be something that's part of the uh the the thing that damien's building um but like.

C

Meaning, like my plugin might be, might be using the like the latest. My your plugin might ask for explicit configuration update right.

E

A

I do think this falls into the config of the plug-in, and I think it's okay, if you're, if the remote cams influences how you configure the kms plugin in some way. Right like, I think, that's okay, because if you try to normalize the configuration of all uh of the kms plug-in, you get the like the least common denominator of all remote implementations.

A

Basically- and that's probably not the state, you want right because you basically get the least value out of your keywords like you can't use any special features or anything like that right and that seems to me.

C

But how to interpret the configuration is, I guess, uh is not plugin specific.

C

uh I mean, if I use latest, will it even and would it even work right? The word latest.

A

Yeah I mean it yeah. I think this will be easier to kind of reason and think about once we have a little bit more stuff written down and we can try to play with some of this and be like what would? How would I express my intent of latest versus tagged as right, key or something.

C

And today the configuration is very explicit. Right is key version, that's it, but if we are introducing something new, that's a new field right.

A

Yeah, so it would be a new configuration to the kms plugin itself, but not necessarily to kubernetes right, because it would probably be funneled into that metadata concept. Right like that.

E

C

So what well, what would be yeah.

A

Right because you can imagine that um you got you started, you went from version, let's say you were using latest and you went from version seven to version eight of the key encryption key, the remote one right when you get data back, it would be nice for you to be like. Oh, this was using the same.

A

You know named key encryption key, but it was using an older revision. So I need to ask to decrypt this this blob that I don't understand right now, using that particular revision right, that's kind of what I mean is like there could be value in having this extra metadata, because it's basically a way for the kms plug-in to remember these types of states um that is specific to how they provide you value. Right like I, I I would expect the azure key vault implementation to leverage keyword, specific features to provide a good experience.

A

And, and if your provider, for example, can't do like this nice rotation because it doesn't have like a tagging concept or something and fine, you just have to pin a version and that's all you ever get, and it probably never has to well. You might still want to put stuff in metadata to make it clear what it did, but.

A

Yeah, I mean, I think, that's okay, like we get the best where we can and if your provider doesn't support it, you just get a less good experience, but still like a functional one.

D

Yeah, I mean it'll be great if they can just configure it outside and don't have to restart all the components in the cluster. Like one thing, one other thing that we had talked about was uh maybe just push all of this into the kms plugin. So today we do configure the kms plugin and pin it with a particular version right. But what if the kms plugin implementation can handle multiple versions. So when it's configured, basically, you can say, write is version b1 and then for read.

D

Maybe v2 v3 v4, so it can basically handle these things internally. So, rather than relying on encryption config order, when the api server says hey encrypt this, it knows it's configured with this particular version for encryption, so it always uses that, but for decrypt it can basically go through the list with which it's configured.

D

So in terms of rotation, all we would have to do is restart the kms plugin, which is still if it doesn't come up. We break the api server, but at least we don't have to update the encryption config. We don't have to restart cube api server. So it's only a single component restart. Instead of all these multiple different components- and maybe we also don't have to restart kms plugin like this- would still be kms plugin implementation, but if it's watching on like a config map or something it can dynamically pick up these configurations.

D

So essentially, whatever we are saying about hot reload of encryption config, we can tell kms plug-in implementers that they can do something where they can dynamically pick up. These newer version for reads and then newer version, that's available for the right.

A

Yeah, so certainly a kms plugin could say uh you know I'm going to create this socket at this end point- and I take this file as my config and it itself could keep reloading its own config on this. So now you don't have to restart it at all. You just have to change that one file um so that that gets you closer, which is basically you know, that's the only thing you need to change and the I I think the main thing. That's still there is, um I mean yeah.

A

It reduces the problem, but it's it's still a small. It's still there um yeah. It would be.

C

And also, we have to write to keep that data in that cd.

A

The the yeah, what what you, how you did your.

C

Version used yeah.

A

Yeah and that's where I like, the idea of having at least the key field or a more expressive metadata field, so that way, the kms plug-in has a specific spot where it's like. Yes, this is my contextual data. I need it back to figure out how to actually give you right.

D

A

Because you know you could you could imagine that even if the read configure like like you could imagine, the kms plugin only ever needs to be configured with the right key, because the contextual metadata could always.

C

A

C

Exactly he did and they would just go.

A

Fetch it right and if you can't, then it's like well, it doesn't matter and.

C

Even if you fat finger it's okay like today, it's not okay,.

A

uh Yeah yeah, I.

C

Like that, that's much cleaner.

C

A

I'm trying to write down some notes about this, so I don't forget so so the idea is to express.

C

I, by the way, when I was implementing the initial ak-31, I was going to add the version 2 scd. That was like what I was going to do anyway because to support rotation yeah. But then it got too complicated.

C

It'll be nice if we standardize this in the reference implementation.

A

A

I think the ideal state would be is the reference implementation is just basically broadly adopted and used, because it just it solves a problem in a robust and correct way, and it just gives into reason to reinvent this video totally.

C

No pressure kristoff.

C

But also we're here to help so you're, not by yourself.

B

No definitely so, if, if there's a way we could cooperate, definitely I would invite everyone to do so. Okay, and mostly what I'm doing it's based on on the discussions from the meetings right, so there's just implementation details that I'm trying to figure out.

C

Hey you have access to the doc now. Why does this still.

A

I don't think I refreshed the page so can.

C

You just refresh the page that will that way, I don't have to keep going except.

C

Unless you're logged into someone else.

A

uh This is my vmware account yeah and my account. So that's probably why okay.

C

That makes more sense all right. I got a drop, um but thanks.

A

For the discussion yeah uh all right, I think we can continue this uh next time.

A

See you next week. I still need to investigate. Basically, I think the biggest unknown for us is: could we could we make it at least somewhat observable the current state of all of this from, I guess the kubernetes api? So I still need to sort of check what this all means. I have some like unfinished ideas in my head about how we could make the kms plug-in, maybe like optionally, serve as an aggregated api server so like it.

A

Could it could if it wanted to serve rest apis through the kubernetes api, and it would be optional, like you, wouldn't have to wire these up. If you didn't want them and then maybe that helps it like, basically, you could ask the kms plugin. Please tell me what the state is using your fancy api. That literally exists from you. I don't know if that's a good idea or if it even makes sense, but that was sort of like done thought so far.

D

So at that point basically we would try and decrypt every secret and say which version is being used like using some of the metadata.

A

I don't know like yeah, I haven't thought through it yet, like you know it could be, it could be as simple as like hey if I was to do a write right now. What key would you end up using like that? Would you know that would at least tell you? Okay, though, you've observed some some particular state so far yeah. It's it's still unclear to me. If that actually works, because you have you know multiple instances of this process running so like. Does it make sense or do you do?

A

You need some other kind of api where you can target the instances and ask them specifically, or maybe the instances don't matter at all, and the only thing the api needs to express is what is the current state of ncd?

A

And yes, it's a little unclear to me.

A

I haven't figured out exactly where this would be expressed, because this is sort of like just an artifact of stores. Right like you, you can't ask today hey this object in ncd. Is it stored as json yaml, or you can't ask that right right and it's like we're saying, but I really want to ask that more specifically, I want to ask like some subset of that question and I think okay, let's see what we can do.

D

A

All right cool uh thanks, y'all for hanging out y'all next week.

D