Continuous Delivery Foundation cdCon 2021, 9 Jul 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Spinnaker Security SIG - Jason McIntosh, Armory

Description

Spinnaker Security SIG - Jason McIntosh, Armory

Join Jason McIntosh from Armory for an open discussion around security in Spinnaker and related security issues. Recent CVEs, RBAC, authentication/authorization, and any other security topic!

For more Continuous Delivery Foundation content, check out our blog: https://cd.foundation/blog/

A

Let's go ahead and get started, um threw up a poll, and this is just a simple kind of thing to get started. Sig security covers, you know, sort of security around spinnaker. A lot of what we deal with uh in the sig is just simple things like hey. We get an email, there's a vulnerability disclosed or hey. We've detected some issue on the website. What does that all mean? How do we answer it? The security sig is designed to answer some of those questions in the operational uh aspects of spinnaker from a security perspective.

A

So simple example, we got an email the other day, hey our build system is potentially vulnerable to a domain subdomain takeover, so security sig investigated realized. Oh yeah, it's not actually an issue because one it's an a record, it's not a c name and it actually works on https, not http, so that wasn't actually a vulnerable.

A

We also tend to respond to github issues, so we look at any security issues reported and that can be configuration that can be oh questions around the arbac model. uh Changes to the arbec model, an example of those there's, a discussion lately around whether or not changing the permissions for the ui on who can make infrastructure changes was appropriate or not, and that went through a couple of different discussion points.

A

Some of the other things that we'll talk about is hey ldap uh improvements and workflows and how that performs. So the sig covers a couple of those areas on both questions around security of spinnaker itself, but also security of spinnaker's, tooling and infrastructure.

A

That's the core sort of responsibility for the team and obviously responding to github issues is probably the most time consuming.

A

So that's what the security sig does were, one of obviously even multiple stakes, so I threw out a poll- and this is a reminder of birds of a feather session. So it's a little bit of hey. Let's talk security and let's have a conversation around security, but to have a conversation. I'd like to know what people are looking to know about, so that can be a couple of different topics, including hey our back of spinnaker. It can be, you know, hey how to use uh spinnaker to you know improve your security.

A

What can it do for you, but it can also be hey. You know, how do we respond to those kinds of events, and how do we investigate those issues? So there's a little bit of hey? What can we answer for everybody who's attending so.

A

I think the poll is up so I'll. Let people hopefully join and ask some questions. um I don't want to just sit here and randomly talk about.

A

Security give a minute, and then I will just start picking your topic.

A

B

A

Any questions on security, thus.

A

B

B

A

Now I'm not hearing much anybody have anything they'd like to discuss before I pick a.

A

B

All right: well, we can talk. uh Let's.

A

Go ahead and start on maybe the our back model in spinnaker um and again I said: if anybody has any questions, answers feel free to ask. I will happily do my best to answer on any topic around security. So but let's start off with a simple hello world sort of example,.

A

Spinnaker's securities in the arbec model is really broken around in two key areas. um Your account permissions, which is a targeted aws account or a targeted docker environment, for example, or a kubernetes cluster and spinnaker abstracts that into a spinnaker name for the account like you could call it bob and that talks to ek is cluster x.

A

um When you call spinnaker account bob, you can set up permissions for bob who, in there is allowed to access to, read and or write to bob note that these are not inherited. So if you do write but not read, they could deploy, but they wouldn't be able to tell the status of that deployment. So generally, you want both read and write if you're going to allow right operations uh to give you posture.

A

So what does that actually entail? Spinnaker's permissions model is entirely group based role-based. It does not actually grant permissions to an individual user, so it is entirely based upon a set of groups. Groups can come from saml confirm, ldap spinnaker will automatically. uh You know, use any of the groups that you have it configured for now. You can come in through the assertion. I.E, an oauth claim you can come in through a sample assertion or you can do an external lookup for group information to match a user id based upon the login username.

A

Those external sources are very limited in what they are all that being probably one of the most common uh you can do, ldap to open, ldap, active directory, etc. It's spring security frameworks under the hood, so it works with the uh ldap template uh like uh libraries, the group information once it's there, that is used for how you can assume or how you can assign permissions to your accounts.

A

Now, that's just read writer accounts that does not impact whether a user is allowed to do other operations through the ui that only impacts whether or not they can deploy or list say services in a cluster application. Permissions are the other aspect of spinnaker's rbac model application permissions.

A

Let people control things like hey apps named bob or jane, or you know, foo hey, who can read, write or execute stages and pipelines in that given application, but just because a person can actually execute and trigger a pipeline does not mean that that pipeline can deploy to anything. So you can grant person or a team. I should say permissions all day long, but they wouldn't actually be able to deploy anything.

A

So what does that look like? uh Let me bring up a real, quick demo app here and I'll. Show you a simple example that we use internally. This is pretty much a basic.

B

Let me find my screen share button here.

A

Here's a hello world, app, very basic hello world, app spinnaker when you create a given application, an application is just an abstraction around the name. um All it does is let you set things like hey. What accounts? Could I use for deployments some basic things, but you also note hey, there's some permissions.

A

These permissions come from whatever your auth provider is, and you can configure them pretty. Simply you can see up here. I can set read only read and execute or read, write and execute execute lets you, fire pipelines, read lets you read all of it and write actually lets. You update your pipelines very simple permissions, but again only impacts this application.

A

If you don't define any permissions, you're going to get read, write and execute permissions by default, so, as a general rule, you're always going to want to set permissions.

A

There's this handy little question mark that explains some of these. There now note to use these permissions. You have to have group permissions, turned on and enabled, if you don't you're, not going to get permissions, so you generally want both um drop down. You'll note: these are groups. This is actually coming from a github org as an example.

A

Basic basic permissions, but, as said just because you have permissions here, does not grant you permissions to actually deploy an app that comes from a very different spot. If we look, let's see if this will follow here, uh we document those permissions in spinnaker.io, and if we come in here in your setup, there is set up spinnaker. There is a security and I'll just get a security authorization and authentication.

A

Authentication is just saying that you are who you are. Authorization is about whether or not now that we know who you are you're allowed to do anything. So when we talk about authorization asset saml is there through the saml assertion, we can do an ldap lookup.

A

We also support github teams, google groups, any of these can be sources for group information. You can actually do ldap or sorry oauth groups as well. If the oauth claim includes a role assertion as part of the claim, this page documents here the breakdown- and you can see- I just mentioned- like your application. Permissions are not the same as your account permissions, and I said it's open by default.

A

If you don't define it, anybody can do it. So a general rule you're going to want to set permissions. There is a super admin. You can add to your fiat, which is the permission system that lets them do anything. So you can say, hey group, you know bob is allowed to do anything in spinnaker. Agree will completely bypass your permissions uh requirements. We document here and I'm not going to read through your docs, we document, you know some of the different permissions models and some of the things that we've just talked about.

A

There are a couple things I do want to highlight down here. These are controls around your api, so you can always investigate by right-clicking and you'll, see 403 forbidden in your development console when it talks to the api models. So hey you wanted to access any given application. The api is where the permissions are managed.

A

So your api calls are what controls it. This is very different, though, than pipeline execution.

A

So, let's come back to this here, where we set up a permission for armory, oh and this managed services as an example can read and write.

A

When we have a pipeline, you can configure a pipeline to actually fire an automated trigger and run as a custom permission. I don't have it configured in this environment the way it would normally do. But if we said oh yeah, hey every execute five minutes. You can set this to run as a service account. A service account is not a real user in spinnaker, it is just an abstraction around a set of groups.

A

The set of groups will then be part of this automated trigger and run with those permissions. So service accounts are not actual ldap users or end users, they're, just hey. Here's a set of groups. This pipeline will execute with.

A

Through one of these automated triggers, if you execute a pipeline manually, it will execute with your permissions.

A

Now here's an interesting thing that we'll hear on occasion uh and again, please feel free to speak up and ask questions uh on any of these permissions elevation. So imagine you have a change control process, I.e. You want to be able to say: oh yeah, the development teams can trigger pipelines all day long, but we don't want them to deploy to an account without the account going through a change management approval process.

A

Normally I would prefer not to have those kinds of manual judgments, but you know sometimes you have something like that. Spinnaker will also allow you to do hey fire. These don't set up permissions or set a permissions that are very low level that allows you to fire based on docker triggers, etc.

A

You can actually do in your manual judgment stage.

A

You can propagate your authentication, which basically will change the permissions on follow-on stages to use the permissions of whoever approved the stage. So, for example, I could say propagate permissions and in the rest of the stages iea deploy stage. If I had a deploy, manifest stage would actually use the permissions of whoever clicked the manual judgment. This lets you have one team trigger a pipeline to do like a dev deploy, for example, and then a manual judgment from a change management team to approve it for a production, deploy and use different permissions.

A

There are a couple of key factors for that. If you're going to do that, watch your write permissions on the app because you have a potential where somebody could say: oh yeah, I'm going to change this to deploy something that was unintended just as a caution flag on that.

A

So that's the rough outlines of the pipeline permissions model now you're not going to see your target permissions in here I.e. If I picked an account like one of our internal accounts, I wouldn't see the permissions that that account is set up with, but I would only see accounts that I am authorized to access when I would come into this ui and pick this drop down.

A

There are apis that let you query this and see the permissions on this, but, as a general rule, you're not going to see this in here. This is going to be in your config file. So up in here, you can do it through a couple different places, I'm going to real quick, bring up the documentation that we use operator. This is our armory docks I tend to like going over here.

A

If we look at the accounts provider and our operator documents, you'll know like hey, you have your different providers.

A

Permissions you'll read, write, execute and actually I forgot create was added uh fairly recently, but I don't remember if it's supported for all of them like it is yeah it's uh it should be on all of them.

A

Okay, so, let's take an example of you know: hey your aws permissions. You would define your permissions with read and then underneath you could say here: are the groups it's yaml syntax, if you're using operator and hired reference.

A

In here, when you set up your account, so if we add down here.

A

uh Aws, let me just find it.

B

A

So when you go to add your.

B

Accounts, you can.

B

A

Trying to find the link for where it is, and I am blind there's when you're configuring these you can set the permissions uh for the accounts from a model perspective here. It is write permissions, for example, and this lets you control when you add the account to spinnaker what permissions are set at. That point always always recommend doing one of those if you're using an operator, it's just adding underneath these appropriate ones the both.

A

Now again, the reminder is read and write. You want both, because various stages will fail um because they will be able to get the status of a deployment. If you do not have read permissions as well as write permissions.

A

Now go ahead and pause there. I've gone through some examples. Any questions on the arbec.

A

A

Anybody at all feel free to ask or any other.

B

A

Some things I will bring up there is a new lock up here, create permission where you can actually control a permissions in a little different manner, and we actually document in a pretty good detail the different permissions. You can actually start preventing application creation. Let me find the.

B

A

That's policy engine, that's an armory enterprise. Let me find the permission.

A

Gotta love on the fly here. It is restrict application creation. So one of the things- and I think it's in 220- it's in 227, there's a permissions model. It was in a newer version and I'd have to go back and find the version where you can actually restrict whether or not your users are allowed to create an application.

A

So if you go by default, you can come up here and you can create a brand new application all day long now again, it doesn't give you permissions to actually deploy anything it just lets. You create an application and put pipelines under that application.

A

You can now, though, also restrict who can actually create applications. So there's a custom figure configuration for fiat if you've ever dealt with configuring spinner for your profiles, which are basically custom overrides for any of your default settings.

A

These are getting to the details of you have to be able to configure it directly, but you can control, for example, uh pretty low level things like oh yeah, here's tomcat properties or spring properties by using these one of the things you can do, though, is with this low-level config control, who can create on a given wildcard it's prefix, based, for example, any app named app test. You can use this to also apply to star to apply all and that would, for example, prevent people from being able to create applications.

A

There's also a specific flag on that.

A

Highly recommend, reading and understanding this permissions model, unless you want to let everybody in your company do everything that.

B

Makes work for.

A

You know your company, I don't want to tell you how to run that, but um spinnaker grants a great deal of flexibility on this application model.

A

The caution is, as always, make sure you understand what you're doing, because you can either expose more than you intends.

B

A

Hey lock it down, so you can't even use it. So let me make sure we link this into the chat today. um Your read permissions and thank you for asking. You can always be cautious about it. Your read command gives you access to an account, so you can read all of your infrastructure account.

A

I don't know that this particular. uh If I pick up an actions demo or one of our other demos.

A

I don't know if I have one that has any infrastructure connected to it right now. This is an older environment.

A

Here let me actually just bring up the spinnaker app itself. I believe I've got some clusters here. Okay, so when you have read permissions to an account, you can pull, like instance, ids now, in this particular case, there's not a lot of data here. That may be relevant, but you are potentially exposing information that you may not intend so, as a rule be cautious on it and that's really an organization specific question, but it is something that you do want to be cognizant of.

A

Is this clusters tab when you get read access grants access to this account, for example, this is like our dev account, um so I can get into any of the details, including config maps or other uh things. Now I wouldn't be able to. Unless I had right, do modifications to that account.

A

Any attempt would actually fail, but read permissions can't expose information. You may not intend.

A

And let me link over here also the authentication model.

A

For that, for that docs and I will link, I believe, yeah link the create and then the example commands when you go to set up any of these using hired.

A

You can set the read and write permissions through that.

A

Other questions anything else, people would like to.

B

A

Answer one of my other questions that I tossed in the poll secrets spinnaker oh pipeline permissions, there's a good one. Okay, let's talk about this. Let me document it over here, there's two types of permissions in spinnaker for what we call uh automated triggers and if we come back here, if we remember, I had my hello world.

A

Now this cluster was set up with service accounts and there's two different systems on this. So if we come into your authorization, you'll see service accounts and pipeline permissions. Both are attempting to solve the same problem.

A

How do you control when a pipeline executes what permissions the pipeline executes with from an automated trigger, whether that's cron somebody did a git pumish, there's a new home chart or a jenkins job? Any of these, your pipeline permissions and your service accounts are controlling this. Now a service account is just, as I said earlier, a wrapper around a set of groups, the trigger trick with a service account. You have to pre-create them directly, calling the internal.50, url and you'll.

A

Note here: hey you post to this endpoint with the name of your service account, and you say: hey that service account has these roles associated with it, and then you would be able to come in and pick that service account here. Is this run as user block.

A

Now there are a couple of gotchas you're not going to be able to assign any permissions unless you are a member of all of the permissions associated with that account.

A

Otherwise you get into privilege escalation, sort of situations, so that's service accounts pipeline permissions was an attempt to make this easier.

A

It wasn't even actually visible in the docs until maybe a year and a half two years ago I remember working on the docs. It's like. Oh, this is awesome.

A

Here's what this does pipeline permissions automatically will create a service account for you when you save a pipeline. What, instead, will do is it'll change. This run as user to be the roles that you you again, you have to be a member of those roles to be able to assign it that you want to run your pipeline with now. What does that? Do? I said it when you save it, it will actually create a dedicated service account for your pipeline associated with it using a uid, here's, the good and bad to that.

A

The good to that is it's only for your pipeline. It's only going to be used by your pipeline. The bad to it is it's only for your pipeline.

A

You can't reuse that same permissions pipeline permissions, anyplace else now, there's some stuff in, I think two or 125 126, where you can actually do some and there's a couple, pull requests talking about um enabling reuse of some of these.

A

So you don't have a duplicated set of permissions for multiple different pipelines, but, as a general rule um up until those releases, you would have a situation where imagine you had 10 000 different pipelines all with the same permissions, you would end up with 10 000 service accounts that abstract each one of those that includes a whole lot of lookups. That also impacts your synchronization permissions, so, but it makes it much simpler to configure what your pipeline will run.

A

As with the understanding that you're still not going to know on a given target account, I mean when you pick a deploy, manifest, for example, or a deploy aws.

A

For example, you know if I were targeting aws you're, not going to know the permissions uh of the account you're deploying to we don't expose that, although I believe again, the api does expose it.

A

um So you'd have to know to look at the apa to know what permissions the account is set up with so, but I find this to be pretty useful. I like it, but it is two different models.

A

The advantage to service accounts is typically admins will do it and set up a set of pre-baked ones, but you're not always going to be able to assign it, because you may not have the permissions to do so, um whereas with pipeline permissions, anybody can create these and then control and restrict permissions on their pipelines, good and bad to both it's really tied to. However, you want to manage these there's an automated migration. If you end up going with the pipeline permissions, it's a newer feature.

A

There is a couple of caution flags I hear if you have um yeah I'll, just let you read through this doc versus going through it, because there is some impact of hey of the roles and where okay, you have to be a member of all the groups afterwards to be able to edit a pipeline. So if your roles change, you may have to reset some of the permissions on a pipeline on some of the stuff.

A

So yeah read through some of the impacts on this there's a couple, interesting ones and how it.

A

Operates question though it's a nice new feature, not everybody wants to use that feature, though,.

A

All right other questions, anybody else.

A

uh Visibility of jenkins cia servers to specific users and groups, some of the stuff supports, like your artifacts, do not have, and I'm pulling from memory here the ability to control your permissions.

A

So when you go to your artifacts accounts up in here, where you do your config artifact like where you add a bit bucket reference, you're not going to be able to restrict username or read groups on an artifact, you can only do it on like if you do the docker registry, when you add a provider for that, you can set read permissions.

A

Well, that's dc os!

A

If we look at one of the other config providers, you can do it now. You can also do it on.

A

uh If I remember at your ci and if you look here at like your jenkins, you can set your read and write permissions on your ci. So.

A

Any of your ci servers it it's your artifact triggers that you can't control the permissions on. So there is an interesting gotcha and a caution of hey. If you set up a you know, artifact account responding to example, a private home repo, be aware that anybody in your spinnaker can then trigger off of that prime or that helm repo and pull your helm.

A

B

A

Not too much of an issue, but that one is one: that's not really advertised as a uh yeah, hey, here's the risk for it.

A

Other questions people have on either permissions or back security of spinnaker in general.

A

I can talk to a couple other topics here in a minute.

B

A

Not hearing much other questions so I'll talk to a couple other topics, a couple of things from the ui perspective, when you go to set up your spinnaker ui, you can see up here. For example, I have a smaller armory I o domain. Your domain is key.

A

When you go to setup spinnaker, you have a ui and an api. Your ui.

A

Security uh here it is your api and ui ui is deck. It's static, javascript files matter of fact, you can go to any spinnaker and you can look at it. It's just static files, it's relevant because that can impact like redirect urls and certain other things, and what features are enabled you can go to any spinnaker and look at those. It does not impact and store any secret data in and of itself.

A

Your api is what controls that so your workflow is hey. You load the javascript, the javascript doesn't equal to your api layer and determines whether you're logged in and then to do a redirect to your identity provider.

A

The reason I mention that you're going to get impacted by that, if you do separate domains for api and ui so do be aware of coors impacts on this, particularly be aware that different browsers handle things differently, you're going to want to start enabling https on those through a load, balancer layer or similar.

A

But your security block is what controls these settings, including turning on your rbac models, I.e authn, which is your authentication and auth z, which is your authorization whenever you turn those on you're also going to want to make sure your api and ui are also secure.

B

A

Spinnaker cdes, okay, there's been a couple of cves. uh Let's see.

A

I'm trying to see if I have any of them: bookmarked nope, just some recent cds- I was doing um currently most of the cds- are published in netflix security bulletins in the advisories here, we're looking at potentially moving that over into the spinnaker repo, but in the past. This is where we have uh published these.

A

um I'm trying to think if this is nope. I'd have.

B

To go back and find an example: that's titus.

A

Here we go. uh This is an example where hey there was a server-side request, forgery on pipeline templates. This was you can see here on orca versions. This is published in september of this last year.

A

um I distinctly remember this one because it was tied to some of the ammo processing on templates, so security sig will typically announce these through the announcements uh distribution list. The way these will actually happen is we'll get a report of a vulnerability uh we will respond to internally. On this matter of fact, there is a policy.

A

If you go to spinnaker issues over here um in here, we actually have a security policy. You're welcome to read through that documents. Hey what happens. How do we go through it? Ideally, the more details that are provided, the better that we can do.

A

I do have to. What do you expect? Hey, we give credits, it's an open source. We don't really pay anybody for this. We do issue thanks and we've done that a couple times. um You know when we're allowed to give credit to the reporter.

A

So what does this happen? So the first thing is: send it to security at spinnaker. Io security sig is on this and then there's some internal communications that that triggers uh we end up going through it. Oh is it valid? Yes, oh okay, great. We need to figure out what to do and here's the next steps which, as you can see, is evaluating the products and we have a process here that goes through hey what is involved.

A

uh How do we rank things? How do we track it, etc? um Internally, as so, we track and do some communications until we fix and then eventually we announce it. So um that's where you will see that show up and currently again the security bulletins here, um we'll also announce it in the security sig channel and we'll do it. You know once the release has got it fixed into the uh release notes, so this particular cve, let's see if we've got the link to the cde, it's uh on this advisory.

A

I thought we created a cve4, but I'd have to go back and look.

A

Yeah, I don't know that this is there's. Actually, I think, a cv mitra for it as well um on this particular one, um and we announced it and released it. I said it was. We did some investigation, we discovered it was actually vulnerable. We were able to replicate it and which one we had teams work through identifying and mitigating a fix.

A

If we look at this particular one.

A

I don't even believe they mentioned it in here, but there is a render uh I'll have to find the exact pull request that fixes it obviously there's a lot of changes that go into some of these different uh releases and actually, I think, that's I'm not even sure, that's the right tag. Oh that's yeah, anything less than that uh this particular one. It was, I said, a yellow person on the.

A

If this is the one I'm thinking it is yeah template resolution functionality where basically would try to parse the template and as part of the parser would allow to actually use unsafe calls I.e, load, a file.read ability.

A

So that was an interesting one to replicate fix. It was just changing one of the internal calls to use a safe constructor versus a non-safe constructor.

A

You know, through the expression person.

A

Did I answer your question there on spinnaker cves? Did you have anything particular on cvs? You wanted to.

B

Ask I'm giving that to you, respond.

A

So yeah, that's one of the things the sig security does is help coordinate efforts on those kinds of incidents, investigate them, troubleshoot them and assist with it so yeah it's usually. These aren't made public until they fix this out, at which point we will announce. Hey here's where the fix is. Please go update as soon as humanly possible.

A

And if anybody wants to join, you know we have our regular sig security sessions. There's not a huge amount of topics. Lately, there's a couple on some of the permissions model. We've been discussing some changes there.

A

A lot of what we do is more coordination of some of the investigation and some initial triage, and then we work with some of the various teams to actually get the fixes, depending on what the problem is and where um I've spent a fair bit of time with the docs myself um and answering questions versus actually requiring fixes, um although occasionally there are some areas that I'll step into more but a lot of times, some of the other teams will end up owning the fix versus the security, sig, actually fixing the problems so but we're sort of the front lines of hey yeah.

A

Okay, this is a real issue or no that's not an issue. That's just a you know, misunderstanding.

B

A

That's cdes vulnerabilities. um That's some of the rbac model.

A

There are some nice things without our back model. For example, you don't have to grant your users direct access to aws or kubernetes. You can use manipulate mask access. Sometimes that can be good. Sometimes that can be bad.

A

That's one of the advantages, though, to using a centralized tool like spinnaker, when you have lots of different platforms, but keep in mind. Spinnaker doesn't control the rbac in a platform.

A

So, for example, hey there's nothing preventing somebody from deploying a manifest that contains a service account as long as spinnaker has permissions to create that service account. So you do have the potential where spinnaker can allow a user to create a back door, just a caution flag again on hey when you go and enable an account in spinnaker you're, creating a lot of access to it so be aware of what you're granting.

B

A

Let's see some other good questions, anybody else have anything else, they'd like to ask or hear about.

A

Look at my time here: we've got about 15 minutes, left um secrets. Spinnaker does not normally recommend doing secrets inside spinnaker. You know generally that's better for side cars or your environment injection than it is to actually use spinnaker to do it. We don't recommend it um just as a principle having secrets as part of your pipelines or as part of anything you bake. Don't do it. Please don't embed secrets in your code and I consider docker images to be code from that perspective.

A

So the short answer for that one is yeah, don't use it um with that said, you can use vinegar to deploy stuff and include, like hey, go load this environment from a target account instead of uh you know trying to do it. Spinnaker is very intentionally not doing that. Now there are some things that you do want to be aware of from the secret perspective.

A

So hey when you go for automated triggers, do be aware, for example, hey if you're doing pub, sub or say web hook, you're going to want to add payload constraints. So imagine I set up a web hook and I said: hey go. Allow bob to configure some of these and then allow a trigger webhook bob to fire. This pipeline well web hooks as a general rule. Don't have authentication they're open to anybody. Can you can execute it?

A

Which means, if you have a web hook open to the world, and you allow that web hook to trigger a deployment you're going to get a deployment.

A

You're going to want to add some sort of constraint or secret to enforce that your fire or your trigger is actually what you want to happen, particularly if you're using any part of your trigger as part of your deployment.

A

So what can you do? Well, from a permission, security perspective, each of your triggers supports a slightly different functionality jenkins because of it being more of a pull model. Well, you don't have as much of an issue because spinnaker spinnaker's talking to your jenkins and pulling data from dragons to trigger.

A

As a result, hey you're trusting the remote jenkins to operate correctly.

A

Web hooks, don't have the same thing, but what you can do with web hooks is, you can add payload constraints, so you can say. Field x in payload must be some arbitrary string. You know put something in here now: one caution: this value is not encrypted, which means anybody who has access to your pipeline can come. Read this it's a constraint, there's a certain level of hey.

A

You need to make sure your app has the right permissions, because anybody can come in here and see and modify this, but this is a way you can prevent web hooks from automatically firing unless the web hooks contain some sort of secret identifier. Each of the different triggers supports a slightly different thing: hey you have a docker. You can control, which images you fire on if you're using get.

A

It actually supports get secrets as well, when you do your artifact constraints,.

A

Let's see some of the other ones, you can do.

B

A

Let's see quan course job hey. This is again one of those it's an integration, so it controls it.

A

I don't know if I have all of the configurations here, but each of them is configured a little differently. So when you set up your artifact support.

B

A

Let me find my github secret, like you, can define a sitting here. Yeah you can set up a github web hook. You can have a secret fields that you can configure, and this adds validation that this web hook happened. The way you thought it.

B

Was all right? We've got 10 minutes left I've gone through a couple, different topics. There.

A

Anybody else have other questions things they'd like to ask know about security, our back authentication, authorization.

A

B

Give it another minute.

A

All right, I am rolls for authenticating with eks clusters. Absolutely when you configure spinnaker spinnaker is going to operate within the permissions of the cloud provider you're running it in so, for example, if you're deploying it to eks, hopefully your worker nodes or if you have a newer one using the oidc mapping, stuff it'll, actually map a service account to an im role. Spinner code will operate with that.

A

I am role, so that means you can use eks role, assumption with eks aws off config maps to access remote clusters, because spinnaker itself, when it talks to kubernetes, is just calling kubectl commands under the hood. So when you go through and you configure your kubernetes manifest and you set up, you know your eks config. This contains a coupe config file.

A

Your config file uh cloud driver will have aws im authenticator as well as the awscli. I actually recommend the aws cli for token acquisition over the aws im authenticator.

A

There are certain advantages to it, um so do recommend using that over uh I am authenticator, uh but yes, it will actually use full role assumption um because literally it's just coupe ctl calls under the hood and that supports all the sdks on there.

A

A

Did that answer your question? We're not even going to try and pronounce the names here. Sorry.

B

A

Okay base container images for spinner alpine busy boxer, updated uh cves are one of those interesting challenges from those. Yes, like the base. Images is going to have a library that's out of date, but that doesn't necessarily mean spinnaker itself is vulnerable to those, for example. I think I scanned the other day and I saw like image magic or something I'm like that's not even directly accessible.

A

It's just included on the base container, so general rule is those are updated whenever people get to them in cork secrets or in quark, there is a spinnaker dependencies project.

A

This defines transitive library dependencies, for example, what version of aws or bouncy castle or groovy for all the different spinnaker services to use. So this is probably the most common place that gets a little out of date and does need to be updating and you'll. Note here, like occasionally, we've done hey, we need a version bump, because we've detected this cve, the docker images themselves are updated per service. So if we look at spinnaker gate, for example, you will have three different docker files. We look at the docker file. Slim it's on alpine 311.

A

We probably need to bump this. It doesn't happen as often as we probably should. Unfortunately, I would encourage pull requests and testing the more we improve this, the more contributions we get, the better on that so yeah. Unfortunately, some of these are not updated as often as it would be ideal.

A

Cds in these images, though, are not tracked. Currently, the security sig has been talking about security scanning. Some of these. It has not been implemented any place at this point. We're actually just talking about using one of the off-the-shelf, since these are all open source, we can actually use github's native security scanning for some of this stuff.

A

So, unfortunately the short answer is yeah. It's not actually being tracked, monitored or updated automatically, because a lot of times these don't actually trigger vulnerabilities in spinnaker themselves, it's just they're out of date, and they need to keep up to date and they're using libraries that have vulnerabilities in them. Potentially um that's one of those balance. Acts between libraries with cds versus the use of the libraries, so um keeping up to date on any of that stuff is a very challenging challenging process. So um other questions, predefined web hook, stages that require base64 encoded data.

A

And matthew, please let me know if I answered your question on the cves and, I said: do recommend pull requests on both of these um this one, the gotcha with updating some of these libraries by the way, there's so much interdependencies on some of these, particularly the spring in some of the ok http libraries that you have to be really sensitive like I've tested a couple of spring security changes that just did not work for a lot of services, because, even though spring says they're complete or compliant, it's yeah, they're, not always compliant with each other, I.e.

A

I've seen oh yeah, I bumped spring security from 5.2.3.5.2.4, not that those are real numbers and things broke, even though it was supposed to be a patch version bump. So armory um just have to throw a little thing there. That's one of the things we struggle with on a fairly regular basis is trying to manage some of those versions and keep those up to date um and yeah uh netflix has their own sort of extensions on top of it as well.

A

So yeah it's a challenge um and trying to get those to compile and work is an equally hard challenge um back to the other qa predefined web hook stages that base64 coded secrets.

A

So I said I strongly encourage you not to use secrets uh in that. I don't know that. I have a good answer for you mate on that that I can discuss in the three minutes that I think we get left on the session.

A

um It's probably a topic, that'd be a good blog post for the future, so yeah. um I think I'll delay that one for now, but don't as much as possible use secrets and instead use environment, injected or side cars to reference secrets through a secret provider. So for an example, you would want to use awk secrets manager, vaults, one of the you know various platform secret injection libraries uh to reference, a secret versus having it as part of your environment, very powerful tools.

A

I always recommend that if you have to inject a secret for whatever reason, I would stop back and say: okay, is there something we're really doing correct there with that said, spinnaker is not always the best on how to handle secrets and that's a really challenging topic. So.

A

Yeah, that's the general rule is, if you can have it as a separate system that pulls secrets through an environment and uses like hey automatic secret rotation. Vault's got a pretty good sidecar implementation that I really like um and you can use. I am in kate's authentication with that, for example, and that lets you dynamically inject secrets versus embedding them into your platform.

A

B

A

Embed secrets so.

A

I think I've got one minute left, but I think that's about it on that.

B

Security, it's an ongoing thing. uh Pay attention to it, understand, there's a lot.

A

Of flexibility in the platform and anytime, you have that much flexibility. You can also shoot yourself in the foot really easily. So you know, depending on your environment, depending on your regulations, it's definitely something to pay a significant amount of attention to and understand the good and the bad. A lot of options on this always encourage enable it and please come to the seg, welcome assistance, triaging, pull requests, etc.

A

There's some very interesting ones going on fiat from a performance perspective, and I also got to thank the rest of the community for their support on that. With that said,.

B

Thank you all for coming.

A

And hope this was useful for everybody.