GitLab Manage: Access Group, 4 Nov 2021

Previous Meeting

⏯

youtube image

►

From YouTube: Manage::Access: Service Accounts: Prototype walk thru with AppSec

Description

See related epic for context. https://gitlab.com/groups/gitlab-org/-/epics/6777

A

And if you can open that link and share your screen and we'll go ahead and go through this process and I'll talk a little bit about it, uh what we're trying to do is um create service accounts or bot accounts and I'd like to get your um opinion on the process itself and then any sort of security concerns you think might be exposed here or need to be addressed uh so first, the idea is that this screen would exist at every level, so at the group level and of the projects and instance, because you want to be able to create a service account at the level you want access to with inheritance, um all the way to items underneath it so from here go and create the service account button.

A

And click on the name and the avatar and the axis roll.

A

And just pick one doesn't matter, uh and so now, if you select one of the three tokens there, can you just give me some feedback on when you click on them? Do you think there could be any change or, if there's a concern here, if we create it in this manner,.

B

So quick comment here so I think expiration.

B

Even if I don't choose anything here, I think we have to enforce uh like a default exploration time. I think so. That's.

A

B

That's a common thing that I'm noticing for all kinds of tokens. We don't enforce any exploration.

B

So what happens is that you know someone creates some high privilege token and say they leave the company or you know they move to a different project or something they might still end up, having very high to list access, and uh currently we have a issue of auditability also for admins, so they don't have like a one place to see what are all the tokens that were created by whom and all that uh thing.

B

So, if you're yeah, since, if this is a new feature, that you're working on it'll be nice to enforce a default, uh I mean folks can set it uh as, as you know, based on their requirements. But uh I think we have to enforce uh one by default.

A

Okay, so that's really important. I wanted to get some feedback on that, because right now, currently that field's optional uh and you say it would make sense to have an enforcement date. Do you would there be a reason or need to have uh an indefinite uh expiration, or um is that kind of just like a luxury that we shouldn't be doing.

B

Yeah, I again, uh I don't know how deploy tokens and deploy tokens are used. I.

A

Mean in deploy.

B

Tokens I mean used for deployment right. I think even those you can have them. I guess valid for a long time, but uh again uh there you go for tokens that tend to live longer, it's good to have a rotation policy in place.

B

So so, if say, deploy tokens uh since we are assuming you use these to deploy stuff, uh if it's not good, if they expire too often so, it'll be an inconvenience so for tokens like these it'll be nice to automatically uh rotate them say once in a year once every six months uh but access token. uh I think it should be as short as possible, like uh like one hour. Usually is the uh industry standard uh scim token? I am not sure what it's used for so yeah.

B

I guess, depending on the context you can, you should either enforce uh like a shorter as short as possible, expiration time. uh If it's something like deploy token uh expiration makes sense, but rotation, I guess rotating it periodically, makes sense automatically. So I guess I'm not relying on the advance to do it again. That'll be at least if we can enforce the rotation automatically that'll be ideal.

B

uh If we can't enforce it automatically at least alerting the admins that you know, this token has been created uh more than a year ago, you might want to rotate it as a security best practice.

A

Yeah, that's interesting, so an automatic rotation makes sense to me. We don't necessarily have some sort of system in place to do that, but doesn't mean we can't create one.

A

If you were to create some sort of rotation so say, for example, you select deploy token, it says, deploy token will be rotated automatically in six months or a year or whatever would there be? um You said: we'd have to have a notification system if there was an automatic method of doing it that automatically generated the token.

A

I would wonder if there'd be a security problem where the notification came into place. Saying you've now created this new token. Please view it here, someplace versus having just a notice by saying hey, it's time to replace a token. It's been six months or whatever and then have the admin be notified and go manually, create the process.

B

Yeah I mean uh with the automatic rotation. I think, if the, if the deep, I think the good thing at the automatic rotation is it's transparent and seamless, I guess it's seamless, not transparent, seamless with respect to an admin, but if the admin actually needs to know what the deploy token is, uh I don't think we so say, I'm creating a token. Now I get to see the token only when I create it and once that is done, I'm not able to. I won't be able to see the deploy token right, yeah so yeah.

B

So if I guess yeah depends on the requirement like if the admin does need to know what the deploy token is, then I guess the automatic rotation um yeah might not make sense, because they won't know what the new token is right. So in that case, I guess we have to alert the admin and there could be the potential of like down.

B

Time like we might have to support the old token and the new token uh for uh like there should be some overlap time uh until the whole token is also valid, and the new talker is also valid. But once that overlap time is done, then the new token takes over and my.

A

Implementation is, I think,.

B

You have to keep that in mind.

A

Yeah, if you were to click on the deploy token and then uh just pick any of the scope and then generate token, this is kind of what I had in mind where it would say you have to copy it. We currently have this process, and this is something that I don't know if it's the best solution or not, but it's since it's what we currently use.

A

I thought okay, we'll just use that for now and see if there's a better way of doing that, so this kind of brings into the discussion which you're just speaking of how you would repopulate or regenerate that code. This would be the one my first thought on how to do it. I'd be curious to see if there was I'd wanted to learn more, if there's a possibility to automatically regenerate it um by having it.

A

I don't know if having it visible somewhere to get this copy and paste somewhere would be a safety concern versus having to do it automatically right or not.

B

Automatically I mean like say once: I've generated the deploy token and once I've configured it in my groups of projects. Is there really a need for me to know uh what the deploy token is at that point like if it my projects are working uh as intended, and if the rotation is happening seamlessly um you know.

B

If I really need a deploy token, I can always create a new one right. But to me, I think, uh from the developer's point of view and from the implementation point of view, if you can go the automatic rotation route, it'll be much more cleaner. That way, instead of pinging the admin and asking them to rotate.

A

Okay um right now, currently we are have for access token. We don't generate a token. We generate a user and then that user is the. I guess, bot um it's not really a bot, um it's actually taking a seat, and so this is part of the reason we're trying to change this process.

A

There's a related issue where we are looking at unifying the tokens so, instead of having a deployed token or an access token, it's just the same token in the database uh or the same table, but you can then define the scope and that would differentiate what sort of token it is without having to pick the the type of token.

A

um Do you think there would be any sort of consider security concerns with that.

B

Yeah, I guess, if you're just relying on one token, so I guess all the permissions get uh populated in that. Just one token like say um once I select the deploy token the scopes change accordingly right right once I select the sci and token, let's say access token, uh the scopes are different, so yeah, I guess there is that there is a risk of uh you know, making that uh one token to rule them all kind of.

A

B

A

And uh for this go ahead and click on deploy again and click on a scope and then generate and then at the bottom, click that save button.

A

Now so the idea is that you would have the list of the tokens here and you'd be able to just delete them or create a new one. You wouldn't be able to necessarily change it.

A

I I don't know if there's a necessity for changing or updating the token, rather than just creating a new one, that's current that we have right now is that you can only just delete it. Is there any reason why you want to look at or modify the token here, or does that present security concerns.

B

uh I guess so. I guess the revocation is one place where like say, the token gets leaked somewhere, um uh it'll be nice to rework it. You know if you delete it, uh I'm assuming it goes away from the database and if someone tries to access the api using that deleted token, it wouldn't go through.

A

Yeah the id would be would work the same as the revoke.

B

Okay, okay, yeah, then yeah. If you have the delete option, I guess yeah that makes sense.

A

And one thing that um cynthia had mentioned is that it would be interesting to see um bots from where they what objects they filter down to and where they inherited from so in this iteration. I only have it where it filters down to or where it came from. um That is, it exists here, but the ideas she presented was that, as an admitted want to see both vertically and laterally, because that way, if there's an instance where I don't know where the bot exists, I don't want to go backtrack and dig around.

A

It would make sense to see it in that way. I think that has value I'm curious to know what you think. Does that make sense, or is there a problem with that.

B

Yeah for sure, I think we have a whole issue discussing the groups, sharing complexity right so yeah. So I think we don't want to add that complexity. On top of this feature so yeah any any any approach we make, we take to make it easier for an admin to see, I guess all the complex, all the access or alien inherited permissions and all that that a bot or service account would inherit or would give to some subsequent. You know, projects or groups in like in this location. I think I'll be uh that'll, be great.

A

Okay, okay, uh did you have any other feedback.

B

Yeah, I think uh this uh this issue, I captured a few right so subscribe, so I think you.

A

B

This um and yeah, I think, enforcing uh at list privilege wherever we can. So if we go back to we go back to this I mean we could have this in the documentation somewhere like. If someone is choosing all the scopes, maybe we should have on android saying that hey uh you know, do you really need all those scopes or try to use privilege uh wherever you can so.

A

B

Can have that in the documentation but maybe like, if someone's selecting all the scopes for a given token, maybe you can have like an alert uh saying that hey you know, do you know what you're doing yeah.

A

This is not recommended, it causes a security concern. That's interesting.

B

Yeah and no configuration.

B

Yeah, I think this is more uh visibility. I think uh into permissions that uh token created by a service account- and I guess, ability to add fine, grained access through scopes. So I think we already have scopes here. So there is a way to give more fine grain access and also, once the token is generated uh you can show, like all the permissions uh like say this spot user or the service account uh like all the tokens that they created and the access that they have. In this main view,.

A

Yeah that makes sense.

B

And I think one other comment I had was working on looking at this was: let's go back.

B

uh The token size um uh looks a little smaller, maybe like again, I don't know the entropy of this string, but yeah. If you can make it a little larger. I don't know what the industry standard is, but it's definitely looks uh the industry standard tokens like say odd, zero whenever they issue a token like this, uh I don't remember the length, but it seems larger than this one. So maybe you want to be in line with the industry standard. On the token size.

A

Yeah, that makes sense, I think, in this case I just threw some text in there as a prototype, yeah cool.

B

Making sure so I think this is covered right, like at every level. uh Atoms can see.

B

And yeah, I think this is the second point here uh is related to like a very common vulnerability that we see with respect to access is insecure, direct object, reference kind of issues where so say. Like a token.

A

Access token was the other one that had okay.

B

A

B

Yeah so say this is group, one and uh say security bought service account created a service, a token associated with group, one and group one would usually have like an id in the back end. Right so say, group one has an id one in the back end, the vulnerability that we see is uh using the access token that's tied to group one which has a id one in the back end.

B

uh What I can do is, uh I can use that token to then change settings in group 2 which might have an id2, but this service account may not have direct access to that group. But what we are checking is like in the backend code. Usually is: is the access token valid? If it's valid, we are not checking if the access token is valid on the group that it is trying to access so more like dynamic.

B

uh You know, runtime checks, uh making sure that yeah check to see if the access token is valid but check to also see if the access token is valid on the group that the token is trying to access. So it's a common issue. I think we are saying so something to keep in mind um so that this cro cross, a group cross project access is not present.

A

Is that something that currently exists, because um I'm saying or I'm looking at the settings that we currently have? If you go back and look at the scopes, should those scopes be reduced or is there something that we're not showing here.

B

So I think it's not a scope issue, so whenever there's a it's more like a back end code, that needs to check where usually they check the validity of the token and the next step should be. This token is trying to access group 2, but based on my information that I have this token belongs to group 1.

B

So they shouldn't be. They shouldn't be able to access group two, but that check the second check where you're, comparing uh where this token was issued, versus what this token is trying to access, even if it has the right scopes and everything that check needs to be present.

B

So that say, this token might have say you know this access token might have uh api access to group. One like it can access. It can read, write, delete everything in group one, but that doesn't mean that it should be able to read right, delete everything on group 2, because the token was issued on group 1. so that runtime check and the back end is very inconsistent.

B

Like uh is a common issue that we see in ioc through hackerone, so so the this broader class of class of issues is called like insecure direct object, reference uh vulnerability, so so something to keep that uh something to keep in mind. So.

A

This would be a separate issue for the backend team to address as part of this process. We move forward.

B

Yep, okay, yeah.

A

Wherever we're.

B

Creating uh you know, new accounts or new tokens um back end also keep needs to keep that in mind, uh saying that uh if I'm getting a token from somewhere, I have to check the validity, of course check the expiration um and also, if the token is valid, to do what it is trying to do. So. That's also the important part.

A

B

I guess a good analogy would be you go check into a hotel, they give you a key card, so the key card should only give you access to your room.

B

So, but what what's happening right now is uh I'm able to access any room in the hotel, because what the hotels uh like the keycard reader is doing is. It is checking to see the if the keycard is valid, but not checking to see if the keycard is valid to get into this room or that room. So.

A

B

uh And insecure storage of service account token, so I guess on the back end, because we're just showing this token once and uh we're not showing this token at all in the front end right. So in the back end, we have to make sure this is using secure. You know encryption, algorithms and such and uh yeah encrypting. The token address no um lack of visibility.

B

Today uh I think we've covered this yeah.

A

That'd be the overview screen.

B

I think we've covered this also enforcing expiration and rotation policy and uh yeah. I think the last one is more like a additional friends for us to look at how gcp has done implemented. This feature.

A

B

They have written this nice threat model and all the I guess, the best practices that you have to follow when generating service accounts. um So I think they talk about. You know privilege escalation, not following this police privilege and spoofing yeah. I think stuff we discussed, but uh maybe, if you can go into specifics on uh something that maybe like while you're implementing, if you can use this as a reference to you know not do stuff like this, I think I'll be yeah.

A

B

Very comprehensive.

A

Yeah on this issue, the service request or the review of service account issue that you created uh okay. Well, I was double checking that I had it assigned to me, so I can keep it in my records cool thanks.

A

um Anything else.

B

um That's it for now I'll, take a closer look again at this prototype and maybe if something comes up helping you on slack, okay,.

A

That's great, I know one thing that we're trying to do, like I said, is we're trying to merge the tokens.

A

So I don't know if that's going to present any new problem apart from the scope, confusion, but I'm wondering if there's a way to fix that, with changing the priority or changing the generation process, so you define like the the scope type and then it says you've created this kind of token. Instead of picking the token first.

B

Yeah, I think a good example again.

A

B

How industry has done it is? uh If you look at odd0 how they generate uh you know, tokens is uh like in the oauth context. uh You have different types of clients right, like you, have a.

B

Front-Facing web applications that want to like, say you log into I don't know you want to try to log into ops hops.gitlab.net and you can log in through username or password, or you can log into your gmail account. So when you log in through gmail um it under under the hood, it uses auth to get a token from on, on behalf of gmail and login to log you in that way. So that's one client context and one other client context would be like single page applications where it's a single page application.

B

As the name says uh you don't have uh like you're, not logging in on behalf of uh google, or something like that. So that's a totally different context. So, depending on the context uh like the first question they would ask you is: uh is the client here like a single page application? uh If, yes, then they would only enforce uh this specific grant type called.

B

uh I think it's pixie for single page applications, so the whole flow of how they generate the token and what kind of token it is depends on the client or the context in which the token is being used, and if it's totally a back end oauth client, then they would use like a different grant. I think it's called client credentials grant and the process to generate and refresh tokens is totally different based on the context.

B

So maybe we could use something like that on the high level like what is the use case here and maybe based on that. We can then change how we generate tokens and how we manage them uh accordingly. So if you can log into like odd0 you'll get to see like what I'm talking about yeah.

A

Oh cool that works good all right! Well, thank you! So much for your time. This has been really helpful and I'm gonna use this research to kind of make a bunch of changes and um I'll come back with a new prototype uh to talk to the rest of the team and show uh with the changes we've discussed so far,.

B

Cool sounds good.

A

All right well, take it easy thanks again.

B

Yeah you too yeah. Thank you. Take some rest. Yeah bye, yeah.

A