GitLab Package Group, 1 Oct 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GitLab: Enabling the Dependency Proxy Cleanup Policies

Description

A quick overview of the Dependency Proxy cleanup policies and how to enable them using GraphQL.

Links from the video

Docs: https://docs.gitlab.com/ee/user/packages/dependency_proxy/#cleanup-policies
GraphQL Explorer: https://gitlab.com/-/graphql-explorer

A

Hey everybody. My name is steve abrams, I'm a back-end engineer on the package team at gitlab and today we're going to talk about the cleanup policies for the dependency proxy and how to enable them.

A

So I will start by sharing my screen.

A

And so, if we jump into the gitlab docs, you can find the dependency proxy page just by searching for dependency, proxy or working your way through to the packages and registries area.

A

And what we're going to be looking at is how we can clear the dependency proxy cache. So, as you know, if you're using the dependency proxy, we are caching images into your group's storage and so over time those images can add up and maybe you're not using them all anymore, and so it'd be good to clear out that extra space, previously we've been able to clear the entire cache by using the api.

A

But now we can set up cleanup policies so that we only clean up and remove unused image files that are no longer needed.

A

So when we enable the cleanup policies, we enable them for a given group, and you can configure how many days you want to keep something before it expires and is removed from your cash. So let's say you pull the image alpine latest and then you don't use it for a month.

A

If you set your policy to 30 days, then after 30 days, all of the underlying files for that image will be removed. If any time uh before that month passes you pull that same image and reuse that cached image again, then it will reset that 30-day count.

A

So it will not remove any images that you are actively using initially right now to enable the cleanup policy we're going to be using the graphql api, we're still working on adding a ui to this feature, but I wanted to get this video out so that anyone that could benefit from this feature can start using it right away.

A

So if you haven't used graphql before there's a few different ways to use it, you can use curl commands or your favorite client to make these api requests, but what we do have with gitlab.

A

Let me find the right link here is: if we go to the getting started, graphql issue, it gives an example of how to make a command line request using curl. But then one other thing that we do have here is we have the graphql explorer.

A

And so this is available on every instance of gitlab, including gitlab.com. So when you are logged into gitlab.com, you can make queries of your own um of any of your of anything that's available within the graphql api. So if we're trying to take a look at um if a given group has this feature enabled or not, first, I have a group here. It's just s, abrams public group and we can see we have a dependency proxy that is running.

A

We have a small number of blobs of images, cached, not anything too fancy. But let's say we wanted to turn on the cleanup policy to start removing any unused files that are in here, so that this number of storage capacity can hopefully go down.

A

First, we can start using this to query and see if it's already enabled which, since it's a new feature, it's it's it's disabled until you're going to go ahead and enable it, but just to show you how this tool works for graphql, which is really really intuitive.

A

You can start just typing that we want to query a group and we're going to give it a full path and it'll prompt you with all of the different features that are available, and so, let's give it my full path here, which is let's enroll group, and what we want to look at is the ttl policy, the time to live policy for the dependency proxy.

A

So if I just type in dependency proxy, we can see a list of all the things we can query, and this is what we're looking for is the image time to live policy and then inside of there. We want to see if it's enabled and maybe we'll take a look at what the default setting is for the time to live.

A

So we can see it's not enabled and the default setting is 90 days.

A

So you don't need to worry about doing this to check your policy before you get started if you're just ready to turn it on. We can jump straight back to the docs and use this mutation in order to update the dependency proxy policy and set it to enable this true. So I'm just going to copy this and we'll head back over here.

A

And so now we're going to add my full path in one more time.

A

And let's say instead of instead of using 90 days, we want to be very aggressive and start expiring. Anything that hasn't been used in seven days, and so what this is saying is that we're going to change these fields for this group and then once the change occurs, we would like to receive back these fields to see what changes were actually made. So if I do that now we can see that, for that group the policy is now enabled, and it has a time to live of seven days.

A

So any images that are older than seven days will start to be removed. The cleanup policies for the dependency proxy run once daily. So once every day, if you have your policy, enabled your dependency proxy cash will be scanned and anything that is older or that sorry that hasn't been used in seven days will be removed.

A

So I hope that that helps you enable the new cleanup policies, please feel free to open an issue or message us on the existing issues. If you run into any problems or have any other questions thanks again for watching.