Cloud Custodian Community Meetings, 22 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cloud Custodian Release Process

Description

In this video AJ does a live release of Cloud Custodian. We're working on automating this process but figured it would be neat to just record it. In the future releases will be automated.

Find out more at cloudcustodian.io!

A

All right, aj take it away we're doing a release. So let's, let's try to do the thing uh I'll share.

A

Now I wonder if I can share, I don't want to share your faces, I'm going to try to share the release checklist and a terminal where I'm doing things.

A

Oh, you just went muted there, george yeah, I usually just minimize the meat window and then just share your screen in the display.

B

A

Yeah we do blow out of slack because who knows what's going on? uh Okay share this other screen.

A

Good luck! What could go wrong um cool? So, do you all see a release checklist.

A

Yeah yeah cool, yep, okay, so some of this uh I some of this was a checklist that existed already. I was going in doing going through these steps for the first time myself and just added some pieces in um this.

A

For anyone who has used pypi uh to upload packages before you can set up a pipi accounts, I've just added in some some kind of basic setup for getting uh for defining both the the normal, the real pi pi repository and also test pipea, where I published last week and just linking to their to the the publish python packaging authority authorities documentation on their their recommended method of using keyring to store your api tokens rather than baking that and hardcoding that into a config file.

A

So that's just some background setup so that when we do, when we go to publish later, uh twine will pull that stuff out and theoretically just work.

A

So next bits are clean check out and clean virtual live so I'll. Do that? Let's see we'll do cloud exterior and can you update font size a little bit? I can.

A

There we go: isn't that good cool, so we're doing we're going fully complete this. Isn't this isn't a new work tree? This isn't just a new, a new branch or anything we're going a full clone, not even a bear clone, because that would be, I mean uh a shallow clone, because that would be smart. No, I'm doing the whole the whole thing into a new directory.

A

Okay, so we've got a release directory and I'll create a one.

A

We've got an active environment here.

A

And we'll install all the things into this environment.

A

And this piece does take a little while so it as it's going. If any anyone else has questions comments have done similar things. uh I don't know. If I have either have any of you try to uh hey darren. I know you've done plenty of uh devin stahl's uh steven view. Have you done any of this stuff.

B

I tried to do the unit testing thing and I yeah I couldn't figure it out at all.

A

B

Guess this and let everybody else fix my phone.

A

And you know what that's that's totally fine, because right now it is a bit awkward and I know we've got some some work in progress to make this install a little bit cleaner um because right now yeah every time I see this and it's resolving these dependency trees separately yeah. It hurts a little bit so we'll get there.

B

I tried I actually tried. I forget what my issue was a few years ago and um I ended up just patching. I would install on the patch.

A

Yeah, that is fine, so uh would a so if you were gonna to work on something like this. Would you prefer to work just on your local local machine? Do it all locally or have some kind of a a web-based or container-based dev environment? Do you have.

B

Your container base.

A

Would be my program you'd want container how about you darren, I'm just pulling while we have that, while we have time.

B

For me, I'm more used to just doing it locally yeah.

A

Okay same I've been doing a bit of both, but I'm feeling like going the container way might be a little more welcoming for some new folks or at least having that as an option. But yeah.

B

I work mostly on on arm and okay. I don't even have that at home. I don't think maybe my one of my max might.

A

Gotcha, okay, so for the folks following along at home, we should have a an environment that has all the custodian bits installed into it. Then the next piece, which sort of already done.

A

Last release was 16.

A

A

Oh this, so this was something that I ran into when I was testing, so it's good good to see it uh immortalized on recording. uh I had to go and pip install the docker module, because this changelog tool uses previous versions. If we're going to generate a change log from 0916 or 917, or to current it's going to spin up the docker, uh the docker version of c7n. From from that previous version, and it's doing that through python code. So we need the docker module installed.

A

I'm actually not going to install that right now, because I've got a let's say.

A

Got a release and I'll pump the font size here a release drafted with the change log, um so I would call that out. As a note one other thing that I noticed from the the changelog is that again because it uses docker and we don't have the aws cloud control api provider, the aws cc included in docker.

A

It will detect any awscc changes as schema changes, so it just balloons this whole section so manual, editor notes here or that I I had to generate that change log with that docker module installed and then trim out, trim this game of changes section that I suspect going forward, we'll install that awscc into docker. uh But I don't know that for sure.

A

um But okay, so that's the that's the changelog, that's the only piece we have to do there. uh The next step is this: freeze, free setup.

A

I'll make a frontend setup and what this one's going to do is right. Now we have, uh we have all the dependencies specified in.

A

We have all the dependencies specified, as uh version ranges uh it already finished in here.

A

Let's see and george. If this is too much detail, let me know I'm just trying to things that would have been useful for me going forward.

B

A

Good yeah, okay, yeah, so the normal setup.pi that we have in in all these projects here, it'll show a version range, so it'll say hey for our complete. We just need something.

A

That's at least this version and that comes out of our poetry, dependency specification out of the lock file and when we do this frozen setup, it's going to say no we're not just going to say this version range we're going to we're going to pin that we're going to say this is our latest thing from that: we're that locked out we're going to specify exactly this.

A

So when you go to pip install if our complete happens to bump a new version next week, when you do a pip install of c7n you're still going to get this 2.0 because that's pinned in our release so that we don't have you know the adders or our complete change or something, and it brings us uh between releases and that's that's similar across all the dependencies uh for most changes. We will uh as we're going through and creating release bits we'll go in and update the changes we'll commit them back to the repo for this.

A

As far as I can see, we don't commit these changes because they're just they're baked into the wheels that we publish and we don't actually commit these back to master. uh That's uh I'm not sure why, um but, but that's what I've seen so, I'm gonna keep doing it.

A

Oh okay, and so after we got the the publish, so we we had a step in the release checklist about commenting out pi win32. I believe that we've got that fixed, so we shouldn't need to comment anything out. There's um we were losing at some point in the past.

A

This line was uh it was losing this windows platform constraint, so we it was showing a pi 132 dependency uh on multiple platforms, so folks would try to install it and it would say: oh we've got no 5132, so we had to have to remove that and I haven't seen that anymore. So that's good.

A

A

And the poetry install, we would have already done so. That's cool uh the next step, and this is the change since the last release is that there used to be a make uh pkg uh uh build, uh published wheel uh and what I've changed is making a pkg build wheel, because then we can build all the wheels locally and install them into a new environment without publishing them anywhere and just test to see. If there's any uh gosh darn it.

A

I know I'm already in the poker show this would work. Oh, this is for for checking okay. So what this? This is probably an issue with.

A

Is one of those packages that we've seen that will kind of flap between dependencies? This is one of those cases where it's it's uh tedious, that we have multiple dependency trees. So here we go top level we're installing colorama gcp we're removing colorama and then nothing else is installing it again.

A

So this is one of those cases where it would be really nice to uh to have a single dependency tree. What we're doing with colorama there is uh is using twine check to just validate that our wheels look. Okay, so I think, for that purpose we can do a uh we'll just install the top up level package and that should just a colorama back in and remove remove typing extensions.

A

B

A

I'm not too worried about losing that typing extensions here, while we're building uh but that'll that'll be good, we'll have the we'll be able to do the wheel, install into a new environment.

A

uh This all builds since we have colorama and that's cool uh okay.

A

So what we have now is we've got wheel files built and for all of our all the projects and uh something.

A

We can inspect those. This is not really in the checklist. This is just curiosity or fleshing things out a little bit uh wheel files just a zip file with some metadata in there, and if we look at the metadata, this is what's actually going to get pushed out to pipi.

A

As so, if we go and look, look at c7n0917 we'll see these as all all dependencies, and so that's just worth going through and seeing that we've got the the pinned versions, the hard versions. uh This is as much for me as it is for people following along just to make sure this looks legitimate.

A

That's cool and the next thing would be to create a new fresh virtual environment. Let's call it release test.

A

Okay and then for that.

A

So we've got nothing in here now.

A

I just did that just keep that okay, so for here I'm going to try to find all the wheel files that we've built and then install them directly. We're not going to go.

A

Go pull the packages from pipi we're just going to try to install all the locally built wheels into the same environment and make sure that we don't see any weirdo errors and make sure that everything that everything in there is everything installs into the same environment. No conflicts.

A

That yanked version is because of an older azure dependency. I would love to get rid of that. If anyone on the call or anyone watching has familiarity with azure data lake store, then it would be cool um seeing if we could update that.

A

Okay, so that installed a bunch of stuff.

A

A

The locally built versions that we have looks like we're all matching, so we've got our 917. It installed on 917., all of those all those dot. 16S look good uh cool and we didn't know so what we would have seen on previous releases, where we've had conflicts is when we tried to install that c7n wheel next to say azure, it would throw a pip error, it would say no can't do that. There's a there's, a conflict with james path between between z7 and c7, azure and pip's resolver would scream at us.

A

So the fact that that's not doing that and then it installs, uh and that we see that the wheels have all the pin versions. That makes me feel pretty confident about about things working.

A

The next thing would be to uh so. Let's see, I'm gonna run something run some policy.

A

Okay, so I'm running I'm able to run policies across a few different uh few different environments.

A

Okay, I can run ec2.

A

Can run some azure stuff, I can run gcp.

A

I'm seeing the current version.

A

Cool I'm sitting a bunch of stuff in here.

A

Across all different providers, that's great lots of filters cool all right, so those are some basic sanity checks. Just showing that we've got we're able to install everything. uh We see the right schema. We see the right versions got a release drafted uh george any uh since you, I think you've you've been through and seen some of this stuff uh anything like especially fishy. So far, uh no none so far cool. I do want to check this file into the repo, but then.

B

A

Also, a goal to not have to do this so yeah exactly.

B

A

Been going through some of this stuff, it's some of the updates I've made were with the eye cord like what can we automate here rather than looking up? What can we just kind of script? Pattern match things: uh okay and like this, this manual, this manual update of the the package repo.

A

A little bit fishy, but okay, so with all that stuff done.

A

Oh and then the next piece would be to actually do the release, so this release is drafted.

A

A

Look at previous releases to just confirm the tag.

A

Yeah tag is just zero and again this would be something that we would do. Typically just do through.

A

You know typically do through the the command line we'll script this, so we don't have to do it manually, but all this stuff looks okay. uh George, I'm gonna hit the create a discussion for this release in the announcement section just so that we see something pop out there. Yeah I'd love to see what act. Would it I'd love to see it actually work?

A

Okay, let's, uh let's see.

A

Oh, we weren't able to create a discussion.

A

Oh, oh, I think you had to give it a title.

A

Go back down. I thought I did get this title.

A

huh um Yeah, I thought this was the title.

A

All right well that piece, I guess, we'll go back, it's good to know yeah. I guess uh uh okay.

A

Back here just because and then.

A

The last piece should be to do this and then we'll see so.

B

I actually haven't done.

A

This piece yet the the publish to real pipey so we'll see if this works or if I get oh, no, that's because I'm getting a password. So that means that I didn't set up my gearing properly um and I'm gonna have to do that on another screen.

A

Yeah, I think I'm gonna have to either stop sharing for that one or do something weird.

A

A

You can all watch me paste my password and tune in perform repeatedly.

A

Rather than troubleshooting the ring but hey when I wasn't recording and published a test ti, it worked flawlessly.

A

Is there a different set of creds between the tests? That's got to be everything interesting, so I can't do. I can't publish policy stream.

A

A

uh Yeah well, it is a different set of credits. That's for sure, but interesting that I couldn't do policy stream.

A

A

All right: well, I'm gonna have to troubleshoot that one and that we can do I'm okay doing that off uh offline because we went through the first few, at least, um and let's see, let's look at that one again, so that would have gotten gcp cube, openstack mailer, uh so we didn't get to awscc or azure right, that's not great, but if uh stephen or darren, if one of you want to try to do like a pip install with the upgrade on um on c7n, that should pick up 0917 now.

B

We wouldn't know.

B

It's 9 17 right.

A

Yeah yeah cool all right so hold that okay, so I've got a troubleshoot policy stream, so I can continue publishing the rest.

B

Rebuild all my dockers, I noticed today I had to um with the new c7 and work I had to change the build order, I'm installing on cli2 on the same container, and I was I had to. I noticed that if I I did the pip install of c7n before the install of this cli2, it didn't work.

A

Oh interesting, do you have? uh Is there I.

B

Can redo it yeah.

A

Okay, yeah I'd like to see the issue and try to reproduce that one yeah.

B

I didn't I I couldn't figure it out. I'm like. Why is this? You know, I always say the only thing that's happening after the install was the cli and it worked fine in my test bed. So I was like it's gotta be just some of the cli. It's messing up custodian, but um that was yeah. I just rewrote my docker call.

A

Okay, yeah interesting: I.

B

Wonder why it would have right it shouldn't have.

A

No, no, I don't think that that should make a difference, but I'm curious what else? What might have changed that made that made that ordering matter the.

B

Install for the cli.

A

Is ridiculous anyway,.

B

A

Yeah, how are you, how are you installing it are you grabbing like audi.

B

Install you know the dot, slash, install.

A

Yeah that okay.

B

Horrible, you know, definitely let me know, because.

A

Yeah, I'm trying to think I I think I tend to do it through like homebrew linux, brew or whatever, but I know there's the install that that the shell script that they get works too.

B

Well, if I do, you can, do you know if you install with yep it only installs the old one, whatever old version of cli yeah.

A

uh All right, so I think uh george, we can probably kill the recording and I will go, follow up and troubleshoot the policy stream uh unless.

B

There's anything.

A

Else recording relevant, you think thanks. Everyone.

B

It's working fine.