IPFS IPFS þing 2022, 9 Aug 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: GitHub Configuration (of IPFS organization) as Code - @galargh - Project & Community

Description

No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).

A

I'm peter as steve all research. I work on the ip2 arts team, specifically on the ipdx team, which takes care of developer. Experience needs for ips tours now, but our long routine goal is to expand further beyond that, and you know steve being the great encourager he's. He encouraged me to explore. Github configuration space and that's what's brought us here today, I'm going to talk about how we're thinking of configuring github as code, instead of just relying on ui transfers for that.

A

So the agenda for this talk is that first, I want to explore the reasons why we even once on this endeavor, because it's pretty big one. So we better have good reasons for that.

A

Next, I'm going to quickly go through what we decided to use for to actually achieve that, and, finally, I'm going to show you how it works in ipfs organization. So why configuration is code? Well, the main reason is that doing it through ui on a scale just doesn't work, so many different check boxes. So many different settings pages like if you want to manage anything beyond a scale of a couple of repositories and couple of users just really hard to track and trying to enforce any best practices across bigger space. It just becomes unfeasible.

A

So what we were really looking for was to be able to ex oh, to represent github configuration in a human-friendly format and store it in github repository and then have some tooling. That is able to translate that into actual github changes and it as it turns out like it comes with quite nice properties, because it's for months it enables collaboration, and when you can imagine making github changes through through pr's, you can discuss them. You can get counter proposals for people involved in the trench. You can get reviews for them, which is just nice.

A

You can preserve context next to the changes you made, not only in the prs, but also in line with the things you're changing it's more secure. If you don't need high permission levels, because you can achieve the same through making a pr repository like not so many people needs need admin rights of everything it opens up, possibly for automation, because since most of your changes or all of them, eventually make it through a single single repository that handles github configuration.

A

You can do interesting stuff like reacting to a new repository being added and ensuring that it has branch protection rules set up. For example, it increases visibility because you no longer have to rely on making hundreds and hundreds of api calls to retrieve information about your organization. You just look at this human-friendly representation and quickly glance through it to figure out. What's there, you can also also write scripts that extract the information you're really interested in without worrying about hitting any github ratings.

A

uh Yeah and, most importantly, it's feasible at scale like with that many different resources to handle. uh These are the numbers for ipfs organization yeah. Without the help of machines like it's, it's really impossible to track all of it, yeah, oh last, but least like if you open it up to people like you. No longer have to find this one person who has enough permissions to do some action for you. Instead, you just create a pr and request a chance. You want to see to be made.

A

So is there anything, that's actually enables touch in the wild. So, as it turns out, there wasn't at least not to the extent in which I imagined it.

A

It's starting to change a bit like recently github itself, open sourced a project called entitlements which, to some extent covers it, but only on a small level in which it can manage access to the organization itself and access to specific themes based on text files.

A

But what we did we went ahead and built it from scratch. A project called github enactment which is terraform based so yeah, it's terraforming, the back, which takes care of actually applying the the changes you expressed in a yaml format to github and back, and everything in between is handled through github actions.

A

And what it can do it can help you with managing level access, control of organizations, teams and repositories, but not only that also, you can tweak any settings.

A

Any configuration settings that organization, team or repository has, and what I think is really important is that it can co-exist with people making changes through ui, because I, on day one when you start using this framework, you don't have to switch everyone you can you can it can work with phantom with people using the ui and because it supports the wasting transaction feature from the ammo to github and the other way around pulling changes from github and reflecting them in the yammer.

A

And yeah it all sounds good, but does it really work as it turns out it? Does we also have it started up, for example, for the p2p organization, and not too long ago? I went on this endeavor of trying to remove inactive users from the organization itself, teams and repositories it affected more than 100 different individuals. So if you can imagine trying to track that many people in any other way, that would be like pretty much just going to w3c and trying trying to achieve anything there.

A

So instead here, I could just create a pr mention everyone and start the discussion. As you can see, it was quite fruitful. There were some counter arguments being made there, so uh it was productive in a way.

A

That's that we changed our approach to do this cleanup and instead of removing people from the organization, for example, we decided to just create a team that has no heightened particular taxes to any resources in the organization, but still, let's see, display the honor badge of being part of the organization and with all that setup, I was able to merge that pr within a week, which I think is quite impressive, given the size of the trench and, what's even better there is, there is a bonus point to be made here, because, later on, I found out that this exact same discussion about not removing people from the organization had been had a few years back and we even had already a team set up in the organization for exactly this reason.

A

But since we had no way of storing the context next to the resource itself, I had no way of finding that out. So I only found out about this post action and now yeah we have it.

A

We we have the context where it should be so, hopefully, the next person that comes after me can just read the trans and figure it out. So how does it all come together in pfs arc, specifically yeah? Let's go and see.

A

So this is what github management is it's just a github repository in ipf's github management. It was created from a template repository in the protocol arc. Github management template uh what's critical about it is that it comes with an auto upgrade features.

A

So whenever we make improvements to the to the template repository, uh we can just run a workflow here that pulls in the the new features that that got enabled in github management framework uh when you get here to this repository and want to work with it, you're greeted with with a redmi file that has a bunch of useful documentation, links uh from a contributors perspective. Probably the most important one is to how to work with github management, and it explains things of how to create new things, how to modify the existing stuff.

A

But I think what's the most useful thing here is a full example of a configuration file of a yaml configuration file that lists all the things that can be configured through github management, what they mean: what's the syntax and so on, and so forth yeah. So that's that's the file. I would recommend viewing if you make your way to github environments repository uh and now, let's go back to the the main page.

A

What I want to show you is where to find the ipfs uh ml configuration file, so you would have to go to the github directory here and find a file which base name is the same as the organization name. So ipfs.yaml.

A

Here we have all the resources that github management supports at this point point uh listed in the in the yaml farmers. So if you um so yeah you can view everything here and uh you can start proposing new changes. So, uh let's, let's try to propose a to change code owners in the github management repository itself.

A

uh I would look for github pandas here and uh code owner's content right here. So right now, it's just me, but that doesn't scale to well. If something happens to me, so maybe large trendy to ipdx team instead.

A

And just stay to change change code owners to the team.

A

Cool and let's create a pull request.

A

So there we go, that's how easy is proposing changes through for github management and what's going to happen right now, is that, like a bunch of workflows are going to start uh in particular a plan workflow that is required uh is going to start and what it does. It performs terraform plan action, so it's going to try to find out what kind of changes to github it has to make so that the configuration that that we have in the repository is reflected.

A

uh What's also interesting is that it's going to post a comment on the pr with the output of this pawn and also after we merge this pr, this exact plan that was shown to people viewing the pr is going to be applied like so we we have certainty that these are exactly the actions that that we can expect to happen after the merge but yeah.

A

What is missing, for example, here and what would be a cool automation, is that uh I know that ipfs ipdx team doesn't exist yet so it would be nice if we had some workflow. That could check that for me and tell me that oh you're doing a thing that won't work.

A

uh So all things like that are possible, but just not yet implemented, but I have another pr here so that you can see how it looks like. I got a review from my teammate that this is the pr that creates ipdx team in ipfs organization.

A

I have a. I have a plan that tells me that a team will be created with two members to add maintainer level in it all sounds good. Oh.

A

Wait: oh there we go so yeah, let's go ahead and merge it and he's going to create this thing for me just for a second, because after a merge and apply, workflow get started that actually downloads the plan that was created during pull requests workflow and applies it to github and in the meantime, while it works.

A

uh Let me just show you one last little thing that I mentioned before is the sync workflow, so this one uh is the one which actually goes to github makes a bunch of api calls tries to find out what the current state of the github configuration of the real github configuration is, and it compares that with the yaml configuration file in this repository and if it finds any differences to between actual github configuration and the yaml file, it will create oh it's frozen. It will create a pr that updates the yaml file.

A

So, for example, here we have a pr created by a bot that tells me that someone added kyle to as a collaborator- if I remember croky yeah as a collaborator- go grab some repository with push access, so this trench was made through ui, but it was like really simple to just pull it in into the our framework. So I think by now uh our ipdx team should be here yeah there. It is so the more worked.

A

I think that's that's my queue and time. uh Thank you for her attention. uh If you have any questions, I'm around, uh I go by galler in falcon slack and and github yeah come say: hi. Let's, let's talk more about how it works.