GitHub Demo Days, 11 Dec 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: An efficient SDLC that leverages your toolchain #DemoDays

Description

You've migrated your project to GitHub, now what? We're going to take the example of a Node.js project with planning via Jira, coding in VS Code, GitHub Actions for CI/CD and deployment via Azure to walk through how you can expand GitHub's capabilities with your own toolchain.

0:00 - Start
1:22 - Structure through the lens of DevOps
2:29 - 3 pathways to extending GitHub capabilities
4:50 - Bringing Jira planning to GitHub via the GitHub Marketplace
9:28 - Closing the loop on the Jira side
13:57 - Suggested GitHub Actions based on your technology stack
16:30 - Configuring the workflow and deploying to Azure
23:90 - Making a change and following it end-to-end
24:03 - Download the project locally or open up a Codespace?
27:50 - Commit changes and sync back to the Jira ticket
33:37 - Customizing the workflow file to add automation
43:00 - Leveraging the integration

https://github.com/marketplace

A

Hello: everyone thanks for joining me today on demo days today we're going to tackle what to do after you've migrated a software project over to github and how to make your tool set work for you, instead of the other way around giving the demo today is adam barlow solutions. Engineer welcome adam how's, it going it's going well. How are you doing good good? Also, if you find any of today's subject useful and want to talk about pain points that your org is specifically trying to solve?

A

Just please scan the qr code that I've just popped up into the corner and book a meeting with us with that said adam. Where should we begin.

B

Yeah, so um I think the best place to begin is at the beginning. um I want to thank everyone for taking the time and joining me today for this session. I'm so excited to kind of talk to you about how to expand github's capabilities and leverage your tool chains. um I thought it was an excellent exercise to take it from the scope of bringing a project over as aj was saying after the migration like what do you do so from my customers and who I speak with?

B

There are several common trends that I was kind of hoping to tackle on today's call and if you're looking at this from the lens of devops, there's planning coding, cicd and then your development environment, like how you're going to push it out and provision that so I have probably the more common scenarios and and we'll go through how github's, extensibility and marketplace can help you leverage this so for planning we're going to be configuring jira to work with this for coding.

B

If, if you're coming from a particular ide, that's local to your machine, we're going to see what this looks like going into code spaces. But I think the most popular ide that I've seen is vs code out there today and we're going to take this from the perspective of actions as being our our uh cicd system of choice and then the last thing is our deployment environment which is going to be azure.

B

um So I think a a really great place to start here is kind of level setting on what are the capabilities of expanding github because, like github, does a lot of things really well and we have for many years in terms of inner sourcing collaboration and um we're starting to go into the realm of continuous integration and security with advanced security and several other spaces.

B

But we can't develop for every single scenario and leveraging your tool chain is something that we kind of um push to the marketplace right. So what are the items that you can find in our marketplace? Well, there's github actions.

B

Github, apps and github actions are the two primary items now, if you're unfamiliar with github apps, this is a powerful and secure mechanism that that integrates into github in the github ethos. Now, typically, they materialize as anonymous um bots that can act uh in a very fine grain manner, for whatever action that you're trying to do and then what github actions is. This is our automation system, that's built within our ethos.

B

If there is a an event that occurs on github, you can create a workflow for this, so we're going to be looking at the jira github app to make that connection into jira and for actions. This is where we're going to build test our project. It is a node project, very simple and we're going to deploy it to azure and then the last thing here. I won't dive too deep into this in today's session, but our apis are fantastic.

B

If you haven't looked at this, um here's a really great rundown of it. You can find uh find the source at this location, but we, if we look at all the different references, you can basically pull and push on any one of these items that are within uh within get up so with that said, let's go ahead and dive into it.

B

So here I have our project and this is a again a sample project. We're taking this from the perspective of we've just migrated it over to github it's an existing one that our team has been working on, and there are several other components to it that we need to cater in our software development lifecycle.

B

um So the first one of those components is jira. So I have this giro board very simple, and what I want to be able to do is extend the capabilities of jira and this project management tracking system with actions that are happening within github, so the first place to start, when doing anything that has us reach out to a third party service is go into the marketplace.

B

The marketplace is the the one-stop shop for any everything right. You can find almost anything under the sun, in fact, even with actions it's grown exponentially since it's first started we're at like 11 000 um results here, 11 000 actions that are within our marketplace- and this is because we have such a broad developer community. I think it's uh 70 million developers are now on our platform and this if we can leverage their work to create these integrations and we'll look at this today.

B

But when so focusing on on jiren and what we're going to do, we're actually going to take a github app, I'm going to search for jira, and this first one is- is the one I want to focus on. But that's not the only result. um There's actually.

B

Let me remove this there's actually several github actions that we will be um a little bit of a spoiler that we're gonna look into right and this uh so we're gonna. We can connect into our jira instance. We can create comments, this way, transitions and leverage the capability of these components within our automation, workflow, but with the github app, we can do some really cool stuff as well.

B

So what this is telling us that we can do is that within jira issues we can connect pull requests, commits and branches.

B

Now this is actually leveraging atlassian components and and within smart commits and we'll take a look at this in a little bit, but really what smart commits. um There are several keywords that we can add into our commit history that can trigger the connection into these these tickets.

B

So let's go ahead and set this up. It's very simple: I can just click on setup, make sure we're on the correct organization and then it's going to request us a request from us the proper access, so it does need two levels of permission into our metadata and members.

B

So it's it's going to need this like read access. It's also going to require, read and write access into actions and deployments, so we're going to say, install.

B

Hopefully, it's not that big of a deal. That's.

B

See what's going on, of course, during the demo, um let's try this going to my settings.

B

Third party access all right: well, I had it configured, I don't know why didn't go over all.

A

B

That's good um looks like it was configured correctly. The next thing that we're going to do is uh so most third-party systems, at least the ones that I've seen have a configuration back into github right. So what I want to focus on is uh actually making the connection from the get from the jira side into github, so there is actually a uh marketplace within jira that we focus on it's actually right here.

B

This is one that we want to take a look at, so I'm just going to click into github, and this is going to allow us to do uh the same thing and we're going to connect this free marketplace app into our jira instance. So we can make this collaboration. So once it's synced up, we can get started and under the configuration settings we can connect our github board.

B

The one we want to focus on is our demo days. There's one repository in this click connect now this might take a moment to process and get everything lined up, and while that's going, we could take a look at our let's go ahead and take a look at our project.

B

So the first thing I want to do to be able to make sure that this connection is working properly before moving on to other aspects of our software development lifecycle is creating a test file.

B

So I'm going to create this test file directly within the repository that we were looking at.

B

We're just going to call this github euro and test so down here in our comments section, this.

A

B

Where we need to start to reference, some jr components and if we look back at our smart commits this is the syntax that we need to follow right. So we need to have our issue key, which is this ddc16.

B

um Whatever text and or comment, so we're going to make a comment: have it link back into this? Let me just double check that this is done, though,.

B

B

Double sided c 16.

B

Comment create a new file.

B

So this went through right. We see it here. It was just created right now and might take a a moment here to sync up, but what's a refresh.

B

It has our commit history.

A

B

This is awesome right, so, if you could imagine having epics stories, this full agile environment built out in jira and not having to replace it right, you can still have it make use of the invite the tools that you've been using for years, however long and connected into to get a using this, like extensibility, using the the marketplace.

B

um If we take a look at this actual comment, yeah you can see what we just created was was uh committed.

A

B

We can even click on this deep link and it'll it'll direct us back into github perfect.

B

So the next thing on our to-do list is to create an action that does both uh building up this project testing and deployment.

B

So it shouldn't be too difficult difficult when leveraging uh the marketplace. They do stuff like this, but um there's a couple ways that we can take a look and find out what we want. So the first spot is actually going back into the marketplace itself. Now we do have to click on a external site to do this, for the for one of three ways right.

B

This is just the first way that I'll show you, but if we go into maybe like our node project find out ways to build it, we can just leverage one of these items. So it's like set up this environment.

B

We have um basically.

B

This syntax that we can copy over into our workflow file. We can pick and choose which modules and tasks that we want to add to a workflow to kind of like build it out. I think the easier way to kind of conceptualize. This is directly within the actions tab itself.

B

So if we go into actions, I do have a couple actions here, but um if you're setting this up for the first time, it'll actually look like this, so we make suggestions based off of the technology stack and the tool chain. That's within your repository.

B

And based off of that, you know you can click through so like one is, is this node.js right? So we can configure this? It's not 100 the one that we want to focus on, because it's just one component of the three things that we want to do right and it's actually. This is this is the one that we're going to pick today is creating that node project and deploying it to an azure web app.

B

um There is one other thing I want to point out before we get into the configuration step, and that's this down here. So what's really cool about actions?

B

Is that, since this is our underlining automation system, you can perform automated tasks that are outside the soft uh ci cd realm, for instance, with stale stale is really cool because it'll check issues, it'll check, pull requests, see when they were interacted with last and if it hits a certain threshold contact, the the creator of it. Let them know it's like. Is this still live, so you can really do some maintenance, like cool maintenance, stuff on on your repositories.

B

um But if we wanted to configure this, we can click on our item. It's going to have us do a couple things here right. So since this is pushing out to an azure environment like a web app, we need to to actually add in like a publishing profile right. We need to specify the package path which node versions are being used and even the triggering system. So anything that's underneath this on. This will be our track triggering mechanism.

B

Now we can actually use the marketplace items in here too. If we wanted to build this out, um which is really nice, that we don't have to keep going back and forth from different tabs, it's all in one space and that still rings true with documentation.

B

So if we wanted to change anything in this in this yaml file, we could pull it directly from here. So if we wanted to um have it get triggered on a pull request like a cron job, we can do all of that.

A

B

Adam, I'm sorry to sorry to.

A

Break your flow, but I've got a request from chat to zoom in a bit on your screen. Please.

B

Oh yeah, absolutely.

A

B

This is a little bit better yeah, so um I'm actually not going to go through the configuration on this side because there's an easier way to do this, and I don't want to go through the trouble of linking up my uh publishing profile and and creating like secrets for this within our secret store. But azure actually does this for us, which is really nice.

B

So here I have um hopefully that that looks better I'll zoom in one more time.

B

um So we have two web apps here one's for the completed project, just in case the demo gods strike again- and I I need to show you the finished product, but the one we're going to work on. Is this demo days calc?

B

Now it's already up and running right. We have this url and this is our production environment right. So our calc is very simple. um Has this blue background and if you had noticed, one of our other issues is to change the background uh for demo days, so we're gonna make this a little bit. Snazzier, we're gonna go through that workflow and what it looks like once we have everything hooked up and configured um but to make that connection from github into azure.

B

There's there's this deployment center now that this kind of goes back to what jira was doing with its marketplace items um since developers use our tool right, 70 million of them a lot of other third-party environments are going to create configurations into our system to make it as easy as possible to kind of like leverage their products with what we're doing- and this is no different, so I'm going to create I'm going to connect into the source here. I've already signed in with my github username. So I have access to this.

B

Select the organization demo days.

B

My repository and branch, so now it's going to give us two options. The first option here is to actually create a workflow, which is what we'll do, or we can use an available one so creating this new one. Let's preview, this file and it's a little bit wonky, because I've um zoomed in here it'll probably make a little more sense once I actually deploy this.

B

But this is the same workflow file that we were just looking at the only difference is is that I don't have to create the variables and save them into my secret store. So I'm just going to save this.

B

You can see that it's connected and if we go back.

B

Into our project we can even see a run. That's that's going on right now. Maybe this will look better. We see a run, and this is because I just added our new file. In fact, if we go into our workflow file, you could see this like master demo days, calc yaml file and in it it's the same exact flow.

B

um The publishing profile is referencing an element, an item in our our secrets, key vault right. So all that's that's going now. I I do want to edit this a little bit. So the first thing I want to do here is actually change this part. So I want this to get triggered not on push and commits, but I want this to get triggered on pull requests right, so this pull request, syntax or I'll, replace it.

B

So that looks good. The next item is my testing. So spoiler, the test is going to fail and I don't want to fail right out of that right. So during our testing, we're just going uh to make sure the connection is, is working and we're deploying properly I'm going to omit the the test so perfect, let's commit this back, we're going to save it to our main line, and we should be good now. If you remember, the action was kicked off and it failed.

B

So once we go into the section, it's already detecting that we have a build component and a deployed component doesn't go into the ploy section, but under builds we see a test. That's that's! Not uh that's not good, so we'll dive into this in a in a second but the connection works and we could start our workflow right. So the first workflow that I want to do is creating a new background.

B

So if I open this up, we could see what the key to this item is and.

B

um Yeah, so you can see what the key to this item is and this ticket, because we're going to need that here in a second.

B

So the first thing I want to do is let everyone know that this is in progress, I'm going to make this transition, and so we can start to work on this item now, when you take your code at this point, there's a couple things that you could do.

B

You can go through um downloading a copy of your repository, your to your local system, either through the github cli ssh into some server or use https right code. Spaces is relatively new, and this is really great because, instead of configuring, your local environment, especially if you're new to a project and there's a lot of dependencies that you have to worry about, you can have a yaml file. Have that configuration stored for you. So any new developer that comes to a project can just spin this up and it's a virtualized environment of visual studio code.

B

So we can just get started without having to worry about any configuration of the environment and start working on our um item here. This change background, so the first thing I want to do is create a new branch.

B

I'm going to call this change background demo days.

B

Hopefully this is.

B

Large enough I'll zoom in here a little bit more too so once we're switched over to this new branch, let's head in to our um yeah, let's go into a public folder. We have this default css.

B

So right now um it's referencing this shade of blue.

B

um I have my code already commented commented out so so we don't have to waste any time with that, but um just to test to see where we're at and get them level set to make sure this is at the uh at the same state that production is which should be.

B

um I can run node server.js to get this up and running so once you click on it, we could see our calculator app is showing up as expected.

B

So let's go through it and change our background and see and see what our changes come out to be. So I'm going to comment this back in.

B

B

And there's actually one other thing I want to do and there's actually there's a background on the calculator itself that I want to comment out.

B

Awesome so let's make sure that our changes are showing up as intended.

B

And look at that gorgeous uh aj you'd, be, I feel, like you're you're, very proud of me right now um that I'm putting a little bit of demo days branding on this thing. So this is our intended outcome. We want to go at this point. We can go ahead and and check this back in.

B

So we can connect into our repository. In fact, you could do this. It doesn't have to be within like code spaces if you're not familiar. There are a lot of github extensions within vs code and other ides that allow you to do the same thing. If you don't feel like going through a terminal and using git commands to kind of push this up, but with our source I'm going to commit our changes and sync it back to our jira ticket so again say ddc 15.,.

B

It's a key phrase, comment and change background.

B

Now that we have that going we'll push it and we could create a pull request right here which would trigger our workflow. If you guys remember.

B

I won't do it at this point, but let me actually go here and we can and we can compare it.

B

So now that we're creating this pull request, let me make sure it's going back into.

B

My repo and not the one I forked from awesome, so you can even see that it's taking this comment. So what's really cool about this, it should show up in our first comment. So we have. We have one comment here, but as we.

B

Create our pull requests and go through this flow that we're all pretty familiar with. I'm assuming it's going to kick off this action that we just created. We can start uh collaboration with our other team if, if there's issues that were found- but at this point um it should all be pretty straightforward right, so we could see it kind of go through the process now this is going to take a like a few minutes.

B

So what I'm going to do at this point is look at that error that we had previously.

B

So one of so, if we go back to our actions results, we can see that this build failed.

B

B

It's going to show us that one of our tests in our test directory didn't pass, and this is a arithmetic um equation right. So what do we normally do if we're trying to resolve a bug that we find well first off, we have to track this, so we have to go back into jira uh label it. So eric uh bug.

B

Now, at this point,.

B

We'll put it in this section here at this point we can start to fix, fix our issue which, which we'll actually do right so we'll do this we'll do the same thing open up uh code spaces here, I'm gonna create a new branch call. This fix bug.

B

That's probably that's a terrible name. So.

B

Sorry, I'm more normally more creative, but that's fine, um so all of our controllers or in inside this directory here and we can go in and and this is actually the problem right- we're in javascript.

B

This is concatenating it it's not like proper um addition.

B

If I just add in these plus signs.

B

And we should be good to go like this, should fix everything, um so I'm gonna go through the same process of referencing, our new ticket that we just created.

B

Oh, you know what.

B

And publish this so same thing right: it's going to go it's going to create a build for this and and go through the same workflow file that we just saw before, but the one thing that was at least annoying to me was the fact that I had to go into jira create a new ticket for this when uh that's a manual step that we could probably automate.

B

So I want to expand on our workflow file right.

B

Let me just double check here yeah. This is still going.

B

So it's going through the deployment.

B

So um actually did we create oh yeah. We didn't even create a pull request for this. Oh, my god we can create. We can push this back. It's going to do the same thing uh I want to do. I want to do this.

B

You know what no, let's not do it quite yet, because what I want to do is make a change to the workflow file, so we can see an issue created automatically for us, so we don't have to go in, and this is just like another instance of um how you how you're able to uh look at that I went into or or yeah. That's not the right spot um yeah, because, like any manual task that you can automate it's going to increase efficiency.

B

So let's let's go ahead and do this: let's go into our marketplace and search up jira actions.

B

So there's a couple of them that we can choose from.

B

We have our jira login set up jira issues from to do, but this is this is the one that I want, so we want to create an issue ultimately so creating this issue requires several parameters that we have to fill out. Some of them are yeah the three of the five that they're listing here are required.

B

So we need a project id issue, type and summary, um but we do also need to log in so I'm going to go into jira.

B

And yeah, so once we're in this login, we can perform all of these actions. We can. We can do transitions comments, uh create task.

B

B

B

Do this right, let's, let's go ahead and and create our connection so like the first thing I want to do here is take our login.

B

Go into our workflow file.

B

A

B

And underneath, um where installs builds our project and all that good stuff? Well,.

B

Let's bring all this over.

B

B

Actions artifacts.

B

Breaking that over.

B

Okay, so once we have this entered in there, we're able to.

B

Take in fact, I'm gonna omit those two.

B

We want to create our ticket and we want to yeah create our ticket at this point after we log in.

B

So we'll tweak a couple things in here and also I forgot to mention, so I'm not I've already set up these secrets within the repository. So if you go into your repository settings secrets, um this is where you can kind of put in, like your workflow environment, depend um uh dependent keys right. So I have three of them here.

B

One that's comprised of your api token base url and my email address.

B

So that it's going to be pulling from that and then it's going to create uh the ticket now. The things I do need to change here is point to the right project.

B

Make sure that it's the correct issue type and just have to say, like pest found.

B

Bad mapping, this need to be.

B

B

Great, so we have all this set up. Let's put in let's comment this back in, so our test can can actually run.

B

B

We can commit it, I hope no one's cringing at the fact that I'm not putting in actual comments for this, but great. So we have that going.

B

We've altered our workflow so now, when we take this new bug fix it, should it should create it. We didn't put we.

B

We could take this a step further and actually add in a conditional statement, saying if test was a failure, go ahead and create the the ticket and even leveraging like uh composite actions or um like modularizing, this action into a separate uh sub yelmo file, which we probably should do, because this is those jira that jira component can probably be used elsewhere within this organization right, we we can refine it a little bit more to get more utilization, but for the purposes of this demo I think that's pretty good.

B

And now what we'll do is compare this new branch um yep, we'll push back to our main line, connect in to this okay, so we'll say: ddt.

B

B

Comment fix this bug all right, so now it should once go once it goes through our workflow. We should get automation, uh automation in action and actually create some additional jared tickets for us, while that's going on. Hopefully this isn't, you know too disjointed, but with the two builds that we had running is here. Let me close this.

B

So this one passed, so we could see here that both uh the build the deploy were successful. When we changed our background, um we even have a a link for the production environment. So if I click on this, here's our updated version, so everything worked properly.

B

We can go back into azure and our overview see some utilizat utilization spikes, so it it worked, we're in we're in prod and um it even created like a nice little artifact that we can save to so again. For for the underlining message for this, uh for this demo is how you can incorporate extensibility within github, so we've connected into jira, utilizing our same project board. We've pushed um out a new deploy to our azure environment, utilizing actions and even leveraging code spaces. We have like a development environment. We've done all that.

B

um I do want to highlight some more stuff. That happened, because what I showed before was just a commit, but this connection into jira is really cool because it was it picked up on everything it picked up on our build pull requests the different branch that we created for it just because we made that link and it's all available within this environment.

B

So I think the last thing here is to see if yeah look at that- oh man, my timing's spot on, because this is our new issue right. So now, if there's like a bug, that's found from um like a test or something like that.

B

We can create issues directly from github into uh in into jira, and you can utilize this in a lot of different ways.

B

So, if you're, if you have uh advanced security enabled and there are bugs and vulnerabilities found within advanced security, you could set up a workflow that periodically scans a repository and if any new vulnerabilities are found, you can push it into your. In fact, there's there's a couple projects out there and actions out there that allow you to do this, but you can leverage that same workflow and components with any tool out out there, um even if there's not a github, app already created or an action or to create that.

B

Allow you to do that with our api. You can basically perform these the same functionality.

B

So that's it for my presentation today. Sorry I kind of like dived into it, not many breaks into here, but I I definitely wanted to show kind of like the full functionality of like our marketplace and how you can extend the capability of github into other third-party tools.

B

um Let's see, if there's any, is there any questions happy happy to handle that uh I don't know aj were, were there any out there that uh you saw that we should focus on.

A

No, I think, we're in a pretty good spot. Actually, okay, nice all right! Well, I guess that concludes it for today's demo days, thanks again adam for uh showing us how to get your tool set in line uh for your my great github project and we'll see you again soon, hopefully,.

B

Absolutely thanks for thanks for inviting me on. It was a pleasure.

A

All right see you bye.