GitHub Demo Days, 17 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: How to run end-to-end tests on cloud native apps BEFORE you spin up a new Kubernetes cluster.

Description

#DemoDays #cloudtesting
14:24 - Start of the live demo

Testing is a critical part of any CI/CD pipeline, but running tests in Kubernetes can absorb the extra time that automation saves. Enter kind. kind stands for “Kubernetes in Docker.” It’s an open source project from the Kubernetes special interest group (SIGs) community, and a tool for running local Kubernetes clusters using Docker container “nodes.” Creating a kind cluster is a simple way to run Kubernetes cluster and application testing—without having to spin up a complete Kubernetes environment.

Join Sarah Kalife and Adam Barlow, Solutions Engineers, and Grant Griffiths, Tech Lead at Portworx(http://portworx.com/) as they use kind in concert with GitHub Actions to eliminate the inconsistency of local dev environments when testing cloud-native applications.

https://github.com/learn/devops

A

Hello, everybody welcome to our devops demo day, so we'll kick it off with some introductions we'll be talking about testing your cloud applications on on github actions with using kind. So um let's get everybody on the screen, so we can go through some introductions, all right I'll, let I'll kick it off! Really quick! So hey everybody! I'm sarah khalifa! I'm a solutions engineer here at github!

A

um I work on a lot of the enterprise accounts. uh Previously before I joined github, I was a senior software engineer there working on some of their cloud applications using kubernetes and a couple other things um I'm originally from cyprus and lebanon. I currently live in san francisco.

A

You'll see some cool pictures here from a couple of my favorite spots to visit, and if I were a tour guide and you're visiting san francisco, I would actually take you up route 1 to see some of the beautiful uh coastline there and then visit the miss trail at yosemite park, as well as the famous famous golden gate bridge, but specifically from the battery spencer uh side of things.

A

So I'll pass it over to grant.

B

Hi everyone hi, I'm grant I'm a software engineer at a company called pure storage and I work on a product called pullworks. um Orex is a cloud native storage platform and specifically I'm the tech lead for the pork csi driver. I also am a contributor to kubernetes, specifically the kubernetes csi project and kubernetes sig storage as well, but I'm not coding or in the office. You can find me climbing mountains rock ice and snow. So those are some pictures of me uh out.

A

There where's that one picture where's this picture taken where you're on top of this tiny, huge uh mountain.

B

Oh, that's, that's in the french alps uh and then the one on the right is uh justin tahoe nearby, california,.

A

That's awesome.

C

And every time I see that photo, it's incredible. I love it.

B

It's crazy photo.

C

So nice to see everyone logging in here and joining us today. My name is uh adam barlow I'll, be the moderator. So I won't be speaking as much as uh my other two colleagues that are on the button, but um I love to travel. I've been to um close to 30 countries.

C

I have also been skiing, since I was four years old. So if I can travel someplace and ski I'm a happy camper, and during this uh like covet time, I've picked up um becoming a chef. So I'm uh this is like one of the my favorite dishes that I've ever made is a homemade pasta. So it only took seven ingredients and it was superb.

A

That sounds delicious and where did you take these skiing pictures adam.

C

So the two on the left, uh the selfie and the one with my wife, that's in tahoe, and that um one on the right, that's actually in whistler. So whistler is probably one of my favorite places to ski.

A

That's awesome, um everybody in the chat. Let us know where you're from or what are some of your favorite places to visit when.

C

You're in the bay.

A

Or wherever you're from.

A

All right, so what we'll be doing we're covering some topics that we we actually have a um demo, that we did a similar demo that we did for kubecon, so within this demo, we'll be presenting and talking about how you can test your cloud applications across uh using github actions um and kind to have a homogeneous testing environment overall. This was somewhat presented at kubecon as well, when we're taking that and making more specific to how you can leverage the specific github actions, tools that are available to you today.

A

If you pre, if you participate at kubecon, let us know if you have any questions, feel free to post in the chat and we'll you know answer them live if we can all right and I'll be kicking it off.

A

So, first we'll do a quick overview. Talk about kind, testing, automation and takeaways, we'll also being due we'll be doing a live demo, so stay tuned for some of the live action that will be happening there.

A

As you see throughout all these presentations uh kubecon, we were supposed to go to coupon europe, but it was uh cancelled or it was postponed and made virtual, because you know the pandemic. That's going on today, so you'll see a lot of pictures throughout of amsterdam just so, we can feel like we're hanging out there, while we're talking through all the cool fun kubernetes capabilities that we have today.

A

Awesome so we want to be able to collaborate successfully in both internal open source projects. That was our main motivation both grant, and I worked on a lot of kubernetes projects and we'll talk a little bit about that later on today as well, and we noticed that kubernetes does a really good job in maintaining a lot of things um very consistently.

A

But one thing that we've seen is that that's how it's successful? So how do we improve that? How do we maintain that? So what are the some challenges that a lot of open source projects and internal projects have been seeing?

A

Overall, collaboration needs to be more transparent, consistent and rigorous. We need to test our kubernetes applications consistently and it varies per developer environment because everybody has to set up a different dev setup locally, and then it takes up time and resources to constantly spin up kubernetes clusters. You don't want to spin up a full blown kubernetes cluster if you're doing some quick, end-to-end testing.

A

As I mentioned earlier, we'll cover some topics we'll set some steps to create and automate our testing environment, use, kind to automatically run our end-to-end tests across a common environment and then we'll be automating. The creation of uh these tests purple requests and run the test suite before even merging.

A

So I'll pass it back to grant to talk a little bit about kind.

B

Yeah, so uh just a little bit about kind, so kind is basically kubernetes and docker uh kubernetes, um since we haven't really covered it too much. Yet it's a container orchestrator, um specifically, you can run docker containers or any kind of containers that you have. So it's pretty easy to set up kind kind. Actually, just requires docker to be running on your system, and all you need to do is on a mac. You can type brew, install kind or linux.

B

If you have brew installed and then you could just do kind, create cluster and that'll bring up a small kubernetes cluster on your local machine, so yeah, if you can click on the kinds.sigs.ks.

B

This is the kind website, so kinda, specifically a a sig within kubernetes within the kubernetes project. So what that means is it's a special interest group? So, if you're interested in getting involved with kubernetes, you can join the kubernetes slack and join the kind channel. You can go ahead and check check it over there. So here's the github page for kind as well.

B

um So if you are interested in contributing to kind here's where you could go to see it, um so yeah we'll send a link if anyone wants to see that and yeah, so that's pretty much kind in a nutshell, this is the thing we'll be using to spin up our kubernetes clusters for our testing in this demo today,.

B

So you may have heard of a couple other tools similar to kind there's, k3s mini cube. Those are just a couple, others that are similar in usage. So what makes kind different is that it's just requires docker running on your system and the overhead is very small. You just need docker, so it's really easy to run in things like jenkins or github actions, or anything that you want to run um that you want to run your ci cd in.

B

So it's a great tool to basically spin up a easy kubernetes cluster uh mini cube is a single node kubernetes, also a low-buried entry, and then k3s is a minimal cut down version of kubernetes. That has some features removed to keep the footprint really small. So um we'll also, we have a link uh to this blog post right here, so um this is a blog post. That kind of went over a comparison of all three. If you are interested in seeing more of this.

A

For those on the chat, um if you have used any of these tools, what are some pros and cons that you've seen um if you haven't used any of these tools? What are you interested in uh seeing for another demo? Possibly awesome? So, let's couple: let's talk about a couple of our goals, so we want to run our end-to-end validation testing. We want to set up a pre-configured environment block and emerges to production if those tests aren't passing and automate everything, nothing should be manual because it allows you to keep it consistent.

A

We want it to be a little transparent or we want it to be very transparent and maintain that rigorous um testing, so we're doing it per pull request and whatever we are. Whenever we um are running our tests, we're actually hoping to block the merge, because we don't want to introduce any new bugs into our production environment. At the end of the day, we want to increase the frequency and maintain everything very repeatable to use kind in actions. We actually found a uh github action, that's already provided by the open source community.

A

So if you think of actions, think of them, if you think of actions as an open source tool, think of each action itself is an open source uh automation tool that you can use. In this case, I was able to find an open source action that does kind uh cluster creation, so I didn't have to do all the pre-work to create a new kind cluster action, because it's already out there and you'll be seeing this uh in the actual workflow, while we're uh covering it later on.

A

So, let's take a walk by the uh by the river here. um This is a beautiful river that is in amsterdam that we wanted to go visit while we were there, but it was something that we could not do. So I decided to build our workflow pipeline across this, so our kitty cat dev, will be introducing code change and creating a pull request in this overview.

A

In this pull request, she'll be introducing new changes, so we need to run a build and publish docker image, just to make sure that all those changes are being leveraged and then to really test the changes that kitty cat dev made. We want to jump into a kind cluster creation and run a kubernetes end-to-end testing, and what we also really looked are planning to do is testing across all the kubernetes or the latest kubernetes environments that we have available.

A

So today, we'll be testing against uh 1.171.18.1.19 and once all the tests pass we'll be merging the pull request.

B

Cool, so if we want to run some tests, we need something to test against, so uh we'll be testing against the product that my company builds called pull works, so works is a storage platform for kubernetes. um Specifically, it offers things like storage disaster, recovery, migration, autopilot and security. So just a lot of automation and devops and intelligence around storage within the kubernetes ecosystem.

B

We also have a backup product called pxbackup, so we'll specifically be testing something called the torx csi driver which I'll talk about in the next slide. So yeah, I mentioned pork security. This is the main feature that we are going to be testing. This feature prevents all unauthenticated users from accessing our platform, so pretty straightforward at a high level um and then we'll be using something called the forex csi driver to do that. Testing.

B

So csi stands for uh the container storage interface and is a project maintained by a lot of folks that work on different container orchestrators, so dcos kubernetes and a couple other container orchestrators are all working together to make this common storage interface and kubernetes has its own implementation, and that's the team that I that I work on um the open source side of things so yeah. So we're going to be testing the pork csi driver um and we're going to be using something called open. Storage. Open storage is the open source control plane for portworx.

B

It's also where the csi driver sits. So that is the code that I'm going to be demoing today, fixing it so yeah. This is the uh the e2e test that we're going to be running. So let's go ahead step by step, so first we're going to create the kind cluster. Then um you can click to the next one, so yeah, then we're going to deploy the csi driver, and so the pod is going to come up in the cube system.

B

Namespace then we're going to create the token secret and then next, what we're going to do is create, what's called a storage class. So a storage class is basically a template, for a volume basically tells uh the container it tells the storage driver, what basically, how to create the volume and what kind of properties you want for this volume.

B

So yeah, then we're going to create the persistent volume and then next we're going to create a pod that uses that volume so for this demo. What we're going to do today is use mysql and we're going to have a volume using we're going to have a volume, ports volume, that's provisioned by the porrix csi driver, and it's going to be used by the my sql pod. So I know that's a lot of words to take in if you're not familiar with kubernetes, but.

D

In a nutshell, what that means is that we're going to be running an application that application needs storage. It needs persistent storage um because there's data that we want to hold so we're.

B

Going to use what's.

D

Called a persistent volume we're.

A

D

A

Awesome so let's jump into the demo.

B

Yeah so I'll share my screen and what we can do is go into the live demo. So what we have here is vs code, so we actually have the open storage code base open right now. So let me actually pull up open storage, real, quick and I'll. Just show you what I mean by that.

B

So here is the main open, storage, repo. This is the open source control plane for portworx. um It's just a go: go repo with a lot of different apis, a lot of co code for authentication uh migration, backup that kind of stuff. So, if you're interested in how that kind of works, you can check out the github repo at github.com storage- that's open storage, but specifically we're going to be using a fork for this demo because we didn't want to mess with anything on the main repo because we wanted to work quickly.

B

So if you are interested in checking out a lot of the demo code that we've been showing today, you can actually go to github.com escalife, openstorage and um yeah. So there's some helpful links there as well for you to check out that we've talked about today and yeah so now that we've kind of gone over the repo that we're going to be contributing to let's go ahead and get into the code. So I have some code here. It looks like screen is big enough. Let me know if you need me to make it bigger.

B

We have some code here um that I've written last night really tired when I was developing. So let's see if everything is good, so we're just doing. um Actually, let's take a look at the branch name. The branch name is just go lint fixes. So what my task here today is. Basically I have a bunch of like linter fixes that I need to do just to clean up pr to kind of clean up the code base and make it look a little nicer, so go ahead and check out what changes I've made in the diff.

B

So pretty straightforward changes to some renaming more renaming and import fixing a log line and cool. It looks like we are good, um let's go ahead and commit all of these changes.

B

We're just going to do git commit dash s, sign the commit and we're just going to do fix some bullet issues.

B

Very innocent pull request, nothing wrong with it. So let's go ahead and create this pr, so I actually started using the github cli this past week, so um it's actually pretty easy to use. If you can just you can just download it and, uh I believe, there's a brew command to install it, but we can include some docs about that in the end. But let's just check out what the github cli can do. um Cli can do a bunch of different things.

B

It can you can list your gists issues, prs releases, repos and, but specifically, what we want to do is to get pr create. So now that I've made my branch I made committed my changes in the branch I can go ahead and create this pull request.

B

So it's going to ask where we want to push this branch to we're working on sarah's fork. So what we're going to want to do is push to that fork and the title you can just keep it the same as the commit for now and nothing really in the body is necessary for now, so we go ahead and submit this pr, so cli is doing now is sending a pull request to our fork, and so it's sent the fork and everything it sends the pr.

B

So we can do is pull this pr up in the window and cool. So my pr is submitted to our open storage fork. We have some very straightforward changes. Like I mentioned, just a lot of go lint fixes pretty straightforward, stuff um cool. So now we have some tests running and uh yeah. Sarah did you want to go into what exactly is happening now that I've opened the pull request to our repository.

A

Yeah, so what I'll do I'm actually going to go over the workflow? So if we can switch back to my screen here, we have our github repository in order to build a github action. um If you are familiar with github actions, let us know in the chat and we'll cover some even more in-depth topics. But if you aren't familiar with github actions, you can actually go to the actions tab and then I actually have a couple of jobs running, but what you would normally do.

A

You can just click on your workflow when you click on new workflow. It automatically detects that the library that the repository is using go so it says hey. Do you want to just build your go pipeline and set up this workflow?

A

If we're doing a little more cd, we can also deploy to all of the different different cloud providers, but we have a lot more ci integration flows, um continuous integration flows that really allow you to build uh more end-to-end uh testing, but what we've done because we wanted to use kind these, so these are star workflows, developed by our ecosystem, so you can take these and customize them or you can take them and use them, as is you can easily deploy to azure building a docker image and everything, but today we wanted to use kind action, the kind action, so we actually built a end-to-end kind yaml here within the actions yaml you'll see that we have a trigger it's so it's being triggered on and let me zoom in a little, so people can see you much better.

A

um You can trigger on pull requests uh previously. We actually had this trigger on pull request and on every push, and we weren't specifying the branch, but in reality nobody needs to run end-to-end testing every single push it makes um it makes unnecessary work. I think it in overall it uses up resources that you might otherwise just only need on pull requests, so we actually cleaned it up a little and made it only on pull requests specifically to our production branch because end-to-end testing we don't like.

A

I said we don't want to test that every single push but unit testing makes more sense. If we wanted to test every single push a unit test that would we have for open source that could work, but for end to end it only makes sense to do the end-to-end integration testing when we're pushing to our production branch or our active development branch.

A

So we have a couple of environments set up here, so if you notice we're actually using a ghcr.io, so if you're unfamiliar with that github container registry was recently announced in beta so under github packages, so you have maven, you have nuget. You have a couple other type of package container package registries. We now can do container registry as well. We could do that before, but now it's more focused on building out a, uh I would say, end to end where it's not just docker images.

A

It could be any type of oci type container so with that we also, if you notice, we're setting the run id as the version of the of this uh image. Well, in this case, we wanted to keep it simplified and we didn't want any of the images to collide in case we're running concurrent jobs. So every time a new run is spun up, the run id is used, so this allows us to make sure that, for this specific pull request, this is the run that this is the image that is being built.

A

If somebody introduces a new code change, that will be a different run, but if you're just restarting this workflow, if you're just doing a rerun all workflows for the specific one, then you wouldn't even have to change the run id again. This is an environment variable, so it automatically does that for you.

A

What we also have here is we have jobs, so you can have a couple of concurrent jobs you can have so jobs are concurrent uh and asynchronous, and then you have steps that are very current and in a specific order.

A

In this case, I have a build job and a test job in this build job, I'm actually building out the docker image and then pushing it to our github container registry, we're doing a simple, build, docker image, we're doing a make docker build, so we have a make file and a docker file that we're compiling, and then we have our docker login very common to if you're familiar with docker. Overall, it's the same way you can log into github container registry.

A

The really nice thing about this is that it uses the github's uh github's user management. So you don't have to go, create a new set of credentials. You can use your existing credentials today if you are an enterprise user. This comes with, I think, 50 gigs of packages available, even if you're an open source contributor. There are packages available for you today, so all of this is available for anybody to test it out.

A

We're also doing an image push here. So simple, simple overview of this: we are tagging the image with the specific test id image, name that we provided at the environment variable here and then we are doing a docker push image so now that image will be stored in github container registry.

A

Here we have our test job, so this is usually jobs can run concurrently, but in this case I really wanted these test jobs to wait for the build job to pass, because it doesn't make sense to spin up tests without having the docker image pushed.

A

So what we have here is also a matrix build, so you can see we have labeled here: okay, it's version 1.17, 1.18 1.19.

A

This allows us to spin up three concurrent jobs to run our tests, so with that we were able to use this variable and put it in our kind action that we were leveraging from the open source community and spin up this job three different times with all kubernetes versions.

A

If you want to see how this action runs, you can obviously it's fairly simple here. You can see I'm setting up kind and spinning up the cluster, so it's using the we're naming it the open, search cluster, it's using our yaml configuration here and we're spinning up the kubernetes versions here and then we're using version 8.1 for the kind, the kind version.

A

If you want to see specifically what this kind action does, you can actually take this um this uh url and you can put it at the end of github.com and pull it up. So all of these actions are open source. You don't have to reinvent the wheel. You can just work on something that is actually differentiating and leverage existing tools. So in this action, if you look through all the readme section here, it gives you specifics on how you can set it up what you can configure, what tools?

A

uh Sorry, what um variables you might need and what are the some of the best practices, and you can do that with almost any open source action. If you wanted to find more actions, you can actually go to the marketplace and talk um and uh uh uh pull any actions from there.

A

So with that, I'm we'll uh well in this case we're starting. So we had our kind job being uh spun up where it spins up the full kind cluster which is kubernetes and docker, and then we start our open storage uh configuration and then we do an environment check here and then run our end-to-end tests. So overall a simple workflow and I will let grant talk to it in a second and I just wanted to go back to the pull request and see what's happening huh interesting. So we can see the job failed grants.

A

uh Let's go check what where it failed. So we can see the bill job pass. So it's not in the build. So I guess it's when it's spinning up the kubernetes cluster. So let's see here, if you look through all the logs are available for each of these jobs that were spun up. So in this case I'm looking at the one that was spun up for test um for version of granny's 1.17 and if I zoom in a little here, we can see that there's an on unauthenticated interesting.

A

So normally I could just tell grant hey you broke this, but in general, remote collaboration isn't as easy as that. So what you can do is actually copy the link to the actual error. So all of the logs are linkable and that's a super. I'm telling it's like a super convenient feature because when things break, you just want to make a comment and say: hey: this doesn't look right what's happening, so I'm gonna go at my review. I'm gonna comment and tell grant please look at this error because I don't understand why it's.

A

A

Then I can just comment so in a remote world: grant will get a notification and see that I commented hey the air failed and then you can click on this url. So now that the error failed I'll pass it back to grant. So you can actually fix this issue.

B

Cool yeah so uh looks. Like sarah commented on my pearl request. uh I can start sharing and I'll go ahead and take a look so yeah it looks like sarge has commented, let's see what she said. um Oh my also looks like my tests have been failing. So let's go ahead and see what sarah has found.

B

Okay, so looks like uh the porix csi driver is not able to authenticate with the back end for bulwarks. It looks like poor works is denying the request. So good news is work. Security is working. It's not letting people in, but bad news is users can't create their pvcs in this case. So let's go ahead and take a look at my code. Again, let's see what I have here.

B

Some golan fixes bill and fixes more comment fix. Oh looks like we have a bug that I introduced as part of this pr, so it looks like I added an equal sign back sentence. I must have just typed this by accident, my editor or something so let's go ahead and fix that. So let's go ahead to our csi.go, so okay looks like we have bearer equals. Let's fix that.

D

B

We can do is just do the same thing. Submit our changes.

B

Expert token issue so just add a new commit, so it's easier for sarah to see what I've changed in this pr instead of uh create, instead of committing to the same commit and at the end, what we can do is squash the commits into one when we merge, so that it's all merged as one feature cool so um oops. I pushed to the wrong.

D

B

That's fine push to sarah's remote.

B

So we're adding the new commit on top of this pr that we've already opened, so we can go back to the pr and see two commits have been added. I just added the new one, and now the tests are running again. So speaking of tests, let's actually just go into the test code and check at what exactly we are testing. So we have the csi driver right here.

B

This is the thing we're actually testing, um specifically we're testing that when kubernetes passes the secret, um when you basically a user, passes a secret in, as we mentioned before, that token will actually get put into the headers and will be sent to the request to poor works. This is how porwick's authentication works. It basically looks at the authorization header and validates that your token is the correct one.

B

So now that we've kind of gone over the code, let's quickly go over what exactly is happening in our test.sh, so to give the full picture. Actually, let's first go to the workflow, so sarah mentioned the workflow, I'm just going to pull it up so that we can really easily tell so. The build job happens. First, my build was passing just fine, so that wasn't the issue she was actually in this test job. So basically in every kubernetes version you noticed it failed.

B

um What actually happened was inside of our integration test. This demo slash test.sh.

B

We tried to create the pvc, the persistent volume so that our mysql pod could use it, but unfortunately, because of the bug I introduced, it wasn't able to do that. So let me go into the test.sh code and just see what exactly we're doing so here is. uh Let me turn word wrap on. So here's what we're doing in the test.sh. This is a very straightforward demo test.

B

It's you wouldn't probably actually want to write tests like this in the real world, but for demo purposes it's a nice little test suite to run so the first thing that we're going to do is define a token now before you go ahead and copy this long, jwt token down. It's not used for anything special, so no need to put in that.

D

B

Not a good practice to commit this token like this. This is just a test token for running our tests. So next, what we're going to do is we're going to create that kubernetes secret, so cube cuddle create secret. This actually creates an encrypted uh secret stored inside of kubernetes, and what it'll do is it'll.

B

Just do auth token equals, so the the the key for the token is off dash token, and the value will just be this token that we have right here next we're just going to get that token secret and make sure that joker is created successfully and then what we're going to do is create a storage class.

B

Let's go ahead and look at this storage class to give you an idea. So this is what a storage class looks like in kubernetes. As I mentioned, it's kind of like a template for creating volumes and volumes are what hold your data in kubernetes.

B

So, if you're using mysql, then you would want to use a volume to store your data.

D

B

Sure that it's persistently stored and not taken away when the pod goes up or down so here's kind of what we're doing in the storage class. We have a reference to our token secret that we created, and this is just the way that csi actually um allows you to it- um allows you to pass token secret. Allows you to pass secrets into the requests that we're talking about.

B

So specifically, the provision or secret name is token secret and the namespace is just the default namespace.

B

So let's go back to our test.sh, so we got the storage class just fine and then next, what we're going to do is create that volume. This is what this line does right here. If we do, q put apply on our pvc.yaml that'll actually go ahead and create this yaml. So this is a persistent volume claim. It basically has two gigabytes of storage and it uses that storage class that we had mentioned before.

B

So it's saying hey, I would like a persistent volume of size, two gigabytes, but please use the settings from that storage class that we've already created.

B

So next, what we're going to do is get that pvc and make sure it's bound. So this is actually the line. That's been failing so because this request, right here to create a volume, is protected by px security.

B

um What was happening was that when we tried to do the pvc creation, it was failing because the token wasn't being passed correctly to the porex off back end, so this line was failing, and you may have noticed this line right here. It was failing specifically and forget about this line. I wasn't even able to start using the mysql pod because the pvc wasn't created correctly, so um that's kind of what the test is doing.

B

Let's go ahead and look back at the um let's go ahead and look back at the pr and see what's happening there. So it looks like my test is still running so yeah.

D

B

Yeah sarah, do you want to talk a little bit about what's happening right now,.

A

Yeah, so we are, let's see: can we share both screens at the same time? Okay, so we can't so it's okay. So what you can see here is actually, if you wanted to follow the logs along the logs, you can click on details and pull up the um the steps that are being taken. So you can see here where we're setting up kinds similar to what we were showing that diagram earlier, as well as running our open storage setup.

A

So, while these tests run I'm going back to our conversation here within this conversation that you can see, these tests are required. Who knows why these are required? um Does anybody if you do? Let me know in the chat and then I'll jump into how I did it and why it's important.

A

All right, so these tests are required because I set our production. I set up production branch as a protected branch, so you can go into your settings for your repository and you can decide on which branches you um see value in blocking, merges or causing or enforcing some sort of policies. In this case, our master branches are product, a production branch. So I really don't want anybody merging there unless they had somebody code review.

A

I do a code review as well as all as all of our test, passing in this case we're testing again three different kubernetes versions, as well as our build job, so at any given point in time. Anybody that creates a pull request into our master branch needs to have all of their tests configured correctly and all of their environments. Configured currently and any code changes that they made to not break the tests.

A

So if these tests don't pass, we can't merge the pull requests. In addition, your github sorry software engineering overall is more of a team sport. You don't necessarily sure you could develop a couple of features together, but you really, if you're working in an environment where um there's a lot of collaboration, there's a lot of other team members that can do code reviews for you as well, so going through your code with just a couple. Other people allows you to really understand what's happening and share that allows.

A

The other person understand what's happening when you're introducing these new code changes and provide feedback and have a conversation of why these code changes are necessary and why you implemented them in a certain way. So this requires a pull request reviewer. So if I'm adding any code- and it does for some reason- it doesn't make sense. At least somebody else can see it and say hey.

A

This doesn't make sense, um but at any given point in time, if anything that will uh be a breaking change, your test will be, um uh will notify you right off the bat.

A

So, let's go now that we've seen our required. Oh on a couple. Other things that you can do, if you haven't noticed, is to disable or you it's automatically by default. Disable disabling force pushes and deletions for that master branch.

A

uh Previously I worked on a team where somebody decided that it's okay to delete the master brand for some reason and there was a definitely a a huge uh slowdown in our production and our print and productivity that day in general, you don't want to delete any branches that are your core production or um you know consistent development branches, uh smaller branches, they don't. You know like the golan branch. It doesn't that's not as required awesome.

A

So now that we've covered some of these protective branches, let's go back to the pull request and actually see if the tests have passed all right. We see a little check mark here and if we scroll here sweet all the checks have passed. But again the purpose is still blocked until a reviewer since grant created this pull request, I can actually go and review it and provide my approval. So let's looks good to me. Let's ship, it.

A

All right so now that it's approved by the one reviewer and all the checks have passed grant has the power to merge the pull request so we'll pass it back to grant to march that pull request.

B

Cool so yeah sarah mentioned uh go ahead and merge.

D

B

So merge it. There.

D

We go, I have my.

B

Feature added to our fork of open storage.

D

D

A

So a couple of the other things we wanted to just touch upon is some of the lessons learned.

A

Overall github actions allows you to do a couple different things and if we can go back to uh my screen as well I'll just cover some of the main topics that we touched on. So we took that walk by the river we had our pipeline. We did our builds and pushes and everything so some of the things were to know that we triggered this event on pull requests. This event.

A

uh We didn't really need to run the integration test end-to-end for every push, so it only made sense to run it on a pull request specifically to our production branch, or it might be specifically to your deaf branch.

A

We kicked off our build job first and then broke down and broke down our testing job and had three jobs run parallel in a matrix, build the great thing about originally. What we did was actually have our build job and our test job all in a single job, and it was convoluted.

A

So what happened is every single time I spun up a concurrent parallel uh matrix, build a build job was happening, so we had three build jobs happening all at once, and it just didn't make sense so splitting up these two, especially because overall, the strategies for one and the other are different, uh made it much easier to set up a workflow and, last but not least, protective branches.

A

Let me tell you this saves you so much uh worry and concern in case somebody does not follow the best practices right off the bat, but that's fine mistakes happen, and you can block these mistakes from happening uh with your branch protection rules. uh These definitely have to be taken in mind where which branch you're using branch protection rules. You don't need to set these up for every single branch. It only really makes sense to block the specific things. For um you know the specific scenarios that might be detrimental for your production.

B

So sarah, I noticed on our workflow that we uh we only test against.

A

Kubernetes 117.

B

118 and 119., uh let's say a customer of forex, wanted to run on kubernetes 116. and we really wanted to get some test coverage. How might you add that to the matrix job.

A

Oh, that's a really good question. It's actually super easy to test this in a different kubernetes version as well, just because we set up that matrix environment variable right off the bat. So if it's a simple change like that, I don't even need to pull the uh branch down and do it locally. I can just edit the file in the ui as well and edit it here in general.

A

If I just wanted to how to add a whole new action, I can actually search for a marketplace uh for the marketplace here and we can see that we're using uh actually we're using the engineered kind action and we can see helm actually created a kind action as well and there's a couple different things here.

A

There are some documentations on how to configure your environment, your workflows within the same setup, environment, variables and such within the editing itself. So it's really easy, but since I'm only doing a version change, let's just simply do that. Let's test this live, we have not done this before. So, let's see how it turns out.

A

All right, so I created a v1.16.15. I believe. That's the latest uh 16. That's the latest miner uh fix miner patch for 1.16, so let's do a pull request and update it. Okay, it's virgin.

A

Perfect, so I'm creating this pull request now, so I don't know if you also notice, when creating a pull request, there's some templates that are done here, so you can set these as best practices from the start. But uh since this is just a demo, let's just skip these and say updated. K8 versions include 1.16.

A

I'm creating this pull request and the really nice thing about this. I just made that simple change that one line change and now I can test on a completely different kubernetes version.

A

Hopefully everything will pass, but let's see what happens, and we can go into our actions pipeline to see that in action, um my corny jokes aren't always the you know the bestest, but so we can see that it's actually just spinning up a build job similar to what we have before and then, once this build drop passes, it will spin up the three, uh the four different kubernetes versions, uh jobs for those four different kubernetes versions and so yeah.

B

Cool, so while this is running sarah, we also want to test against different cloud providers. Actually, so we want to make sure that our coverage is there for all the different uh cloud providers. I want to make sure that what works works on every cloud out there uh as part of our business statement, so I was wondering if you could show us any any way that we could kind of deploy this to any cloud provider.

A

That's a really good question grant yes. So, as I touched upon at the beginning of the session, you can click on this new workflow tab um within your actions. Workflows, if you didn't, have any actions before it's super easy, just it'll pop up pop up right away, but in this case since I already have some actions and I want to add additional workflows, I can set up an azure web app deploy or amazon ecs or gke.

A

In many cases these are some of the four of the three core uh cloud providers that you want to deploy to, but it makes sense. uh I would say some of the lessons learned work great. But what are the next steps for making our workflow a complete end-to-end test? uh Not only do we test it in the different kubernetes environments, but then, finally deploying this into a production environment, testing it against the different clouds. So this really sets a uh easy, I would say workflow setup.

A

That gets you started and if it doesn't necessarily meet the exact standards that you want, you can always customize them. These are really nice because again it's they're developed from the ecosystem. So you can treat them like open source code. They are provided from amazon themselves from azure themselves from google cloud themselves, so they're setting the standards and we're just allowing the standards to be set across the board and providing these as like template, starter workflows.

A

What's really nice actually is the terraform integration as well. So if you have any infrastructure built out with uh using terraform the terraform cli, you can just run your github actions with terraform and run your terraform applies within github actions and set it up from that. What that really allows you to do is move slowly slowly to that get offs mentality where things are triggered, based on git events and rather than manual triggers or automated triggers that are not necessarily standardized across the board.

A

So having these capabilities and setting up these workflows make it very easy to just kick off your deployment pipelines as well, and a lot of new features have been coming out about the uh for these, and you can actually check out all the new features that we have. If you haven't done this, yet you can go to the github roadmap and it is public and it does it in the most githubby way possible oops.

A

It doesn't in the most good heavy way possible and if you go to check it out, um oops- and I have to do my single sign-on- but um let's not this- is there- we go um we're very secure now, so we have to do our biometrics as well cool. So now that that went through this is actually public. So uh you can actually check out the github roadmap and see exactly all the new features are coming out.

A

A lot of our capabilities are called go to cloud, so you can actually search for specifically um the tags that we have for all of our environments. You can set up, you can click on cloud as a label. You can talk about github enterprise and you can search the roadmap, but anyway I've talked a lot, so I actually wanted to bring it back to grant so grant. How did you get involved in kubernetes?

A

What made you uh you know what introduced you to it and how did you get so involved and even contributing to the open source project.

B

Yeah totally um so I got involved with kubernetes through my um through my company, so at full works. There are a couple features well to start. We works as a company uses kubernetes a lot and we develop a product that works on top of kubernetes and helps connect different kubernetes clusters. So we wanted to get involved in the community and contribute back and add a couple features that we also needed out of kubernetes.

B

uh Specifically when we were adding our work security implementation, we needed to add a couple things into there to actually um make security work within kubernetes uh with a uh with csi volumes. So specifically for each csi request. We wanted to be able to pass in token secrets and, as I mentioned in the demo, and uh that needs to be done through the csi spec, so that's actually something that we worked with the team with to to kind of get that working.

B

So that's kind of how I got involved um specifically, I I helped out with the feature in kubernetes called kubernetes snapshotting, so that's basically being able to snapshot and protect your volumes um in case something bad happens. You will have a snapshot of your volume that you can restore from, let's say like if there's a server outage or a disk failure or something um and you need to restore from a snapshot, then that's how that's the feature that you would use out of kubernetes. So I helped with uh yeah.

B

I helped with some of the testing and some of the implementation for some smaller features, I'm about like 50, 50, half half working on my pork, stuff and half working kubernetes stuff, but that varies greatly based on kubernetes releases and forex releases, and so that's kind of how that works.

A

That's a pretty cool rule to have to be able to contribute to both your internal projects as well as open source projects.

D

A

You have examples of what that kubernetes testing looks like.

B

Yeah, let me share my screen actually and I can go over how kubernetes kind of uses a similar model so uh yeah. I can share my screen and show you all what.

D

That looks like.

B

So I have uh the tech kubernetes test grid right here. So um actually let me go to the kubernetes cty.org and show you some of this in action. So this is all the technology behind the interaction between kubernetes seaside and the pork csi driver. So these are all different sidecar containers. If you're familiar with kubernetes um most of it's written in go so let's go ahead and open up one of the sidecars.

B

We can look at the pluricasts open, so a whole bunch of pull requests are open now and they use kind of a similar structure that we talked about in the demo. In fact, a lot of the um a lot of the ideas we talked about in the demo are are inspired by how kubernetes does things. So that's one of the nice things about working in open source is that you could look at how other projects are doing their development and kind of see what you can kind of adopt as a result. So let's go ahead.

B

This is just a uh a let's look at this pr, for example, this is just a read me update. I think uh nothing like too intense. It looks like some reviewing happens, but at the bottom you can actually see they have a very similar structure that we have. So this is where we got the idea from the inspiration from they run a they run. This sidecar go code on different versions of kubernetes, so they make sure that the code that this person sent as a pull request is actually working on all these different kubernetes environments.

B

They initially run unit tests. They make sure that they've signed a cla, so the this books cannot be merged until there's approval. There's the cla signs, all three kubernetes jobs have passed and then the unit tests have passed. So this is kind of that rigorous type of um testing and pr validation that sarah talked about in the beginning. That really is important when developing a high functioning.

D

B

Product so that's kind of how csi kubernetes csi works from an open source perspective and.

A

B

And from like a cicd perspective, um and now we can go to the test grid, so this is actually where the test results actually are, um but I think the plural questions.

A

B

Yeah, I think the pull request did a better job of showing showing all the different versions, but uh here's all the history, um this isn't through github actions is through. I think I forget what they're running on, but it's a similar model um that we use as well. So these are all the different tests that they're running so very thorough testing and, as you can see, uh some have failed in the past and but most.

A

B

Pretty green, so I won't go too in depth there. It's kind of complicated but uh yeah, so that's kind of how kubernetes csi uses this same model um and just a quick shout out to kubernetes slack. If you are interested in contributing to csi very sorry to kubernetes, you can go to slack.kubernetes.io and you can join the kubernetes slack. You can go and find out how kubernetes is being developed. You can go join any slack channel. You want it's all open source, it's all open to the public.

B

There might be some private ones for people that are maintaining certain projects, but everything's pretty open and you can go and check the kind. For example, you can go and check the kind kind slack channel, so I'm actually a member of kubernetes org and kubernetes slack, and I follow kubernetes uh kind as well. So here's the slack channel for kind uh people are asking about how to use it might have pull requests or issues. So this is kind of how kind is developed through this slack channel and also a lot of work behind the scenes.

B

Of course, a ton of work behind the scenes to make kind possible, but this is kind of the main spot where people go and ask questions um other than github issues and mailing.

D

B

And that kind of stuff so yeah, similarly with csi as well, you can check out there so a quick shout out if you're interested in contributing um there's a lot of good uh videos out there as well. If you just search on youtube, how to contribute to kubernetes um and sarah, I think you also wanted to show how you might be able to find out issues that you want to work on for kubernetes.

A

Yeah- and I know we have like one minute left but I'll just show it really quick on what that will look like. So, if you go into the kubernetes project, you can actually search for specific issues to find um which ones are good to start working on and if you're new to open source or just new to kubernetes, you can look at the different labels that are out there and if you just search help, there's a couple of labels that are really good to leverage.

A

It's the help wanted label and the first issue label these labels, and um these labels are usually associated to issues that can help uh you get started within the open source community and overall, the or overall, the open source community, and even specifically, for the kubernetes community. But with that I will be. I think we are at the end of our time, and we hope that you have learned a couple different things that you might not have otherwise known.

A

um If you have any questions and want us to do any, um uh you know, other demos feel free to comments in the chat and we'll take all your ideas and we'll talk about them within these devops demo days. uh At the end of the day, we are partnering with our community are partnering with our not only open source community, but our enterprise customer community as well we're helping to build these conversations and help build out our community even stronger.

A

So let us know if you have any questions or if you have any ideas and thanks everybody.

A