Argo CD and Rollouts Community, 5 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Argo CD and Rollouts Community Meeting 5th May 2021

Description

No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).

A

All right, hello, everyone and welcome to the may 2021 argo cd and rollouts community meeting, uh I'm your host jesse suen, I'm a principal engineer at intuit and one of the maintainers of the argo project.

A

We have a great agenda for today today, code fresh will be presenting a project that they've been working on and we've been consulting with called argos cd autopilot, which is a opinionated way for you to manage your argo cd application installations through in your git repositories.

A

Just a reminder: these meetings are recorded and will be uploaded to youtube before we get into the autopilot just a quick announcement about. Let me share the screen: a quick announcement, um argo rollouts, the first release candidate for argo rollouts- has been released last week, um so we want to make this available so that we can get some big time in the community.

A

It's been installed in the half. A dozen clusters inside are at intuit so far so good, and um some of the kind of highlights of the release are that we do have a new ui, which we um I think we presented this previous meeting this one we might not have presented or demoed yet, but with rollouts, it's now possible to reference existing workloads instead of inlining a pod template a whole pod template.

A

So if you are migrating from say a deployment, you can actually leave that deployment in your namespace scaled down to zero and actually reference it using a rollout. So this is a new feature in 1.0 and then there's some more.

A

We have support for ambassador there, which was also presented a while back and the ability to do some different canarying in istio using destination rules.

A

So I encourage you, if you're a rollout user, to give this a try and within hopefully in two weeks or so it should be um generally available. um Okay, um that's all I had um it's. I do you want to take over.

B

Yeah, thank you, okay, so I will share my screen and second, okay, so, and so we've been working in code fresh for the past one two months on a project we called it. The autopilot I was the autopilot.

B

The main motivation of the autopilot tool is to provide an opinionated way of installing argo cd in a github's approach way and also to provide a github's approach way to fully maintain applications that are eventually deployed on multiple environments and to do all of that in a completely itops approach in idiomatic way means that you can potentially delete entire clusters and be able to install it again and- and this is basically the autopilot tool and what we will do today.

B

We'll show you a very nice demo that the team members have prepared it's basically a summary of a blog post. We already created that you can check out later and afterwards, we'll move a little bit to some q a session and we'll show you the roadmap and we'll be happy to to answer any any other questions. So and I'll move we'll move to norm which will show the demo.

C

Okay, hello, everyone.

C

Okay, hello, I'm going to show you the the quick demo of how to create the github's repo and how to manage it using that gaussian, sili, xd5, sorry, um okay, so first thing I've done before I I've exported a git token. So basically I have it already, but you just export it from from your github code to support github and then all the commands will ha will use it automatically. It's not a mandatory step.

C

You can always just support the token on each command and the first command I'll just create a new repository, and this is we'll just create a demo under my name in github.

C

Okay, so this is this command is basically uh not doing much key tops or anything it's only. The only thing it has done so far is just an empty repository completely empty, but now this command sorry now I'm exporting the repo as well, and this is doing all the magic.

C

This command will um install argo cd on my current cluster record and connect it to a gke cluster. It will create some photo structure on the repo that I just created, and it will also create the applications over there. One application will manage diagon city itself, and another application will be used later to create more projects and applications and also by the cli so running. This.

C

It's basically cloning, the repo it's doing some stuff over the on cluster, and apart from the bootstrap, the the other commands you will see later and most of them do not require access to the cluster. They will only work on the on the git repo and the few commands that do currently require require the cluster. We are working on, improving that and making everything work only through githubs, so our aim is to have only the bootstrap touch the the cluster.

C

This is the what we're going for so currently we are. um We have applied the aggro cd and application set on the cluster. Now we're waiting for the server to be ready. We can see here the folder already starting to to run. Everything is running.

C

This is the applying some of the applications, and now we fell back at the ripple. I can see this this basic structure uh over here we have uh the bootstrap folder. This contains two applications. Two argo cd applications, one application, the argo cd application is referencing, this folder, and over here we have. We have a customization, basically installed installed argo cd, and this repo has argo cd and application set, and it also creates the namespace.

C

So we have it here and the second application that we are installing and is the root application, and this one is referencing this folder, which is currently empty. It only has some md file. You can read here how to create more projects that I'll show in the demo as well, and the third folder we're creating is the customize folder and again this one is currently empty. This folder will hold all the applications we will deploy to our clusters.

C

So again we have some information, but there's no real information, no real manifest in here currently and I'll also run this spot forwarding here.

C

And this is the initial password and I can see the address in the ui and see those applications, so we have two applications ago, cb and root, and we have the third application is the bootstrap application which actually looks at the bootstrap folder.

C

So this is the only application we applied directly to the to the server to the cluster. So we apply this one and it caused the other applications to to be deployed.

C

So this is the state correctly after the bootstrap, and now I can start creating the project.

C

So here I'm creating a project called prod or just just the name. Basically, an in autopilot the project is a has tourist resources, it's an argo cd project and an application set as well.

C

So now we have to refresh I look at the application and refresh it.

C

Here we see two um resources that basically the the autopilot project, but still we still don't, have any applications, and now we follow as an application. So now an application and give it a name.

C

I give it a name and I give it a source and basically we are just creating a customization file that references this, and this is the project I'm adding it to.

D

C

Now we should look and refresh do it here. This is the application we just created and it's now getting deployed to my cluster.

C

The course forwarding isn't working well, okay, so it's getting deployed to my cluster and if I look also in the repo I can see, but this is the project I created earlier. So this yaml has the app project, obviously the app project and application set and if.

E

I'm looking at the customized.

C

Folder, I can see the app I just created and it has a base which this points to the to this. So the base is referencing, the the parameter I supply and the overlay which is currently we only have one overlay, is referencing the base and over here I can add more stuff, add patches or images and show you later.

C

But this configuration is what's creating the application using the application set, and I can also see.

C

Something it's not very well here with my ui.

C

Okay, it's not loading.

C

Okay, so now I'm going to add.

E

C

Project and this time, I'm adding a project which will be connected to run on a different cluster completely. So it's not I'm running on this one. This is mine and I'm running on a different one. I'm going to create a product on a different one.

C

um Currently, this command does require access to uh to the argo, cd and.

C

Okay, it's working and now we can add an application.

C

So I'm basically adding the same application because it has the same name, but I'm editing it to a different uh project. So now, if I look here and refresh the root.

C

Application here, so this is the staging app and over here we can see it's in a different destination. Cluster.

C

And again, it's just being displayed deployed over there and everything is managed through the githubs every all the the cli everything you did was commit here. So now in overlays I have two different overlays one is the project project one is for staging, and over here we can see how it's uh directing that application to be on a different cluster.

C

Okay, so now, basically no it doesn't work here. So I have those two applications and two projects. Two clusters. I can also.

C

Have you can see the list of my projects and I can also list the app in a specific.

C

C

Okay, so here's just the list currently on the staging project um you can. I can also create an application on a different cloud. The project, the project cluster, is just the default cluster for applications.

C

um So now that I have two different applications that are like on different projects, I can start adding uh here I can add, maybe a different image for for this overlay.

C

This is just basically, I will override the image over there.

C

E

I'm doing is a.

C

Simple commit on git like any other usage of ocd, but we hope to be having these kind of features inside the door. Paracetamol itself so it'll be easier to do.

E

This kind of thing, but.

C

Here, it's thinking, it's creating a new.

C

Service, of course, I can also just edit the the base, if I want to, and then it will uh affect all all the different uh projects at the same time, if I'm adding it here, it will affect both the staging and the production. So, and I can do anything, I want in either location and affect all the different like.

C

If I need a specific project or other projects, and basically that's it, we have some we're working on some more commands to delete projects, delete applications and then we want to also have the the power to promote environments from staging to production or from testing to staging or whatever.

C

So this stuff we have. You know.

C

Basically, that's it.

C

This is the demo.

B

Thank you. No, so I I saw a few questions in the chat, so I we'll be happy to answer the questions before moving on to to showing a little bit about the whole map and architecture so feel free to ask. I saw a question about the albert for the projects, something like that. So no, if you can elaborate a little bit about that and by what story about what are the outback configurations for the project, setups.

C

At the moment,.

C

Sorry, I'm not sure if I like everything currently is uh over here, the like uh defaults, but, of course everything.

D

C

Change or add, more flags.

B

Yeah, the default configurations for the arbuckle are just open for for everything, so this is like default behavior at the moment,.

C

The installation of the agua city itself by default is cluster installed, but we have a flag to install it on a namespace option and also the entire bootstrap command can accept a different source for your august installation. So as long as it contains iocd and application set, it should work so the bootstrap can have whatever you want in there.

F

Are there any questions, maybe so my question was: how how will we maintain the r back? Will the r back for these applications and projects um be stored in the kubernetes at the back end, or will that also be in github.

C

Currently, anything you put in github will be applied to the cluster, so you can manage it through here and so.

D

So yeah basically.

C

If you have the resources for for your uh for your r buck here, you can have it in the githubs. It all depends on what what you applied. What's your application.

F

Okay, so so is there a structure I should be using inside github for that.

C

um So so this structure is uh at the moment. This is the way autopilot is building the the ripple everything I didn't mention it before, but again in the bootstrap.

C

um It is possible to have all of that inside the sub folder, so you can have an existing repo that has other things and bootstrap into a specific folder in there, so it doesn't have to be on the root of the repo, but other than that. Those three folders are rms. That's how the bootstrap is working, um but then a specific application that you created a specific manifest or your customized folder, and you can manage it.

C

However, you want and then just install from it and then, when you install the app that basically will just points to wherever you put your application.

C

There is also an option if you want to, if you have the mini, manifest files locally on your phone on your hard drive, then, instead of having a customization here, the base will just contain uh install the ammo file with everything flat or all the resources. So it is an option.

C

I imagine it would be mostly useful for quick, pocs and demos. I know I don't think so, but maybe or use cases for it.

A

I have a question: if um autopilot will would work with, uh can it manage many repos or is it currently a mono repo.

C

So yeah the cli itself. Currently I have the.

C

I just I set up this end variable. I don't have to set it up. I can send it in on every command and I can have every command go to a different repo and and work on it. But but um currently every repo is has to be a complete solution, so it has to be bootstrapped and it has to contain a lcd and installation on it.

C

So there's nothing to prevent you from having several different, uh such a github, github's repo, but they're, not all managed by the same. Obviously, at the moment, okay,.

A

Yeah, and um can you talk a little bit about when this, the relationship between projects and the applications? Is that something the user can choose for themselves or is there some kind of mapping that is um autopilot decides for that project in the application.

C

So, basically, just uh yeah the the term we use for project. The august of the autopilot project contains two two resources, so it's a another tv project and application set. So just here in the ui, you can see there's a two of each application, certain projects for every single autopilot project.

C

um It's nothing like the user shouldn't care about it. I I hope um the application set is the one that's in charge of creating the application, because, uh when you're adding an application, we basically add a config.json file on a specific location and then the application says.

C

Show it here and then the applications that will create the the specific application. So let's say the project is basically just looking at.

C

Here so everything under customize, whatever application overlays prod. So all all the configurations that are matching this path will create an application. So this is how we basically commit the configuration and if you have the files around it- and this is what causes the application to appear and then the project is used to separate applications and also in the future, we'll add more.

C

Ways to define all the permission on which cluster and which, from which a github repo, so it will be more fine-tuned to whatever every specific project project needs. So this is just some simple defaults.

C

B

So if there are no more questions, maybe let's pass through oren to talk a little bit about the roadmap, which is super interesting uh to see where the autopilot will move ahead from here, and we have all sorts of thoughts and things that we would like to get feedback. And so we can decide about the prioritization. So oh then go ahead. Yeah.

G

Okay, thank you so uh so. First, I would like to mention that uh the roadmap uh is available on the on uh the read the docs site and the reference the reference to the documentation is found in our in the git in the rcd autopilot repository, and I will also would like to take this opportunity and encourage encourage every one of you uh to also to check our issues page. So this roadmap is actually based on on the issues page uh that is associated with uh with the repository.

G

um So you can, you can either uh add more issues, thumbs up or thumbs down any issues that you would like that will uh again a priority and so on so and so forth. Okay, so regards the the roadmap uh that we we thought about it. uh I would like. uh um We talked about the next few items and the repository update and uninstall. uh For now, uh we don't support uh upgrade by upgrade means that the ability to upgrade the argo cd, the obviously that we are currently installing in the boots up command.

G

So this is the first item. Second item is about the way how to install this orgo cd, that is being it is being installed.

G

We would like also to support a way to upgrade an app, so uh so, when an app so, for example, whether a new a new tag that is associated with some app is uh being uh published. So we would like to that. Argo cd will be aware of this uh of this tag, and we will offer either manual or automatic way to to upgrade the app and uh to create pull requests that will be applied uh to the cluster. Sorry will be applied to the um to the grid, repository and delete app.

G

Is we already already implemented?

G

Next item is a story secret as for today, uh the git token, that is, uh um that we are using uh to create uh to uh to have uh the access to the git repository, is being stored on on kubernetes. So we would like to to find a a good way uh to use secrets in in the github's approach and, of course, uh storing also the g token in in this way.

G

So we are, we are thinking about uh integrating uh with external secret stores like like sealed, secret or or vault, or any other uh secret store providers and and and if and we would like to provide out of the box store solution in case there's, no one exists on on the cluster.

G

A very cool feature that we talked about is the promote feature.

G

I think it was always already raised in the contributor forum last week, so we would like to provide a way that one can promote exchanges from one environment to the second event, for example, that we have a staging environment and we would like to promote the changes after after being verified into your production environment, so we're going to introduce such a command and such a process and and also a way to to to apply to apply, to promote a specific environment to all the environments- and I mean environments- I mean I mean projects.

G

um Another item is the the multiple aggressive installation, so as for today, we have a one argo cd uh to rule them all so uh like like nom demonstrated, uh there was one obviously installation that uh actually controlled uh two cluster, so we'd like to to support a weight, a way that that that either either argo cd was already installed and the autopilot will work with it automatically.

E

G

Of course, a support, a scenario where, uh where august cd is going to be um installed on a per cluster, so this is also an an important icon on our roadmap. Also, we got. We got a request to support support n.

G

For now we are only supporting the customize, so we need to support other other templating languages, so probably the next one will be helm uh uh other or everything the community uh will choose to implement and uh regards the ripple create we for now we support a gita people create, so we would like to support also bitbucket and gitlab.

G

So uh any question is on the roadmap, and this is the the issues page uh just like we thought.

H

I thought all right real, quick um as initial as an end user. um If we wanted to vote for a certain prioritization, um where would we go to uh submit that vote.

G

Okay, so, for example, if you would like to to turn back for so introduce the the local upgrade command, just click in and there is uh um where? Is it sorry.

C

On the smiley icon, you can have a thumbs up next between the collaborators.

B

B

Smiling next to the collaborator, oh.

G

B

G

Okay thanks, oh I just I just did it today.

G

Sorry thanks, thank guys so just a thumbs up or any other reaction, and, of course, according to community, we will pick uh the items, the the issues that get more thumbs up or any other affection icon and, and we will try to implement it, that's perfect! Thank you. Okay,.

G

B

Thank you so last thing that we wanted to show a little bit is uh show a little bit the architecture and show what is the process of contribution and the release life cycle a little bit. So, let's move away if you can share a little bit starting from the architecture, so.

D

Yeah, I don't have the architecture ready just a second.

D

Yeah, I hope you can see that.

D

Can you see it guys yeah yeah, that's fine, okay! So, basically, uh what you can see is. It is just what the norm um bran in the in the demo.

D

So we have the autopilot cli, which is separated from the all of the argos, cli and basically what it does the first time it accesses the the cluster to apply the bootstrap manifest and after that, the the only times it accesses the the cluster or it never accesses the cluster after that, but the only time it did access the augo cd was to actually connect it to the other cluster, so both of these actions will probably be performed only once and after that, all of the accesses will be directly to beat to create new applications and manage the environments and all of the already created stuff.

D

So of course, you can also do it uh manually through git, but uh we would like to provide uh doing the the trivial things uh like adding an application deleting it uh through the cli, but all of that can also be managed through git.

D

So that's the architect architecture um now to the actual release, life cycle and contributions. So the the repo itself is is managed, like any other open source ripple similar to many of the other algo projects.

D

So yeah, of course, you're welcome to create issues and submit a pull request and I'll just show you uh one pull request, for example, so you can see once you create a pull request. We run a sci pipeline, which you can see here, so we use code fresh currently to run the ci pipelines and you can easily just click on it. It will also show next to each commit and it will link you to the actual build.

D

So you can check what went wrong or just check the results and after you're ready to to have that pr checked. You can just ask for a review and then, when we are ready to merge, we we will merge it to main to the main branch and then from there once we are ready to create a new release.

D

We will just uh go to a release branch and then from there we will run the release pipeline, which you can see here. So it's basically pretty simple. It's just running the ci pipeline and builds the the binaries and creates a github release and does all the the boring stuff.

D

So, of course you can see all of the releases here with the installation and the documentation of the new features.

D

That's pretty much it another thing I will say is we have a slack channel? You can see it here. uh Also. We have a link to the official proposal documentation.

D

So you can see it here and also probably suggest some changes and we also have a getting started link which will just link you to the documentation.

D

So that's pretty much. It.

B

Thanks, so if there are no other questions or.

E

B

We pretty much finished what we wanted to do.

A

All right, thank you code fresh for that um awesome, presentation. Well, autopilot looks like it's in the already really far long shape from the last time or like the last like a month ago, when you guys were talking to us about it, um okay, so um so, actually that was the agenda for this month. um We had one that we pushed off till next month with, but actually next month, we wanted to present about a hackathon project that we did and intuit to support um uh argos cd style notifications, but for rollouts.

A

So the idea is that um you will annotate your your rollout with the same style of annotations that you do with argo cd notifications and then you can then get kind of life cycle messages or, like slack messages or emails or pages on um interesting events about your rollout. So look forward that to the next meeting, but I knew we. We knew that this would take most of the meeting, so I push it out to next.

A

um As always, you can always add items that you want on the agenda that you want to discuss and we still have some extra time now in case anyone wants to bring up any topics.

E

um I was just going to add on that argo autopilot thing I just posted in the chat, a link to the blog post, announcing it so feel free to share that and use it to follow along, and um I I had a pretty good experience with it. So good work, good, good work at thai and no I'm in the rest of the team.

A

Yeah great job.

A

All right, um so, if there's no other agenda topics, we can end a little bit early today, um so yeah again feel free to uh update the google doc with agenda items um for demos or anything you'd like to discuss about the projects.

A

All right, thank you. Everyone and we'll see you in about two weeks for the workflows um community meeting.

E

Thanks, everyone take.

E