Argo Workflows and Events Community, 17 May 2023

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Argo Workflows & Events Community Meeting 17 May 2023

Description

This Argo Workflows Community Meeting covered an end-user story from Forbes and Raymond Wong, Senior Architect at Forbes, who championed Argo adoption. We also reviewed the ArgoCon Europe 2023 talks. Playlist here: https://www.youtube.com/playlist?list=PLj6h78yzYM2MqzIJyc5OBXAgFxjW6ZyBq&themeRefresh=1

Community Meeting agenda, notes, and schedule: http://bit.ly/argo-wf-cmty-mtng

Argo Workflows talks @ ArgoCon Europe 2023:
Leveraging Argo WorkflowTemplates Within a Data Platform at Bloomberg & Pipekit - J.P. Zivalich & Yao Lin -
https://www.youtube.com/watch?v=g8TCgnrM648&list=PLj6h78yzYM2MqzIJyc5OBXAgFxjW6ZyBq&index=30&t=1s

Managing Artifacts at Scale for CI and Data Processing at Pipekit & Intuit - Caelan Urquhart & Julie Vogelman -
https://www.youtube.com/watch?v=ucnOQuNkIbE&list=PLj6h78yzYM2MqzIJyc5OBXAgFxjW6ZyBq&index=23&t=2s

Lightning Talk: Configuring Volumes for Parallel Workflow Reads and Writes at Pipekit & AWS - Lukonde Mwila & Tim Collins -
https://www.youtube.com/watch?v=QZI-LXJGWYI&list=PLj6h78yzYM2MqzIJyc5OBXAgFxjW6ZyBq&index=24

Scaling Gene Therapy Research with Argo Workflows and Hera - Flaviu Vadan, Dyno Therapeutics - YouTube
https://www.youtube.com/watch?v=h2TEw8kd1Ds&list=PLj6h78yzYM2MqzIJyc5OBXAgFxjW6ZyBq&index=26&t=1s

Training and Ensuring Reliability of ML Models at Wolt: The Power of ArgoCD - Stephen Batifol - YouTube
https://www.youtube.com/watch?v=p3EaPotovBs&list=PLj6h78yzYM2MqzIJyc5OBXAgFxjW6ZyBq&index=29

Argo Workflows in Action: Rabobank's Journey to Cloud Native Banking - Sudesh Jethoe & Karly Huijbrechts -
https://www.youtube.com/watch?v=q7tRK9uPW0k&list=PLj6h78yzYM2MqzIJyc5OBXAgFxjW6ZyBq&index=19

Lightning Talk: How to Avoid a Kubernetes Doom Loop - David Collom -
https://www.youtube.com/watch?v=skOTc_evJnw&list=PLj6h78yzYM2MqzIJyc5OBXAgFxjW6ZyBq&index=31

A

All right, I'll kick things off here and we'll. Let folks come in, hopefully a few more folks to join um so yeah. Here's the agenda I want to do a quick overview of the talks that we saw at argocon EU, which was a surprisingly yeah um filled to the brim very popular event. We'll do a few of our contributors shout out as we like to do, and then um talk a little bit about some of the 3.5 release.

A

Roadmap, that's in the works, um and then the highlight of today will be presentation from Raymond from Forbes and Robert from synthesis.ai yeah thanks. You guys for coming and presenting today. um So to kick things off just want to call out ago Khan, EU talks or Live on YouTube check these out.

A

I think there's a good 10 workloads event talked specifically in here um from end users from primarily end users, some some vendors, like Pipe kit, um uh my team, myself JP, is on the call we gave Cox um in partnership with a few uh other maintainers and end users like Bloomberg, get into it um and then got to see some really cool um end user presentation from Folks at rob. A bank was a good one: um yeah Rook out was another really solid one and and from a machine learning perspective.

A

Walt had a really interesting presentation about using market workflows and Argo CD for ML orchestration, so yeah highly recommend checking these out um I'll post specific leaks links in here, but feel free to go through the playlist there um yeah quick shout out. We got some big news: congrats Carrie who's on the call for being promoted to project lead, long overdue, yeah, Round of Applause, uh Standing Ovation, so yeah um well deserved and yeah. It's been a pleasure working with you on the project um the last couple years now and you excited yeah leadership position.

A

um I know you probably haven't prepared comments, so I won't have the mic over quite yet, but yeah I just wanted to call it out that Terry is now a project lead for Argo work, clothes, we're really excited um in the community on that front and Bala is the other Project Lead along with Alex Collins, and then there will be a um I think plans in effect the transition Alex Collins to approver, um so yeah exciting plans in the work.

A

uh We have a couple other maintainer promotions, so it's uh from the pipefit team, it's now a reviewer um and Julie was a reviewer um and she just got promoted to approve her. So thank you both for all the awesome work um and leadership in the community on the code base, especially um yeah, really looking forward to working with you all going forward.

A

Yeah you're, welcome glad you were able to hop on.

A

um And then last shout out for other folks, we see in Argo workflows and events, Community, um David, Ray, Allen, Robert, um yeah, awesome, work and great to have you now as formal members on the project.

A

B

A

Definitely Round of Applause for y'all happy to be here.

A

A few other shout outs: we had some other community members that got pretty active um since our last community community meeting, so I wanted to call that out or awesome work, getting some fixed and even a few features in related to pod names.

A

um Yeah, Hooks and um templating really appreciate that um and want folks to um yes that you know continue contributing on that front. um Another shout out for Ahmet I'm, getting a pretty helpful feature for templating uh host node name, which will help a lot with node selectors and using those in workflow template um nice work. Thank you for the contribution and a last shout out to Mac, um for he had thrown in some Texas um like this.

A

So I really appreciate all the work, you're doing and and we'd love to keep working with you all um to keep up these contributions and don't hesitate to reach out to us if you're interested in membership on the project or um yeah following the road down to reviewer and approver.

A

With that, um just some quick comments on the upcoming uh potential release for a version 3.5 Vargo workflows, folks in the community, have been chatting about this.

A

um The big things the call out are UI ux and quite a few fixes and features we have on master that are going to be slated for a 3.5 release um if you're interested in contributing UI ux is a great way to get going so Terry put together this helpful GitHub project that organizes a lot of UI ux features and enhancements that we've seen uploaded uh we'd love to get more folks involved on this um shipping.

A

Some of these features ahead of releasing a version 3.5, so I, see yeah Terry is the phone here already JM great to see you um really appreciate yeah helping out on this front. um If you have questions, don't hesitate to post in the contributors, Argo contributors channel on slack, we can support you in opening these PRS um and then yeah. We have quite a few features that are on Master but haven't been released. I believe so um we're mindful of this and looking to nail down a 3.5 release date and we'll update on the community.

A

Once we have that, um so those are the big updates on a 3.5 released roadmap and if I guess I'll pause there and see if anyone else would like to chime in or has any comments or questions.

A

Right on, if uh yeah you have future requests um again, don't hesitate to speak up at Community meetings and we have contributors meetings every uh two weeks. So look on the Argo project calendar for those and that's a good place to ask about yeah when a bunch of features might be on the roadmap.

A

So with that I'd love to hand it off the array at Forbes, um if you're ready, I can hand over screen sharing to you and feel free to take it away. Will.

C

Do thank you. Kellen.

A

C

Man can everyone see my screen.

A

Yes, uh but we see the Google Doc yeah.

C

Okay, great all right: let's, let's go into slideshow Moon hi I'm, Raymond, Wong I'm a senior architect at Forbes. If you don't know us, we are a global brand that are well known for our billionaires list.

C

uh So a little high level um on our Argonauts Journey. We started off with Jenkins like everyone else and you know started to run into difficulty, maintaining and scaling Jenkins.

C

um The main impetus is really the docker deprecation um Jenkins running in kubernetes with Dr and docker was afraid that you know it'll it'll not be available anymore, so started looking at new Stacks. You can read more about it at this uh website down here.

C

So this is our CI CD pipeline I used a hello world container, but I can give you a good view of how we're doing things so, um first we're doing a get clone and the reason why we're doing this first is because uh we taped the outputs from git clone and we use it in other places like uh building the image. Slack notifications PR comments.

C

What have you so that is our uh entry point, uh then you'll see that we're doing things in parallel, like uh building an image and doing a slack notification and commenting on a PR and you'll also notice that we're doing things in container sets so we're trying to minimize the time spent on the entire pipeline.

C

Then you'll see that we have a whole security section where we are one either trying to figure out what language your repo is using or if you pass in a um tag, we'll use that tag, and then we will do various uh scanning like for your code, your dependencies, your container and then there's a whole exit Handler. That will notify you um of the results and if we so choose, we can start getting deployments.

C

That's probably something it will turn on in in the future.

C

uh Moving on, we have our CD portion so over here, we'll actually do a pull request into a app Set, uh so we're using Argo CD and once the pull request is merged, then it will actually start doing the deployment.

C

uh In the meantime, while the deployment is rolling out, we're going to figure out all the different applications that are being rolled out and uh there's actually a sink and weight portion, so that will tell us when the deployment is healthy and we have some other things that might have been in parallel, such as DNS workload, identity, deployment, markers and finally, once everything is done, we have a exit Handler, which will upon success, send out a slack notification or if there is a failure, it will gather all the failure messages and then send a slack notification or a PR comment, and then there's also a PR comment.

C

If this was triggered through a pull request, so moving on as a developer or a QA person, there are multiple ways that we can trigger a cicd workflow. One of the cool things that we developed in-house is a slack app which will take slack commands so over here you see there is a slash, deploy and then there's like a drop down for all the apps that we can deploy and some parameters that you can pass in, like your branch, your environment and then finally, it will trigger that's.

C

Hey I've done something another way that we trigger workflows is through GitHub. So one of the neat things that we've done is on a pull request. If I'm a developer and I'm working on different features. If I add this label a Dev box, it will actually trigger a workflow to deploy a developer deployment onto kubernetes where I can test my changes. Other neat things that we can do are watching for pushes to main or cutting releases, and all of this is done with a web hook to our go events same thing with our slack app.

C

It's going to trigger to our go events. We could go into Argo events at another time, but moving on all those notifications that I was telling you about. This is how they might look. So on the left hand, side is um saying that the build has started and you'll notice that there is some git commits mentioned in the links and then over here you have some security results and then, and then over here you have uh the successful notification on the right hand, side, you'll, notice, uh PR comments. We are using a legacy, GitHub user.

C

So that's why it says Jenkins Cloud, but it is all on Argo and uh you have a notification that your Dev box is being prepared and it is done with some useful links so moving on uh the interesting part, lots and lots of yamos so I mentioned and get clone. uh We do have a lot of outputs which we are using in the future, one of them being a artifact and you'll notice as an input artifact. We are using a git artifact with Secrets over here.

C

uh You also notice that we have a lot of Vault annotations, so we are using hashicor Vault and we are using Vault injector to pull in uh secrets.

C

So you'll see that over here moving on, um basically, you know, do a git authenticate and then we might switch to a specific Branch or revision and then based off on some input, Boolean Flags, which are actually strings. We might inject some Secrets like npm RCS or doing some good mod vendoring, and then we, you know we're massaging our um get information and echoing them all out to outputs, which we will be using later.

C

I will note that if you don't actually need to use all of these uh get inputs or get outputs, uh you could just simply do the uh get artifact on your build stage.

C

uh Moving on to our build image portion you'll see that we actually Define some defaults over here for kaniko we're using kaniko, because it does not need Docker, sockets and it's secure. um Then we have some more Vault annotations and then an output for your image name.

C

So in our container set here we are going to generate some image data so, depending on some flags that we pass in, we will generate a image tag either using this random hash sure it passes an image tag and then we're going to Output that into some text, files that we'll use later and then over here you'll see a bunch of those git outputs that we had before we'll be using them as labels, so that if we do a container inspect, you can see the attributes of the container from build time.

C

Okay, moving on, uh we do um pass in some build arguments as Secrets such as your GitHub personal access token, maybe some APM Keys, depending on certain flags that are passed in and then over here. We just do the kaniko build from all those arguments that we combined together, uh you'll notice that we do have a security context over here, and that is because Eco requires some elevated privileges.

C

um Noteworthy is running as root and adding some capabilities, and the reason for that is by default. In our workflow template defaults, we are actually dropping uh privileges, dropping caps, making read only root file system running as a non-root user. So specifically, we are calling that out to elevate privileges, there's a great blog post from Alex Collins about securing your Argo workflows, which is a pretty good read.

C

uh Moving on this is our Argo CD application set, so you'll notice we're using a list generator. So we have um you know, information about the environment, the image uh who deployed one and now we're combining that into basically your cluster information moving on so once we have the image created and we have the image output. So we know the image name, then we're going to do basically a bunch of Y cubed depending on if it's an existing deployment or if it is a new deployment.

C

So we have all this complex, yq logic which will basically insert into that app set, which I showed you previously and then once we have that we will do a pull request and emerge.

C

uh So this is some more code. People can pause the screen later in the recording and um watch it.

C

So this is the get diff of an existing deployment, so you'll see that it's actually just changing the random hash and the deployer and the deploy time. So this makes it pretty easy to revert back. If there's some sort of change, you just go to the pull request and revert it and now you're back into a good state.

C

uh Moving on so I didn't mention that, once we do do the pull request and it's merged it, there is a web hook which will trigger our ghosts. They need to start syncing automatically. However, we do uh specify a single weight, so we do know when the rollout has completed- and you can see this over here- that we're trying to one first determine all the applications which are affected. So in some cases we have multiple deployments, multiple applications, because we are multi-regional, so we're highly available.

C

So there are two applications, perhaps one in say, East and once say in West. So that's why you might have multiple targets to wait for the rollout to finish, and then finally, you see over here that we do do the sync and we and then there's a specific timeout.

C

Hopefully you finish your deployment before then, but the deploy the timeout is there to notify. If there is something wrong with the deployment, for instance, uh you've push them back code and your app is just crash looping and it never finishes the rollout. We want to know about it and then, when we finish our deployment, we get a nice hello, Argo community.

C

So I want to give some shout outs to my team, uh Nazar and Omar kalush they're great and our intern at one point Matt he had contributed so uh I cannot have done this all by myself, so I just want to give props where props are deserved and I will open it up for questions.

A

Nice thanks a lot Ray yep.

D

That was cool, um um I didn't catch uh the motivation for dropping Jenkins. Can you clarify that.

C

Sure um do you have a blog post about it here, but uh high level Jenkins kind of difficult to maintain. We did have a shared Library, which was collecting a bit of tech, debt and Jenkins. The control plan doesn't really scale itself well in kubernetes um you, you basically get a single pod. So if you're doing any sort of Maintenance you kind of have to restart the whole pod and and there's some down time, um yeah.

D

C

Mean Jenkins uh Jenkins is a good Workhorse. It works well, but um it can be difficult to manage and also with the Docker deprecation that they had announced and is now um GA and 1.24.

C

uh We were worried that using the kubernetes plug-in for for Jenkins, um which relies on Docker and Docker, would not be available in the future, so it Reliance upon the docker socket right, so the docker socket is gone. Then um a lot of this would not work. So that was the primary motivation in moving to a newer stack.

D

Cool. Thank you.

C

A

I'll add a question for you right yeah. What's next on I, guess your to-do list with Argo at Forbes, um either with cicd or other use cases like.

C

A

Yeah, what other needs, or or.

C

Priorities, do you have great question um so I did mention. We are using in our uh build image.

C

Kaniko I'm actually evaluating build kit at this time and what's cool about Argo workflow templates is I, can I just swap out a container set or template, and you know if my inputs don't change, uh it's it's pretty easy to just swap it in, and the main impetus behind that is noticing that kaniko is a little bit slow and I'm just trying to compare build kit versus kiniko and see if it's worthwhile to switch over um other things that are in our pipeline um Nazar who I have mentioned earlier.

C

uh We are doing some things with uh ytt for templating um and now we're kind of noticing there's some sprawl um amongst uh our ytt templating across multiple repos, even though they are more or less the same. Some of them diverge slightly and we want to be able to easily maintain that so we're we're looking at something called vendor v-e-n-d-I-r, and he already has the pull request for that. I just need to find some time to review it and we can test it out so that that's something interesting. That's that we're working on.

C

Perhaps not Argo specific but interesting, stuff yeah.

A

I sense, another future talk about how you're using ytt yeah with Argo.

C

There's a lot of topics: I could talk about. ah Okay,.

A

um Is there any other questions yeah.

A

I'll, throw one more here um for you: Ray oh, go ahead! Yeah go.

B

Ahead, I was just gonna say how long did uh this is a lot of work. This is really great by the way. How long did this? uh What we saw today take your team from start to finish just just curious. Oh.

C

Gosh um there have been many iterations on this, like I, think we're almost like a year and a half into our Argo Journey, but I mean a lot of it's.

C

You know, figuring out, like ARCA CD had bootstrap everything with Argus CD um coming out with the workflow templates. There's, definitely a uh High overhead when you're first starting in out, but once you kind of chain everything together, it's it's not too bad and we've been iterating. You know we didn't first start out with container sets. We first figured it out. Everything has like a as a step level and then we're like okay.

C

Maybe we can reconfigure this as a container set to get like some speed improvements here and there uh I mean we don't always have a security scan portion. We just kind of uh figure that out later on and injected that um there have been several uh uh different Frameworks for Argo CD, like we were using a um app of apps pattern. So we didn't initially have all this uh yq uh stuff, so I mean things have been iterating and evolving. You know over the past year and a half, so it's a constant progress.

C

Okay, um any other questions or I can just stop sharing and hand it off.

A

Yeah you actually mentioned the container sets I was going to ask you um if you had any sort of um the speed, Improvement or cost savings um numbers that you would be able to share with the community around like going from yeah, stepwise templates to then using um containers that, for those those um those steps that you mentioned on the previous slide,.

C

Yeah, unfortunately, I did not capture that information, but we definitely did see um some gains um mostly. You know what you're doing, if you're doing it step wise you're waiting for um like your pod to be spun up, get scheduled, um I mean with this. You just have one part with multiple containers, and hopefully you know you're You're Building constantly. So your your image cache uh is, is pretty good, um so you don't have that wait time for uh the image or container to come up.

C

uh I mean there are things that you can do that, maybe in the future we'll look at like um and then in in our cloud provider. There's something called like image streaming, which does some magic and, if you're, using their um container Repository, uh it should just like start up instantaneously and then kind of bootstrap things later on. So that's that's interesting to look at.

A

Yeah I think that was a feature that came out right on Google and AWS last year. The container.

C

Streaming: okay, definitely something worth looking into just haven't. um Had the.

E

A

C

A

Cool does anybody else, have questions.

A

Well, yeah, thank you so much Ray for preparing all this and sharing um all these details really appreciate it. um Yeah I.

E

Had a random question, sorry go.

A

E

A

E

That the dark reader.

C

Plugin for your browser, because.

E

C

Aware that, okay, that.

E

C

E

That looks awesome. I was just curious.

A

Yeah, that's a good Pro tip for dark mode yeah with Argo workflows.

A

We should probably post that afterwards.

A

um Cool yeah thanks Ray um I guess next up Robert um is going to be presenting uh potential feature that he's developed um so yeah Robert I'll. Let you take it away.

B

Awesome yeah I appreciate it and um yeah before I even get started. uh Apologies in advance I've been sick for the past week or so so, if I sound, confused, I'll blame it on that um so yeah I, appreciate uh the opportunity to share this feature is something that uh we've been kind of chatting about uh for the past year or two and implementing and and uh I am excited to share what we've done so far in this meeting.

B

I think before I dive into the slides here, the I think the main goal here is just to get feedback on like how useful like. Would this be broadly useful, or is this like a problem that just one or two people might have, but maybe not big enough, that it's worth actually merging in uh to the main branch?

B

So that's kind of the context, um so my name's Robert uh I am a lead engineer at synthesis Ai and basically what we do is that we generate synthetic training, data for AI, and um so our use case actually seems to be different than many of the other community members that I've talked with so I I'm curious.

B

If this particular feature is also kind of a niche thing or not, um so the motivation here is that uh so what we do is that we run workflows on behalf of our end users, so we could have 5 10 20. However, many end users running workflows at the same time and some of these templates use finite resources.

B

So in our case that might be a license, but there are other cases where you could imagine that you know you have this smaller set of resources that that you need to share amongst users, and so the idea is that we want the users to have equal access to those resources.

B

Instead of uh you know, one user submitting 10, 000 templates and then the next user, who submitted one second later, has to wait for all of those templates to go through before their templates run. That's kind of the high level so just to visual visualize it, my slides, don't have a lot of visualization. So this is. This is the extent of my my artwork here, but you know suppose. Hopefully it makes sense.

B

But, like suppose, you have these two end: users, Alice and Bob, and these gray circles here excuse me other workflows, so you have this template that kind of dynamically or, however, spawns uh four sibling templates of the same type and then Bob for Simplicity. Let's say his workflow does the same thing now these templates use a resource. So in our case it's a license. So every every template or every pod. In our case, it's running.

E

B

uh Is claiming one of these licenses, so you have eight total, but you only have two resources, so we want to give one of Alice's at at any given time and one of Bob's until either Alice's or Bob's have completed, and then the other person will get both of them.

B

I'll stop there. If that doesn't make sense to anybody, I can dive into a little more detail, but that's kind of the premise here.

B

So yeah, like I, mentioned its synthesis. We're running uh third-party software. In our case this is um rendering software. So these are some popular CG tools that are running on the pods and when the Pod spins up it claims one of those licenses when the Pod spins down it releases it. So we want to distribute how those licenses are distributed.

B

um You know there are other examples, I I think, but you know another one that came to mind as I was writing. This was suppose, there's a smaller cluster with finite number of nodes, and you want to, and- and so you want to distribute a particular instance class amongst, amongst your end users, just another idea that came to mind um I'd be definitely curious to hear how this might be more broadly useful. What kind of cases this might apply to so like I mentioned, we've been working on this for uh quite a while I I.

B

Actually, last fall. I opened up. uh This second pull request, but uh our conversation on this has gone back even further by a couple months. At least um I have a whole slide show around the initial pitch, which kind of walk through my implementation. At the time um it was kind of a first pass, so it wasn't ideal I think it was a little more complicated than it could have been. It had more new Concepts that I think than I.

B

Think um people were interested in in entertaining so uh I did create a PR that worked um got some feedback from the maintainer meetings. Paula was, amongst others, were very critical in helping me to kind of iterate on this, so that it might be more appealing to the broader community and, as a result of those conversations, went back um from the ground up again re-implemented this a second time using a new approach.

B

This time um this fell kind of under the category of like or in the the part of the code that was related to semaphores, um without going into too much detail the way that those were because that they run on top of a queue, and um you know, whenever a new lock is available, um we basically pull the next resource that should receive that lock. Well, the new attempt we create a new type of queue that can be swapped out with that default one.

B

We decided in the end to call it rebalance Q, which basically, what what happens is when a lock is released from a currently running template. We quickly go and we sort the top few items so that we always maintain that kind of even distribution amongst the end users.

B

um So I went and I implemented that I wrote. You know unit tests and integration tests, so we're we're quite confident that this works well.

B

um Yeah I won't go through the whole PR, it's it's quite big, but it is there um 9992. If you want to check it out um now, I I was weighing doing a live example. I get kind of nervous doing those I feel like anytime. Something can go wrong, it does so I wanted to just kind of share and in the end, there's actually not a whole lot to see.

B

You just see a bunch of pods running, so I wanted to step just a little bit deeper into the implementation by sharing um kind of the user guide. This is um documentation that I wrote in in the proposals directory for this PR and um I. I think that in the end, we settled on a pretty nice and straightforward implementation, um and this was a kind of a group effort.

B

This was based on some feedback that I got in those earlier iterations and- and the idea here is that currently you have config Maps, where you designate kind of a threshold for for semaphores um in the current docs. We there's an example that has template and template is set to some value.

B

So, if you're running five of those templates they're all going to run at the same time, if you're running six, it's still going to run five until one of those finishes and and then that, finally, that sixth one can begin so we're building on top of that concept, but in this case we're just adding one more property called basically template and then Dash strategy um is kind of where we landed so just adding the dash strategy and then setting it to rebalanced by default.

B

If you don't, if you don't add anything, if you don't change anything you're still using that default queue you're still using the default Behavior, but if you set it to rebalanced, suddenly you're, unlocking this, this kind of new underlying queue that will that'll open up this functionality.

B

um So that's the configuration, that's involved, that's it! If we step down into the workflow itself, uh let's see what so what's happening here is that we we have two steps. The first step is generating a list of numbers.

B

That's all, and- and so you you can see here- that it's going to just spit out 15 15 numbers and that's going to spawn 15, basically siblings as the next step, using the with param tag or the with param property and um and if you look in the acquire lock, we have all this kind of standard, synchronization, stuff, semaphore config map ref, where you're setting the key, and this key corresponds up up here to the template. You know this is all existing functionality, um but we add one more thing: a rebalance key. So this key.

B

If, um if it, you know, if two templates have the same key they're considered to belong to the same user or that basically they're considered to be equal when it comes to um acquiring locks, so that's basically it! So if you were to spin up say uh two of these examples and you were to pass different rebalance Keys into you, know the first and the second.

B

You would see that uh you know we're sharing these five templates um and one of the one of the users, one of the workflow instances would have be able to run three templates at a time and the other would be able to run two there's no guarantee on the ordering there. It's just going to be three and two um until one of those completes, and then it will start to shift over um all of the locks to the remaining uh workflow.

B

So that's, oh I didn't mean to do that. That's kind of uh in a nutshell: you know what we've been working on. um It's pull request. 9992 um I'd really appreciate. If any of this resonated even just to go and give a thumbs up um or leave a comment um describing how this might be useful to your team or your project um and that's all I've got uh any any feedback, any any questions happy to help.

A

Thanks Robert yeah that was impressively coherent. Given your disclaimers at the beginning, sorry.

B

About all the uh hacking away over here.

A

It's been a rough week. Thanks sounds like it um yeah any questions for anyone.

A

Yeah I can say I've seen similar use cases where um yeah the cluster is getting overwhelmed with demands and, as certain users are getting blocked out of jobs from executing. You know, without this rebalance strategy that you mentioned. So it's a really elegant way of solving that that use case um interested to hear if anyone else um yeah ends up running into that and, um like like Robert said, please leave a comment on the pr on the issue um or not vote. If you'd like to see it um released at some point in the future.

A

Cool well yeah, there's no other questions or comments um thanks. Everybody for coming and I'll give a quick shout out again to two other argocon EU speakers that popped on ya Lin from Bloomberg and Michael Crenshaw from Intuit um definitely check out the argocon talks that are on YouTube, um so go um up up both those share, those um get them in the YouTube algorithm, but uh but yeah I'm exciting to see all the awesome talks across CD and workflows there, um thanks Ray and Robert for presenting today and we'll catch you next time.

E

Is open so if you have any use cases from other workflows, please submit a proposal to share with the Border community and the deadline is gonna be like or about around like in a month or something.

B

That was kubecon right.

E

A

Deadline on that, thank you, Terry for mentioning is uh June 18th, so um yeah I believe right around the next community meeting. So yeah thanks for bringing that up. If you have kubecon or further talk, ideas, um we'd be happy to get feedback on them. um You know if you'd like it and and help set you up for a great submission um anywhere. We can support our workflows folks and users in a talk, feed, we'd love to be there to support you um and yeah.

A

Hopefully, we'll get more Argo representation in the next kubecon that will be happening in Chicago on November, I, believe 6th through 10th and then um agricon. uh There will be an articlecon, North America, so stay tuned for updates on that um in the fall as well. But um cfp, uh processed and detailed haven't yet been released.

A

Cool thanks: everybody have a great day.

E

Thank you have a good day. Thank you.