VMware Carvel Community Meetings, 12 Oct 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Carvel Community Meeting - October 12, 2022

Description

Carvel Community Meeting - October 12, 2022
We meet every 2nd and 4th Wednesday at 9am PT. We'd love for you to join us live!

This week we talked about Carvel at the upcoming KubeCon, podcasts/blogs/videos featuring Carvel, new releases in imgpkg, and what the team's been working on. Check out full details here:
https://hackmd.io/F7g3RT2hR3OcIh-Iznk2hw#October-12-2022-Agenda

A

Hi everyone welcome to this week's edition of the Carville community meeting. Today's date is October 12 2022..

A

uh If you're watching this from home I want to come, join us live. We meet every second and fourth Wednesday at 9, A.M Pacific time. It's a good opportunity for you to come and meet the team meet other members of the community. Ask questions, listen in on what the team's working on and, uh if you have like you know, any any feedback, or if you have any suggestions on on what you would like to see from Carvel you can you can do that as well.

A

And for those that are here today and if you have a discussion topic, please be sure to add that at the bottom of the agenda right here, the discussion discussion topic section and we will get to it towards the end of the meeting.

A

If you can't come and join us live, you can still find us in other areas to reach us. So you can find us in the Carvel Channel and the kubernetes slack workspace. You can email us, you can find us on GitHub with any of our project repos and you can find us on Twitter at carbol underscore Dev if at any point, you're interacting with us or any other members of the community. We ask that you please read and abide by our code of conduct.

A

If you are using any of the cardboard tools, we want to know more details about how you are using this, like any of the cardboard tools. So we've created this pinned issue that asks you to provide some details about your company, uh how you're using Carvel, how can we reach you, etc, etc? And you can also add your logo in your comment so that we can add it to the adopters page, which is optional, but we would love to promote you on our adopters page as a as an adopter of carbol.

A

And then all those that are attending today, please be sure to add your name and any organization you represent, so that we can keep track of those who are outside of VMware that are coming to our meeting.

A

Moving on to announcements, so carpool's going to be at this little conference called kubecon and uh in about two weeks uh so we're super excited. We have a couple of of areas where we will be um I know that uh outside of this, we are working on some opportunities to be at the uh the Tag app delivery Booth, so so that cargo can do some demos there and as well as the VMware Booth, but once we have more details there, I can I can share it here.

A

Otherwise, we have a co-located session at gitupscon, Tuesday, October, 25th, 4, 15 to 4 45 pm and that's experimenting with q and cargo to enable getups for your applications with Dimitri and Rupa, then, for the main show we have on Wednesday October, 26th, 9 40 to 9 45., the we are doing a keynote and Rupa is going to be delivering that and it's beyond automation. Kubernetes success requires a get UPS mindset.

A

So if you are attending kubecon and any of the coolest sessions, we encourage you to come and hang out with us and listen in and and uh meet meet the team.

A

Okay. Moving on to content. So since last meeting Brett Fischer posted his podcasts for devops it devops and Docker Carvel tools for kubernetes.

A

We also had a video tutorial done by Robusta Dev, just as a recap on that Robusta is doing a hundred kubernetes tool initiative and going through all the different tools of Carvel, in addition to several other kubernetes tools out there, so the other one that we had done was with ytt, and this one right here that was just recently published is cap for cube cuddle, apply on steroids, so be sure to check that out and keep an eye out for the additional tutorials that come out from Robusta.

A

And moving on to the third one, we have blah a Blog that was posted the hidden costs of misconfiguration by John and varsha I believe this was going over.

A

uh What's it ytt validations or yeah, um do you wanna talk more about this.

B

Sure, just that um part of what we're highlighting in this post, uh that coincides with our release of ytt 0.43, which includes generally available data values, validations and this talks about. Why do you even care? So it's a short read but hopefully helps helps you understand where do validations fit in.

A

Awesome so be sure to give that a read, and let us know if you have any questions thanks, John hi, moving on to upcoming content on October 12th, which is today, is that accurate? Is it coming I believe so varsha? uh Do you wanna? Is this where you wanted to showcase your video and highlight a little bit more about it?.

C

Yeah sure so let me share my screen.

C

Are you all able to see the screen, and so this is the ytt getting started tutorial series we are trying to um get here. This is especially for someone who is coming to Carvel and ytt and trying to understand ytt and just coming coming there for the first time and trying to figure out what ytt is, what it is doing and how they can just get started, because uh sometimes the documentation- and here like there are a lot of things they have to go through.

C

But this is a video where they can just quickly get started. We will upload it along with timestamps, so they can just directly jump to whatever section they want to learn about and learn from there and then the part two and part three will come um for some details about other features of piety team.

C

So um I'll just play this a part of this video uh for first five minutes. It is the introduction to ytt what problems ytt is trying to solve and then uh we'll jump into playground. Examples. So yeah, let's get started here.

C

Hello, everyone welcome to the series of Carvel ytt getting started tutorials. We are going to take a tour of many ytt features through real world examples. So, let's get started.

C

Today's agenda is Introduction to ytt what problems is white. It is solving and demo C ytt in action.

C

So what is ytt ytt is a yaml templating tool. Yaml is a data serialization language that is widely used for writing configuration files. What is meant by templating templating is a static. Config templating, a static configuration file means extracting parts that I know my end. User will likely want to change or add logic to, but why do you need templating at all and what problems is why it is solving.

C

So when you are working with kubernetes configurations- and you are dealing with lot of yaml trying to maintain your configurations for different environments like prod QA staging, you can see lot of boilerplate code, and on top of that say, if you want to get some user inputs and update your configurations per environment, then you need to keep tracking, maintaining and updating various ml configuration files over and over again.

C

So if you want to reduce the boilerplate and make your configurations reusable and extensible solution here is ytt templating, why did it helps reduce the boilerplate by extracting values to variables extracting repeated yaml, fragments to functions and extracting larger chunks to separate files?

C

Another Advantage is like. Are you tired of counting spaces and indentations to make sure you have valid yaml and repeating this for each and every configuration file? What if you have thousands of yaml files, ytt templating is useful for avoiding parsing errors. Ytt operates on yaml structures instead of text based templating. The first thing ytt does when it is processing a yaml template it parses that file as yaml.

C

Why did it take CML in and renders guaranteed valid yaml out? This eliminates the need to count spaces or Escape codes, including pipes and filters for indentations, as in traditional text-based templating, so users can now focus on their configuration needs. Instead of issues associated with text, templating ytt uses yaml comments to annotate structures so that it's no longer text templating but structure, aware yaml, templating,.

C

um Another issue: you might be well aware of pain, of maintaining patches or edits and configuration files manually by following same steps. Every time you do the deployment, for example, say if there is a third party yaml, and you need to make changes to that, yaml that you don't own.

C

So when you have such source tml, and you can't make changes to the original file you you may need to make changes and keep track of the changes or keep log of it for future edits solution that ytt provides here is Advanced configuration via overlay feature to capture the edits needed and apply to the configurations, so overlays are basically logical, edits and with ytt overlays patches are articulated on the yaml structure instead of edits based on the line numbers, so the maintenance of these files becomes as low as possible.

C

It works with your own yaml as well as third party ml configurations.

C

Another problem you might have faced is when you need to template and apply patches together. You need to learn and work with different tools for that um which obviously increases the complex setting.

C

So ytt allows you to template and Patch as needed together to easily make your configurations reusable and extensible one more advantage of using tool like quantity is. um Would you like to be sure that your configurations are secure? Definitely, yes, like no secrets getting exfiltrated no network calls are made. Ytt provides secure, fast and deterministic execution.

C

Why did the execution environment is hermetic and side effect free with no access to file system, Network, timed Randomness and operating system interfaces? This guarantees that templates produce identical output with the same input. Your configuration changes only when you change it now, let's start playing with ytg on ytd's playground. This is the interactive playground and we'll start by working with simple hello. World example. Then we'll just slowly introduce extracting variables using functions, loading data values using conditionals events and using for Loops. So let's get started, I'll open my playground here and share that window here.

C

This is the interactive playground to see ytt in action if you are new to YTD. These are some examples, organized by topics so that you can familiarize yourself with white. It is key features if you are more advanced user and if you need a quick way to validate how ytt would render your files, then you can put your files using add file option over here and run it through ITT and see write it in action. What what output you will get.

C

So, let's start with a simple hello world example here.

C

In this example on left, we have SIMPLE template and we are attaching a string to the title. Hello world, welcome to ICT.

C

um Yeah, so these kind of examples I have added I'll um not take too much of time here. People can obviously go on YouTube and watch these. We will upload the link to this tutorial series on our website and along with the timestamp, so Advanced users, or who are already little bit familiar with it. They just want to go to examples. They can view them um all this example, so I'll just show couple of them, like um I, have plus time times time here so from, for example.

C

Here um here in this example, we are extracting variable, with name as a front-end service.

C

um In the next example um say we are using data values, file and extracting data values and using them in our config um in in the in this example, we are defining a function, so this tells you how to use functions in ytt.

C

um This is a another example where so I'll just run this for Loop example, and if conditional example- and that's it that's what we can do for today and then uh people can go through these tutorials as and when they get time basically go through. Look through all the values that are present in.

C

Saw how to use functions and uh without how to extract two variables. We saw how to use data values to get the values that you need now. The next section is, we will see how we can use conditionals and um for loops.

C

So um this is our example. Now what if I have multiple Services added in the data values file.

C

So say this is my one service called printing.

C

And I have another service call backend and um I want to generate the configurations for these two services or any new services that can get added on the Fly.

C

So for that, let's I I need to basically go through look through all the values that are present in this values file and then generate the configurations. So let's see how how we can do that.

C

For this service um I'll add a for Loop here for before that, I have to add hash, add for C service in data dot values, Dot services.

C

And I'll add in over here.

C

This is how I will um add a for Loop and.

C

So, for name instead of taking it directly from data values, name I'll, add service dog name. So now this name is getting rendered from data values. File and I got two separate configurations: one for front-end service, one for backend service. This front end is appearing because I have it hard coded I'll update it here as well.

C

So now, as you can see, this is a configuration for front-end service and this is the configuration for backend service so for each new service that gets added to data values, you're gonna get entire kubernetes service manifest, and this is showing you the power of for Loop and uh for end user. You write one line and you get the whole manifest, for example, name and I, say and deformed so I got one more default service over here.

C

Let me update um the version as well, so.

C

I'll add a version.

B

C

0.1.0 and that I need to page like this, so all of these configurations get the same version if I make any change it all gets reflected. This is follow.

C

Now say if I want to add HTTP and https support for these Services I want to support, say only HTTP for back front end and not for backend. Let me first add few um the lines over here ports, this kind of configuration, so here, as you can see, it is appearing for all of these services, uh so I need to make it conditional like I, want the HTTP only for prime time and and HTTP is not allowed for backend. So how do I do that, for that? um I can add.

C

Something like say: allow the sleeping through and say, allow HTTP parts and for now I'll just remove this default, and then here for HTTP I'll just add. If.

B

C

Service dot allow http through then show this chunk, and so now, as you can see here, I'm configuring, the um these services I'm configuring- how these services are being exposed.

C

And this is how we are making the chunk uh chunk of yaml optional.

C

So, as we see here, runtime service and only front-end service has HTTP allowed and back-end service does not have.

C

So yeah, this is how we can um use various features of ytt. um Let me get back to the presentation. Other way you can try out ytt examples is through the terminal.

C

So to see why didn't action locally? uh You need to do few steps like follow few steps, as mentioned here. These are available on the website too. So I'll share my screen for the terminal and, let's see here and right now, I have already installed. Ytt I have installed through growtab VMware, tanzo, Carvel and then reinstall YTD. So if I do ytt version gives me the latest version that I have and then I have already cloned the examples repo, but you can clone it using this um using Buzz directory.

C

You see these kind of examples that you can try out so to try any of these examples so, for example, um LCD to example, plane.

C

And it has one ml file.

C

It's basically an example file that shows you all the common data types. So to run this kind of file, you can directly give ytt hyphenate plot and that will give you the output or or you can give directly full path of the yaml file.

C

So that's it for today, please let us know how you find these tutorials to get started with your ytt journey and stay tuned for upcoming tutorials. Thank you. So much happy templating.

B

A

C

A

Awesome job Marsha, thanks for sharing that any questions on this.

A

Okay and so for those watching from home, just a reminder, you can go check out the full video when we upload it to YouTube I'll, be sure to share that on Twitter and make sure everybody has access there right and if you have any feedback or questions, please feel free to reach out to us all right. Let me share my screen again.

A

All right, so, as far as content goes, it's not limited to just the team. So if you have something that you are working on with Carvel or you have anything interesting, you want to share. As far as your knowledge with Carvel goes, we would love for you to sign up here in this uh Wiki weekly content sharing posts that we have. You could add your name date that you wish for it to be published.

A

um The topic on which you're going to be talking on and the type of media Medium you're going to be delivering the knowledge, so it could be a blog post, a video podcast, uh anything that that you know you are most comfortable with the asset.

A

You add that there and if you have a Twitter handle, you can add that and we'll add it to the promotion when we do the tweet and additionally, we would love for you to come and join the community meeting itself so that you can talk about what you are sharing and so that people can ask questions live as well. If you do sign up and and you um post something, we will send you a t-shirt as a thank you moving on to releases so image package, 0.33.

A

D

Hey so the best release after the previous one for September there we go so um image package. uh Two things to note about this, um this particular version. So we made a big change on node 32 uh that changed the way image package handles keychains, uh that by default it no longer uses keychains. What does this mean? It means that, if you're inside a a is sometimes you can authenticate using, for example like the gcp services, or you can authenticate using the the Azure Services internally.

D

uh Previously image package was just doing this by default and just trying to connect these services, like the logic, has changed, and now it's not gonna. Try that and it you have to tell it that you want to authenticate to those Services.

D

um So there's like a minor issue there, where you could only provide one uh you can provide more than one so this this is what it's being fixed on, that with that bug, to be fair, like there's, really, no reason for you to authenticate with more than one is because you can only run that single instance of image packaging in a single place, but who knows uh so that, and we kept on expanding our API now.

D

Tag list is also available uh that basically I I'm not sure like how to explain what this does, but it basically lists tags.

B

D

B

D

It like we're gonna keep on expanding on our uh on our apis uh to add more things as as the as there are like there are some requests for copy as well, so maybe that might be something that we'll be adding Max on our API.

B

Question uh is there like a regression in the Behavior now where, before you were able to off with multiple, is um and now you can only specify a specific one.

D

No, but so the the difference and the breaking change is that, like we change the variable name that you're using uh so previously by default, image package was crying to see if you had gcp installed or if you were in, if you had like the Azure tooling installed and tried to connect to it or the eks and so on, and then went on and tried to authenticate you with a different method so that stop that no longer happens, uh and now you have to explicitly tell it.

D

I am on this particular I asked and I wanted to enable that keychain. So this is this, this is a little. This is a breaking breaking changes here, because we change the variable name that you use to provide that information. So I was listening and.

B

I understand all of that, but my specific question was about what you were talking to about authoring with multiple is.

D

Providers yeah is.

B

It a regression from the previous behavior that you were able to off with multiple and now you're, not able to.

D

When you say regression like that was fixed with this was.

B

It was it changed where, before you didn't have to specify anything, I have like say the Azure and gcpa tooling installed at the same time, can I do operations on both at the same time and now I can't.

D

So this only works for is right, so like you need to be on. That is in order for this to you to to be working like for this keychains to work. So there are like a couple of principles that need to have first.

B

D

All yeah I mean.

B

People do like, uh like Google Cloud stuff, on azure um like they're. What do you mean by a GPA cluster to Azure so like there's, there's weird Edge stuff? That I can imagine like it's not very common, but but I just want to know like if there was a behavior.

D

Before that's now not possible, so it is still possible. Right like ends this bug here. So on the previous version, we changed the behavior in itself of image package right.

B

D

The problem with the change of the behavior is that there is a bug on it. Where you couldn't provide multiple keychains. You could only provide one there. It was about right and the Buckle speaks on this version, so from 33 on you can again specify the keychains previously image package. I was doing it by default. Now you need to tell it I want to use these keychains. Okay.

B

So you can use multiple keychains, okay, perfect, okay, so so no there's! No regression! There was a bug with multiple things. Before now,.

D

There was a regression, but it was fixed. It yeah.

B

Okay, next, thanks.

D

Any more questions back to you, Nancy.

A

Thanks Joelle all right, moving on to status updates, what's the team been working on Project roadmap, anything changed or updated since last meeting.

B

I, don't think any changes have been made in here since the last meeting um this. This is always a good prompt for reyno and I to come in here and update it. uh So I think a number of these things are still holding true. It's just the timelines of them are needing to be adjusted um so Rhino and I can update that, but nothing to report on my end, renew anything that you want to add.

B

Oh I have updated the timelines for cap and key control, uh I think so the October time frame is there and we want to hold to that. So we are hoping to make a key control release, uh probably by 20th, and if there is I think if there's a cap controller, it is also happening. I think you should sync up on that.

A

Okay awesome. Thank you. Both any questions on the roadmap.

A

All right, moving on to backlog, stuff uh image package, working on improving the output of describe command Jo wow on to you, maybe.

D

I can use some light on that uh since I'm working on it. So it's basically previously uh the image package, if you're trying to describe a a bundle and if it fail for some reason to get any image, it would just fail and it just stopped process right uh in this.

D

This was causing people uh having some problems, especially because we also by default, try to find if there are images, if there are signatures associated with uh the bundles in some Registries behave in a strange way where they, instead of saying that they can't find it, they give you an an authorizer, so they just stop like if, for some reason, you didn't have a a signature to just fail and you wouldn't get any permission right so this this is going to change the the output a little bit where it's going to do everything that it can and in the end, it's going to display you the full information that it has, including if there was an error trying to to find those images.

D

So that's the work, that's implied on that.

B

A

Any questions on that.

A

All right, moving on to vendor working on some found bugs and preparing to make a new release.

D

Nobody wants to shine me and I can't mean on this one as well. uh So um we've we've had some some contributions here. um We uh Winder is about to be uh released, a new version of it that contains a couple of interesting things. For example, you will not need to have image package installed anymore in order to use bindir. uh It will just have everything on it, so you don't need to solve any image package if you're pulling images in and things like that. So that's that's a new thing.

D

That's going to come up on the next release, uh that's uh already there and done um so. We found uh we found a bug that cemeterys is fixing right. Now, as we speak, and as soon as as we have that uh settled, we will.

D

um We will create the new version. So basically it is a problem where you can tell it tell when you're not to unpack a particular tar file and if it's, if there's anything after to sync, it just fails because there's there's an issue uh so Judiciary is taking care of that. So at the same time, uh peeped uh well I think I got his name. I! Think it's! It's not! uh You made a he made a PR as well to to fix a little bit of a a problem on video.

D

uh That was the fact that, even if there was no changing nothing to be pulled from Winter, the second time you do like a vendorsync vindi would rewrite the the log file, and this was causing some problems, for example, for make files. He had a make file where he was running something if the the vendir log file was changed, and this was uh causing it every time that he did a vendor sink it.

D

The make file was the soon and like the file of the new date, so um it was uh always running those those actions and now from from after his PR uh the image back uh sorry, the video lonely write the file if there's something new, so those are like. Basically, the three big things I believe are going to come out on the next version of India.

D

That will happen somewhere this week. I hope.

D

Any questions about the 20 minutes of talk from me.

D

Awesome Nancy take it on from here all.

A

Righty uh next up we got cap.

A

Exploring how we can be as kind as possible to the API server wrapped up work around the new Max duration, annotation.

E

Yep, oh I could probably add, like some context to that, so Praveen and tanoshi have been sort of exploring like making a tree of sorts of all the calls that cap makes today and they sort of trying to figure out what are the source Parts like, which calls are more frequent or more taxing on the API server and sort of figuring out. If we can do things better and uh the second annotation is something that allows users to specify a Max duration for a resource using an annotation um I'm.

E

Sorry, it's not called Max duration, it's actually called renew duration, and um what that does is the main use case was actually for. Let's say a user wants to check if a certificate is older than a certain amount of time and trigger an update on it if it is right and so yeah. So that's the use case- that's addressed using this annotation and we sort of completed that work last week, um yeah I think I think that's. It forecast.

A

Awesome. Thank you so much uh any questions on that.

A

All right on to K control, continuing refactoring work, removing cluster dependency, while package authoring wrapping up work on generating open API schema for Helm charts wrapped as packages who can talk more about this.

E

um Yeah, oh I'll probably talk about that as well, um so the refactoring work is just some like cleaning up that we've been doing uh nothing that the users will notice, and so uh the removing the cluster dependency here means.

E

So when, when kctrl is building images well like releasing a package, what it does, it uses cap controllers reconciler to run the commands, but instead of um deploying the deploying the resources, the end goal here is that cable treats a cable, config available, creates those images and then like resolves all of the images in the config. That's there so um the way it was set up today, the user would need to have a cluster connection to release packages which shouldn't be the case.

E

So that's something we are planning on changing and wrap uh so yeah, so the generating open, API schema for health charts is KC. Travel. Follow like provides users with like a three or four step process which lets you wrap a Helm chart uh into a Carvel package, and so for packages which have let's say we are building a package from config which has a mighty value schema files.

E

We were generating an open, API schema using that so what's new is if you're wrapping a health chart now we'll be reading the values.yaml file, that a lot of uncharts have commonly and generate an open, open, API key models out of it so yeah, uh that's something we're working on. We plan on closing that out this week,.

E

um Yeah, um that's it.

A

Thanks sumik uh any any other questions here,.

B

B

Do you want to add on the kind of comments that we are looking on to uh generate this open API schema.

F

uh Yeah, so uh what we have seen or uh is that the the helm expects a valid yaml file right and not necessarily that a valid yaml file will generate a valid open. Api schema as well. uh So there are. uh You know these are very, very edge cases, but uh maybe users can uh you know the uh not so properly well written Helen chart values. File can have the, for example, in uh in an array they can Define uh many items with the same key value pair, for example the environment.

F

It can have multiple uh name and value as a pair, so what we are doing is We are following what ytt does today, which is that, at least for help we will be generating the default value out of the only the first array item and will not consider generating from rest of the array items and based on the feedback we'll see.

F

If we want to enhance that so ytt actually limits that today that you can't have more than one items in the array, but on Helen there is no restriction like that, and then also around the the description for a key helm doesn't have any strict format over there.

F

Like the comments can be starting with hash or hash, and the key name or double hash, and things like that or like bitnami follows a very separate set of templating, so at least to generate the description for a key We are following hash: double hash and hash with a key name, so we will be generating based on that, there is anything else. Apart from that, maybe we will not. We are not doing that now.

A

All right, thanks, Roy.

F

So uh the any inputs like how we can improve it or if you have seen some other use cases which we should definitely handle, so those suggestions you can drop in.

A

All right, thank you so much any questions or comments.

A

Okay, um we've got about eight minutes left. Is there anything the team wishes to discuss before we part ways.

A

Okay, all right! Well thanks everyone for joining using us this week and we meet every second and fourth Thursday. So if you're watching this from home, please come join us live it's at 9am, Pacific time we love to meet you and hear more about how you are using Carvel. So with that, we hope to see you next time have a good day.