Cloud Foundry CF on K8s Forum, 31 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cloud Foundry on Kubernetes Working Group Forum May 31 2022

Description

[JH in absentia] Logging Format Proposal
[KB] Use of eventReporter in controllers
User friendly debug info
Updates info
Example: https://github.com/cloudfoundry/korifi/pull/1155/files#diff-58eb0aa24cc37d5046b0ce2c2ecebd69d2e006014778c2f72c3a263776ddd745R87
[BR] Korifi 0.1.0 https://github.com/cloudfoundry/korifi/releases/tag/v0.1.0

A

I have linked the working group form document in the chat. um I can link it again. If you have any topics you'd like to add. Please do so.

A

um We can get the recording start at any point that you'd, like uh we don't have a ton of topics right now, but feel free to add them. The uh the document or uh and zoom chat, I can add them uh for people, um but once again thank you for attending the bi-weekly uh cf on kubernetes working group form and we'll get rolling on the topics, so the first one we have is, I believe, from julian. Even in absentia is a vlogging format proposal. There's a document linked there.

A

You can do a site reading if you'd like for a few minutes and then sort of start to talk about. It also provides a few minutes for everyone to add topics or think about. If there's anything they'd like to bring up.

A

Usually, how we do the silent reading is you shut off your webcam and then, when you're ready to come back, you re-enable it. So I'm gonna go ahead and do that. So I can read this really quickly and I'll. Join you again for a few minutes.

A

uh For those of you that are just joining us, we're looking at the document and there's a proposal about logging that we're looking at, I can link both of those directly and we're just doing searches. You might be tend to be muted, I'm not sure, uh but anyway, so I'm adding the uh links to the chat.

A

So everyone can go, read it um the normal procedure. Is you turn off your your webcam? While you read it and then we come back and all that but uh feel free to do whatever you want alrighty.

A

So uh the proposal in brief is about how we should standardize all of our logging formats and what information we should contain in that and how we should structure it. So the gist of it is uh that it would be nice to sort of have all the information we want in a machine, readable format and sort of align everything within that so I'll. Let uh anyone else that has an opinion about that. Kick it off before I offer my thoughts.

B

Looks perfectly all right to me: uh it's definitely good to have consistency. The format seems sensible, so no objections really definitely an improvement.

C

In some of the bad examples it had logging coming through from like controller runtime. um Hopefully it's easy enough to go and make that use the same logo as us. ah Isn't that a really example it was, it was trying to do a leadership leader election. Wasn't it um and we use their like pre-made code for that, but I think there's a method where you can tell it which logger to use and that should do the.

B

Trick I mean it's generated code. Isn't it I mean, I guess up top point? Yes and then it's library code.

A

Yeah, so that that is one aspect that I hadn't really thought as much about, um but we can kind of explore that and hopefully find out what the bounds of the logging format are in the controller.

A

um But in general I think it's a good idea to keep everything pretty well standardized and having a machine. Readable format is always good because you can have anyone can.

D

Write their own parser.

A

To extract the information rather than just magic strings, so personally, I think it's probably a good idea to move forward with this and sort of explore, but I would be happy to hear other pitfalls because, as you mentioned, some things have their built-in loggers, so we would have to figure out how to configure those so yeah yeah. We.

E

B

Have um a bunch of ideas like we have a bunch of ideas of changes, we'd like to make around logging? This is one of them. I think we have other. We should have proposes about other stuff as well like in general, login style like deciding when and what to log, and then we have the one about like.

B

I guess, we've already done some work on the. What is it the unique id being how's that called the correlation.

D

Correlation ids.

B

We have something already, though, so maybe that's fine.

D

Yeah this one touches a little bit on what the log hill usa says: don't log client level errors and definitely I don't think we're logging any secret contents, but it hammers home. Don't do that.

B

Yeah, it's like it's one of the classical thing that, like you, you kind of neglect a little bit at the beginning, and then you have to go back through the whole code base and be like okay, let's review all logs and see if they make sense.

D

Yeah structure-wise, this all makes sense to me it's just like a hybrid of what we've moved to with a lot of the cf for vms components in their um date time format, along with how kubernetes like json logs look. So that sounds good.

A

Cool well, I then think we'll probably move forward with that and just sort of explore out get some more information and yeah it. uh It should, regardless of where we end up, it should be progress in the right direction, so cool.

A

uh The other topic I see if we're done discussing vlogs is uh use of event. Recorder and controllers.

C

um Yeah, um so we started writing a task reconciler this morning um and that's just creating an arena task object, and then you see the job get created, there's lots of things that can go wrong there, because the tasks needs an out breath on it. Yeah prep needs to exist the uh so yeah yeah. The app needs to exist for that ref.

C

The app needs to have a current droplet on it that build needs to exist. That build needs to have a droplet image on it.

C

As you go through those things I mean your code has to take different paths, and um I don't know you can return the error and try again and you can log it, but it's not particularly useful for the user.

C

um I was just thinking how you see that when pods go wrong, like you know, it can't grab the image to run the pod and you you describe it in cube, cutting or canines, and you see the events saying you know: here's a whole load of user information saying that I can't grab this image from this repository credentials are wrong or can't ping it, and it turns out. That's really easy to do with controller runtime.

C

The manager thing has a method on it saying get event reporter for and he's just passing some string, and then you can just call event and event f on that passing in your objects that you're reconciling and it goes and shoves events on it and it deals with all the the nasty stuff you have to do like. If you send the message twice, it's got to go and patch an existing one and increment the counter on it.

C

You don't have to worry about all that stuff, so we've written one of those before in irini, which did all that nasty work and it was a pain. We clearly missed that thing that exists and control the runtime already. That does it all for you.

C

So I think it's a really good opportunity to get some user-friendly debug information out for when things are going slightly wrong. So it's not the sort of stuff that would go into a status when it's just like you know. Task is failing it's more like here's, something that's failing at the moment, because these reasons and when we retry it might go away, but it gives people an idea of what's going on so yeah, I don't know just suggesting that it might be a useful thing in all our different resources and reconcile us.

A

Yeah, that sounds pretty nice. um If there's a built-in library for this- and I think we should definitely leverage uh all that. So uh I will personally go explore that because I'm not as familiar with it but uh yeah, I would say, find the areas where we need to sort of work on this and open issues and let's go ahead and look at if it makes sense that people.

C

And maybe we can encourage people like k-pac to adopt it as well. Yeah.

B

Yeah, I mean there's probably a bunch of things that we log, but it's not that useful to log them, because we would actually like the users to see them and the users don't usually have access to our controller logs, so it would be better. To I mean we can still log them, but it would be good to also surface them by others, because it seems to be the only way given like how asynchronous the whole thing is seems to be like the only way to kind of surface issues.

B

I mean we when we use the kubernetes native resources, that's what we do like when we look at stuff on canines, etc. That's what we look at. We look at the events. If things is going are going wrong are not working, it's literally the only way we can get any sort of info, so it would be nice if we used it across the board on our resources as well.

C

Yeah, I think the system tidies up nicely as well, so these things I think they only last an hour or something and then they'll be right.

B

There collected.

C

Or automatically collected.

A

Cool well, we can definitely sort of start doing that and hopefully ask our friends in components to also look at that sort of thing. So I think it's it's worth, leaving the charge on that so yeah.

A

Cool. Thank you uh anonymous, otter for adding a link in there.

D

Yeah, it's weird that it's calling me that, because I I logged in, but it still like, tells me I'm anonymous like every other. Second, I don't know what's up with it.

A

I'm sorry it's doing that to you tim, but uh I thank you for your contribution. Whether in human or otter, form cool, oh yeah, that's our yeah, okay cool! um I don't see any other topics, so the brief one I will add on just for funsies is that we had a release. 0.1.0.

B

Oh, we haven't talked about it yet yeah.

E

E

A

uh Major milestone for us to have a release out there for people to play with uh still very much an active development. Obviously, so a lot of things are going on, but we kind of reached the point where we felt like we could drop it out for people like play with experiment with and give us feedback so that we can continue evolving the product.

A

That's got all the basic functionality we wanted, um but yeah there's always room for improvement. So if you find some rough edges, please let us know join us, contribute open full requests, whatever you feel like um yeah. So I just wanted to drop that in there that all right, we got a release up there. The first of many so um congrats to everyone. That's worked on it and thank you for your hard work and I hope we can continue charging ahead and making it the best product. We can.

B

Yeah, I saw an issue from a few days ago uh complaining about the installed instructions because the installer instructions mentioned um like they, they kind of refer to files that come in the table that you download from from the release page, but the person was trying to use just a checked out copy instead like a git copy which will not have those files because uh those are generated.

B

So I wonder, maybe we need to update the install instructions to make it very clear that you need a release. Tarball, not the not the checked out copy because they checked out copy, uh doesn't work that way. I pointed them to the hacking file. I don't know how up-to-date that is but like if you want to use the checked out copy, that's what you should do like the flow is different. You need to use the make file and stuff, but yeah other than that. I guess pretty good.

B

I def I manually tested the install instructions on that table, so I know they work.

E

E

A

Yeah we're good uh yeah, I mean that's, obviously, a point where we want to shore up any weaknesses is to make sure our instructions are at least clear, if not the most simple, for people to follow. So I mean.

B

It's cool, it means someone actually tried to install it.

E

A

Good yeah, so uh we'll we'll try to think about that going forward, try to keep an eye out for making sure our instructions are at least helpful.

A

So but, like you said at least, uh if you follow the released hardball flow that they work so we'll just have to keep improving there. So yeah.

F

Awesome work, everybody really happy, uh they got cut and um sorry if we pressured you a little bit and cutting the release when we were a couple, articles came out like two days before uh saying that it had been released, but anyway it's all good now and um yeah. Congratulations everyone um also. I will mention that we are uh definitely plan on going to kubecon detroit.

F

um So, hopefully lots of you can attend that and work the booth on the floor um rom and I should be there some folks from paketo should be there. uh Hopefully, lots of 500 folks will be there for for the cloud foundry event and also the cfp for kubecon detroit closes on june 3rd, which is very very soon so if you have been thinking about putting anything in um or if you can get them together in the next few days, um that'd be great. That'll save your employer.

F

Some some money on the kubecon ticket, uh among other things, um be great to you, know, get a talk or two approved so um yeah. That's that we haven't actually even signed anything on the event yet, but that is going to happen um it's so early, but um that we'll make that more public with the cloud founder community uh in the next week.

A

Awesome uh excited to hear that should be fun to get up into the community and have some fun palling around with everyone and yeah coupons, always a great time. I see in chat a message from paul. Thank you so much for your comment. um All decided to take around and play with it and found that uh on a local install on a macbook air of kind installed in under seven minutes, which is you know, always a good experience when you can install these things in less than an hour uh yeah. So awesome.

A

Thank you for the comment. Paul, that's awesome. I hope you uh continue to experiment with it so yeah.

A

uh If there are any other comments uh or questions topics, people would like to add right now. Please make it known, uh if not, then we'll kind of take a little break for the next week and change until we come back, hopefully with some feedback from the release and some new exciting topics to pursue.

A

All right, I'm not seeing too many panicked faces or a flurry of chat, so I'm gonna go ahead and say. uh Thank you for your time for attending uh we'd love to see you all again in two weeks and just you know, have a great time and feel free to leave any uh comments in the slack channel or uh on issues on the repo or anything like that. We'd love to hear from you. We want to make sure we're doing the best we can for you, so yeah thank y'all.

A