Red Hat OpenShift OKD Working Group, 11 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: OKD Working Group 2021 05 11 Full Meeting Recording

Description

OKD Working Group 2021 05 11 Full Meeting Recording
https://groups.google.com/forum/#!forum/okd-wg

https://okd.io

A

B

Has started all right folks welcome to this edition of the okd working group meeting. uh This is our meeting for may 11th of 2021, and I would encourage folks to um go to the uh meeting group notes and put in your name uh for the little uh attendance sheet there under may. 11Th on that link is uh in the uh invite and I'll also put it here in the chat, real, quick.

B

Here so we'll uh get started with uh vadim and then we'll move from there so vadim. What do you have for us in terms of updates.

C

um Some engineering points of on the yokiti we didn't release this weekend because our previous signing key has expired. We submitted a fix for the new one, but that fix is not being accepted by cbo. Yet we have a request to fix that. We would need it to get reviewed and hopefully this weekend will make a new 47 release other than that.

C

um I think just a bunch of small fixes landed in 47, so it shouldn't be anything groundbreaking um last week or was it the week before that we released a release candidate for okd for one for eight, the main difference is well, it's kubernetes, 121, based and the way we pack machine. Os content has changed significantly longer real life on fedora cos comets.

C

Instead, we build from the same configuration using the same rpms, but we build our own comet layer, cryo, all the necessary rpms like uh open, vm agent for um for alert and vm tools for vmware, um and that helps us to avoid using os extensions. So that should make initial setup and upgrades a bit faster, because all we need to do is just turn back the us3, and uh once you have the new deploy new os 3 deployment, you would be able to check the versions of the components using rpmqa command.

C

So that's very helpful to figure out what versions have been installed and most important releases with extensions. You no longer need access to fedora uh repos. In fact we disable them on every upgrade, so that should make it less uh rather more error. Prone um 48 is in so-called feature complete status, meaning significant features won't be added, but still more fixes are landing. So please stay tuned file, box report, issues and and general feedback would be very appreciated.

C

um I think that's all we have from the engineering standpoint.

B

There any particular bugs that stand out of uh in things that are filed right now that are open, that people would be able to help you with by testing or at least taking a look at.

A

B

Anything stand out in terms of issues, nothing.

C

Else to mind really.

B

C

I think um a usual testing of sphere upi especially, would be very useful.

C

We could ask for a copy of development version of documentation which lists some new features in 48 like proxy protocol support in ingress. That might be useful for those who want to preserve the forwarded, ips and some more features coming in parade. These would be very useful to test early and report feedback bootstrap in place. That one is a tricky beast, but some early testing would be very appreciated.

C

But something to fix right now, I think a lot most significant issues are in our infra and those are very uh because they require a lot of tinkering and a lot of carefulness so that we won't break nightlys at all.

C

But channel testing and for aid features would be great to have a look.

B

And so next thing I wanted to do is I wanted to sort of highlight the discussion section of the repo which has gotten a little bit of activity lately and um uh wanted to sort of go through these um to see if there's anything outstanding or just to bring it to people's attention. So one of the first ones that came in was uh the suggestion um uh to have exact links to um the fedora core os, um there's nothing that I know of right now. Actually, my script, the my oct cool tools.

B

Script has the ability to do that because it parses through the json to get it, but I don't know of anything else other than what we see coming forward.

B

um Any comments on that before we move to the next one. I think that would that was probably.

C

It effectively would be a part of that would be a part of openshift install sub command to print the exact url to iso, but it's coming in for eight. I don't think we can backport it at 247, but the fix long-term fix is in the pipes.

A

Yeah, sorry, it's it's easy to download the federalist images for install on offline or yeah yeah. So that's that's! The video we added in voice yeah.

B

And uh now bouncing over to um 609 automated solution to backup scd on a schedule from within the cluster. Where did we land on that? There was. A lot of discussion um is sri on uh today.

B

I don't see no, um but just to highlight this. I think this was a great discussion. um Automated solutions for backing up at cd, um so folks can check out that thread again for those that are just joining or for who are watching the video there's.

B

Now a discussion section of the okd repo that's opened up and folks are having sort of more nuanced technical discussions there about uh features and things related to website and whatnot and um yeah automatic backups would would be awesome for sure, um but there's some complexities there, obviously, um and okay, any comments or thoughts on that or anything folks want to mention before we move on from that one.

C

I think it's a great a good starting ticket for users who want to play with okidi on.

C

And we have various approaches to the same problem because well the use cases might be very different. Some want backups on requests, so maybe a text on task would be easier for that. Someone to persistent uh snapshots, so an operator is probably the best pick here. um I think various approaches having some code scripted for a start would be great and then we'll see which, which one is more widely adopted and wins effectively.

C

The other thoughts.

D

No something like that dude yeah um yeah, some crunch up that at least would be sufficient. I think for the most critical backups once a day, or I think I think I remember that we have to call a backup script that is always present on the notes to do a ptcd backup and such a cronjob simply could call that script regularly and yeah, maybe copy that away to external location that you can define by changing the current shop scripts like that.

C

Right, I think a current job with a container is a great starting point. The problem, is you don't get any reports if it fails, so that might.

D

Assume that it's.

C

Passing but it's actually even failing all along that's right.

D

C

And the restore procedure probably should be somehow I automate it, but that's something to look forward to. But uh the container with this, which launches the script on the notes is probably the most critical part reused in every single approach.

C

We tried, especially like so so um current job is a great starting place, which we can easily evolve into tax on pipeline, and then we could write a more complicated operator on top of that and have a proper reporting, maybe even biotech tone so that that would be a great showcase of how various technologies connect with each other.

B

Let's see if we can round up some folks there's a lot of ideas floating around it'd be good to sort of get a concerted effort, maybe get um a repo together, I'm happy to contribute to the tech town aspect of it and I think it would uh it'd be cool if a couple folks from the group got together and created a little subgroup to come up with something uh all right. Next one is transition to c groups v2.

B

I threw this in to sort of have documentation of the conversations between uh the fedora core os group and the okd group um video. If you just want to talk to this for a second basically, it's covered in there. But if there's anything you want to add.

C

Sorry, I missed which ticket I'll be discussing.

B

uh The transition to c groups v2 that fcos is doing.

C

Right um so we have all the basics in for eight nightlys, the only missing part is run c used by builders. So um all the features are working. We have a photocross which can enable it, but the test, failing because all the builds are effectively crashing immediately.

C

um What we could do is to experiment with building our own ogde setup with updated builder and where we pull it, the problem is that it might be using uh rail packages, so testing it in ci would be pretty complicated, but as an exercise to build a builder container using centos streams, fedora would be great is excellent, because all we have is a docker file starting part, and that would be very helpful because the ticket is well filed.

C

But it's not the priority for for age release for sure, but and we'll be pushing it to be released in 4 9.

B

Any other thoughts on that timothy. Do you have any uh thoughts you wanted to add to that.

A

um I'm looking at the picket uh details and uh yeah, even though it's not a goal right now for for openshift product. It's it's definitely on a radar. So I guess this. Should uh I don't know how it's going to get fixed exactly but uh yeah? That's it's still kind of a priority for us. So.

A

B

Knows have any comments and comments. Your thoughts on uh c groups, v2.

D

No opinion of that.

E

I should probably know this, but what are the advantages of c groups? Two versus one I mean: what's the? What do we gain from it.

C

The largest benefit is probably a fair actual fair. I o scheduling.

C

And I don't think kubernetes uses it, but what we could have is not just for the memory, but you can also have something like a middle or intermediate limit which tells the application that it's time to release a memory if it can do that.

C

So that allows you to schedule workloads in more well flexible fashion, but other than that. That's mostly a lot of low-level things which well just don't use parts of the kernel which are created in 2004.

C

That's all. We know.

A

Yeah, as definitely as valencia, the the main one is all the disk access checks, the disk access accounting and the memory accounting. I think everything makes sense in v2 much more than v1, where you have a lot of discrepancies between between accounting and memory or for processes.

A

F

Biggest thing, the biggest thing for me from a v2 perspective is you: can trivially, tie resources to a process and be able to track that through its children, because the groups aren't based on this concept of controllers of different types of resources, they're based on where you're instantiating a scope for a process. So control groups are actually singularly grouped in v2, whereas they're multiply um grouped in v1, and that makes it a lot harder to track and make sure that things are actually being allocated correctly and tracked.

F

And that's why a lot of things like umd, psi, tracking and stuff, like that? They all depend on v2, because you can connect all the resources. You're you're, allocating to the process in question that you're trying to instantiate. So in this case with a container that'll, instantiate, a scope and a slice and a and a resource, you can tile those resources directly to that and make it the distinct owner of those resources.

E

Cool something to look forward to then.

B

uh Next one up, so this one is, was an error or a perceived error. um So I'm looking at 622 vadim. Did you want the discussion section to be a place for people to put errors, or would you prefer that they actually go in the issues section, as opposed to in the discussion section.

C

Now I think, starting with the discussion, is a good idea. Well, two starting points are equally fair because we don't know if it's an issue in oakley initially or it's just here- I I might have typoed or using an old julie, so it might have been fixed. um The we can anyway convert any discussion. Should you into this discussion, so um that's interchangeable.

C

um I think it's a great place to share uh some logs and error lines, but of course, no guarantee that we'll look into this and uh somebody actually actively supporting this. So um discussions like that are probably a good place. We probably should create a new, um a special topic for this yeah yeah category.

C

I'm not sure how to name it correctly, but um I think it's a great idea to group these kind of discussions and.

B

Tickets um and and this one in particular, okay, d47 storage operator, degraded- um basically it's it's resolved. um Did you want to say anything about that vadim, add anything other than what you have.

C

It's a common uh problem in for seven, including a recipe where folks don't know what they want. They have a vsphere platform, but they're not sure if they would be using machine api or they would be using storage, they might switch to csi, which doesn't use entry drivers, so the credentials might or might not be valid and the installation we pass.

C

If you don't use those features with invalid credentials and in order to track that, we added a degraded condition for this to verify that if you have a vsphere cluster and your credentials are invalid, we won't proceed with upgrade because it might break a lot of things uh that caused a lot of discussion internally, because a whole bunch of people are using vsphere without using storage or machine api, so they use fake credentials.

C

The bottom line is that it will be eventually converted into an alert saying.

C

You may not use that, but you should be aware that your credentials are wrong and the upgrade would pass, but the admins would still be notified.

C

At this point, it's a requirement for the credentials to be correct, but might not be the case in 47 later.

C

I don't think I think it was uh it's a good starting point where we can link people, because it's not effectively an issue in okiti, but it's really a good example of how discussions should be how they should work.

C

B

Thank you. Anyone else uh have anything they want to chime in. On that one.

B

All right, uh I threw another one in here: uh that's from the f cos working group uh fcos moving ip tables to nft backend um vadim, again anything you want to add to that. It's pretty straightforward.

C

I don't think it should affect us significantly, mostly because um okay defaults to ovn, meaning the q proxy, the most fragile component here would not be using, is not in fact used at all and uh railroads.

C

If I remember correctly from day, one has been using nft, so all these cases have been tested and it's just us lagging behind. um In any case, the instant 4a to effectively control the whole configuration for 48.

C

um All we need to do is just pull one podman container and that's a basic case where um which, for the request ensures so once we can control the whole configuration and roll back the nft change. uh If we have a good reason to do that, but um I'm thinking, even if we hit issues these would be reported to the sdn team and they should be fixing it because eventually nft would be default in rail if it's not yet.

C

So I think if we have a good nand in case the the issues would appear.

B

Timothy, did you want to add anything to that or.

A

Yeah, I'm just thinking right now, but it needs its. um The nfd background has been default in real eight, so I agree.

A

It just realized that that actually we we were lagging behind in federacress in a sense, but it's not by choice, it's more by by mistake but yeah, so nothing should break on rkd side. Hopefully.

A

E

A

Oh sorry, yes, sorry, it's still the same interfaces. It's just changing the back end for ipc able. So essentially it's the same command line, the same tools just using a different vacuum underneath yeah.

B

Anyone else want to add anything to that.

D

I, like I, like that new format. I have to tell you that that we discuss about issues. I love that.

B

D

B

Well, we'll bring that feedback to diane. This is generally how I do meetings of this sort, but um we'll bring it back to diane and see what she says: um okay, so the next one that I added is um clarify. Okd's community support model.

B

I threw this one in because um red hat employees, one in particular, keeps getting uh personal emails and people tagging him in the channel and all sorts of things trying to get support from the person that they know is um sort of the red hat employee, that's working on okd, and so my thought is that we scour through the website, the repos and whatever and actually have a boilerplate a couple sentences, paragraph that says what does community support mean?

B

We have that in the big banner on the website, uh but we don't actually explain what is community supported and compute community driven and there's two reasons that this is important. I think number one um it's unfair and to the people who are red hat employees who are um providing their time uh to help out um and probably wears on them I'd. Imagine that it sort of um uh stresses them out.

B

uh The other thing is that, if more, if, if one person gets tagged and one person is the target for it, then the community can't help, because we know we've got our institutional knowledge and memory and whatever to help on these issues, and also we don't learn, because if we're not privy to these discussions or we're sort of boxed out of them, then the rest of us can't learn about these particular things. So there's there's multiple advantages to this. So we talked about this at the doc groups.

B

Meeting um doc's group meeting last tuesday and we'll be talking about it at the next meeting. If anyone um can join us at the doc's meeting uh to chip in on this um coming up with a couple sentences that we can put somewhere, uh then um that would be yeah bruce says reference the goose that laid the golden egg fable.

B

Then I think, if if we can come up with something in the next like week or so, and then just go through all of the okd references uh and plaster this up, so that we can um relieve the load on on the red hat employees who have been bearing so much um and and get ourselves more up to speed on things.

E

Sorry would it be worth putting in you know to the slack channel. Occasionally you know a boilerplate message indicating that you know this is community supported. You know, along with the working group email, you know saying that this is not vadim supported. I mean not specifically saying vadim, but you know the idea is that community members are helping. It's not just one person it just. You know once a week put something out there or whatever, so people get the hint. I mean I'm trying to help people.

E

I know vadim is, but you know, other people are gonna have to start jumping in also.

B

And we can do uh channel announcements and things like that for sure I think that's a great idea. Anyone else have thoughts.

G

Yeah jimmy uh it sort of just occurred to me earlier today that it actually might be useful to somehow put to get us some information of this is what you can do.

G

Okay, because uh I think what happens to a lot of people and as I was telling video, I would include myself on that list on occasion, especially when I'm tired and irritable or lazy, which is often is that you know the easiest thing to do is to seek help, but that that's not really a long-term strategy, and it's not that easy in the documentation to find out uh like we do have a faq and we do have some information there.

G

So that's certainly good as far as it goes, uh but it might be useful to try and pull together some like checklists or try this, or did you look at that or things that are important?

G

I don't know I mean it's, it's a big topic, uh so that's that's just a a vague idea of something that might help as well.

B

And we can always link to that website. What's the website, that's like how to ask questions, there's like an actual url, where someone set up a site that, like you, go to, and it's actually like. How do you ask questions for support and for for uh for troubleshooting and whatnot um john? Did I hear you say start something.

E

Yeah, I have a I have a delay here, so I think you're done, and I start talking or whatever I thought. One of the what vaneem said earlier is doing something you know how to debug. You know looking at you know when you get that huge log bundle, you know, how do you analyze it? I think something like that would be great um to send us a go. Go look at this youtube, video or something to give you. You know the basics. Is you know? How do you get the log stack?

E

You know, how do you do a basic analysis of it? What are the things in it that are important um and that might help yeah and it might help people who are who want to help too, because some of that stuff in there is very esoteric um and sometimes you just have to stumble across it. So um I know I'm not trying to put more work on vanim, though, but that might be well.

C

E

C

Yeah, that sounds like a that sounds like a very useful thing. I'm just not sure about the format. Should it be um a youtube thingy, a blog post um text.

G

C

All of the above.

G

Yeah blogs are easier to reference. You know with the with the video you sort of have to know okay, this was it uh one hour and 23 minutes. Okay, how do I find it uh can.

B

You do a short video. I can. I can see a youtube video having some advantages of like someone wanting.

E

B

Oh actually, how do you search through it? What particular things would you be looking for? I.

H

B

I yeah so, but this we shouldn't be limited.

C

I think I think we should start with a youtube video, because it's easier and I'm not sure which parts of the process should we focus on, for instance, there's a bunch of code which generates certificates. I have no idea how it works. um I can barely handle the open, ssl cli, so I probably would rather discuss the versions and the bootstrap process, but this thing also needs to be covered in some kind of a blog post and so on.

C

But initial response from the video would help us to shape up the basics of the of the markdown document we would put, and then we could extend it later on. uh Another concern is that things get outdated.

C

Yes, well, not that badly on blog post is still like top 10 in top 10 of openshift blog posts, and it still refers to openshift. Oh kitty, four five, some pieces there are very old and not used anymore, but in general it kind of works.

C

I suppose we would still have to update this document very very frequently in the beginning and then with every major release. We could we could update it, but it sounds like it's a required thing.

C

Yeah, I'm not sure if I would have time to create it from from scratch to the very end, but I guess I could have some.

C

I guess I could create in in a week or two until our next meeting. Some draft word helpful.

E

I mean if you want to put something together. You know you know a rough template. I mean I could try to put a video together. If you don't have time.

B

Yeah, I think that that would be, I think, john's making. A good point was really if we could just get some pointers from you, like just a template of of items that you think would be important to hit. The group can handle john himself, who can do the video anyone else? Can chip in and same with documentation. We don't want to add more work to you, but you're, the one who handles the majority of the tickets and also knows the in innards uh best of anyone in the group.

B

So if you just gave us a template, if these are the things to hit, then we could run with it in all of the the venues be it uh you know blog post or video, or anything like that.

D

Good idea, okay.

C

Sounds like a plan yeah.

B

Excellent anyone else have thoughts.

B

F

Well, it sounds fine to me, but I was also just going to say that I've got to drop now because I'm gonna go get uh go on my merry way to go. Get my second vaccine shot so uh I'll, see y'all later.

B

Awesome see you in health meal take care.

F

Yep you too bye.

D

D

I think it would be a good idea to have a recording um yeah, maybe a like a role play um if you get a lock bundle, what you do first and just to see some typical yeah typical steps that may be reproducible by others.

D

I I'm sure, because you are always so fast that you have some typical um spots where you look first.

B

Well, folks should feel free to um chip in so vadim can get us something within, let's say within the next month right. We don't want to put too much on his plate, but if vadim gets us a template, then folks from the group will all sort of divvy up the tasks of getting it into the various formats coming up with something in the various formats blog whatever and the docs group I'll mention this at the docs group, because there's some people going to the docs meeting that aren't coming to this and vice versa.

B

So all right anything else on this topic is there anything else that we can do that folks can think of to make it um to clarify what the model is. The support model is: is there anything else we can do other than providing this documentation and putting some boilerplate language into various uh places, though, where we have a presence anything else.

D

I think I think a problem may be that vadim has sure a connection to a huge internal network of people that know each corner of the system and most people from outside don't have that yeah, and I think I think yes, that's that's the reason why lots of people ask him yeah, because he is a the gateway to this internal network and we don't have it and if maybe it's someone has a good idea, how to get a substitute.

D

To know which array pose? Are they working on how we can contact them through issues and so on?.

C

um That shouldn't well happen. uh The internal knowledge is, of course, a huge source, but it's not being used in every single report. Most issues are very trivial um and I think, like three or four of our architects are in openshift dev, so you can think I can name drop later, but I don't think they would like it.

D

C

And the real the the real way to handle is probably starting with some. We have a bunch of issues logged for uh okay and showing activity. There would be very helpful. Just some basics um here is how I understand bootstrap here is what I see from log bundle. I'm stuck here, I don't know what's happening.

C

uh I could jump in and help and extend this of course, but a lot a lot of issues are just I'm thinking. Folks are thinking me directly just because I respond there and and respond in every single uh issue, and people assume that I'm the only one here. Well, um that's the belief.

B

All right any other thoughts on this.

E

Well, until we get cloning available, we can't clone him.

B

Right exactly, I remember, sweep.

D

I remember, as we were searching for um jon vadim and I was searching, were searching for an ovn kubernetes problem and, finally, um I think vadim got us the first first steps and john and me were searching through the community and finally, I think the guys at john. Do you remember at the network manager?

D

um um I don't know what the jet is called now they helped us and it's just. Finally, I think it was a solution, though you don't need.

C

Always a single.

D

C

I think it was a great example of how it should be structured. I, my network, knowledge is very limited and I would rather not extend it actually. um So this is why I would rather pass it to some professionals who can chase folks on irc and help with some details. I don't they don't fully understand.

C

We should we should learn lessons from this and structure our a similar fashion.

E

And it may be that there are bugs that you know you know like we did. We created that sort of private slack group or whatever with the three of us, but there may be issues that you know we can get if we need to get three or four people in where it's easier to have a discussion in that private channel. You know versus the open discussion, then publish the the findings afterwards, because sometimes you get chime in after chime in after time and and it gets distracting.

E

It's all good, but distracting yeah.

C

There was, there was a good good idea, which we implemented here. Some ad hoc chats for the interested people was certainly a good.

C

A good idea, yeah.

B

Well, let's uh move on now I think we we've got a great foundation now for this particular topic uh to move forward with um and I'll go back and fill in the discussion uh item with sort of what we discussed here at the meeting, so that we've got sort of a clear record in that actual threat, and that's the other thing is: if you put something in the discussions and we talk about it at the meeting, if you can update uh the discussion item, if you created it folks, that's helpful um and sort of keeps things organized uh and then the last thing in the discussions is insights operator is degraded.

B

Vadim. Is there anything you wanted to add on that? One.

C

No, I think we need logs, because there are multiple um things which could be causing it. It could be proxy, it could be actual well unlikely, but uh actual uh downtime of the inside separator for some short time. um Well, if we would have logs we would. We would have some something to discuss there. Yeah.

B

We'll point out that vadim says that's great, but I'm not a dedicated okd support person and this community should have access to it, and I do think that's an important thing again is the community. You know we can't lift the community up if we don't have access to these logs as well.

B

We talked about um the issue of identifying information. um Does anyone did anyone come up with or know of like just a simple bash script that cleans things up um to make it easier that we could just post and people could download it and scrub anything?

B

You know, because you know post names, people feel that that's identifying and you know any ip numbers and stuff like that. Maybe we just come up with a simple bash script that cleans that out.

C

B

Have something like.

C

And now for skater, or something like that? Yes, the problem is that they have to be consistent. You can't replace every host name with localhost and well expect us to make sense of it.

C

That's a well that's a complicated topic, of course, if we scrub to art that might not help us say, maybe your password is wrong or maybe well you scrubbed it wrong. um Yeah. We don't have a great solution for this, especially anything privacy related solution.

B

C

Uploading this.

B

Sorry hold on, who is that I didn't see? Who was that was talking.

H

Hi, uh here's eric um what is the sos tool doing, because this is now also scratched with logs. I think, since maybe half a year ago,.

E

Yeah but aren't sos logs going to red hat and stuff. I mean you have a certain concept of security if things are being sent out. Yes,.

C

We have the whole customer, we have the whole customer portal and gdpr things and ccsa, which was as a community and okay. You certainly don't want to deal with um and legal. Yes,.

H

um Is the shell tool right? You call it and it collects stuff for you.

C

H

C

The problem is sharing the results. The log bundle is built in a similar way, but uh some information might be considered sensitive. Let's put it like this, I don't think host names are sensitive, some folks thinks they are um and we need some way how to have them replaced with some fake names, but still make sense. In the end, um that's.

H

Virtually impossible.

C

um No, rather it was not built for that and another problem is we also copy a lot of certificates which effectively, in the end, have the host names embedded in them? You just have to extract them.

C

Probably our solution would be to avoid scrubbing but do up long, um temporary places which would have to delete it after a couple of hours days.

C

That would give us the full information and severely limit the time for attack.

C

Again, this is a security throw obscurity, but that's probably the best we've got if we have any other ideas.

C

um That would be very welcome because the log bundles are expected to not have all the sensitive information, and the same applies to must gather, but in order to identify all the issues they effectively have to also read data from user name spaces so that we could understand. Maybe it's a pdb blocking the upgrade or something.

B

um I don't want to spend too much time on this, because we've only got 14 minutes left and we've. We actually had a larger discussion a couple months ago about this that took up a significant amount of time, let's regroup on this at the next meeting, but maybe if we came up with a list of things that people do feel are um uh concerning uh uh items from the logs that are concerning. If we generated a list and then said okay, how could we tackle this? Can we tackle this?

B

Then we actually know what we're looking at, um and some folks in the group may not be familiar with, must gather so, let's, let's table this, but in the meantime, if folks, who, just in their minds who are familiar with, must gather think about some of the things that would be problematic and maybe I'll send something to the group um share out like a google doc or maybe discussions yeah in the discussions.

B

Actually, we could do it in there um in the discussions just generate a list of things that we think could be viewed as problematic and again we're just we're having to put ourselves in other people's place and and what they would think would be um problematic in that and uh then from there um at the next meeting or future meeting discuss ways in which we could ally those concerns. A little bit. Does that sound good.

B

Yes, okay, all right uh and that's it for the discussions. um uh Timothy did you want to bring up anything from the uh fedora coral west world that that you think that uh folks should know about.

A

Some right now on the federal side. We did this last week on testing for federal 34, but that's not a direct impact on otd, because ogd doesn't switch when federal court switches switch when machine os content gets pushed uh but yeah. So essentially it's it's coming.

A

um Apart from that, I think uh we've already discussed c group d2, which is one thing: the move to nfg, which is second one. The account me changes are coming later in august, just a reminder for folks, starting from the releases in august. So maybe not the releases you know in august, but a little bit later defaults.

A

The the default will be on fedora cores that you will send account me requests to the fedora servers, so essentially it's a very privacy friendly way of counting the number of federal cross, node leavings running on the planet and for us to have some kind of statistics, of how many peoples are running federal courts and yeah. So this one is coming around august and there's already instructions. There's a magazine article coming up like probably next week or this week or next week, to explain how to disable that.

A

Maybe we could have something specific for key to disable started, but it's essentially a two-line machine kind of thing, so it should be fairly easy and yeah.

B

A

Think we have anything more like really really important right now happening beside all of that, so I guess should be good.

B

uh Did you want to talk a little bit about about the signing discussion that was that took up a lot of time at the last meeting and it I think it does relate to okd nodes in the way that people might keep them around for a while if their cluster is running a long time, et cetera.

A

True, but just I I think about that, one, but I'm does it really, I'm not sure it impacts okd, because the updates on kd does not happen the same way. They happen on fedora, crest classic.

A

I would say so. Maybe maybe I'm wrong here, but I think okd should not be impacted by that. So the the issue in general is that if you start from a very old federacy node and you update, you try to update to a fresher version, you'll get issues because you won't get. You won't have the signing keys to verify that the latest version is actually a valid one, because you're in a very old node and potentially you're out you you're way out.

A

You don't have the latest keys to verify that, because on federacrs we well in general, we only ship the next two releases keys um in in in an image. So if you're, starting on fedora 30 for example- and you want to upgrade to fedora 35- then you won't have the key for federal certified, but that should not be an issue as far as I know on the kitty, because, uh honestly you, the updates happen.

A

They are at the mco so and, and the machine is content, you beat it on the cluster and there's no it's uh it's. It's not it's not pulling uh any content from a from a sign or repo. So it's it's not exactly updating the same way that we update the classic federal course notes. So yeah I made I'm not 100 sure so, maybe about him. You could confirm that, but uh yeah.

C

um Right there is a very unlikely case. Bruce has mentioned it that if you start with the dora third say core os is a bootstrap note, and then you want to install the key, which is probably based on fedora 49.

C

That case you would of course fail, but that's pretty easy to prevent by using a fresher federal cross.

C

As for upgrades right, we don't use native core os mechanism, we upgrade from one major version to the other, so it may not even span fedora releases at all, but in the worst case scenarios fedora is 33 to 34 and we can also pull out the edge to prevent that upgrade from happening. If we find some issues with that um so yeah, I don't think that would affect us.

B

I just wanted to bring that up in case it did, and at least we're aware of that, which I think is a significant change, because there are people going between both groups and sort of playing with fcos and playing with okd. At the same time, um in the last few minutes we have seven minutes left. I want to talk about um the uh kubecon office hour. What did we learn from that uh in terms of our audience in terms of our um uh our ability to communicate our ideas, our ability to answer questions?

B

What how do folks think that came out uh for those that were there for folks that haven't? um We can put the link to the video which was posted.

D

For me, I had big problems um to follow the correct chats, because there were lots of chats and q and a chats and main chat and background chat. I think it's a little bit too much. um I don't know, what's your impression.

B

I think that's why they have a moderating person to sort of sift through the questions for us right.

E

Yeah, a lot of.

C

E

C

Following the twitch chat was apparently.

E

C

Yeah I was following the twitch chat and apparently had copies from everywhere, like at least from youtube, but following the moderator was apparently a better idea, because they know how to like switch topics and slowly move from one topic to the other. So you wouldn't have a whole mess of different chats.

C

um My impression that was that it was great except when folks started uh with, what's coming in four eight and four nine, which are very technical issues. um We shouldn't have to answer them, but we should have a prepared answer like here is where we post the release. Notes here are the links to our workgroup meetings and so on.

C

um What I think I would add, some more discussions about that. It's in not just a free version of ocp, which is not true, but it's rather a community version where you can affect every single detail of your okd cluster folks who are asking why you're not starting with railcar os. That's exactly the reason, because the community cannot contribute to it directly. They would have to go all the way from upstream to fedora to rail, and that shows the value like.

C

If you found a bug in it, you don't have to wait for rail support to pass it to engineers, have it fixed and pass it back. You can start tinkering with it immediately. Just like we did a lot of times.

C

uh The next suggestion is probably mostly for internal developers, but uh we we should mention that with a lot of things which landed in ocp like the whole ignition version, 3 started inokiti and later only based on a bunch of fixes.

C

All of us have submitted landed in ocp and the whole ovn as a default was a great testing platform for the folks to decide. Should it be a default sdn or not.

C

uh That means you might hit some more bugs yes, but that also means you also get a feeling of um how the distribution would look in in a couple of years. The whole city groups it's a perfect example.

C

We might want to enable this and ok if it works great if it brings benefit to the community, because ocp is more conservative and we have our hands on time. We can do whatever um yeah complex topic is because folks might think that we test new features on them.

C

This is not well, that's not entirely true um and probably uh that's a very, very complicated topic, but once we should actually make a point of it that uh okd is a place where you can get latest and greatest features and since the whole community is looking at it, your chances to have them fixed sooner, are actually multiplying, but other than that, I think it was.

C

It was great. It felt refreshing to see a lot of new faces, asking questions. Both the the beginner ones and more complex ones feels like the community is actually growing and look. It is not even one year yet we'll get e4, I mean so that felt um very rewarding.

B

I thought one thing is that we'll have to find um this is assuming that it uh moves forward. Diane was saying that if, if it went well that we had the chance of doing like a bi-weekly, um I think that would be fantastic. I haven't heard back. If that's the case, um one thing I think we would want to do is find what's the right level of support to provide in these office hours. How are we going to start looking at people's logs?

B

I we got a question along those lines of like here's, a big log error. Are we going to start looking at that or at when you know we'll have to figure out? What's the threshold low and high um for saying, okay, this needs to happen offline from the show, because otherwise it would take up. You know a significant amount of time on this one particular issue and we wouldn't get to anyone else um so that that would be my only thought. um Anyone else.

A

I I guess um well I don't know the weekly might be a little bit too much for for the office hour, but certainly not up to me to decide um but yeah. We could maybe monthly or something like that and uh well. I I'm not sure office hours are great for debugging session either, because they'll just pin down a lot of people for just one issue, one specific issue so yeah, um I'm not really in favor of that but yeah. It's uh it's still open for question.

A

E

Yeah, I think live debugging is a little chancy.

B

All right, we have one minute left, and I do want to be mindful of people's time. uh Is there anything else that uh folks want to bring to our attention before we um uh step away from this meeting.

B

All right well, thank you so much folks for your time and thank.

A

You for bringing.

B

Your meeting oh good, meeting jamie yeah, you're you're, welcome um and if folks like this format, then um we can, um you know mention it to diane, um I'm happy to facilitate in the future. I don't I don't know what what's uh uh behind that, but uh she might be interested in letting me co-chair or whatever for we'll see. I don't know if it has to be a red hat person or not, but um uh this was great and uh we'll uh talk in two weeks and also online and use the discussions and don't forget.

B

The dots meeting is next tuesday. If you're interested in doc's.

E

E