Ceph Developer Monthly, 5 Apr 2023

Previous Meeting

Next Meeting

⏯

youtube image

►

From YouTube: Ceph Developer Monthly 2023-04-05

Description

Join us monthly for the Ceph Developer meeting: https://ceph.io/en/community/meetups

Ceph website: https://ceph.io
Ceph blog: https://ceph.io/en/news/blog/
Contribute to Ceph: https://ceph.io/en/developers/contribute
What is Ceph: https://ceph.io/en/discover/

A

Let's start hello: uh this is a developer monthly Emira uh friendly Edition today is a 5th of April I'm driving uh I'm igniting uh this meeting on behalf of Laura. uh She ran into uh into a conflicting meeting. uh So uh let me, let me pinpoint the today's agenda. Actually, there is uh one. Only one item there uh introduce open Euler to save Upstream by Kevin is out.

B

A

Hello: everyone, hello, okay,.

B

So, uh first day it's my honor to share this information here to the safe monthly. So uh in fact that we have already shared this information to the self Dev and the user user monthly uh several weeks ago. So uh so, basically, the idea here is to uh introduce life to the open, Euler operation system and to, and we want to make sure that uh and doing some contribution to make sure that the staff Works mostly on the open, yield operation system. So there are some. Basically a background is open.

B

Usually is uh it is a open source operation system uh which was born three years ago and now it's a Chinese largest open source operation system, community and so uh and also it has a lot of the users and the commercial releases. So, for example, uh some uh some company, like Susie, has using the open, Euler Upstream to build their commercial release and so inside open Euros. There is a SDS seek, so we are working. It has gathered.

A

Oops can't hear you anymore.

B

A

Okay, now you are back.

B

Oh, oh, sorry, sorry, yeah yeah! So, okay! So could you hear me now.

A

Logan clearly.

B

Oh okay, so now it's better now or not.

A

No glitches uh sounds good.

B

Okay, good good yeah, so uh yeah so uh currently inside open you last year is there is a stsc, so it has gathered a lot of the developers and the company there so and also the users has a desire, a strong desire for the itself on over Europe, and we have doing some. uh We have. We have already doing some uh capability uh ties down the development to make sure that the open, Euler LGS support now can running the size so and also we have submitted pages to the safe Upstream.

B

So and- and also we want to see that yes, so uh let us just introduce yourself to a new average system and we want to see that the is there a foremost Dives for introducing the site uh to a new operation system. For example. What is the next step to doing uh maybe aiding the Ci or yeah in the CI and adding the resources or or something else so and- and we also want to hear some feedback from the step upstream and from the open, Euro side we can commit that we will contribution.

B

We will continue contribution to uh the self support on the open user, and also uh we will also how to see that if there is uh some new user cases for ciphon wheel yeah. So that is what I want to share today so and also want to see the state of feedback from the dev Upstream.

A

Well, that sounds that sounds really good. uh From my personal uh perspective, uh let me I will try to address your uh your last question about uh well, basically, the places related to supporting a platform instead, according to my uh to my best. No, according to my fragmentation, uh fragmental knowledge.

A

uh There are multiple uh things here, starting from the technology testing technology is basically automated uh UA for uh for that, and we do starboard multiple platforms. There, however, tautology just uh just tests, there is a builder infrastructure. That's another place where operating uh that is aware about an operating system aware about uh a new distribution.

A

There is web interface, shaman safecom,.

A

uh There are also Jenkins notes responsible for uh for validating uh PRS. You can see uh you can see uh the results of of doing that. Even uh under your ER, there should be Auto a bunch of a bunch of uh text, summary objects. We have things like the API test, but but it's interesting from from the from this perspective, it's like some objects like make check the version for a arm 64 make the general make track Etc, so it that's another Junction uh with an operating system.

A

A

Please go high right: okay, uh I was actually I was I was finishing, uh I'm, pretty sure this. This list is not exhaustive.

A

Even it comes to former uh validation releases, Etc I think I I'm, pretty sure that there is a lot of things I, just uh I, just forgotten, yeah I would we would be super cool to hear uh output from uh from other folks.

B

A

But literally speaking, it's along and and staged process and I personally I think you don't need to uh accomplish all step all stages in one. In the same moment, it could be a gradual.

B

A

So the first, the first step is, is to uh is to tune uh the the build the specifications for building packages which you already did.

B

Yeah yeah so from uh the open, Euro side, so uh our first step may be uh produce the uh packages, so that would be better uh and from from my point, I I see that.

B

There are some architecture related tests here so, for example, the accident 6 test and also the unsaturated, so did that one also needed for introducing a new operation system to running on a test, for example running the May check or the Tesla adverty test, uh uh uh so I'm not quite sure. This is also very. uh This is this is also an essential style here. So two two two four four new operation system to do this or this or that one are more related with the architecture side.

A

uh Which one in particular.

B

Oh I I mean that the Matrix test.

A

Success I see yeah.

C

I, don't think uh uh so. It's like it's like uh Radix, said. uh Most of these steps are you know at a bare minimum. uh There needs to be. You know, packages need to build, but yeah uh you know make check. So, for example, uh we uh just just as an example uh I think we do uh builds for CSA, so I have at least one csm-based distribution. If you look at the shaman.7.com uh which, which rather stuff linked in the chat there is a uh there, are repositories produced for.

C

Well, I'm I'm, not sure if the repositories are actually produced, but the builds uh I mean we certainly built for for opens user uh 15.3.

C

It looks like uh and and the the reason I'm bringing this example is that uh there is, um there is a build, but it's not used for make check because, uh like we just don't have the uh the resources in in our Upstream lab to run, make check you know on on each pull request on each change every time something is pushed on multiple distributions, uh so we do this like so so we don't do this once user, even though the build is produced, and the same goes for pathology testing. So we we do test.

C

uh uh You know generally, we test in uh and Debian based distribution, and this happens to be Ubuntu uh and uh to uh RPM based distributions. uh This happens. You know this is uh this is uh Centos and Rel. uh My understanding is that open Euler is a kind of a direct descendant of from Centos and.

B

C

Yeah and the spec file- actually you know uh like pretty much all your changes, except for the um except for the RPM config package are just uh you know, you're, just adding an additional condition for, um and it behaves exactly the same as I say it looks like you're expected to behave exactly the same as Fedora or Rel. So from that perspective uh again, I don't think there is a requirement to run uh all tests. You know make check pathology. You know everything that we do. uh We just don't have the resources.

C

We don't have the hardware to duplicate that. um Okay, okay, but it could be. uh You know, use and build, for example, could be set up uh and uh you know maybe maybe so that it runs.

C

uh You know just like this user build, so we don't produce the the actual um I'm, not sure if the packages actually produced, but at least you get the notification or you get the status, whether the build is red or green, uh but all of that is I think you know like right, exact, there's a kind of gradual, uh the first and obvious requirement is just the uh the spec file changes, um and so the question I had so that this was just a kind of uh to explain the background.

C

The question I had is you mentioned that you have a storage, Sig.

C

Do you have any any automated builds set up there.

B

Oh, we we have some uh automatic views uh so inside open usai uh so but uh we have not actually run a lot of times for each build yeah. So I just wonder so. If the community needs some resource, I mean the hardware resource or VM resources to execute the building in the self Upstream side. So if so, we can contribute some resources to the art to Upstream to help set up the automated job.

C

Oh, that that would that would be cool.

D

C

um You know I'm not sure what is the, what would be the best way to do it because, uh right now, all of our builds are you know the build machines are all hosted in uh in in our lab.

C

uh So it's just you know, they're physically, in the same in the same location, uh so I I, think you know some tweaks uh to the to the you know, build infrastructure scripts might be required to uh to kind of get them going uh for uh for a node or a set of nodes which are not posted in the same Data Center.

C

um So if you could, you know if you uh if it could be like if you're going to be able to provide access to a set of Hardware nodes? Yes or you know that could be virtual machines that you know it doesn't really matter just you know some execution environment uh where uh we could run. You know whether it's make check or you know, even just builds or starters uh May check would come later.

C

um uh There is uh I'm pretty sure there is. You know it's there's going to be a way to wire that up. uh It's just you know it just. It may not be entirely straightforward because, like I said right now, it's all uh you know it's all internal, and so there might be some dependencies on some uh internal services.

C

Definitely something to uh to investigate um so is this: like is this spec file uh that that is that you've submitted? You know the basically, what I'm looking for is to as an answer like is that spec file um like tested so is it? Is it known to work because yeah.

B

Yeah that one has.

C

B

Been tested so- and we also use this to produce the packages.

C

What what uh so, uh then a related question? You initially submitted a pull request against the Pacific branch, which leads me to believe that you are producing uh packages for the Pacific release, whereas uh the pull request that we actually you know like from an upstream perspective, the the the the way things happen is that they get merged into changes, get merged into the main branch, and then they get back ported to uh older. You know releases that are still yeah um yeah and the the question is Io Io.

C

Is it known to work on the main branch.

B

Actually, my colleagues has also some major of PR choosing Branch, so that can work also uh yeah, because so uh previously it has made the pr to the Pacific Branch, because the open, Euro LTS uh has a dedicated doing some compatible ties towards it with the specific. But actually uh the the main branch can also work so that you can see.

B

I listed the the length hair uh for photos, yeah and so so and I say that we should actually contribution to the membrane first and then Backpage to the dedicated branch that we would be uh better than the previous day plan.

C

Okay, uh yeah well as long as as as the spec file is known to work on the main branch. I see no reason, um you know no reason to block it uh or not to merge it um um I uh it's!

C

It poses virtually no risk to um uh to uh to the uh to the Fedora and Royal branches uh since you're, just adding an additional condition and for open Uber. So that's totally fine with me. um Okay, great I'll, just go ahead and improve it right now um and uh I. Think from there we can.

C

We can pick it up and just just make sure that it just make sure that the Centos and uh who had had like Corel packages uh still build, and then it continues I've added the meat Skillet label to hopefully make that happen uh in the next week or two okay. Great, thank you, I think! That's it.

B

C

Thank you. Thank you for the contribution.

B

Okay, well, that's my prior yeah and and thank.

C

B

Yeah so, and also so, uh we will see that uh and- and also so, you have reprodite the the what we are doing- some actually uh the regular building in our uh in open Euler Upstream side, so that one can.

B

uh So if, uh if it is hard to uh adding some uh CI job is to uh uh their job, is to the the uh yeah I mean the using the external resources for the say, a bigger inside Upstream, uh and it is it possible to doing some uh Ray link to the open Euler rifle for, for example, if the user wants to use the RPM based uh uh for, for example, if a user wants to use the rpm-based, uh open, Euler packages or open urine set packages.

B

So I'm just curious. So well.

C

As long as as long as you're going to provide them in the storage, seg repos, uh yes, I think we can definitely mention that uh somewhere on the 7.io website. uh This is uh you know it's the same thing for for the existing distributions that we. Currently they don't know that we that we support today uh there are packages that we, the Upstream team produce and those are hosted on download.sap.com.

C

But then there are also packages that are built by the distribution teams, uh so Ubuntu builds their own and then toss uh storage. Sig. Does that too? And those are those are different sets of packages uh so open Euler could just you know you, the the storage thing in your organization could produce uh the packages and open the user. Users would definitely be expected to use those.

C

uh Instead of you know the packages from download.com, which are only Built, For, You Know, in this case Centos. That might be fairly close, but still you know it's, it's not what what an open uh either a user would want to to deal with.

C

Okay, that's good, even if even if they are, you know, relatively close. uh As far as the content.

B

E

Hey Kevin, um just on the name, I thought I. Would uh it's very commonly mispronounced. It's actually Euler. Oh.

B

Okay, a Wheeler, oh all, right, good, yeah,.

E

Spoiler, like uh like, like uh oil used to for cars, spoiler all.

B

Right good, thank you.

A

And just a quick note on the uh PR, uh if you will be able to put some information about the testing procedure, you know taking into account uh ability of other people to replicate, it will be super cool. It will be super helpful uh to have everything uh in one in one place. Basically, the comment plus some instructions examples how to how to verify that. If this will go to the comment, description.

B

All right, so so you you mean that adding some information for other developers to reproduce right.

A

I'm trying to put I'm trying to put myself into a shoes, a developer that reads this comment: let's say six months from now, all right: okay, how to verify.

C

Yeah and another thing that would be useful to that. So if the storage Sig, uh like you mentioned that you're doing builds uh in the storage segment, if there is some dashboard, so some some web interface uh similar to this timeline.sam.com, which rather stop linked where you know it just lists, builds and you know what versions are built and whether they are green or red that type of stuff. uh So if there is, if there is a link to that that you could provide, that would be. That would be useful as well.

B

Okay, good, so I I will add more information for for this. Thank you.

A

Cool by the way, the the code changes, uh look, good, I, think uh I think it's. uh It's all a part of the coming description. It's ready to match.

B

Okay: okay, no.

C

Problem, we still need. We still need to go through the uh through the uh you know, basically procedure or to make sure that the uh Centos and rail packages still build over this change, but other than that, yes, I. Think it's ready.

D

D

Just push the branch to Shaman, it doesn't need to go through any of the sweets right.

C

I think it would be actually useful to just include it uh in any of the you know in one of the uh integration branches, uh on the main branch, uh and uh just you know just to make sure that those packages when, when they're produced you know, work exactly the same way as as before. This change.

D

D

Availability I'll make sure it gets in.

E

So I just want to point out one legal complication that there may be um because open Oiler is a product of Huawei I'm, not sure if we can put that in our Upstream builds um or accept Hardware. To do the builds we'll need to follow up with players.

B

Oh, oh yes, also uh here, uh I have some more background for four days of open reader open over there, yeah yeah, so uh the uh although we are open, we're Upstream yeah so was uh so Huawei is proposed to set up this uh Community before and uh yes three years ago, and they have gave them some more contributions to this community and and also offer some more developers. But so this uh operation system Community, has a has now uh not owned by Huawei. So it has also uh owned by the open Anthem Community.

B

So this is a neutral Community. uh Yes, sorry! uh This is a neutral uh open source Foundation uh in China, so that that one is uh uh not a Huawei owned uh projects now so for the open waiter. So it is a neutral operation system.

B

Yeah. But again so Huawei is a uh it's a contributor in this community yeah.

E

Okay, that sounds good.

A

A

Anyway, uh I see that uh Laura has joined after after uh another complicated meeting uh Laura. Do you want to take cover.

D

Thanks radic for leading.

A

D

So, let's see do we have any other topics. I think that was possibly the only topic on our agenda. Is there anything else anybody we wanted to uh that's not on the agenda.

A

Well, uh just one thing related to uh uh to uh to counters I I would love to ensure that we can uh in Reef that we can switch uh some components from using uh the old, unlober and labeled counters to new version. Two newer ones, I'm, not sure what was well. My main take care was that we wanted to provide some kind of uh of transition period uh has basically count the original counters uh be claimed, are claimed to be part of the of the of the interface.

A

Please correct me I wrong here.

D

C

D

Thanks just really quick, there actually is a PR out. That's that adds uh some label per counters. I can link it. I have to go ahead. Elia.

C

Yeah so I just had because I was uh I. You know ended up being very involved in this. uh For some reason, um the uh there isn't so a transition period uh is.

C

I I, don't think uh it's it's it's necessary, uh because uh reef is a major release and a transition period would basically be equal to uh like delaying any change uh in this. You know in this area. So if someone, if every component wants to take advantage of uh label Turf counter- and there is an existing unlabeled one since you really don't want to have both- uh then a transition period would just mean you know, delaying that change by a year. uh I, don't think that's reasonable.

C

A major release is exactly the the the place uh you know and the the time uh to uh to to do this sort of thing, because uh it uh it's going to be some time before people upgrade uh they're, definitely going to read, release notes, uh whereas you know that that might not happen for Point release, for example, uh and um uh so I I, don't think a transition period uh is is necessary.

C

uh It's also. uh This is kind of the other aspect of this is I. Guess the question that I raised before and I I I I, don't think a straight answer was kind of given or agreed upon is: do we do we even promise any sort of compatibility for perf calendars, or is it just a you know we're just not we're just trying not to break them uh fundamentally, both counters are they expose uh internals and any you know any significant refactoring uh can affect sample of counters.

C

uh It's um there's some things like you know the number of open connections um you know taking the messenger as an example, the number of open connections is probably always going to be there, because, no matter what you do internally, you know there's always going to be a connection and it can be either open or you know, closed, and in that case it just doesn't exist, but I'm pretty sure that if we, uh if we like, dump all the turf counters that let's say OSD exposes we're going to find some uh that are that expose uh some.

C

You know internal implementation, details of particular subsystem or of a particular class even and committing to keeping them stable, whether it's uh if this is com by the way, completely orthogonal to the switch from uh old style, unlabeled Earth counters, to label your calendars uh uh in cases where that's desired.

C

um It's just a question of. Can we even maintain.

D

C

Know can we even maintain the semantics or even the presence uh for a particular curve counter that just happens to uh to be exposing some particular class or even some particular functionality uh in a in a in a you know, in one of the classes, my opinion is that we just can't and.

C

There's just no, uh you know if, if you treat uh good counters as a public API uh and then the development is going to be pretty hindered um yeah. So that's that that's that's!

C

That's how I view it, and and from that perspective the switch from from old style and label for counters to label curve counters is just is just you know, it's just another kind of kind of change right, uh it's nothing new and uh since we are, uh since the proposal is to do it in a major release uh and with a release, note added I see no problem with that at all.

A

I hope this is uh I hope this is uh the entire the view of the entire project, because well, as you pointed out, uh making counters a part of our public operators interface, it's really bullderson and I. Think it's a recurring uh discussion. We are always hitting the topic whether there is any uh there is somebody who scripted uh and built on top of them, and you still I I, show the perception that still there is no definite answer. uh The question is: are per counters part of the public interface or not area.

A

How about bringing this topic to the to the broadest crowd? We can have I mean deaf Summit during this. How about using this this opportunities, this Summit, to make a definitive stance.

C

um I think that that's a good idea, uh I'm, not sure if that's the broadest, because you know I, think the the email lists uh would would still be. uh You know that that would be my definition of broadest um and I think this topic was.

C

What's brought up there before, maybe not you know, maybe maybe it wasn't um kind of as as explicit, and uh we may need to you know um to to do it again and make sure that it's circulated not just uh across the uh kind of the usual Dev and users, because we, you know some some operators who are scripting. Things may not be subscribed to that list.

C

There was no expectation of that, so uh users list would definitely need to be included, but also some of the um um we do have some also some uh monthly meetings, uh for example the scientific community meeting uh where a bunch of uh people who I suspect have quite a lot of scripts.

C

They they they do attend that meeting and it could be right up there as well, but as far as the the projects, dance, um I think uh as far as I understand it right now, it is uh you know it is just you know, try not to break it, because we're definitely remove perf counters in the past. uh I'm, not sure. If again, if, if that's, if that was made um kind of visible- and it was elevated to the kind of discussion that we're having now, but that happens in the past so.

A

C

A good idea, I, think I think just to start with discussing this. You know bringing this topic up at several time uh and then also following that on the on the on on the mailing list. That that we have uh would be uh would be uh would be. You know the the right path forward, but as far as the they are, that kind of I think uh you know uh that started this discussion, I guess uh I think uh again, it's it should be.

C

You know it's completely fine to do this in a major release, because, even if, even if uh for some you know, even if we consider, even if we considered as a project of counters, uh to be part of the public API, which we do change public API in major releases, uh we do that too. We reserve that right because otherwise again, there's no way to uh there's no way to to to develop things and there's no way to move forward uh eventually.

C

uh So, even even from that perspective, it's still fine to do it in major release. uh With the release now.

A

I see my intake is that the first uh my take is that it is that we would need to have to speak uh in single voice because the, in other words to form a consensus. uh Okay, cdms are public meetings, Etc, uh usually tech leads are governed there, but not not all of them. I think it cannot be perceived as a voice of entire project.

A

If I call on I think could be, we will have far higher component lead density than any other meeting uh over there. I believe.

A

But just to summarize I'm Andy bird I would really love to switch, for instance, messenger uh to labels, country to label counters. Otherwise, you've got a PR uh that brings uh two new labor like counters, while other others are still all about being unlabeled. It's it's mixing. It's it's birderson, we'll have to rectify that.

C

Yeah it definitely, uh it definitely doesn't make sense to split uh so that we are, it doesn't just bring two new curve counters it. Actually, you know there is uh there. Is it it it does this uh for for a particular class. uh So it's it's not.

C

You know just messenger at large right, it's a very particular class in the messenger that would suddenly report um uh some connection related metrics uh in one format and other connection related to metrics they're, very closely related uh they're related to the point that you know you could even try to derive uh one from the other. You know to some degree uh and they will be reported in another format.

C

That just makes zero sense, and that's exactly why I'm saying that, uh no matter whether it's considered to be part of the public, API or or not, we still can make this change in a major release and for the question of whether it's considered to be part of the public API discussing this Etc on, uh like like you, like you, said, May uh makes makes perfect sense just to finally, uh you know kind of put a stake in the ground and uh make it formal to.

A

Have the definite definite answer.

A

Cool I'm I'm, putting I'm putting uh this identity uh to the topic uh list for uh for the dev Summit thanks.

D

All right on that note is there anything else we'd like to discuss in today's CDM.

D

That's a great discussion.

D

All right, I think that's it thanks for everybody, uh for all of your uh the discussion on open Oiler and for kickstarting, the reef, perf counters, I, hope everybody has a great day and I'll see you guys next month, thanks Kevin.

C

Bye thanks. Everyone thank.

A

E