Project Harbor Community, 8 Feb 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: CNCF Harbor's Community Zoom Meeting - Feb 8, 2023

Description

CNCF Harbor's Community Zoom Meeting

A

Okay, we're recording hello. Everyone today is February 8th and, as the official community meeting for Harbor, my name is Julian vasilev and I'm. The community manager for Harbor I also try to do some devrel kind of stuff recover. uh For those of you who don't know me um in the slack I'm, sorry in the zoom chat, uh I'm gonna paste once again the link to our community meetings, notes and I'm, also gonna open them and share them.

A

So we can all see uh what we're gonna talk about. Alrighty.

A

All right, I hope EO can see that single window shirt um and let's see what kind of topics we have for today, yeah. So uh a few updates from my site. uh First before we begin, um our community maintainers track for kipcon was accepted for kubecon. So congratulations.

A

There will be the two maintenance who's going to represent uh our little project at kubecon to the maintenance track. Also we have a kiosk boot for the like the the afternoons for the three days, so whoever is going to join kubecon is more than welcome to take some some time to spare. On the boot talk to people. uh We also requested to have a community uh sorry project meeting at kubecon and what else.

B

A

I think that's it right.

A

um Unfortunately, all the others Harbor rented talks were rejected, um but that's that kubecon is facing again a record. A record um number of cfps submitted so I think it's just super hard uh for everyone to to judge. I can say that because I was part of the the team who uh made the discussions on the community and track 101 and we had like double then the last time in Europe, so it's just insane um yeah. So for this one I'm gonna open the floor for everyone else.

A

uh Oh and another thing I'm sorry, um we have been accepted to the Linux Foundation mentoring, mentee kind of program, uh thanks again to yanwan and Vadim for jumping in to this one.

B

A

By the way, that's open for everyone else who wants to do the Montana Mentor thing, um so we were accepted with four projects. I'm gonna show that uh in a second let me let me see if I can find that link super quick.

A

um So we have. We were applied with four uh with four four projects and they were all accepted or project ideas.

A

You can see them over here, so we practically translated some some of the most common issues that we can see opened by the community, uh some more complex regex for the application rows, uh an effort to kick off a golang, AP, client, uh API client and to the next one, was to implement um uh an overview of the CV per project or the whole power instance and the last one was uh the robot accounts implementation. So they can have a full um full access to the API.

A

So um there was an issue by adding us to the Linux Foundation Mentor mentee program, and we have an extra week because they screwed up something to be added in the list. So if you want to be part of the mentor list, uh we have listed right now, Vadim and Yan, and for some I'm listed as well. uh But if you want to do the mentoring stuff, you can open the link over here.

A

I'm gonna place that in the in the community meetings- and you can see how you can sign up so with that some stuff popped up last week when we were discussing some stuff and, for example, this proposal to be able to implement something that we spoke with Tom uh once or twice a global disable or enabled public repositories per the whole instance.

A

So if you take, if you can take a look at this one and add your thoughts on this, one will be great, so we can have more. We can prioritize it um and.

B

We can start working.

A

Unfortunately, that brings me to another topic which is Roger is not the PM anymore from our site for Harbor. I cannot share more news about this one or reasons, but I'll keep you posted. uh What's gonna, what's gonna happen in the future with the row of PM, so that said, anyone else I'm speaking too much today, so I'm going to shut up and leave the floor for everyone else to talk about some issues or uh something that we want to have discussed. I can see Tom, uh you've added something they want to take it away.

C

Yeah I can quickly uh I have to go in about 20 minutes, but I can quickly uh run through this.

C

So one thing we've noticed is that basically, because this is kind of a uh schedule- basically a Cron job, for running uh vulnerability, scans right, so we use Travian, R setup, uh which I would imagine most people do, because it's quite easy um and we've got it, set up to do daily scanning and, as we see the amount of artifacts in the registry increasing over time, we see kind of the the queue for scanning getting longer and longer and longer, um and it also has quite an effect on the performance of the database.

C

um So uh you know, CPU usage on the database generally throughout the day is kind of like nominal right like quite low, uh and then it comes to vulnerability scanning time and suddenly it's basically sitting at 100 for uh hours and as more artifacts get added yeah. That's only going to get worse and worse and worse and I can imagine it kind of spreading into a workouts. At some point um and I wrote up this proposal because it kind of seems to me that it could potentially be done in a different way.

C

The the vulnerability is getting like the scheduled kind of thing. So at the moment, it's basically a crown job that says: okay at this time go and uh just scan everything uh which obviously has some issues as stuff gets bigger potentially, and this is kind of what I'm proposing potentially, instead of doing it. That way, it could be like a continuous job. That's looking at okay!

C

When was this image last scans, you can kind of set goals like uh you know, we want to uh have images scanned every 24, 48 whatever hours and then there's a basically a constant running process that looks at when was this image last scanned? Should I scan it again now, yes or no to kind of so there's a kind of constant background hum of scanning, rather than kind of just this all-in-one thing. Obviously it has some implications because then you've got more constant load over time.

C

You can probably get quite still spikes and then it's a bit harder to predict when the spikes will be, at least with the current setup. You know uh okay. Well, we know that there's always going to be a spike at this time. uh When trivia starts its uh thing um but yeah, that's basically the proposal, I'm, not sure I, don't know if it's a good idea. This is kind of just a kind of what popped into my head.

C

When we were looking at this thing, uh because at the moment we're looking to have to to increase our database instant size purely to account for this uh Spike uh that we notice uh the rest of the day, we don't really have a huge amount of uh consistent usage, um yeah.

D

That did you look at the optimization side of the database. So what are the? What are the long run, running queries that might be optimized? You know, maybe you know some indexes some.

C

Yeah, unfortunately, we we didn't uh look too uh closely at the uh using RDS and the default um RDS logging is not great, so we've realized that, after the fact, so we're going to be increasing the instance uh type, no matter what so, when we do that, we're going to enable the kind of slow log, uh slow query, logging and all that kind of stuff, um so that we can kind of get better insight into it.

C

But at the moment we're using a T-Type instance, which is like a meant for burst, but we're kind of getting beyond the burst stage. Now, with this.

A

Everybody related question to this one: do we have some com, some kind of guide for optimizing the database for this stuff.

D

It depends on on the workloads right, so it's more depending on the what type of workloads they are um and operations they are so that you know you need to. Currently you need to opt. You need to see what is the database load and to optimize for different scenarios, but there is no to my notice, there's no documentation on that.

A

Okay and same question goes to one young and Jolene: do we have something from VMware I? Suppose we've done such things for for for customers.

E

So you mean uh database, uh optimization or or yeah.

A

Something like a wallpaper or best practices for optimizing database for Harbor or I. Don't know how to name it, but something.

E

Yeah yeah, actually we did some code change and and produced by release to to resolve some bottleneck that reported by the community users and our customers. um I can share some uh experience from one of our uh internally Enterprise user. um They set up the ha post postgres.

E

The the HRA cluster works well on at their site and and we did some tuning on their, uh they have a DBA database that I mean to help them to uh uh figure the bottleneck of their uh uh the database instance to uh as some memory or or or enlarge the database connection size account and yeah.

E

Since thank you, but.

F

E

Is by case because.

F

E

A

We should because that's not the first time when we have someone saying hey our database is super struggling with the with the loads. What we can do so maybe we can think about it, like stream, aligning it into like to take experience from VMware, continuous Solutions and now NS and and combine it into one document and you'd be like yeah. We don't guarantee the best performance out there, but if you do these simple steps that will give you a bit more right. Yeah.

C

Yeah yeah I've been kind of um encountering this because now we've been using it for a while, so we're encountering kind of more. uh You know day two uh operations kind of stuff, um and it's it's more like the devil, is in the details right so, for example, uh you know we're using S3 for um artifact storage and what we're noticing we've got. Versioning enabled we're noticing kind of overtime.

C

uh You know the amount of storages uh kind of climbing quite intensely um so like what would be really nice is if we had a kind of community kind of contributed to page with, um for example, just implementation guidelines like use at your own risk, but this is kind of from our experience. We've noticed this kind of stuff, like S3, enable a lifecycle, roll to expire, non-current versions after X amount of days and blah blah blah. That kind of thing uh I think it might be handy.

E

Yeah, like a handbook for for have a user right, yeah yep, yes,.

A

B

E

A

By the way, I still don't have the book, but uh I'm still writing my copy by the way, uh but.

B

uh I'm getting aside.

A

uh Is there anything in the book about this stuff.

E

uh Give me the database or or we have some uh FAQ released, some hard topics that ask from the community and we give some answers on that. um Yeah.

A

Okay, great yeah by.

G

The way Tom, uh what is the current version that you you gotta use? I mean Hubble version.

C

I think we're pretty up-to-date I think. uh Let me just grab my card I think around two seven at the moment or something that's.

G

That's really the latest one yeah.

G

Or team member have done some uh performance Improvement uh either in the uh Circle I mean tuning the circle or doing some color change to improve the performance. So just so curious. What is the version you are using? So if you use still see there are some performance or issue, maybe you can open a ticket and put some details on that. So we can do some investigation to see the code change. You needed to support that.

C

Yeah, my gut feeling is it's just the amount of uh um the amount of scanning that's happening, um but we're gonna. What I'll do is once we've uh enabled um the uh slow log uh slow query. Logging then uh we'll have some more data. um Unfortunately, we need to basically reboot the instances to do that, so we're gonna we're trying to delay that a bit until we actually upgrade the instances, but um once I've got that I'll create an issue for it.

C

Okay, yeah yeah, we're on version uh we're using Helm chart. So one ten two at the moment so I think that's pretty recent.

D

C

D

C

D

Okay, sorry I'll talk to you yeah.

G

E

So I appreciate that you, you can uh okay happy issue and provide some uh fundings or details, database, usage or or something else. You know CPU memory yeah. So then.

B

E

We Can Work Work video to to provide some tuning experiments, yeah.

C

Sounds good thanks.

A

um By the way, I mentioned a book and if you don't know I'm not on mute right, uh we have that book out from from young one and and I'm sorry. But I cannot read that name properly, so I'm not going even to try uh so congratulations of the book. Yet.

A

um Is that the only place where folks can buy this book by the way.

E

B

E

I, don't know sorry, I, don't know. Okay,.

A

All right, I'm gonna link it in the community meetings anyway and yeah.

A

Let's put this, uh so you can, if you're interested, you can get a copy. Okay.

A

um What's next.

A

um This one wants to take this one: do we have the.

A

I'm not sure who opens this. One I cannot recognize this. This handle.

D

I think he was here before no I think no okay, but maybe we can open it. Yeah I.

A

I did that okay.

D

All right, let's discuss.

A

D

There uh quota checking yeah, so thank you so much a minor Improvement here um I checked the code.

D

E

I have a review of this VR already, but uh uh since the the change will impact the quarter and the push scenario, so um I like to take more time to do some more deep, deep dive or investigation on this far since.

D

E

The the change is not big, but the the impact that the push scenario. This is the one or the most important scenario of Harbor, so uh um yeah uh I have already uh talked with the uh another. A maintainer Who uh uh wrote the code on the code, so um yeah I will follow up this. This.

A

E

A

But if you, if you can provide some feedback in.

F

Into this okay, I'm.

B

E

D

And also does someone know what the changes I mean. It's like optimized quota but and I didn't understand the code correctly. To be honest,.

E

Yeah yeah the change, the current logic to uh um intercept the the approach request for uh manifest and Broad. This is the uh uh quite important part of Harbor, so we should be sensitive for this PR um I I will not approve that before I. I totally understand that the change and the.

B

E

All the scenario that I I can think about that yeah. So, um yes, it will take time.

D

Yeah, okay, I understand yeah, because I didn't understand it correctly. What it! uh What is the change, because the only changes is here at the function here? No, that's the test function, that's the no! It's! This initiate a lot of thought.

D

Well, this is the test function, so it doesn't um it's this one here in YouTube, go because it doesn't really remove the checks check against current used resource and.

D

And then it does a you know: resource List It returns a resource list. Instead of no.

D

So yeah I I, don't know.

A

Okay, so I'm not sure if it makes sense to discuss it now or do we want to take a look at it and.

E

A

Okay and I wanted to open another one about the test environment. Give me a second to find it.

A

These guys that are trying to to use this to um everything. Can you can you record this one we're taking a look at this one together once uh these guys that put it their uh private key to the certificate in the epr.

D

uh This was the um test environment.

A

Pr, environment, yeah yeah. But uh what was the name of stuff? um Something like you and.

D

Updated, no, not update description, just sort open, we are open and then.

A

uh To sort them, how used.

D

A

D

Yeah and it is scroll down.

D

A

D

A

You PC whatever, yes,.

D

Except this one.

E

A

So I wanted to bring that up because it sounds pretty cool.

B

A

And if you can take a look uh practically, they want to as part of the CI they want to integrate Harbor into so it's part of our CI to spin up a cover in their environment. So we can do the tests on live machines and then, when the pr is closed, they're going to destroy the instance, so this stuff, like the uh the quarter testing, we can have them more easily up and for user kind of testing. So if you can take a look at this, one sounds cool, so yeah I'm not voting problems.

F

A

One it's just it's just interesting PR, uh which is not related to the project, but from user and contributor perspective, it's going to be useful right now,.

E

um I I saw him. I saw hey, um uh created, several pull requests for different open source projects.

E

So I I mean that the we can monitor their action to see whether they decide to accept that uh uh um yeah, because that it seems that they wanted to automatizing their product right. Yeah.

A

E

Yeah within the CNC community, so so we can uh see other teams.

A

Do you have examples or on other projects that have dotted that thing.

E

E

A

Mean if you have them, we can take a look all together and okay.

B

F

A

Good or not for our project.

E

A

Mean um Ryan uh I'm, sorry man, some are you gonna dance or what you're gonna do.

E

I was uh send your link.

A

Okay and I'm gonna add it into the community meeting all right um any other topics, if not I I, want to give the time we have two new folks on on the call to present.

D

This uh no regarding the issues- and there is one issue not discussed um the stable uh kind of the first one. It's it's yours, like oh yeah, I, think you missed it right. Yeah.

B

A

A

I think we I, maybe I I- was thinking about mentioning it. Thank you very much anyway. um There's a proposal to team Samantha uh Global enabled disable Anonymous success. um So if you can take a look at this one.

D

A

We can take some some yeah go ahead with him.

D

I think I would like to add to this proposal something, and it is related to you know, a few vulnerability reports that we received recently with these false positives that we got and and I think this will address this issue so because, when you have the possibility to make projects like, if you go to the description and in The Proposal, um because the proposal just says like Anonymous access, disabling it but I think if we can make it like a bit like a git gitlab scroll down scroll down to tianon mentioned this and I think it's really good idea.

D

So if we would have something like this in Harbor right, uh like restricted visibility, levels like private internal public, and this would allow, for example, the internal users to access the the images or public or private. So this would make you know, obviously, projects uh projects and also the the whole registry, private, internal or public and I think this would be a great Improvement in um in Harbor. If this must be, this would would be existed.

A

Yeah and Tom left, but uh we were having a chat with him and he said that his first job when he arrives at work, is to open.

F

A

And to see how many public repositories there are created and to go, and after the folks who created them to close them, because that can bring some some issues and if we, as far as you understand that bank, uh if we implement this thing, that issue go away right.

D

Yeah, because mostly people put it private or internal.

A

Yeah that to allow to encapsulate the the whole instance, let's put it this way right.

E

Yeah, um but the per tianist or Commerce, it's it's not related with the this proposal. I I believe so the the title of the sphere is enable disable Anonymous access provide uh three states opinion for both.

F

E

But they are seems not not on the same right. uh It.

D

Goes to the same direction, but it's not the proposal, that's correct, so maybe we either should close the proposal and create a new one. Oh.

E

You mean you mean the for for the tienes commas. We want to introduce one more state for for project like in.

D

Internal right, yeah, I I, would Pro I would yeah I would actually favor the proposal from tianon and then I either changed it 124 or close it and create a new one, so create a new uh one and close 124 or accept 124 under the condition it is implemented uh like tianan suggested. So it depends how we How We Do It.

E

Yeah I prefer we can close the the one two four and then we can reach another, and you can you can talk to Tiana Tiana and actually it was a a maintenance of Harvard.

A

E

D

He I can I can create one new one if you yeah.

E

A

We all agree that that such functionality is needed. uh We can close this one and all the interested parties in this one to implement it. We can like set up an hour, oh just to figure out which direction we want to go and to craft out the the new proposal. What do you think?

A

Is it waste of time or.

D

A

D

Think it's clear from the requirement you know: I think they yeah so I would just create a new proposal and there.

H

D

Much to discuss I would say.

A

Okay, so uh Vadim, are you gonna, take this one or yeah.

B

A

Can create the one okay cool.

E

It sounds like a commercial of requests, so we can uh sun up to the use a group to let them know on whether they wanted this feature or not. To give us some give us some feedback From perspective of their uh Enterprise use, so uh so.

F

E

We have the your proposal, so we can send a link to the cnaf hardware user, Channel, all right so to hear their thought.

F

E

Part right: okay,.

A

Good okay, uh but then, if you think, I can help out with with this one just drop my line: okay, okay, yeah, like wording it out or whatever you think I can I can be in Hope.

B

F

A

um Okay, so uh anything else that I.

E

I just paste a link, so you can open that um at.

A

The base yeah the same close this one.

E

Oh yeah, that's.

A

Why he's the same person right so.

E

Yeah same PR, same person, okay,.

A

I can I'll track down all the others that he or his company open to other projects, because I think it's relatively the same, naming and we'll add them to to our wiki page. So we can take a look and decide how we can go with this one.

E

Okay, yeah all right, but but yeah. Okay. Thank you.

A

Because that's no! That's not ours! That's that's! Theirs! Again! But yeah yeah! It's pretty much the same wording so yeah you.

B

A

Super complicated to find all this okay, yeah yeah, because it was like a red flag for me at least that they've committed their uh private keys to the to the pr.

B

A

um Okay, so uh I'm gonna link also that thing to the um to our community meetings and until when I'm doing this one I'm. Sorry, if I'm mispronouncing your name but first can you can you introduce yourself please.

H

uh Definitely so hi everyone I'm part agrawal I'm pressing, my undergrad at idbhu from India and I've worked on open source project like vectil and other communities like bar data and CNC and Peters I'm, currently, an Outreach in Mentor at Bechtel and I'm, looking forward to apply for reject application project at Harbor.

A

Nice um did you, did you apply through the uh the um the mentoring program or.

H

Yeah I've applied I've.

A

H

And submitted my cover letter and drafted me.

A

Yeah: okay, because yeah as I said in the very beginning, um there'll be some delays in the process of approving all the applications and stuff. So welcome and I think you also um uh submitted some some PR study to the website right.

B

A

Some some fixings some words good first step. Thank you and welcome to the club. You.

F

Can see any other.

A

Folks over here, so practically that's the the core team that works on on Harbor. uh Of course, there's some some missing, but.

B

A

Feel free to reach out to us anytime. You need some some help.

A

I cannot promise we'll respond right away, but yeah we'll be around so yep welcome and yeah Tom left, but I can I can I'm not sure if everyone else has seen Tom but Tom is from the uh the Netherlands Railway system and they have adopted Harbor.

A

uh So we had a chat a week or two ago about their implementation and that they want to be more active and more contributing to the project as they rely by a lot in Harbor so that they feel responsible to commit back um to the community to contribute back to the community.

A

So yeah I'm not trying to take his words out about yeah. That's that is pretty much what we chatted about uh yeah. So with that I. Don't have anything else for today: um I'm gonna work on the kubecon organization, so we can have some swag, some yeah we can do some breakfast or even dinner depends on the amount of people that we can get there. And what do you think that I can get.

G

Planning we have a new Milestone named Peach of Reed right at that time. The Dell me stone. There is uh February 6th, so I think we have already reached that page so say. Payload for to the latest, release will be uh I mean to be finalized. uh We were not introduce any new functionality, it's uh except that is a big issues or other uh important functionality impact the ticket. So that's something I want to share in this meeting. So.

A

So can you add that to the Q to the community notes, please um no.

G

Problem uh I will add that.

A

So you're saying we reached to the point.

G

Yeah, if you, if you scroll down to the yeah.

A

G

Yeah here right.

B

A

Okay, so that means we want at new candidates and targets for two eight from.

B

Right yesterday, the day before, okay yeah.

A

So whatever new stuff comes in uh to be targeted for two nine or right.

G

uh Two: nine uh yeah to right right: yeah, yeah, that.

A

Is the intention yeah, okay, cool this.

E

Should be two night candidate right? Oh.

A

F

I was just trying.

A

To say we're gonna Target them to get into two nines that they will become two nine candidates: the first yes, okay, yeah, sorry, okay, all.

H

A

All right, okay, anything else, um folks, anything that needs public attention and we want to discuss it.

D

Not from my side.

A

Okay with that, thank you very much. It was in my uh eyes it was very protective meeting, so welcome to the new folks on board and talk to you in two weeks right. Thank you bye-bye. Thank you bye. Thank you.

A

Thank you. Bye-Bye.