Project Harbor Community, 25 Mar 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Harbor Community Meeting - China/Europe Time Zone - March 25, 2020

Description

Harbor Community Meeting - China/Europe Time Zone - March 25, 2020

A

A

Hello, everybody good morning or good afternoon good evening.

B

B

C

B

Okay, let's start.

B

Can you see my screen.

A

Not yet, yes, you can.

B

Okay, uh hello, everyone and welcome and the host of today's meeting. This is the agenda. First, I will try the progress of harvard version.

B

2.0 then stevenson will introduce harbour operator for us after that appear from over over html or demo the operator.

B

The last is the time for the three uh this this current.

B

Okay, let me share the progress of harbor version to point zero. Last week, all features of version 2.0 in cloud oci, refactor integration with travis scanner or trs for the interlock combines and others are completed.

B

Rc and ga were being hyper.

C

Will we we sorry for interrupting?

C

Did you play your slides? I only see the first slide.

C

It's my problem.

B

Now can you say my slash yeah, oh okay,.

B

Today and yesterday we did the first testing activity for version 2.0 and some issues were created in the following weeks. We will focus on two fixes issues and add more more e2, e2 e test test tasks and clean up course.

B

This is all the progress of hub harbor version 2.0. Thank you.

B

Qualcomm's archons, even though to introduce the harvard operator.

C

Okay, thank you willy. Please stop the shirt. Let me share my screen.

A

C

Okay, can you guys see my slides.

B

C

Okay, so before pr uh do their demo, I'd like to use several minutes to quickly share the overview of our hub operator, so here is a simple diagram to describe what we will deliver our hover operator.

C

So far we have a uh operator, uh we call it a hardware carburetor this operator is donated by pr the ovh team, and uh this operator will focus on the you know manage the lifecycle of the only the hardware service components. It will not cover any. uh You know dependent service uh hardware relay on like the database, radius and the storage, so this is a pure operator for the pure harbor. Okay. So, as I think that's not enough, so, uh based on this operator, we are uh uh design and uh develop a hyper class operator.

C

This operator will, you know, provide a hard cluster cr. This cr will be covered, uh we'll cover the you know, all in one solution. That means, besides the harvest service components, we also, you know, manage the uh it's dependent service like foster securities cluster and possibly some in-cluster storage will use the miao to take that responsibility.

C

So the harbour cluster will be all in one solution, for you know: deliver a high available and stability harbor uh studies, okay for the uh for securities and the meal. That's we'll leverage some community operators. They are already there so we'll leverage those operators to deliver our uh operators. So under the last part we also cover the backup operator. So in future this is our last part after we complete you know really some hardcast or hardware operator. We are back to to see if we can deliver the back cover and recover scenario.

C

Okay, so far the harbour coverage and the harbor cluster bridge are developed in parallel. They are you know, maintaining different uh repo code repository and, in you know, in future, not too long term. In future, will you know will merge those two operators as a one operator. That means that we'll have a very flexible operator to cover. You know you can use only external dependent service or you can deliver all in one hover. Okay, so that's the overview of the uh operator.

C

So I think for this. This here is a diagram to show the relationship between the different operator so so far, based on current uh design. When you deliver all you, one um use a hard cluster to deploy all in one hardware service. There will be unit really on uh maybe five different operators. They have different controller to manage the different cr. I think the top one is the hardware cluster.

C

This cover classes here we own the harbor cr and uh also on the prosecutor, redis and uh ocr, and uh you know, for the harvard cr, based on the current uh design hardware cr will rely on the kubernetes, build building in resource tab like deployment security, configure config map or service to construct the overall hardware service so and the yin in filter the harbor operator. That means maintains the harvester will do some improvements for each service.

C

For you to service component of harbor. We will, you know, defend a new cr4 for that space component, so in future, harvester will own different service component crs. That will make the reconsider process. Very. You know highly efficient, and uh you know maybe more simple and more powerful. So that's.

A

So, and also steven in in some of the cases who might use crs built by other community members, for example, for postgres or for readys, there may be existing crs custom resources out there that manage their entire lifecycle right.

C

Yeah yeah, correct, okay, so so for the uh operator release plan, I think uh the hyper carburetor will release the first release. That's the zero dot file uh before end of april and for the 1.0 we so far. We still have no any concurrent plan. Once we finalize the plan we'll share with the community okay and for the harbour cluster operator, we have three community over uh contributors we have from uh um from netease from qing cloud.

C

These three guides will work on the hyperclass operator of course-based hardware operator and other community uh database already operator to deliver the class operator so far. uh This this work is only you know in the desired phase. Not any coders uh is is introduced, so no uh any conquerator is a release plan same with the. uh If, once we have a confirmatory display we'll share with the uh with the community, so that's uh overall view of the hub operator uh visa and uh the map.

C

Okay, now back to, maybe I think maybe we can give it appear to do the more detailed introduction and demo.

D

Okay, let's go. Let me share my screen, hello.

A

A

Hello yeah appear here. We can't see yourself.

D

Yeah! Okay! um Sorry about that, I'm not on.

C

C

Hello hi, I guess pierre is gonna log in on our on his desktop as well.

B

Hey steven, can you share the style about the operator.

B

So can I get hello.

D

Can you hear me.

B

D

Yes, yeah, okay, good, sorry about that, I'm not in the office, I'm in quarantine here or not easy!

D

Okay! My brother is preventing access to share my screen. Okay. Why? That.

C

So you, you have problem to share your.

D

Yeah exactly I can't allow firefox to share my screen.

D

It's a systemd here due to quarantine. Sorry.

A

Hey stevens out, do you have the slides up here wanted to show? If you have the slides, then just share them and then pierre can walk through them.

D

I spent slide in a fact.

C

D

But I wanted to show a few comments in my terminal so trying hard here to share my screen again.

A

Pierre, you may want to download the zoom client either for your mac, linux or windows. Instead of I believe you may be logging through the web browser which might be harder to share.

D

Yeah, but uh if I'm not wrong, it requires roots to install the zoom client.

C

Yeah, I have helped to share your slides.

D

C

D

um So here I'm here to present you the operator, I'm pierre from ovh, which is a cloud provider so in the future story behind this operator is why why we needed an operator and we, I will show you how we built it and how it works. So, stephen, let's go thank you.

D

As a cloud provider we have needs. The first one is to manage hundreds of our stacks.

D

We see that you guys worked on the m chart to deploy arbor, but on our side it's not enough, because we cannot manage hundreds of hours through m charts. It's a really pain in the ass.

D

So we need to have management in demand that mean with api and sadly, m chart is not an api and the last, but not least, is to have a reaction of a external or internal event. Imagine adding a dns imaging, adding iplb or whatever external stuff can can trigger in our configuration.

D

The operator can react and interact directly with the right component component. I mean artwork or notary or whatever that's that that are our needs, so yeah, thank you. 7. Can you go to the next slide.

D

So how it works, how does it works? We have a custom resource definition.

D

Maybe- and maybe all of you know what what it is, and in that customer resource definition, we have a few specifications like public, url database and storage configuration.

D

That's the minimal requirement for an arbor to work, then, with that resource, which is only a body in a api call, we have events that are triggered by the kubernetes system.

D

We can add an arbor resource in the kubernetes api. We can update it and delete it. What does the operator.

D

So in the first case, add an arbor is simply react to the event, receive the event and construct the kubernetes resources to deploy the stack so construct internally in memory I mean and then apply to the kubernetes.

D

Api, so what does it mean? It means that we only use the public arbor image. We we use our core from docker hub and run it in the kubernetes. So the operator is pretty simple. In fact,.

D

Then the other simple use case is to update the arbor resource, which is simply change the go object in memory and apply it to the kubernetes. By the way, it also delete unwanted resources. Imagine a specification with chat museum which are obeyed to delete the chat museum. The operator take care of this cave and simply delete the pod where.

C

D

Museum is ready.

D

The last case is to delete another resource. In that case, the operator simply does nothing, because all is done by the kubernetes garbage collection system and all resources are linked are owned by the arbor resource.

D

Stephen can can you go to the next slide please.

D

So I wanted to show a few comments in my terminal but, like I said, I'm not in at the office, so I I was afraid about the the live bag, and I I set up a few screenshots here um to deploy the operator. It's simply one single, but you can see it in red. The hardware core operator, controller manager, xxx, and this this is the main and only pod, which contains the operator.

D

All other parts are dependencies. Like steven said, the operator have many external requirements about databases about certificate management, about radius about whatever. So here to install the operator, we need a third measure and the application details.

D

um Okay, listen.

D

So here we have, we installed the operator and now we install arbor requirements. So we have management about certificate, and now we have management about databases and redis.

D

um If I'm not trying that, I know we also have the router. Maybe you know the ingress controller. Inquest controller is here to to root traffic to the right harbor, depending on the request. We wrote a request thanks to the public domain, I'm I'm in the the tldr not sure how to say that english, but.

B

D

Thanks to the domain, um I will show you where we can find it and where we can change uh update it.

D

So here is the arbor requirement and then stephen, I will show you the last, but probably not least, the sample, however, which is deployed and thanks to zimming, it's easy to deploy the.

B

D

Sample from the repositories, so how about sample? What? What and no yes thank you our sample is simply a custom resource.

D

Stephen. Can you go back to the harbour some purpose? Yeah, we have in the last line. Arbor global dot. Io harbor sample is a harbor resource with a public url. You can try it out and go to the arbor that oh no.

D

I forgot how about that registry that paper and ovh, but yeah, it's a so we have the public url and it owns all pod. You can see over there, um so this harbor under the chat, museum, the clear core and job service notary features and all are managed by the operator. You just saw.

D

D

I wanted to do some shook event, but okay, uh thank you steven, so yeah yeah. Please keep this slide.

D

How to access to arbor I mean that's nice. Please go back to the to the password slide, the previous one stephen. Can you go back to the previous slide? Thank you.

D

So how to access to harbor I mean we we have deployed. We deployed an arbor stack, yes, that's nice, but how to serve this password to customer how to expose that password how to handle the permission and error or back to access to this password.

D

All this permission, management are handled by the kubernetes api and this is exposed to a specific resource which is secret. So if a user can access to this secret in the right namespace in the right, kubernetes cluster, it can display the password and go to the to the arbor ui and connect thanks to the admin user.

D

So this is simply automation about management of the arbor and exposing features and all of those old stuff.

D

The link between this password and the arbor resource is in the next slide that even we'll press the right key. Thank you.

D

I talked about specifications about arbor resource, so customers, customers, clients or people or users or whatever will set up their own admin password secret, which is the same name that this secret. The previous cut, as I shall show you. um So that means, if the easier.

D

I don't know we are thanks to this link. We have we handle many cases where people can generate password and other can only read only so. We have read, write access different for four people.

D

We have this: okay, not sure how to explain that, but thanks to the this kind of links between arbor resource and secret resource, we handle many permission case use cases.

D

We have the same kind of architecture with database secret. um So in the middle of the slide we have core database secret and job service ready, secret and notary database secret. You. We have database this secret, pretty much everywhere, because every component requires the database.

D

But thanks to that kind of list, with a kind of link, we can manage pull up databases in secret resources and link to the arbor resource and that's pretty nice to automate automate dive. All of the stuff you can see in the end of this specification, the public url, which is sample registry, that apparent that ovh, the simple demo that I wanted to show you is to update this public url and you will see the this change live.

D

But just please trust me: it works and let's go ahead uh about the title of this side. We have arbor stakes, specification and arbor config.

D

As you can see, we have no configuration about about itself in this specification, except probably the public access it could be. Cural.

D

Yeah we we wanted to split that kind of.

D

Job configuration I mean the real core, the real signification about the hardware configuration and the technical infrastructure and architecture about that with databases, the access to this and whatever. So we have a difference between our boss specification and airborne configuration.

D

Okay, stephen, can you go to the next slide? Please.

D

So in this slide we have two things: the first one is at this moment. We have configuration in specification, it's an issue in the first development, but we can improve it in the next to the red, the blue, the blue, uh the blue dash. We have the narrativity sources and worker, which are probably configuration.

D

This kind of information are in the.

B

D

Configuration itself and should be in an external resource which is configuration, but the way we will under it stephen explained it uh uh earlier, is to have splitted specification with pleated um resources. We will have the portal resource the job service resource at kdr, and that means we.

D

We will have some kind of merge about configuration specification where, in fact, portal specification is like portal configuration and job service specification service configuration because it it will be simpler for everything and so yeah. That is.

B

D

Release, 0.5 is simply if you agreed about the installation to be easier and documentation and the release 1.0 is about splitting the resources.

D

One other point I would like to to watch statues is: I mean we in this slide. I only show you the specification that kubernetes are as also statuses in every resource, and I would like to I don't know, set up the job service only if the redis is really is really ready and to don't know if it is ready uh just it should be based on the kubernetes status.

D

So that's what I mean about statuses.

D

So here we go at the end. If you have any questions, don't hesitate.

A

uh Thank you. uh Thank you so much pierre. So I also wanted to mention that uh uh the with a lot of the work that the ovh cloud team did on harbor uh we're very fortunate for them to be uh huge contributors to a community and and donate the operator to hardware. So we've released a blog post, announcing that and uh very important now.

D

And we are happy.

A

To be part of that, so yeah, absolutely and then jeremy and and pierre are also now maintainers of harbor, and they will continue contributing to our community and enhancing the operator, like both pierre and steven zell mentioned earlier, and we will. uh We welcome other future contributions and we look forward to seeing the operator uh be the method for life cycle management of hardware moving forward for everyone.

A

B

Thank you here. Everyone has has any questions.

E

Hi, this is daniel from aqua, just it was somehow explained, but just to make sure that I understood correctly uh so far. I see you support claire as a vulnerability scanner.

E

I'm wondering how difficult or how easy it would be to extend the operator to include other pluggable scanners.

E

Is it something that should be contributed by the vendors or are you going to implement the you know crds for most popular scanners? um If you could comment on that, that would be great.

D

Yeah, um there is true answer on your question. We have at this moment it's uh we have to copy past many files about claire and rename it to 3v.

D

That's not hard, but pretty painful. In the next release, the 1.0, you will have specific resource and it will be easier for anyone to implement their own scanner and their own statuses.

D

I don't know if the scanner is ready, okay, but is it up to date? Is it.

D

Overloaded, I don't know there is many statuses possible and.

B

D

The next release it will be easier to for everyone to handle.

C

I'd like to see more words about uh danielle's question, so uh for my understanding for our operator, we only cover the uh default scanner so for other scanner. We will not current in operator, so that will mean after two uh dot zero. We will both cover both claire and the tray, but for other platform scanners. They need to you know to install with their way- and you know, configure through the hardware. Dashboard not operator operator will keep consistency with other. You know installation approach we provided so far, so we only cover the default scanner.

C

Yeah got it make sense. Thank.

C

B

B

D

We have a question in chat.

B

D

Yeah, can we can we scale specific services uh yeah? I'm not. uh We provide this replication count on every component, so you can deploy two registries and to our worker independently, but we do not handle every use case. For example, if you deploy two hardware core you will, you may have issue to share session and with two registries you may have issue to to share secrets the http secret- whatever I don't remember, but yeah you can scale it, but be I mean caution about the shared secrets between your application replicated component.

D

You see what I mean.

D

B

Everyone has other questions to ask.

B

How about the supporting commercial nitrates.

D

So we are a huge fan about the formative matrix and the operator itself already exposed few metrics. But honestly it's it's very poor and we have metrics about other components. For example, the registry exposed matrix this matrix is not handled by the harbor operator, but it is natively.

D

Natively handled by the parameters operator- maybe you know that operator that parameters have its own operator and under the scrap of every slash matrix on every component.

A

All right folks we're out of time as well, so um we should probably end this meeting and if there's any additional questions or concerns, uh please post them on slack on the hardware channel or on the mailing list, and uh we can try to answer them.

B

Okay, thank you. Thank you. Everyone.

A

Thank you all have a great day or evening.

B

Thank you thank.

A

D

D