Project Harbor Community, 8 Sep 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Harbor Community Meeting - Sept 8, 2021

Description

CNCF Harbor's Community Zoom Meeting

A

There we go hello, everyone and welcome to the harbor community meeting. Today we got two demos that we want to show off. The first one is by kian deng. um Do you want to start off.

B

Okay, thank you.

B

Let me share my screen.

B

Oh, can you see my screen now yep, okay, uh uh uh today, uh hello, everyone today, I will show you the diamo of distributed tracing and therefore more background. You can check the proposal of distributed tracing in github and you can also watch some recordings of our paris mill apps. So let's get started today's demo. uh First of all, uh we added some configuration item in the configure. File of hardware uh is related to tracing and recoil trace and uncommand uncommand.

B

These configurations and uh setters are enabled to chew and the sample rate the sample rate means uh uh one means. uh A hundred percent of the tracing data were collected by the vacuum and uh if you only wanna sampling half of the status, you can set it to 0.5 and set it to zero.

B

It means never tracing and the namespace.

B

You can add the namespace. If you want- and this is a if you set the namespace- you can using the namespace and the service name to differentiate different applications and the attributes is the uh the attribute you want to add to the tracing data and, for example, if you're using some third-party tracing stories like wavefronts.

B

You know I want you to set up this item, so, if you're using this kind of service, you can edit to you can add the attributes here and the exporter. uh Sorry, the the tracing. So it's back in the way, uh whereas with the two of them one is geiger and the other one is the hotel and the jeiger is. uh The popular I think, is a very popular and a great uh uh tracing backhand and is also since they have projects, and we can.

B

um We can export data to geiger directly and we can also uh in uh explode our tracing data using our helpful tackle and using this protocol, and you can never see the other uh open telemetry component. We call uh called the ultimate collect and it will consume this kind of data and exports. The data to, I think, almost all the um tracing surveys. Okay, let's get started we first we we're dyno, uh explore the data to geiger and, uh like the configuration here, we set a dagger in the point and.

B

That's that our stories.

B

B

B

As you can see, the hubble service is uh getting started, it gets started and uh because the remote enabler traces. So at the same time the tracing data is sent to the gyre. Let's say the geiger ui, okay.

B

As you can see, and we added the uh that, because we added the http or we traced all the http requests and and uh in hubble we have a health check uh in all components and, as you can see from this trace data, you can see we're using hardware. The hubble uh use http get method to request the uh register.

B

uh Resist control and with the api has.

B

So, oh yeah, I think, oh in this uh next release we just uh enable three component is uh two of the tracing and the one is harbor core. The other ones are uh how about job service and the last one is hover, resist control and, as you can see, both of them have the tracing data. Here.

B

B

And then, let's do some more communicative job uh like uh replication.

B

I added this uh replication job here is uh uh for the image data from localhop to our hubble and the right.

B

As you can see on all the because the application will be styled from the job service and they started some back-end tasks and to replicate jobs. You can see it's a push, which is our data, all the layers, data to our core and from the.

B

From the geiger data dashboard, you can see every request of the hubble surveys and, for example, there are, if there are some arrows, you can also find uh here.

B

This this is because the page power, the head requests, are retained for 404, so you can say arrow here and it's very convenient for people. Debugging distribute your service, and that has I have another hover running and which, using.

B

So open autofocus like this and as you can see, we can also export using this protocol and from these series on this server, you can see the open source or obtaining hotel protector using the hotel collector we can consume the hotel protocol and explode. Also exporters means the exposed service. We can also export using yeager.

B

We can also using other services which supports hotel, like cans, observability and, as you can see, oh, I enable it here and we can send the data to uh 10 uh away from which is kind of observability proxy, and it's also running on this server.

B

B

As you can see here, we can, this is the observability we can collect the it also can consume the data sent from the hubble.

B

Yeah it from the request you can draw the service graph here, because we just enable three components. So there are three ones: three components here.

B

Okay, I think this all my today's timer is an impression here.

A

This is great, thank you um which parts were you did you say you were going to be in the next release of harvard.

B

B

What component enables the hubble tracing right.

A

Yeah yeah exactly yeah.

B

Only three components: uh the harbor core harbor job service and the hybrid resistor control. I think uh this three component is the most important one: uh the hubble core. Every request will uh pass through the harbour core and all the back-end tasks are running from the job service and some the radius control is related to gc, and we because we have some.

B

Issue using this, we can monitoring the process of the gsa process.

B

C

B

C

Then, uh if I want to trace the distribution component, so I have to modify the source code to enable javascript right.

B

Yeah yeah, I I already uh enable the industry components if you wanna tracing the functions or other apps other things. You can add the tracing points in your code, for example, you can add some data, you can add some notes of this tracing and you can also add some more informations.

A

Thank you any any more questions for ken.

A

All right, thank you, ken.

C

Thank you. I have another.

A

Question about the.

C

Trace now I thought you are conflicted in the uh avayama right, so uh so for the second admin there is no chance for them to update their jaguar points from the online.

C

They have to restart harvard to make the conflict.

B

Yeah yeah, because uh uh you you are always uh if you want to start, uh stop the tracing. You need the uh restart the hardware, but there's just some other solutions. For example, you can always expose the hardware to hardware data and using uh another component uh like the open, animatric connector and using that component to control uh if to send the data to the end.

B

So if you using the third component, you um you will not stop your service.

C

C

A

All right um so with that. Thank you again, kian for the demo and answering all the questions. Next up, we're going to move over on to shanghuang.

D

All right, thank you, jonas and today I will be representing some um stop scanning future and demo, and uh in here actually there are two scenarios. One is to stop single scan job of a given artifact and the other one is to stop scan or job.

D

Just to make to give you guys some background in this demo, um I will use the fake scanner or use the fake scanner to mark the scanning activity and each scan job is set to take 20 seconds to do the scanning activity, and this gives us some time buffer to click a scan, but just stop scan button. Let me oh okay, let me show you the the.

D

Second bottom: here this, we use a fix, as I said before, when we use a fake scanner to set uh for this for to mark the scanning activity and in that fixed scanner. We said each second job is taken to try it's taken, 20 seconds, to finish the scanning job, and that gives us some time buffer, approximately 20 seconds to click. This stop second job to click. This stops again bottom here and.

D

And therefore determinate uh currently running scan job, and uh let me show you guys the uh the scanner I'm I will set you for this demo and.

D

And I put a few artifacts to this demo environment and uh when I click and the what immunities here is like low vulnerabilities um before, because I do, I did some practice before to to prepare for this demo. Normally, it will be not scanned right, but either way. If I click scan and then stop scan, the vulnerabilities here will be stop scan. It will not be in other status or by the way. I also enabled a webhook for stop scanning.

D

Let me show you the webhook.

D

I created a created a web hook for scanning fields, getting stopped and scanned finished.

D

And let me jump to.

D

So the webhook respon request will be received here in this uh in this uh unbrowser session, and let me click start scan exclude and scanning will take approximately 20 seconds and let me stop scan and trigger stop scanning successfully and the status availabilities field is changed to scan stopped and we can also check the webhook should be.

D

There is a webhook for this.

D

It's triggered at 9, 19 p.m, and it's for the engix image and that is getting stopped and the next is to show you guys, stop scanning all right and prior to stop scanning all. Let me and change the stats availabilities from start from scan stopped to uh low vulnerabilities, because we also want to see this will be changed back to stop scanning all will be changed back to get stopped. When I show you guys, the show you guys the the scanning stop. Oh.

D

It will take approximately 20 seconds, and this is just to make sure we prepare for uh stop scanning or make sure the vulnerabilities is not scanning stopped, because we want to check the vulnerabilities sphere here later on and.

D

And let me jump, you will also trigger a webhook for scanning and completed. It is not because we just finished the scan, uh completes a scanner and.

D

So it's time to demonstrate of the stop scanning stop scanning all and before to do that. Let me show you guys the the few. The few images there are, no vulnerabilities in these vulnerabilities fields and the other ones are normal abilities as well and just to show you guys. This is known abilities, and this is no vulnerability right and let me go to here to show you guys how to stop scan all and let me clearly this request, uh scan or start scan, or it will take approximately 20 seconds to finish it.

D

Let me stop it change it to to stop scan right. Let's stop it. They're all stopped and let's see, if there's any web hook being triggered.

D

There are four images as as we saw beforehand, and there are four. Therefore there are four webhook requests here. One of them is like photon.

D

Next one is reddish and next one is ngx, and next one is alpha and and let's double check the vulnerabilities field, it should be.

D

It's changed right from no vulnerability to scan, stopped right and the other radius is changed from scan, known abilities to scan, stopped.

D

And let me check photon same thing here and off open something here, and this is essentially how to stop uh scan scan all and there's. One thing left, though, is like. I also want to show you guys how to stop schedule scale, schedule scan or before doing that that essentially we can do that if, like we, create a schedule for scan all right here, but before that, let's also prepare for that environment is to do that.

D

I scan all to make sure all these uh all these images is changed from not scanned to and to know abilities, because we also want to double check if this vulnerability is fit on a lower. This vulnerability field from normal ability to stop scan, if not right as prepare for this.

D

For this test purpose that success- and so we essentially we just need to make sure there are. There are no vulnerabilities, because we want to check this later on. When we stop them, they should be. They should be again stopped.

D

And I just checked two of them right. The other ones should also be known, abilities everywhere.

D

No one abilities.

D

No vulnerabilities, and once once we did a stop scan all for scheduled scale scan or it should be changed from novel abilities to stop scans right. So to do that, let me just create a schedule scheduled job for this.

D

This essentially means every meetings wrong. These scan or job right and they're all success.

D

uh If right now it's like 9 26, let's save this uh in approximately one minute, it should trigger a scheduled scan or job right. Let's refresh it.

D

And let's wait for for a few seconds.

D

Because this schedule is not triggered at this moment and let's wait for other a few minutes. A few seconds.

D

Okay, this schedule drop is running currently running and, let's click it.

D

And click it and also want to change it back to no, because we don't want it to run again to overwhelm our webhook. Let's see the webhook, there is a get scanning stopped right. It's getting stopped scanning stopped it's getting stopped for because we have four images: uh radius, ngx and photon and opaque right. Just stop scanning stopped and let's double check the vulnerabilities field.

D

And for engix and it is scan, stopped right.

D

And for radius is again stopped if, before it was no vulnerabilities right and for photon is again stopped and for open is getting stopped.

D

So essentially that, basically, is how stop single scan job and the stop scan or job works from from user experience, and essentially we have uh stop single scan job for of a given artifact and scan, stop scan or for manually triggered and stop scan or for scheduled uh for scheduled, scan or jobs, and that's essentially, my short demo, for this stop scan job feature and right now, I'm happy to take any questions.

C

uh I have questions, can you switch to the website uh web outside.

D

C

Yeah, I wanna see the chasing data of the stock. uh I noticed that the the operator is r2 so uh who, who is r2.

D

Okay, you mean this operator should be adding or something you should be adding right. Okay, that's a good question. Maybe we should do some change on the code base to make it to admin, because the current operator is adding right.

D

Yeah. Thank you for this uh catcher and yeah. That's a good question. We may need to change it. Make further changes to the code to make it to admin instead of autograph.

C

Yeah, I suppose it should be the operator, the name name of the user who's, the.

D

C

D

Instead of auto right.

C

D

Okay, yeah, let me see if, uh if, if, if there's anything, I can do to to change this operator from auto to admin to to the current login logged in user right and accept this operator. Is there any other um other issues you want to raise up.

C

I suppose I have a thousands of artifacts, so if I stop scan or in the middle of operation, I should receive thousands of one.

D

That's it that's a great question. Yes, I saw this.

C

I I it should be a noise for for the uh like cool system. I I I believe.

D

Yeah, that's insanity I or what I implemented when I was implementing the webhook for stop scan, or I also think about is like, should we receive um a webhook for every scan job so in the case of scan all? If there is a thousand second job for that second old task, then there will be they will in in this. In in the currency implementation, there will be 1 000 web hook, requests being sent to the webhook receiver.

C

Yeah, I I I think that this this could be discussed.

D

Even the webinar channel will be overwhelmed by the stop, stop scan or web hook. Payload.

C

Yeah, so I suppose there should be only one one hope for stop scan or just to give the summary summary information about how many is scanning and how many stuff, like the ui issues,.

D

Because yeah because the webhook is sent on a job-based, not on uh execution based, so I uh so I guess there is barely anything that we could do to make. This stop scan all uninstant one web hook instead of 100 or 1000 web hook, requests if there is a 1000 scanning job at the back end, because the webhook from my knowledge, like the webhook, is like a job-based, if not an execution based. So we I I I currently I I can't see.

D

I I can't see, there's anything that we can do to change it to the to change the webhook to a job based webhook request.

C

Dearborn you go ahead, I adjust the for, uh I mean from the user perspective. The song should be, uh can be a noise for for my system, because I just take one action in in harvard so yeah um from a coding perspective. Well, we can discuss, I think, yeah.

D

Yeah, if not only scanning stopped issue, if there are 1 000 jobs at the back end and they are all scanning completed, they will also be 1. 000 scanning completed or scanning finished webhook being sent out right, yeah yeah. Essentially, this is because the webhook is like a job-based, not an execution based, I see so yeah, but we can discuss it offline and see if there is anything that we can do to make. It happen like, instead of overwhelming the uh the webhook channel with all these tons of stuff scanned webhook.

D

Yeah is there any other questions for stop scanning feature.

D

All right, that's it! Thank you, everyone and that's essentially my demo for stop scanning feature and let me hand it to you. Jonas.

A

Yeah, this is great. Thank you so much england is that right. Does anyone have any other questions or comments for penguin.

A

All right uh does anyone have any other updates that they want to share on the community call today.

A

Any other discussion topics.

A

All right, then, we're done for today. Thank you. Everyone for joining have a fantastic rest of the week and see you all in two weeks. Thank you. Thank you.

D

Thank you. Thank you. Thank you. Bye.