Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2020 - Virtual, 4 Sep 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Handling Container Vulnerabilities with Open Policy Agent - Teppei Fukuda, Aqua Security

Description

Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Handling Container Vulnerabilities with Open Policy Agent - Teppei Fukuda, Aqua Security

A vulnerability scanner for containers doesn’t help a user decide how to handle vulnerabilities. For example, even if a critical vulnerability is found, some organizations may accept the risk of it. The policy for vulnerability handling depends on the organization, and in many cases, the person in charge has to make a manual judgement based on this policy every time. This is time-consuming. This talk demonstrates how to automatically handle vulnerabilities detected by a scanner using OPA. - The vulnerabilities found by a scanner in CI are handled automatically by Open Policy Agent - Applying custom policy, OPA shows users which vulnerabilities to address This automatic vulnerability handling in CI will be demonstrated live, along with Trivy, which is an open source vulnerability scanner for containers. The same policy handling model could be used with any scanner.

https://sched.co/Zekd

A

Hi everyone thank you for taking the time to attend my session, I'm tepif kuder from the open source team in aqua security and the maintenance review, which is a vulnerability scanner for container images.

A

Today, I'll be talking about vulnerability, handling with open proceed agent.

A

There are a lot of software availabilities in the world. Usually cv id is assigned to each vulnerability in the case of hot bleed. Cbe 2014 0160 is a sign. Do you know how many vulnerabilities are reported per year?

A

This chart is from nbd national vulnerability database, basically listing all cbeid assigned vulnerabilities in 2019 there were 17 000 vulnerabilities disclosed.

A

As you can see, the number of vulnerabilities has increased year by year, in other words, about 47 vulnerabilities are reported a day. Basically, you have to understand how vulnerabilities work in order to know how much impact it is. But you know it's not easy.

A

You might take a few hours to understand. Only one vulnerability, 24 hours is not enough to figure out the 47 vulnerabilities.

A

If your organization has a huge security team, you might even be able to do that, but you should not do that. Since 47 vulnerabilities have a lot of noise, it may include vulnerabilities of windows, open ssl, python, kubernetes.

A

What what? If your organization doesn't use, windows and kubernetes?

A

You don't need to look into those vulnerabilities, so asset management is essential for this purpose. You should know which os package, programming, language, library and equipment are used in your system.

A

If you remove vulnerabilities related to software, your system isn't using. You can reduce vulnerabilities. You have to look into, for example, it might be five or seven vulnerabilities depending on your organization, should we do this process manually?

A

Let's think about the container image. A container image should include all software it's gonna use.

A

All you have to do is extract information from the container like the image is based on debian and uses java, then close reference to security advisories from vendors.

A

It sounds easy, but it's still hard to do it for all container images manually and periodically.

A

For that purpose, you can use the vulnerability scanner, which automatically does does the things I explained just before. There are many security solutions in the cloud-native area, mixing open source and commercial products. It also includes 3b, I developed, applying vulnerabilities scanning reduces the number of vulnerabilities you have to investigate.

A

Do you think five and or two or ten vulnerabilities are small?

A

This number is just an example and it should be different depending on your system and your organization, but I'm sure it must not be zero. You will see sound vulnerabilities. I don't think this value is small enough. As I told you earlier, it might take a few hours to look into vulnerability.

A

How can we reduce vulnerabilities more take a look at this chart? A vulnerability basically has a cvss based score, indicating the stability. The score varies from 0 to 10. 10 means the most critical cbss consists of three metric groups base temporal and environmental, but in this presentation I'll be talking about the base metrics and score because nvda provides base metrics only as you can see, there are not so many high risk vulnerabilities only five point. Seven percent of the vulnerabilities have a score of eight or more filtering based on cvs.

A

Scores can further reduce vulnerabilities, it might be zero or one vulnerability. If you look into critical vulnerabilities only of course, zero or one depends on your your system.

A

Let's consider whether the services score is reliable in the case of hard blade, which affects open, ssl, the cvss v2 score. It sets 5.0 media according to the cvss score. It doesn't look like it's a dangerous vulnerability. Can we think of this vulnerability as negligible and ignore it? As noted in this article hackers exploited this vulnerability, so this vulnerability is not negligible.

A

In addition, the services score is provided by each vendor. In this example, nvidia says the score is 9.1 critical, while record sets 5.9 moderate. The values are often different.

A

I'm not saying don't use the services score, but it's a bit risky to rely on cbs scores too much. We should define our own policy for vulnerability handling. For example, our container image uses bash, but we can accept the risk of batch vulnerabilities as bash is not internet facing.

A

Of course, if you don't need it completely, you should delete it from the image. Also, you might be able to accept the risk of cross-site scripting. If your system is a static website and you perhaps can ignore the vulnerability which requires user interaction like opening the email and clicking the link, this policy should be customized depending on your system and requirement, and the system managing customers assets like bank account and system just publishing an article like the blog, must have different policies.

A

I recommend defining the policy for vulnerability handling if the vulnerability is bash, related or cross-site scripting or requires user interaction, then you'd classify it as a negligible vulnerability.

A

If it doesn't match any rule, you have to fix the vulnerability by updating the version or something another way. Actually, there is thermal useful information for vulnerability handling such as services, vector and cwib.

A

Cbss vector string consists of eight elements called cbss metrics. Each metric value is used to score the vulnerability, for example, attack vector this matrix reflects the context by which vulnerability exploitation is possible. This metric value will be larger. The more remote an attacker can be in order to exploit the vulnerable component.

A

In addition to attack vector, you can find privilege required, user interaction and so on.

A

Cwe common weakness enumeration aims to provide a common base to identify the type of software vulnerability. For example, cwe 17 8 is os command. Injection and cw 89 is sql injection. Now you can rewrite the policy using these metrics.

A

If you want to know if the variability is xss, you can check if cw id is cw in 79 or not.

A

If you want to know if the vulnerability requires user interaction, you can check if user interaction in service spectrum is required.

A

So it allows you to handle vulnerabilities automatically, and I propose to use open process agent to automate vulnerability. Handling open agent is open source policy, engine and cncf project. You have to write a policy written by rego and pass it to opera with input. Json opera evaluates the policy and returns the result, it's usable as a library and service.

A

Now you can put the policy into lego. The policy written in regal should be like this on the right. You can see the vulnerability detail of the input you can see. The package name, cwid cb id services score vector on the left. We can see the pretty simple rules if the package name is bash or cw id is 79 or user interaction is required or services based score is less than 7. The vulnerability is going to be negligible.

A

Opera should reduce the vulnerabilities to 0 or 1 a day depending on the policy. It's so small that you could look into old note that validate information is not always correct. If anybody says if the interaction is required, it might be wrong.

A

Don't trust the use variable information too much, even if it's correct and the policy makes it a negligible, it doesn't mean there is no risk. It means you decide to accept the risk. The balance of risk and cost is important.

A

If there are many people in a security team and the organization can take the cost of security, you may be able to look into the patches and the primary source of all vulnerabilities.

A

If it's not the case in your organization, you probably need the policy to accept some risk. Here's a summary of what I've been talking about so far at first vulnerability scanning filters to only those vulnerabilities that are relevant to your organization, then oprah filters out vulnerabilities that you can accept the risk. This whole process is automated.

A

You can just focus on the vulnerabilities in the final output as a case study. Here's a look at open integration in 3b.

A

In this example, trivia scans, an image and detects vulnerabilities then operates the result.

A

Okay, so before that, let me explain 3b a little. It's an open source channel for container images. 3B has a bunch of features, and especially it can be integrated into the cs system, such as guitar actions and jenkins. Easily.

A

Recently, trivia supported, open integration to filter vulnerabilities. You can pass a lego file, defining your policy, be a minus minus, ignore policy option.

A

If your policy marks the variability negligible it will, it will not be displayed in the result.

A

The structure of each vulnerability input is the same as for the 3b json output. It includes severity, cwids, cbss, vector and score, and so on.

A

Also, it provides helper functions in rego like pass c basis, vector p3, which converts services vector string to an object so that you can easily access each metric.

A

This example ignores some packages like bash binder, license rpm and bim because they are not used in the production environment, as mentioned earlier, unnecessary packages should be removed from the container image, but if it's difficult, you can ignore packages in the level policy, then vulnerabilities with low and medium stability can be accepted.

A

Also when it's not exploitably motoring, or it requires high privilege, like administration or needs user interaction, and they will be marked negligible in this example. Okay, so let me show the demo of integration into vcli at first. Let's try scanning center 7 without any policy in this demo. The result of detail is not needed, so I am redirecting it to the null.

A

We can see the number of vulnerabilities 622, it's a hard task to check all vulnerabilities next. Looking into the policy example, as I explained, it ignores some packages, some disabilities and a vulnerability that can't be exploited remotely or requires high privilege, like administrator or requires user interaction or a cross-site request. Forgery. In this case.

A

Okay, let's apply this policy to the results to be shown. You can pass the regular file via minus minus, ignore policy option.

A

As you can see, there are seven vulnerabilities only in the result with the policy, as you saw in the image center. 7 shows 600 vulnerabilities without policy, but it would be only 7 with the policy.

A

Less is not always better. You have to tune your policy according to the requirement in your organization.

A

At last. Let me explain open integration in kubernetes. This technique can be applied to any scanner, but as proof of concept I developed kubernetes operator called 3b enforcer.

A

It works as custom controller to scan an image in advance and admission controller to validate an image. A user tries to deploy. Let's take a look at the architecture at first, a user registers, an image name. They want to deploy as a custom resource to be enforcer, watches the custom resource as custom controller and fetches the image name.

A

Then it sends the image name to 3v server, for vulnerability scanning to be server returns, a result and finally, 3d enforcer updates the custom resource with those vulnerabilities.

A

The second result of the image must exist before you try to deploy it. Besides, a user needs to upload their policy to config map, cube management provided by the opera team, watches, configure map and loads the policy into open service.

A

What matters is you can take advantage of the same policy you use with 3vcli and 3d enforcer calls open service through rest, api.

A

When the user is trying to deploy the image, kubernetes api sends admission review to trivia enforcer and the request contains the image name to be retrieves the custom resource that matches the image name and extracts the result of the vulnerabilities.

A

If there is no second result, it rejects the image immediately. Then it sends vulnerabilities to opera and oprah evaluates those vulnerabilities. If one of them violates the policy, it returns denied response.

A

If there is no vulnerability by relating the policy to the enforcer, allows, the deploy of the image all right so I'll demonstrate how turbine processor works with open agent in the kubernetes cluster and prevents a vulnerable image from being deployed at first. I am checking parts in the default namespace.

A

There is no resource and checking the 3b enforcer system so where to be enforcer, is deploy. It works as custom, controller and admission controller and also checking the opener name space.

A

You can see one point: it includes two containers, one is the open service and the other one is cube management to load regular file from configmap to open service. Okay, let's check the configmap map in the open namespace.

A

So you can see the one rego file, including helper functions.

A

This dragonfire is deployed by default from by 3b enforcer.

A

Let's check the content of this reg file functions.

A

Okay, as you can see, this logo file provides a helper function like pass services, vector b3, converting services, vector string to the object. Okay, you can import this package and calls these functions.

A

A

Next, you can see the same policy I used in the previous demo example rego. It ignores some packages and stabilities and some other rules as well. You can reuse the same policy in equivalence, integration to make use of it. You have to deploy it as a config map, so let's run create config map.

A

And 3b from file example rego and namespace over okay, create it all right. Let's check the comic map.

A

Okay, you can see the 3d copy map, so this regular file is uploaded to configmap and load it into open service through curve management.

A

In kubernetes integration, you need to define the package kubernetes by dating because 3p enforcer queries it. So, let's define the policy web.

A

Package name must be kubernetes, validating, 3b and import data.3b. This is what we uploaded just before, and also the there must be deny rule.

A

But we already have the logic to evaluate the vulnerabilities in tribute to ignore, so we can just call the imported rule here uh to contribute to ignore, but the ignore means you can accept the risk while denies that you cannot accept a risk. So we need a not and you can define the message so, for example, the image for okay. That's it.

A

And also, we have to upload this regular config map with.

A

A

Okay, so there are three config maps in the open name, space and web fork. Imports 3b, also 3b imports to be helpful functions. Okay,.

A

We've completed the preparation of measure assurance, so let me try to deploy ngx mage hx 1.18, so it was rejected because there is no scan result for this image.

A

This is because so we've never added a custom resource with this image name. So there's no second result in the cluster okay. So we need to add the custom resource, including this image name.

A

Okay, so this injects.yammer has a kind image vulnerability, so this is the custom resource. The name is ngx and image name is ngx 1.18.

A

So before uploading, let's check in the custom resource, okay, so there's no resources. So let's uh add it. Let's add this custom resource and checks.

A

Okay, so here you can see the ngx custom resource and 3b enforcer, watches this custom resource and it detects the change so now to be enforcer scans, the image of ngx 1.18 and update the vulnerabilities okay. Maybe it's already finished image from this piece into x, yamu.

A

So you can see the labels, the namespace registry repository and tab. Also, you can see the status field. It includes vulnerabilities, as you can see. So there are vulnerabilities of this image, for example the packaging name apt and stability row and the probability id cbe, 20 11 374.

A

A

Now we have the second result of njx 1.18.

A

So let's try to deploy this image again to the control run mgx you image, gx 1.18, but it still failed because this image has a high severity vulnerability, and you know this rego file doesn't allow high security vulnerability.

A

So let's try editing the config map to allow high stability.

A

A

I'm adding the high stability okay so after editing, this peg file the cube management loads, this regula file to open service. So this the price should be succeeded.

A

Ngx manage ngx, 1.18, okay created, so now we confirmed that it works properly with open proceed agent.

A

If you use herba as a container registry, 3b enforcer can clearly hover to fetch vulnerabilities of the image because harvard supports image scanning in the registry. In that case, you don't need to add a custom resource for free scanning, so I have the demo of open integration with hardware register.

A

So now we have the hardware deploying the kubernetes cluster and the browser is 2 0. One.

A

So it's the latest version and the domain name is called hardware domain. So there are some images under the project, library, so library, ngx and library alpine. So let's have a look at ngx. It has only 131.18 and it has some vulnerabilities, including high stability.

A

We need a different policy than the one we just used with 3b, because hubble has a different format of the scale result so defining the process for harder. The package name must be kubernetes by dating harbor and includes deny with the message in this example. It just denies the vulnerabilities of high and critical stability and remove streams policy used in the previous demo.

A

So you can see the only one regular file, and so we have to upload the policy to config map.

A

This policy is also loaded to open service, create copy map from file hardware.lego, open, namespace, okay, create it and load it to server, and it can find the policy of hardware and config map.

A

So let's try deploying the image in the hub registry. So in this case I'm using the core hardware domain, slash library, slash, ngx, 1.18,.

A

But it was rejected because, as you know, this image has a high stability vulnerability, so the this image violates the policy so modify the policy to allow high stability.

A

Okay. So let's remove this uh visibility. High stability from the policy.

A

And try to try to deploy the image again around ngx image or harbor domain library, ngx 1.18, so the pro is supposed to be allowed.

A

So, as you can see, the image has been created, so you can probably image assurance throughout the development life cycle in local development and continuous integration. You can run 3b cli with minus minus. Ignore policy option then to the enforcer can be used in a kubernetes cluster to reject the drawings image with critical vulnerabilities.

A

The important thing is that you can share the same policy across the whole process.

A

Note that they are still experimental features and not stable. What I wanted to get across in this presentation is that we should define a custom policy for vulnerability handling based on your organization's risk acceptance.

A

Then open policy agent can be used for that purpose. You can find some proof of concept of integration with 3b in those plc. You can take advantage of the same urban policy throughout the development lifecycle.

A

That's it thank you for your attention.

A

There are a lot of software vulnerabilities in the world, usually cv id is assigned to each vulnerability in the case of hot bleed, cbe 2014 0160 is assigned.