Cloud Native Computing Foundation KubeDay Japan 2022, 9 Jan 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Generalizing Policy-as-Code for Compliance Posture Management on Multi-Cloud Infra... Takuya Mishina

Description

Don't miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from 18 - 21 April, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Generalizing Policy-as-Code for Compliance Posture Management on Multi-Cloud Infrastructure - Takuya Mishina, IBM

Compliance posture management is an essential feature of enterprise Kubernetes-based infrastructure. There exists various tools and rule sets ("policy as code") to ensure the compliance posture of Kubernetes clusters, but some of complicated Kubernetes usecases such as multi-cloud and managed services require additional functionality for such off-the-shelf rule sets. This presentation shares the extension of ComplianceAsCode, an open source predefined rule suite. The new functionality of ComplianceAsCode enabled Compliance Operator, an open source compliance posture management tool, to support multi-cloud and managed usecases achieved by HyperShift, which deploys multiple Kubernetes control planes on another Kubernetes cluster. Attendees as users will be able to learn how to manage the compliance posture of multi-cloud and managed Kubernetes products and services using ComplianceAsCode, and as developers they can learn how to design and implement flexible policy-as-code for various types of Kubernetes usecases.

A

So, thank you for uh joining my talk. uh The title is generalizing policy as code for compliance project management on multi-cloud infrastructure.

A

So today, I'd like to share with you about the value of automated and community-based compliance, posture management, using compliance operator and the uh for managed multi-clusters with high passive. So this uh presentation consists of three subtopics. One is about hyper shift, which is an open source, managed cluster enablement tool. So you may have an experience using the some of the monitor tools, but then I. Let me introduce the open source tool. Hypothesis and second topic is a components operator and compliant code, which are open source compliance, posture management or CPM to land policy.

A

So the compliance posture management is another important workload for the many cluster or multi-cloud environment. So I'd like to talk about that, the second in the second topic- and finally, this is the a new feature in this year, so adoption of the the compliance operator to high passive for consistent compliance, posture management that did today's topics.

A

Okay, let me start with Hyper shift.

A

uh Hyper shift is an open source managed cluster, which focuses for the red hat, open shift clusters and the uh you may have an experience to use the managed cluster tools offered by cloud service provider.

A

So you can create or delete your cluster by just clicking the button and the you may also have an experience that the uh you use the some Standalone tools to manage the life cycle of the uh multicluster or managed cluster like terraform, Argo, CD or tecton through the uh life cycle management API like kubernetes cluster API, so the high passively the similar tool for such Standalone tools.

A

So you can create or delete your own cluster on your bare metal or the infrastructure on the cloud service provider, and it's a unique feature is a hosted, control, plane. So I think it is better for me to introduce the what uh uh control plane. So you may know that, but let me uh introduce that the uh the left hand side uh small box uh shows the uh inside of the standard kubernetes cluster.

A

uh It consists of two kind of kinds of components: one is control, plane component and another component, so uh control plane, in short, the infrastructure components for kubernetes clusters, for example, API server etcd for the scheduler or controller manager. So it is a very important components and they run on Master nodes and other workloads like users. Application like database or web application or Alexa are classified as a Waka component and they run on vacanos.

A

So in a standard kubernetes cluster, they they consist of one cluster with two kinds of the nodes, but in the hyper 60 environment the Contour plane uh hosted on another cluster called the management cluster. So this the uh Center picture shows the uh architecture of the hyper shift. If a user attempt to create a cluster, then the Waka will be created for that cluster, but the its Contour plane will be hosted in another cluster and the the con control plane exposes its apis to the worker component.

A

So the uh Waka and the exposed apis virtually forms the uh kubernetes cluster. So when I, uh when a user attempts to create another cluster, then the another workout will be deployed, but the control plane will be divided into the same management cluster. So in other words the multiple control plane will be deployed in a single cluster. So what the benefit of the hypersf2? The first benefit is that the we can reduce our workload by uh centralized management I mean, for example, uh an infrastructure engineer.

A

I need to apply patch to the control plane, and in that case the uh the infrastructure engineer, I just need to log in the single management cluster and and perform the management operations at a single operation. So that's the first benefit and second benefit. Is that the uh the managed cluster our users cluster can be deployed quickly because the basic components like basic resources, like Network or stress, have already been deployed as a part of management cluster and the?

A

uh What we need we need to do is just deploying the control plane component spots like APA server and so on, so it can reduce the time for a deployment and, finally, the include resource I utilization as similar to the virtual machines and Battery machine monitors. Many control planes can be packed into a single cluster and it improves the resource utilization. So that's the advantages of high passive and we can leverage it to uh how to say optimize. The money is the workloads.

A

And what uh other unavoidable workloads we can deduce? That is the compliance posture management.

A

The compliance posture management is the process to ensure that an ID system complies with internal or external policies. So you may think about that.

A

The what the difference between security and compliance, so uh compliance policies, often contains many security rules like the minimum password link should be longer than eight characters or the uh the encryption key The Links of encryption Keys should be larger than 1024 bits or larger, but the uh the important difference is that uh in the compliance process, we uh not, we need to not only configure the I.T system secured, but also we need to ensure or check that the system is configured as expected.

A

That's the difference, so the compliance posture management uh consists of four steps.

A

Let's go through how is example, our policy. This is about one of the rules in Shia's Benchmark for kubernetes. In this lure, API server must not run with option in Secure. Buying address in this case. First step is collecting fact so fact is, can be thought of as a configuration to the I.T system and the in this case. The facts can be collected by this file path in the master, node and then the second step is applying policy.

A

The uh we will say the string insecure bind that should not exist in this fact file and the results should be reviewed by the compliance engineer and then fixed violation. If this kind of violation of is found so the in the compliance posture management, uh it is important that the Automation and open Community Based technology play important roles, especially for cloud native compliance, social management.

A

Why so, why automation matters the uh as you may use the cloud native tools like devops and detox based tools, it frequently or continuously changes the ID system and therefore we need to uh they perform. This process are frequently well continuously, in that case, how to say the Legacy process like monthly or quarterly or annually such annual check doesn't matter. So we need to perform a frequent check. So then, therefore, automation is essential and the second is that open, Community Based.

A

So why are open matters so, uh as I wrote, the external policies, so the many laws and Industry standards exist and that common common that can be commonly applicable? uh Many organizations, for example in the financial company in Japan uh such a organizations, need to comply with the uh the security standard given by the uh Japanese Financial agency. So such and in addition to that uh effects, infrastructure is now common. You know the kubernetes uh we can.

A

uh We use q and this as a standardized infrastructure and therefore we can apply a same policy to uh similar our identical infrastructure over multiple multiple organizations and therefore the uh open Community is very important for this topic.

A

Okay, let's move on to the concrete tools. The first one is compliance operator, which is an open source. Compliance posture check engine, so this is an Operator kubernetes Operator, but they are inside of the operator. There are many mature Technologies, but it is wrapped by Cloud native Technologies, so this figure is same one in the previous slide, but the uh these steps consist of multiple techniques.

A

The uh for collecting fact, the uh Cloud native Technologies, like Cube, API or storage access to the host, will be used to collect the fact, for example, file in the master node and then the up for the applying policy. The very mature technology is used for 2008. This is the openness cap started in 2008 and policies is written in xccdx, which was standardized in 2005.

A

and the uh to review. The result we can use the both material and Cloud native Technologies and xcdf, and we can also use chromatase, metallics and also the sum of the violation. Fixed fixed process is automated with the kubernetes Manifest or ansible Playbook.

A

So this complex operator, we can gain several benefits. The first biggest one is that we don't need to write policies from scratch, so we can use existing xccd rules and and I will introduce it in detail in the next slide, which is called compliance as a code and also we can integrate it with both Cloud native tools and Legacy tools. For example, we can deploy and configure compliance operator with Cloud native tools, including operator, like cycle manager and the rocd or tecton, and as I told you that leave you without obserably.

A

Details can be used for the uh leaving the result, so the complex operator exposes its result as parameters metallics, and we can use uh Prometheus over the grafana for the uh graphical user, user, interface or dashboard, and also we can use the material dual Legacy Technologies, like XML, xslt or css, to review the results through the uh browser or the uh pass it to another tools via through the XML technology.

A

And the compliances code is a set of technos which are maintained by the open community, and it consists of the rules and Remediation script and the lures are written in yaml file, as shown in the bottom of this slide. So it consists of location for fact, for example, in this case, so this is the same one that I introduced previous slide, so disabled use of insecure bind address. So in this case the fact can be collected through the kubernetes API with this URL, and the it contains also contains the logic for a check.

A

In this case, the string in Secure bind address should not exist in the in this Factor data, so it supports the whole infrastructure. I mean the it contains the operating system rules and the middleware on application use, including the open shift cluster, and it supports various regulations. For example, in case of the Clusters, an East SP 853 cs20 Mark PCI DSS are supported, so they all of them are very popular components, uh litigations or standards in the I.T industry.

A

So uh let me uh share the architecture slide of the compliance operator working with uh compliance as a code. So, let's rectangle represents a component components of compliance operator. uh First, the resource collector collects the effects from cluster, which is indicated in the location in the policy, I mean compliance as a code and then fact will be scanned by the apple plus operator, with technology provided by policy and the result will be consumed by various tools, including the metallic exporter and Report generator, and the limit automate automated remediator for clusters.

A

Okay, so far, I have talked about two uh kinds of twos one one is hyper shift form, one is the cluster and another is the compliance operata for the compliance posture management. So the next change for us is, uh you know the you'd like to apply the compliance operator not only for the Standalone conductance, but also for hyper shift, managed clusters. So challenge here is how to support the hosted controller plane feature uh as I uh introduced in the second Slide.

A

The uh managed clusters under the high passive are split into two, how to say two or multiple Parts one is the clustered and analyze management cluster and the control planes are hidden from the user's cursor. It just exposes the API, then the Contour playing component demonstrates are hidden and the components of it are running in the users. Cluster can't access the controller plane itself, so we need to use another compliance operator instance on the management cluster and the this instance need to support multiple control planes on a single cluster.

A

While standard compliance, Operator just need to check single controller plane. So that's the challenge, but the we. If we can generalize this compliance operators both for the worker and the host this uh hosted onto the planes, then we can uh enable consistent compliance. Posture management I mean we can use same tools and same policy, so the result of check can be thought of uh very consistent result.

A

Okay, the uh this is how uh such a feature is achieved. Actually this is what I contributed to the open Community, uh the adoption of compliance operator, the high passive hosted control brains. So the most important thing is that the each controller plane is how to set encapsulated so so control one control plane for one managed cluster is located in a single name space. So if there are two control planes, there are two specific name: spaces for each controller plane.

A

So in a standard uh lose the namespace name is hardcoded, but the uh we need to replace them with Blissful place for holders like this, and some of the resource name is changed and therefore we also need to replace them with placeholders in the components are the contributors and also we need to add a runtime parameters? I mean speed, touch feature to compliance of data. So when compliance operator instance on the managed cluster try to perform a check, then we can.

A

uh We can assign a specific parameter for specific control planes, for example the namespace name in cp1 or the tool. The scan for the control plane to the namespace name might be CB2 electron.

A

So that's the uh new functionality and the with these. uh With these features, we can apply the components operator or automated compliance project management to the managed clusters powered by hyper shift. So these are critical arrays.

A

So first I have introduced the high passive to one of the uh manageda enablement too, and then introduce the complex position, management and the concrete tools, complex operator and complex as a code, and the uh I have also mentioned about the importance of the open Community for this uh this operation and then I have introduced the adoption of compliance operator this High passive. So thank you so much so! That's all uh from me. uh Please ask me any question about my presentation. Thank you.

B

I have actually multiple questions so, like my first question is, like is hypershift and your compliance operator only works with openshift, or it can work with normal kubernetes. Also.

A

Thank you very good question. So actually it is a QB operator and it can run on any kubernetes clusters and you can check the uh underlying operating system with compliance operator, but the afford the uh the check for kubernetes. uh It only supports the high passive.

B

Okay, and uh is there anything like uh it, can check during the deployment as well or like it will check only after the deployment is done and it will fetch metrics.

A

B

You checking uh like, when you say, Prometheus and metrics right like things which are deployed right after that only it can scan and it can filter out the things right like saying that, okay, this is not matching the compliance right, but if I can catch those things during the deployment itself, is that possible like using some kind of GitHub actions or, like you know something like that?.

A

uh uh No predefined uh like something like that doesn't exist in the compressed operator, but uh the output is created not only for the promise uh Primitives metrics, but also the custom resource uh generated for the results. So maybe uh we can use some Fook for kubernetes resources to trigger some other actions like positive actions.

B

Okay, so means like after deployment only is it possible, but not during the deployment or like it can catch. What.

A

Deployment I mean.

B

Like when we deploy applications right like like you know um the deployment service, the state full set or config map secrets right. So during the deployment like you know, while it's happening, can it stop saying that? Okay, you are doing something wrong, don't push the code.

A

I see yeah I see yeah. um Actually the answer is no, so this checks the run time so I mean the maybe your. uh What you want to do is the prevent.

B

Correct, like admission controller.

A

Yeah, so that uh operation should be done with some other tools like the on the uh GitHub. You can use Travis or something with you and when you'd like to prevent on the kubernetes some uh yeah, so the uh controller like equivalent or oppa can work for that.

A

Thank you. Thank you for your question.

A

Any other questions: okay,.

A

Thank you. Thank you for joining my session.