Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2021, 14 May 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Intro + DeepDive: Kubernetes Cloud Provider Project for IBM Cloud - Sahdev Zala & Richard Theis, IBM

Description

Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Intro + DeepDive: Kubernetes Cloud Provider Project for IBM Cloud - Sahdev Zala & Richard Theis, IBM

In this session, the project leads will provide an overview of the IBM Cloud Provider subproject, its activities, and learning resources. The introduction will be followed by the newly added support in Kubernetes IBM Cluster API Provider for IBM VPC Gen 2 and PowerVS. The session will also deep dive into the IBM Cloud Provider and provide updates on the general Cloud Provider SIG. This session will be of value to a broad audience including active contributors, new contributors, and those with an interest in the IBM Cloud Provider features and capabilities or the Kubernetes Cloud Provider in general.

A

All right, hello, everyone uh welcome to this talk, kubernetes cloud provider project for ibm cloud. My name is sadie zala. I am a senior software engineer and open source developer at ibm.

A

I contribute to kubernetes uh I'm one of the maintenance for the fcd project and I am uh one of the co-lead for the provider ibm cloud project. I have a pleasure of having richard thais with me today for recording of this talk. So richard, would you like to introduce yourself please.

B

Yes, thank you. I'm richard tys work for ibm as a software engineer on the ibm cloud, kubernetes service and red hat, open shift and ibm cloud managed services as the release lead for delivering openshift and kubernetes releases to our platform and with uh satay, I'm co-chair for the cloud provider project for ibm cloud. Thank you.

A

Well, uh thank you richard um all right, um hello again. uh So for the agent uh you know we will provide uh an overview of uh sig cloud provider and uh this sub project provided ibm cloud. uh You know this is a part of the maintenance track, so we'll definitely give you some introduction here of the project, uh we'll briefly cover activities and then we'll talk about cluster api provider for ibm cloud and the ibm cloud provider which richard will we give a deep dive all right.

A

So um you might already know about special interest group cloud provider. It owns kubernetes cloud provider, interface, uh the code and related work. uh You know which is responsible for running all the cloud providers, specific control loops right. So you know you know that when you run money like kubernetes on different uh cloud providers right they have their uh uh own requirements, they have their own uh functionalities. uh uh For for the communities right to run, the keyboard is on on their site, so, for example, load balancer right, that's one of the examples there.

A

um So uh you can learn more about uh the the code there uh in the github repo for the cloud provider. I have put a link here.

A

um The seek also ensures that the kubernetes ecosystem evolves in a way that is neutral to all the cloud providers right. So there is no favor given to one cloud provider or other that kind of things.

A

uh It also ensures a consistent and high quality user experience across different cloud providers and the sig also owns uh the sub-projects from various cloud providers. So, for example, uh the project we will be talking today, uh the ibm cloud provider, it's one of the sub project- and you know some other examples- are like provider aws required azure.

A

You can learn more about uh all the sub projects in the link that I have provided here, all right, so um the provider ibm cloud. uh As I said, it's a project of cloud provider- and you know you would be interested in the project, especially if you're interested in building deploying maintaining, supporting and using kubernetes on ibm cloud platform right and the project also owns the cluster api code of ibm cloud.

A

We will talk about it and then I will own the code from ibm cloud provider once we have that open source as you as you can imagine uh you know as part of this sub project, we have ibm uh cloud teams, developers leader, you know they're talking uh through the open discussions uh about uh you know, what's happening there, you know sharing with the community working with the community uh so that uh you know uh folks can follow the evolution of ibm cloud platforms uh even with respect to kubernetes and other cncf projects.

A

So one thing I would mention here is uh the sub project. Is uh you know it's for the open uh source, related discussions for keywords and other projects right? It's not uh for any sort of commercial uh kind of discussion. So we we totally discouraged that kind of discussions.

A

um A little bit about the structure, so we have three uh different uh leads for the project. uh From the different areas of ibm cloud side. We have khalid ahmed, who is uh ibm distinguished engineer, working on multi-cloud management. We have uh richard theist, who is a speaker here as well from the ibm cloud community service and redact openshift community service myself from the open source site from the community side.

A

uh I have given the link for the mailing list here. So please join the distribution list there. We love to. Have you part of that? We have a provider ibm cloud slack channel on kubernetes.com.

A

uh I would love to have you know, discussions there, questions uh any kind of you know other other thoughts, uh and then you can learn more about the sub project in the link provided here.

A

Okay, um uh the the sub project team right, the the folks uh who are uh interested in the in the project, contributions the leads right. We we meet every uh last wednesday of the month so once a month, uh unless there are, you know things like holidays or no agenda kind of things.

A

uh Again, we would love to be part of these meetings. uh Provide your inputs right, contribute to the betterment of the subproject and take a leadership role. uh If you, you know miss meetings uh and interested to see. What's going on, we have recorded all these meetings. We do record, as we you know, have, meetings and and upload the videos on the kubernetes slack channel uh again the link is provided here. You can take a look and you know, watch the videos.

A

uh We also participate in the general activities of sick cloud provider right through their bi-weekly meetings during uh face-to-face conferences we meet in person uh and in our brainstorming about the the ongoing things about strategy.

A

um As I mentioned, uh you know, the the the ibm cloud provider owns a couple of things here: the cluster api provider ibm cloud. That's excuse me. It's a github repo uh that I'll I'll be sharing with the next slide and then uh the ibm cloud provider which richard will talk. Okay, all righty, um so the cluster api program cloud. uh uh Before I talk about it. Let me uh briefly mention about the cluster api project. uh As you might already know, it's uh it's a community project uh from kubernetes community.

A

He was created some sometime back with the goal of uh managing the life cycle of cubase cluster right, so creating scaling um and destroying cleaning up kind of things um through uh nuclear apis right, which are basically cuban style apis.

A

um You can read more about the cluster api project lot of good documentations. uh There is actually a whole book there. um So it's a good reading. You know to learn about cluster api.

A

uh As I said, you know it's a it's a similar style declarative apis like kubernetes. uh You know also, you know the cli, uh as you can see here, cluster ctl, similar to cube ctl uh to work with the with the management of cluster. It's there.

A

So you know different cloud providers. They typically extend this. The cluster api project uh they extended uh to meet their own requirements right to provide the support for a specific cloud provider.

A

So from ibm cloud side, uh the cluster api provider ibm cloud, the github repo we have again the link is provided here. uh We basically extend the cluster api project for ibm cloud uh for the different infrastructure of ibm cloud. So we already have a you know: a stable support for classic infrastructure.

A

What we call, uh but, uh as you might know that you know uh lately the vpc gen two was announced. We have a power virtual system, a virtual server right, so we have a work going on. uh There are a few pr's out there for review. There are some issues out there.

A

So would love uh to have you take a look. You know provide your review comments uh towards the work. You know. That's that's like in progress right now to support uh different types of infrastructure.

A

uh You know that involves some refactoring as well. So, as I said, we would like to have you uh review those things support your feedback contribute as much you can um with that. uh Let me have richard to talk about iks and and then ibm cloud provider. So richard would you like to uh take over from here? Please.

B

Yeah definitely thank you so yeah um we'll continue the conversation here into the call provider pro project here um I will set the stage a little bit talking about um iks and rocks first and then uh go into the details under under the covers there. So we'll start with iks, which is ibm cloud kubernetes service, which is the managed offering that we provide to create clusters good clusters on ibm cloud. Okay, one of you know many cloud providers out there that provide such a service.

B

So it's a starting point for where our cloud provider is used right, um as you can imagine, ibm cloud provider running in this service.

A

B

Want more information on the service feel free to check out the link? I provided there key thing here: is it's a certified kubernetes offering um from the cnc app? So this is a really important piece for the kubernetes ecosystem to provide certifications, whether it be a managed service or um or a distribution right that it can that you can say with confidence if you're running your applications on this particular thing that it's certified kubernetes to run the kubernetes um apis.

B

So with that we'll jump to the next slide and we'll go over some of the release processes for iks and how they relate to the community. So this year, 2021 we've released 1.20 release of kubernetes. Last year we had three releases in 2020 time frame, 17, 18, 19. and um we've found from our users and internal and external that maintaining currency with kubernetes and the entire ecosystem of cncf can be rather difficult because of the speed at which they move.

B

So you know looking into the more details, we've seen, not only us, but it's really a lot of industry-wide a lot of a lot of folks seen the similar thing. So it's not just unique to folks running cube on ibm cloud. It seems to be very pervasive a lot of discussions on the topic and there's a new cap sig release cap to add release cadence for kubernetes, and this is a really um assuming it gets in improve, approved and merged.

B

This cap would have some key impacts to the project and to every uh all those consuming the project downstream, so proposals to go to three rather than four releases of kubernetes every year, um believing that you know a lot of background investigation, that's gone into it feedback surveys and such um so that I think, have a good direction for the community to to pursue, um and um so, if you're interested in following that and how that's going to play out.

B

Certainly a lot of the ecosystem will be impacted by the results of this so and I think for the better I think for the better um if uh in as far as patch releases are concerned. So this lays out the high level the major minor release updates, which usually have been four year coming down to three. Potentially um they also kube releases um patch updates every month and we've done the same for iks, so very good cadence there on delivering patches and, for the most part, very good quality coming out of kubernetes community.

B

All right next slide. Please um rocks, which is red, hat open shift on ibm cloud. We call it the red hat, openshift kubernetes service internally, here um it's another managed offering this time allowing you to create openshift clusters on ibm cloud, so we're all about. You know in a cloud provider being able to provide both cube and openshift through our cloud provider.

B

Same cloud providers used for both um if you want more information on openshift in the service here check out the link again certified kubernetes, so under the covers same stuff, all right next slide, please just like icast rox has a very similar release cycle: okay and it's again based on openshift and openshift's based on kubernetes. So you can see the cascade. This is one of many examples in the in the kubernetes community with respect to applications running on top of kubernetes and how a lot of them have a similar release pattern to underlying kubernetes.

B

So for rocks, we've had one release this year, uh four six open shift on top of 119 kubernetes that is and three releases last year you can see them listed out, outdates and so on again, both for kubernetes users, but also openshift users. They see the same pattern of adoption and and issues and concerns, which is, um can certainly be alleviated through vendors supporting things longer, but it makes it more difficult, especially when the underlying support is not there from the community.

B

So again, this cap will have impact like I mentioned, for coup, but also for openshift going forward, and I suspect we might see some changes potentially coming from these downstream providers as well. All right next slide, please all right. So how does this all tie together? So, as uh saudi said we're?

B

We always love talking about community stuff, kubernetes on ibm cloud right and most people deploy it through the managed services, but certainly they could deploy it other ways as well um and so supporting those in whether, however, they want to run it um usually an important piece of that is the cloud provider and then the cloud controller manager in particular so rewind a few years ago, and how the community used to manage and work with cloud providers within the kubernetes code base.

B

They were actually statically linked into the binaries built for kubernetes, and so they were very tightly coupled both in the control plane and in the worker nodes. So now the new architecture which the community has done a great job, moving forward on and getting really close to the end game here, is to be running a cloud controller, loop, controller manager. If you will I'm in the control plane that handles the connection to the cloud and that's that's the end game, we have gotten there, especially in 120 we've.

B

um The community has completed a lot of the extraction and migration work, there's still more a few more things to go. But um a lot of groundwork has been laid for that, so the architecture here in the diagram showed comes from the kubernetes documentation um detailing where the cloud controller manager usually sets within a cluster similar to the kube controller manager. But it is responsible for cloud loops um and delivering those reconciliation loops within kubernetes to deliver your you know, load, balancers and other main pieces of the of the cloud.

B

Some cloud providers can run their cloud controller managers. They don't have to be in the control plane. They can actually run the data plane within the workers as well like in a demon set. That's also seen as well all right next slide, please.

B

So if we dig under the covers a little further look at the cloud controller or the cloud provider, the interfaces, what do the uh those specifically mean um in the context of kubernetes? So the first main interface is load balancers, every cloud provider has you know some type of load, balancer service, they'll, provide and then different implementations of that different options for it right we're no different on ibm cloud.

B

We have a set of options here for load balancers that you can run whether it be on the classic infrastructure that we provide, which would be your network load, balancers version one and two they run in cluster and then, if you're on the vpc infrastructure, you have a vpc layer, 7 load, balancer, a vpc network load balancer, and we continue to work on providing you know, enhancements new new feature function for load balancers, so that's the main interface.

B

Then you've got another main interface here, the instances interface, which is also the nodes interface, if you will, along with that, is the zones interface. So these these are really important pieces needed to bring up your cluster. Of course, you got the load. Balancers you've got the nodes on the infrastructure that you need to manage, um so the instances um has two versions of the interface instances: v1 and e2 v1, the v1 interface.

B

If you will is like uh the interface of the old architecture, it works on the new one, but they wanted to enhance it. So I believe in 119 they started working on a new v2 interface. I think it's beta now on 120., um so we're going to look at pulling in and using the new interface a little bit more efficient for the new architecture, the cloud provider. So that's more of an implementation detail, but nonetheless you can see the the moves forward in the community on improving the uh interaction with the cloud.

B

So that's the nodes, interface and then zones is similar, um but from our standpoint on ibm cloud, we rely on node bootstrapping process to help set things up to implement this interface. Another interface is the clusters. We don't implement that and then lastly, we have routes again not implemented for our cloud provider, because we rely on calico to provide the routing necessary, but um now these these interfaces all being available have their own controller loops um to go along with them.

B

uh The nice thing, with the new design from the community is that we're able to now turn on and off interfaces that are interesting to us, um for our I shouldn't say: interfaces turn on off the um control loops that are of interest. So, if we're not using a control loop, we can shut it off um now. In the new version 120 from kubernetes, so we'll go to the next slide and we'll look at some of the um enhancements and activities that we've been working on for the cloud provider.

B

So some of the big things here in 120, which is really awesome, is that the community has made great strides forward again on on extracting uh the cloud specific code from kubernetes, so the cloud provider code, if you will to make it more cloud agnostic, so that you can have some base controllers there, that folks can leverage, and then we have a little set of build tests.

B

Examples to build cloud providers, a release process guide that helps cloud providers, have some consistency across that not necessarily a binding requirement, but at least a good um process going forward, and so these have all been been very beneficial allowing us to instead of what we always had to do is vendor or have kubernetes the core kubernetes be a dependency of our cloud provider.

B

No longer has to be the case anymore, so we can build a cloud provider much smaller, less dependencies, less conflicts. If you will that can go into that. All the you know the large set of vendors that kubernetes has today, so that's a real benefit of 120 that we were able to take advantage of a much cleaner, build process. We were able to move to gold modules for dependency management, also a nice move forward um and then we're looking towards the future.

B

Here, like I mentioned, we're looking at um v2 interfaces implementing that for instances um looking at open source for the cloud provider, it definitely makes things a lot easier, as the community has made the extraction of migration uh much smoother, um improving our documentation and as always, adding more feature function.

B

Of course, in the area of load, balancers is one of the one of the big ones, all right all right next slide, please so that that kind of takes us through the um the ecosystem of kubernetes um and how we leverage it on our you know in ibm cloud space and cloud provider and how it's used within that, and now, let's, let's talk a little bit more about kubernetes and openshift, and how they're, similar and different so from our standpoint, running openshift on ibm cloud is like running kubernetes on ibm cloud.

B

A lot of similarities, they're both certified um and from red hat's perspective. It talks about their what their documentation red hat openshift is kubernetes distribution, so it contains additional features beyond kubernetes, um and this is not uncommon. There's many many providers out there that have a distribution or managed service. You take kubernetes open source and you provide a little bit more value added to that, and so this is no different.

B

You have a few a few interfaces that you work with on kubernetes and openshift that are very similar, but yet have a few differences. Oc is the cli for open shift and cube. Control is a cli for kubernetes as you're well aware, very similar, you can effectively run very similar commands and then, in fact, just replace cube, control with oc and usually the same command. Work is right. um After that there's ui for openshift console administration operations and kubernetes. The community provides a dashboard.

B

You know your pro. Your distribution or your managed service may or may not provide it with you, but there is a community dashboard for kubernetes that you can do a lot of similar things there tooling for image, creation and deployment. One of the really nice things and I think, a lot of the ecosystem around kubernetes takes advantage of it's like cube's, a real good building block, but it's not the end solution in the end for your business. So a lot of times, people want to build.

B

On top of that, so give you day two operations, so openshift has some of that as well. Automated installs updates and so on. Catalog, a catalog of operators and applications provided by both red hat, but also the community at large.

B

So a lot of benefits there and then, if you go back to the earlier conversation, you're talking about the releases and this release, cadence of kubernetes changing that certainly has a broad range of impacts to openshift, but also all those operators, applications that are running on it, um on top of it to make sure that they can stay current with those releases coming out, and the last thing I wanted to touch on is security.

B

Obviously kubernetes and security is huge, and a lot of a lot of effort has been put on that in the community to securing kubernetes by default and addressing potential weaknesses and open shifts. No different. A lot of focus is on that as well. I'm keeping your images clean of cves and so on.

B

One of the key differences that most folks may be not aware of off off the bat is that pod security in particular, is a little bit different on openshift than it is kubernetes and, in particular, openshift provides security context constraints to help manage what pods can and can't run and what they can and can't do.

B

In kubernetes, you have pod security policies, they're very similar, but there's uh there's a lot of significant differences in how they're used and yeah how they're used and how they work and and to achieve the end goal, which is basically the that pod level, security and security of the entire cluster container level. So the thing with pod security policies being provided by kubernetes is that there's now active conversation about actually um deprecating pod security policies and looking at what's the next thing to replace it.

B

So an important conversation I think, for the entire community ecosystem around kubernetes is that that security aspect, so I'm looking forward to seeing where that goes, because it certainly will have a broad range of impacts to our our community, both in kubernetes and openshift space and the applications running. On top of that so that'll, be. I look forward to that in the coming years to talk about um you know this.

B

If folks have interest in discussing these topics is a very good topic for our project group uh with that we'll jump over to the next slide, which I think takes us to the end and a thank you to all for attending and wrapping it up from here. So back to you.

A

um All right, hey teacher, that was great. Thank you um thanks. Everyone again, uh if you have any questions, please reach out to us. Otherwise, we'll we'll see you during the uh playing of this recording at the cubecon, and you know we will take questions uh at that time. uh Well, thanks again, thank you.

A