Cloud Native Computing Foundation Kubernetes Batch + HPC Day EU 2022, 19 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Keynote: High Performance Computing on Google Kubernetes Engine - Maciek Różacki, Google Cloud

Description

Keynote: High Performance Computing on Google Kubernetes Engine - Maciek Różacki, Google Cloud

Google Kubernetes Engine is already a platform of choice for highly demanding high-performance computing workloads. We will present how we're investing into pushing the capabilities of our product further to maximize users' scientific output with ease, cost efficiency and industry leading performance.

A

Hey my name is michael kozke, I'm a product manager working on gke, so indeed my role is more oriented internally on how how, on the offer that we provide to to users with google kubernetes engine, I'm I before I start actually, I want to say, like I'm, really surprised like how many people we have like. I remember like three months ago when we were discussing with the guys here and with our colleagues at cncf. If you should organize this event for the first time, I think it's worth to recognize.

A

It's really the first first collocated event on on batch and hpc we're concerned, if we'll have actually a room, we're targeting around 50 people, which, like kind of looks like guesstimating around that that that group so really cool, to see all of you here, like a quick show, maybe of hands like who, among you like is a kubernetes user. uh It looks like one third, maybe almost half and like who, among you, is a contributor to core kubernetes or projects that run on it.

A

ah That's actually interesting, um interesting and then who's like a vendor. That's providing products uh that uses kubernetes, yeah, okay, cool who didn't raise their hands. Then, if we okay, we need to talk them later, but cool. uh That's uh that's! That's! That's um yeah! That sounds so really excited to see. You here like in this short, like really five minute sponsor keynote. um I would really. I would like to show you just in like five minutes.

A

um What's our thinking process in terms of how do we embed kubernetes in the overall offering of google cloud so that we help our users and other companies in the ecosystem to either consume services or provide them to to end users and what I would love as an objective here I mean like, as ricardo said, the objective of the whole event is really to like build a community around the topic of hpc and batch processing in kubernetes. So I'd love.

A

If, after the session during breaks, we could like spend some time together, discuss like if you had any feedback or or maybe set up some some meetings later to to maybe dive deep into some of the thoughts that you might have on on how we're approaching the the problem space.

A

So with that then, um like- I probably, I don't need to explain to anyone in this group, especially that is so dominated by users, that the importance of high performance, computing and data processing in general, for all of the scientific workloads that researchers are doing is only growing in the modern world. We face more and more challenges that require uh robust, um robust it infrastructure for for scientific workloads and data, processing and kubernetes comes with a very important offer in this context.

A

That is, it comes with that capability of offering an open api that abstracts a lot of the infrastructure and the differences between on-premises and various cloud providers, so that batch, admins or platform admins are able to offer provider agnostic, infrastruc primitives, to consume by researchers. But at the same time, um kubernetes is low level enough in its abstraction layers.

A

Is that it is possible to offer that platform um without making sacrifices or major sacrifices in things like performance efficiency that are critical for many high performance computing workloads and as we think about what we are trying to offer in the google kubernetes engine and gcp in general, when it comes to kubernetes color colors could be a little bit better, maybe so for this.

A

But um here uh so we we are dividing the capabilities that we're uh offering to you and we we have uh shipped or we're working on on launching uh into three main domains. One is job management or you could convert it into like ease of use.

A

So there is a lot of investment that also comes close from us into the open source community and how kubernetes orchestrate jobs, uh how queuing operators and other capabilities in here, but it also comes like integration of the ecosystem, but it also comes with some capabilities associated with uh how just manage clusters, like the the ease of upgrades in creating clusters, especially in this domain like female clusters.

A

Other domain is like performance, and here it's both improvements into the open source stack in terms of its efficiency.

A

In how it operates, and also launching a variety of capabilities in the infrastructure or other parts of the ecosystem, like gpus, new cpu, architectures, better integration of kubernetes and our google stack with with the underlying hardware and, last but not least, its cost efficiency and all of the capabilities associated with the fact that high performance computing requires massive amounts of resources that are expensive but then, which makes it a very sensitive topic to to be able to fit into the budget and financial constraints of workloads.

A

I'll. Just flash this to visualize like what was the level of our commitment to the high performance computing capabilities. This just flags some of the both hardware and software capabilities that we have been launching and we have on the roadmap.

A

They both cover in, like not only hardware and not not only stuff related to kubernetes and consumption, but we're also supporting those users who would prefer to use some alternatives when you think about bernadette's engine itself, I'll just highlight like the most recent uh important uh capabilities or the most notable ones that we see among the hpc community. So definitely the fact that uh we commit to to supporting clusters of a size like three times, fifteen thousand notes so three times larger than what is being tested by the open source community.

A

uh We provide um our auto scaling. uh Algorithms and capabilities are like custom and internally built to offer the best integration of our infrastructure, and they all are built in a way that they optimize the function of the cost for the customer as the primary primary objective of what they do.

A

Ease of use, upgrades ease of creation of clusters, then performance capabilities, integration of hardware, etc, gpu, slicing or orchestration of spot vms. So, like there's more to come, uh if you want to chat and share your thoughts on like where we think we should go, etc, I'll be very keen to talk, and now I will pass to aldo and the next presenter one more one more, maybe just on these cards, you can find a url and you can find it also here on the bottom.

A

It will take you that we have some small present for you. uh We didn't know how many people will show up online versus in person. So we as a swag form of sponsored swag, we are offering training credits that you can.

A

You can redeem by logging to that link in the middle or the one that you have on these cards that are on your table and with that.