Cloud Native Computing Foundation KubeCon + CloudNativeCon Europe 2022, 1 Jun 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Keynote: 7 Years of Running Kubernetes for Mercedes-Benz - Jens Erat, Peter Mueller, Sabine Wolz

Description

Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.

Keynote: 7 Years of Running Kubernetes for Mercedes-Benz - Jens Erat, DevOps Engineer; Peter Mueller, Lead Expert; Sabine Wolz, Product Owner, Mercedes-Benz Tech Innovation

Years ago, software engineers faced hard times at Mercedes-Benz: spreadsheet operations, manual processes, grown infrastructure and strict governance. A grassroots initiative of engineers accepted the challenge to change the game – and their silver bullet was Kubernetes. Join us on our journey from introducing Kubernetes 0.9 on managed servers to an on-premises self-service cloud platform with close to 1000 clusters on Cluster API. You will learn about our stake transforming a data center with a young team that mostly did not know enterprise processes before. We describe how mixing naive visions and a strong believe in open source with lots of resilience made the project a success.

A

Hi and a very, very warm welcome here at kubecon in valencia. It is overwhelming to be here after this very long time of only virtual events due to covet. It is so great to be here in person and to actually meet people before we start. I would like to briefly introduce ourselves, so please welcome my dear colleagues peter one of our lead experts.

A

And jens one of our excellent devops engineers.

A

My name is sabine and I'm a product manager at mercedes-benz tech, innovation.

A

A

So if you hear mercedes benz- and you see the mercedes-benz star, you might think about things such as luxury, lifestyle, great quality and, of course, cars, and still we are not at any kind of car conference here. So you might be wondering what is a car manufacturer doing on main stage at cubecon now the answer is as mercedes-benz, tech, innovation, we don't build cars, we create software and thus play a decisive role in shaping the digital future of mobility.

A

We are a subsidiary company to mercedes-benz and will be formerly known as daimler tss. So just in case, you want to check for our commit history.

A

Our headquarter is based in ulm now who of you has ever heard of ulm? Oh, oh, okay! That's more than I have expected.

A

I thought hardly anyone, but I'm pretty sure that most of you know stuttgart, which is famous again for cars for swabians and for moultos and the yummy answer to ravioli, and I'm very sure you all know munich, which is famous for great beer, lederhosen and, of course the oktoberfest.

A

So if you take the german autobahn from stuttgart to munich, you will drive by ulm and although um is not that famous and not that big, it is hosting something really great. It is hometown of the currently highest church tower of the world.

A

Now. Why am I telling you this? Because one message I would like to bring to you today is you don't have to be very big and you don't have to be very famous to be able to host something that is really great and worth to talk about, and this is the case for us.

A

So last autumn we had priyanka visiting us and we told her our story about the last seven years and us and kubernetes, and I have to say she was as impressed about us as we were about her and this whole cncf, and therefore we have decided to share our story with you today as well.

A

We will let you know from where that we start. What did we do and what do we have now? I would say peter. Let the story begin.

B

Thank you sabine, so.

B

So why did we build our own cloud native ecosystem? Why? Why did we build our own cloud native olmenster to understand that we have to take a look back to the life of a software engineer at mercedes-benz in let's say, 2014 and yeah? It was quite different. It was a kind of different age of id and we were caught in a classical enterprise id.

B

Not everything was bad, but you know it. It was built with a different mission: it it was optimized for for cost and not for speed. It was optimized for governance and not for developer experience or innovation.

B

I will give you a few examples.

B

So ordering of a vm or firewall clearance often took weeks, we had a very narrow corridor of software tools, frameworks middleware that we were allowed to use and most of it was closed, source and open source was forbidden if not explicitly allowed, and maybe the worst death and ops were strictly separated.

B

Deployments often were done manually based on a description and change tickets and and that really caused a lot of pain. I can tell you and yeah it led to a situation where, on the one hand, our death teams they yeah, they tried to be as agile as possible.

B

They they had pressure from the business to the level features and uh yeah they built their their increments, their product increments, but then they often failed to deploy them to production, and that was the setting where some guys from one of our competence groups and um they said together to make our life a bit better again, you joined them pretty early. How did we build our old moving stone.

C

So every building needs a strong foundation. Ours was a young team of developers. We wanted to modernize the world of I.t operations in our company, starting with a greenfield approach.

C

We did not care about the enterprise way of running I.t in 2015. We analyzed the options for modern microsource orchestration and shows the new cool kit on the block huber needs soon after we started with a single shared cluster, running kubernetes 0.9 already in production and for the deploy for the applications we hosted deployment. Velocity was yet unheard of in the company.

C

If you visited the mercedes-benz car configurator website like five or six years ago, parts of which have already been hosted on cuba needs and deployed with a cicd pipeline all this in times when even startups they talked about docker and the keynote audience in here was like the very first five or six rows.

C

During the first days, we soon realized the single shared cluster will not fit our own needs, so we compared the alternatives for kubernetes fleet management, but there was no vendor distribution out there at the time fitting our requirements.

C

On the other hand, we had engineers with in-depth knowledge.

C

We understood the tech and decided to create our own solution instead and cloud management system.

C

We built it based upon 100, free and open source software, um both developed and operated by the same devops team and soon realized the benefits something's broken.

C

Instead of worrying about licensing issues and sending in support requests, we enjoy the four freedoms of open source. Not only are we able to run the code, but we also deeply dig into reading it and modding it modifying it. If we see need- and of course, we shared back our improvements.

C

During the first days, we still lacked the formal contribution process in our company, so we had to send in patches during our spare time with private mail addresses, but eventually this was resolved.

C

In the year 2021 mercedes-benz, the car manufacturer, ranked on the top 20 companies worldwide, contributing to kubernetes.

C

Remember we still contributed as timely tss back then, if you check the commit history today we are part of the cluster api provider, openstack maintainer team and right now donate cluster api state metrics to the community.

C

A customer once asked me now the application is completed. Who will I send this docker image to so you can run it and he was pretty puzzled. When I replied back well, you build it. You run it there's an api for that you cannot, you will have to do it on your own.

C

This was uncommon in a world of ticket operations. Instead we went automation first, even automation, only as the platform grew and with us the number of applications we hosted.

C

The number of changes just started to pressure surrounding I.t processors still stuck in those enterprise days, driven by excel sheets handled with manual tickets, but the cloud approach spread among the I.t in the company from getting a thyroid clearance to internet exposure from managed databases to our company internal, let's encrypt service for retrieving certificates.

C

All of this reduced lead times in deploying a new application from month to days or even minutes.

C

As the ulmer minster, even after completion always seems an everlasting construction site for renovations, we are too constantly renewing our setup reinventing core technology following the newest upstream development.

C

By joining in community efforts like cluster api management or for kubernetes fleets, we are able to add amazing new features all the time, while reducing the amount of internal code and improving quality right now we roll out kubernetes 1.23, always balancing out enterprise requirements for stability and developers needs for breeding edge technology and, of course, we support our app teams.

C

All the way on backway to cloud native to keep up with the pace, if you're interested in our experiences in not only renewing our infrastructure, all the time and following the newest upstream development, but also driving upstream, be sure to join. My colleagues, tobias and sean's talk on thursday afternoon in the operations track so peter. Where are we now.

B

B

At the moment, we're running more than 900 kubernetes clusters worldwide, 24x7 all on premises, but we're moving to public clouds at the moment and yeah cluster api is a perfect match for doing that.

B

We do have five platform teams which are steadily growing and yeah. We started with kubernetes as a service, but we see we soon realized that this is not enough. So we had to reduce the cognitive load of our devops teams, so we added logging and monitoring as a service as well as a database as a service.

B

We do have a team which is offering a container security service and we do have teams which are doing golden passes. That means a kind of pre-configured solution for mercy to spend specific challenges like the integration of our identity and access management systems, and so now, there's just a hem chart for doing that.

B

Most of what we are doing is open source or inner source. Inner source is a kind of yeah open source within the scope of a company, and we are supporting our platforms in a community approach. That means, instead of this first level, secret level, tickets, stuff we're just doing yeah.

B

We have select-like communities channels where um the other application development teams, they can directly contact our platform teams and they can solve problems if there are- and it's it's much more- we're also helping our our dev teams in there with kubernetes specific topics or cloud native specific topics, and we also can see that application teams are helping each other, which is pretty cool, and I think just this really is one of the success factors of our platform.

B

um But we've we've reached much more than that. So we were part of the movement in the in our whole company from this classic enterprise iit. I showed you towards in the cloud native mindset and it's pretty cool to see that now you build it, you run it and death.

B

Ops are common practices for us for years and, of course, our kubernetes clusters are yeah provisioned on the fly and- and we do have apis for fireball experiences yeah, but there's even one more thing to add some guys from the from this first yeah from this first movement or from the first team they moved on, and they did lobbying for open source within the company, and this led to our mercedes-benz foss manifesto, which I think is pretty cool and we're getting pretty good feedback on it.

B

It basically states a prefer open source strategy so from open source not allowed to prefer open source pretty good, and it also has a clear commitment, also from our management from our cio, to uh yeah, contribute to open source and to play an active role in open source communities. I think it's pretty cool it's on the web. You can read it if you want to, you can also use it. It's under a creative commons license so feel free to check it out and yeah use it. If you want to sabine what else did we learn.

A

To sum our story up, we have learned a lot of the last last seven years. One of our main success factors was, we simply started. We were brave, we took heart, we started small and thought big and that worked really well for us.

A

From our end user perspective, we can say you will learn so much from using open source technology and the cncf landscape is a great guide to let you know what is out there and what is good to use as an end user. You are a valuable part of the con community and you can contribute in many different ways, and we would like to encourage you to actually do so and last kubernetes still is hard.

A

Therefore, don't leave your devops software and developer teams on their own support them on their journey to cloud native talking about journey, if you are ever close to ulm, just make sure to stop and visit the ulmer munster, and, of course, if you want to you, can also visit us at one of our fast growing locations. We would love to invite you and have you there.

A

We will be around at kubecon for the next couple of days, together with some of our colleagues just look out for the t-shirts, and we would love to talk to you and get in touch with you exchange knowledge exchange experience.

A

So if you want to feel free to talk to us, thank you very much enjoy kubecon and talk to you later.