Cloud Native Computing Foundation Online Programs, 18 Aug 2023

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Multi-Arch Cloud Infrastructure: Why and How to Get Started

Description

Multi-architecture infrastructures enable workloads to run on the best hardware for the task (Arm or x86) and optimize price/performance ratios while boosting design flexibility. However, migrating from a single- to a multi-arch framework can be tricky.

Join Cheryl Hung, Senior Ecosystem Director at Arm to learn about:
- What is multi-architecture infrastructure?
- The trend for moving cloud workloads to multi-arch
- How early adopters handled the challenges
- The growing software ecosystem of Arm
- Comparing the price-performance of common workloads on x86 vs Arm
- Resources to get you started

This talk prepares developers for the road ahead with the ability to run workloads on the best hardware without needing to be concerned with the underlying architecture.

About the speaker:

Cheryl brings engineers together to build the future of infrastructure, especially cloud native and open source. As Senior Director at Arm, Cheryl leads ecosystem strategy to drive adoption across cloud, 5G and networking. She also founded the Cloud Native London meetup and runs events for 7000 architects, SREs and DevOps engineers monthly. Previously Cheryl was VP Ecosystem at the Cloud Native Computing Foundation / Linux Foundation, bringing Spotify, Apple and Airbnb into the End User Community. She was a C++ engineer on Google Maps and and engineering manager at Apple, and holds a Masters Computer Science from the University of Cambridge.

Find more resources to plan your migration and get started:
Whitepaper: The Insider’s Guide to Building a Multi-Arch Infrastructure - https://armkeil.blob.core.windows.net/developer/Files/pdf/white-paper/arm-whitepaper-insiders-guide-to-building-a-multi-arch-infrastructure.pdf

Hands-on tutorials to create Arm-based Kubernetes clusters
More multi-arch cloud resources: https://www.arm.com/campaigns/multi-arch-cloud-infrastructure

○○○ Further Arm Software Developer Resources ○○○
– Twitter: https://twitter.com/ArmSoftwareDev
- Reddit: https://www.reddit.com/r/ArmSoftwareDev/
- GitHub: https://github.com/arm-software
- Discord: https://discord.gg/armsoftwaredev
- Arm Developer Program: https://www.arm.com/resources/developer-program
- Arm Developer Hub: https://www.arm.com/developer-hub/servers-and-cloud-computing

A

Very delighted to be joined here today with um Cheryl who's, our senior director here at arm leading our ecosystem strategy driving adoption across Cloud, 5G and networking fields and.

B

A

Sure we'll be talking about multi-arch sport today, um so yeah Cheryl without further Ado. um Take it away.

B

All right, thank you. Thank you! So much Josh and hi everyone. It's really nice to be here. Thank you for joining today. So my name is Cheryl and um as just Josh Josh said, I'm a senior director at arm and I have some slides that I'm going to talk through today, so.

B

Okay, let's see if we can get these slides going.

B

Okay, fantastic so multi-arch Cloud infrastructure from the ground up we're going to talk about why you should consider it and some ways to get started.

B

This presentation is going to be targeted at mostly sort of senior technical people, so Architects engineering managers, devops engineers and platform engineers and some application engineering Engineers as well we're going to look at what is multi-arch cloud infrastructure.

B

Why are people moving towards it and look at some use cases and benchmarks that we've done for web servers, nosql data stores and machine learning we're going to look at some real life experiences from Fusion off and honeycomb and hear about the challenges they faced, how they got through them and what the results were. They saw of migrating from x86 to arm and then I'm going to give you some hints and tricks for how to get started.

B

So we'll look at a three-step approach towards a smooth migration towards multi Arch and then just look at some example deployments briefly, we'll look at deploying a basic WordPress and my support on AKs on Azure kubernetes service, and we will look at using GitHub actions with arm-based burstable VMS on azure.

B

A little bit about me um as I said: I am senior director of ecosystem at arm um in the infrastructure line of business, so that is cloud Telco and networking, and my mission is to get um to cross the chasm so crossing. The chasm is this term from probably 15 20 years ago now this book called crossing the chasm which.

A

B

All about early technology adoption, so in in this crossing the chasm terminology, you have the early markets, you have the innovators and the early adopters, and then you have the mainstream market, who are the early and late majority and between the two of them you have a big Chasm. This is usually where a lot of new technology fails to get adopted, because the early Market segment, companies and individuals are usually willing to put a lot of work in to make changes.

B

But if you want to hit the mainstream markets, then you need to go from just having these minimum feature sets to a whole product solution, so you need to make it easy and convenient to adopt and you need Mass adoption and math support.

B

So my mission, as I said, is to get arm across this Chasm to go from where we are in Cloud today, which is pretty early Market towards this whole uh whole mainstream market.

B

So I and my team work on a bunch of different things. So we work on software supports including open source. We work on making sure the hardware is easily available widely available.

B

We work with various standards, bodies and different foundations and make sure that arm and arms users are appropriately represented and we do developer Outreach. So things like this, where we talk to directly to Developers and tell them what is the latest and greatest ly?

B

I was an engineer and engineering manager at Google and apple I was a VP at the cloud native Computing Foundation I've also been very involved with Community for a long time, so I run a meet up with about with over 7 000 members, which is around Cloud native infrastructure and Technologies in London.

B

So, let's get into multi-arch Cloud infrastructure.

B

And let's look at some context, so Cloud spending and energy pressure energy usage are both under pressure. On the left hand, side this growth, this chart shows you the growth in revenue between cloud and traditional data. Centers and, as you can see, cloud is not only going, but it's still projected to keep growing for the next few years, whereas traditional data centers have kind of um flatlined more or less so.

B

Companies who are moving to cloud or using more Cloud need to make sure they're using those resources efficiently, because the spend the spending is continuing to go up and what we call maximize price performance. So for the amount of money for each dollar that you're spending you want to get the maximum performance, but back out of it, and then this chart on the right hand, side shows the internet traffic and the data center usage, so data center energy usage compared to 2010, so starting from 2010, you can see.

B

Internet traffic has gone up 17 times, but data center energy usage has only gone up, 0.1 or 10 during that time, and that's really really important. Come not companies, um Cloud providers and companies have put in a lot a lot of work to make sure that the energy usage doesn't follow this top line, but instead it stays mostly flat and in fact, there's now a EU goal of climate neutral data centers by 2030..

B

This internet traffic is probably going to go, continue to go up and up and up there's ever more streaming ever more things like cryptocurrency, machine learning, very compute, intensive workloads and while the hype scalers are put in a lot of work to in terms of better software and better hardware and better coding, Technologies and so on, um we need to all find ways to make sure that we are using this energy as efficiently as we can as wisely as we can.

B

So into multi-archy infrastructure, so computer architectures are either risk, which means reducing instruction set computers or cisc, which is the complex instruction, say: computers, um arms risk architecture, targets, power, efficiency, performance and you can license it for many use cases.

B

So, for many years arm, trips have been widely available in mobile devices because when you have a mobile phone- and you want to you- know- maximize the battery in your mobile phone and it's also starting to come up in automotive, internet of things and Cloud. Now, because of that those data center NG usage, we just saw that power, efficiency and performance is becoming really important for the large Cloud providers.

B

The goal of multi-arged infrastructure is to run workloads on the best hardware, for their price performance needs without developers needing to be concerned with underlying architecture, so whether it needs to run runs best on risk or cisc-based architectures shouldn't matter to the developers. All they need to know is, it will run on the best hardware, for the price performance needs.

B

What kinds of workloads can benefit from a multi-arch approach? I've taken these from the arm, Community infrastructure, Solutions blog, where we look at different workloads and try and Benchmark them as fairly as possible. Benchmarking is a little bit of an art, but you know we'll take on the same cloud provider, x86 versus Arm instances, we'll use industry standard benchmarks and we'll use vanilla installations of things of Open Source software to test on so for web servers.

B

We looked at nginx performance and found you can improve the performance by up to 32 percent for distributed memory cache we looked at memcached and found a performance Improvement of up to 41 percent and then really impressively on machine learning. We found you could increase the performance by 1.8 times, 80 better for deep learning influence workloads.

B

There are many many more examples on this solutions: blog, so I highly recommend you go and check them out. But, generally you know performance in place, go hand in hand right. You know. If you get better performance out something you can pay less for it. That's what it boils down to.

B

And in general, what kinds of workloads can benefit from multi-arch there's broad support now for arm across almost all of the top used open source projects. Arm based instances are available on every major public cloud provider.

B

So AWS is your Google Cloud Oracle, also a lot of the Chinese ones, so Baidu Cloud Alibaba, Cloud testing, Cloud JD cloud and the team's put in a lot of work to make it really really easy to recompile your software for arm, sometimes just a few tweaks needed, but on the whole we want to make it as easy as possible for you. So we're going to look at two real-life case studies, one from fusionor from one from honeycomb and we'll look at their experiences, what challenges they faced and what results they saw.

B

The first one Fusion auth is a developer, focused API, first auth provider, Deployable anywhere.

B

um This is a sort of smallish startup and pretty early on in 2020, one of their community members just reached out on the forums and said hey I'm, looking at running Fusion North on a Raspberry Pi, because he was a Raspberry Pi enthusiast.

B

So that's the first time this considered running Fusion North on R. About a year later, after graviton AWS graviton came out, they started load testing six months after that it was officially supported on the downloadable version of fusion North and it was rolled out to the SAS platform in June and within a year more than 70 percent of those SAS instances ran on arm.

B

Here's some more details about their technical timeline, so the first step that they need to do was to find the right jvm to support arm, especially on Mac, because they were using Macs and they found that Java 17 was the correct one for them to do so. They took three months to add Java, 17 and arm support into the code base.

B

Then they had to update and test the install scripts and update Docker to support these multi-arch builds back in 2021. Armed support was not available in every public Cloud region, so they had to make sure that on support was there for the for the SAS regions. They were using the region so using for their SAS instances and then last but not least, they updated the application to actually use the correct arm architecture in terms of performance testing.

B

They know they knew that their logins were particularly CPU intensive because they had to do password hashing. So the team load tested 50 000 logins. They found arm handled between 26 and 49, more logins per second and cost eight to ten percent. Less than Intel on ec2.

B

And here's some quotes from the the CTO from the team, the user, so the CTO said because we run on Java, our lift was pretty small. We just had to find a jvm built for arm and then work out any remaining Kinks.

B

This is a quote from the user who said: I just switched a fusion North instance to arm 64, and the transition was so smooth. I couldn't even tell whether it's actually running the armed 64 version, so very smooth, very straightforward on both sides.

B

For the Second Use case study we're going to look at honeycomb.io, so honeycomb provides four stack, observability enabling Engineers to deeply understand and debug their production software, so they started experimenting with graviton 2 in March, 2020 about nine months, sorry, one year later than that, um their first workloads were in production.

B

Those were the ingest workers six months after that, pretty much all of their workloads and environments were on arm and 92 percent of vcpus were on arm and then another six months after that they turned off the last of the x86 uc2 instances, and now they run 99 of their workloads on arm on Lander.

B

So, let's look at the technical timeline from honeycomb, so honeycomb chose to migrate, those in just workers first, because they were stateless, the performance, critical and they scale out horizontally and they're. Also written in golang, so compiling for arm was pretty easy. They were just deployed in the dog food environment and saw positive results.

B

um Initially the honeycomb didn't use kubernetes or container orchestration at all. um Now they do, but everything at the time was in telephone with Chef, so they could just switch to the arm Amis very, very easily.

B

The Next Step were their own workloads, which were the easiest to recompile and they spent the most on and Kafka, and then the last were these ad hoc, one-off services and those that were difficult to migrate.

B

So after a year of work said they turned off the last x86 instance and moved almost entirely to AWS Lambda on arm and what they found was they saved 40 of their ec2 instance bill from doing this migration over the course of a year again, here's some some quotes from them, so graviton has enabled honeycomb to scale up our compute without increased operational, toil, spend less on compute and have a smaller environmental footprint and then Liz who uh who's.

B

One of the engineers who was working on it said I personally approached it as an idol experiment with a few spare afternoons and was surprised at how compelling the results were. Saving 40 on the ec2 instance bill for this service is well worth the investment.

B

So uh hopefully, you've seen that there are some really good results coming out of these migrations for arm and in fact the chief evangelist at AWS has said publicly that 48 of the top 50 Amazon ec2 customers also use AWS graviton processors for their workloads.

B

So in many ways this is not uh there's an early majority thing anymore, we or early adopter. Actually, the early majority have already started moving over and started using arm in the cloud very very successfully.

B

So hopefully that gets you interested and now you want to know. Where do you get started.

B

So there is a lot to consider in a modern, Cloud native infrastructure. So here are just some of the things that you might need to look at. So you might need to consider. uh Is your? Do you properly use infrastructure as code? Do you have your ocicd, Pipelines and reproducible builds?

B

What about packaging binaries the images registries when you actually get ready to test and roll out so testing scheduling, roll out performance testing and then, when you need to upgrade your infrastructure? How is that going to work, and this diagram shows what you might consider a modern Cloud native infrastructure, so kubernetes at the bottom, a bunch of different microservices who each have their own different um responsibilities, backed by different data stores at the top you serve it through nginx. You manage to manage all these microservices with something like Envoy.

B

You also have a test environment, so you have CI CD with various tools and maybe get up to tours, and then you need to monitor so you have various.

A

B

Tools running as well, so there was a lot a lot going on. So the question is: where should you get started? Thank you.

B

There's three major steps that you can consider when you're migrating towards a multi-arch infrastructure, so I've called these ones in form, optimize and operate. So we'll start with inform Zone form means. Do you understand your software stack really? Well, so you need to make an inventory of everything that you are running. The operating system, container images, libraries Frameworks once times any tools that you use to build, deploy test, any tools that you use to Monitor and manage, and then you should identify the hot spots.

B

So what are the workloads that are costing you the most money and check each of those for arm support and slight distinction here? So GCC calls this ah 64 and then it's kind of caused this arm 64..

B

So from that first step, you should now feel like you know, which are the first workloads to take on. Secondly, you is called optimize, so you need to provision a test environment. So it's based on arm 64.

B

upgrade each of those contained images and test make sure that the um it's working as intended and do some performance testing as well make sure that you are hopefully getting a better performance out of it.

B

For example, there might be various flags that you might need to change between x86 and ARM version, so you might need to add if statements, depending on what architecture you're running on and then, if you are running kubernetes, then you need to think about how you're going to change your kubernetes cluster so, for example, you're going to have the control plane or the work plane, nodes on x86 and so control plane on an xa6 worker nodes on arm or you're, going to have the other way around or you're going to have a mix of them.

B

Each of those have different pros and cons.

B

You probably have some scripts about how you create a cluster that need updating and then you'll. You may have some Demon sets so things that run on every node and you need to make sure that those are selecting the correct for the correct architecture. The correct applications for that architecture and then, lastly, the operate so update. The CI CD make sure that your arm 64 builds, are reproducible and then you can start deploying them using a canary or blue green deployment approach.

B

So in terms of controlling how your, uh how your applications roll out, then different approaches you might need to take include node affinity, taints and tolerations, and then lastly, your different architectures might have different limits and requests on them, so that will you'll become aware of those through monitoring and how you have to change those so very, very broadly.

B

This is the approach that I would suggest to you so inform understand what your your software stack looks like optimize. Take each image at a time and provision them into a test environment and update your cluster and then do the actual roll out and just monitor the results.

B

So we're going to look at a couple of really really simple um deployments here, because it's very difficult to give definitive advice, because everybody runs infrastructure in quite different ways. So I'm just going to look at some very, very simple, straightforward deployments and see what are the kind of changes that you might need to make.

B

So the first deployment here is going to be deploying WordPress on top of kubernetes um WordPress with my MySQL and we're going to deploy it on arm-based instances to AKs as your kubernetes service.

B

So the way to do this is you assuming that you you're using telephone to create your clusters? You use a different kind of VM.

B

You set this VM size to deploy on-based VMS, and then we have some scripts for you, which just are slight variations of the basic WordPress bicycle Scripts and you just run those against the kubernetes cluster and that's basically it so. As long as the cluster is set up to run on top of arm-based VMS, then it should be very straightforward. You can run kubernetes WordPress and WordPress my SQL on top of all of uh on top of those arm-based VMS. Without too many changes, basically with just this one.

B

And we're going to look at a second simple deployment, this one's really hot off the press. So this one is, if you're looking to save some resources, save some spending on CI CD, because in general cicd is very, it's very busty. You know it might sit idle for a long time and then you need to have a little Spike where you actually use the use, those resources, and then it will sit idle, so obviously you're spending money, even if it's idle, but a different way to approach.

B

This is to use burstable VMS and those burstable VMS will accrue credits over time that it's not being used and then, when you actually need to use them, you can use up those credits, so saves a little bit of money and then these are also integrated really nicely with GitHub actions.

B

So you can just when you create your instance, you can create it as a on 64 architecture and you can choose the correct size that you want and when you create your GitHub learner again, you just choose your arm 64.

B

architecture here and then you sit back and you look at some dashboards, and hopefully you see that you have some CPU credits because you're not using your cicd and then when you actually commit something to the cicd pipeline, you're, going to see a little best where you actually consume those saved credits. So that's quite nice.

B

And uh let's go to the summary, so Cloud spending and NG usage are under pressure, so companies need to maximize their price performance and use their spend and the energy as efficiently as they can.

B

The goal of Montage infrastructure is to run workloads on the best hardware, for your price performance needs without the developers having to be concerned with the underlying architecture and the real life case. Studies show, somewhere between 25 and 40, in savings 20 to 60 percent performance increase compared to x86.

B

Of course, this can vary very wildly depending on your setup and your workload, um but there is lots of evidence for this kind of level of savings.

B

Arm-Based instances are available on all major public Cloud providers and arms widely supported across the software ecosystem, both open source and commercially, and if you want to get started, then think about this inform optimize and operate model so find your hotspots and then manage those transitions smoothly and monitor, monitor the changes that you're making and then. Lastly, you can go and look at other examples from the arm: Community infrastructure, Solutions blog for tips and tricks, um other widely used open source projects, and- and hopefully you find some inspiration there.

B

And that is it for me. Thank you so much thank you for listening. Hopefully, uh we'll be around to answer any questions.