Kong Destination: Automation, 31 Aug 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: 8 Cloud Cost Optimization Tactics for Kubernetes

Description

In this session recording, Annie Talvasto goes over the eight ways you can reduce your Kubernetes cloud costs.

Read the blog version here: https://bit.ly/2UZt6pW

Check out all recordings from the Destination: Automation 2021 event: https://bit.ly/3AwzL9X

#Kubernetes #K8s #CloudOptimization

A

A

Everyone, to this session, super excited to have you all here.

A

The session is kubernetes cost optimization: how to actually do it.

A

So very much a welcome on my part. and thank you for thank you for kong for organizing such a great event.

A

But let's get straight to it: essentially.

A

So let's go to the next slide., so kubernetes optimization is the topic of the day.

A

So this, in here,, I'm going to just talk about expectation management,. What to expect.

A

From this session., so the session description,, if you've seen it,, is very comprehensive.

A

And that's what I aim to always deliver in conferences.

A

But I do have to say that this is a 20 minute, session.

A

obviously,, we're limited in time and in in the things that I can show. So we cannot show.

A

Everything., but that does not stop us from trying to be as comprehensive as possible.

A

So very, very happy to have you here.

A

So I will essentially be taking you through the basic principles of cost optimization,.

A

Then do a deep dive, ish type of thing into eight best practices, then give you open.

A

Source project suggestions that you can use to take care of cost optimization to the next.

A

Level and then present you with a one automation tool that you can use, automate your.

A

Cost optimization., so that's amazing. and let's get to it essentially., but yeah, it will be..

A

The session is rather cloud agnostic., so any cloud vendor you can use. Essentially these tips and tricks.

A

But we will definitely be focusing on cloud and communities today.

A

Yeah, so then to that,. I think one of the important things is to say,, who am I and why am.

A

I speaking here today? so hi everyone., I'm annie.. I'm a product marketing manager at cast ai,, which is a kubernetes cost.

A

Optimization company., so you can always reach out to me on linkedin,, on twitter,, on github and in any of the.

A

Places. and usually I'm there with the handle, annie talvasto and I can really easy.

A

Just first name, last name.

A

lovely, super excited to talk with you all at any time.

A

So I am a cognitive technology, marketer and kubernetes specialist.

A

I have been managing the kubernetes and cncf meetup finland for over three years, now,.

A

Almost four years at the moment., I am the co-host of the cloud gossip podcast that you can find at cloud: gossip dot, net.

A

So- and I also love speaking at meet ups, local meet ups, international conferences,, all.

A

Of these., so it is a passion of mine to be sharing technological things to everyone in.

A

These kind of ways. so then first,. Why does cost optimization matter?.

A

Why should it matter to an organization?? Why is it so important? and I think, on a personal and a company level,, everyone has probably been in the.

A

Situation where we have received only one has received a surprisingly large cloud. Bill.

A

And the surprise, and the kind of what happened that that comes with it.

A

So on that's personal and organizational level., it definitely matters quite a lot to be able to manage your cloud bill.

A

Then I think in the big picture,, why cost optimization matters nowadays,, particularly.

A

Because in the history it mattered,, but in a bit of a different way.

A

So, even to this day, over provisioning is norm rather than the exception.

A

Because I understand it, because, you know, back in the day,. People had to actually go out.

A

And buy physical hardware to make sure that their service is kept up and running.

A

So obviously, you could you have to overprovision to prepare for possible fix.

A

But that has all kind of changed with the cloud., so cloud makes flexible, flexible scaling possible and it has introduced so many new.

A

Services one can use,, which obviously increase our compute use from there.

A

So cost optimization, really has, I think, has become and will become even a bigger topic.

A

In the industry going forward, because it's hugely important.

A

And I think a good thing to point out as well. Is that usually, when companies start doing.

A

Cost optimization, they actually also start investing more into the cloud,, because when.

A

You do cost optimization, that opens up the possibilities. Of, you know,, investing more.

A

Into the services that you want to use, investing more into new team members,, finding new.

A

Engineers, all of these things., so cost optimization, doesn't necessarily mean less cloud..

A

It just means smarter way to use cloud.

A

So it is highly important then, to the next slide,. Why is the cost of the cost? Optimization.

A

Then so complex? we are in a situation where I think cost optimization should matter to everyone and it.

A

Should be a talking point., but then why is it so complex if everyone should be doing it?.

A

Here is a snapshot of aws pricing that I've taken,, and this is literally just a snapshot.

A

There's so many more instances, there's so many more info about it.

A

So it is very confusing., it is very hard to follow, is very hard to keep up your knowledge around it.

A

And even with particularly with the kubernetes space,, it's really difficult and.

A

Expensive., so there's so many questions that you have to answer from how many nodes do.

A

You need and how big they should be,, how many ports,, how many containers do you need,? How.

A

Big, they should be. so there's so many questions that need answers.

A

And obviously, you don't you can't only answer the questions one during deployment.

A

You have to continuously be ready to answer the questions and keep up with your knowledge.

A

And with your optimization., so no wonder that it's hard. it is very difficult to keep up with.

A

and I think in general,. The system is also rather built so that the cost optimization is.

A

Not the default., so over half of it is applications use less than 30 percent of the cpu resources that I.

A

Booked them for them, according to datadog., so it's a place where it's it's it's the norm to just overprovision and not to think.

A

About it kind of. and I think that's that's easy to leave it at that.

A

But but since you are here, you're interested in cost optimization.

A

and that's wonderful because it can really open great doors,, as mentioned in the last.

A

Slide. so then getting to the actual recommendations. So you can do so many things with cost.

A

Optimization., there's anything from policies to resources, heat maps,, rightsizing.

A

Reserved instances,, instances,, multi cloud, understand your bill or autoscaling.

A

Allocation review, goal setting so many.

A

I'm not going to even name everything on the slide, because it will take a bit of time on.

A

My session. so there's so many things you can do.

A

I'm going to focus on a few things. and first, before I focus on the eight practices that I've highlighted, today, first,.

A

Let's cover some base with the basic things I think.

A

So these all that I'm highlighted here are governance, things that you need to really.

A

Handle. you need to have a basic understanding of and have a basic thing,, because to do.

A

Cost optimization well,. You also have to do optimization,, ok, or well in your company.

A

But I will not be focusing on these governance aspects, because these depend largely on.

A

Your organization., so large companies might have full teams focused on forecasting only.

A

And a smaller companies do not have that luxury..

A

So I decided not to focus on this side or rather focus on things that everyone can.

A

Actually implement and utilize in their work.

A

But essentially,, I think one of the important things here that is usually often.

A

Overlooked is understand your cloud bill.

A

So, even though it is very confusing, there's so many mysterious codes,, it might look.

A

Completely different than the development side of it., so it's very hard to grasp., but I think understanding that really gets. You started with your journey into doing cost.

A

Optimization well., because when you understand your bill and you understand how it affects your environment,.

A

The new architecture,, then you can take it to the next level as well.

A

And all the other things obviously important as well.

A

Yeah, and I will provide our resources in the end where you can check out these a bit.

A

More in depth., but then going to the best practices,, the eight ones that I'm sharing today is the kick.

A

Off this session., so let's get to the first one,, which is define your requirements.

A

So, essentially, what it means that you should only order what your workload's needs across.

A

The compute dimensions from cpu account and architecture to memory, storage and network.

A

So, even if an affordable instance might look tempting,, but what happens if you start.

A

Running a memory, intensive application- and you end up with the performance issue.

A

Affecting your brand or customers,, it's not a good place to be at.

A

So you should pay attention when choosing between cpu and gpu instances for sure.

A

So the second one,, which is choose the right instance types,. So as mentioned here, aws,.

A

Offers more than one hundred fifty different instance types,, sorry to quite a lot,, so.

A

There's really no denying that cloud sources are often the biggest line item on your.

A

Bill., so picking the right type can go actually quite a long way. Already.

A

So cloud providers offer many different instance: types, matching a wide range of future.

A

Cases,, which, with differently combinations of cpu, memory, storage and networking capacity.

A

And every type comes in one or more sizes., so you can scale your resources, properly., but really, know this, cloud providers, roll out different computers for their vms and the.

A

Chips in those computers have different performance characteristics., so you might pick an instance type with a strong performance that you don't actually need.

A

And you might not even know about it., so it can be quite a lot with so many business types,. The best way to verify a few.

A

Capabilities is by benchmarking., so by dropping the same workload on every machine type and checking his performance from.

A

There. so then third, verify storage, transfer, limitations,, so data storage is definitely another.

A

Cost optimization aspect worth your time: spending on.

A

So each application out there comes with unique storage, needs.

A

So make sure that the vm that you pick the storage throughout your workflow needs.

A

So also avoid expensive drive options like premium ssd, unless you're planning to really.

A

Use them to the fullest., so far, check if your work is spot ready,, so spot instances are a really great way to.

A

Save on your kubernetes bill, are at anything.

A

And, but before jumping into this spot space immediately,, you need to really decide on on.

A

How you build your framework and how to use for instances then and how you implement.

A

So here I highlight a few questions that you should think about before you before you go.

A

With the spot instances., so how much time does your workload need to finish? The job?

A

is it mission and time critical?

A

can handle interruptions?? Is it tightly coupled between instance, nodes??

A

What tools are going to use to move your workload when a cloud provider pulls the plug?.

A

So these are just a few beginning questions and you need to start answering before.

A

Jumping into the spot, world.

A

so continuing on the same topic,, you should cherry pick your spot instances,. So when.

A

Choosing spot instances of the slightly less popular instance, because they are less.

A

Likely to get interrupted. so after you once you pick your instance, check its frequency of interruption there at the.

A

Rate with which the instance we claimed capacity during the trailing month.

A

So you can see the aws of instance, advisor ranges from there and the ranges are from.

A

Under 5 percent, 5 to 10, 10 to 15, 15 to 20 and then more than 20.

A

So don't shy away from using spot instances for more important work,, both as long as you.

A

Know what you're, doing. and aws offers the type of spot that gives you an uninterrupted.

A

Time guarantee for up to six hours in hourly increments, and you just pay a little more.

A

For it and still achieve a discount on the on demand, pricing as well.

A

So then, six a bid, your price on the spot.

A

So once you find the right instance,, it's then the good time to set the maximum price. You.

A

Are ready to pay for it. the instance will only run on the marketplace price below or.

A

Equal to your bid., so here's the rule of thumb.

A

Set the maximum price to the one that equals the on demand. Price.

A

Otherwise, you risk getting your workload interrupted once the price goes higher than what.

A

You have set. so to boost your chances of snatching spot instances,. You can set up groups.

A

Of spot instances called in aws, spots and request multiple instant types at the same.

A

Time., so you will be paying the maximum price per hour for the entire fleet rather than the.

A

Specific spot pool.

A

and it requires a lot of work,, but the cost optimization will definitely happen.

A

So it works, even though it takes a bit of time to manage it.

A

So number seven, use mixed instances,, so this is something that's specific to kubernetes,.

A

Only a mixed instanced strategy gets you a great availability and performance at a.

A

Reasonable cost., so you basically pick different instance types in.

A

Some of them are cheaper and just good enough,, but might not be suitable for high.

A

Throughput, low latency workload., so depending on your workloads,, you can often pick the cheapest machine and make it.

A

Actually work. or you could have a smaller number of machines with higher aspecs.

A

So this can help slash your kubernetes bill,, because each node actually requires to be to.

A

Have kubernetes installed on it. and it adds always a little overhead,, but mixed instances do present some scaling.

A

Challenges. so in mixed instance, scenario, each instance uses different type of features.

A

While you can scale up the instance in your order, scaling groups using metrics like cpu.

A

And network utilization, you'll get inconsistent metrics.

A

But then cluster or a scalar helps here by allowing you to mix instanced types in node.

A

Group. but yeah, so there's there's always good and bad sides due to everything.

A

But then the last of the eight recommendations is make multiple regions work for you.

A

So instances spanning across several regions increase your availability.

A

So it is recommended, for example, by aws to configure multiple node groups, scop each of.

A

Them to a single region and then enable that balance. A similar node group's feature.

A

And if you create only one group,, you can scope that node group standard across multiple.

A

Regions., that's very nice, as well.

A

So that's it for the eight best practices, tips for general kubernetes and cloud cost.

A

Optimization. but then I think we can, I think it's worth looking into what is actually.

A

Holding teams back.: is it the lack of knowledge or is it something else?.

A

So because, if this knowledge is all very not very commonplace,, but it is out there, why.

A

Isn't more people doing cost optimization cost something?.

A

I think really great report by a phoenix foundation,, the status of the twenty twenty one.

A

Phoenix foundation is under the phoenix foundation.

A

In their report,, they they have asked their members on how what are their business.

A

Practices and how do they manage these things.?

A

So what challenges do business teams face?

A

thirty nine percent said that getting engineers to take action is one of the main.

A

Challenges that they have. and then there is dealing with shared cost,, accurate forecasting,, reducing waste or.

A

Unused resources. and then what tools do have not practitioners use?, so they need to tools, third party vendor platforms.

A

And surprisingly, to me,- and I think a lot of people probably is that 32 percent use.

A

Spreadsheets and nearly half of the respondents say that they have little to no.

A

Automation., so what I think this tells us is it's hard to get people to take action.

A

It's hard to get people involved., it's difficult to kind of approach, cost optimization.

A

But then, at the same, time, there's little to no automation..

A

So I think this is the perfect spot to be focusing on,, because if you have automating.

A

Tooling,, it's a lot easier to take cost optimization off.

A

It's a lot easier to manage it.. It doesn't require so much manual work..

A

So this is something that you can actually take your engineers to take action. A lot.

A

Easier., so this is a great way to increase cost optimization.

A

Just to highlight a few cncf project solutions that can be used to kind of build up. Your.

A

Cost optimization., so cncf is the cognitive computing foundation,, which is home to.

A

Kubernetes. so in there you have even driven or scaling,, which is keda, a really great,.

A

Wonderful project., you can use application management tools such as helm and monitoring.

A

Is highly important., so you can use prometheus, for example, and then container workflow,.

A

Flux works, great. and then I highly, highly recommend using a service mesh as kuma, for example, works.

A

Wonderfully., but I think the one of the best approaches to this is actually using a.

A

Comprehensive solution,, an a.I.

A

or automation, driven cost optimization,, because then you can get full benefits from all.

A

The things that I've mentioned without too much work needed on your part.

A

And you can obviously build your own kind of solution bit by bit from the cnf solutions.

A

As well., but I think, automation, driven cost optimization- is really great.

A

So there's multiple places,, multiple platforms that offer it.

A

But the one that I'm most excited about is definitely is cast a.I.

A

And I do work for casani,, but I'm also personally incredibly excited about it, because I.

A

Think it's absolutely a wonderful thing.

A

And we have been building cast a.I..

A

The kubernetes does cost optimization engine for a while now and it's amazing to hear.

A

Great feedback, feedback on it,, but also amazing, to hear any feedback on it.

A

So if you have, if you try it out,, let me know and tell me the positive,, the negatives,.

A

Everything. we're always building even a better product. Every day.

A

So no worries there.

A

so then I will show you how easy to use automation to optimize your cost. Optimization.

A

So all the things that I've said, totally important.

A

But now a few steps that you can also take advantage of pretty much all the steps that.

A

I've shown before. so in here,. This is how you get started with cost optimization.

A

Automation., so you connect your cluster in here.

A

You can see it's a simple script that you'll paste to your terminal and then you will get.

A

A read only access to cast a.I.

A

to get started. and no worries there. Nothing will change on your environment.

A

We just take a peek to see what there can be optimized and what are the kind of needs in.

A

Particular in your cluster., so it's completely customized to you., and then this will produce after you connect this kind of really great report,, which is.

A

Already really nice, provides a lot of information..

A

It's the savings report., so you in there,, you see your current compute cost..

A

You see how many policies you've been able. You see your progress on how much how close.

A

Are you to an optimal set up in there in the middle.

A

I can highlight, for example, that you have the workloads there..

A

You can already see there that some of them are spot friendly.

A

And so then you don't have to go manually kind of thing through the solution, workload by.

A

Workload on its path. bandiana,, the platform will tell for you and then current cluster configuration optimized.

A

Cluster configuration., and so it takes a lot of the guys guessing out of the optimization. If you use an.

A

Automated platform., so then, if you want to take it further,, you can then enable a cost.

A

Optimization automation to your cluster., so you just add some more access, rights,, but not too many.

A

It's just to to optimize the cluster,, nothing more,, no environment,, no application changes.

A

Necessary at all., and then you get this kind of a field where you can then see all the policies that you.

A

Can enact. so you have policies from cpu policy which ensures that your cluster stays.

A

Within the defined cpu, a minimum and maximum accounts., you have horizontal put on a scalar,. You have spot and preemptive instance policies,.

A

Which I really want to highlight is a really great thing, because this is something that.

A

Takes that automates the spot instances and they are really hard to sometimes manage and.

A

People kind of shy away from them., but this takes that makes them easy and makes them automated, lets you handle everything.

A

Super easily., so it's really nice.

A

And in addition to this,, there's also deletion of notes of victory. Policy.

A

But it didn't fit my screenshot, unfortunately,, but yeah.

A

And after that, you will get an updated saving's report., but that tells you what your current configuration is.

A

and we are also building, for example, ongoing savings. Part where you can see an ongoing.

A

Savings., so a little teaser there, if you see in the future.

A

Then I want to highlight that if you're interested in finding out more about cast a.I.,.

A

You can go to this link cast.ai/annie, there's a really nice google forum there, where you.

A

Can give input, your email address essentially, and you will get email from us.

A

Essentially,, but also a surprise gift that might have something to do with kubernetes,.

A

Plushies, stickers and t-shirts,, you know, these kind of things,. It's the only way to.

A

Give really nice things and in this kind of virtual event,, so.

A

Happy to happy to get you all there and connecting, and you can always send me a message.

A

As well, if you're interested.

A

as a recap,, we went through eight best practices that I highlighted.

A

We went it looked a bit on cncf project, solutions,, keda,, helm,, prometheus,, flux,, kuma.,.

A

And then we went through an automated option that you can also use to manage cost.

A

Optimization in your projects and in your company.

A

So here are some resources that are really great, that I want to highlight.

A

So phoenix foundation has really great principles that that anyone can kind of take and.

A

Build upon top and get further in your cost, optimization journey.

A

Then eight best practice is to reduce your aws bill. Opportunities.

A

It's cast a.I. block fees,, which is really great resouce, a lot of the things that I've.

A

Spoken here is mapped there already., so you want to kind of find out more about those.. You can check that out. and also the governance policies are more described. There.

A

And as always, with my presentations,, you can find my slides from my github.

A

That's it for the presentation, part.

A

And thank you so much for kong.