►
Description
What is Kubernetes cost allocation? How do you setup your Kubernetes environment to accurately trace the costs tied to the many different components involved in your Kubernetes clusters? In this video, Lukonde Mwila explains the importance of cost allocation in Kubernetes and demonstrates how you can accomplish this using Taints and Tolerations, Karpenter, KubeCost & Cost Explorer.
Activate user-defined cost allocation tags: https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/activating-tags.html
00:00 - Introduction
00:05 - Why is Kubernetes cost allocation important?
02:30 - Using taints and tolerations to create dedicated nodes for workloads
03:12 - Using Karpenter to manage lifecycle of dedicated nodes for workloads
03:53 - Using Cost Explorer to see costs of dedicated nodes
04:25 - Using KubeCost for granular cost allocation breakdown for workloads
05:02 - Code walk-through and demo
#AWS #Kubernetes #EKS #karpenter #kubecost
A
One
of
the
more
difficult
challenges
when
running
kubernetes
at
scale
is
cost
allocation,
there's
so
many
components
typically
involved
in
or
under
the
umbrella
of
your
eks
cluster,
especially
in
multi-tenant
environments.
It's
certainly
not
that
helpful
to
Simply
get
a
total
cost
as
the
output
of
everything
going
into
this
kubernetes
operational
funnel.
One
of
the
steps
to
cost
optimization
is
to
have
enough
transparency
around
the
expenditure
tied
to
each
of
these
different
components.
Let's
take
the
example
of
an
organization
running
multiple
micro
services.
A
Each
microservice
could
represent
a
product
and
is
probably
going
to
be
owned
by
a
single
product
team
or
a
product
team
may
have
a
subset
of
microservices
that
it
owns
in
the
bigger
architecture.
A
company
might
have
20
50,
80
or
more
of
these
products.
In
cases
like
this,
it's
especially
important
to
have
the
visibility
to
trace
the
cost
related
to
specific
components
like
nodes,
Ingress
gateways,
namespaces
allocated
storage,
a
specific
workload
and
the
replicas
tied
to
those
workloads,
the
more
granular
the
better.
A
This
kind
of
information
can
serve
as
a
precursor
for
decisions
related
to
cost
optimization,
otherwise,
cost
optimization
becomes
a
lot
harder
when
all
you
have
to
work
with
is
a
single
figure
for
the
total
cost.
It's
also
not
prudent
to
try
and
cut
costs
from
a
high
level
like
a
total
cost,
because
you
don't
know
what
impact
a
reduction
in
cluster
resources
will
have
on
your
workloads
and,
by
implication,
end
users
or
other
clients
dependent
on
the
high
availability
of
your
workloads.
A
There
are
other
tools
that
you
can
use
to
achieve
a
similar
outcome,
but
perhaps
this
will
serve
some
teams
looking
for
a
model
or
example
that
will
help
them
accomplish
this
before
I
switch
to
my
editor,
I'll
start
by
briefly
explaining
what
purpose
each
of
these
components
is
going
to
serve.
I'll
start
with
taints
and
tolerations
with
taints.
You
can
influence
where
the
scheduler
places
pods
in
your
cluster.
A
You
can
essentially
tell
your
nodes
to
reject
or
repel
certain
pod
placements,
as
well
as
influence
how
strictly
the
scheduler
should
take
the
effects
of
these
tanks
and
is
a
great
way
to
dedicate
certain
nodes
or
compute
capacity
to
teams
or
products.
If
you
think
about
a
multi-tenancy
model,
where
multiple
teams
are
sharing
a
cluster,
it's
definitely
one
approach
to
set
resource
constraints,
but
it's
also
very
useful
to
dedicate
certain
nodes
to
teams
or
products.
A
Explorer
and
I've
placed
a
link
to
the
docs
in
the
description
that
will
walk
you
through
setting
this
up
and
finally,
there's
Cube
cost
I'm
going
to
use
it
for
more
granular.
Breakdowns
of
the
costs
involved
in
my
workloads
with
Cube
cost
I'll
be
able
to
see
the
spend
based
on
a
namespace
each
application
in
that
namespace,
as
well
as
CPU
memory,
networking
load
balancing
costs
all
of
these
components
tied
to
each
application
in
terms
of
their
costing,
with
both
cost
Explorer
and
Cube
cost.
A
You
can
generate
and
Export
the
relevant
details
in
a
standard
format
like
a
CSV
or
PDF
for
reporting
purposes,
and
the
end
result
is
the
visibility
of
what
the
costs
are
for
the
different
components
for
a
given
product
or
Team,
all
right,
so
first
things,
first
I'm
calling
my
example
application
an
e-commerce
app,
but
it's
very
bare
bone,
one
graphql
service,
an
order
service
and
a
product
service.
The
graphql
service
is
the
API
Gateway
and
data
aggregator
for
the
other
two
services,
but
that's
just
for
context.
It's
not
our
focus
at
all.
A
A
If
my
Ecommerce
app
shares
nodes
with
other
applications,
that's
going
to
obscure
the
kind
of
cost
transparency
that
I'm
hoping
for
I
want
to
be
able
to
go
to
cost
Explorer,
as
well
as
Cube
cost
and
filter
to
see
costs
just
for
the
e-commerce
project.
Now,
firstly,
I'm
going
to
be
focusing
on
the
aspect
of
the
node
costs,
so
I
want
the
right
kind
of
visibility.
That
is
specific
in
cost
Explorer.
A
I
have
my
taint
properties
that
will
be
applied
to
the
e-commerce
nodes
and
the
no
schedule
effect
means
pods
that
don't
tolerate
the
taint
won't
be
scheduled
on
the
Node.
However,
if
pods
were
scheduled
to
the
node
before
the
taint
was
applied,
they
won't
be
evicted
and
will
continue
to
run
as
per
Norm,
but
this
rule
or
effect
applies
to
any
subsequent
pod
placements.
So
remember,
the
purpose
of
taints
is
to
repel
certain
pods,
but
we
don't
want
to
repel
every
single
part.
A
So
how
do
we
place
pods
on
nodes
that
have
taints
and
that's
where
tolerations
come
in
and
you
can
see
over
here?
I
have
my
manifest
for
graphql
orders
and
products
each
of
our
micro
services
in
order
for
a
pod
to
tolerate
a
taint
it
must
match.
Let
me
scroll
down
slightly,
so
you
can
see
the
Toleration
over
here
and,
as
you
can
see,
it
matches
the
exact
same
key
value
pair
and
the
effect
of
the
taint.
That
is
a
requirement
in
order
for
a
pod
to
tolerate
a
taint.
A
Another
important
property
when
matching
tolerations
detains
is
the
operator
property.
The
operator
value
can
either
be
equal,
which
is
the
default
value
or
it
can
be
exists,
and
if
the
operator
is
set
to
equal
or
not
defined
altogether,
because
it's
the
default
value,
then
the
value
from
the
taint
has
to
be
specified
in
the
Toleration.
On
the
other
hand,
if
you
use
the
exist
operator,
then
you
don't
need
to
specify
the
chain
value.
The
key,
in
effect,
will
suffice
as
long
as
they
match
the
taint
now
I'm
going
to
switch
over
to
K9s.
A
A
But
I
want
to
see
if
the
I
want
to
see
this
in
cost
Explorer
as
well,
because
remember,
cost
Explorer
is
where
I'll
be
able
to
actually
export
a
CSV
file
which
is
great
for
reporting
purposes
and
can
later
be
merged
or
Amalgamated
with
any
other
reports,
even
from
Cube
cost.
So
let's
switch
to
that.
A
A
You
can
see
the
tag
key
over
here,
carpenter.sh,
the
provisioner
name
when
I
click
on
the
list
of
values,
you'll
notice
that
I
have
both
e-commerce
and
express
node.js,
and
each
of
these
represents
the
different
sets
of
nodes
that
I
have
now
I'm
only
interested
in
e-commerce.
So
that's
what
I'm
going
to
select
and
apply-
and
you
can
see
over
here,
the
custom
usage
graph
has
updated
now.
I
can
see
a
cost
breakdown
for
the
e-commerce
nodes,
as
opposed
to
just
a
total
cost.
A
Also,
if
I
wanted
to
I
can
export
these
results
as
a
CSV,
and
this
way,
even
though
I
have
multiple
nodes
and
workloads
in
a
shared
cluster,
I
have
visibility
and
Reporting
capabilities
for
the
infrastructure,
costs
of
specific
teams
and
projects
using
the
tags
applied
to
the
carpenter
provision
nodes
and
because
of
the
taints
and
their
Associated
effects.
I
can
tie
the
node
cost
to
the
appropriate
team
project
or
department,
but
this
is
just
one
layer
in
our
kubernetes
cost
bundle.
A
Rather
I
want
to
focus
on
one
of
the
columns
or
categories
you
can
see
here
we
have
load
balancing
and
you'll
notice
that
orders
and
microservices
do
orders
and
products
rather
do
not
have
any
costs
tied
to
them
in
the
load
balancing
category,
whereas
the
graphql
service
does,
and
the
reason
for
this
is
because
of
the
role
that
our
graphql
service
is
fulfilling.
It
is
actually
receiving
requests
from
a
public-facing
load
balancer
and
then
aggregating
data
from
the
Upstream
services
that
it
communicates
with
orders
and
products.
A
So
it
is
actually
fulfilling
the
role
of
a
load
balancer,
hence
the
cost
that
is
tied
to
it.
But
let's
drill
down
even
further
and
now
the
cost
breakdown
is
at
a
replica
level,
I'm
going
to
scroll
down
even
further,
and
this
is
very
useful,
especially
over
a
period
of
time.
If
and
when
your
application
undergoes
certain
changes.
A
For
example,
an
app
may
start
using
more
CPU
cores,
which
will
result
in
a
cost
increase
and
can
be
seen
in
this
same
view,
and
that
would
also
be
the
case
if
we
have
a
stateless
application
that
starts
making
use
of
storage.
It
will
then
have
a
persistent
volume
cost
tied
to
it
there,
and
these
are
cost
changes
that
we
can
see
at
a
granular
level
for
replicas
of
the
same
application
over
a
period
of
time.
A
I
covered
how
taints
and
tolerations
with
Carpenter
can
help
dedicate
nodes
to
certain
projects
or
workloads.
Then
looked
at
costs
tied
to
those
specific
nodes
in
cost
Explorer
and
just
wrapped
up
a
brief
walkthrough
on
how
cost
allocations
in
Cube
cost
can
provide
even
more
granular
details
that
will
eventually
inform
your
cost
optimization
exercises
as
usual.
Let
us
know
if
you
found
this
helpful
by
providing
feedback
and
stay
tuned
for
more.