►
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Sustainability Research the Cloud Native Way - Chen Wang, IBM & Huamin Chen, Red Hat
Speakers: Huamin Chen, Chen Wang
Do you want to help combat climate change? Are you interested in sustainability research? Then join our open systems for Cloud-native sustainability infrastructure. We present the research opportunities of using Cloud-native patterns, observing, optimizing, and executing, to improve Cloud efficiency in sustainable architecture. Core to this architecture is Project Kepler (Kubernetes-based Efficient Power Level Exporter) and its integration with Kubernetes ecosystems. By leveraging eBPF programs and other system libraries, Kepler probes the full spectrum of energy, performance, and resource measurements to enable energy-relevant observability and further empower advanced sustainable management on clusters. Kepler is an open system for exciting research topics like energy-efficient workload scheduling, energy-aware autoscaling, and frequency tuning. In this session, a sample Kepler integration is case-studied to help researchers build their advanced vertical autoscaler to improve the energy performance objectives of the Kubernetes applications.
B
Thank
you
Sean
for
the
introduction
and
it's
my
pleasure
to
meet
you
all
and
let's
celebrate
for
this
afternoon
and
with
some
of
the
good
ideas
and
the
introductions
of
what
we
have
done
for
sustainability
and
for
the
research
community,
and
so,
if
you
I
just
want
to
put
a
certain
words
and
certain
criterias
of
what
we
do
and
what
we
see
is
cloud
native
sustainability
Republican
using
a
few
characters,
some
colors
to
paint
the
picture
so
I
call
it
IMC
coupe
not
to
understand
not
too
confused
with
mcsquare.
So
for
m
I
mean
metrics.
B
So
as
we
many
of
you
have
noticed
for
cloud
native,
its
observability.
That
gives
us
a
lot
of
power
for
scheduling
for
scale,
for
scaling
and
for
a
lot
of
management
and
costs
and
availability.
So
having
enough
mess
with
metrics
will
give
you
the
insights
into
what
we
what
we
can
do,
so
the
metrics
we
build
is
for
energy
conservation.
So,
specifically,
we
want
to
know
how
much
energy
consumed
by
a
workloads
in
Cloud
native
in
kubernetes.
That
means
the
part
level
how
much
energy
you
use
by
the
node
by
the
parts.
B
The
second
is
one
we
call
correlation
so
having
this
Matrix
makes
sense
to
end
user,
but
start
not
making
use
for
the
end
user
in
terms
of
doing
the
energy
conservation.
So
when
you
schedule
a
workloads,
how
do
you
correlate
with
the
metrics
with
the
hardware
characteristics
such
as
the
scheduling
of
your
workloads,
will
be
optimized
for
energy
consumption?
B
So,
after
the
scheduling
the
workloads
landed
on
the
nodes
it
keeps
running
for
days
and
weeks
and
even
for
months,
if
signage
of
the
original
assumption
about
the
resource,
consumption
resource
assignments
goes
wrong.
How
do
you
correct
this
runtime
Behavior?
So
we
call
this
the
correction
phase.
In
this
phase
we
use
an
auto
scaling
capabilities
in
the
kubernetes
community
as
very
last
one.
B
Once
you
go
beyond
a
single
cluster,
you
want
to
take
into
account
of
the
carbon
Deltas
Around,
the
World
abusing
different
clusters
that
is
about
the
multi-clusters
use
case
comes
in,
and
we
want
to
account
for
the
carbon
intensity
differences
in
between
multiple
clusters
and
find
the
best
attest
animations
for
your
workloads
to
land.
On.
B
Let's
come
down
to
the
projects
we
are
trying
to
the
problem
with
our
project
is
trying
to
address
the
projects
in
other
problems
in
the
cloud
native
power
management
system.
First
of
all,
you
want
to
have
accuracy
right.
Accuracy
means
there
are
metrics
we
presented
to
you
about
the
part
about
the
containers
as
accurate.
As
we
can
see,
everybody
will
agree
on
the
methodology.
I
will
put,
everybody
will
agree
on
the
end
results.
That's
the
way
we
want
to
achieve.
B
The
second
level
is
the
fairness,
as
you
all
know,
there's
a
having
something
accountable
on
a
shared
offering
system.
It's
not
so
easy
right.
So
it's
just
like
the
same
way
you
live
in
the
Condominiums.
You
pay
the
management
fee.
You
have
to
account
for
every
of
the
usage
by
the
square
feet
by
each
of
the
tenants,
but
they
will
shield
areas
in
the
lobbyists
the
swimming
pools,
how
to
account
for
this
Shield
area.
So
this
the
founders
we
want
to
take
into
account
is
in
the
methodology.
B
So
when
we
present
the
numbers
they
are
often
represented,
they
repeat,
the
representation
will
reflects
the
ideas,
the
founders,
the
accuracy
we
want
to
tell
the
End
customer.
The
very
last
one
is
the
multiple
granularities
you
want
to
present
the
information
not
ever
enough
for
developers
who
are
interested
in
developing
or
want
for
applications.
They
will
focus
on
the
process
right.
The
running
process
will
talk.
B
Lots
of
story
numbers,
however,
efficiency,
the
the
application
is,
but
for
the
containers
we
are
talking
about
the
container
level,
the
power
level
and
for
deployment
level
there's
multiple
layers
of
our
aggregations
and
for
end
users,
maybe
like
a
system
that
means
they
worry
about
the
tenants.
The
names
left
space
novel.
How
do
you
cast
the
information
from
all
these
levels
to
all
different
personas?
So
we
won't
have
all
these
things
considered
into
account
in
their
projects
when
we
build
up
the
project
so
for
capture
projects.
B
If
you
have
already
seen
the
projects,
the
names
actually
stands
for
the
kubernetes
efficiency
power
level
exporter.
It
makes
sense
to
us,
as
you
know,
the
NASA
tactical
Capital.
Let's
give
you
the
insights
into
the
universe.
We
want
to
have
the
insights
into
the
universe
of
the
kubernetes
community,
so
the
capital
has
three
goals
in
mind.
B
One
is
the
reporting,
as
I
said,
you
won't
have
the
capability
to
tell
the
customers
can
end
users
developers
how
much
energy
used
by
the
parts
on
different
levels
of
you
know:
the
electronics,
the
components
are:
CPU
theorem,
GPU
and
eventually
going
beyond
that
level
to
fpga
and
almost
like
a
networking,
it
could
be
the
next
one.
So
we
also
want
to
use
the
capabilities
of
running
Capital
across
the
across
the
hybrid
Cloud
environments.
B
You
what
we
wanted
to
run
on
bare
metal
machines
on
a
private
clouds
on
your
data
center,
and
we
also
run
the
capital
to
run
on
hyperscalers
on
YouTube
to
you
know,
Azure
and
gcp
on
these
environment.
Most
of
the
the
runtime
computation
form
is
virtual
machines.
So
we
want
the
model
to
be
reported
as
a
portable
as
across
the
different
subspections,
and
we
also
support
the
Prometheus
and
that's
the
you
know.
If
you're
going
to
the
keynote
Prometheus
becomes
a
lot
of
a
lot
of
you
know
just
a
lot
of
love
by
the
community.
B
So
we
want
to
support
the
previous
level
of
exporting
if
you
are
getting
all
these
resources
and
if
you
want
to
guess
all
the
details
it
seems
like
this
is
a
very
heavy
task,
so
you
want
to
reduce
the
footprints
of
capital
projects
itself.
We
do
not
want
to
consume
too
much
resources
too
much
CPUs
too
much
ethereums.
As
a
matter
of
fact,
in
my
two
hour
observation,
we
usually
stay
at
two
percent
of
the
CPU
level
and
40
megabytes
of
memory.
So
it's
very
light
weighted.
B
We
do
not
want
to
introduce
our
own
methodology,
I
mean
we
do
not
want
to
invent
something
that
any
customers
have
no
visibility
into,
so
we
are
taking
into
accounts
of
the
scientific
research
over
the
last
two
decades.
I
put
off
the
two
color
code
here.
So
if
you
have
interest,
you
can
just
take
a
look
into
the
research
papers.
B
What's
the
formulas
is
about
I'll
just
give
you
one
example
of
one
of
the
linear
regression
formulas
how
to
take
into
account
of
the
security
usage,
the
performance
counters
and
imagine
the
secure
the
power
consumption
by
the
CPUs.
But
this
is
not
the
formula
we
are
using
As
a
matter
of
fact
if
you
are
going
to
the
next
slide.
So
this
is
our
architecture,
so
we
have
three
layers
of
actually
we
have
three
columns.
B
So
if
you
come
from
the
very
end
of
the
on
the
left,
we
have
the
eppf
that
collects
the
information
from
the
universe
from
the
kernel
level.
The
information
includes
the
CPU
instructions
are
the
cycles
and
the
secure
time
and
also
process
and
container
correlation,
and
we
pump
these
information
up
to
the
user
space.
B
The
user
space
will
get
another
layers
of
information
and,
like
the
CPU
power
consumption,
that,
if
you
are
running
on
x86
platforms,
you
have
the
rapper
runtime
average
power
level
that
will
tell
you
different
domains,
how
much
electricity,
how
much,
how
much
power
consumed
by
each
of
the
CPU
or
drams
and
on
a
different
time
and
also
collects
six
stats.
C
group
stats
are
from
the
c
groups
and
we
also
collects
information
from
kubernetes.
So
we
put
all
this
information
together
and
we
use
the
machine
learning
models
to
predict
how
much
power
used
by
different.
B
You
know
of
this
processes
and
containers
and
parts,
so
that
is
our
methodology,
so
we
are
not
dictated.
We
are
not
dictating.
The
eventual
model
will
ask
the
customer
to
build
the
model
on
your
site,
so
that
will
be
a
true
representation
of
how
much
power
used
by
your
environment
by
your
workloads.
B
So
this
is
the
one
of
the
dashboards
we
can.
We
are
using
thanks
to
the
community
contributors
from
from
here,
and
we
are
trying
to
give
you
some
intuitions
how
much
carbon
Footprints
this
your
workload
will
generate
and
what's
the
variations
over
time,
the
workloads
on
different
of
the
components
on
CPU
on
ethereum
on
GPU,
and
we
also
keep
you
certain
rankings
as
different
or
namespace
level.
With
this
Matrix,
that's
a
unlimited
potentials.
B
How
much
information
you
can
build
up
for
your
final
visualization
and
we
also
want
to
introduce
yet
another
project
by
Chan
he
has
built
after
this
wonderful
projects
called
clever
using
the
Matrix
and
before
she
comes
in
that
I
want
to
give
the
your
community
our
shout
outs.
So
over
the
time,
this
project
has
been
contributed
by
different
people
representing
from
different
companies,
and
we
really
appreciate
their
help
in
shaping
up
the
projects
and
giving
a
lot
of
feedbacks
and
even
contributing
code
to
the
project.
C
So
because
we
have
capillary
in
place,
we
can
have
all
types
of
observative
abilities
in
terms
of
getting
the
energy
related
metrics,
including
energy
consumptions,
for
different
components
of
your
workload
right,
including
CPU
memory,
and
you
can
also
understand
what
is
the
current
frequencies
of
your
CPU
and
then
whether
you
have
tuned
down
the
frequency
to
save
energy.
So
here,
I
just
want
to
give
an
example
on
like
how
we
can
leverage
Kepler
to
do
some
optimizations
on
your
cluster.
C
So
you
can
both
reduce
your
energy
consumptions
as
well
as
guarantee
your
performance
of
the
workload.
So
the
question
we
try
to
answer
here
is
when
the
frequency
tuning
knobs
there's
a
lot
of
frequency
tuning
knobs
available
in
the
kubernetes
community
and
then,
when
you
increase
or
decrease
CPU
frequencies,
and
then
how
you
can
guarantee
the
performance
of
the
workload.
C
So,
let's
assume
you
have
something
like
a
tuner
frequency
tuner
to
update
the
CPU
or
GPU
frequencies
according
to
any
targeted
metric
or
your
energy
consumption
budget,
and
then
we
will
lower
down
the
frequencies
intuitively
your
performance
of
the
workload
will
decrease
and
then
what
we
can
do
is
we
can
obtain
like
the
status
of
the
workload.
The
cluster
state
will
work
on
the
CPU
focuses
from
nodes
where
the
the
frequency
are
changed,
based
on
Kepler
exported
Matrix,
and
in
this
way
we
know,
there's
something
change,
changes
in
your
cluster
or
node.
C
C
We
will
be
done,
then
the
recommender
with
the
recommender
does
is
we
develop
a
recommender
called
clever
and
it
can
trigger
to
resize
your
part
to
either
increase
or
decrease
accordingly
to
your
frequency
change,
to
guarantee
you
get
the
similar
performance
for
your
workload
and
then,
when
you
use
the
clever
recommender
working
with
all
other
controllers
in
the
vertical
part,
Auto
scaler,
the
vpa,
updater
and
animation
controller
will
go
ahead.
Change
resize
your
path
to
guarantee
you
have
the
same
performance.
C
So
the
idea
is
like,
like
let's
take
the
computational
Intensive
workflow
for
example,
and
then
you
relate
the
the
most
important
metric
you
are
watching
is
the
instructions
per
second?
And
if
you
look
at
ipis
formula,
it's
related
to
your
frequency
of
the
CPU,
your
allocation
of
the
resources
on
CPU
and
something
like
that,
the
average
number
of
circles
per
instruction
and
then,
if
you
want
to
Target
the
particular
IPS
here
and
then
you
would
have
a
default
request
on
that.
C
Assuming
you
have,
you
have
the
CPU
in
full
the
maximum
frequency
running
in
the
full
capacity.
And
then,
when
you
tune
down
the
frequencies
and
then
you
will
derive
the
current
IPS
and
what
the
clever
project
is
doing
is
trying
to
force
the
IPS
Target
equals
to
IPS
current,
and
then
you
later
find
out.
It's
not
related
to
the
number
of
average
cycle
pre-instruction
and
the
the
current
request
that
you
should
tune
to
is
only
related
to
the
maximum
frequency,
your
current
frequency
and
your
default
request.
C
C
C
So
we
go
ahead
to
create
capture
and
deploy
Kepler
demand
side
for
all
the
nodes
in
the
cluster,
and
you
can
also
check
if
the
Kepler
demonstrator
is
running.
Okay.
Now
it's
running
just
for
for
exporting
all
the
metrics
to
Prometheus.
We
go
ahead
to
create
a
customer
resource
called
service,
monitor
to
actually
expose
the
Kepler
endpoint
to
script,
the
capital
endpoint
for
all
the
data,
and
that
Yama
also
includes
all
the
Outback
juice.
We
need
access
to
Prometheus.
C
We
can
double
check
if
the
service
monitor
resources
is
successfully
created
under
the
capital
namespace,
and
you
can
also
check
from
the
Prometheus
AI
whether
the
Kepler
service
support,
monitor
endpoint
is
available.
It
takes
some
time
for
it
to
be
up
and
the
other
way.
You
can
also
check
this
Kepler
service
endpoint
to
get
whether
it's
an
owner
and
the
port
number
is
9102.
C
Let's
go
and
load
the
Kepler
exporter
dashboard
and
the
dashboard
is
already
also
available
in
the
GitHub
repo.
There
is
a
folder
called
grafana
dashboard.
You
can
import
it
in
the
dashboard.
You
can
quantify,
you
can
see
the
like
the
equivalent
CO2
amount
and
patrolling
amount
of
energy
consumptions
for
for
a
particular
part,
and
then
it
also
so
shows
the
energy
consumptions
for
different
components
over
time,
and
then
it
also
accumulated
the
energy
consumptions
for
different
name
namespace
of
your
workload.
C
C
C
C
Then
then
we
want
to
clone
the
project
of
clever
clever
is
just
rating
a
simple
code,
vpa
recommender
written
in
Python
and
it's
less
than
200
lines
of
code,
and
then
you
can
take
a
look
and
write
your
own
recommender
as
well.
The
way
to
deploy
it
is
very
simple.
We
already
prepared
the
Manifest
file
for
that.
C
And
the
clever
yamo
includes
all
the
necessary
deployment
cluster
rows,
classroom,
binding
and
service
account,
and
the
only
thing
you
want
to
configure
is
the
involvement
variable
premises
host
and
then
let's
use
the
default
endpoint
here.
So
it
will
get
all
the
necessary
metrics
from
the
from
premises
from
also
Kepler
exporter.
C
Okay,
so
the
next
step
we
are
going
to
test
is
we
want
to
create
a
workload
using
the
all
the
vpa
and
using
help
clever
as
the
customized
recommender
for
the
vpa
and
for
any
workload.
So
now,
if
we
go
back
to
see
the
the
clever
dashboard
and
in
the
clever
dashboard,
we
will
show
like
something
like
what
is
the
current
frequencies
of
the
node,
the
maximum
minimum
frequencies
of
the
node,
and
this
dashboard
is
also
available
on
the
clever
project.
Ripple
and
you
see.
Currently
we
have
one
node.
C
C
The
third
one
shows
the
instructions
per
second
performance
metric
for
the
workload.
By
the
way
the
IPS
metric
is
also
available
in
Kepler.
So
if
you
install
Kepler,
you
can
not
only
see
the
energy
consumption
Matrix,
but
all
types
of
performance
metrics
for
your
for
your
workload.
It
is
using
ebpf.
So
it's
very
lightweight.
C
C
C
C
Now
the
the
default
value
is
250..
Let's
wait
a
moment
for
the
Kepler
to
get
the
actual
frequencies
of
the
now.
The
actual
frequencies
are
already
jobs
right.
You
see
the
the
top
window.
The
frequency
drops
from
almost
four
to
two
gigahertz
and
then
at
the
same
time
the
ipis
drops
for
the
workload
and
it
will
take
a
while
for
the
vpa
to
kick
in.
C
So,
actually,
when
we
tune
to
the
frequency
down,
the
IPS
drops
full
well,
but
the
VP
and
the
clever
controllers
kicking
to
increase
the
allocation
of
the
CPU
and
equivalently.
We
will
keep
the
same
ipis
over
time.
This
is
how
clever
utilizing
the
Kepler
metrics
to
guarantee
you
have
the
same
performance
as
well
as
saving
some
energy.
C
B
So
this
is
my
observation,
so
we
have
seen
a
lot
of
one
for
skating
mechanisms
for
performance
for
resource,
but
we
have
never
seen
something
to
scale
for
the
purpose
of
energy
conservation
without
disrupting
your
service
and
I
think
this
is
one
of
the
greatest
achievements
of
what
this
integration.
Thank
you
chairman
for
this
huge
contribution
so
looking
forward.
We
believe
Kepler
and
integrative
with
other
ecosystems
with
other
metrics.
We
can
make
bigger
things
happen,
so
our
going
back.
B
B
We
are
discussing
bootstrapping
mechanisms
to
Auto
scaling
based
on
carbon
intensity
and
we
potentially
can
also
use
the
same
mechanisms,
build
up
even
high
impacts,
Auto
scaling
mechanisms
for
service
workloads
for
auto
scaling,
so
we'll
take
both
our
energy
and
carbon
into
accounts.
So
we
can
reduce
the
energy
and
carbon
Footprints
with
high
impact
workloads
and
that's
what
we
have
for
this
presentation.
A
D
How
you're
sort
of
thinking
about
energy
Beyond
just
the
CPU?
So
like?
Obviously,
you
know
if
you're
using
a
cloud
provider
they
have
to
power.
You
know
everything
in
the
box
right
yeah.
In
some
ways
it
can
make
more
sense
to
bin
hack
denser
nodes,
because
you
amortize
the
cost
of
things
like
next
over
sort
of
more
containers
right.
So
how
do
you
sort
of
Juggle
that.
B
Yeah,
so
if
I
don't
understand
crap
today,
so
why
is
the
question?
Is
plus
the
the
the
components
like
the
CPU
and
the
other
one
is
like
the
type
of
workloads.
So
just
let
me
ask
you
the
first
question
and
then
maybe
you
can
help
with
help
me
with
the
second
question
about
the
type
of
workloads,
so
I
think
that
you
are
definitely
right.
We
are
targeting
CPU
a
lot
in
this
presentation.
B
One
reason
is
that
the
CPU
actually
accounts
for
60
of
the
whole
energy
consumption
of
the
server
I
mean
relatively
speaking,
because
if
you
have
huge
amounts
of
ram,
that's
good,
the
number
could
be
vary,
but
our
CPU
is
the
adaplex
around
that
those
things
they
are
workloads
is
GPU
and
we
also
have
the
GPU
capabilities.
The
metrics
we
can
reports.
The
challenge
with
GPU
is
that
the
frequency
and
eventually
QRS,
are
hard
to
Define.
Like
gcpu
can
Define
this
instruction
per
cycle.
Gpu
will
probably
have
different
metrics,
it's
very
hard
to
Define.
B
If
we
know
what's
the
metrics
we
can
use,
we
can
build
that
into
a
yes
another
exporter
and
a
capital
could
provide
the
energy
information
and
you
probably
can
provide
this
usability
metrics
and
we
can
correlate
with
each
other
and
you
can
build
up
yes,
another
vpa,
based
on
the
new
metrics
and
for
the
type
of
workload,
maybe
Channel.
You
know
she
knows
much
better
about
scheduling
and
scaling
than
I
do.
C
C
E
C
That's
a
very
good
question,
so
it
takes
a
while
in
our
current
setting,
because
we
can't
do
like
checking
the
frequency
changes
only
every
minute
to
reduce
the
resource,
consumption
so
clever,
and
if
you
want
you
can
just
increase
a
little
bit
the
frequency
you
can
tune
it
to
10
seconds
5
Seconds,
whatever,
as
it
is
needed
because
in
our
case
it's
like
the
frequency.
We
don't
two
frequency
that
frequently
so
one
minute
interval
will
be
fine,
but
it's
configurable.