►
Description
Does your OpenShift node need a performance tune-up? With the advent of edge computing in Telco and 5G spaces, decreasing latency and congestion issues while enhancing application performance has become a prime concern. In this episode, Franck Baudin, Senior Principal Product Manager, joins to take a look at the Performance Addon Operator. We’ll walk through installation steps and a few of the different settings and modes to consider when using it to tune nodes for performance. We'll also look at what you need to know for debugging latency degradation issues. So if you're an admin looking to achieve low latency performance for OpenShift applications, join us to explore the Performance Addon Operator and learn more about how to tune software nodes for real-time running and low latency.
Learn more: https://access.redhat.com/documentation/en-us/openshift_container_platform/4.6/html/scalability_and_performance/cnf-performance-addon-operator-for-low-latency-nodes
Follow us:
Andrew Sullivan , https://twitter.com/practicalandrew
Jonathan Rickard, https://twitter.com/jrocktx1
Subscribe to Red Hat's YouTube channel: https://www.youtube.com/redhat/?sub_confirmation=1
About OpenShift:
Red Hat OpenShift is an open source container application platform based on the Kubernetes container orchestrator for enterprise application development and deployment.
Openshift.com https://red.ht/33Fh6uV
OpenShift on github https://github.com/openshift/
OpenShift on redhat.com https://red.ht/2o4ccDk
#RedHat #OpenShift #ContainerPlatform
A
A
Hello
and
welcome
to
this
week's
ask
an
open
shift.
Admin
live
stream.
We
are
here
and
you'll
notice
that
it's,
I
don't
have
my
usual
co-host
with
me.
It's
johnny
is
on
some
well-earned
pto
this
week,
but
I
do
have
our
guest.
Mr
frank,
baden,
joining
us
today.
So
if
you
happen
to
miss
the
title
of
this
particular
episode
or
the
subject,
we're
going
to
be
talking
about
the
performance
add-on
operator,
so
we'll
get
into
much
more
details
there.
This
is
a
this
one
pokes
all
my
all
my
nerd
buttons
it.
A
It
makes
me
happy
to
be
able
to
mess
around
and
when
frank-
and
I
were
talking
yesterday,
it
was
you
know
it's
it's,
this
one's
a
little
dangerous
right.
You
can
break
things
and
when
you
break
things,
that's
that's
bad,
but
I'll.
Quit
talking
about
frank
and
I'll,
let
frank
introduce
himself.
If
you
don't
mind.
A
Yeah
yeah
and
I
I
know
the
technical
product
manager
aspect,
there
is
really
important
because
you
are
one
of
our
more
technical
product
managers
for
sure,
and
I
think
that's
something
that
is
really
good
and
really
exciting
to
our
audience.
Here,
let's
see
you're
very
modest
about
about
all
of
this.
You
keep
telling
me.
Oh
it's
probably
going
to
be
a
short
subject.
I'm
I'm
not
so
sure
because,
like
I
said
this
is
one
that's
interesting
to
me.
A
So
with
that
being
said,
let's
talk
a
little
bit
about
the
top
of
mind
topics
and
just
as
a
reminder.
These
are
things
that
johnny
and
I
have
found,
or
we
think
that
might
be
interesting
to
you
all-
that
we
discovered
kind
of
since
the
last
live
stream
that
we
did
so.
The
first
thing
that
I
want
to
talk
about
here
is
going
to
be
a
blog
post
that
was
recently
published.
You
can
see
it
was
published
on
monday,
and
this
is
around
ansible
and
open
shift
integration.
A
So
what
this
means
is
you're
kind
of
you
know,
unlocking
unblocking
the
scale
of
ansible
and
ansible
tower
to
unlimited.
You
know
as
many
jobs
as
you
need
to
run
as
many
concurrent
jobs
as
you
need
to
run.
They
can
all
be
scheduled
against
a
kubernetes
cluster
or
an
openshift
cluster,
so
that
you
can,
you
know,
take
care
of
those.
So
I
thought
this
was
really
cool.
A
It's
something
that's
interesting,
particularly
as
you
know,
we
start
to
see
more
and
more
cross-infrastructure
cross-application
integration
happening
within
our
customers
and
the
applications
that
they're
doing
you
know
I've.
I've
seen
and
I've
worked
with
a
number
of
customers
who
they'll
you
know
hey.
My
application
needs
to
deploy
something
or
my
application
team
needs
to
deploy
something,
and
some
of
it
is
an
open
shift
and
some
it
is
in
virtual
machines,
and
some
of
it
is
physical
and
we
need
to
reconfigure
this.
A
We
need
to
provision
storage
off
of
that
you
know
and
have
everything
mesh
up
in
the
end.
So
me
this
one
was
really
cool
and
really
exciting,
but
I
I
won't.
Hopefully
I
won't
oversell
it.
Let's
see
where's
my
notes
document.
Oh
I,
the
first
one
I
actually
I
forgot
to
bring
up.
Is
that
a
reminder
that
there
will
be
a
no
or
excuse
me?
There
will
be
no
ask
an
open
shift.
Admin
live
stream
next
week,
so
february
the
16th.
A
A
So,
essentially,
if
you're
not
familiar
the
red
hat
product
management
team
has
two
live
streams
that
they
do
have
basically
every
quarter,
so
one
is
what's
new,
it
happens
with
each
release
of
openshift
and
it
describes
all
of
the
new
things
that
you'll
find
happening
inside
of
that
release.
We
typically
schedule
those
for
90
minutes.
I
don't
know
that
one
has
ever
gone
less
than
90
minutes.
Usually
it's
closer
to
two
hours,
they're,
really
thorough,
they're,
really
good.
A
Typically
here
on
the
ask:
an
openshift
admin
live
stream,
we'll
follow
up
with
that,
and
I
like
to
pull
out
a
couple
of
the
things
that
I
think
are
really
important
to
you
all
and
kind
of
dig
down
into
them
a
little
bit
and
explore
what
that
means
for
us.
So
if
you
happen
to
be
watching
the
live
stream
feel
free
to
you
know,
send
me
a
message,
and
let
me
know
you
know:
hey,
can
you?
Can
you
pull
this
apart?
A
A
So
if
you
have
any
questions,
comments,
etc,
as
they're
going
through,
that
presentation
be
sure
to
leave
them
here
in
the
live
stream.
We
always
shuttle
those
over
to
the
product
management
team.
So
if
there's
anything,
we
can't
answer,
we
can
get
those
answers
from
them
and
of
course
they
always
like
to
know
what
your
thoughts
and
opinions
are
too
okay,
so
that
being
said,
do
subscribe
to
the
channel.
A
If
you
haven't
already
whatever
platform
you
happen
to
be
on,
we
do
have
a
number
of
really
interesting
shows
coming
up
towards
the
end
of
the
month,
so
I
don't
have
the
list
in
front
of
me.
So
I
I
don't
remember
off
the
top
of
my
head.
What
they
are
I've
been
completely
consumed
with
one
of
my
tasks
this
week
I
I've
so
far.
A
It's
only
wednesday
and
I've
spent
like
20
hours
on
this
thing,
so
my
brain
is
only
half
here,
but
yeah
do
please
get
subscribed
if
you
don't
want
to
subscribe
on
youtube
or
on
twitch
or
anything
like
that.
Definitely
check
out.
So
it's
red.ht
live
streaming.
That's
page!
That's
the
landing
page
for
red
hat
live
streaming!
A
That
has
the
whole
calendar
of
all
of
the
live
streams
there
and
it
is
a
google
calendar
so
down
in
the
lower
right
hand,
corner
there's
a
little
thing
that
says
add
to
google
calendar,
so
you
can
subscribe
to
the
calendar
there
if
you'd
like
so
that
way,
you
can
get
notified
if
we
cancel
so,
for
example,
if
you're
subscribed
to
that
calendar,
you
would
see
the
canceled
one
for
next
week
as
well
as
yeah
supplement
your
brain
with
fried
turkey.
That
would
not
be
a
good
idea
tiger.
A
Oh
stephanie,
thank
you
very
helpfully,
so
hopefully
we'll
be
talking
about
odf
essentials.
We
might
be
doing
a
dive
into
that
and
what
that
looks
like
inside
of
clusters.
I
think
there's
some
other
4.10
specific
things
that
we
wanted
to
call
it
as
well.
One
of
the
ones
I
wanted
to
talk
about
is
l3
mode
with
you
know
so,
bgp
mode
with
metal
lb
in
them
states.
Yes,
thank
you!
That's
another
good
one,
so
I
have
a
show
scheduled.
A
I
don't
have
the
guest
lined
up
yet,
but
in
em
state
is
going
ga.
We
think
we
we
or
should
be
going
with
4.10
and
there's
some
really
cool
stuff
that'll
be
happening
there
at
the
installer
level,
so
stay
tuned.
For
that,
I'm
not
going
to
spoil
the
pm
team.
Surprise
all
right.
Moving
on,
we
had
so
internally
a
lot
of
the
red
hat
folks,
particularly
the
ones
that
are
close
to
the
bu
here
and
engineering.
We
had
what
we
call
the
the
openshift
networking
summit
last
week.
A
A
It's
been
talked
about
since
we
released
ovn
kubernetes
as
a
fully
supported
sdn
option
back
in
four
six.
I
think
four,
six
or
four
seven
you
know
which
one
of
these
do
I
use,
which
one
should
I
choose
and
really
it
comes
down
to
the
default-
continues
to
be
openshift
sdn
and
as
of
right
now,
there's
no
real.
A
There
is
a
goal,
a
longer
term
goal
to
change
that
from
the
default.
I
think,
if
you
remember
we
had
maybe
a
year
ago,
I
don't
think
it
was
a
year
and
a
half,
but
about
a
year
ago
we
had
mark
curry
on
the
live
stream
and
mark
had
said
you
know,
yeah
4.9.
We
intend
to
switch
the
default
to
ovn
kubernetes,
so
that's
been
pushed
out
for
a
number
of
different
reasons.
The
the
the
default
continues
to
be
openshift
sdn.
A
So
my
perspective,
my
personal
opinion
is
continue
to
use
the
default
unless
you
don't
have
a
reason
to
so.
If
we
share
my
screen
here
in
the
documentation
we
have
this,
you
know
what
are
the
different
features
available
and
you'll
notice
that
ovn
kubernetes
has
a
number
of
different
things
that
it's
capable
of
doing
so,
for
example,
ipsec
encryption,
ipv6
that
openshift
sdn
isn't
capable
of
so.
If
you
don't
need
any
of
these
features,
openshift
sdn
is
the
default,
makes
perfect
sense.
I'm
going
to
post
this
link
into
the
chat
here.
A
Openshift
sdn
continues
to
make
perfect
sense
and
the
good
news
is,
you
can
migrate.
So
if
you
deploy
openshift
sdn
today
and
you
discover
that
you
need
ovn
kubernetes,
you
can
migrate
in
between
them.
So
you
can
see
here
in
the
documentation.
The
very
next
page
is
migrating,
so
today,
right
this
moment,
the
only
thing
that
or
or
the
only
thing
that
comes
to
mind
that
would
prevent
you
from
doing
this.
A
Migration
is,
if
you
need
features
that
can
only
be
turned
on
at
install
time
and
specifically,
what
I'm
thinking
of
here
is
the
hybrid
networking
option.
That's
required
for
windows,
nodes
and
windows
containers,
so
that
has
to
be
turned
on.
You
know
it's
basically
an
option
when
you
do
the
install
it
can't
be
turned
on
day
two.
A
A
And
aside
from
that
yeah
what
about
trip
to
trip
to
tahiti
yeah?
I
wish
that
would
be
nice.
I
keep
trying
to
convince
my
wife
that
we
should
go
to
that
part
of
the
world
and
my
wife
is
not
a
flyer.
So,
while
I
have
been
to
you
know
the
the
apac
region
multiple
times,
I've
been
to
australia
and
all
that
a
few
times
and
absolutely
love
it
there.
My
wife
is
not
on
board
with
the
you
know:
14
15
hour
plane
ride
to
get
there.
A
A
A
I
know
calico
offers
a
lot
of
really
cool
things
so
like
they
do
all
layer.
3
sdn,
I
think
that's
one
of
their
options,
so
everything
is
routed.
Everything
is,
you
know,
there's
not
an
sdn
on
an
sdn
or
anything
like
that.
If
you
happen
to
be
using
another
sdn
at
the
hypervisor
level
stuff
like
that,
so
they
also
have
some
pretty
powerful
network
policy
stuff.
A
You
know
we
talked
was
it
two
weeks
ago,
three
weeks
ago,
when
we
had
the
vmware
folks
on
you
know,
we
talked
about
the
possibilities
there
with
antria
and
with
nsx
same
sort
of
thing.
You
want
to
use
those
capabilities-
great,
that's
wonderful!
It
will
absolutely
work
with
openshift
it's
certified.
I
think
we
have
a
whole
certification
program
for
that,
so
no
harm
in
doing
that.
A
Cncf
community
had
a
webinar
last
night
and
using
submariner
make
services
across
clusters
yep,
so
submariner
is,
as
far
as
I
know,
and
I
will
be
clear-
I'm
not
the
expert
here
as
far
as
I
know,
decoupled
from
the
specific
sdn.
So
essentially
it
creates
a
layer,
3
overlay
network
between
the
clusters
so
that
they
can
communicate.
A
A
So,
if
you
create
a
cluster
pool
inside
of
acm,
you
can
then
go
in
and
tick
the
box
and
say
deploy
submariner
and
then
it
will
deploy
it
across
there.
So
I'll
see.
If
I
can
get
some
more
details
there,
one
of
the
things
that
we
had
talked
about
in
the
not
too
distant
future
was
talking
about
service
mesh,
and
I
think
that
integrating
in
with
all
of
that
would
be
a
great
opportunity
to
talk
about
submariner
and
some
other
various
networkingy
things,
depending
on
the
scope
of
that
or
or
how
large.
A
That
topic
is,
we
may
even
have
a
dedicated
stream
for
it,
so
keep
keep
an
eye
out
for
that
one
and
of
course
I
always
try
and
remind
people
of
what
our
upcoming
topics
are.
When
I'm
paying
attention
to
my
own
schedule
ahead
of
time,
so
yeah
ebb,
ebpf,
yeah,
there's
lots
of
really
interesting
stuff,
that's
happening
in
that
area.
A
A
So
if
you're
not
familiar
openshift,
commons
is
kind
of
our
community
set
of
stuff.
So
the
the
commons
group
regularly
has
these
commons
gatherings
where
they
talk
about
a
bunch
of
different
topics.
This
one
just
happened
to
pop
up,
because
christian
is
a
friend
of
mine
and
he's
always
telling
me
about
get
ops
things.
But
definitely
if
you
have
some
time,
you
can
see
it's
right
when
our
stream
ends,
you
can
hop
over
to
hop
in
no
pun
intended
and
watch
their
event.
A
A
So
yeah
lots
of
really
good
and
interesting
stuff
christian
who
I
was
just
helping
him
review
a
chapter
for
his
upcoming
get
ops
book.
So
I
I
there
is
just
a
ton
of
stuff
and
if
you're
not
familiar
with
get
ops,
it's
definitely
something
that
I
I
am
encouraging
everybody
to
get
familiar
with.
Now,
you
remember
last
week
when
we
had
the
validated
patterns
team
on
remember,
all
of
their
stuff
is
driven
through
get
ops.
So
it's
just
becoming
more
and
more
prominent
and
I've
been
doing
a
lot
of
work.
A
This
work
this
week
with
acm
right
acm.
We
can
leverage
get
ops
in
order
to
deploy
applications
across.
You
know
clusters
as
they're
joining
not
just
applications
but
features
of
open
shift.
So
it's
it's
really
interesting.
There's
a
lot
of
power
behind
that
and
it's
only
getting
better
as
time
goes
on.
So
I
I
I'm,
I
guess
this
time
I
have
to
send
christian
the
check
because
I'm
the
I'm
the
fanboy
as
it
were
tiger,
I'm
a
flyer
yeah
and
I
know
tiger
you
live.
A
A
A
The
person
was
specifically
asking
about
prometheus,
but
really
it's
the
metric
service
broadly
and
the
answer
to
that
is
sort
of.
So
I
brought
up
the
documentation
page
here
which
is,
and
the
important
thing
here
is
we
we
can
effectively
set
the
operator
to
be
unmanaged
and
then
basically
have
it
uninstall
all
of
those
components.
A
It's
not
a
supported
option.
It's
not
a
supported
action
inside
of
the
cluster,
and
the
reason
for
that,
in
my
perspective,
is
because
there's
a
number
of
features
and
capabilities
that
rely
on
metrics.
You
know
horizontal
pall,
auto
scaler,
vertical
pod,
auto
scaler,
all
right.
All
of
those
things
rely
on
metrics
data
in
order
to
function.
A
A
Nate
is
choosing
a
third
party
sdn
and
install
time
decision
and
how
much
complexity
does
it
add
to
the
install
good
question
so
generally?
Yes,
it
is
an
install
time
decision,
I'm
trying
to
think
I
don't
know
of
any
off
the
top
of
my
head
where
you
can
migrate
after
the
fact,
but
considering
we
have
a
migration
from
openshift
sdn
to
ovn
kubernetes.
It
wouldn't
surprise
me
if
it's
possible,
the
complexity
is
really
going
to
vary
so
at
a
minimum.
A
A
You
know
the
vmware
nsx
container
plug-in
and
I
believe
that's
also
how
it
works
with
calico.
So
the
complexity
is
going
to
vary.
I
don't
know
if
flannel
is
a
supported
sdn,
but
that
one's
one
of
the
quote
unquote
simpler
ones.
You
know,
maybe
it's
a
little
bit
easier.
Maybe
it's
just
less
stuff
happening
underneath
the
covers,
I'm
not
quite
sure
there,
but
yeah
it
shouldn't
be
too
hard
compared
to
many
things,
just
an
extra
step
at
the
beginning.
A
A
Okay,
so
if
anybody
wants
to
take
a
look
at
that,
that
would
be
how
you
can
forcefully
disable
the
metric
service
inside
of
the
cluster
all
right,
so
I
I
built
it
up
a
little
bit
here
at
the
beginning
of
our
session
here.
You
know
talking
about
the
performance
add-on
operator,
and
this
is
one
that,
like
the
name,
is
really
intriguing,
and
you
know
it
sounds
like
it's
this
magic
tool.
You
know
like
like
tunde
right,
I
can
use
a
2d
profile
and
I
can
set
all
these
things
inside
of
my
cluster
and
magically.
A
A
So
while
it's
incredibly
powerful-
and
I
can
do
a
lot
of
stuff
inside
of
there-
it's
also
just
a
teensy
bit
dangerous
right.
We
all
like
to
live
a
little
bit
on
the
edge.
So
frank,
I'm
curious,
and-
and
thank
you
very
much
for
joining
me
today,
this
one
I
know-
we've
been
planning
it
for
a
little
bit
and
I
I'm
excited
to
learn
more
about
this.
So
can
we
start
with
like
what's
the
use
case
when,
as
the
product
manager
for
the
performance
add-on
operator?
B
B
B
It's
a
cloud
native
network
function,
which
is
basically
you
want
to
make
sure
that
you
will
dedicate
really
dedicate
some
resources
to
to
some
containers,
so
you
make
sure
that
your
container
will
never
be
preempted
by
the
scheduler
by
an
interrupt.
So
this
is
where
you
want
to
use
a
performance,
addon
operator,
basically
fine
tuning,
to
make
sure
that
you
can
really
dedicate
resources,
but
not
from
a
kubernetes
perspective,
but
also
from
an
operating
and
system
perspective.
B
B
You
have
other
ways
to
do
it
also,
but
the
thing
is
as
soon
as
you
start
to
configure
a
cubelet,
you
want
to
make
sure
that
you
have
one
source
of
cubelet
configuration
and
not
two
components
colliding
in
competition
and
at
the
end
you
end
up
with
something
not
coherent,
but
anyway
the
the
let's
say
that
if
you
think
that
you
need
a
tuned
d
profile,
so
first
you
need
to
know
what's
a
tund
profile.
If
you
need,
if
you
think
you
need
to
do
some,
I
need
to
do
some
interrupt
rebalancing.
B
I
care
about
numa
locality,
about
my
resources
and
and
finally,
potentially
I
need
to
have
a
specific
boot
parameter
on
some
worker
node.
This
is
where
you
should
think
about
pau
and
when
you
start
to
use
pio,
you
have
other
tools
that
you
use
to
potentially
use
to
configure
cubelet
and
you
should
forget
about
them,
because
pio
becomes
your
front-end
for
the
whole
configuration
and
if
you
you,
you,
you
configure
the
one
component
at
two
places.
Guess
what
happen.
A
B
And
it's
even
worse
than
that,
it's
you
want
also
to
configure
cryo
or
very
low
level,
cryo
parameter
as
well,
and
you
want
to
be
coherent
between
your
cryo.
Let's
say
you
want
us
some
specific
cpu
in
cryo
configuration.
You
want
the
same
in
the
boot
parameter
and
you
want
the
same
cpu
list
in
the
tund
profile
and
if
you
do
it
manually,
you
may
succeed,
but
you
will
need
to
rewrite
basically
pio.
A
B
If
you
want
to
change
the
cubelet
configuration
you
don't
want
to
go
via
mco.
So
mco
is
something
like
something
that
you
would
like
to
add.
Rather
than
configure,
I
would
say,
or,
and
you
we
we,
you
need
to
to
to
be
very
aware
about
all
of
the
consequences
about
what
you
do
so
yeah.
So,
first
thing:
first,
can
you
use
pau
nmcu?
Yes,
of
course,
we're
always
doing
that
to
set
the
same
parameter?
No.
A
B
A
B
A
B
Yes,
indeed,
so
the
performance
addon
operator
is
is
also
helping
to
configure
the
real-time.
No,
so
installing
the
real-time
kernel
is
one
thing,
but
then
you
need
to
have
all
of
the
associated
boot
parameter
the
tund
profile,
because
when
you
install
real
time
is
you
want
real
determinism?
You
need
to
have
a
cryo
config
with
or
without
hyper
threading,
because
you
can
be
more
or
less
soft
on
your
real
time,
so
the
best
practice
to
be
hard
real
time
is
you
disable
hyper
threading.
But
when
you
want
more
throughput,
you
enable
hyper
threading.
B
So
pao
can
do
that
for
you.
One
thing
we
didn't
talk
about
is,
if
you
think
about
disabling
hyper,
threading
pio
could
be
a
good
fit
and
some
application
are
faster
when
hyper
threading
is
disabled
and-
and
this
is
something
that
the
application
developer
should
know,
because
his
application,
for
instance,
requires
a
big
cash
right
yeah
and
when
you
have
hyper
threading
on.
Basically,
you
divide
your
cache
by
two.
A
Yeah,
it's
you
know,
I'm
an
old
old-school,
virtualization
admin
and
when
we
used
to
do
sizing
exercises
you
know
it's
a
a
hyper
thread
only
counts
as
half
of
cpu
when
you're
doing
sizing.
You
know
if
I
have
four
cores
with
hyper
threading
turned
on,
so
it's
eight
virtual
cores,
but
from
a
capacitive
perspective
I
only
want
to
count
that
as
six
the
four
full
cores
plus
two,
which
I
I'm
curious
to
see
or
curious
to
know.
A
B
A
pool
of
faster
cpu
yeah,
so
meaning
the
other
there
so-
and
this
is
something
that
we
work
on
upstream
with
intel
to
enable
it.
So
you
would
have
some
kind
of
first
first
class
workload
very
demanding
as
okay
give
me
those
cpus,
but
but
if
you
start
to
to
be
in
this,
if
you
want
to
consume
them,
you
may
have
a
very
specific
use
case
could
be
telco.
B
So
telco
can
be
a
radio
access
network,
as
you
said,
and
the
most
stringent
in
telco
is
a
vdu
as
we
call
it,
which
is
a
really
close
to
the
antenna.
And
then
you
have
the
cu,
and
then
you
have
the
core,
and
now
you
have
others,
okay,
so
the
real
time
is
really
for
the
really
close
to
the
tower
and
the
other
use
cases
so
far.
Don't
use
don't
use
real-time,
but
use
fine
tuning
and
and
with
pao.
B
So
if
we
step
back
just
one
thing,
kubernetes
is
about
sharing
and
cloud
native
telco
is
about.
Okay,
I
want
determinism,
so
it's
kind
of
the
opposite,
because
either
you
dedicate
either
you
share
okay.
So
the
idea
is
to
to
find
the
recipe,
the
trade-off
to
be
able
to
collocate
isolated
workload
and
committed
workload.
So
we
are
globally
efficient,
so
you
have
some
kind
of
I.t
like
pods,
that
can
be
over
committed
running
on
hyper
threats,
no
problem
and
you
have
the
others
that
cannot
suffer
from
an
extra
cash
miss
on
the
same
platform.
B
So
this
is
something
that
is
is
going
to
be
one
level
beyond
with
intel
ucpu,
and
do
you
want
this
cpu
to
run
more
over
committed
workload
or
dedicated
workload?
I
would
say
this
is
up
to
your
application,
but
when
you
start
to
think
about
it,
it's
really
application
specific.
I
I
think
that
fsi
hpc,
so
even
so,
it
really
depends
on
the
workload.
B
A
I
can
imagine
you
know
aiml
type
workloads,
all
of
that.
So
there's
a
couple
of
questions
here.
I'm
actually
going
to
ask
them
in
the
reverse
order,
because
I
think
nate
you
I'm
going
to
use
your
question
to
to
ask
frank
to
do
a
demo
on
there.
So
pnuw
asks
when
using
when
doing
net
app
modernization-
and
you
know
basically
having
windows,
server
windows.
Containers
in
the
cluster
does
performance
add-on
operator
work
with
those
windows
nodes.
A
B
A
Yeah,
so
I
thought
I
was
going
to
use
nate's
question
to
to
I
kind
of
ask
you
to
launch
into
a
demo
so
nate's,
asking
with
performance
add-on
operator.
Can
you
use
labels
node
selectors,
to
limit
the
nodes
that
the
the
profiles
are
being
applied
to,
and
I
I
I
suspect
that
if
we
look
at,
you
know
how
to
deploy,
how
to
use
how
to
apply
the
the
performance
add-on
operator
to
a
cluster
that'll.
Probably
answer
a
lot
of
those
questions.
B
Yes,
so
so,
unfortunately,
on
my
cluster,
I
only
have
two
worker
physical
node,
so
they
are
all
on
the
m.
They
are
on
the
same
machine,
configure
machine
mcp,
but
if
you
have
multiple
ncp,
basically
you
can
have
one
pa
or
one
profile
performance
performance
profile
per
mcp.
So
that's
as
many
as
you
want.
Okay,
you
even
have
inheritance
and
you
can
do
things
which
are
very
complicated
on
top
of
something
complicated,
so
we
can
go
through
my
setup
and
and
the
values
parameter
if
you
want.
A
B
And
the
point
is
what
I'd
like
to
to
to
to
to
to
talk
more.
If
it's
of
interest
of
the
audience
is
it's
about
the
use
cases,
and
what
does
it
mean
this
parameter
because
you
have
two
kinds
of
user?
You
have
users
which
come
from
a
very
low
level
and
discover
kubernetes,
and
they
know
what
they
want
to
do
exactly
and
say.
B
Okay,
how
do
I
translate
my
linux
configuration
into
an
openshift
configuration
and
I
can
help
these
people
to
understand
how
to
to
to
translate
and
go
on
top
of
the
layers,
because
there
are
a
lot
of
things
in
between
that.
You
need
to
configure
that
you
are
not
aware
of
when
you
join
the
kubernetes
ecosystem
like
cryo,
etc,
and
you
have
others
yeah
and
you
have
the
others
which
are
more
cloud
native
people
trying
to
discover
the
low-level
stuff.
B
A
Yeah
it
jumping
back
a
little
bit
when
you
said
that
you
can
have
a
a
performance,
I'm
going
to
call
it
a
profile.
I
don't
know
if
that's
the
right
term
per
machine,
config
pool
that
kind
of
makes
sense,
because
you,
you
know
you
would
assume
that
all
of
those
nodes
are
going
to
have
the
same
configuration
the
same
set
of
hardware,
the
same
set
of
things
so
that
to
me
that
that
is
logical.
A
I
also
like,
when
you
describe
it
as
as
two
groups
of
people
like
that,
because
I
think
I
probably
fall
into
that
latter
group,
which
is
you
know,
okay,
I
need
to
deploy
an
application
now.
I
need
to
you,
know,
tune
it
and
figure
it
out,
whereas
earlier
to
your
point
earlier
of
in
theory,
the
developer,
the
application
team
should
know
exactly
what
they
need.
So
they
should
be
communicating
that
and
kind
of
working
from
the
bottom
up.
B
So
if
we,
if
we
go
on
on
one
of
these
compute
node,
so
cats,
so
just
for
the
audience,
so
those
are
reference
machines
from
intel.
So
my
lab
entry
and
the
code
name
was
a
wildcat
pass,
so
wild
cat
pass.
So
I've
got
three
of
these
physical
machine
and
one
of
the
machine
is
hosting
my
three
virtual
masters.
B
B
B
Default
so
the
default
smp
affinity
is
it's
not
fff,
because
I've
got
already
pods
running,
so
this
is
something
that
I
will
explain
a
bit.
So
if
I
tear
down
all
of
my
pods,
you
will
see
that
the
mask
of
the
core
available
to
process
interrupt
is
ffff
and
as
soon
as
I
start,
an
isolated
pod,
the
the
cpu
is
not
available
for
interrupt
anymore,
so
they
are
pulled
away.
So
this
is
thanks
to
the
tunde
profile
that
that
is
applied.
B
B
I
want
to
configure
some
cctl
which
are
unsafe
so
because
my
cnf
wants
to
to
be
able
not
to
accept
router
advertisement
ipv6,
so,
okay,
so
I
can
do
that
with
pio
with
the
syntax.
So
with
this
cube,
so
it's
called
experimental,
because
we
have
hope
that
at
some
point
we
are
going
to
be
able
to
have
a
hierarchy
called
tree
like
rc.1
rc.2
for
cubelet,
but
this
is
a
long
shot
and
even
if
it's
experimental
here,
it's
fully
supported
by
by
our
teams
and
for
our
customers.
B
So
here
this
is
my
cpu
topology.
So
I've
got
a
dual
socket
system
with
all
of
these
cpu
on
socket.
So
this
is
okay,
zero.
This
is
socket
one
and
the
siblings.
Are
this
way
so
0
36,
so
that
guy
is
sibling
with
that
guy
I've
got
hyper
threading
on,
and
I've
split,
my
cpu
between
reserved
and
isolated,
so
reserved.
If
you
see
this
one
so
here
I've
chosen
to
to
to
to
let
a
two
cpu
per
luma
node,
that's
a
that's
a
lot.
B
Usually
you
you
just
let
one
okay,
just
it's!
For
the
sake
of
of
being
so
it's
to
do
some
r
d.
So
that's
my
lab,
so
one
one
thing
is
and
those
are
isolated.
So
when
you,
when
you
look
at
isolated,
you
think
okay
they're
isolated,
so
I
cannot
use
them,
and
this
is
where
sometimes
the
labeling
is
misleading.
B
These
cpu
are
candidate
to
be
isolated,
but
until
the
pod
is
isolated,
they
are
going
to
run
whatever
needs
to
be
run.
So
this
is
what's
a
little
tricky.
Isolated
is
not
really
isolated,
and
this
is
a
kubernetes
syntax.
It's
not
a
pau
syntax.
So
then
I
configure
the
huge
pages
so
64
in
total,
so
32
32.
This
is
how
it's
working
so
one
gig
huge
pages.
I
don't
like
to
make
just
huge
pages.
A
B
A
B
B
B
B
So
that's
a
translation!
Look
aside
buffer!
So
it's
it's!
It's
when
you
have
a
you
know
that
your
virtual
mem,
so
it's
all
back
to
the
concept
of
physical
memory
and
virtual
memory.
So
your
cpu
manage
virtual
memory,
but
it's
backed
by
physical
memory
and
you
need
to
do
the
mapping
right
and
to
do
the
mapping.
You
have
page
tables
and
every
time
you
go
because
you
want
to
resolve
a
physical
address
to
virtual
address
or
you
you
may
have
a
page
which
is
missing.
So
you
you
know
when
you
go
through
the
memory.
B
A
You
don't
share.
This
is
all
interesting
to
me.
You
know
I
I
have
I've.
Had
some
awareness
of
pneuma
and
kind
of
the
you
know
when
folks
would
want
to
use
it
again,
I'm
an
old
virtualization
administrator.
So
you
know
we
would
have
workloads
that
need
you
know,
pneuma
awareness
that
need
to
be
penned
to
a
specific
pneuma
node.
You
know
for
that
performance,
but
learning
the
details.
This
is
all
super
interesting
to
me.
A
A
B
B
Exactly,
and
here
be
mises
is
what
you
want
to
avoid,
because
this
is
one
of
the
most
costly
event
that
you
can
have
because
you're
executing
code
and
then
you
access
to
memory
and
boom.
You
have
you
have
to
walk
through
the
pages
and
you
have
like
five
cache
misses.
Four
cache
misses
worst
case
so
because
we
have
no
virtualization
if
you
with
virtualization
it's
even
worse,
because
you
can
have
like
up
to
16.
B
so
by
so
you
want
to
have
huge
pages
and
about
single
luma,
so
kubernetes
permit
to
have
so.
You
have
various
policies
with
the
topology
manager
and
you
can
ask
the
topology
manager
to
say
okay,
my
pod
will
have
all
of
these
its
resources
on
the
same
luma
node.
What
are
the
resources
memory
devices
cpu?
B
B
So
it's
so
because
of
generation
after
generation.
So
the
the
chief
vendor
there,
the
the
they
have
done,
a
great
work
and
the
throughput
is,
is
okay,
but
still
you
have
latency
penalty
and
lentency
matters,
because
it's
not
only
it's
not
only
how
many
packets
per
second
I
can
get
in
and
out
of
my
network
application.
B
It's
also
the
latency
in
particular,
when
we
think
about
5g.
We
have
a
very
limited
quota
in
term
of
time
to
to
get
through
the
whole
run,
to
the
run,
to
the
do
the
roll
core.
So
the
packets
needs
to
be
to
be
processed
and
sent
out
on
the
wire
as
soon
as
possible,
because
your
quota
is
ridiculous
compared
to
4g
so
single
pneuma.
It's
really
locality
between
memory,
cpu
and
devices
devices
being
in
in
the
case
that
primarily
network
interface,
but
it
can
be
gpu.
A
Yeah,
I
know
even
some
high
performance
databases
benefit
from
from
like
new
awareness
and
all
that
and
p
and
uw.
Thank
you
for
linking
the
documentation
there.
That's
that's
very
helpful.
So
frank.
I
feel
like
I've,
I've
distracted
you
long
enough.
I
I
want
to
make
sure
that
you
have
time
to
to
finish.
B
B
Okay,
so
one
thing
which
is
very
important
is
in
pao.
You
want
to
make
sure
that
all
cpu
are
listed
here,
because
if
you
forget
one
cpu
kubernetes
is
going
to
schedule
workload
on
it
and
there
will
be
absolutely
no
tuning
applied
and
it
can
and
kubernetes
may
schedule
an
isolated
pod
with
containers
that
require
isolation.
And
it's
not
going
to
be
isolated.
B
So
you
want
to
make
sure
that
you
all
of
your
core
are
isolated
or
reserved.
No
core,
no
cpu,
left
behind
and
again
isolated
does
not
mean
isolated,
it's
candidate
to
run
isolated
workload
and,
as
I
can
show
you
by
killing
my
pod
after
you
will
see
that
my
interrupt
my
interrupt
mask
is
going
to
be
increased
to
cover
all
of
the
all
of
the
cpu
as
soon
as
my
isolated
pod
will
be
destroyed.
So
it's
really
dynamic.
A
B
And
and
finally
something
I
I'm
going
to
show
you
on
the
on
my
on
my
workout,
which
are
related
to
interest.
So
why
am
I
talking
so
much
about
interrupt
so
interrupt
are
asynchronous
events
that
are
going
to
trigger
kernel
activity
to
process
packets
typically
or
to
do
I
o
when
you
have
an
interrupt
related
to
a
disk
and
every
time
you
run
the
kernel?
Well,
you
don't
run
your
application,
so
you're
you
still
cycle
from
an
application
that
needs
hundred
percent
of
the
available
cpu.
B
B
You
want
to
make
sure
that
you
can
pin
this
interrupt
to
a
cpu
which
is
not
isolated,
but
you
have
a
constraint
which
is
on
a
given
cpu.
You
cannot
put
more
than
224
interior
because
that's
a
hardware,
that's
that's
part
of
the
silicon,
so
you
have
a
limit
at
250
255,
but
you
have
32
already
taken
so
224..
B
B
So
basically,
with
user
networking
equal
true,
you
are
going
to
spread
your
interrupt
on
this
reserve
cpu,
at
least
because
you
are
going
to
spread
your
interrupt
on
more
or
more
cpu,
basically,
all
the
all
the
reserve,
plus
all
the
cpu,
which
are
not
running
isolated,
bots.
Okay,
I'm
going
fast,
it's
complicated
well.
A
And
just
to
be
clear
so
effectively
without
having
those
cpus
reserved
for
interrupts,
it
has
kind
of
two
effects
right,
so
one
is
it
has.
It
literally
interrupts
the
processing
of
the
application
which
potentially
slows
down
the
application,
but
two
because
you're
having
to
rely
on
the
cpu
scheduler
across
you
know
all
of
those
already
busy
cpus
with
the
application.
It
can
potentially
slow
down
things
like
network
throughput.
You
know
so,
if
you're
pushing
100,
gigabit
or
something
like
that
and
your
cpus
are
running
near
capacity,
suddenly
you're
not
able
to
hit
that
throughput.
B
B
So
in
telco
we
have
a
very
popular
application,
which
is
the
dpdk
test.
Program
is
namedtestpmd
and
I've
got
a
lot
of
annotation
here.
But
what
matters
in
the
in
this
in
this
container?
It's
it's
a
it's
like
it's
a
it's
dedicated
resources
right,
so
you
have
a
east.
You
have
east
and
west.
Sorry,
because
what's
running
is
not
what
you
see
because
I
got
too
many
machines.
B
So
this
is
that
issue
as
well.
So
this
is
this,
is
this
is
the
right
one?
So
so
what's
interesting
here
is,
for
instance,
okay.
This
is
a
cctl
that
you
have
seen
and
I'm
setting
the
value
to
zero,
and
I
can't
do
that
because
I've
put
that
in
in
in
pio,
so
it's
a
safe
and
here
so
I'm
having
four
cpu
two
pages
of
one
gig
of
huge
pages
and
one
gig
of
generic
memory,
because
you
always
need
a
good
memory.
B
B
What
I
say
is
interrupt
load
balancing
disabled,
so
I
disable
the
so
this
pod.
What
I'm
saying
is
when
you
start
this
pod,
please
move
all
interrupts
away
from
my
cpu
okay,
so
this
board
has
four
cpu
no
interrupt
on
this
for
cpu,
so
in
in
the
former
version
of
openshift
b4.7
by
default.
That
was
a
general
option,
but
now
it's
opt
out
so
each
and
every
pod
needs
to
say.
Okay,
I
want
to
get
the
c
interrupt
out.
B
Then
this
one
is.
We
have
a
bug
in
cryo
that
is
going
to
be
fixed
upstream
at
some
point.
But
until
this
bug
is,
is
a
fix.
You
need
to
put
this
quota
disabled,
because,
even
if
you
request
a
complete
cpu
cryo
is
going
to
cpu
throttle
your
workload.
So
this
is
bad,
so
you
need
to
put
this
option
and
finally,
you
can
also
unplug
the
cfs,
though
so,
the
linux,
fair
scheduler
that
so
here
I've
got
four
core
four
cpu,
two
cores
for
hyper
threads.
B
B
So
here
what
you
say
to
the
cur,
so
basically
you
disable
the
fair
schedule.
So
once
the
task
is
started
on,
one
cpu
is
going
to
stay
on
the
cpu
until
it
dies,
but
the
application
on
top
can
do
via
task
set
the
pinning
on
another
cpu.
So
it's
not
like
so
the
the
linux
scheduler
is
not
going
to
move
the
application
to
another
cpu,
but
if
the
application
wants
to
it's
possible
also,
the
application
can
start
can
start
yeah.
B
B
B
And
as
you
see,
any
cpu
can
take
interrupt
again.
So
every
time
you
put
an
isolated
pod
with
a
proper
annotation,
this
mask
will
be
updated
and
all
interrupts
of
the
system
are
going
to
be
moved
of
the
cpu
which
have
been
allocated
to
this
pod.
So
to
so,
these
cpu
are
really
isolated
from
kernel
isolation.
Of
course
they
are
isolated
from
other
pods
from
kubernetes.
B
A
So
you
can
tune
in
in
two
weeks
we'll
talk
about
it
at
the
top
of
the
stream
during
the
top
of
mind
or
feel
free
to
send
me
an
email,
andrew.sullivan,
redhat.com
and
as
soon
as
I
can
get
that
answer
out
to
you
also
included
in
the
blog
post.
That
follows
each
one
of
these
streams
as
well,
so
keep
an
eye
on
cloud.redhat.com
blog
mark
curious
to
know
about
hyper
threading
for
an
open
shift
cluster
running
in
virtual
machines.
B
When
you
run
in
a
virtual
machine,
you
run
on
a
virtual
server,
a
server
is
a
server,
so
this
server
is
exposing
you
hyper
threads
or
not.
So
it
depends
on
how
linux
is
configured
and
the
hypervisor
is
configured
to
show
you
cpu,
which
are
number
zero
one,
two
three
four
or
something
a
little
more
fancy
showing
you
up
a
new
topology
and
hyper
threading.
B
So,
basically
you
know
what's
a
vm,
it's
just
like
you
enable
vtx,
and
you
run
the
same
thing
so,
of
course,
and
by
the
way
we
do
that
on
openshift
on
openstack
we
have
a
complete
end-to-end
tuning
for
telco,
so
hit
apply.
So
I
I
did
a
demo
of
this
before
coved
in
denver.
So
if
you
search
for
my
name,
open
interest
submit
denver,
we
did
a
demo
live
on
stage
that
was
kubernetes
on
openstack
and
to
end
with
all
of
the
tuning
manually
applied
without
bao,
and
I'm
glad
we
have
pio
now.
A
A
I
noticed
that
the
recommended
setting
for
x86
architecture
is
one
thread
per
core
cpu,
so
in
that's
going
to
vary,
so
I
think
what
you're
asking
is
you
know
if
the
physical
server
has
hyper
threading
enabled
the
hypervisor
is
taking
advantage
of
that
you
know.
Should
you
basically
take
advantage
or
use
those
hyper
thread
cores
for
your
your
openshift
nodes,
and
the
answer
to
that
is
maybe
right.
It'll
depend
on
a
bunch
of
different
things
in
your
application.
A
A
You
know
in
rev
there's
things
with
I
o
threads
as
well
as
cp
reservations
and
threads
per
cpu
as
you're
calling
out
there.
That
can
all
affect
that
performance.
So
if,
if
you
are
seeing
performance
degradation
in
openshift
as
a
whole
or
or
most
especially,
the
control
play
nodes
utilize,
those
hypervisor
features
to
kind
of
dedicate
it
or
give
it
more
priority.
Inside
of
the
resources
that
it
has,
we
have
built
our
cluster
before
the
rev
integrations,
so
I
don't
think
the
red
hat
virtualization
vms
are
using
anything
special.
A
So
please
reach
out
to
me.
We
have
a
the
the
rev
engineering
team,
they're,
really
good
about
answering
those
questions
and
they're
really
curious
about
that
stuff.
So,
and
I
know
the
product
manager
would
be
very
interested
in
any
use
cases
or
concerns
you
have
there
so
mark
feel
free
to
send
me
an
email,
andrew.sullivan
redhat.com.
I
posted
it
just
before
your
chat's
there.
Thank
you
stephanie
for
putting
my
contact
information
up
there.
That
works
too.
A
Excuse
me,
so
I
know
we're
we're
about
six
minutes
over
according
to
my
clock
here
I
don't
want
to
intrude
into
your
evening.
I
know
over
in
france,
it's
it's
getting
later
in
the
day,
and
I
definitely
don't
want
to
to
interrupt
your
your
your
off
time.
Your
family
time,
frank,
so
if
anybody
has
any
last
questions,
please
go
ahead
and
submit
those
into
the
chat.
You
know
we'll
address
those
as
they
come
in.
B
So
I
recommend
people
to
read
the
pao
documentation.
The
page
is
really
long.
It
reminds
me
of
the
of
the
tcp,
the
tcp
code
of
the
bsd
stack,
because
it's
one
file
with
everything
in
all
the
tcp
stack
in
one
file,
so
it's
pretty
the
same
size,
but
everything
is
in
there
and
at
the
bottom.
You
will
have
something
that
you
know
when
people
talk
about
a
lot
about
devops,
but
at
the
end,
what
the
best
practice
is
when
you
do
something
you
want
to
test
that.
What
you?
What
you
get
is
what
you
expect.
B
So,
when
people
use
pau,
you
need
to
think
about.
Okay.
How
am
I
going
to
make
sure
that
I've
got
the
isolation
I
want?
And
in
this
page
you
will
have
some
handy
tool
explaining
you,
okay,
how
to
make
sure
that
okay,
if
something
was
wrong,
you
run
a
tool.
You
will
see.
I
see
cpu
preemption,
that's
bad!
If
this
is
what
you
want
to
see,
for
instance,
so
it's
it's
not
only
about
understanding
it.
A
All
right
well,
thank
you
so
much
for
joining
today.
This
has
been
like.
I
said
it's
been
a
really
interesting
one,
and
I've
learned
a
lot
about
lo,
the
the
lower
level
performance
tuning
performance
capabilities,
stuff
that
I've
had
I'm
gonna,
say
the
luxury
of
not
having
to
pay
attention
to
over
the
last
few
years,
but
nevertheless,
it's
still
really
interesting
to
me
again,
it
punches
all
those
buttons
for
me,
the
the
nerd
knobs,
so
to
speak
for
our
audience.
Thank
you
for
joining
us
today.
A
We
really
do
appreciate
it
again.
If
you
have
any
questions,
if
there's
anything
that
comes
to
mind
after
you're
after
you
have
watched
the
stream
or
if
you're
watching
it
not
live,
feel
free
to
reach
out.
You
can
reach
me
on
social
media,
at
practicalandrew,
on
twitter
or
via
email,
andrew.sullivan,
redhat.com
and,
of
course,
I'm
I'm
more
than
happy
to
to
bring
frank
into
the
conversation.
A
If,
if
needed,
to
help
answer
those
questions,
one
last
reminder
we
won't
have
a
stream
last
or
excuse
me
next
week
because
of
the
what's
new
presentation
be
sure
to
tune
into
that
in
two
weeks,
we'll
be
back.
Johnny
will
be
back
off
of
pto,
so
it'll
it'll
be
the
whole
crew
here
stephanie.
In
the
background.
Thank
you
so
much
for
all
of
your
help
today
and
posting
links
and
keeping
us
on
point
and
with
that
I
hope
everybody
has
a
great
rest
of
their
week
great
weekend
and
stay
safe
out.
There.