►
From YouTube: What’s in the Box? Resource Management in Kubernetes - Louise Daly & Ivan Coughlan, Intel
Description
Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18).
What’s in the Box? Resource Management in Kubernetes - Louise Daly & Ivan Coughlan, Intel (Intermediate Skill Level)
As the cloud native ecosystem evolves, it is determined to align platforms and technologies into a well-orchestrated infrastructure for optimum utilisation of compute, network and storage. This infrastructure needs to cater to the needs of a variety of workloads in fields such as telecomms, scientific computing, machine learning, financial services and data analytics. In order to provide the best performance for such workloads, features to provide performance isolation (e.g., CPU pinning, huge pages and accelerators) are required. The resource management working group (RMWG) in Kubernetes is working on the design, development and implementation of such features through the introduction of CPU manager, Huge Pages and Device Plugins. In this presentation, we will demonstrate how to leverage some introduced by RMWG via specific use cases from networking and machine learning (ML).
About Louise
Software Engineer at Intel. Working towards cloud native orchestration of workloads, currently looking at enabling compute resources in Kubernetes to improve application performance, provide predicable performance and effectively manage server resources.
About Ivan
Ivan Coughlan is a Senior Software Architect working for the Intel Software Defined Datacenter Solutions Group (SDSG) with a broad range of experience from embedded real time product development for the Telecoms industry to manufacturing IT systems development, integration and control.Currently, Ivan is working for Intel Corp on next generation network platform enabling in Cloud Computing environments and specifically on the enabling of Cloud Native environments for Telecoms NFV use cases.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
Join us for KubeCon + CloudNativeCon in San Diego November 18 - 21. Learn more at https://bit.ly/2XTN3ho. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Okay,
so
my
name
is
Louise
Daley
I'm,
a
software
engineer
in
Intel
and
my
co-speaker
is
sitting
down
in
the
crowd.
Ivan
Catherine
he's
our
architect
and
I
just
want
to
call
out
Balaji
who
was
my
original
co
speaker,
but
he
couldn't
actually
attend
the
event
so
I
just
like
call
him
out.
He
would
have
been
the
machine
learning
aspect
on
the
abstract
on
the
networking
side
so
for
those
that
were
waiting
for
machine
learning,
there's
not
much
in
here
so
today,
we'll
be
talking
about
what's
in
the
box
resource
management
in
kubernetes.
A
So
the
next
slide
is
just
noticing
disclaimers.
So
the
agenda
for
today
I'm
first
gonna
talk
about
the
current
status
of
resource
management
and
Kure
is
just
a
basic
overview
of
how
it
works
today
and
what's
available
and
then
I'll
go
on
to
the
challenges
we
found
that
our
customers
had
with
this
current
state
of
kubernetes
resource
management
and
why
they
needed
it
to
change
and
then
I'll
go
into
the
steps
we
have
taken
with
the
community
to
make
those
changes.
A
So
these
include
features
such
as
node
feature
discovery,
CPU,
pinning
huge
pages
device,
plugins
and
numa,
and
then
going
to
go
into
a
quick
demo
of
our
experience
kits
for
containers
bare
metal,
so
I'll
show
how
CPU
pinning
benefited
and
networking
workload.
So
that's
the
plan
for
say
and
then
I'll
leave
some
time
for
questions
at
the
end.
A
So
first
up
is
a
lot
of
you
may
know
this,
but
it's
just
how
a
resource
compute
resource
management
works
in
communities
today.
So
we
can
see
on
our
left.
We
have
our
pod
specification
and,
in
its
specification,
has
resource
requests
for
CPU
and
memory.
These
are
the
core
compute
resources
available
in
kubernetes
today,
so
our
pods
requesting
a
thousand
mini
CPU
and
500
Meg's
a
memory.
So
this
pod
gets
into
the
kubernetes
scheduler,
where
the
Challenger
looks
at
the
nodes
in
the
costume
decides
where
I'll
place
that
pod.
A
So
it
looks
at
the
allocatable
on
each
node,
so
the
allocation
will
is
the
total
available
compute
resources
to
the
community
scheduler
that
it
can
run
in
pods
on
so
there's
a
total
capacity
available
on
the
node
as
well,
and
this
is
for
either
the
system
or
the
cube
of
the
queue
processes
to
run
on.
So
these
are
taken
away
from
allocatable
as
well
as
this
security
scheduler
will
take
into
account
the
resources
being
used
by
pods
running
on
the
node.
A
So
in
our
case
here
node
one
has
2000
milli,
CPU
and
512
Meg's,
a
memory
bush
512,
milli,
CPU
and
256
Meg's
of
memory
are
being
used
by
pod
B,
so
I'll
take
that
into
account
when
it
makes
its
decision.
So
in
this
case,
our
only
available
node
for
scheduling
for
our
pod
is
at
node
2.
So
that's
the
current
state
of
kind
of
compute
resource
management
in
communities
today.
So
what's
wrong
with
this.
So
as
kubernetes
clusters
are
growing,
heterogeneous
nodes
are
being
introduced.
A
So
not
all
hard
worker
capabilities
are
the
same
across
your
cluster
and
kubernetes
has
no
insight
into
this,
and
it
has
no
age
for
a
workload
to
request
that
needs
specific
hardware
or
needs
a
specific
capability
on
a
certain
node.
So
we
want
to
extend
the
kubernetes
resource
management
to
take
into
account
the
many
different
resources
available
on
the
cluster
and
then
enable
our
users
to
request
that
their
workloads
require
a
certain
level
of
resources.
A
So
there's
a
bunch
of
challenges:
we've
been
trying
to
address
mainly
for
networking
applications.
Vnfs
virtual
networks
functions
on
query,
so
there's
a
bunch
of
these,
so
some
of
them
include
multiple
networks
for
vnf,
high
performance
data,
plane,
east
and
west,
north
and
south,
and
then
we
have
ability
to
request
and
allocate
platform
capabilities,
CPU
pinning
in
isolation
dan
I'm,
a
huge
page
allocation,
I'm,
not
sure
if
you
can
see
that
one
is
device
plugins.
So
it
discovery
at
advertisement,
scheduling
and
management
of
devices
and
guaranteed
new
born
old
placement
and
in
platform
telemetry.
A
So
these
challenges
kind
of
fall
into
different
buckets,
so
if
we
put
them
in
so
the
first
is
kubernetes
networking
data
playing
acceleration
resource
management
or,
as
we
call
it,
enhance
platform,
awareness
and
telemetry,
so
today,
I'll
obviously
be
focusing
in
on
that
yellow
bucket
there
of
source
management.
So
this
might
be
a
little
hard
to
read.
Sorry
can't
so.
Each
we've
implemented
a
set
of
solutions
that
are
all
open
source.
A
We
aim
to
get
everything
into
upstream
kubernetes,
but
if
not
we'll
open
source
a
project
for
it,
so
for
the
resource
management
set,
we
have
a
node
feature
discovery,
which
is
a
KU
mary's
incubator
project.
So
an
incubator
project
is
one
that
needs
to
get
community
support
and
backing
in
news
cases
to
get
pushed
into
communities
upstream.
So
that's
currently
there
since
November
2016,
and
we
still
don't
know
when
it'll
be
graduated
and
I,
believe
that
the
incubator
may
be
deprecated.
A
A
But
that's
currently
in
proposal
stage,
still
being
discussed
so
I'll
just
go
into
each
of
these
in
detail
in
the
resource
management
pocket
now.
So
first
up
is
node
feature
discovery.
So
what
node
feature
discovery
does
is
discovers
the
capabilities
of
hardware
configuration
our
capabilities
on
each
node
in
the
cluster,
and
then
it
advertises
those
as
node
labels,
so
pods
can
make
requests
to
be
landed
on
nodes
with
certain
hardware
capabilities.
So
we
can
see
here
in
our
diagram.
A
We
have
a
node
feature,
discovery
NFT,
pod
running
on
each
node
in
the
cluster
and
if
T
can
be
deployed
as
a
daemon
set
running
continuously
on
each
node
or
it
can
be
deployed
as
a
one
shot
job
just
discovering
at
the
current
capability
on
the
node
and
then
labeled
a
node
as
I
said,
with
different
capabilities.
So
in
this
example,
our
node
2
as
SR
IV,
secure
boot
boot
card,
a
VX
and
turbo-boost
enabled
and
then
for
node
1
has
nothing
enabled.
A
So
then
our
application,
our
pod,
can
request
via
affinity
RN
to
affinity
for
labels
in
its
pod
spec.
So
it's
requesting
to
be
placed
on
a
node
that
has
SR
IV
and
boot
guard
enabled.
So
then
our
scheduler
will
see
that
it's
requesting
this
capability
and
Lander
firepower
on
node
2
and
then
the
other
application
which
isn't
requesting
anything
can
land
on
either
node.
It's
not
stopped
for
lending
on
the
node,
with
the
the
abilities,
so
I
just
want
to
give
you
a
actual
use
case.
We
have
for
NF
t.
A
So
in
the
case
of
your.
If
you
have
an
IG
b,
u
io,
base
DVD
k
application.
You
don't
want
it
to
run
on
secure
boot,
enabled
systems
as
the
kernel
doesn't
allow
it.
So
we
can
use
something
called
node
anti
affinity,
so
we
match
a
label
on
a
node
and
we
say
that
we
don't
want
to
land
on
any
nodes
that
have
this
capability.
So
this
will
give
us
the
assurance
that
our
IG
v-
u
io
activity,
K
applications
won't
land
on
nodes
that
they
can't
run
on.
A
So
this
previously
wasn't
possible
in
in
kubernetes.
So
as
I
said,
and
if
T
is
a
an
incubator
project,
so
we're
currently
kind
of
looking
for
use
cases
and
walk
apps,
it
fills
and
why
there's
something
that's
useful
to
the
community.
So
if
anyone
has
good
use
cases,
we're
really
really
interested
to
hear
from
you.
The
link
is
on
the
sides
which
are
available
as
well.
So
if
you
have
any
issues,
please
log
them
with
us.
A
A
It
uses
c
RDS
for
logging.
What
nodes
have
what
cpu
pools
and
whether
garbage
collection
has
been
done.
What
tasks
have
been
cleaned
up
so
how
cmk
works?
Internally,
then?
Is
it
there's
three
different
pools?
The
first
pool
is
for
exclusive,
isolated
CPUs,
so
in
an
isolated
CPU
I
mean
one
isolate
V
via
currently
ISIL,
CPUs
and
exclusive
means
that
when
I
were
close
requests
it
gets.
No
other
process
is
going
to
run
on
that
CPU.
So
that's
the
first
pool
the
second
pool
is
exclusive
or
isolated
and
shared.
A
So
basically
it's
a
set
of
isolated
CPUs
that
can
run
complimentary
workloads,
so
they're
they're
shared.
But
the
understanding
is
that
the
workloads
will
work
well
together,
as
you
place
them
on
this
isolated
surface
CPUs
and
then
our
third
pool
is
the
infrastructure,
so
it's
just
shared
and
on
isolated,
regular
on
the
rest
of
your
processes.
So
cmk
allows
you
to
request
to
be
placed
on
one
of
these
pools
and
then
it
will
place
your
container
on
this
pool
and
as
well
as
this
another
feature
that
cmk
has
is
thread
level
pinning.
A
So
you
can
request
to
place
your
container
on
exclusive,
isolated
core,
but
cmk
won't
do
the
actual
task
set.
It
will
leave
that
up
to
the
the
process,
so
it
can
pin
a
particular
thread
to
an
exclusive,
isolated
core
and
the
rest
of
the
threads
will
run
on
your
infrastructure
pool.
So
your
shared
on
isolated
course,
so
the
the
one
of
the
major
use
cases
for
it
is
your
noisy
neighbor.
So
currently
you
could
get
placed
as
you
can
see
in
the
top
one.
A
You
could
have
your
performance,
sensitive
workload
or
any
workload
running
on
a
CPU
zero
which
might
be
hyper
threads,
a
blink
and
then
you'll
have
your
noisy
neighbor
workload
come
along
and
land
on
the
other
hybrid
siblings.
So
it
will
basically
trash
the
performance
of
your
performance,
sensitive
workload.
So
there's
no
way
of
currently
stopping
that
name
kubernetes.
So
then,
with
cmk
you'll,
obviously
get
your
exclusive
isolated
core
here
for
your
poor
insensitive
and
then
the
noisy
neighbor
will
run
somewhere
else.
So
it
won't
affect
your
performance,
sensitive
workload.
A
So
then,
once
we
ops,
I
open
sourced
cmk,
we
worked
on
getting
these
features
into
kubernetes
natively.
So
we
worked
with
the
community
to
add
CPU
manager
natively
in
coop
cublas.
So
what
this
does
is
currently
there's
a
static
policy
for
CPU
pinning.
So
if
you
make
an
integral
request
for
CPU,
so
either
one
two
three
CPUs
in
your
pods
back
you'll,
get
pinned
to
a
CPU
and
you'll,
get
guaranteed
pod
level
and
CPU
pinning
so
we're
hoping
to
extend
the
CPU
manager
to
take
care
cater
for
more
of
the
cmk
features
in
the
future.
A
So,
as
I
said,
the
cmk
is
open
source
and
available
today
and
then
CPU
manager
in
kubernetes
is
available
off
as
of
one
daddy.
So
next
I'm
going
to
talk
about
huge
page
support.
So
previously
you
had
to
manage
huge
pages.
The
coaster
off
would
have
to
manage
the
number
of
huge
pages
available
on
the
node
manage
applications.
How
many?
A
How
many
huge
pages
an
application
is
using
and
then
figure
that
how
many
more
they
can
run
on
it
and
they
also
had
to
manage
the
huge
table
FS
and
just
assume
good
behavior
by
applications
sharing
the
huge
table
efest.
So
we
worked
again
with
the
community
to
introduce
huge
page
support
natively
in
Corelli's.
So
now,
pre-allocated
huge
pages
are
our
first-class
resource
in
communities.
So
you
can
request
them
in
your
pods
back
along
with
CPU
and
memory
as
well
as
this,
we
extended
the
empty
d'oeuvre
volume
to
have
a
huge
huge
pages
type.
A
So,
basically,
there's
a
provocation
will
FS
into
your
container
of
the
sizes
huge
pages
you
requested.
So
this
means
that
your
coaster
off
no
longer
has
to
manage
that
huge
table
of
s
as
its
also
cleaned
up
when
the
pod
dies,
so
that
support
is
there
as
of
Cooney's,
what
not
H
as
well-
and
it's
just
it's
just
recently
moved
to
beta,
as
well
as
a
110.
A
So
next
I'll
just
give
a
quick
overview
of
device
plugins.
So
they
see.
If
you
had
wanted
to
add
your
device
to
kubernetes,
you
would
have
had
to
fork
the
kubernetes
code
and
right
into
cube
list
to
give
your
devices
to
a
pod.
So
there
was
an
effort
made
to
create
an
extensible
api,
so
a
plug-in
based
system
so
that
vendors
could
advertise
and
manage
their
devices
in
communities
via
a
plugin
without
having
to
accept,
extend
the
kubernetes
code
itself.
So
that's
where
the
device
plug-in
API
came
in.
A
So
basically
it
allows
you
to
advertise
the
number
of
devices
you
have
available
on
the
node
via
extended
resources
and
then,
when
a
device
resource
request
is
made,
a
call
is
made
out
to
the
device
plugin
where
you
can
do
any
necessary
allocation
of
the
device
to
a
particular
container.
So
you
can
change
the
C
group
for
container
to
mount
in
the
device.
You
can
change
mounts
and
you
can
add
vibrant
variables
to
indicate
to
the
container
what
device
it's
been
allocated.
A
So
this
is
in
since
1.8
as
I
said.
So
it
gives
you,
obviously
a
better
resource
utilization
and
reduces
overhead
of
having
to
maintain
additional
code.
So
again,
I'll
give
an
actual
use
case.
We
have
for
device
plugins
in
kubernetes,
so
qat,
which
is
quick
assist
technology
is
basically
an
offload
card
created
by
Intel,
and
it
allows
you
to
offload
cryptographic
or
compression
were
close
to
this
qat
card
and
alleviate
stress
on
your
CPU
for
other
tasks.
A
So
acuity
also
takes
benefits
from
SRA
V,
so
you'll
have
a
single
QT
device
on
an
older
and
you
may
have
multiple
SRA
v
v
FS
of
your
qat
device.
So
with
the
device
plug-in
were
able
to
advertise.
Those
v
FS
is
available
on
the
node,
V
extended
resources,
and
then,
when
an
allocation
request
is
made,
we
can
allocate
the
VF
to
a
particular
container
and
as
well
as
that,
we
can
advertise
the
the
PCI
address
of
the
VF
to
the
Container.
A
So
we
can
do
whatever
offloading
it
needs
so
that
this
code
is
currently
we're
working
to
open
source.
It
is
not
currently
available
and
there's
also
a
slight
limitation
in
that
the
device
plugin
doesn't
cater
for
the
allocate
situation.
So
we
can't
deallocate
the
VF
at
the
moment,
so
we're
hoping
they'll
extend
the
API
to
to
do
so
and
if
you
want
any
information
on
QT
I've
added
those
links
in
there
as
well.
A
Ok,
so
next
is
a
new
manager
for
kubernetes,
which
is
a
proposal
we've
created.
So
it's
the
introduction
of
all
these
components
that
are
actually
our
resource
aware
in
kubernetes
oops.
Sorry,
we
need
some
way
of
coordinating
their
allocation
because
currently
they're
I'm,
currently
there
they
don't
talk
to
each
other
and
there's
no
way
of
communicating
what
has
been
allocated
to
it
to
a
container.
So
the
first
case
we're
trying
to
solve
is
CPU
and
device,
which
are
the
two
components
that
have
been
added
so,
for
example,
in
lis.
A
If
your
container
requests
that
it
wants
to
be
pinned
to
a
CPU
and
it
wants
to
use
qat
device,
there's
no
way
of
telling
cubelet,
I
allocate
those
Numa
local
to
each
other.
I'll
just
allocate
them,
and
so
this
could
affect
the
performance
of
your
workload
as
it
has
to
go
over
the
the
interconnect
between
the
sockets,
and
so
the
idea
for
a
new
manager
is
to
create
an
interface
that
components
that
are
numeral
where
can
extend
in
order
to
make
more
numeral
and
resource
decisions.
A
So
what
this
means
is
a
pod
admission
time.
The
new
manager
will
make
a
call
out
to,
in
this
case,
CPU
manager
and
device
manager
to
get
what
allocations
they
can
give
to
that
pod.
So,
for
example,
in
this
case,
you
can
see
maybe
that
the
CPU
manager
can
satisfy
the
request
on
newman
o0
and
neumann
old.
One
it'll
send
back
that
information
to
new
manager
and
device
manager.
We'll
do
the
same
thing.
A
New
manager
will
then
read
this
information
and
create
a
best
fish
for
that
container,
but
it
doesn't
do
any
allocation
itself
and
then,
when
CPU
manager
and
device
manager
make
the
actual
allocations,
they
can
consult
new
manager
to
get
the
best
fit
for
the
container.
So
then
the
resources
will
be
aligned
and
we're
hoping
that
the
proposal
is
extensible
enough,
that
if
any
additional
components
are
added
to
cubelet
or
elsewhere,
that
they
can
extend
the
the
API
and
also
be
accounted
for
in
this
decision-making,
so,
for
example,
networking
devices
or
storage.
A
So
this
is
currently
in
proposal
phase
we're
also,
as
I
said,
with
NFT.
If
there's
any
use
cases
that
you
have
four
new
my
wariness
enquiries,
you
would
really
like
to
hear
from
them.
So
I've
put
down
the
pull
request
to
the
proposal.
So
if
you
have
anything
that
we
really
yeah
we'd
like
to
hear
from
you
so
then
so
that's
all
the
the
steps
we've
taken
to
extend
resource
management
in
kubernetes
bought,
Intel
and
the
community
couldn't
have
done
it
without
the
community.
A
So
then
we
created
these
container
bare-metal
experience
kits
were
basically
kind
of
a
best
practice
development
guidelines
for
deploying
containers
on
bare
metal,
with
an
orchestration
system
such
as
kubernetes,
so
they
have
multiple
components
in
them,
so
they
have
feature
briefs,
feature
application,
note,
scripts,
benchmark
reports
and
demos,
so
they're
all
available
online.
You
can
download
them.
I
have
the
link
on
this
slide
as
well.
A
So
sorry,
I
went
too
far,
so
these
are
the
the
four
we
currently
have
so
there's
a
reference
architecture.
So
as
the
hardware
that
we
recommend
deploying
on
and
then
there's
three
instances,
if
you
go
back
to
the
the
side
where
I
dressed
all
the
challenges,
these
kind
of
each
dress
one.
So
this
is
this:
one
here
is
kubernetes
networking
telemetry
and
then
we
have
our
hands
platform
awareness,
so
I'm
going
to
give
a
demo
of
the
enhance
platform,
awareness
ones,
and
then
you
can
run
the
other
ones
yourselves.
A
A
A
A
So
if
we
go
into
the
enhance
platform
awareness,
one
it'll
give
you
an
overview
of
the
different
components
of
a
see
what
I've
gone
through
so
nft
CPU
manager,
huge
pages,
and
then
we
also
use
a
SR
IV
in
this
demo,
because
it's
a
networking
workload
and
then
it'll
give
you
overview
of
the
the
impact
of
IPE
on
your
performance.
So
then
it
also
goes
through.
You
can
go
into
each
of
the
components
individually
and
find
out
more
it's.
A
Basically
what
I
showed
you
and
then
as
well
as
that,
there's
also
how
we
set
up
the
tests,
the
information
of
how
the
hardware
setup
the
results
of
the
tests.
So
you
can
see
the
benefits
of
running
a
CPU,
pinning
huge
pages
of
space,
your
hands
platform
awareness
with
a
networking
workload.
So
it's
a
DBT,
Kate
SPMD
workload
and
then
the
details
of
the
tests.
So
what
we
use
to
to
run
the
actual
tests,
a
software?
A
So
then,
if
you
go
to
the
demo
area,
we
can
see
our
demo
for
enhanced
platform
awareness
so
basically
on
the
Left,
we're
going
to
show
a
notice
that
noisy
neighbor
workloads.
So
we
have
our
DB
DK
workloads
running
on
a
core
to,
but
on
one
of
the
hybrid
siblings.
We
also
have
our
noisy
neighbor
running
and
then,
when
you
use
this
is
with
cmk
the
open
source
project
we
can
see
here.
A
We
have
an
isolated
set
of
core
or
cores
here,
so
our
DB
k
workload
is
going
to
run
on
one
of
these
and
then
our
noisy
neighbor
is
going
to
land
in
the
on
isolated
pool.
So
they
can't
affect
each
other,
so
you
can
run
actual
packets
through.
So
these
are
basically
live
recordings
out
of
data
through
that
setup.
So
you
can
see
that
if
you're
using
64
byte
packets
that
you're
getting
3.9
times
performance,
so
you
can
see
to
put
packet
rate
on
both.
A
A
Yeah,
so
the
CPU
manager,
if
you
request
multiple
CPUs
up,
trying
to
allocate
you
on
the
same
Numa
node,
but
there's
no
guarantee.
So
if
you
request
eight
and
there's
four
and
four
it
lelik
eight
you
the
four
and
four
yeah.
But
if
there's,
if
there's
eight
on
one
socket,
it
will
give
you
those
yeah,
exactly
yeah
and
it's
the
same
for
I
betrayed
siblings.
If,
if
high
returns
enabled
it
will
try
and
give
you
the
the
siblings.
A
No,
so
we're
kind
of
we
have
an
effort
to
align
CNI
and
network
manager
to
communicate
so
that
we
can
cater
for
networking
devices,
because
it's
kind
of
a
funny
area.
So
we
would
like
for
the
device
plug-in
to
take
care
of
the
allocation
of
the
device
and
then
CNI
to
do
the
actual
plumbing
to
the
network
namespace.
A
C
A
In
Franny
the
components-
yours,
yes,
not
that
I'm
aware
of
the
scheduler,
there's
no
scheduler
extender,
so
there's
no
worry
of
having
HTTP
connection.
There's
G
RPC
with
the
device
plugins
an
old
feature.
Discovery
is
using
in
cluster
config
to
communicate
with
the
API
server.
So
I
believe
there
is
no
additional
overhead
to
the
system
that
I'm
aware
of
a.
A
Once
again,
not
that
I'm,
aware
of
I,
don't
I
believe
I
guess,
there's
additional
complexity,
maybe
but
none
of
its
propagated
to
the
pod
spec.
This
additional
complexity,
complexity
in
kubernetes
itself,
but
none
of
that
is
exposed
up
so
I,
don't
think
the
user
has
any
additional
effort
to
make,
and
neither
does
the
cluster
operator
so
I
would
say.
No.
Thank.
D
A
Try
and
make
a
generic
open
source
of
a
solution
so
that
we
don't
tie
it
to
a
particular
platform.
But
then
for
the
case
like
with
device
plugins,
we
can
use
qat
and
kind
of
boost
that
with
kubernetes
as
well.
We
can
use
SR
IV
with
our
Nix
and
we
can
with
C
and
I
and
device
plugins
with
that
as
well,
and
then,
with
NF
t,
we
can
advertise
CPU
capabilities
like
a
VX
and
turbo
boost.
So.