►
From YouTube: Kubernetes Resource Management WG 20180314
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
Okay,
everyone
welcome
to
March
14th
edition
of
resource,
which
meant
working
groups.
Community
meeting
looks
like
the
main
agenda
today
is
to
discuss
idea
me
and
device
plugins.
So
it's
going
with
face
once
your
name
correctly
and
he's
going
to
talk
about
his
experience.
That's
device
plugins
on
GPUs
and
IDM.
A
go
ahead
like
floor
is
yours.
Take
your
time.
B
B
B
D
E
Sorry
to
interrupt
here
para
here,
we
would
like
to
use
our
DMS
C
group
as
well.
He's
yeah,
yinz
plug-in
currently
does
not
use
this
and
I
have
slides.
That
I
would
like
to
walk
through
to
give
a
first
a
big
picture
once
he's
done
with
the
demo
of
what
are
the
modes
and
knobs
that
we
have
at
an
RDM
a
level
and
the
requirements
that
we
would
like
to
integrate
into
the
kubernetes
at
the
device
plugin
level,
as
well
as
CNI
and
at
other
places.
B
Ray
again
well
protected,
it's
a
it's
very,
it's
very,
very
simple,
I
say:
firstly,
you
should
install
the
the
a
if
you
open
the
texture
software
and
start
with
a
device
for
luggage
into,
and
then
you
can
start
the
device
clogging
on
all
or
deploy
you
with
kubernetes.
Then
you
can
run
again
a
container.
You
can
run
a
container
as
they
either
example.
F
A
A
E
No,
probably
I'll
ask
Korean
I,
get
a
branch
first
question,
so
for
our
DMA
there
are
two
things
needed:
one
is
the
the
kernel
level
drivers
which
usually
gets
loaded
as
part
of
the
PCI
device
and
the
networking
device
gets
loaded
for
Ethernet,
as
well
as
the
InfiniBand
network.
So
the
the
you
day
of
system
and
the
auto
load
of
the
kernel
loads,
these
drivers,
on
top
of
that
there
userspace
libraries
and
applications
which
will
access
this
RDMA
devices.
E
D
Our
I'll
just
say
one
more
thing
on
our
on
our
distribution:
sorry
Red,
Hat's
distribution.
There
is
a
system
D
unit
that
called
our
DMA
that
gets
all
of
the
drivers
lined
up
assuming
their
upstream
I
notice.
Here
you
have
something
called
Mellanox
mo
fed,
so
I'm
wondering
if
you
have
cost
you
are
using
mo
fed
user
space
in
this
in
this
demo
or
like
the
customized
Mellanox
ofat
or
if
you're,
using.
E
I
think
so
so
usually
the
the
Mellanox
offer
brings
lots
of
features
every
quarter
or
so
and
though,
usually
the
OS
waste
does
not
have
those
features
at
the
same
pace.
So
the
users
usually
uninstall
the
drivers
and
user
space
packages
that
come
with
the
OS
and
they
tend
to
install
the
Mellen
of
software,
which
is
equivalent
to
what
they
have
in
their
OS.
D
B
E
E
G
E
To
a
different
screen:
okay,
let's
see
great
okay,
all
right,
so,
let's,
let's
go
over
it
so
before
I
jump
to
this
lights,
little
more
bit
more
step
back
on
our
DMA,
so
our
DMA
is
usually
a
kernel
bypass
method
that
allows
application
to
directly
talk
to
the
hardware,
bypassing
the
networking
stack
and
system
calls,
and
that
allows
is
to
do
it
much
much
faster
rate
where
two
nodes
can
talk
to
each
other
in
0.7
microsecond
to
to
exchange
the
messages
from
application
to
application,
and
this
is
done
through
the
DMA
subsystem
which
exists
in
kernel
for
more
than
twelve
years,
I.
E
Since
it's
a
kernel,
bypass
method,
there
are
various
various
security
aspects
being
added
to
make
sure
that
there
is
a
right
isolation
and
right
access
being
provided
to
each
containerized
environment.
So
if
we
jump
here
to
the
slides
here,
there
are
two
use
cases.
The
first
one
is
first,
we
would
like
to
to
have
customers
that
user
starts
using
it
in
this
mode.
There
is
one
IB
device
per
pod
and
all
the
containers
would
share
this
InfiniBand
or
a
DMA
device.
Infiniband
and
RDMA
are
usually
interchangeably.
E
Called
I
would
update
this
latch
to
call
our
DMA,
but
for
for
this
conversation,
just
we
can
in
that
it's
our
demon
device.
So
in
this
more
we
would
like
to
have
one
idml
device
or
shared
a
dedicated
to
a
pod
in
all
the
containers
of
it.
Now
what
from
orchestration
point
of
view,
what
consists
IDM
a
device,
so
edema
device
consists
of
one
character
device,
which
is
last
name
InfiniBand.
You
are
0
1,
2,
3
and
the
number
based
on
the
device.
Then
it
has
got
a
cease.
Fire
cease.
E
The
third
one
is
a
RDM
SCM
device.
This
basically
allows
applications
to
establish
the
connection
between
the
pods
and
between
containers,
so
they
use
this
device.
This
device
is
common
across
across
all
the
pods
and
all
the
containers.
So
it's
there's
one
such
character.
Device
per
system
through
which
all
the
applications
talk
and
perform.
The
listen
bind
connect
such
such
calls,
similar
to
what
socket
API
used
to
do
with
the
socket
calls.
E
The
fourth
one
that
consists
an
edema
device
is
that
is
one
net
device
for
each
RDMA
device,
and
this
net
device
would
live
into
the
namespace
network.
Namespace
of
the
pod
shared
among
all
the
containers
and
how
this
and
the
intent
is
that
this
kubernetes
to
isolate
these
three,
these
four
entities
of
an
eidm
a
device
on
a
per
container
basis,
I'm
sorry
on
a
per
for
basis,.
E
In
second
point
with
regards
to
this
character
device
and
third
one
is
that
our
DMA
cm
device
that
should
be
accessible
to
all
and
that
character
device
kernel
implementation
ensures
that
whenever
it
is
accessed,
he
provides
our
access
only
to
network
device
that
belongs
to
that
namespace.
So
this
device
is
okay
to
share,
among
all
the
whole,
the
of
the
container
instances.
A
A
A
E
A
E
The
libraries
do
make
use
of
this
character
device
file
so
typically,
what
they
do
is
the
open
character
device
and
then
the
issue,
the
read,
write,
calls
this
character
device
and
that
sets
up
the
connection
between
the
two
nodes.
Once
that
connection
is
set
up
through
the
character
device,
then
all
the
data
path
operations
to
send
and
receive
the
message.
That's
all
done
through
the
user
directly
talking
to
the
hardware.
E
A
E
Okay,
moving
to
to
the
second
use
case,
so
here,
if
we
see
it,
there
is
a
there
is
a
limitation
of
the
scalability
of
how
many
containers
and
pods
we
can
run
depending
on
the
how
many
devices
we
have
in
the
system.
This
is
typically
done
through
SR,
iove
or
other
virtualized
more
and
some
users
like
to
do
a
really
large
scale
in
more
than
three
digits.
The
use
case
one
can
do
up
to
hundred
or
odd
or
a
smaller
number
of
scale.
E
The
second
use
case
is
where
the
user
wants
to
do,
thousands
of
them
the
RDI
device.
Typically,
one
device
can
really
do
really
large
scale
operation,
but
the
servers
are
in
range
of
256
to
500,
GB,
RAM
or
more,
and
the
device
can
also
really
scale
up
to
a
large
number
of
millions
of
connections,
and
in
this
mode
we
would
like
to
have
one
device
that
is
shareable
across
multiple
ports.
E
So,
in
this
case,
what
you
will
see
is
this
will
be
only
one
device
like
previously.
We
saw
one
one
multi
character,
devices,
multiple
sea-surface
entries
and
so
on.
Here
there
would
be
only
one
such
device
and
one
sea-surface
entry
or
maximum
two
of
them,
and
in
this
case
the
isolation
of
the
resource
should
be
done
through
the
RDM
SE
group.
That
says
how
many
resources
of
one
device
you
can
consume,
and
in
this
case
there
would
be
one
network
device,
nine
device
per
namespace.
E
That
would
help
to
establish
the
connections
between
the
containers
and
pots,
so
the
net
device
is
pretty
much
same
between
the
two
modes.
The
only
big
difference
that
we
see
here
is
one
character
device.
One
sea-surface
entries
versus
multiple
of
them
and
in
this
is
relatively
less
secure
model,
but
there
are
use
cases
where
they
tend
to
you.
They
tend
to
have
single
tenant
and.
A
A
E
A
A
E
A
E
E
H
A
E
E
A
A
E
D
E
Very
much
and
I
mean
$0.10,
representing
is
one
of
the
perfect
example
where
they
are
using
outside
of
the
labs.
You
know,
and
N
and
I
think
there
are
other
MPI
users
too.
It's
just
that
I'm,
probably
not
in
sync
with
them,
but
you
know,
even
if
they
don't
go
for
a
scale
of
thousands.
I
would
still
see
that
they
might
come
up
with
a
few
of
you,
four
to
eight
four
to
eight
parts
running
on
arm
on
SEO
system.
A
A
D
I
think
the
thousands
is
something
like
it's
funny,
because
we
hear
these
huge
high
density
requirements
and
we're
like
wait.
How
come
process?
Isolation
isn't
in
it,
for
you,
I
mean
once
you
start
talking
all
the
additional
cubelet
plumbing
on
top
and
it's
most
like
you
know,
you're
paying
a
lot
of
overhead
for
I'm,
not
really
sure
that
people
fully
calculate
the
the
upside
and
downsides
to
these
things.
I
So
this
is
a
Renault
from
Nvidia
speaking
and
I've
I'm,
actually
working
on
a
design
document
for
sharing
devices
and
there
I'm
about
to
release
that
document
either
today
or
early
tomorrow,
and
there
are
two
parts
of
that
document
that
specifically
match
very
much.
What
your
what
you're
suggesting
here,
the
first
one,
is
sharing
a
device
between
multiple
containers
inside
the
same
pod
and
that's
something
that
we
do
have
a
use
case
for
GPUs
at
Nvidia
and
I.
Think
you
provided
an
additional
use
case.
E
Yeah
and
as
current
features
grows,
it'll
eventually
become
secure
right
now.
This
is
this:
is
the
current
state
and
the
currently?
What
we
have
to
make
them
secure
is
various
cardinal
knobs
one
is
cgroups
to
isolate
the
resources
and
there
are
SELinux
policies
that
can
be
set
up
so
to
isolate
between
who
can
talk
to
whom
in
is
being
done
through
the
SLE
next
policy
setup,
which
is
part
of
the
4:14
Colonel.
H
E
Two
resources
that
the
C
group
allows
to
control
right
now,
so
RDMA
level
has
Q's
or
like
Q
pairs
and
completion
Q's
through
which
it
does
the
radio
transaction
to
send
and
receive
messages,
and
so
there
that's
one
type
of
resource
which
we
call
HCA.
It's
here
objects.
These
are
like
all
the
objects
of
that
see.
A
shared
q,
CQ
q
pairs
memory
reaches
protection
domains.
There
are
like
six
or
seven
such
objects.
These
are
controlled
through
HC
objects
of
the
C
group
and
the
second
one
is
called
HCA
handles.
E
H
E
H
J
H
E
E
F
E
D
D
E
A
A
A
G
A
H
A
Okay,
I
apologize
for
my
laptop,
so
what
I
was
saying
it
was
like
device
plug-in
is
the
way
to
go
for
exposing
any
sort
of
countered
hardware
devices
and
when
it
comes
to
hardware
devices
that
that
doesn't
meet
scheduling,
I
would
recommend
going
down
the
CSI
route
or
just
using
host
path
volumes,
because
those
big
volumes
were
meant
to
also
expose
character,
device
files,
for
example,
or
block
device
files.
It
doesn't
really
matter
as
long
as
you
don't
need.
You
don't
need
any
sort
of
scheduling.
A
Primitives
go
with
that,
and
I
was
also
saying
that
unless,
unless
we
get
to
a
point
where
some
of
some
subset
of
these
hardware
devices
become
so
ubiquitous
that
workload,
portability
becomes
a
blocker
for
cumulus
adoption,
I,
don't
think
we
really
need
to
prioritize
resources
or
anything
of
that
sort.
For
these
kind
of
special
devices.
H
I
think
that
perhaps,
like
maybe
a
separate
discussion
by
the
way,
the
even
agree
with
me,
if
all
people
want
to
do
it
to
create
a
character
device
in
the
container,
perhaps
using
the
best
plug-in
it's
a
bit
heavy
weight
because
you're
your
own
use
telling
you
to
manage
your
demons.
That
are
some
kind
of
way
to
to
use
put
the
device
plug
in
service.
H
But
if
you
also
want
to
do
other
things
like
monitor
the
device
house
to
automatically
install
the
drivers
and
those
sins,
perhaps
you
can
use
the
best
plugin,
but
I
agree
with
evasion.
If
all
you
want
to
do
is
to
import
the
device
into
the
into
the
container,
we
may
want
to
look
at
other
ways
that
is
more
straightforward
for
vendor.
To
do
this.
E
One
is
the
the
character
device.
Second,
one
is
sea
surface
mount
points.
The
third
one
is
is
a
querying
to
basically
a
locator
device
or
set
of
devices
for
when
the,
when
the
body
started
and
I
think
all
the
three
requirements
sort
of
tend
to
match
from
the
device
plug
in
right
now,
but
I
would
let
others
also
to
comment.
H
E
I
agree,
and
and
for
second
use
case
since
there
is
only
one
device
that
we
want
to
share
among
all
the
containers,
there's
practically
no
need
of
a
device
plugin.
We
just
need
the
knob
at
the
cubelet
or
the
kinetics
level
to
allow
mapping
that
device
character
device
into
the
to
the
runtime
environment,
and,
if
that,
if
that
is
done,
I
think
that
is
sufficient.
You
in
that
case,
we
would
only
use
the
C
group
and
the
P
key
policy
of
these
cleanups.
A
A
E
Sure
I
want
to
go
through
just
last
three
slides,
most
of
them
is
straightforward
and
I
have
been
yeah
in
have
a
document
also
to
elaborate
more
so
right
now.
The
issue
that
we
face
today
in
terms
of
requirement
is
the
first
issue
that
we
face
is
the
device.
Flooding
does
not
have
a
know
anything
about
the
pods
and
therefore,
when
the
bodies
are
located,
he
just
says
a
locator
request.
Give
me
one
device
without
knowing
anything
about
the
pot
ID
or
for
name.
So
what
happens?
E
Is
that
when
multiple
containers
in
the
corner
run,
the
new
devices
are
located
for
each
container
and
that's
not
desired?
So
we
would
like
to
extend
the
device
plug-in
API,
where
allocate
requests
tell
us
about
the
pod
name
or
for
ID,
so
that
the
the
device
plug-in
can
do
one
device
allocation
for
all
the
containers
of.
A
H
A
Sounds
a
lot
like
the
CNIB
and
we
haven't
really
solved.
How
do
we?
How
do
we
take
care
of
like
accounting
for
or
network
devices?
So
this
is
currently
an
open
problem.
I
mean
if
you
can
find
some
way
to
work
around
it
for
now,
by
all
means,
go
ahead
with
that,
but
before
opening
up
API
any
further
I
personally
would
like
to
see
the
overall
picture
and
overall
alignment,
both
DNI
and
vice
plugins,
before
extending
the
device
plugins
for
this
specific
use
case.
So.
E
So
when
the
cni
plugin
is
started
and
the
cni
plugin
moves
one
network
device
here,
let's
see,
if
you
look
at
this
diagram
here
right
for
each
of
the
InfiniBand
device
I
should
have
shown
here
there
is
one
net
device
also
exist.
So
the
cni
plug-in
moves
the
network
device
into
that
namespace.
But
here
he
doesn't
have
the
details
or
the
above
one-two-three
details
that
he
also
needs
to
do
a
bind.
A
A
E
Yes,
so
you
would
like
to
avoid
the
hacks
and
the
proposal
we
can
discuss
it
and
for
the
meetings.
The
idea
that
yeah
in
and
we
were
thinking,
is
to
have
for
ID
being
exported
device
plugin
for
two
reasons.
One
is
to
allocate,
and
second
one
is
to
free
that
resource
when
the
power
goes
away,
so
that
the
same
resource
can
get
allocated
again
and
therefore
knowing
the
product
is,
is
good
thing
to
have
and
the
second
one
is.
E
If
the
cni
plugin
also
knows
about
the
power
ID,
then,
along
with
the
device
parameters,
then
he
can
do
the
he
he
can
map
the
right
network
device
that
belongs
to
this
character
device
into
the
network
namespace.
So
maybe,
after
this
meeting,
we
should
look
at
his
document
where
he
has
described
these
details.
You.
H
Know
definitely,
I
think
I
you
probably
want
to
work
with
him
to
to
make
sure
y'all.
Your
case
is
also
a
raptor
incited
you
his
a
document,
but
also
a
great
base
like.
I
think
there
are
a
lot
of
missing
pieces
to
to
really
make
it
work
her
to
set
up
the
network
at
device
under
the
network
namespace
so
that
the
best
pocket
yet
no.
A
E
H
H
H
G
H
E
If
you
see
on
the
slide
here,
if
the
cni
plugin
does
based
on
the
network
device,
then
for
this
network
device,
he
needs
to
know
what
is
what
is
the
character
device
and
what
are
the
sea
surface
files?
He
can
possibly
find
that
out
by
himself,
but
he
still
needs
to
have
an
access
to
the
amount
to
bind
mount
namespace
for
the
host
and
the
container
so
that
he
can
do.
E
H
K
H
A
D
Don't
think
there's
a
lot
of
like
debate
about
it.
Some
minor
concerns
around
making
sure
we
have
Bobby
in
the
room
when
we
need
him,
as
relates
to
any
like
more
intense.
You
know,
scheduler
topics,
yeah,
there's
a
finalized
agenda
on
the
on
the
on
the
mailing
list.
I
spent
a
couple
days
ago
well,
last
week,
I
believe
I
just
dropped
the
list
and
the
link
in
the
chat
so
I'm
now,
as
of
the
jinyang
and
others,
are
sending
Docs
to
the
list,
I'm
adding
them
here
to
the
individual
topics.
D
H
A
It's
not
just
think
we
need
representation
from
scheduling
in
general,
so
I
think
they
have
a
meeting
tomorrow.
Maybe
like
Jeremy,
you
and
I
should
actually
go
to
the
meeting
and
talk
about
this
case
and
make
sure
that
enough
enough
eyeballs
I
are
on
this
face-to-face
meeting
and
then
we
can
have
more
more
license.
Yeah.
H
A
The
point
of
this
working
group
is
that
we
actually
have
a
presentation
from
different
six
and
if,
for
example,
if
you
go
to
like
restricted
this
to
a
single
sink,
then
we'll
just
operate
as
I
said
right.
So
I'm
just
saying
that
we
need
to
think
we
need
to
do
a
do
our
own
due
diligence
or
advertising
across
all
the
stakeholders.
Six.