►
From YouTube: Make Kubernetes Networking Ready for World Class AI and HPC ... Sunyanan Choochotkaew & Gaurav Singh
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Make Kubernetes Networking Ready for World Class AI and HPC Workloads - Sunyanan Choochotkaew, IBM & Gaurav Singh, Red Hat
While use of Kubernetes for various services is growing rapidly, it is still behind in the world of HPC and AI clusters. Part of the reason is that the lack of support for advanced features like multiple 100G networks available in HPC/AI Systems. Vast majority of AI systems in hyperscalers such as IBM Cloud, AWS, Azure, and Oracle Cloud come with two to 8 100G network interfaces on the A100 GPU nodes. However, by default in Kubernetes, a pod has only one network interface, but attaching multiple interfaces is often a requirement in the scenarios. Multus unlocks the potential of multi-networking feature in Kubernetes, but there are still challenges in usability, manageability, and scalability. We present Multi-NIC CNI, a new open-source project, to democratize multiple interfaces capability for everyone. This CNI saves users from the concerns regarding environment heterogeneity and acquiring CNI specific knowledge. This talk will introduce the architecture, use cases, and performance of the CNI, then show how beneficial it is for HPC/AI. We will demonstrate the CNI on a large scale GPU Cluster consisting of over 1400 GPUs and two 100G network interfaces that we build in IBM Cloud.
A
A
So
I
gave
a
session
in
this
morning.
What
I
really
want
to
bring
in
front
of
the
community
is
stuff
that
I'm
listening
from
my
customers,
so
that
you
get
the
perspective
of
the
use
cases
that
we
are
dealing
with
right.
We
have
the
session
with
Darren.
We
talk
about
scaling
on
the
cloud
bursting
and
you
you
saw
a
gentleman
from
nnl
Lab
who's
working
with
IBM
and
red
hat
to
to
to
do
to
this
Flex
scheduler.
A
So
these
are
all
based
on
the
use
case
that
we
are
seeing
in
the
field
and
we
are
helping
them
right.
So,
like
clouds
and
HPC
AI
what
again
right?
What
we
are
seeing
is
customer,
looking
forward
to
have
like
Cloud,
saving
and
elasticity
type
of
use
case.
Where
you
know
is
any
like
easy
to
GPU
based
ec2
instances
are
expensive.
You
don't
want
to
grab
that
and
hold
it
for
you
know
for
the
job,
and
if
it's
finished
you
don't
want
wanted
to
be
associated
with
that
job.
A
You
want
to
get
it
free
so
that
either
it
goes
back
to
the
pool
you're
you
can
you
don't
get
built
for
the
usage
or
it
being
used
for
other
job
right
and
similarly
elasticity.
You
need
to
make
sure
that,
if
there's
more
jobs
in
in
the
queue
you
are
scaling
the
same
cluster,
adding
nodes
because
think
about
creating
a
cluster
from
the
scratch
and
adding
the
whole
operators,
on
top
of
it.
Take
time,
take
five
to
fifteen
minutes
right,
so
you
it
cost
saving
elasticity,
reliability,
right,
reliability
in
terms
of
the
job.
A
Let's
say
you
are
running
a
training
and,
and
then
a
cloud
instance
by
inheritance
is
sometimes
unreliable.
So
what
what?
If
you
are
in
a
day
of
your
training
and
your
Cloud
instance,
got
deprovision
or
or
disconnected
so
you
need
to
figure
out.
That's
you
know,
State
reliability
of
that
training
of
a
job,
maybe
taking
a
snapshot
or
putting
the
put
dividing
a
job
into
sub
jobs
and
running
across
the
zones.
A
You
know
the
communication
from
GPU
GPU
needs
to
be
fast.
There
needs
to
be
a
faster
bandwidth,
pipe
going
from
one
GPU
to
other
and
sanasan
we're
gonna
talk
the
whole
presentation
on
how
how
we
can
use
multi
infinity
band.
Second,
is
the
fast
loading
of
images.
These
AIML
images
tends
to
be
very
heavy
right.
Think
about
you
have
download
the
image
from
an
image
registry.
It's
giga
byte
of
images,
you're
downloading
it.
What
what?
If
you
want
to
Cache
it?
A
Do
you
want
to
cache
in
on
the
same
node
or
you
want
to
Cache
it
on
every
node.
Is
your
East
to
East?
West
traffic
is
more
faster
than
north
south
right,
rather
than
going
to
the
to
Rice
tree
all
the
time
you
have
a
cache
somewhere
right
so
think
about.
That
is
we.
We
are
seeing
a
prevalent
use
cases
around
that
again
Q
we,
we
have
talked
about
Q
in
couple
of
ways.
You
know
we
talk
about
Q
as
in
project.
A
This
MCAT,
that
is,
IBM
team
is
working
on,
but
really
the
problems
that
we're
trying
to
solve
here
is
when
your
jobs
comes
in.
They
need
a
place
where
they
can
stay
until
unless
a
resources
is
available
for
their
job
to
run,
which
is
also
the
concept
of
the
gang
scheduling
where,
where,
if
you
have
a
resources
for
10
five
node
five
jobs
out
of
ten
you,
your
job
doesn't
schedule.
B
Okay,
thank
you,
goras,
okay,
for
my
part,
I
would
like
to
start
from
this
slide,
as
gorov
is
already
mentioned,
it's
his
first
slide
that
cloud
brings
so
many
mid
feet
to
the
HPC
and
AI
workload,
and
that's
especially
with
their
containerizations
and
organization
systems.
That
is
why
we
are
working
so
hard
to
make
the
full
cycles
of
the
AI
and
also
the
HPC
workload
work
on
the
open
shift,
especially
on
top
of
their
workshop
instant
running
under
our
our
GPU
servers
on
the
cloud.
B
We
hover,
we
get
the
machines
with
their
support
of
the
SRI
that
we
can
bypass
this
with
layers
of
the
hypervises,
and
we
can
enable
almost
a
full
bandwidth
on
the
virtual
instant
that
we
have
and
are
in
some
of
our
meshes.
We
have
two
of
them
and
we
got
like
about
200
gigabit
per
second
for
the
two
plus
and
20
microseconds
for
the
latency.
B
The
problem
next
is
that
or
how
can
we
bring
this
month?
Linux
Network
Solutions
from
the
cloud
vsi
into
the
port
level,
and
this
slide
will
show
a
two
part.
One
is
like
what
we
have
in
the
default
networks
on
the
kubernetes
system
that
we
provided
right
now
right
now,
as
you
may
already
know,
we
usually
help
when
primaries
are
interfaced
on
the
Pod
that
make
the
connections
to
the
control
plane,
those
stuff
and
with
that
one
we
usually
have.
There
are
two
pass.
B
So
what
what
we
need
for
the
AI
and
his
PCS
that
are
okay?
We
can
leave
the
first
primaries
for
the
control
planes,
but
what
we
need
is
we
need
to
bring
the
secondaries
in
Phase
that
we
attach
to
the
high-speed
networks,
connect
the
library
to
the
part
in
arms
so
that
we
can
have
the
same
triples
and
the
same
latencies
that
we
have
in
the
install
level,
and
this
is
the
key
that
we
can
achieved
that
part.
B
One
thing
is
that
we
have
to
directly
expose
the
package
of
the
of
the
pot
straightforward
to
the
to
the
East
Diamond
delays,
so
that
we
don't
have
our
overhated
encapsulations
of
their
address
translations
and
another
thing
that
we
have
to
work
on
is
that
or
after
we
make
a
straight
state
route
to
the
to
the
host.
We
have
to
make
sure
that
the
part
package
is
rootable
on
the
underlays.
B
And
in
kubernetes
we
have
we
have
their
projects
called
maltas
that
are
already
mentioned
in
the
previous
talk.
That
is
this.
The
project
that
do
very
excellent
job
or
like
allow
us
to
attach
multiple
network
interface
into
the
pot.
However,
to
make
that
happen,
there
is
like
multiple
steps
that
I
still
have
to
do.
Manuals
and
like
peoples
are
files
suffers
from
like
doing
those
stuff.
That
is
like
this
post
step.
B
The
first
step
is
that
the
administrator
have
to
like
sneak
into
their
arm
into
the
instant
they
have
to
know
about
their
interface
World
interfaces
available.
What
is
that
in
the
face
name?
What
is
the
network
address
and
those
stuff,
and
then
they
have
to
make
a
configurations
to
make
the
the
the
part
address
through
tables,
and
then
you
have
to
Define
one
by
one
for
each
interface
to
make
the
attachment
available
and
annotate
that
to
the
pod
in
sums
are
in
some
providers.
B
There
is
infrastructure
supported
as
well
handles
all
the
product
IP
address
to
make
sure
that,
even
after
you
export
the
packet
directly
to
the
host,
it
can
be
routable
to
the
order
instance.
However,
the
other
step
is
to
leave
to
the
administrator
and
the
users
to
to
handle
all
of
the
stuff,
and
in
this
in
this
thought,
I
would
like
to
represent
our
research
work
on
the
Monday.
Next
are
cni
operators.
These
operators
will
like
handle
their
handles
their
configurations
and
their
discoveried
interface
and
handle
everything.
B
So
are
the
benefits
of
the
monthly
Leaf
is
not
just
only
their
adoptabilities
and
usability
that
you
can
see
from
the
previous
slide,
that
are
admin
and
the
user?
Don't
have
to
do
much
thing
about
the
network
and
they
can
they
can
enables
or
multiples
infinite
band
into
the
pod.
Furthermore,
in
the
mountains,
we
also
consider
the
dynamicity
of
the
cluster,
like
the
Clusters
machines
can
be
scaled
out
and
scale
scale
and
scale
in
any
times
depends
on
their
workload.
B
Demand
and
also
is
the
interface
itself
can
be
added
or
removed,
or
this
can
be
unavailable.
Any
chance
and
like
the
noise
can
be
unconnected,
can
be
reset
anytime.
So
this
is
the
thing
that
we
have
to
handle
with
this
dynamicity
and
also
our
reconsiders
about
the
scaling
as
well
like
we
expect
the
north.
Can
we
scale
like
up
to
100
or
more
than
that,
and
we
have
to
handle
that
simultaneously
with
the
with
the
large
numbers
of
the
product
IP
allocation
as
well.
B
So
this
is
a
key
that
how
can
we
achieve
the
Nile,
mystery
and
scale
by
the
mountainix?
First
thing
is
that
we
do
the
isolations.
We
isolate
the
root
configurations
of
the
for
the
each
Network
so
that
we
can
handle
the
dynasty
separately
from
the
what
the
horse
is
working
on,
and
then
we
do
the
synchronizations
time
two
times
to
make
sure
that
okay,
our
the
latest
stated
of
the
host
interface
is
up
to
date
and
we
can
handle
the
failures
and
do
the
recoveries.
B
The
third
thing
is
that
we
minimize
the
communications
to
the
API
servers
and
choose
some
of
you
may
are
facing
the
same
problems
when
we
like
depends
on
anything
on
this
custom
resource
and
we
have
to
connect
to
API
servers
so
many
times
and
we
can
get
like
a
rate
limits
problem
and
those
issues.
So
we
are
because
most
of
the
custom
results
that
we
create
is
automatically
grade
and
controlled
by
Zuma
Linux
cnis.
So
we
can
catch
that
we
can
do
the
minimizations
on
the
API
server
Communications
and
the
force
are
pieces,
their
distributions.
B
We
do
the
IP
management
and
the
distributed
way
and
independently
from
each
other
post
to
make
sure
that
it
can
scale
are.
If
you
want
to
try.
Maintaining
Cris
now
is
already
available
on
the
operator
hubs,
not
just
only
on
the
openshift
operator
office,
but
also
like
their
communities
communities
as
well
to
get
started
with
the
month
inning.
There
is
only
a
simple
three-step.
B
The
first
thing
is
okay,
install
operators,
second,
is
Define
the
network
and
the
next
Network,
just
only
a
single
one,
and
then
no
matter
is
how
many
is
of
the
interface
you
have.
It
can
be
attached,
like
a
network
like
interface
pool,
and
the
third
step
is
like
annotate,
that
to
the
party.
This
is
where
it
is
on
top
of
the
malta,
so
the
peoples
who
get
used
to
models
will
also
simply
understand
like
apply.
This
are
madani.
Cnis
are
from
this
from
this
line.
B
Next
slide,
I
will
go
on
the
details
about
their
modeling
cnis,
like
how
his
work,
how
it's
composed
of,
but
I,
will
go
very
quickly.
It's
because
I,
don't
think
you
are
interested
in
into
the
details,
but
of
course,
if
you're
interested
you
can
reach
out
to
me
and
see
more
detail.
Basically,
the
cni
is
composed
of
a
three
components.
The
controllers
part
the
main
thing
and
the
demons
and
the
cni
similar
to
the
other
cni
plugins,
for
the
custom
resource
that
women
age
is
that
there
is
a
full
custom
resource
to
manage.
B
The
main
one
is
the
monthly
Network
that
you
still
have
to
Define
for
the
last
three
is
this
managed
is
auto,
created
and
managed
by
their
maintenance
control,
so
you
may
not
have
to
are
concerned
about
it.
Basically,
there
is
a
trick
operating
for
that
may
happen
with
the
mandinix
cnis.
Let's
say
it
can
be
even
triggers
to
the
cni
the
basic
operate
operating
for
starting
from
the
user
deployed
operators
and
then
deploy
the
Monday
next
networks
create
a
part
delete
the
part.
B
This
is
a
simple
figure
that
happens
to
the
mountainix
cnis
additional
to
that.
If
you
search
a
scale,
the
node
like
increased
number
of
the
node
or
change
the
interface
on
the
note
that
will
triggers
the
the
synchronization
change
and
also
like
if
the
node
is
restart
or
is
failures,
so
it
will
trigger
another
synchronizations.
B
Starting
from
the
basics
of
reading
for
like
when
you
deploy
the
operator,
it
will
automatically
create
a
demons
that
are
deployed
at
the
host
with
the
cnis
and
also
it
will
like
automatically
discover
about
the
host
interface.
It
know
about
the
secondary
interface.
What
is
the
device
ID
is
this
or
the
stuff
is
listed
in
the
host
interface
custom
resource?
B
If
you
change
a
node,
it
will
like
period,
create
synchronize,
and
they
take
that
and
after
that,
it
would
update
all
of
the
custom
resource
to
the
reconfigurations
again.
If
the
node
is
distressed,
this
can
be
detected
by
the
rules
is
not
available
on
the
host
anymore,
so
it
can
do
the
re-configs
and
make
the
route
become
available
again.
B
B
So,
in
summary,
like
we
have
a
modeling
cni
that
can
make
their
kubernetes
networking
ready
for
their
Ai
and
his
PC
workload
with
not
so
slowly
adoptability
and
usability.
We
are
so
considered
about
the
Dynamics,
D
and
scale
with
four
or
key
pieces,
that
is
our
isolation,
synchronizations,
minimizations
and
distribution.
D
B
D
Of
had
two
questions
that
I
guess
were
related:
if
you
have
a
pod
with
multiple
I
guess,
IP
addresses
Associated
to
it.
How
do
you
do
service
Discovery?
For
it
like
say
I,
you
know
want
to
talk
to
that
pod
I've
got
four
different
addresses
to
use
to
I.
Guess,
take
advantage
of
all
that
bandwidth
until
I.
Do
it
do
I?
D
B
I'm,
not
sure
I,
understand
your
questions
clearly,
but
what
the
marketing
seed
or
discover
is
this?
Something
does
our
vegetables
in
the
host
level,
like
we
have
the
demons
monthly
demons
that
it
can
do
the
same
thing
like
I
have
configs
with
the
net
links
library,
and
then
we
can
get
the
address
and
we
can
figure
out.
This
address
is
the
primaries
or
the
second
race,
and
we
can
get
the
info
informations
for
most
most
of
information
from
the
net
link,
Library.
C
B
B
Era
is
still
based
on
the
IP
address
and
are
in
the
monthly
series.
We
will
do
like
a
host
interface
locals
ipam.
So
we
like
our
with
with
the
cider's
results
that
we
have
is
we'll
keep
the
informations
of
the
okay.
If
you're
pausing
in
this
host
with
this
interface
view
in
this
range-
and
this
will
use
that
range
for
put
that
in
the
root
tables
so
that
their
host
can
know
that,
okay,
if
their
this
IP
address,
is
coming
from
this
point
from
this
host.
In
this
interface
sure.
B
C
B
D
So
you've
got
the
host
is,
you
know,
has
its
four
additional
or
three
or
four
additional
Nicks
and
you've
kind
of
you
know.
You've
exposed
those
directly
into
the
pods
Network,
namespace
and
you've
set
that
all
up
and
the
routing
works.
I
understand
that
I
guess
my
question
is:
can
we
make
it
so
that
those
four
individual
physical
interfaces
are
exposed
into
the
Pod
Network
namespace
as
a
bond,
rather
than
individual
interfaces
like
using
lag
or
something
so
that
you
don't
have
to
deal
with
it.
B
A
E
E
A
E
B
B
C
Do
you
deal
with
setups,
where
you
have
multiple
necks
at
the
node
level
and
you
want
to
assign
one
neck
for
each
part?
So
it's
not
multi
neck
at
the
no.
At
the
Pod
level,
for
example,
you
have
a
neck
for,
like
you,
have
a
numa
architecture
and
neck
connected
to
a
GPU,
and
you
want
to
assign
a
pod
to
that
GPU
and
get
that
Nick
and,
like
does
your
like
plug-in,
handle
this
setup.