►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Deep Dive: CNI - Bryan Boreham, Weaveworks & Dan Williams, Red Hat
CNI, the Container Network Interface, is a small but critical piece of infrastructure linking runtimes such as Kubernetes and CloudFoundry to dozens of different container network implementations. This session is aimed at implementers of network plugins and runtimes using CNI, as well as anyone interested in contributing to the project or becoming a maintainer. We will recap recent changes, look at the roadmap ahead, and have plenty of time for discussion and Q&A.
https://sched.co/Uaga
A
A
A
A
A
The
in
the
same
repo
is
that
is
a
is
a
library,
Lib,
C
and
I.
That's
that's
a
go
library,
that's
used
by
things
that
call
C
and
is,
and
we
call
them
runtimes.
So
the
one
many
of
you
will
know
as
kubernetes
but
CNI
is
is
cross-platform.
So
it's
used
by
what
can
be
used
by
me
sauce,
it's
used
by
cloud
foundry.
It's
used
by
pod
man,
there's
a
bunch
of
alternate
runtimes
that
can
call
into
CNI,
and
we
do
stress
that
in
the
the
cross
project
cross-platform
nature
CNI
is
a
CNC
F
project.
A
Kubernetes
is
the
CNCs
project.
But
that's
you
know
we're
all
friends
but
they're
not
the
same
thing.
One
is
not
part
of
the
other
yeah,
so
a
couple
of
tools.
So
that's
it.
That's
on
github,
it's
all.
Under
the
container
networking
organization,
the
CNI
repo
has
the
spec
and
the
library
the
so-called
base
plugins.
B
A
Who
we
are
so
I'm
on
there
Dan's
on
there?
These
are
the
maintainer.
These
are
the
people
who
show
up
regularly
who
have
commit
rights
to
the
repo,
but
we
do
get
lots
of
contributions
and
I'm
sure
there's
lots.
People
in
the
room
have
made
contributions
to
the
lib
CNI
or
to
the
spec
or
to
to
the
reference
plugins
and
then
there's
this
dozens
more
plugins
outside
of
the
CNI
project.
Those
are
all
CNI
plugins
and
we
love
them.
We
host
references
to
a
lot
of
them,
but
that's
that's
you
know.
A
One
of
our
aims
is
to
make
it
possible
for
everyone
to
be
on
the
same
playing
field
and
everyone
who's
got
a
network
implementation
can't
that
into
a
container
runtime
via
C&I.
That's
what
it's
for
a
little
bit
about
how
we
work.
The
spec
is
quite
slow
in
evolving
intentionally,
so
yeah
yeah,
oh
yeah,
because
we
have
to
be
very
careful
and
really
agonize
over
things
that
might
break
someone.
A
B
In
one
point
under
plugins
is
we
need
to
rev
the
plugins
more
quickly,
because
you
know
people
find
bugs
and
plugins,
and
you
know
bugs
in
the
other
libraries
and
stuff
like
that,
and
so
we
want
to
get
those
fixes
out
to
people
more
quickly
than
the
spec
of
balls.
Because,
like
Brian
said,
the
spec
needs
to
move
a
little
bit
more
slowly,
because
we
need
to
be
very
careful
about
what
gets
added
to
the
spec
in
the
language
there.
But
we
don't
want
to
block
point
releases
of
the
plugins.
B
A
A
You
have
a
you,
have
a
runtime
on
the
left,
so
maybe
that's
kubernetes,
maybe
not
you
have
a
container
or
in
kubernetes
case
a
pod
on
the
right,
and
you
have
a
network
at
the
bottom.
So
this
this
could
be
something
that
someone
is
set
up,
that
this
could
be
calico
network
or
weave
network,
or
you
know,
shifts
network
or
something
from
from
Cisco
or
juniper
network.
So
you
know,
there's
a
a
network
out
there.
A
We
want
to
connect
to
our
container
and
conceptually
the
the
plugin
is
just
doing
the
glue
in
the
middle
I
was
talking
to
someone
after
the
last
talk
and
I
realized.
Maybe
I
didn't
explain
this
very
well,
so
so,
in
the
case
of
my
own
plugin,
we've
net,
when
when
when
kubernetes
calls
the
plugin,
it
basically
makes
one
rest
call
because
we're
running
a
daemon,
an
exit.
You
know
the
plug-in
itself
does
nothing.
It's
talking
to
an
implementation
of
a
network.
That's
somewhere.
A
So
so
both
of
those
things
are
CNI,
plugins
and
and
I
recognize
that
might
be
confusing
to
think
about.
But
basically
what
needs
to
happen
is.
Is
the
runtime
makes
a
call,
which
is
very
simple,
just
add,
to
make
a
make
a
network
connection
or
dowel
to
take
one
away
it's
configured
by
a
JSON
file,
or
you
know
some
Jason,
whether
it's
in
a
file
or
not?
Is
that
me?
Okay?
A
The
Jason
is
the
configuration
of
what's
going
to
happen
on
the
network
and
the
plug-in
makes
it
happen.
The
plug-in
makes
an
interface
appear
in
the
container.
So,
whatever
needs
to
happen,
we
don't
care
because
we're
we're
just
making
it
possible
for
everyone
else
to
play
on
the
same
playing
field.
A
B
A
A
A
There
was
a
problem,
though,
that
they
would
go
ahead
of
rules
that
someone
else
had
on
their
machine
and
in
particular
that
could
be
a
blocking
rule
such
as
a
firewall
rule,
so
just
kind
of
depended
on
the
order
in
which
things
were
set
up
and
ended
on
whether
you
had
a
port
map
or
not
and
so
on,
but
eventually
someone
realized.
This
is
vulnerability.
A
The
the
kind
of
the
the
interesting
part
was
that,
oh,
you
know
how
this
how
I
became
aware
of
this.
It
was
essentially
people
people
screaming
at
us
on
github
saying
when
are
you
gonna
fix
this
issue,
and
so
we'd
go
like
okay?
Well,
I
personally
would
go.
Okay,
tell
tell
me
more
about
the
issue.
They
go
I
can't
it's
embargoed.
A
A
The
security
committee
of
kubernetes,
who
were
the
people
screaming
at
us,
were
you
know,
I
tightly
tightly
knit
group
they're
keeping
the
information
under
wraps
until
they
had
a
fix,
so
they
weren't
going
to
publish
that
on
github,
and
you
know
we
sorted
it
out
in
a
day
or
so.
The
the
fix
was
really
easy.
Just
don't
insert
a
pen
the
rule,
but
yeah
we
learned.
We
learned
a
number
of
lessons
that
that
that
week,
CNI
is
a
very
critical,
tiny
but
critical
part
of
the
infrastructure.
A
We
didn't
have
the
communications
channels.
We
didn't.
We
didn't
have
a
way
to
make
a
private
build
of
CNI,
so
the
fix
itself
could
be
kept
under
wraps
and
while
there
was
more
fun
and
games
in
actually
getting
it
kind
of
vendored
into
kubernetes.
But
that's
that's
really
down
in
the
weeds.
Ask
a
question:
if
you
want
to
hear
about
that,
but
yeah
the
kind
of
the
kind
of
life
life
of
the
project,
this
is
touchwood.
A
A
B
It's
something
that
we
know
we'd
seen
for
a
long
time
as
a
need
in
the
cni
project.
We
went
through
a
number
of
iterations
of
it
and
I
think
it
took
maybe
six
nine
months,
maybe
even
a
year,
to
figure
out
exactly
what
the
right
pattern
there
was
for
the
check
command.
So
you
know,
based
on
that,
let's
just
kind
of
go
through
it
and
talk
a
little
bit
more
about
it.
B
Dr.
shim
currently
assumes
that
the
networking
is
up
if
the
pot
has
an
IP
on
the
primary
interface
of
the
pod,
but
after
that
it
basically
kind
of
forgets
about
it,
and
even
if
it
doesn't
really
have
that
IP,
it
assumes
that
things
are
more
or
less
okay.
So
there's
kind
of
a
disconnect
here
between
what
is
the
actual
state
of
your
container
and
what
does
the
runtime
expect
that
state
to
be,
and
if
something
changes
does
the
runtime
react
intelligently
to
that?
B
Hey
si
and
I
plug
in
I
asked
you
to
set
up
pod.
Well.
Is
that
pod
still
healthy?
Is
the
network
of
that
pod
still
healthy?
Do
you
think
it
is?
There
were
a
couple
of
complications
with
that
and
that's
why
I
kind
of
took
us,
you
know
maybe
almost
a
year
to
really
get
right
and
pushed
through,
because
correct
means
something
very
different
to
a
lot
of
different
network
plugins.
It's
things
like
is
the
IP
address
of
the
container
the
same
as
was
originally
assigned
to
that
container.
B
First,
you
usually
have
something
like
the
bridge
plugin
and
then
you
might
have
an
eye
Pam
plug-in
in
there
to
do
the
actual
IP
allocation
assignment.
Then
you
might
have
something
like
a
firewall
plug-in
to
poke
rules
through
the
firewall.
You
might
also
have
a
tuning
plug-in
or
a
bandwidth
plug-in
to
set
up
bandwidth
limitations
on
the
container.
Well,
all
of
these
different
plugins
can
interact
in
different
ways,
and
that
was
one
of
the
big
things
that
we
kind
of
had
to
discuss
and
figure
out
how
to
get
exactly
right.
B
So
the
spec
language
says-
and
this
is
pretty
sure
we
copied
this
directly
from
the
spec-
that
the
plug-in
should
return
an
error
if
a
resource
included
in
the
cni
results
type,
which
are
things
like
interfaces,
addresses
routes.
You
know
MAC
addresses
of
the
plug-in
other
things
if
that,
if
a
resource,
including
the
result
that
was
created
by
this
plug-in
and
is
listed
in
the
result
that
is
given
to
the
plug-in
as
part
of
the
check
operation,
is
missing
or
an
invalid
state.
Now,
what
exactly
does
that
mean?
B
It's
probably
worthwhile
to
back
up
a
little
bit
and
talk
about
how
you
know
what
proof
result
is
and
how
the
check
operation
works.
So,
when
the
runtime
calls
the
check
operation,
the
spec
requires
that
it
passes
to
the
plug-in
the
actual
object,
that
the
add
call
returned
to
the
runtime.
So
first,
the
runtime
calls
add
and
says
set
up
this
container
and
all
the
plugins
in
the
chain
do
their
thing
and
they
return
one
JSON
structure
to
the
runtime
that
describes
things
like
the
IP
address.
B
B
The
two
most
important
ones
are
check
and
delete
in
the
case
of
check
that
pre
result
describes
the
final
results
of
the
network
operation
on
that
container,
and
that
is
what
the
plugins
and
check
need
to
take
a
look
at,
because
if
some
further
chain
plug-in
decided
to
modify
the
results
after
your
plug-in
ran,
then
that
gets
fed
back
in
and
your
plug-in
can
see
that
you
know.
Maybe
the
state
is
not
exactly
what
I
thought
it
was
when
I
set
it
up,
and
so
that's
why?
B
If
the
resource
that
you
configured
is
listed
in
the
pre
frizzled,
then
you
can
take
some
action
on
it.
But
if
it
is
not
listed
in
the
pre
Frizzle,
then
your
plugin
should
not.
What
really
needs
to
happen
here
is
that
if
the
plug-in
thinks
that
something
is
invalid,
it
needs
to
return
an
error
to
the
runtime,
and
then
the
runtime
is
supposed
to
do
something.
Like
tear
the
container
down
so
that
it
can
restart
the
container
with
a
healthy
Network,
and
so
let's
you
know,
take
one
example
here.
B
So
if
your
container
sets
up
or
your
excuse
me,
your
CNI
plugin
sets
up
a
route
inside
the
container
and
then
a
further
plugin
later
on
in
the
chain,
decides
to
remove
that
route
or
change
routing.
Now,
when
that
comes
back
for
a
check,
call
to
determine
whether
the
containers
healthy
or
not,
your
plugin
might
know
through
some
other
mechanism
that
it
was
supposed
to
configure
that
route
inside
the
container.
But
your
plugin
will
see
that
in
the
pre
Frizzles.
B
B
Hopefully
that
makes
sense-
and
this
is
kind
of
one
of
the
reasons
why
it
was
complicated
to
decide
how
check
should
really
work.
But
in
the
end
we
did
implement
check,
support
in
all
of
the
reference
plugins,
and
so
that
serves
as
a
pretty
good
example
of
exactly
how
the
cni
project
and
the
spec
expect
check
to
work
and
again,
if
you
see
a
resource
in
your
plugin
that
you
think
should
be
there,
but
it's
not
in
the
pre-freeze
alt
that
you
get
as
part
of
that
check
operation.
That's
not
an
error,
but
anything
else.
A
B
You
covered
that
already.
The
one
thing
the
important
thing
is
there
is
a
skip
check
safety
valve.
We
could
come
up
with
some
situations
where
this
behavior
could
return
or
could
result
in
unwanted
behavior,
and
so
you
can
specify
in
your
CNI
configuration
that
check
should
not
actually
happen
result.
Caching
is
the
other
thing
and
this
kind
of
ties
directly
in
with
the
check
behavior.
B
B
It
turns
out
that
a
lot
of
plugins
don't
necessarily
want
or
need
to
cache
the
state
that
they
set
up,
and
so
it's
very
useful
for
those
plugins
to
get
the
result
of
the
add
operation,
and
then
they
can
decide
using
that
information.
What
exactly
need
to
clean
up?
That's
things
like,
for
example,
the
subnet.
That
was
that
the
container
has
an
IP
address
on
so
that,
for
example,
firewall
rules
can
be
cleaned
up
and
you
can
use
and
the
firewall
plug-in
you
can
match
on
that
subnet
and
do
the
right
thing.
B
So
there's
also
config
caching
and
the
big
issue
here
is
what
happens
if
you
want
to
change
Network
plugins.
Well,
the
cluster
is
running,
and
this
actually
is
a
thing
that
people
want
to
do
it
in
some
ways
it
is
more
complicated
than
it
should
be,
and
it
might
not
give
you
the
right
results
that
you
want,
but
there
was
a
pretty
big
gap
with
C&I
around.
B
If
you
change
that
Network
plug-in
out,
nothing
actually
remembered
what
the
network
plug-in
was
that
a
particular
container
was
set
up
with,
and
so
you
plug
in
a
and
you
get
a
container
set
up
with
plugin
a
then
you
swap
out
the
network
plug-in
and
all
the
rest
of
the
containers
get
started
with
plug-in
B.
But
then,
if
container
a
is
removed
or
deleted,
nothing
remembered
that
it
was
supposed
to
be
deleted
with
the
first
plugin.
B
It
would
then
be
deleted
with
the
second
plug-in,
and
that
obviously,
is
not
going
to
give
you
the
results
that
you
expect
and
garbage
collection
and
things
like
that
would
not
do
the
right
thing.
So
config
caching
basically
saves
the
configuration
that
any
given
container
was
started
with
and
then
it
will
use
that
configuration
for
subsequent
operations
like
check
and
Dell.
B
This
is
we
implemented
this
both
config
and
result,
caching
into
Lib,
C
and
I,
and,
as
Brian
said,
this
is
the
library
the
go
library
that
the
cni
project
provides
as
a
reference
for
runtimes
to
encapsulate
a
lot
of
this
behavior
so
that
the
runtimes
don't
have
to
all
rewrite
this
themselves.
So
if
your
are
involved-
or
you
write
a
runtime
that
uses
CNI,
this
is
mostly
automatic
for
you
with
Lib,
C
and
I
and
go.
A
B
As
long
as
you
wrote
it
and
go,
but
most
of
them
are
written
and
go
so
that's
that's
not
a
huge
issue.
It
is
fairly
automatic.
The
one
thing
that
is
not
automatic
with
it
is
the
config
caching
for
delete
your
runtime
does
need
to
actually
pull
that
out
with
API
so
that
we've
added
to
Lib,
C
and
I,
so
that
you
can
then
pass
that
configuration
back
to
the
delete
call,
but
it's
actually
fairly
easy.
A
A
We
therefore
have
a
stable
spec,
which
is
what
1.0
means
really.
We
would
like
to
have
more
complete
test
coverage
and
we
need
to
get
our
build
process.
Signing
builds
that
we'd
like
to
do
that
conformance
test
is
a
is
a
great
idea
actually
this.
So
this
is
a
ask
for
those
of
you
who,
in
the
audience,
are
really
interested.
We'd
we'd
love
for
people
to
step
up
to
help
us
with
that.
A
Currently
CNI
is
a
program
interface
like
CGI.
If,
if
you
old
enough
to
remember
that
the
the
the
caller
execs
a
program
which
is
the
plugins
and
as
I
described
for
instance
in
in
in
the
network,
I
work
on
all
that
program,
does
his
mix
makes
a
network
call
a
rest
call
to
make
something
happen
somewhere
else,
and
a
lot
of
implementations
are
like
that,
a
lot
of
the
bigger
ones.
A
So
what
we
might
want
to
do,
and
then
this
has
been
done
in,
for
instance,
a
storage
interface
CSI
is
making
a
G
RPC
interface,
so
so
the
kubernetes
or
the
runtime
or
whatever
would
somehow
be
aware
that
there's
a
demon
running
and
call
it
directly
that
would
that
would
be
more
efficient.
Give
us
better
facilities
to
know
what
the
heck's
going
on
I
would.
B
A
A
The
idea
of
garbage
collection,
when
you
win
so,
for
instance,
the
bridge
plug-in
the
first
time
you
call
it
if
there's
no
bridge
it
makes
a
bridge
and
then
you
add,
you
know
20
containers
or
whatever,
and
then
let's
say
you
take
them
all
away
again:
they're
still
a
bridge
because
there's
no
nobody,
nobody
said
to
take
the
bridge
away,
there's
no
command
to
take
the
bridge
away.
So
that's
the
idea
of
garbage
collection
when
those
when
there's
nothing
left
using
a
thing
same
applies
to
firewall
rules
and
stuff.
Like
that,
there's
it's.
B
A
B
A
A
A
D
D
E
B
Sure
you
do
have
to
be
a
little
bit
careful
and
basically
our
default
implementation
for
check
is
success.
So
if
you
don't
want
to
implement
check
at
this
point
in
your
plugin
or
you
want
to,
you
know,
do
some
more
research
first
to
figure
out
the
right
way
to
do
it.
You
don't
have
to
do
anything,
it's
backwards
compatible
and
it
will
still
work
exactly
like
it
currently
does.
B
If
you
do
want
to
implement
check,
then
you
do
need
to
be
a
little
bit
aware
around
what
other
chain
plugins
might
do
after
your
plug-in,
if
they
do
alter
some
of
the
things
that
your
plug-in
sets
up,
that's
explicitly
allowed
by
the
CNI
specification,
and
so
that's
why
we
have
the
wording
in
the
specification
around.
If
you
see
something
you
did
in
the
previous
alt,
then
you
can
check
it.
E
How
do
I
identify
in
the
changeset
that
what
corresponds
to
me
and
how
do
I
look
forward
and
then
say
that
okay,
some
somebody
next
to
me,
might
be
modifying
if
I
know
in
advance
what
the
result
is
going
to
be
for
that
modification
by
another
plug-in.
You
know,
I'm
implicitly
creating
a
dependency
on
something
that
is
going
to
happen.
So
is
there
a
way
for
me
to
identify
in
this
result
like
what
did
I
do
and
how
does
in
other
change
impact
my
change,
yeah.
B
B
A
Yeah
sorry
I
said
you
know:
I
spent
half
my
time,
I
emphasizing
that
they're
completely
separate
projects,
and-
and
you
also
might
need
to
remember
that
we
all
work
on
both
of
them-
we're
all
members
of
Signet
work
and
kubernetes,
and
so
there's
where
different
hats
at
different
times.
But
but
yeah
specifically,
we
want
everybody
wants
to
take
the
network
specific
code
out
of
kubernetes,
particularly
couplet
or
docker
shim,
and
so
that
is
why
we
created
a
specific
implementation
in
CNI.
E
So
port
mapping
is
not
about
stitching
a
network
inside
a
part.
Actually
it's
it's
something
about
additional
IP
table
rules
or
routing
or
matting
that
you
are
doing.
It
has
nothing
to
do
with
the
core
networking
word
and
you
move
a
vet
inside
a
network.
Namespace
assign
an
IP,
and
you
know
just
have
some
roads
added
into
the
container.
B
F
B
So
network
plumbing
working
group
is
kind
of
a
informal
group
that
is
part
of
kubernetes.
It's
not
formally
part
of
the
kubernetes
project,
but
it
is
related
and
it's
formed
by
people
who
are
members
of
cig
network
and
some
other
groups
as
well
around
a
common
interest
of
how
you
might
do
multiple
networks
in
a
pod
within
the
context
of
kubernetes.
B
That
said,
it
is
not
directly
related
to
CNI,
although
some
of
the
same
people
are
in
both
groups
but
network
plumbing
working
group,
also
in
the
standard
or
de-facto
specification
that
that
group
developed
specified
behavior
of
multiple
networks
when
a
CNI
plugin
is
used
to
implement
that
behavior.
So
there
are
I
mean
as
Brian
said.
B
Many
of
us
are
members
of
many
different
groups
and
we
all
try
to
you
know
kind
of
work
together
and
develop
things
that
we
think
will
push
different
areas
forward,
but
then
also
try
to
make
sure
that
you
know
individual
projects
work
for
a
number
of
different
use
cases
and
groups
as
well.
So
if
you
want
to
talk
more
about
that,
I'm
happy
to
to
talk
more
I.
A
Mean
part
of
the
essence
of
it
is:
is
things
move
forward
when
someone
shows
up
to
do
some
work?
You
know
a
lot
of
a
lot
of
people
show
up
to
demand
things
or
complain,
or
whatever
and
I
sympathize.
You
know
I.
All
these
people
are
correct
about
the
way
they
feel,
but
it
doesn't
get
anything
done
so
things
get
done
when
people
show
up
so
the
network
plumbing
working
group
is
some
people
who
showed
up
to
do
work.