►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Kubernetes Networking: How to Write a CNI Plugin From Scratch - Eran Yanay, Twistlock
CNI (Container Network Interface) plugins are the cornerstone of Kubernetes networking. CNI is the standardized way used by Kubernetes to expose network devices to pods, responsible for pod to pod communication across physical nodes in your cluster. During this talk we will: - Explore the details of the CNI plugin interface - Understand how it is used with Kubernetes - Provide a detailed walkthrough of a simple CNI plugin from scratch Attendees in this talk will gain insight into the process of creating a CNI plugin and get familiar with networking decisions required for having their pods connected and reachable from within the cluster and the internet.
https://sched.co/MPd2
A
I
hope
you'll
enjoy
the
last
talking
loop,
conne
kiss
for
the
random
talk.
So
a
little
bit
about
myself.
My
name
is
irani
and
I.
In
the
past
few
years,
I
walk
a
twist,
lock,
I've
been
a
software
engineer
and
now
in
a
team
leader.
This
is
my
email.
You
can
feel
free
to
reach
out
later,
if
you
don't
have
time
for
the
questions.
A
No
one:
okay,
how
many
of
you
have
learned
this
command
and
you
have
no
idea
what
it
does?
Okay,
so
I'm
going
to
cover
many
topics
about
CNI
plugins
today
and
sinaia
in
general,
and
hopefully
within
a
few
slides
I
will
get
back
to
this
example,
and
you
will
have
a
better
understanding
of
what
it
does.
So.
A
The
agenda
for
today
is
to
eventually
understand
how
to
write
a
scene
I
plug-in
from
stage
in
bash,
using
only
simple
command
and
tools
and
we'll
start
with
an
overview
of
what's
in
II's,
we'll
continue
with
understand
how
it's
been
used
in
high
level
in
kubernetes,
what
it
does
when
it
operates,
and
eventually
we
will
conduct
a
small
demo
that
the
most
as
everything
that
we
will
soon
see
in
this
talk.
So
what
is
CNI?
The
United
stands
for
container
networking
interface.
A
It's
the
intervals
between
the
container
runtime
in
our
case
kubernetes,
and
the
network
implementation.
When
the
cinah
plugin
is
running,
it
is
responsible
for
configuring.
The
network
interfaces
and
possibly
routes,
and
also
another
thing
that
both
understand
is-
is
that
the
sinner
plugin
concerns
itself
only
with
the
networking
connectivity
of
the
pod.
This
is
the
link
to
the
latest
see
an
iceberg.
It's
0.4,
you
can
follow
the
link
and
read
further
about
it.
For
the
rest
of
this
talk,
I'm
actually
going
to
focus
on
the
latest
on
the
previous
spec,
which
is
0.2
8.1.
A
There
aren't
many
crucial
differences
between
them
for
what
we
are
going
to
touch
in
this
talk.
It's
going
to
be
practically
the
same.
The
only
reason
that
I'm
doing
it
is
for
the
sake
of
the
demo,
which
for
kubernetes
1.14,
is
still
referencing
the
whole
spec,
so
CNI
plugin,
often
composed
of
two
components.
The
first
component
is
the
CNI
binary,
which
is
the
plug-in
itself
to
think
that
is
implementing
the
interface
and
one
is
flying.
It
is
being
executed
by
the
couplet
and
is
responsible
for
configuring.
A
The
network
interface,
it's
deposit
setup,
the
connectivity
of
the
container.
The
second
part
is
the
diamond:
well
essentially,
most,
if
not
all
CNI
providers
for
kubernetes.
They
also
provide
a
demon
that
is
going
to
run
on
every
node
in
the
cluster
and
when
it
is
running
it
will
be.
Managing
the
legibility
of
Phil
Paul
saw
the
dull
cluster,
which
ability
means
that
every
port
is
accessible,
reachable
from
every
node
and
in
when
talking
about
networking
in
kubernetes.
A
It's
not
such
a
trivial
task,
because
we
are
talking
about
virtual
addresses
that
are
local
to
each
node
and
are
not
necessarily
known
at
any
point.
In
the
cluster,
meaning
that,
if
you
have
two
nodes,
then
each
node
is
allocated
with
some
address
space
for
its
pods,
and
this
address
space
is
virtual
and
not
only
to
the
node
that
is
responsible
for
this
address
space.
A
So,
for
the
rest
of
this
talk,
I'm
going
to
focus
mostly
on
the
first
part,
which
is
the
Siena
implementation
itself,
meaning
that
we'll
see
code
examples
and
talk
about
the
way
that
the
pod
acquires
its
network
interface,
business
languages
in
IP
and
so
on
and
living
the
which
general
eligibility
about
lost
eligibility,
internet,
internet
and
and
so
on.
We
will
leave
it
for
the
end
of
this
talk
for
the
demo
and
then
we'll
experience
the
problems
that
arises,
though,
so
just
to
make
it
clear.
A
The
CNI
binary
itself
is
only
composed
of
those
two
components
so
on
the
left,
you
can
see
a
configuration
file
which
is
simply
as
a
JSON
file
JSON
formatted
file.
It
contains
a
few
required
fields,
such
as
the
cni
version,
which
references
the
respect
that
the
plugin
is
fulfilling
the
name,
which
is
the
plug-in
name
and
the
type
which
is
the
name
of
the
binary
to
look
for
in
the
in
the
folder
holes
of
the
cni
like
binary
implementations
themselves,
it
can
contains
also
additional
optional
fields
which
I'm
not
going
to
discuss
about
today.
A
They
are
not
shown
here
as
well,
and
it
can
contains
also
custom
fields,
in
this
case
I'm,
showing
an
example
of
the
posada,
which
is
a
custom
field
which
I
decided
to
put
in
the
config,
and
we
will
soon
see
why
I
chose
to
put
it
there
and
how
I'm
going
to
use
it
later
on
on
the
right
is
the
implementation
itself.
It's
a
binary
executable
in
this.
In
this
case,
I
decided
to
do
it
in
bus
and
for
the
point
3.1
specification.
A
It
has
to
support
those
four
commands
that
command,
which
is
called
when
the
cni
plug-in,
is
executed
for
configuring.
Networking
interfaces
for
a
new
container
the
delete
command,
which
is
called
when
a
container
has
been
deleted,
and
now
we
need
to
possibly
perform
some
sort
of
cleanup,
maybe
release
an
IP
perform
some
other
operations
without
them.
Regarding
our
own
implementation,
oh
and
what
you
choose
to
do,
the
gate
command
is
rather
than
bigger,
so
rather
not
to
talk
about
it
today
at
all.
A
A
So
going
back
to
the
wave
example
that
I've
shown
you
before
so
the
command
that
we've
seen
simply
download
the
ml
file
and
runs
it,
and
what
is
in
this
file
is
simply
a
demo
set
implementation,
a
configuration,
and
you
can
notice
that
it's
going
to
run
the
host
network-
and
this
is
important
for
CNI
plugins,
because
one
it
allows
us
to
deploy
even
before
the
cni
plug-in
is
configured.
And
then
we
don't
need
an
IP
to
be
allocated
for
us,
so
our
deployment
can
run
everywhere,
even
when
the
amount
is
still
not
ready.
A
And
the
second
reason
for
being
on
the
host
network
is,
of
course,
because
we're
going
to
manipulate
the
host
network
stack
by
performing
some
ID.
Maybe
IP
table
changes,
outs
and
other
stuff.
So
we
need
to
understand
folk,
and
the
second
thing
was
noting
here
that
you
can
see
that
the
wave
container
itself
it
has
mounts
to
specific
paths
on
the
disk
to
the
locations
where
we
will.
We
need
to
place
the
configuration
file
and.
A
A
A
Okay,
so
you
can
see
that
the
wolf
container
is
going
to
have
mounts
with
specific
folders
in
the
discs
that
are
going
to
hold
it
CNI
Benelli
itself
and
the
configuration
file
and
when
it's
going
to
run
it's
going
to
on
the
LAN
script.
So
if
take
a
deeper
look
at
the
landscape,
you
can
see
that
eventually
it's
going
to
do
multiple
things
and
then
it's
going
to
mention
operations
such
as
make
sure
that
the
CNI
binaries
folder
exists,
created
it.
A
So
now
that
we
have
a
simple
but
good
understanding
of
the
overview
of
what
CNI
plugins
is,
let's
see
how
it's
being
used
in
kubernetes,
so
a
new
pod
is
scheduled
to
run
and
then
the
API
server
is
going
to
communicate
with
the
cubelet,
and
it's
going
to
give
it
configuration
about
the
pod
and
it's
going
to
tell
him
to
schedule
the
pod
right.
So
the
next
step
that
is
going
to
happen.
If
the
de
cubit
is
going
to
set
up
the
pod
nattokinase
space,
can
anyone
guess
our
does
it?
A
Okay?
So
it's
going
to
set
up
the
path
container,
the
post
container,
which
I
assume
most
of
you
all
of
you
are
familiar
with.
It
has
a
couple
of
walls.
One
of
them
is
to
maintain
this,
the
pod
network
namespace
and
once
it
is
initialized,
but
the
same
I
plug-in
it
is
going
to
be
shared
across
all
the
containers
in
the
pod
and
then
they
can
properly
communicate
with
whoever
they
need
to
communicate
views.
Now.
A
At
this
point,
the
punitiveness
space
only
has
the
loopback
interface,
and
the
next
thing
that's
going
to
happen
is
that
the
cubit
is
going
to
invoke
the
CNI
plugin.
It's
going
to
tell
him,
there's
a
new
port
that
I
need
to
set
up
here,
that
it
was
about
it
and
configure
it,
and
then
the
sinner
plug-in
is
going
to
do
multiple
things.
It's
going
to
create
the
interface,
the
network
interface
in
the
for
the
pod.
A
So
so,
let's
see
how
the
cni
plug-in
is
been
actually
used
by
the
cubelets.
Oh,
so
it's
going
to
receive
information
in
two
different
forms.
The
first
piece
of
information
it
receives
is
the
container
info
stoke-on-trent.
The
container
info
will
is
received
by
environment
variables,
and
it
contains
information
such
as
the
command
in
this
case
add
forward
in
a
new
container.
A
The
container
ID,
which
is
the
unique
identifier
of
the
container,
the
net
NS,
which
is
the
path
on
the
disk
in
the
POC
file
system,
for
the
network,
namespace
of
the
container
and
the
name
of
the
interface
that
we
wants
to
assign
to
assign
to
it.
The
single
piece
of
information
is
the
same
I
config,
which
is
essentially
all
the
content
of
the
center
config
file
that
we
chose
with
all
the
fields,
the
fields
that
we
chose
to
embed
in
it,
and
it
is
as
if
we
are
the
standard
input
at
this
case.
A
At
this
point,
the
senior
plugin
is
going
to
do
multiple
things
and
it's
going
to
set
up
mental
interfaces
according
to
the
given
spec
according
to
get
the
given
configuration
and
eventually
it's
going
to
output
to
the
standardout
quotes
indicating
to
the
couplet
the
things
that
you
have
done,
such
as
the
list
of
of
interfaces
that
it
set
up.
The
MAC
address
the
name,
the
default
gateway,
the
IPS
and
so
on.
A
So,
let's
take
a
deeper
look
at
the
internals
of
what
the
sinner
plugin
is
actually
doing,
so
each
pod
has
its
own
at
work,
namespace
and
an
easy
way
to
connect
the
port
network.
No
space
to
our
host
is
to
set
up
a
bridge
and
collect
the
pod
into
the
bridge.
You
can
think
of
the
bridge
as
a
physical
switch
that
when
you
take
the
port
and
connecting
to
it
is
it's
like
if
you
were
taking
a
wire
and
connecting
it
to
one
of
the
switch
ports.
A
So
in
order
to
connect
it
to
the
bridge
when
he
was
going
to
create
something
called
the
whisper,
the
visible
will
Charlotte
on
a
device.
Pearl
is
the
power
of
evil
choice
on
a
devices
that
I'll
the
data
created
in
pearls.
There
are
the
inter
connected
pearls
that
are
basically
when
were
creating
them.
Every
packet
is
sent
at
one
side
is
immediately
received
at
the
other
side.
A
So
the
a
very
good
use
case
for
wilt
for
those
spells
in
Linux
in
the
scope
of
containers,
is
that
we
can
take
one
one
one
end
of
the
pair
and
put
it
in
then.
In
the
opponent
open
space
and
keep
the
other
end
at
the
at
the
host
network
near
space,
and
then
they
can
actually,
as
done
as
the
cosmetic
next
pieces
connecting
the
pod
to
the
hosts,
so
we
took
one
end,
would
connect
it
to
the
to
the
pod
and
the
other,
and
we
kept
in
the
host
connected
connecting
it
to
the
bridge.
A
Then,
at
this
point,
most
of
the
work
is
done.
Every
port
that
is
going
to
be
scheduled
to
learn
will
will
be
the
scene.
I
plugin
will
be
called
and
it
will
set
up.
The
virtual
Ethernet
devices
spell
connecting
it
to
the
bridge,
setting
up
an
IP
for
the
device
and
then
with
help
of
common
Linux
tools
such
as
IP
laws,
I
beat
a
basis
and
another
well
going
to
make
the
make
the
appropriate
outs
for
traffic
to
flow
call
it
to
the
port
and
from
the
pod
to
other
pods
in
the
cluster.
A
So
now
I'm
going
to
get
to
go
even
lower
to
the
technical
details
so
on
the
right
you
can
see
an
example
for
an
environment.
I
have
confronted
on
my
laptop
that
I'm
going
to
use
later
for
the
demo
it
it
has
two
nodes:
node
1
and
node
2.
It's
know
the
both
nodes,
communicating
over
the
NP
network
interfaces
over
the
tender
10.8
and
when
I
created
the
cluster.
Each
node
got
allocated
with
some
pod
address
space
for
the
pods
that
are
running
on
this
node.
A
So
the
config
that
I
need
that
that
I
decided
to
create
here
is
the
configured
says
that
holds
the
port's
either.
The
pole
setter
for
the
specific
node.
It's
going
to
be
different
for
every
node
that
I'm
going
to
connect
in
Mecosta,
so
the
possible
will
be
10,
12,
4000
24.
This
is
the
example
of
the
config
in
node
1,
and
this
is
the
skeleton
for
the
cni
executable
itself.
So
for
the
rest
of
the
few
sides,
I'm
going
to
focus
on
the
add
command.
A
So,
as
we
said,
the
first
thing
that
we
want
to
do
is
to
set
up
the
bridge
so
I'm
going
to
calculate
the
port
side,
which
is
by
extracting
the
config,
which
is
ID
in
the
standard
input.
So
I'm
going
to
cut
this
down
at
the
8
port,
pass
it
to
the
JQ
tool,
which
is
a
simple
tool
to
manipulate
JSON
strings
in
bash
I'm.
A
Going
to
take
the
pole
setter
for
me,
once
I
have
the
port
side
I'm
going
to
do
just
some
manipulation
on
it
to
calculate
the
gateway
address
which
is
going
to
be
the
address
of
the
bridge
and
then
I'm
going
to
use
the
bridge
Atul's
package,
which
contains
the
bill.
Ctl
command
line,
a
command
line
tool
to
create
a
bridge
I'm
going
to
create
a
bridge
that
is
called
CNI
0,
then
I'm
going
to
enable
it
wing
it
up
and
I'm
going
to
set
it
with
the
IP
of
the
port
idle
gateway.
A
The
next
thing
that
we
said
that
we
need
to
do
is
to
move
the
eth0
to
the
coordinate
of
night
space.
So
something
was
nothing
about.
Network
name
spaces
in
Linux
is
that
they
exist
in
two
forms.
The
first
form
is
defined
by
the
process
and
under
walk
the
peed
name
and
been
discarded
when
the
process
dies
and
the
other
form
is
under
of
Allen
athena's,
which
does
name
Network
no
spaces,
they
long-lived
they
are
created
and
deleted.
A
When
we
decide
they
are
not
bound
to
any
process
and
what's
interesting
about
those
Network
no
spaces
is
that
they
can
be
manipulated
using
the
IP
link
commands.
They
can
be
manipulated
in
a
way
that
we
can
perform
actions
inside
the
network
namespace.
Similarly
to
if
we,
if
we
do
Hugh,
Cottle,
exact
or
doctor
exact,
so
we
can
still
manipulate
the
the
network
namespace
in
these
cases
of
the
first
case
that
doesn't
reside
in
Van
Etten
s.
But
if
we
create
the
same
link
into
this
folder,
we
can
still
manipulate
it
using
this
tool.
A
So
I'm
going
to
set
the
seam
length
of
the
Sina
internals
from
the
poke
folder
under
Vallon
18s,
and
it's
going
to
be
called
CNI
container
ID,
which
is
the
unique
name
of
the
container,
then
one
once
it's
there.
I
can
perform
various
actions
on
it
and,
in
this
case
first
I'm
going
to
use
the
hyperlink
command
to
set
it
0
I'm,
going
to
set
its
namespace
to
be
in
CNI
container
ID.
A
A
Take
is
from
the
point
of
view
of
the
CNI
that
ready
tools
such
as
the
DHCP
and
host
local
and
other
means,
as
well
I
decided
to
do
something
that
is
very
simple
and
is
good
enough
for
the
sake
of
this
talk,
which
is
this
I,
am
simply
going
to
use
the
file
that
is
going
to
hold
it.
The
last
IP
and
I'm
incremented
by
1
every
time
update
the
file
and
then
I
have
the
IP
and
I
can
use
it
and
then
I'm
going
to
do
some
other
trivial
operations.
A
The
first
operation
I
am
going
to
execute
a
command
in
the
Palmetto
Connect
space
that
pulls
the
interface
up
I'm,
going
to
assign
the
IP
to
it.
So
you
can
see
I'm
going
to
ripen
at
an
exact
in
the
container
ID
set
that
IP
to
the
device
in
I've
name,
which
is
it
it
s,
0
and
lastly,
I'm
going
to
set
the
route
to
be
the
default
gateway,
which
is
the
blue
JP.
A
A
So
a
little
bit
about
the
demo
before
we
start.
As
you
already
understand,
there
is
no
demon
implementation
in
here,
so
we
only
have
the
sinner
plug-in
itself,
yeah
and
I'm
simply
going
to
deploy
a
simple
three
pods
that
are
going
to
run
on
both
nodes
and
we'll
see
what
works
and
what
doesn't
and
for
everything
that
doesn't
work
essentially,
in
the
common
use
case,
it's
going
to
be
set
up
by
the
demon
so
for
the
demo,
we'll
just
do
it
by
hand.
So
this
is
the
my
environment
we
can
see.
A
A
So
not
one
got
allocated
with
the
posted
l
to
be
10,
200,
4000
and
notes
with
a
point
1
and
a
deployment
that
I'm
going
to
are
now
she's
going
to
deploy
an
alpine
container.
That
is
going
to
learn
or
note
1
an
engineer's
container.
That
is
also
going
to
no,
not
one
and
another
nginx
container
or
note.
2
and
I
already
have
the
Senate
plug-in
installed
from
like
before
we
started.
This
talk
so
by
now
expect
my
pots
to
already
have
an
IPA
assigned
yeah.
Okay,
so
all
pods
all
will
be
running.
A
We
need
the
designer
plugin
was
already
executed
and
you
can
see
that
the
IPS
are
indeed
in
implemented
by
one
between
them.
You
can
see
that
up
and
wine
is
with
point
5
and
and
engineer's
point
four.
So
I
already
expect
a
common,
simple
connectivity
to
be
available
from
the
host
to
the
to
the
pod
because
they
are
already
connected.
A
So
what
to
put
a
pod
to
pod
communication
in
the
same
node
is
not
working,
and
why
so?
The
first
problem
is
this:
every
port
resides
in
a
different
network,
namespace
and
traffic
flowing
from
one
port
to
the
other.
From
the
point
of
view
of
our
host
is
considered
to
be
external
traffic,
it's
very
similar
to
the
case
of
well.
A
You
have
a
router
and
you
have
multiple
users
connected
to
it
and
you
want
to
allow,
although
of
connect
a
packets
flowing
between
them
in
order
to
packets,
to
allow
packets
to
flow
from
one
pot
to
the
other.
We
need
to
accept
it
and
wear
something
in
Linux
called
I.
So
most
of
you
know
IP
tables
and
there's
the
forward
chain,
which
is
the
one
responsible
for
allowing
such
traffic.
A
So
if
we
investigate
and
look
at
the
full
chain
consider
this,
we
have
multiple
rules.
One
of
them's
belongs
to
docker,
which
is
what
particular
doing
something
very
similar
to
what
we
just
seen,
and
there
is
no
ruling
here
that
says
to
allow
traffic
from
our
port
side
erects.
So,
let's
add
one
so
I'm
going
to
append
to
the
full
chain
a
rule
that
says
that
anything
that
originates
form
my
port
side,
arrange
and
I.
Don't
care
about
the
accuracy.
I
can
just
say,
deposited
the
length
of
the
entire
cluster
to
accept
it.
A
A
A
Is
that
communicating
with
other
pots
is
still
not
working?
We
can
talk
from
here.
It's
going
to
be
similar.
If
you
are
going
to
try
from
the
pot
itself,
it's
not
going
to
walk,
and
the
reason
is
that
we
said
that
the
poles
either
in
each
node,
is
known
only
to
it
and
it's
not
not
known
to
anyone
else,
and
we
need
to
instruct
our
kernel
to
say
if
we
have
a
packet
that
is
going
to
notes
to
IP
address
then
routed.
A
A
A
Okay,
so
now
we
have
packets
flowing
from
port
to
port
locally
and
also
from
port
across
the
cluster
itself.
In
the
last
thing
that
we
want
to
fix
is
Internet
connectivity.
So
if
I'm
going
to
try
to,
for
example,
ping
the
internet,
it's
not
going
to
walk,
and
why
so
we
need
logs.
We
have
something
called.
A
The
common
way
to
fix
this
in
linux
is
by
using
sauce
nuts
sauce
must
is
a
tool
that
allows
us
to
replace
the
sauce
IP
in
the
packet
when
it
leaves
the
host
so
that
when
the
packet
leaves
the
host,
the
host
will
change
the
source
IP
to
the
host
IP
and
then
when
the
packet
is,
and
when
the
responses
arrived
arrives
back
to
the
host,
it
will
replace
it
back
and
center.
The
response
to
the
container.
A
This
is
something
that
is
called
masquerading
and
we
can
apply
a
rule.
That
says,
add
the
rule
to
the
IP
tables
to
the
post
holding
chain,
which
is
the
last
chain
that
that
is
going
to
evaluate
outgoing
packets
that
are
living
our
host
and
I
need
to
add
it
to
the
not
stable
and
every
packet.
That
is,
that
originates
from
the
pods,
which
is
this
pot
sidelines,
and
it's
going
out
not
to
the
CNI
plate,
meaning
that
it's
going
out
to
the
Internet
masquerade.
It
meaning
apply
the
sauce
nut
and
then
this
should
walk.
A
A
B
A
A
B
A
C
Hi
I'm
Jenna
deli.
Thank
you
for
the
talk
and
introducing
us
with
writing
simple
cni
plugins.
So
this
is
like
similar
to
calico
right,
because
calico
works
in
a
similar
way.
I
looked
at
it
like
a
long
time
ago,
but
they
are
also
using
some
route
stuff
and
other
things.
So
is
this?
Like
calico
Coleco
also
works
like
same
right.
All.