►
From YouTube: Network Plumbing WG Meeting 2018-09-27
Description
Kubernetes Network Plumbing Working Group meeting for 2018-09-27
A
All
right,
hello,
everybody!
This
is
the
plumbing
working
group
meeting
for
September,
27th
2018,
and
the
first
item
on
the
agenda
is
Peter.
They
have
a
demo
using
Malta
and
they
would
like
to
show
this
demo
and
talk
about
it
so
before
Peter.
If
anybody
else
has
things
they'd
like
to
talk
about
I'm,
going
to
put
that
on
to
the
agenda
doc.
B
Okay,
so
what
we
have
it's
not
so
much
a
demo
is
I'm
gonna.
Tell
you
a
little
bit
about
what
we
did.
We
thought
about
showing
a
demo
but
it'd
be
an
awful
lot
of
screenshots
of
containers
running
and
stuff.
That
wouldn't
tell
you
very
much
so
let
me
let
me
share
a
screen.
I've
got
a
couple,
slides
I
think
there's
about
five
slides
in
total
that
just
talk
through
what
we
did.
B
Couple
slides,
okay,
so
background
this.
Obviously
you
guys
have
been
doing
a
great
deal
of
work
in
putting
together
multi
network
capability
for
kubernetes
and
we've
been
interested
in
that
from
the
point
of
view
of
what
works
on
the
customers
and
what
were
respectively,
selling
to
customers,
and
we've
now
done
some
work
where
we
actually
took
what
we've
got
out
of
this
group.
B
So
that's
the
most
referenced,
plugin
and
so
on,
and
the
spec
and
applied
that
to
a
real
telco
app
ran
it
in
a
kubernetes
cluster
and
got
some
interesting
discover
some
interesting
things
from
that.
So
let
me
talk
you
through
what
we
did
and
what
we
found
so,
firstly,
a
little
bit
of
background
about
what
we're
doing
here
and
why
we're
interested
so
we're
running,
tell
collapse
here
and
I
tell
collapse,
that's
mostly
telephony,
but
it's
also
Network
management,
apps
and
there's
a
whole
bunch
of
multiple
network
requirements
here
that
come
from
various
different
customers.
B
All
of
our
customers
different
are
different
and
they
all
slightly
different
requirements,
but
there's
a
whole
bunch
of
motivations
I'm
not
going
to
read
you
through
them
all,
but
typically
the
two
key
things
to
think
about
are
many
of
our
customers
are
trying
to
get
there.
Many
of
our
customers
are
telcos
or
service
providers
who
are
tying
together.
B
The
networks
of
multiple
other
customers
and
so
they've
got
little
different
physical
networks
and
they
are
the
junction
point
between
them
and
so
they're,
adding
as
a
session
border
controller
or
sometimes
as
a
firewall
in
telephony,
a
little
bit
more
telephony
specific
than
that,
just
a
straight
firewall,
but
they're
between
networks
and
doing
function.
It
relies
on
them
being
active
having
access
networks
and
then
there
are
all
kinds
of
security
and
regulatory
requirements
which
I
won't
bore
you
with
to
do
with
different
things
needing
to
be
on
different
networks
and
so
on.
B
I'll
talk
a
little
bit
more
about
what
the
application
does
and
how
it
fits
together.
In
a
moment,
I
mentioned
for
space
routing
though,
but
the
source
routing
is
an
interesting
thing
that
we
we
realized
was
not
available
from
the
multi
network
group
so
far,
and
the
idea
of
source
based
routing
is
that,
if
you're,
given
an
IP
address
to
send
traffic
to
it's
only
the
application.
B
That
knows
whether
it
wants
to
send
sip
and
therefore
goes
out
over
the
sit
over
the
signaling
network
or
media
and
goes
out
over
the
media
network
or
some
database
access,
and
it
goes
at
some
different
destination.
And
so
in
order
to
make
that
routing
work,
we
generally
have
sourced
based
routing
and
our
VMs
and
there's
no
in
containers
to
do
that.
With.
C
B
B
For
the
most
part,
we're
not
in
we've
not
been
thinking
about
service
chaining,
we're
thinking
largely
about
telephony
applications,
I
suspect
we're
going
to
find
more
interesting
edge
cases
as
we
get
into
more
and
more
complicated
applications,
but
we
haven't.
We
haven't
thought
about
that
use
case
yet
so
the
other
thing
I
mentioned
is
we
thought
of
it
digs
demo
that
involve
fast
networking,
it's
our
iove
ukrs.
We
haven't
done
that,
that's
primarily
because
that
is
more
complex
and
difficult
to
set
up,
and
we
wanted
to
go
for
a
good
use
case.
B
Let's
just
be
quick
and
easy
to
set
up
so
here's
a
network
diagram.
This
is
a
bit
schematic.
There's
a
lot
more
pieces
in
the
picture
than
this,
but
once
you've
seen
this,
you
kind
of
understand
we're
getting
at
the
black
line
is
the
management
network.
That
is
a
network
that
is
not
exposed
to
internet
in
any
way,
it's
purely
internal,
and
then
there
are
red
and
blue
lines
to
signaling
a
media
network
to
e
I.
Put
that
up
thinking.
B
Of
course
everyone
knows
for
the
two
years,
but
of
course
nobody
does
unless
you
work
in
the
telco
space,
two
years,
telephone
user
interface.
That's
the
thing
that
actually
manages
signaling
for
phone
calls
and
the
media
server
is
actually
playing
audio
and
phone
calls
and
on
the
Left
we
have
the
internet,
there's
a
whole
bunch
more
pieces
in
the
actual
product
that
we're
running,
which
is
MBM
mobile
voicemail,
and
that
picture
gives
you
the
flavor
of
a
whole
bunch
of
different
components
on
a
network
and
a
couple
of
them
need
access
to.
B
In
our
case,
a
second
network
I
would
put
signaling
and
media
together
for
purposes
of
the
demo.
So
let's
see
enough
for
what
we're
trying
to
achieve
and
what
we
what
we
wanted
to
demo
and
what
kind
of
capabilities
we
wanted
to
show
off.
Here's
a
little
bit
about
how
we
did
it.
So
we
took
em
VMs
I,
just
said
more
voicemail
as
a
case
study
as
a
nice.
Easy
application
was
fairly
well
contained
requirements.
B
We
were
running
in
kubernetes
in
Azure,
we're
actually
using
the
standard
Azura
kubernetes
service,
but
with
the
networking
replaced
by
some
careful
use
of
daemon
sets.
We
use
the
Malta's
plugin
with
network
attachment
definitions
all
according
to
the
spec,
and
we
had
a
final
management
network
and
we
used
there
zero
Network
for
telephony,
so
that's
all
fairly
straightforward
and
then
a
couple
of
interesting
things
that
came
out
of
this
I
mentioned
source
based
routing
in
or
to
get
source-based
routing
to
work.
We
wrote
to
CNI
plugin,
it's
not
a
very
complicated
CNI
plugin.
E
B
The
sort
the
source
paste
writing
source,
so
the
way
source
based
routing
works
is
that,
let's
imagine,
the
application
wants
to
send
a
sip
message.
I
may
say
that
it
is
going
out
as
a
signaling
message.
There
are
two
parts
of
that.
The
first
part
is
the
application
needs
to
decide
before
it
starts
sending
out
UDP
packets
to
bind
to
the
correct
address
on
the
great
interviews,
and
of
course
it
knows
that
this
is
signaling,
because
well
it's
the
application
doing
signaling.
B
It
knows
it's
trying
to
send
out
sip,
so
it's
configured
to
bind
to
the
correct
address.
The
second
part
of
that
is
that
the
correct
IP
rules
and
IP
routes
must
be
set
up
so
that
traffic
from
that
address
goes
into
the
correct
routing
table
and
the
routing
table
has
the
correct
rule
set
up
in
it.
So
the
first
part
is
just
done
in
the
application.
The
second
part
is
set
up
in
advance
by
the
CNIB
plug-in,
and
this
CNI
plugin
is
a
change.
Cni
plugin
that
runs
after.
E
B
It's
a
different
reading
table
one
per
interface,
yeah
so
fairly,
straightforward,
the
in
technical
terms,
the
we
use
town
to
install
everything,
I'm
a
big
fan
of
helm.
It
makes
everything
much
much
better
and
easier,
so
we
both
installed
all
the
networking
daemon
sets
using
helm,
I
think
it's
rather
amusing
that
you
can
run
helm
at
on
a
running
kubernetes
cluster
and
blow
away
its
entire
networking
infrastructure
and
replace
it
without
anything
going
wrong.
But
that
seems
to
seems
to
be
the
case.
We
also
use
Tom
to
install
our
applications.
B
We've
got
helm
charts
for
our
application.
We
already
had
those
we
just
added
the
ability
to
have
multiple
networks
in
them
and
the
last
little
technical
thing
to
mention
is
we
had
to
get
those
IP
addresses
into
the
containers
to
continue
with
its
IP
address
was,
and
the
way
we
did
that
was,
it
was
reading
an
environment
variable.
We
had
an
init
container
that
would
read
the
annotation
out
of
the
pod
and
write
effectively
right
into
environment
variables
by
way
of
a
little
bit
of
plumbing
the
files.
B
B
Let's
talk
about
what
we
actually
found,
so
the
conclusions
I
was
expecting
that
to
be
much
harder
than
it
was.
It
worked
very,
very
quickly.
Everything
just
worked.
I
found
precisely
zero
bugs
outside
my
own
could
so
that
was
all
good.
It
was
really
nice
and
quick.
A
couple
of
interesting
things
I
did
find.
One
is
surprising
to
me
number
of
CNI
plugins
only
support
a
single
instance.
So
sir,
the
AWS
V
PC
plug-in
final.
They
all
have
baked
into
into
them
to
some
greater
or
lesser
degree.
B
The
idea
that
there's
only
a
single
CNI
plug-in
deployed
in
the
box.
I
guess
maybe
that's
not
that
surprising
people
have
not
run
CNI
plug-ins
for
multiple
interface
as
much
before,
but
if
you,
for
example,
are
running
and
as
the
Azura
CNI
plugin
for
one
network,
you
can't
have
the
Azzurri
and
I
plugin
connects
to
one
interface
to
two
different
interfaces
on
two
different
networks
and
have
two
instances
running.
B
They
just
get
in
each
other's
way
and
the
configuration
gets
very
confused
it's
possible
as
a
way
of
setting
it
up,
but
I
never
worked
to
that.
So
there's
one
of
the
conclusions
I
came
away
with
was
in
order
to
work
with
a
lot
of
the
existing
CNI
plugins
out
there
either
the
cni
plugins
are
going
to
have
to
change
or
will
have
to
be
very
clever
with
configuration
or
something.
B
The
third
thing
is
something
that
I
know
everyone's
aware
of
which
is.
It
was
a
bit
of
a
nuisance
not
being
able
to
do
DNS
lookups
and
have
services.
So
when
you
create
a
bunch
of
instances
and
they're
on
the
signaling
network,
it
would
be
nice
to
have
a
service
that
at
least
get
alleged
DNS
lookup
and
see
what
all's
I
I
P
addresses.
Are
we
had
to.
D
B
That
manually,
by
putting
those
IP
addresses
into
into
a
service
via
the
endpoints
API,
so
it's
a
little
bit
tedious
I
know
that
puerile
I
think
was
putting
out
a
document
about
resolving
this
and
there's
been
some
discussions.
But
that'll
be
good
thing
to
have,
and
the
final
thing
as
a
conclusion
is
source
based
routing
CNI
plugin,
so
we've
written
a
CNI
plug-in
that
does
source
based
routing
and
we're
looking
to
open
source
it.
B
We
don't
want
to
be
an
internal
thing
that
only
we
use
we'd
like
to
open
source
and
somewhere
we're
still
figuring
out
how
to
do
that
and
we'd
like
to
hopefully
find
a
good
place
to
put
it
on
the
web.
I
deal
with
other
plugins,
possibly
with
all
the
cni
plugins.
If
we
can
get
the
CNO
maintainer
to
agree
to
that,
although
I
haven't
asked
them
yet,
but
maybe
somewhere
else,
we're
not
sure
but
I
think
the
meta
conclusion.
On
top
of
all
that
is
we
needed
to
run
with
multiple
networks.
B
We
got
multiple
networks
working
and
this
is
a
real
application,
that's
running
in
a
real
cloud
and
you
can
make
phone
calls
to
it
and
it
is
using
multiple
networks,
which
was
very
positive,
so
I
was
extremely
pleased
at
how
quickly
the
whole
thing
came
together.
It
took
us
I
think
less
than
a
week
to
get
everything
up
and
running
and
working,
including
all
the
figuring
out
and
how
things
work
and
writing
the
cni
plug-in
as
part
of
that.
So
that
was
all
good
news
and
I
think
that's
it
I
think
I'm
done
any
questions.
B
A
Awesome,
thank
you.
Peter
yeah
I
had
a
question
around
Malta's
in
the
network,
C
RDS,
and
things
like
that.
Did
you?
What
were
you
experiences
with?
That?
Did
you
run
into
any
issues?
I
guess
you
know,
as
perhaps
one
of
the
first
more
productized
uses
of
the
stuff
that
we've
been
working
on
here.
You
know,
what
did
you
find?
Were
there
any
kind
of
rough
edges
with
that?
How
are
you
creating
those
CRD
objects
that
kind
of
thing
sure.
B
So
we're
creating
them
with
helm,
charts
and
the
way
we
created
the
helm
traps
was
I,
just
handcrafted,
the
ammo
files
and
put
a
Jason
config
into
there.
The
biggest
problem
I
came
across
was
I,
kept
missing
commas
out
and
I
cannot
blame
Malta's
for
me
being
unable
to
remember
to
fit
all
comas
into.
B
But
far
from
that,
I
found
absolutely
no
bugs
it
just
all
worked,
which
was
extremely
gratifying.
Okay,
I
stole
straight
out
the
box
and
it
seemed
very
straightforward
to
do
so.
I
was
what
I
was
expecting
to
find
more
things
wrong
than
I
found,
and
it
just
all
worked
very
smoothly.
So
so
no,
it
was
great.
So.
A
Who
did
the
work
on
this?
Do
you
think
that
the
admission
controller
that
Carroll
and
his
team
have
written
and
I
think
is
available
out?
There
might
help
with
this
syntax
issue?
It's
at
least
meant
to
do
admission,
control
on
the
objects
and
perhaps
point
out
syntax
errors,
the
Jason
the
lot
earlier.
Then
it
would
get
down
to
Malta's.
For
example,
yeah.
B
I
think
that
would
be
good
actually
because
the
biggest
error
we
but
the
biggest
error
I
found
was
definitely
silly
little
Jason
errors,
you
go
and
you
set
something
up.
Man
it
fails,
and
then
it
I
mean
it's
a
pretty
obvious
error
because
things
start
working
quite
dramatically,
but
the
one
problem
is
once
you've
done.
Something
as
fundamentally
is
broken
all
the
networking.
It's
not
all
that
easy
to
diagnose
that,
because
things
start
falling
over
quite
complicated
ways.
So,
yes,
I
think
that
problem
would
be
useful.
Okay,
we
got
away
without
it.
B
This
is
probably
totally
impossible
and
impractical.
Checking
that
as
valid
Jason
would
be
very
good,
but
this
the
second
level
would
be
checking
that
it's
valid
Jason,
that's
consistent
with
whatever
the
particular
network
plug-in
requires
and
that
second
bit
is
almost
certainly
impossible
in
practice.
Right
because
there's
so
many
different
plugins
in
there,
they
all
require
different
stuff.
B
The
routine
that
we
were
using
was
very
basic
in
that
the
routing
the
written
came
in
through
the
CNI
plugin
config.
So
the
particular
way
that
we
did
it.
We
did
a
mixture
of
flannel
and
the
final
routing
rules
are
very
trivial
in
the
final
really
just
sets,
so
it
says:
send
it
all
the
traffic
out
through
this
one
IP
address
and
let
everything
else
that's
something
else
deal
with
it.
B
However,
you
could
add
more
routes
to
the
final
config
for
the
default
network.
So
that
way
you
can
get
Network
config
and
you
put
them
in
through
network
attachment
definitions,
and
that
was
flexible
enough
for
us.
The
way
that
the
source
based
routing
plugin,
worked
with
routing
and
in
terms
of
putting
him
into
separate
tables
was,
it
would
effectively
come
along
after
the
ordinary
plugin
had
said.
It's
routes
up
the
routes
for
that
ordinary
plugin
would
be
set
to
send
traffic
to
some
particular
interface.
B
It
would
take
all
of
those
just
move
them
into
a
separate
table.
So
the
assumption
was,
you
said:
you
set
all
your
routes
at
once
through
the
ordinary
cni
plugin,
by
passing
them
in
through
the
network,
attachment
definition
and
then
the
source
based
routing,
plugin
kind
of
goes
and
moves
them
into
a
separate
table.
That
might
not
be
bomb-proof
in
every
case,
but
it
was
good
enough
for
the
case
that
we
were
doing
I.
B
Should
say
if
anyone's
interested
with
the
source
base
routine
the
actual
code
for
that
we're
gonna
make
that
public
in
India
course
we
have
to
go
through
some
jump
through
some
hoops
before
we
could
do
that,
and
we
also
I've
got
the
helm
charts
if
people
are
interested
in
seeing
the
fryer,
we
set
up
the
network,
attachment
definitions
and
install
the
daemon
sets,
and
so
on.
So
we
could
share
those
as
well.
A
E
D
A
A
Basically,
there
were
some
persistent
problem
with
forget
action,
which
I've
probably
talked
about
in
the
past,
at
least
some
of
you
on
the
call,
and
so
we
ended
up
changing
that
get
action
into
a
check
action
which
preserves
the
ability
of
plugins
to
report
errors
in
The
Container
Network,
which
was
one
of
the
two
big
things
we've
get.
So,
instead
of
mixing
the
report,
errors
from
the
container
and
the
plug-in
and
trying
to
reconstruct
the
result,
which
was
the
the
two
goals
of
get,
we
changed
the
spec
and
this
the
spec
is
not
released.
A
So
the
this
change
would
be
part
of
the
zero
for
version
of
the
or
zero
five
version
of
the
cni
spec
zero.
Four
sorry,
it
is
now
broken
into
a
check
action
that
plugins
can
implement
that
will
report
those
errors
and
then
some
additional
code
in
Lib,
C
and
I,
which
is
kind
of
like
the
driver
that
runtimes
like
kubernetes,
would
use
to
cache
the
results
on
disk.
A
It's
really
meant
to
provide
all
the
information
that
the
plugin
originally
sent
back
to
kubernetes
in
a
way
that
kubernetes
doesn't
have
to
add
more
code
and
add
that
so
those
are
the
the
ideas
with
that,
and
so
now
that
we've
got
the
spec
part
there.
We
are
converting
the
plugins
over
to
support
this
and
it
should
be
pretty
easy
because
the
caching
parts
already
been
done
for
a
while.
A
It's
just
moving
the
plugins
from
some
of
the
get
implementation
we
had
into
check,
but
that's
kind
of
a
net
code
loss,
and
we
expect
that
to
go
fairly
quickly.
Okay,
well,
this
doesn't
give
you
a
hard
date
for
when
the
C&I
spec,
0
4
will
come
out
and
the
plugins
version
0
7,
which
implements
specs
before
yeah.
It
should
be
a
matter
of
weeks,
two,
maybe
at
most
like
once
I
think
oh
cool.
D
A
Maxim
now
we've
got
some
additional
resources
at
Red
Hat
that
we've
been
able
to
put
to
work
on
some
of
this.
So
hopefully
that
will
move
things
along
up-stream
awesome
thanks
and
then,
of
course,
the
the
other
side
to
this
is
APR
to
actually
use
some
of
this
functionality
in
kubernetes,
but
I.
Don't
think
that
would
be
too
awful
first
off.
That
would
be
updating
the
docker
shim
CNI
driver,
but
then
also
we
probably
want
to
update
the
creo
OCI
CNI
driver
as
well.
E
E
F
E
C
Okay,
side
note
down
like
I,
didn't
know
the
process
for
this,
but
I
took
an
action
item
to
do
the
consistent
interface
naming
so
I
did
put
a
description
in
the
document.
Okay,
so
like
it's
a
longer
description,
so
whenever
you
want
to
handle,
if
we
can
actually
take
a
look
at
it
maybe
next
week,
because
I
don't
know,
Mike
is
on
the
call,
but
he
wanted
to
take
a
look
at
it
because
he
was
on
asking
for
more
details.
But
I
did
put
it
in
okay.