►
From YouTube: Lightning Talk: Integrating Envoy with DPDK-based virtual networks - Raja Sivaramakrishnan
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Lightning Talk: Integrating Envoy with DPDK-based virtual networks - Raja Sivaramakrishnan, Voltera Edge Services
This talk will focus on the work we have done at Volterra to
integrate Envoy with a user space network stack built on top
of DPDK (Data Plane Developer Kit) with support for load
balancing across multiple virtual networks.
A
I'm
not
just
around
Krishna
I'm,
a
software
engineer
at
Volterra
we're
a
startup
based
out
of
the
San
Francisco
Bay
Area,
and
today
I
will
be
talking
about
the
work
that
we
have
done
to
integrate
on
ROI
with
DP
DK,
and
what
your
networking
quick
overview
about
our
company.
We
are
an
edge
services
company.
A
We
believe
that,
with
the
advent
of
many
smart
connected
devices
and
the
explosion
of
bandwidth,
that's
going
to
happen
with
5g.
Increasingly,
a
lot
of
the
traffic
on
the
Internet
is
going
to
be
generated
at
the
edge
and
pushed
upstream
as
opposed
to
the
early
days
of
the
internet
when
most
of
the
content
was
downloaded
by
users
from
upstream.
So
that
is
the
problem
we
are
trying
to
solve
and
towards
that
end
we
are
building
our
own
infrastructure.
A
So
it
might
be
preferable
to
write
those
filters
in
in
JavaScript.
We've
also
added
support
for
multiple
virtual
networks
making
envoy
multi-tenant,
and
we
do
that
with
networking
which
I'll
talk
about
in
later
slides
and
finally,
in
this
scenario,
we
expect
that
performance
would
be
critical,
and
that
is
where
DP
DK
comes
in.
A
Another
thing
I
want
to
mention
is
that
in
in
our
deployment
on,
why
does
not
run
as
a
site
guard?
It
runs
out
as
its
own
process
inside
a
container.
It
gets
packets
from
the
wire,
and
if
it
chooses
to
load
balance
them,
it
would
send
them
back
out
on
the
wire,
and
there
are
cases
where
it
will
process
them
locally.
A
A
So
the
net
result
of
all
of
this
is
that,
with
a
kernel
based
application,
we
can
process
a
few
hundred
thousand
packets
per
second,
and
not
all
of
that
can
be
attributed
to
the
overhead
of
the
TCP
stack
itself.
There's
all
this
additional
infrastructure,
that's
around
the
network
stack
system,
calls,
interrupts
context,
switches,
and
all
of
that
that
is
what
contributes
to
the
bulk
of
the
overhead.
A
Network
stack,
so
DP
DK
does
all
of
the
I/o
in
user
space,
so
we
completely
bypass
the
kernel
and
it
continuously
pulls
four
packets.
So
that
way
we
don't
have
the
overhead
associated
with
interrupts.
It
also
uses
huge
pages.
So
instead
of
the
typical
4k
page
size,
you
can
use
either
2
megabytes
or
one
gigabyte,
huge
pages
and
that
helps
with
performance
by
reducing
the
number
of
TLB
misses
and
the
DP
DK
libraries
have
api's
that
allow
for
easy
communication
between
multiple
CPU
cores
without
using
locks
and
api
is
to
do
batch
processing
of
packets.
A
So
the
net
result
of
all
of
this
is
that
with
DP
DK
we
can
achieve
an
order
of
magnitude
better
performance.
Then
you
can,
with
the
core
network
stack
the
amount
of
processing
that
you
can
do
determines
how
many
packets
you
can
process
per
second.
So
if
you
have
an
application
that
does
a
lot
of
per
packet
processing,
of
course
the
numbers
would
be
a
little
smaller,
but
for
simple
benchmarks.
A
A
So,
on
top
of
DP
DK,
we've
added
support
for
virtual
networking,
and
this
is
what
allows
support
for
multi-tenancy.
So
each
tenant
would
gets
get
its
own
virtual
network,
which
is
basically
its
own
VR
F
and
on
the
wire
we
use
overlay
tunnels
and
the
overlay
tunnels
are
what
separates
the
virtual
infrastructure
from
the
physical
infrastructure.
So
you
could
have
changes
in
the
virtual
network
without
having
any
impact
on
the
physical
network
infrastructure
itself,
and
for
that
we've
built
a
video
module
and
integrated
that
with
DP
D
K.
A
So
it
is
this
module
that
handles
packets
coming
in
from
the
wire
and
also
in
the
other
direction
and
applications
can
run
on
top
of
the
video
either
in
in
a
container
or
a
VM,
and
performance
is
best
when
the
application
also
uses
DP
D
K.
So
if
the
video
uses
DP
DK
and
the
application,
doesn't
it
doesn't
improve
performance
significantly,
and
for
that
reason,
we've
extended
envoy
to
also
use
DP
d
k,
because
envoy
is
an
application
that
runs
on
top
of
our
video.
A
A
Initially,
the
V
host
back-end
was
inside
the
kernel
and
a
couple
of
years
back
that
infrastructure
was
extended
to
move
it
into
user
space
so
that
it
could
run
in
virtual
router
or
virtual
switch
environments,
and
that
is
what
we
use
to
communicate
between
between
Envoy
and
the
Videodrome.
So
you
use
a
space.
V
host
sets
up
the
shared
memory
and
then
any
packets
exchange
between
Envoy
and
the
V
doubter
are
sent
over
the
shared
memory.
So
I
mentioned
that
we
have
one
process
which
runs
on
ydp,
DK
and
F
stack.
A
So
you
have
the
orange,
which
is
packets
on
one
virtual
network
so
because
we
use
overlay
tunnels,
there
is
an
identifier
inside
the
packet
that
identifies
which
virtual
network
the
packet
belongs
to
and
we
have
the
V
out
process
which
itself
uses
TP
DK,
so
it's
continuously
pulling
for
packets.
So
the
video
outer
process
polls
for
packets
then
does
whatever
is
necessary
to
handle
the
virtual
networking
part.
So
typically
that
would
you
know
include
decap
sedating,
the
packets
and
figuring
out,
which
virtual
network
that
packet
belongs
to
and
in
the
opposite
direction.
A
A
It
puts
the
descriptor
to
the
packet
in
shared
memory,
and
we
have
this
other
process
which
is
envoy
linked
with
DP
D,
K
and
F
stack.
That
process
is
continuously
pulling
the
shared
memory
for
packets,
so
the
packet
enters
initially
through
DP
DK
and
then
into
F
stack
and
as
I
mentioned
leap
event
is
now
continuously
pulling
four
packets,
so
it
pulls
the
packet
up
from
DP
DK
and
dumps
it
into
the
socket.
So
we've
added
the
ability
to
tie
a
socket
to
a
particular
VR
F.
A
So
if
you
have
multiple
virtual
networks,
you
could
have
socket
one
belong
to
belonging
to
one
virtual
network
and
socket
to
belonging
to
a
different
virtual
network.
So
once
once
the
packet
is
in
the
socket,
then
envoy
invokes
F
stack
calls
instead
of
the
traditional
socket
calls,
and
then
it
does.
Whatever
processing
needs
to
be
done.
A
So
this
is
a
quick
summary
of
the
changes
that
we
had
to
make
an
envoy
to
get
this
working.
So
we
added
Basel
external
repositories
to
include
DP
D,
K,
F
stack
and
live
event
live
event.
Poles
continuously,
as
I
mentioned
and
live
event,
calls
into
F
stack
to
accept
new
connections
or
to
listen
for
connections.
A
Inside
half
stack.
We
added
some
support
to
support
AF
packet
interfaces
and
vertigo,
and
we
also
added
support
for
ipv6
in
F
stack,
because
that
was
not
supported
and
we
added
support
for
multiple
via
refs
in
an
envoy
and
that's
the
API
to
tie
a
socket
to
a
particular
virtual
network
and
inside
the
or
annoyed
dispatcher
again,
instead
of
sleeping,
we
continuously
Pole,
and
we
can
do
that
on
a
per
socket
basis.
So
there
could
be.
A
The
main
thread
of
one
work,
for
example,
could
be
still
be
sleeping
on
inside
live
event,
but
the
worker
thread
could
be
continuously
moving
on
suck
on
sockets.
And
finally,
we
had
to
move
some
of
the
code
from
the
main
thread
of
envoy
into
the
worker
thread,
because,
if
stack
expects
that
any
sockets
that
are
created
are
created
in
the
context
of
the
same
thread,
so
we
couldn't
have
sockets
that
were
created
in
the
main
thread
and
then
handed
off
to
the
worker
thread.
A
This
is
work
in
progress,
so
please
treat
it
as
work
in
progress.
These
numbers
are
on
VMs
running
in
in
Azure
cloud,
so
we
have
a
client
connecting
to
an
HTTP
server
and
in
the
first
scenario
it's
going
through
on
wye
1.8
unmodified
and
in
the
second
scenario
we
have
on
Y,
with
F
stack
and
T
ptk
and
in
terms
of
requests
per
second,
with
the
unmodified
on
Y,
we
got
up
172
with
a
latency
of
about
4
milliseconds
and
with
the
Envoy,
with
F
stack
and
DPD
K.
A
That
number
was
close
to
500
and
the
latency
was
a
little
over
2
milliseconds,
we're
now
in
the
process
of
measuring
these
same
numbers
with
with
bare
metal,
because
on
on
ashore,
we
see
some
variance
in
the
numbers,
so
it's
hard
to
reproduce
these
numbers.
So
what
I'm
presenting
here
are
are
the
best-case
numbers
and
we
don't
get
the
same
numbers
on
every
run
with
a
sure.
A
So,
in
summary,
our
main
contributions,
we
believe
our
edition
of
multi-tenancy
into
envoy
and
improved
performance
with
integration,
with
DPD
k
and
and
of
stack.
We
plan
to
publish
a
blog
in
the
future
once
we
have
numbers
with
bare
metal
and
and
more
test
scenarios,
and
our
website
is
best
or
I.
Oh
yeah,
our
blog
will,
which
published
off
of
that,
and
we
believe
that
this
work
could
also
have
other
applications.
A
A
So
the
question
is:
can
we
characterize
the
performance
aware
of
the
overhead
of
the
kernel
Network
stack
as
compared
to
the
filters
that
execute
with
an
envoy
with
lower
JavaScript,
so
for
the
performance
test
that
we
ran?
We
did
not
have
any
Louisville
JavaScript,
so
I,
don't
really
have
numbers
there
and
hard
to
say
how
that
compares
with
the
overhead
of
the
kernel
stack
itself.
A
Yeah
so
I
think
I,
don't
think
all
of
the
overhead
that
we
see
with
the
kernel
stack
can
be
attributed
to
to
envoy.
In
fact,
I
think
it's
not
even
a
majority
of
the
overhead
because,
as
we
saw
in
the
numbers,
the
latency
went
down
from
four
milliseconds
to
two
milliseconds,
but
we're
still
running
the
same
Envoy
code,
but
just
not
using
the
core
network
stack.
A
So
there
is
some
overhead
for
sure
because
for
the
same
scenario
we
also
tested
client
going
directly
to
the
server
and
the
number
was
about
520
requests
per
second
and
the
latency
was
one
millisecond
and
when
be
used
on
Y
with
F
stack
and
D
PDK.
The
requests
per
second
was
about
500
and
the
latency
was
about
2
milliseconds.
So
we
added
about
a
millisecond
of
latency.
A
Yeah,
so
there
were
two
questions
there.
The
first
one
was:
does
DP
D
K
really
deliver
on
the
promise
of
order
of
magnitude,
improvement
and
performance
when
combined
with
everything,
that's
going
on
with
an
envoy
with
potentially
with
other
filters,
and
things
like
that.
So
you'll
see
a
lot
of
benchmarks
with
DP
DK,
which
show
fantastic
numbers,
and
usually
those
are
for
cases
where
you
know.
A
There's
very
little
packet
processing
that's
happening,
so
you
get
a
packet,
you
do
a
Mac
look
up
and
forward
and
for
that
kind
of
benchmark,
you'll
see
line
rate
with
10
gig
NICs
on
a
single
core.
But
what
we've
seen
in
practice
is
that
that's
really
not
practical
in
most
cases,
there's
enough.
That
needs
to
be
done
with
every
packet
that
achieving
14
million
packets
per
second
is
close
to
impossible.
A
So,
for
example,
we're
on
B
router
widths
with
DP
DK
and
the
amount
of
work
that
we
do
and
B
router
itself
brings
the
number
significantly
significantly
down
from
say:
14
million
packets
per
second
to
a
handful
of
million
packets
per
second.
So,
as
you
add
more
processing,
it's
expected
that
that
number
will
continue
to
go
down,
but
we
still
believe
that
it's
going
to
be
significantly
better
than
what
you
can
get
with
the
current
network
stack
and
sorry.
What
was
the
second
question
again
yeah.
A
The
second
question
was:
how
good
is
the
TCP
stack,
so
we
looked
at
what
options
we
had
and
the
reason
we
picked
F
stack
is
it's
based
on
the
freebsd
networking
stack,
which
has
been
around
for
a
long
time,
and
it
has
a
pretty
good
reputation
so
whether
it
supports
everything,
that's
there
in
the
Linux
kernel,
stack,
I'm,
not
sure,
but
I.
Think
of
the
options
that
we
have
I
think
PBS
D
is
a
pretty
good
choice.
A
Yes,
the
question
is:
if
the
application
does
not
use
DP
DK,
but
V
router
itself
uses
DP
decade,
the
performance
is
not
as
good
and-
and
why
is
that?
And
the
reason
for
that
is,
the
application
would
inhale
incur
the
same
overheads
that
are
associated
with
the
kernel
network
stack
yeah,
so
you
would
have
to
make
system
calls
to
send
a
packet
to
the
application
which
hurts
reroute
or
it
also
hurts
the
application.