►
Description
Want to view more sessions and keep the conversations going? Join us for KubeCon + CloudNativeCon North America in Seattle, December 11 - 13, 2018 (http://bit.ly/KCCNCNA18) or in Shanghai, November 14-15 (http://bit.ly/kccncchina18)
SRv6LB: Leveraging IPv6, Segment Routing, and VPP for Very Fast, Reliable, and Efficient Distributed Data Center Workload Balancing - Mark Townsley & Pierre Pfister, Cisco (Advanced Skill Level)
In this talk, we present performance and scalability numbers from our open source implementation of the Maglev data-plane (part of Google’s load balancing architecture as defined in [1]) in fd.io/VPP, as well as extensions that leverage IPv6 and Segment Routing (SRv6LB [2]) in ways that improve the fairness and reliability for workload balancing in a Data Center. In systems like Kubernetes that serve a large number of connections to micro-service instances in containers, our analytical and experimental results show that processing with SRv6LB is more fairly balanced than with Maglev alone. This results in significantly faster overall response times for end users and more efficient utilization of compute resources, especially under very high load.
About Mark
Mark Townsley is a Cisco Fellow, Ecole Polytechnique Professor, and co-Founder of the Paris Innovation and Research Laboratory (PIRL). Before Joining Cisco in 1997, he held positions at IBM, the Institute for Systems Research (ISR) and the Center for Satellite and Hybrid Communications Networks (CSHCN) at the University of Maryland. Mark served as IETF Internet Area Director from 2005-2009, IETF L2TP Working Group Chair from 1999-2005, IESG Liaison to the Internet Architecture Board (IAB), and IETF Pseudowire WG Technical Advisor. Mark was the lead developer of the original implementation of L2TP in Cisco IOS as well as lead author of IETF L2TP protocol specification (RFC 2661). One of the original architects of the World IPv6 Day and Launch, Mark contributed significantly to the deployment of IPv6 on the internet, including lead author of RFC 5969, IPv6 Rapid Deployment (6RD). In 2011, Mark co-founded the IETF Homenet Working Group, and served as chair until 2017. In addition to his Faculty appointment at Ecole Polytechnique, Mark lectures on Future Internet Architectures at Telecom Paris Tech (TPT), and serves on the steering committee for the joint TPT-Polytechinque Advanced Computer Networking master’s degree. Mark holds a Bachelor of Science (summa cum laude) degree in Electrical Engineering from Auburn University and a Master’s degree in Computer Science (magna cum laude) from the Johns Hopkins University Applied Physics Laboratory. When not traveling, he lives with his family in Paris, France.
About Pierre
Pierre Pfister is Software Engineer at Cisco's CTAO organization. He is an active participant and author at IETF (homenet, 6man, bier and hackathons) and co-developed the reference implementation of HNCP on OpenWrt platforms: hnetd. He is now commiter to FD.io's VPP-Sandbox project and active contributor to the VPP project where he co-developed MAP, ILA, vhost-user and MagLev. He graduated from the Ecole Polytechnique in 2012 and now mentors students on various projects.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Work
on
many
different
things
load,
balancing
being
one
of
them.
Maybe
it's
just
French
people
like
to
work
on
load,
balancing
I,
really
appreciated
a
Damien's
presentation
beforehand
to
introduce
some
concepts
that
I'll
get
to
skip
over
I
have
a
really
long
whoa
there.
It
is
again
a
really
long
title
up
there.
So
I
wanted
to
shrink
that
down
a
little
bit
by
pointing
out
the
I've
been
having
troubles
with
this.
There
we
go.
A
Those
are
the
pieces,
we're
really
going
to
talk
about
ipv6
segment,
routing
and
VPP,
and
how,
together
they
can
make
a
better
pointer
there.
We
go
how
to
gather.
You
can
assemble
these
pieces
and
do
some
interesting
things
with
load,
balancing
so
first
ipv6
and
since
we're
at
a
kubernetes
or
cube
con
conference
a
little
bit
on
the
status
of
ipv6.
This
is
a
work
being
done
by
my
colleagues
Dane
the
blonde
for
Rob
Peltier
and
Paul
McCauley
mishali.
A
Ok,
wherever
there's
before
we're
gonna
make
v6
and
see
if
everything
continues
to
work
with
the
idea
of
eventually
moving
up
to
preparing
and
planning
for
some
of
the
things
that
I'm
gonna
be
talking
about
today
that
when
you
have
v6
what
you
can
do
so
where
are
we
with
v6
in
the
internet
that
was
kubernetes?
Of
course,
v6
has
been
around
for
a
long
long
time,
but
recently
it's
been
taking
off
quite
a
bit
the
top
chart.
A
This
is
going
to
be
problematic.
I'm
just
gonna
have
to
sit
over
here.
I
usually
usually
like
to
walk
around,
but
I'm
gonna
have
to
try
to
stay
planted
here.
The
blue
line
is
as
measured
by
Google
and
reported
on
their
website.
There's
the
link
there
they're
at
the
bottom
and
my
colleague
Eric
Vinci,
he
scrapes
their
site
every
day
and
record
and
stores
it
and
then
publishes
these
charts.
A
So
Belgium
is
that
50%
of
ipv6,
as
seen
by
Google,
so
every
time
you
Google
or
YouTube,
or
whatever
they
do
locate
to
Belgium
and
they
publish
these
up
to
each
country
and
they
publish
these
results.
Now,
that's
without
any
mobile
activity
from
the
ISPs,
the
ISPs
haven't
moved
v6
to
v6
in
the
mobile
network,
yet
so
50
percent,
just
from
enterprise
and
landline
Germany
and
USA
pretty
straight
lines
up
and
to
the
right
around
the
35
percent
range.
Now
it's
not
just
big
economies.
It's
it's
our
smaller
ones
as
well.
Greece.
A
There
started
a
bit
later,
but
ramped
up
quickly
to
catch
up
with
Germany
in
the
USA
at
around
the
30
to
40
percent
range.
This
is
the
tortoise
and
hare
argument
here.
France
was
a
back
when
I
started
working
on
ipv6
in
2011
2012,
and
we
did
the
world
launch.
France
was
the
leading
country
in
the
entire
world
with
ipv6.
With
this
you
know
around
five
or
six
percent,
mostly
from
the
isp
free
telecom,
and
that
stayed
that
way
until
orange
kicked
in.
But
then
India
came
online
boom.
A
Reliance
Geo
deploys
183
million
in
points
within
two
years
on
ipv6
and
don't
want
to
throw
the
country
under
the
bus.
That's
hosting
us
here,
but
I
would
you
know
we
keep
stats
on
everything.
Denmark
is
three
point.
Five
percent,
so
not
all
the
countries
are
like
this
globally
were
around
one
in
four
users
have
ipv6
now
just
to
get
across
and
I
think
it's
important
for
an
audience.
That's
gonna,
be
thinking
about
using
ipv6
and
putting
it
in
the
kubernetes.
A
Is
you
know
how
different
v4
and
v6
is,
and
that's
just
it's
not
just
the
how
the
protocols
operate
and
everything
but
how
it's
being
used
and
how
it's
being
deployed-
and
this
is
a
really
interesting
data
point
from
my
friend
over
at
Akamai,
so
this
is
from
their
infrastructure.
Looking
at
unique
ipv6
addresses
that
they
see
in
a
given
period
of
time
here
it's
in
one
week
and
they
see
almost
ten
billion
unique
ipv6
addresses
within
a
week.
A
Now,
this,
of
course,
far
less
than
they
see
a
unique
ipv4
addresses,
because
there's
only
four
billion
ipv4
addresses
in
the
world,
they
even
see
more
unique,
slash
64
than
they
see
ipv4
addresses.
So
what
this
suggests
to
me
is
there's
a
lot
of
churn
in
terms
of
the
ipv6
addresses
they're,
doing
more
than
just
being
a
single
address
on
the
side
of
a
NAT
or
a
single
address
to
to
an
endpoint
they're
changing
quite
a
bit
and
there's
there's
reasons
for
that.
But
the
infrastructure
supports
it.
Speaking
of
infrastructure
that
supports
v6
at
Facebook.
A
A
A
That's
really
not
an
incredibly
new
thing
when
you
consider
in
t-mobile
in
the
US,
has
been
that
way
for
several
years
and
there's
different
types
of
transition,
technologies
and
people
kind
of
get
scared,
because
there's
so
many
of
them,
but
really
it
started
to
narrow
down
to
a
couple
of
different
ones
with
funny
different
acronyms.
But
the
bottom
line
is:
is
this
idea
that
I
can
take
part
of
my
network
or
part
of
my
deployment
or
whatever?
A
It
is,
and
v6
only
it
and
then
put
v4
out
at
the
edge
now,
because
v6
has
all
this
address
space
we
can
again,
we
can
start
using
it
for
things
other
than
routing
to
a
single
interface.
I
can
take
my
port
space
in
v4
in
a
bit
of
my
v4
address
space
and
shove
it
into
the
v6
address
space
with
an
algorithm
that
allows
a
stateless
translation
between
the
two
and
then
I
can
encapsulate
or
I
can
translate
and
together.
A
We
call
this
thing:
map,
mapping,
address
and
port
mapping
address
import
map,
routing,
ipv4
addresses
and
ports
in
the
v6
infrastructure.
So
you
have
this
v6
infrastructure.
It's
there
if
you've
set
it
up
by
putting
the
ports
and
the
v4
addresses
inside
the
addresses
v6
addresses
in
a
certain
way.
You
get
the
power
of
routing
to
get
the
packets
where
they
need
to
go
without
having
to
set
up
stateful
Nats.
A
A
Almost
put
it
on
there,
but
I've
been
in
a
long
architectural
debate
with
Joe
touch
on
whether
or
not
MPLS
is
actually
a
tunnel
or
not.
What's
that
what
l3
VPNs
well,
they
tend
to
use
MPLS
but
may
use
other
stuff
anyway,
lots
of
mouths,
air
and
I
think
the
proliferation
of
tunnels
happens
because
they're
typically
pairwise,
sometimes
their
group
around
an
edge.
But
if
this
start
of
the
tunnel
is
the
same
protocol
is
the
end
of
the
tunnel.
A
A
Okay,
so,
rather
than
just
going
in
and
then
out,
you
can
it's
sort
of
like
stitching
tunnels
together
if
you
will,
but
in
a
way
that's
that's.
That's
designed
from
the
start
to
be
able
to
do
that.
So
this
lets
you
from
the
source
of
the
packet
or
the
start
of
the
tunnel.
Cinepak
it
to
multiple
waypoints,
just
like
you
see
up
here
now,
there's
waypoints.
If
they're
in
v6
and
you've
got
all
this
address
space,
they
don't
have
to
be
individual
nodes,
they
can
be
services
inside
the
router
or
inside
a
computer
ode.
A
Or
what
have
you,
because
with
128
bits
you
can
I
just
like
I
was
identifying
hey.
This
is
this
address
actually
contains
a
bit
of
a
v4
address,
and
a
bit
of
a
port
I
can
say.
Is
this
address
means
perform
this
function
on
the
packet
install
this
state,
get
rid
of
the
state
and
you're
gonna
see
why
that's
important
when
Pierre
gets
up
here.
A
Sorry
about
that,
so
we're
gonna
talk
about
three
things,
so
that's
ipv6
segment,
routing
just
think
of
segment
routing
as
a
tunnel,
but
one
that
you
can
go
through
multiple
waypoints
and
do
stuff
VPP
is
a
I
can
go
to
the
next
slide.
For
that
I
hate,
not
having
my
little
slide
thing.
Vpp
is
a
project
actually
anything
about
VPP.
You
can
ask
Heather
Kirk,
see
right
there,
it's
the
SDIO
project
or
the
fast
data
project
and
the
little
foundation
or
phyto.
A
It's
sometimes
called
it's
a
vector
packet
processor,
which
is
a
very
fast
packet
processing
engine.
So,
as
you
see
here,
it'll
run
on
bare
metal
and
run
on
VM
and
running
containers.
What
have
you
it's?
A
it's
just
the
packet
processing
engine
and
it
runs
in
userspace,
so
it
bypasses
the
kernel
and
you
get
to
do
all
sorts
of
fancy
stuff
inside
the
the
VPP
system
and
it
lets
you.
You
know,
move
at
a
different
pace
than
waiting
for
the
kernel
in
terms
of
network
features.
A
Also,
it's
running
in
user
space
on
the
compute
nodes
and
there's
convenient
ways
to
share
memory
with
other
applications
on
the
compute
node
and
that's
a
core
of
what
we're
going
to
be
getting
into
in
just
a
moment
for
the
SR
v6
lb
application.
So
if
you
have
VP
P,
which
is
the
fast
data
project
in
the
Linux
Foundation,
you
have
how
do
I
get
it
into
containers?
A
Well,
one
way
to
get
into
containers
is
to
use
con
teve,
which
is
c,
and
I
etcetera
you
can
kind
of
see
it
on
the
on
the
chart
here.
This
is
the
canonical
chart
about
VPP
in
kubernetes,
so
Conte
VPP
puts
VPP
into
kubernetes,
and
finally,
I
stole
this
literally
from
my
from
Lou
Tucker,
who
did
a
CTO
for
us
this
morning.
A
Now,
finally,
SRV
6lb,
you
just
got
a
tutorial
in
load,
balancing
and
that's
fantastic
elf
for
load
balancing,
as
you
just
heard,
you
do
some
sort
of
you
know:
random,
round-robin,
hash,
consistent
hash.
What
have
you
and
then
you
tunnel
or
nap-
maybe
a
GRE
tunnel
whatever
to
get
to
the
chosen
server
destination?
Okay,
this
is
without
monitoring.
If
you
do
monitoring
of
the
app
it
are
you
it's
a
it's.
It's
a
different
thing,
because
the
this
green
box
has
to
do
a
lot
more.
A
If
it's
trying
to
constantly
monitor
the
the
given
load
of
any
server
and
make
an
intelligent
decision,
Bay
done
that,
and
we
want
to
think
of
a
another
way
of
doing
this.
We
base
this
on
Google's
maglev
maglev
has
a
very
clever
way
of
doing
its
consistent
hashing
and
it's
self
described
as
embarrassingly
distributed.
A
Instead
of
selecting
one
possible
destination
with
my
consistent
hash,
I'm
going
to
select
two
up
front
and
it's
a
bit
like
the
power
of
two
choices,
but
we
definitely
use
the
power
of
two
choices.
It's
a
bit
like
the
bounded
load
stuff,
but
instead
of
relying
on
the
end
points
to
chain
the
packet
along
the
way
we
let
the
load,
balancer
who's
got
a
view
of
all
that
all
the
possibilities
select
and
because
I
have
the
segment
routing,
which
is
like
a
tunnel
with
multiple
hops.
A
What's
the
current
load
level
of
the
application,
the
VIP
that
I'm
targeting
on
that
compute
node,
if
it's
below
a
threshold
I
accept
it
if
it's
above
a
threshold
I
pass
to
the
next-
and
this
is
all
done
at
the
network
layer,
the
applications
don't
see
it
other
than
the
network
layer
in
in
the
compute
node
being
able
to
see
the
current
metric
that
the
application
wants
to
balance
on
and
what's
interesting
is
the
load.
Balancer
doesn't
have
to
be
aware
of
that.
A
In
metric
at
all,
it
could
be
CPU
percentage,
it
could
be
a
queue
depth,
it
could
be
number
of
threads.
It
could
be
how
much
memory
you
have
it's,
whatever
the
application
wants
to
balance
upon,
because
the
decisions
being
made
locally
by
that
application
on
the
compute
node,
and
it's
just
a
yes/no,
if
it's
no,
he
doesn't
even
have
to
think
about
where
to
send
it.
Next,
it's
already
in
the
packet,
because
s
r
is
like
a
multi
hop
tunnel.
A
Now,
when
we
first
thought
of
this
in
the
way
it's
it's
architected
is,
we
can
go
through
multiple,
but
we
found
the
same
thing.
That
Damien
did
is
the
magic
number
you
get.
If
you
choose
two
packets
there
we
go,
I
mean
two
destinations.
You
get
a
significant
benefit.
Three
you
get
a
little
bit
more
of
a
bit
of
a
little
benefit
to
one
all
the
way
up
to
N
and
it
decreases
as
you
go
along
the
way,
all
the
way
up
to
the
end.
A
Here's
a
couple
of
results:
I'm
not
going
to
spend
too
much
time
on
it,
because
Pierre's
got
a
live
demo
assuming
the
demo
gods.
Don't
come
down
upon
us,
but
you
can
see
here.
These
are
graphs
of
the
CPU
loads
and
we've
set
a
threshold
and
on
the
Maglev
side,
nothing
wrong
with
maglev.
Wonderful
technology,
don't
want
to
call
anybody,
you
know
invention,
ugly
or
anything,
but
this
is
taking
that
and
just
improving
on
it.
These
are
unlucky
requests.
A
So
when
you're
really
running
hot
every
once
in
a
while
you're
gonna
get
unlucky
and
the
hotter
you
run
the
more
unlucky
you're
you're
going
to
get
and
since
the
load
balancer
has
no
knowledge
of
the
load
of
the
system,
it
can
happen
right
and
so
in
our
testbed
we
were
able
to
you,
know
land
on
a
lot
of
unlucky
requests.
There
same
set
up.
You
turn
on
the
srl
B
and
the
naturally
the
you
move
on
to
the
next
server,
and
you
can
see
that
sometimes
you
get
unlucky
twice.
A
That's
when
oh
I
tried
this
when
I
was
above,
the
threshold
I
tried
the
next
one.
Oh
well,
and
he
accepts
it
anyway,
because
that
threshold
is
not
a
hundred
percent,
it's
something
less.
So
it
does
happen.
But
you
end
up
with
something
that's
much
much
more
fair
in
terms
of
there's
two
ways
to
look
at
the
benefit.
Here
you
can
say
all
right,
same
infrastructure,
same
load
pattern.
A
What's
the
client
page
load
response
time,
so
the
experience
of
the
client
another
way
to
look
at
it
is:
can
I
reduce
the
amount
of
infrastructure
for
the
same
SLA.
So
this
is
the
the
first
way
where
I
can
say
for
a
given
set
of
servers.
Here's
the
page
load
time
maglev
is
better
than
like
round-robin.
It's
doing
it's
doing
a
good
job
here,
but
SRL
B
does
a
better
job
and
you
can
see
as
we
get
up
to
80
and
90%
the
it's
two
and
a
half
times
better
in
terms
of
page
load
time.
A
Peers
demo
shows
all
this,
but
in
better
detail.
This
is
just
that
reverse
situation
for
a
given
SLA
I
can
move
20
to
25
percent,
less
servers
and
get
the
same
response
out
of
the
client.
So
with
that
I'm
gonna
hand
straight
over
to
Pierre
we're
gonna
have
to
change
machines
here.
If
you
want
to
know
any
of
the
excruciating
details,
there's
a
whole
academic
side
of
this
Johan
did
it
he's
somewhere
in
the
audience
there
he
is.
B
B
Okay.
So,
as
you
all
know
now,
VPP
is
via
to
router
and
jet
will
switch
and
it's
very,
very
efficient.
It's
a
code
that
is
beautifully
optimized.
One
of
the
reason
why
it's
so
efficient
is
by
design
all
the
packets
go
through
a
set
of
nodes,
so
they
are
Traverse
graph
and
each
node
does
a
very,
very
simple
operation
on
the
on
each
packet.
So
here
you
see
an
example
going
from
D
PDK,
you
see
the
IP
lookup.
B
You
see
the
transmission
on
the
wire
back,
so
this
simple
IP
forwarding-
and
this
is
very
extensible
as
well
as
well,
so
that
doesn't
work
I
will
have
to
press
the
button.
So
in
our
case
we
implemented
the
load
balancer
on
one
side
and
the
server
agent
on
the
other
side
has
two
different
vvp
plugin.
For
instance,
when
you
receive
a
packet
on
the
load
balancer
well,
this
is
just
a
different
node
in
the
graph
and
packets
are
forwarded
by
the
IP
lookup.
So
we,
this
is
extensible
right.
B
You
can
ask
VPP
to
send
you
as
a
plugin,
some
specific
packets
to
your
own
custom
nodes
same
thing
when
the
server
agent
on
the
server
side
received
a
packet
it's
sent
to
one
of
our
nodes
and
then
based
on
the
destination
IP
that
is
used
based
on
the
segment
routing.
We
can
dispatch
the
packet
to
the
different
nodes
that
we
have
inside
our
plugin
same
thing
on
the
on
the
other
direction.
B
When
you
receive
a
packet
from
the
application
on
the
server
agent
side,
we
leverage
another
way
that
VPP
is
able
to
hand
off
a
packets
to
the
plugins
that
are
called
IP
features,
I'm
not
going
into
too
much
details
here,
but
the
idea
is
that
it's
very
flexible
now,
let's
focus
on
a
small
piece
of
it,
which
is
actually
the
key
idea
of
segment
routing
load.
Balancer.
Is
this
idea
of
having
a
choice
here?
The
packets
on
the
server
agent
is
received
and
it's
a
packet
for
a
new
connection.
B
That's
why
it's
called
connect
if
available,
and
it's
only
if
there
is
there-
are
resources
available
that
the
connection
is
going
to
be
accepted
and
because
that
decision
has
to
be
taken
on
a
per
flow
basis.
Every
time
there
is
a
new
flow
well,
it
has
to
be
fairly
efficient,
and
that
is
as
well
extensible
that
in
our
plugin
we
implemented
a
set
of
different
policies.
One
and
the
one
that
were
showing
in
the
demo
is
based
on
Apache,
because
Apache
provides
an
API.
B
Where
you
have
a
share
memory,
you
can
establish
your
share
memory
memory
where
you
will
see
live
the
busyness
of
the
of
the
Apache
server.
We
also
implemented
engine
something
with
nginx.
We
also
have
a
standard
way
to
do
it
with
Linux
C
group.
So
if
you
don't
have
a
custom
application,
you
can
use
something
that
is
a
bit
more
generic
and
that's
completely
extensible.
You
can
have
other
plugins
that
are
application
specific.
B
If
you
have
an
application,
yours
and
you
want
to
give
to
VPP
vision
of
what
is
your
current
load
and
the
application
is
better.
Is
the
best
place
to
know?
What's
the
current
load
based
on
the
metric
that
matters
for
the
application,
you
can
make
a
VPP
plug-in
and
provide
this
visibility
to
the
server
agents
such
that
when
a
connection
is
received,
the
server
gets
to
decide
whether
to
accept
the
new
connection
or
forwarded
to
the
next
choice.
B
The
next
server
to
accept
the
connection,
then
one
of
the
aspects
I
would
to
mention,
as
well
as
the
the
nice
things
about
using
ipv6
segment
routing.
Is
that
actually,
you
have
plenty
of
bits
in
the
address.
128
bits,
that's
a
lot
and
you
can
use
that
to
optimize.
Well,
your
your
code.
Basically,
so,
even
though
it's
protocol
thing
in
the
destination
IP
address,
you
have
all
those
bits,
some
of
the
bits
that
we
use
are
here
for
the
function.
B
So
those
are
the
bits
that
are
that
are
going
to
tell
this
packet
is
a
new
packet
or
I
mean
a
new
flow
packet
or
a
flow
that
already
exists
and
you're
going
to
do
something
based
on
that
function.
That's
really
used
to
optimize
the
code
path
inside
VPP,
but
I
mean
it
can
be
used
in
other
implementations
as
well,
and
then
we
still
have
40
bits
left.
We
call
them
the
opaque
number,
because
that's
something
that
is
picked
by
the
load
balancer
or
the
server
agent
and
on
the
per
flow
basis.
B
B
We
use
that
for
two
purposes
one
is
CPU
steering
and
the
other
one
is
the
to
optimize
the
flow
lookup
in
our
flow
table.
So
the
details
about
the
CPU
steering,
which
is
very
problematic
when
you
implement
something
at
very
high
rates,
is
that
so
that
you
receive
a
packet
for
a
flow.
The
NIC
received
the
packet
and
the
NIC
is
going
to
hand
off
that
packet
to
a
given
CPU.
You
don't
control
that
it's
not
just
to
pick
it's
the
NIC,
that
does
that
it's
called
RSS
and
so
on
that
CPU
on
that
core.
B
That's
where
you're
going
to
install
the
state
and
because
you
want
to
avoid
using
locks
between
the
different
cores
because
you're
just
going
to
destroy
your
performance.
If
you
do,
you
need
to
do
LOC,
less
data
structure,
so
you
you
can
only
install
your
state
on
one
single
core,
and
so
that's
where
the
trick
happens.
When
you
send
a
packet
to
the
server
agents,
you
put
the
CPU
index,
and
so
when
the
packet
comes
back,
you
have
no
way
of
making
sure
that
the
NIC
is
going
to
send
it
to
the
right
core.
B
Most
of
the
time
it's
going
to
be
the
wrong
one.
So
the
way
you
do
it
in,
if
you
don't
have
that
loud
label.
Well,
it's
you
compute
the
5-tuple.
You
decide
what
color
you
want
to
send
it,
and
then
you
send
it
to
the
car.
That's
a
bit
expensive!
Here
you
get
the
bits
right
inside
the
packet
and
you
can
send
it
to
the
right
car
and
and
then
continue.
B
One
word
about
the
performances
because
we
have
evaluated.
We
have
compared
this
new
approach
to
load,
balancing
a
service,
6
lb
with
maglev,
both
are
implemented
in
VPP
and
both
are
really
well
optimized.
Srv
6lv
adds
a
little
bit
of
overhead.
You
see,
we
go
from
8.3
millions,
packets
per
second
per
core
to
8
millions.
Packets
per
second
per
core
I,
understand
you're
on,
maybe
probably
not
all,
networking
people,
but
basically
that
means
that
if
with
one
core,
you
can
roughly
load
balance
about
22
gigabytes
per
second
of
transfer.
B
The
road
balancer
here
the
load
balancer,
because
it's
a
layer,
3
layer,
7
and
no
sorry,
layer,
3,
layer,
4
and
not
a
proxy.
It
only
sees
the
packets
that
are
going
in
one
direction.
Ok,
so
that's
one
thing
that
you
can
do
with
this
kind
of
load:
balancer
that
you
can't
with
application
level
or
load
balancers,
and
one
thing
as
well
is
that
s
our
v6
LV
is
more
scalable.
It
can
scale
to
1
million
flows
and
more
because
we
have
a
system
flow
table
that
is
now
open
sourced
in
VPP.
So
demo
time.
B
Okay,
now
we
should
should
start
again,
so
this
demo
is
monitoring
and
controlling
live
a
setup
with
one
load
balancer
and
up
to
sixteen
different
servers.
Those
are
nginx
servers.
They
are
basically
computing,
PI
decimals,
that's
why
you
see
the
little
PI's
on
the
screen.
This
is
obviously
pretty
small,
but
it's
kind.
It
could
scale
up
way
much
more
than
just
sixteen
servers
and
it
could
scale
up
to
more
load
balancers.
But
it's
enough
to
to
prove
the
point
that
I
want
to
make
here.
B
So
let
me
check
the
parameters
that
I
have
I,
will
just
first
switch
to
maglev
and
put
my
16
servers
here.
So
does
that
work
yeah?
So
here
so
you
see
the
load
balancer.
You
see
the
different
servers
that
are
here
and
now,
if
you
have
a
look
at
this
graph
here,
you
have
what
Danny
I
explained
earlier
and
what
Marc
mentioned
as
well.
Is
that
with
a
simple,
consistent,
hashing
algorithm,
your
are.
Your
requests
are
going
to
each
servers
that
are
fairly
loaded.
That's
that's
a
problem.
B
You
have
with
consistent
hashing
and
that's
why
you
have
consistent
hashing
with
bounded
load.
Now
our
approach
also
is
plan.
I
mean
also
solves
that,
and
here
you
see
the
requests
that
are
eating
servers
that
are
fairly
loaded.
This
here
is
the
load
of
the
servers.
It's
a
bit
messy.
It's
like
the
one,
the
graph
that
Marc
showed
before,
but
it's
going
to
be
clearer
at
the
next
step,
have
a
look
as
well.
At
the
response
time,
I
mean
that's
what
people
care
about.
B
B
We
are
still
using
maglev
I'm,
switching
to
SR
he'll,
be
singing
trotting
load
balancer
here,
and
you
will
see
immediately
all
the
servers
that
are
receiving
requests
that
are
loaded
more
that
have
more
than
four
different
outstanding
requests
at
a
time
are
now
going
to
send
the
connection
to
the
next
choice:
the
next
server
sometime
as
Mark
mentioned,
you
get
unlucky
twice,
but
most
of
the
time
you
don't-
and
here
you
see
the
difference.
You
have
almost
no
servers
that
are
delivering
weak
requests
and
that
are
more
loaded
than
for
outstanding
connections
at
the
time.
B
So
you
get
the
benefits
that
you
would
get
by
using
a
load
balancer
that
would
know
exactly
the
load
of
all
the
server's.
That's
what
that's
the
kind
of
thing
you
could
get
with
a
layer,
seven
load
balancer
or
load
balancer
that
pulls
the
state
of
the
of
the
servers
at
the
same
time.
But
here
you
do
it
in
a
distributed
way
and
the
decisions
are
made
instantaneously
so
like
when
a
server
gets
a
request.
B
It
doesn't
make
the
decision
of
accepting
or
of
accepting
the
connection
based
on
some
information
that
it
got
like
two
or
five
seconds
ago
or
ten
seconds
ago,
depending
on
how
good
is
your
message?
Forwarding
mechanism
that
lets
you
retrieve
your
load
of
the
server's
here,
it's
instantaneous
and
it's
specific
to
the
application
know.
So
that's
that's
one
aspect.
You
see
the
the
performance
improvement
here.
Well,
that's
the
spike
that
we
get
some
time
here.
B
We
went
from
three
between
three
and
four
hundred
milliseconds
for
the
nineteenth
percentile
to
less
than
less
than
200,
so
180
milliseconds
for
for
the
nineteenth
percentile.
The
median
doesn't
change
that
much
because
we
are
not
really
improving.
The
service
servers
are
the
same,
but
we
are
better
utilizing
the
CPU
of
the
server's.
We
are
more
fair
in
our
distribution.
B
One
thing
you
can
do
from
there
is
reduce
the
number
of
servers.
So
if
I
switch
from
16
servers
to
12
servers,
that's
25%
reduction
in
the
number
of
servers
here,
I
switch
to
maglev
again
and
well.
It's
not
good,
it's
not
good,
because
we
don't
with
maglev
or
any
kind
of
consistent
hashing
algorithm.
You
need
to
kind
of
over-provision
the
number
of
service
that
you
have.
B
Otherwise
you
explode
and
the
response
time
is
is
real
bad,
but
with
SRB
again,
since
we
are
more
fair,
you
are
able,
with
12
servers,
to
have
basically
the
same
response
time
than
what
we
were
having
before.
Well,
it's
now
gone
I'm
going
to
show
it
again.
I
will
put
16
servers
with
maglev
just
to
to
show
that
you
get
the
same
the
same
thing
so
here
it's
maglev
with
16
servers
right
before
it
was
SR
and
we
with
just
12
servers
and
the
SLA.
B
C
B
B
D
B
C
B
A
In
the
case,
maybe
it
was
the
same
with
nginx,
but
certainly
with
Apache
when
we
did
the
tests.
This
is
a
it's
a
it's
an
API
call
that
exists
in
Apache.
Today
we
didn't
have
to
modify
it
or
anything
we're
just
using
this
API,
that's
specifically
for
load
monitoring.
It's
just
we're
doing
it
in
the
data
path.
E
B
E
B
So
that's
a
very
good
question:
it's
because
if
you
don't
do
that,
if
you
want
two
different
cores
to
operate
on
the
same
flow,
then
your
flow
table
as
to
use
a
lock.
You
need
locking
between
the
two.
So
it's
actually
much
more
efficient
to
pay
the
price
of
handoff
between
the
cores
sending
the
packet
from
to
the
right
car.
It's
going
to
make
you
make
your
implementation
much
more
efficient
because
you
don't
need
to
synchronize
the
different
cores
when
they
touch
the
flow
table
and.
D
B
Right
so
the
that
was
mentioned
by
mark
it's
well,
it's
based
on
this
result
the
power
of
two
choices.
Again,
if
you
get
you
can
get
unlucky,
not
once
that's
what
we
used
to
say,
you
can
get
unlucky
once
you
have
a
certain
probability
of
hitting
a
server
that
is
really
loaded.
When
that
happens,
you
forward
the
probability
that
you
get
unlucky
twice.
Is
you
know
exponentially
decreasing
and
so.
A
Balancer
but
I
think
the
answer
is
simply
no:
unless
we're
in
recovery
mode
ability
to
know
the
load
so
load
balancer
I
try
this
server
bounce
to
the
next
server
yeah
I
install
State
for
that
server
for
the
life
of
that
flow.
Okay,
the
second
one,
if
the
first
one
goes
down,
I'm
still
routing
packets
just
to
the
first
one
for
the
life
of
that
flow.
Let
me
do
the
second
one
yeah.
E
A
A
That's
what
some
of
these
bits
that
we
use
and
it's
like
installing
a
it's
like
installing
a
hosts
route,
if
you
will
dynamically
and
then
removing
it
at
the
beginning
and
end
of
the
flow
okay,
so
once
it's
sticky
and
pointing
to
the
right
place,
the
other
server
can
go
down
anything
else.
I
guess
is
a
keynote
coming
up.
Thank
you
very
much.
Thank
you
very
much.
Thank
you
here.