►
From YouTube: IKEA Private Cloud, eBPF Based Networking, Load Balancing, and Observability with... Karsten Nielsen
Description
IKEA Private Cloud, eBPF Based Networking, Load Balancing, and Observability with Cilium - Karsten Nielsen, IKEA IT AB
The digital systems of IKEA are situated in public cloud and private data centers around the world. In this talk we’ll highlight some of the challenges – and opportunities - we faced in setting up a large scale, multi-cluster distributed Kubernetes environment across our data centers. We’ll share how we have used Cilium and its eBPF features to have a better scaling profile, to improve observability and even to replace some of our proprietary load balancers. * Connecting Kubernetes workloads across our BGP network * Protecting multi-tenant workloads with multi-cluster network policy * Cilium support for multi-homed pods * Mimicking availability zones with Cilium ClusterMesh * Use Cilium with XDP, ServiceType Loadbalancer and Ingress to replace our proprietary load balancer fronting workload. You’ll leave this talk understanding how you can use Cilium and its eBPF capabilities to build and instrument your network and obtain great observability.
A
A
A
People
to
join
our
forces
to
actually
deliver
on
our
journey,
so
let's
go
into.
Why
we're
here
today
is
actually
talk
about
the
project
that
I'm
working
on
is
part
of
our
modernization
of
efforts
in
terms
of
modernizing
our
enterprise.
That
is
a
lot
of
other
companies
or
enterprises
is
in
the
same
situation
as
us.
A
As
part
of
that,
we're
also
moving
a
lot
of
our
workloads
from
from
our
data
centers
into
the
public
cloud,
but
it's
not
all
workloads
that
is
movable
in
our
situation.
We
are
since
we're
operating
in
that
many
markets
we
also
operating
on
the
regulated
environment,
so
that
there's
a
ton
of
that
that
needs
to
be
handled
in
a
good
way.
So
we're
setting
up
this
private
cloud
initiative
in
our
data
centers
in
order
to
get
the
same
kind
of
operational.
A
So
what
is
needed
from
the
private
cloud
initiative
is
that
it
should
be
easy
to
use
just
that
you
would
experience
with
any
other
public
cloud
er
it
not.
It
needs
to
have
a
cloud-like
consumption
model
in
our
data
centers
and
also
it.
We
need
to
build
trust
to
our
cons.
Consumers
of
the
private
cloud
and
consumers
in
this
context
is
not
ikea.
A
And
from
this
private
cloud
we
looked
at
different
options
like
this
is
like
I've.
I
guess
a
lot
of
enterprises
has
been
down
the
openstack
in
this
platform,
but
we
we're
kind
of
building
this
environment
to
support
our
needs
of
tomorrow,
not
as
such
as
a
cloud
provider
as
such.
So
we
we
decided
on
moving
or
liberating
the
kubernetes
infrastructure
initiatives,
so
it
gives
us
this
possibility
of
actually
extending
the
apis
a
lot
with
different
types
of
crds
so
that
we
can
get.
A
For
instance,
we
can
run
cube
word
as
an
as
an
operator
to
handle
vms
in
this
environment.
There's
also
a
very
highly
demanded
interoperability
between
our
current
environments,
that
is
being
built
many
years
ago
and
and
this
environment.
So
we
have
a
need
to
actually
be
able
to
co-exist
these
environments
for
a
long
time
going
before
the
old
systems
has
been
modernized
in
order
to
fit
this
new
environment.
A
So
from
our
design
thinking
point
of
view,
it's
it's
an
open
source,
first
platform,
meaning
we
kind
of
went
away
from
dealing
with
traditional
vendor
approach
where
they
deliver
something
that
we
can
use
and
operate.
It's
been
needed,
at
least
for
some,
it's
for
some
time
that
we
actually
have
a
lot
of
flexibility
in
terms
of
picking
and
choosing
from
from
the
different
components
in
the
cncf
initiatives,
so
that
we
can
gain
speed
and
flexibility.
That
way,
and
we
have
been
looking
at
how
a
lot
of
the
public
cloud
providers
has
been
doing.
A
The
infrastructure
with
the
regions
and
availability
zones
in
our
context
and
the
availability
zone
is,
is
a
kubernetes
cluster
and
then
we
can
have
one
or
more
in
the
region.
We
have
load
balancers
as
a
shared
resource
and
the
same
with
the
storage,
and
since
we
are
building
this
environment
for
being
ready
for
tomorrow.
We're
also
looking
at
running
this
as
an
ipv6
single
stack
and
also
we're
looking
at
multi-home
parts
in
order
to
support
some
of
the
more
traditional
workloads
and
to
tie
these
zones
together.
A
We're
using
the
psyllium
cluster
miss
because
it
provides
us
with
some
nice
integrations
to
running
this
multi-cluster
environment
that
we
have
been
trying
to
do
the
same
type
of
thing
before
with,
but
with
covers.
It
actually
supports
having
cross
cluster
lookups,
and
you
can
do
writing
your
own
network
policy
operator
to
make
that
work.
It
seems
to
be
a
lot
easier
when
it's
all
built
into
the
psyllium.
A
What
it
gives
us
is
a
lot
of
flexibility
and
you
can
actually
move
your
workloads
around
much
easier
in
our
opinion,
and
also
this
gives
us
the
possibility
to
scale
out
the
environment,
both
in
terms
of
the
number
of
nodes
and
clusters,
but
also,
if
we're
running
low
in
the
data
path,
we
can
scale
up
or
add
an
additional
layer
of
leaf
switches
in
the
top
of
this.
That
will
bring
us
more.
A
Bandwidth
within
the
total
in
my
environment,
why
is
this
important
is
because
we're
moving
from
like
a
scale
up
solution
into
a
scale
out.
We
also
need
to
have
the
possibility
to
scale
out
networks,
and,
and
with
this
setup
we
can
actually
very
easily
get
our
we're
using
egress
gateways
as
well
to
to
connect
to
the
legacy
environment.
A
A
A
It
also
brings
the
the
need
to
actually
run
your
load
balancers
outside
either
on
a
physical
device
or
through
a
vm
and
in
this
environment
that
we're
building
is
actually
also
meant
to
run
inside
our
stores.
So
it
should
be
able
to
run
at
any
scale
so
from
a
small
environment
with
one
cluster
to
a
larger
data
center
deployment.
A
A
We
found
that,
apparently
it's
not
it's,
not
that
simple
to
get
it
working,
even
though
you're
we're
looking
at
some
of
the
supported,
nic
or
network
device
drivers,
we
ran
into
this
kernel
block
that
we're
currently
working
with
the
vendor
to
get
resolved.
It's
it
locks
up
our
nodes,
but
for
now
it's
doable
at
least
to
run
our
current
testing
and
continued
work
on
moving
forward.
A
This
is
interesting
for
us
to
support
some
of
the
the
multi-tenancy
promise
or
problems
that
what
you
want
to
say
about
it.
It's
you
can
easily
end
up
in
situations
where
a
con
tenant
will
actually
consume
all
of
your
bandwidth
with
doing
things
like
data
migration
or
ai
workloads.
That
consumes
a
lot
of
bandwidth
to
to
consume
data.
A
There
is
a
tool
that
mulches
that
supports
this.
It's
a
it's
actually,
in
our
opinion,
it
brings
also
a
lot
of
operational
complexity
as
it's
not
just
one
networking
component
that
you
need
to
run.
You
need
to
run
multiple
different
ones
to
solve
your
problem
and
then
from
a
zero
thrust.
Slash
a
multi-tenancy
point
of
view.
It
doesn't
really
bring
network
policies,
so
we
again
we
we
teamed
up
so
that
we
can
actually
run
multi-homing
inside
psyllium.
A
A
A
So
how
so,
currently
we're
not
really
routing
ipv6
within
our
data
centers.
So
we
have
some
some
outstanding
things
that
we
need
to
address
in,
for
instance,
we
need
to
isolate
this
environment
so
that
we
can
actually
get
that
six,
four
and
and
four
six
working.
So
we're
working
on
that
feature
together
with
isol
event
to
to
actually
be
able
to
to
have
the
translation
happening
at
the
at
the
low
balancer
notes,
so
that
the
environment
could
run
ipv6
and
and
everything
that
is
communicating
to
it
will
be
addressing
it
as
ipv4.
A
Connect
your
network
from
your
kubernetes
environment,
into
your
bgp
environment,
we're
using
bird
as
the
routing
daemon
on
our
nodes
and,
and
it
has
a
nice
integration
to
localhost-
that's
like
for
those
it's
it.
You
cannot
really
do
multiple
ptp
sessions
out
of
a
single
node,
so
you
need
to
kind
of
collide
them
on
your
node.
So
that's
the
integration
piece.
A
Implementing
this
dual
neck,
xwp
actually
xtp
support
in
order
to
actually
look
at
how
things
were
were
going,
and
how
do
we
then
test
this
when
you're
the
old-time
suicide
man
is
trying
to
use
your
tcp
dump,
and
it
shows
you
nothing.
It's
quite
interesting.
I
think,
like
our
setup
is
this,
that
we
have.
A
A
A
A
A
A
A
A
A
Some
more
convenient
tooling
than
doing
a
hubble
record
and
then
looking
at
them
in
wireshark.
It
would
be
really
convenient
to
have
them.
We're
using
the
grafina
stack
to
do
metrics
together
with
prometheus,
but
it
would
be
really
nice
to
have
a
way
to
actively
to
get
this
into
to
this
stack
so
that
we
can
actually
support
our
developers
in
in
their
journey
on
being
self-serving
on
also
networking
components.
A
D
A
D
Hi,
hello,
I
wondered
you
mentioned
the
bgp
thought
advertising
to
your
network.
I
was
wondering
I
also
sold
like
the
small
yam
base
there,
so
I
was
wondering
how
is
that
integrated
with
with
that
interest
in
the
infrastructure,
because
there
has
to
be
a
like
a
service
discovery
and
different
things.
You
mentioned
the
bird
demon.
How
is
that
integrated.
A
A
So
coordinates
at
least
when
we
did
it.
It
has
a
cross
cross
cluster
plug-in
that
you,
you
need
a
service
account
in
the
other
cluster
in
order
to
for
it
to
do
this
service
discovery,
and
then
you
can
actually
specify
what
we
did
was
to
have
different
types
of
cluster
domains.
So
we
had
different
types
of
cluster
domains
and
we
could
leverage
that
for
our
service
discovery
and
then
you,
of
course
you
need
to
have
that
cross-cluster
functionality
in
all
your
clusters.
C
B
A
E
A
A
A
So,
like
the
the
use
case,
the
initial
use
case
for
the
multi-home,
the
thing
is
to
be
able
to
if
anyone
has
been
trying
to
run
like
elastic,
stack,
there's,
http
transport
and
then
there's
the
transport
layer,
and
we
want
to
separate
those
two
so
that
the
search
queries
goes
on
on
in
on
the
separate
nics
with
it.
In
our
environment,
is
the
client
facing
interfaces
and
then
all
the
the
transport
is
handled
on
the
the
back-end
networks
and
and
you
can
handle
that
with
being
able
to
have
multi-home
parts.