►
From YouTube: Remote Control Planes With Konnectivity; What, Why And How? - Jussi Nummelin & Rastislav Szabo
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Remote Control Planes With Konnectivity; What, Why And How? - Jussi Nummelin, Mirantis & Rastislav Szabo, Kubermatic
Speakers: Rastislav Szabo, Jussi Nummelin
It’s a pretty common pattern to run the Kubernetes control plane on a dedicated node or a set of nodes co-located with the worker nodes. But what if we want to run the control plane as truly separated from the workers? Can the control plane be located in a completely different datacenter than the worker nodes even with some network-level disconnection separating the control and worker planes? We’ll start the talk by looking at why to build clusters with a remote control plane, and use cases for such setups. The second part of the talk will introduce the technical concepts that can be used to make it happen. We will look at how the api-server can be set up to use an egress selector proxy for different use cases. Next, we’ll look at a practical example of how it can be used with the Konnectivity API server network proxy. Lastly, we will showcase how all of this works together in open-source Kubernetes platforms like k0s and Kubermatic.
A
Welcome
everyone
for
our
session
we're
going
to
talk
about
remote
control
planes
and
and
how
to
build
those
or
construct
them
with
with
a
component
called
connectivity
and,
of
course,
we'll
start
with
what
it
is.
And
why
would
you
do
something
like
this
and
and
then
we'll
focus
a
bit
also
on
the
sort
of
building
works
that
that?
How
did
you?
How
do
you
do
things
so
I'm
used
to
from
mirantis
working
mostly
with
the
our
k0
Cube
distro.
B
A
Where
would
you
actually
want
to
do
something
like
this
and
then
what
are
the
building
blocks
and
and
Concepts
in
in
kubernetes
and
and
in
other
components
that
you
can
actually
actually
kind
of
pull
this
off
in
a
way
and
of
course,
we'll
we'll
have
a
bit
of
a
bit
of
kind
of
a
history
lesson
too
that
that
what
what
has
been
there
and
what?
What?
What
is
there
now
in
kubernetes
and
couple
of
real
world
integration?
Examples
on
on
this.
A
A
Why
we
wanted
to
have
this
sort
of
a
talk
is,
is
the
main
thing
is
that
that
there's
very
very
little
documentation
and
knowledge
on
this
topic,
so
we
want
to
raise
the
awareness
that
that
okay,
you
can
actually
do
something
like
this,
whether
it
makes
sense
for
you,
your
use
case
or
not.
That's
that's
a
different
discussion
so
basically
kudos
to
the
all
the
original
inventors
and
and
people
that
have
been
working
on
the
different
caps
in
the
in
the
past
and
to
to
make
this
actually
happen.
B
Okay,
so
as
what
you
say
already
spoke
about
we're
gonna,
We're,
Not,
Gonna
dive
into
the
whole
technical
backgrounds
of
how
we
achieve
things
so
usually
a
kubernetes
cluster
just
looks
like
this.
You
have
a
master
control,
plane
component,
it's
a
node
and
you
have
other
worker
nodes
as
well.
Where
you
have
you
know
your
API
server
or
the
whole
control
plane,
components
deployed
on
the
same
node,
and
then
your
worker
nodes
just
keep
talking
with
that
guy
now
this
was
working
still
working.
It
has
you
know
as
what
we
say.
B
We
have
a
bi-directional
connection
between
the
worker
nodes
and
the
control
plane
notes
it's
unrestricted,
so
they
can
speak
smoothly,
one
another,
whether
that's
you
know
a
requirement
or
not
it's
fast.
Since
again,
there
is
no
a
lot
of
restriction
in
there
reliable,
of
course,
and
secure
and
often
node,
local
or
you
know
you
have.
They
are
on
the
same
layer
to
networking,
probably
and
yeah
nice
things
are
working.
Everything
is
okay,.
B
You
need
a
lot
of
moderation,
a
lot
of
configuration
a
lot
of
tweaking
and
with
that
we
have
quite
a
few
challenges
such
as
probably
One
Directions.
As
you
have
you
know,
you
will
have
some
na
going
between
your
control,
plane,
components
and
the
worker
nodes.
It
could
be
very
restricted
because,
again,
you
might
be
hooking
into
a
different
network,
so
there
should
be
or
I
assume
there
should
be
some
firewall
rules
and
whatever
load
balancers
that
you're
speaking
with.
B
B
Of
course,
again
Network
partitioning
could
happen
between
your
worker
and
your
master,
node,
probably
insecure,
because
there
are
a
lot
of
Plumbing
between
your
worker
nodes
and
your
master
node
or
your
control,
plane
modes
component
and
of
course,
that
everybody
don't
like
I
hope
that
don't
like
that,
there
is
a
possibility
that
you're
doing
things
over
the
internet
and,
if
not,
probably
going
to
do
a
lot
of
you
know
tunneling
into
that.
Guy.
B
B
So
to
begin
with,
you
have,
of
course,
this
the
the
trust
or
sort
of
say
the
the
segmentation
where
you
have
like
the
control,
plane,
components
being
on
a
different
data
zone
or
a
different
data
center,
where
your
worker
nodes
are
next.
We
have
also
like,
of
course,
you
need
to
do
a
lot
of
configuration
and
plumbing,
and
this
could
be
actually
also
something
that
you
know
where
you
isolate
your
control,
plane,
components
away
from
your
worker
nodes,
so
whatever
happens
there
you're
still
safe
in
a
way.
It
is
also
human
error.
B
I
mean
less
human
error
because,
usually,
when
you
have
those
control,
plane,
components
being
somewhere
else,
then
your
your
real
work
is
being
carried
out.
When
you
work
on
nodes,
probably
a
human
wouldn't
be
or
users,
in
other
words
normal
users,
wouldn't
be
able
or
shouldn't
be
able
to
get
access
on
your
main
components
or
control,
plane
components
that
easily.
B
The
thing
that
you
know,
hopefully
everybody
is
now
aware
of
you
know
where
you
might
have
some
connections
to
the
cloud
providers,
but
still
you
have
some
kind
of
somewhere
dark
where
you
still
run
your
physical
data
centers,
and
you
like
to
achieve
this
kind
of
hybrid
model.
Where
you
have
you
know,
things
are
on
multi-cloud
and
other
things
on
your
data
centers.
That
gives
you
a
lot
of
flexibility,
and
this
is
exactly
what
we
are
looking
at
in
this
image
here.
B
What
we
see
here,
as
you
can
see,
on
the
left
side,
you
have
your
control,
plane,
components
being
deployed
somewhere
and
your
workloads
are
completely
isolated,
could
be
running
really
far
away
from
where
those
control
planes
are,
and
they
could
be
even
like
co-located
me,
meaning
that
you
could
have
one
node
or
one
cluster,
where
your
control
plane
components
for
different
clusters
are
being
in
the
same
place.
Of
course
you
should
have,
then
you
should
take
care
of
namespace
isolation.
B
Things
like
that
and
for
you,
you
could,
if
you
are
building
a
multi-tenancy
application
here,
and
you
would
like
to
give
get
a
grip
on
your
control,
plane,
components
of
course
and
clusters.
You
could
come
up
with
a
very
nice
architecture
where
you
can
operate
and
run
hundreds
or
even
thousands,
based
on
how
your
initial
architecture
scales,
of
course,
with
that
you
have
because
eventually,
what
you
have
on
that
isolating
it's
your
game.
B
For
example,
in
one
place
you
could
easily
operate
in
that
without
any
I
hope,
slowness
or
latency,
because
you're,
probably
speaking
to
the
same
Edge,
to
the
same
endpoint,
of
course,
could
be
like
more
a
lot
of
specifications
into
that,
and
eventually
yeah,
it's
cubanidas.
As
a
service,
it's
just
you
know,
you
don't
need
to.
Let
your
users
think
a
lot
about
the
control
planes.
Think
a
lot
about
the
kubernetes
is
not
that
easy.
I
mean.
Probably
you
know
that
already
networking
is
not
that
easy.
So
you
need
to
try
to.
A
Why
is?
Why
is
all
this
like?
Like
a
challenge?
I
can
run
my
API
server,
schedulers,
all
the
control
plane
components,
I
can
run
it
on
one
box
and
then
I
can
have
a
worker
that
connects
to
the
API
right.
What's
the
what's
the
challenge
there,
so
the
API
server
actually
has
to
talk
to
the
in
cluster
components
in
in
many
different
use
cases.
So
when
you
want
to
do
something
like
Cube
CDL,
logs
XYZ,
the
API
actually
calls
to
the
cubelet,
which
then
talks
to
The
Container
runtime.
A
A
A
So
the
API
server
actually
supported
a
thing
called
SSH
tunnels
in
the
past,
but
that
was
deprecated
on
on
version
109
for
various
reasons,
and
after
that,
there's
been
People
Like
Us,
for
example,
building
the
same
or
similar
capabilities
using
various
different
custom
Solutions
like
openvpn
tunnels
or
some
other
other
networking
gimmicks.
Let's
put
it
that
way,
but
then
luckily
there's
there
was,
as
I
mentioned,
we're
standing
on
on
soldiers
of
giants
here.
So
there
was
this
cap
1281
was
born
in
Spring
2019.
A
So
basically,
this
this
aggress
Electro
on
the
API
server
side
acts
as
a
sort
of
a
routing
mechanism
that
that,
when
the
API
server
in
these
different
use,
cases
wants
to
talk
to
to
the
cluster
or
to
some
other
external
components.
How
do
we
route
that
call
that
it
actually
can
reach
the
cluster,
whether
it's
a
pod
or
whether
it's
cubelet
or
whatever?
It
is.
A
That,
as
as
you
see
in
the
in
the
yamo
config
example,
you
usually
have
something
the
routing
the
proxy
component
running
kind
of
an
as
a
sidecar
ish
set
up
for
the
API
server.
So
the
API
server
actually
talks
to
a
local
Unix
socket
and
then
there's
something
behind
that
unique
socket
that
then
routes,
the
the
call
towards
the
cluster.
A
There's
various
different
types
that
you
can
configure
for:
the
experts,
accurate
selection.
So
when
you're
having
a
type
called
cluster,
we
mean
when
API
API
server
wants
to
talk
to
Ports
once
they
get
logs
from
cubelets
when
you
do
proxy
stuff
and
and
those
sort
of
use
cases
at
CD.
Well,
that's
kind
of
obvious.
A
So
the
cap
is
is
like,
like
many
other
caps
in
in
kubernetes,
it's
it's
kind
of
playing
the
architectural
foundations,
defining
the
interfaces,
how
things
work
and,
and
and
it's
it's
really
the
same
here
too
so
the
accurate
selection
and
and
how
the
API
talks
to
the
the
Unix
socket.
That's
like
the
interface.
You
can
have
basically
anything
behind
it,
but
connectivity
is
one
of
the
sick
projects
which
is
kind
of
the
sort
of
like
the
reference
implementation.
A
A
A
A
A
B
All
right
so,
as
you
could
have,
as
you
have
already
seen,
what
uses
showed
the
the
stack
is
quite
interesting
for
the
whole
networking
with
that
at
kubermatic
we
leveraged
how
connectivity
do
things
and
what
you
see
over
here
is
a
true
multi-tenant
architecture
that
we
run
every
day
where
you
have
those
on
the
on
the
right
side.
These
blue
boxes,
that
has
user
clusters.
Those
are
simply
a
kubernetes
clusters
that
only
run
node
applications
so
workloads
not
on
any
of
these
clusters.
B
B
What
we
do
instead
is
on
the
left
side.
We
have
these
seed
clusters,
we
call
them
and
there
in
that
specific
kubernetes
cluster.
We
Compact
and
co-locate
collocate
the
control,
pane
components
namespaced
with
all
what
you
need.
It
is
completely
isolated
from
what
the
user
the
regular
user
is
seeing.
There
is
no
way
that
they
could,
you
know,
communicate
and
see
what's
going
on
and
that
control
plane
namespace,
because
it's
already
on
a
different
clusters
and
of
course
there
are
a
lot
of
you
know
our
back
rules.
B
For
example,
you
have
a
lot
of
network
isolation,
so
it's
not
really
easy.
If
you
get
up,
if
you,
if
you
end
up
with
a
kubernetes
or
cubeconfig
for
the
kubernetes
cluster
on
the
right
side,
for
you,
you
won't
be
able
even
to
see
how
this
control,
how
how
these
pods
for
the
the
control,
plane
components.
Bots,
for
example,
looks
like
they're.
Not
there.
You
only
see
what
it
matters
to
you.
It
only.
B
You
only
see
your
applications,
your
workload,
and
this
is
exactly
what
we
are
doing
and
how
we
are
leveraging
connectivity
on
the
left
side
on
the
seed
cluster.
We
actually
inject.
The
I
mean
the
API
material
like
alongside
the
API
server,
the
connectivity
agent,
and
there
is
a
connectivity
agent
running
on
the
left
on
the
right
side
on
the
user
cluster
side.
What
happens
then,
when
you
create
a
cluster
the
we
have,
some
certain
controllers
that
take
care
of
the
control
plane
are
in
place.
B
Now
all
the
components
are
now
ready
and
then
we
spawn
the
machines
machines
gets
worker
machines
only
gets
what
they
need,
and
then
user
connectivity-
the
agent,
for
example,
receives
a
request.
It
finds
the
right
one
as
what
you
say
already
mentioned,
and
then
the
the
connection
between
cubelet
on
the
right
side
and
the
API
server
on
the
left
side
will
be
there.
And
what
is
quite
interesting
about
this?
You
don't
have
to
run
the
the
connectivity
agent
on
each
node
these
these
notes.
B
B
And
this
is
a
very,
very
real
world
scenario,
as
as
what
you
say
already
mentioned,
like
we
in
the
past,
we
only
had
to
fiddle
a
lot
with
a
lot
of
different
applications.
For
example
openvpn
server,
where
you
needed
to
yeah
manage
this,
manage
that
and
run
an
openvpn
client
on
the
right
side.
It
was
like
a
whole
mess
for
us
for
us.
We
really
wanted
just
to
have
a
reliable
connection
between
the
API
server
or
between
the
control,
plane,
components
and
the
worker
nodes
and
call
it
a
day.
B
It
should
be,
of
course,
reliable,
secure
fast,
and
this
is
what
we
experienced
so
far
with
connectivity
and
that's
why
we
like
it
on
the
left
side.
As
you
can
see.
Also
it's
not.
You
will
see
a
lot
of
containers
like
35
containers
that
were
only
there
for
connectivity.
That's
all
you
need
these
containers
to
have
connections
between
your
worker
nodes
and
the
control
plane
components.
B
What
was
too
much
for
us
and
on
the
like,
left
side
down
you,
you
look
like
you
see
the
the
openvpn
client
that
eventually
will
connect,
and
if
you
I
mean
openvpn
server,
that's
yeah.
Opp
band
is
nice,
but
if
you
do
it
on
that
scale
very
automated,
sometimes
it
could
be
really
hard
to
be
to
debug
and
know.
What's
going
on,
of
course,
never
to
mention
the
a
lot
of
IB
tables
rules
that
we
have
to
introduce
to
make
that
on
the
right
side,
you
see
the
happy
path
where
we
used
connectivity.
B
We
have
one
connectivity
injected
into
our
control
plane
component
part
up
on
top
and
on
the
lower
part.
You
already
see
that
yeah
we
have
only
like
two
agents,
two
parts,
those
are
no
demon
sets.
They
don't
have
to
run
on
each
node,
so
you
don't
have
to
worry
about
having
like
hundreds
of
notes
and
taking
care
of
Agents
being
running
on
each
one
of
those.
B
A
Right
and
and
there's
a
lot
of
similarities,
of
course,
what
I've
been
working
on
is
the
k-zero's
distro
and,
and
in
our
case
we
really
build
connectivity
into
the
picture
from
day
one
and
and
for
us
the
one
of
the
main
driving
use
cases
actually
was
not
dislike,
like
Network
segregation
or
anything.
But
the
fact
that.
C
A
Are
who
can
deploy
stuff
on
masternodes?
So
if
you,
if
you
take
your
traditional
Cube,
ADM
setup
cluster,
where
you
have
actual
Master
nodes
and
worker
nodes,
how
do
you
control
if
Joe
can
deploy
stuff
on
masternodes,
there's
no
R
back
for
that
yeah?
Of
course
you
can
throw
in
some
opa
Gatekeepers
whatnot,
but
but
should
you
have
to
do
that
in
my
opinion?
A
A
A
So
that's
a
bit
of
a
tricky
topic.
Debugging
things
is
a
bit
hard
because
there's
something
in
between
the
call
now,
so
you
can't
do
like,
like
TCP
dump,
at
least
that
easily
on
on
things
and
then,
when
the
connection
tunnels
are
not
working
as
as
expected,
and
you
do
something
like
Cube
CDL
locks.
The
errors
that
you
see
on
your
CLI
are
super
super
weird
you're,
not
gonna
able
to
figure
out
that
it
it's
actually
somewhere
in
connectivity
unless
you've
seen
those
errors
too
many
times
as
I.
Have
it's
like.
A
C
C
B
Yeah,
thank
you
yeah,
so
I
mean
we
can
maybe
quickly
go
back
to
the
to
the
slide.
I,
don't
know
if
we
can
do
this,
but
let's
try
there
so
what
we
have
over
there.
The
root
as
we
call
the
root
cluster.
This
is
actually
a
component
that
would
keep
things
synced
down
like
from
Upstream
to
Downstream
to
those
seed
clusters.
B
B
Cluster
Because
by
the
end
of
the
day,
it's
a
seed,
it's
a
kubernetes
cluster
that
it
has
its
own
Cube,
config
right
and
then,
instead
of
doing
it,
you
know
from
Upstream
to
Downstream.
You
could
just
do
it
Downstream
like
right
away
hook
into
it
and
then
see
what's
going
on
over
there.
So
this
is
like
the
model
that
we
kind
of
think
makes
sense
for
that.
A
E
A
All
right,
so
that
involves
something
kind
of
sort
of
an
external
load.
Balancer
I
mean
external
from
kubernetes
point
of
view,
so
in
in
normal
case,
you
configure
the
agents
to
talk
to
a
load,
balancer
and
and
basically
that
that
connectivity
servers
you
configure
that
okay,
there's
three
servers
running
so
what
happens
in
the
agents,
and-
and
this
is
why
I
said
that
AJ
is
a
bit
tricky.
A
E
A
There's
a
there's,
an
open,
long,
long-standing,
open
PR
on
on
on
doing
the
tunnel
like
like,
so
that
the
agent
opens
up
the
API
port
on
the
on
the
worker
node
and
then
goes
through
the
same
tunnel.
But
it's
it's
in
the
works
but
as
as
mentioned,
we
all
can
help
to
make
that
land
and
help
in
testing.
It
then.
A
A
B
Yeah
same
here,
I
mean
for
us,
as
I
said,
you
could
like
run
hundreds
or
maybe
depends
on
your
main
infrastructure,
eventually
the
underlay
and
overlay
infrastructure
scales,
but
mainly
what
we
saw
that
the
agent
and
the
server
of
connectivity
they
they
run
smooth
as
long
as
they
are
satisfied
with
the
resources
and
I,
don't
think
that
they
need
a
lot
of
resource.
As
we
said,
like
some
cases,
you
really
need
to
have.
As
what
you
say
already
said,
a
lot
of
redundancy,
like
demon,
said
running
on
all
these
notes.