►
From YouTube: How Kubernetes Components Communicate Securely in Your Cluster - Maya Kaczorowski, Google
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
How Kubernetes Components Communicate Securely in Your Cluster - Maya Kaczorowski, Google
How *do* your cluster components talk to each other? In this expository talk, we'll first cover the main Kubernetes components that need trusted communication - that is, the API server, kubelet, and etcd, and how this communication is protected. Then, we'll go over how the cluster certificate authority (CA) works, and how this grants certificates to Kubernetes components. Furthermore, we'll explain what authentication, integrity, and encryption means, and what options are available in Kubernetes, and what you need to configure to address these pieces of CIS benchmarks. Lastly, we'll explain how you can protect other communications within your cluster, if needed for your workload - like node to node and pod to pod. You'll come away with a better understanding of how communications in Kubernetes work, cluster trust, and default protections.
https://sched.co/UaZE
A
Falls,
all
right
so
we're
talking
like
we
just
introduced,
we're
talking
about
how
carbonates
components
in
your
cluster
communicate
and
how
they
do
that
securely,
so
welcome.
My
name
is
Monica
Trotsky
I'm,
a
product
manager
working
on
container
security
at
Google
recently
I've
been
thinking
a
lot
about
the
certificates
that
you
have
in
Kerber
Nettie's
and
the
roots
of
trust
that
you
depend
on
in
your
cluster.
I.
Initially
didn't
know
a
lot
about
this,
so
I
wanted
to
do
some
research
and
learn
a
couple
of
things
and
that's
what
we're
talking
about
today.
What
I?
A
What
I've
learned
so
in
this
talk,
we'll
talk
through
how
carbonize
components
communicate
within
your
cluster
and
the
security
of
these
communications?
First
we'll
cover
what
the
main
components
are
and
what
communications
go
between
those
components.
Then
we'll
talk
about
what
communications
security
should
be
and
like
the
properties
that
you
might
want
and
how
this
is
typically
implemented,
will
talk
about
the
certificate
authorities
that
exist
in
kubernetes
and
how
these
are
used
to
protect
cluster
communications
and
then
we'll
wrap
up
so
that
you
understand
how
all
these
pieces
work
together
in
concert
all
right.
A
So,
first
off
to
start
what
components
of
kubernetes
even
communicate
with
each
other.
So
here's
here's
your
cluster
at
a
high
level
with
the
main
incriminated
components
on
the
left
in
the
control
plane,
you
run
several
pieces
on
master
nodes
to
actually
control
your
clusters,
the
koreas
controllers,
which
tell
your
clusters
what
to
do,
and
at
CD,
which
is
a
database
that
stores
your
cluster
state
on
the
right
in
your
worker
notes.
You
run
your
workloads.
These
are
their
our
kubernetes
components,
they're
deployed
on
your
nodes
that
are
needed
to
actually
manage
the
workloads.
A
These
listen
to
the
canaries
controllers
that
are
in
the
master
and
do
what
they're
told
to
do
and
your
workloads
themselves.
The
worker
nodes
run
your
pods,
which
are
just
collections
of
one
or
more
containers.
So
if
we
break
down
these
components
a
little
bit
more,
you
can
break
down
on
the
left,
the
criminate
controllers
into
the
api,
server,
scheduler
controller
manager
and
other
controllers.
These
perform
the
typical
functions
that
you'd
expect
of
your
of
your
kubernetes
cluster
to
scale
scheduled
workloads
and
respond
to
user
requests.
A
You'll
also
see
that
we
broken
out
at
CD
into
more
than
one
instance
of
at
CD
at
CD
is
a
distributed
datastore
and
run
in
a
high
of
L
Valley
mode.
So
there
are
multiple
instances
that
interact
with
each
other
to
actually
maintain
state
for
your
cluster
and
on
the
right
in
the
worker
nodes
you'll
see
we
have
the
cubelet,
which
is
the
main
component,
each
node
increment
unnies
its
responsible
for
for
what's
running
on
that
particular
node
and
receiving
instructions
from
the
control
plane.
A
It
watches
the
containers
and
makes
sure
they're
running
as
they're
supposed
to
be
running
and
other
company's
components
like
q,
proxy
or
cube
dns
are
actually
run
as
daemon
sets,
which
are
pods.
That
must
be
run
on
every
node
in
your
cluster.
This
could
also
include
other
system
containers
or
add-ons
for
your
particular
configuration
or
your
from
your
cloud
provider,
for
example.
A
So
if
we
add
the
flows
that
actually
go
between
these
components,
you'll
notice
and
don't
worry-
we're
gonna
keep
this
relatively
simple,
that
some
of
these
components
actually
seem
to
get
a
lot
more
attention
and
act
as
the
flow-through
points
for
a
lot
of
communications
that
exist
in
your
cluster,
including
between
other
components
and
specifically,
you'll,
see
that
almost
all
communications
touch
the
cube
api
server,
the
cubelet
and
@cd.
For
example,
say
your
pod
needed
a
secret.
It
would
tell
this
to
the
cubelet.
A
The
cubelet
would
communicate
this
to
the
cube
api
server
and
the
cube
api
server
would
communicate
with
that
CD
to
actually
retrieve
that
secret.
So
the
cube
api
server
is
critical
for
everything
in
the
cluster.
It
amends
talk
to
it.
Other
criminals
controllers
talk
to
it
at
CDs,
talks
to
it
and
it's
the
point
of
communication
with
the
nodes.
Cubelet,
all
communication
paths
from
the
cluster
of
the
master
terminated
the
API
server.
Similarly,
everything
that
goes
from
the
node
goes
through
the
cubelet.
A
It's
the
brain
setter
for
the
node
and
EDD
is
critical
in
that
it's
store
state.
So
it
holds
all
the
important
info
with
your
cluster.
It
acts
like
sort
of
like
a
like
a
hard
drive
for
your
cluster.
So
even
if
it
doesn't
talk
to
many
other
pieces,
we're
gonna
want
to
make
sure
any
communications
with
that.
Cd
are
really
well
protected.
So
with
that
in
mind,
we're
going
to
focus
more
heavily
on
these
three
pieces
in
interest:
cluster
communications,
the
cube
api
server,
the
cubelet
and
xcd
okay.
A
But
first,
let's
talk
about
properties
of
good
communication
security
and
we'll
come
back
and
then
apply
that
knowledge
to
kubernetes.
So
when
protecting
data
in
transit,
there
are
three
common
security
properties
are
gonna
want
to
look
for.
The
first
is
authentication
which
ensures
that
you
know
and
verify
the
source
and
destination
of
the
communications,
whether
it
be
a
human
or
a
process,
you're
verifying
that
the
correspond.
A
In
order
to
keep
it
private
encryption
is
a
process
through
which,
through
which
legible
data
known
as
plaintext,
is
made
illegible
known
as
ciphertext,
with
the
goal
of
ensuring
that
the
plaintext
is
only
accessible
by
parties
authorized
by
the
owner
of
that
data,
and
the
algorithm
is
used
in
the
encryption
process
are
well
known,
but
the
key
required
for
decrypting
the
ciphertext
is
kept
secret.
So
encryption
in
transit
often
uses
asymmetric
encryption
for
key
exchange
to
establish
a
shared
symmetric
key,
which
is
then
used
for
the
actual
data.
A
Encryption
note
that
some
forms
of
encryption
actually
already
have
integrity
and
authentication
as
part
of
the
encryption.
So
there
aren't
necessarily
separate
systems
that
you're
putting
in
place.
You
might
be
able
to
get
many
of
these
characteristics
for
the
price
of
one,
and
we
can
expect
different
kinds
of
communications
between
components
in
our
infrastructure.
There
are
client
to
server
communications
where
a
client
like
Alice
is
the
user
requesting
webpage,
for
example,
and
connects
to
a
server
like
Bob,
which
receives
a
request.
A
You
can
also
have
peer-to-peer
communications,
for
example,
when
running
a
system
in
hive
available,
I
think
high
availability
mode.
When
there
are
several
nodes
of
the
same
service
up
and
running
the
peer
nodes
are
going
to
want
to
mutually
authenticate
to
ensure
that
they're
in
fact,
equally
trusted.
A
This
just
means
that
each
node
needs
to
act
as
both
a
client
and
a
server
in
that
peer,
State,
so
suppose,
Alice
and
Bob
are
communicating
and
they're
encrypting
their
messages
to
each
other
if
they
communicate
using
a
transport
layer,
security
or
TLS,
Alice
and
Bob
perform
a
handshake
and
Alice
knows
that
she's
talking
to
Bob,
because
she
can
verify
Bob
certificate.
Mutual
TLS
means
that
both
the
server
verifies
that
the
clients,
identity
and
the
client
verifies
the
server's
identity.
A
But
if
Alice
hadn't
authenticated
Bob,
their
communications
are
subject
to
what's
known
as
a
man-in-the-middle
attack,
a
man-in-the-middle
attack
or
a
person
in
the
middle
attack
or
a
monkey
in
the
middle
attack.
Whatever
you
want
to
call,
it
occurs
when
another
party
can
intercept
and
relay
all
messages
between
the
two
parties
who
are
trying
to
communicate
so
Eve.
A
The
eavesdropper
in
this
case,
pretends
to
be
Bob,
but
when
talking
to
Alice
and
pretends
to
be
Alice
when
talking
about
and
every
time,
Alice
sends
a
message
to
Bob
Eve
intercepts
it
and
can
read
it
before
sending
it
on
to
Bob.
We
also
have
malicious,
Malory
might
even
change
the
content
of
that
message.
If
they
wanted
to
the
worst
part,
is
Alice
and
Bob
actually
think
that
they're
talking
to
each
other,
they
have
no
idea.
That's
why
authentication
here
is
so
critical.
However,
even
with
on
authentication,
we
face
a
slightly
different
issue
right.
A
A
So
a
root
CA
assigns
a
root
certificate,
like
you
see
here,
which
can
then
be
used
to
sign
intermediate
certificates,
which
can
in
turn
be
used
to
sign
leaf
certificates
in
practice.
A
root
CA
is
one
that's
widely
trusted.
This
allows
users
to
ultimately
only
trust.
A
handful
of
you
know
very
small
number
of
parties,
the
roots
in
order
to
be
able
to
protect
their
communications
and
rely
on
things
that
are
below
those
roots.
To
get
a
certificate.
An
entity
generates
a
key
pair.
A
Then
the
entity
submits
a
certificate,
signing
request
or
CSR
to
a
CA
which
contains
the
public
key
and
the
entity's
identity
and
that's
signed
by
the
CA.
A
self
signed
certificate
is
a
certificate
which
is
signed
by
the
same
entity
to
which
it's
issued
so
note
that,
specifically,
a
root
certificate
is
actually
a
self
signed
certificate,
but
we
place
more
trust
in
it
due
to
who
manages
it
and
some
of
the
processes
they
might
have
in
place
to
actually
manage
those
keys.
A
A
really
important
item
to
call
out
here
what
we're
talking
about
kubernetes
is
is
the
difference
between
public
web
CAS
and
private
CIA's
write
a
public
CA
is
used
to
authenticate
websites
that
you
connect
to
on
a
public
internet
and
has
root
starts
from
a
handful
of
trusted
CAS.
Those
are
usually
shipped
with,
like
your
browser
or
your
device.
These
web
certs
me
need
to
come
from
a
public
CA,
whereas
a
private
CA
might
be
used
within
an
organization,
for
example
between
components
and
infrastructure.
A
Kubernetes
uses
a
private
CA
for
components
and
you'll
need
to
use
a
public
CA
for
any
web
apps
that
you
host
on
Curb
identities,
for
example,
that
you
want
what
end
users
to
connect
you
and
when
we
talk
about
public
key
infrastructure
or
PKI.
We're
actually
talking
about
all
of
this
stuff,
we're
talking
about
all
these
systems
in
your
organization,
the
CAS,
if
you
have
any
what
C
is
you
trust,
your
private
keys,
your
certificates,
how
you
manage
the
keys?
All
of
that
stuff?
A
It's
how
you
organize
that
entire,
an
entire
infrastructure
each
organization's
needs
here,
going
to
be
very
specific,
so
that
how
you
set
that
up
should
actually
be
dictated
on
your
needs,
rather
than
me
telling
you
what's
what's
the
best
practice
in
that
regard.
One
last
item
I,
want
to
cover
to
avoid
confusion,
is
between
keys
and
certificates.
A
When
you
generate
a
certificate,
you
first
need
to
generate
that
asymmetric
keeper,
like
I
just
mentioned
with
a
private
and
a
public
key,
your
public
key
verifies
who
you
say
you
are,
and
you
can
use
that
to
have
people
send
you
messages,
whereas
your
private
key
is
how
you
prove
who
you
are
using
signatures,
and
you
can
read
your
messages
using
decryption.
You
make
your
public
key,
well
public
right,
but
to
verify
that
it's.
In
fact,
your
key,
like
we
were
talking
earlier
about
Alice
and
Bob,
it's
signed
by
a
certificate
authority.
A
The
certificate
authority
uses
their
private
key
to
sign
the
certificate
effectively
saying
yes,
I
attest
the
business
Bob's
key.
Then
the
entity
examining
your
certificate,
like
Alice,
only
needs
to
verify
and
trust
that
signature
in
order
to
believe
that
is,
in
fact
your
public
key.
So
going
back
to
earlier,
though,
with
with
a
self-signed
cert
the
entity,
it's
just
an
entity
that
signs
their
own
public
key
with
their
own
private
key
right.
That's
what
a
self-signed
cert
is
it's
same
entity,
that's
doing
both
okay!
A
So
now
that
we
have
some
PKI
basics
covered,
let's
dig
into
specifically
what
happens
in
kubernetes.
Lots
of
people
have
complained
to
me
that
setting
up
your
equipment
as
CA
is
really
hard
and
it's
one
of
the
hardest
things
to
do.
If
you're
writing
commands
yourself,
it's
one
of
the
first
steps
that
you
take
and
it's
critical
to
your
cluster
security.
A
That's
actually
a
lot
of
work.
If
you
use
cube
cube
ATM,
then
the
certificates
that
your
cluster
requires
are
automatically
generated.
Similarly,
if
you
use
a
hosted
solution
like
gke,
this
is
done
for
you.
An
alternative
is
actually
not
to
have
your
cluster
or
a
setup
tool
create,
manage
and
install
any
of
the
keys
for
you.
That
might
make
sense
for
a
situation
where
you
have
a
CA
that
you
were
on
outside
of
kubernetes,
and
you
just
want
to
import
the
necessary
certificates
to
your
cluster
that
lets.
You
rely
on
an
external
root
of
trust.
A
You
can
also
use
Q
ATM
in
this
case.
It's
something
that
it's
a
functionality
called
an
external
CA
incubating
and
if
you
are
doing
this,
the
root
CA
can
be
outside
kubernetes,
but
you
still
lend
me
at
minimum
needs
some
intermediate
CAS
that
kubernetes
manage
right,
the
roots
there
and
it
signs
things
that
the
intermediates
use
career
days
actually
needs
at
least
two
CAS
to
operate.
A
The
equipment
ease
cluster
CA,
sometimes
called
a
cluster
root,
CA
and
the
xcd
CA
and
optionally,
or
slightly
differently
the
cube
front
proxy
CA,
since
this
allows
routing
from
the
public
internet
you'll
actually
need
that
to
serve
public
certs.
So
it's
an
it's
an
au
category
and
we're
not
gonna
we're
not
gonna
cover
it.
A
These
CA
s,
issue
certificates
for
various
criminals,
components
the
ones
that
we
discussed
at
the
very
beginning,
in
fact,
for
client
and
server
shirts
for
the
cubelet
api
server
and
at
CD.
So
what
does
that
actually?
Look
like
here's,
a
quick
summary
of
which
bits
have
the
certs
that
we
just
talked
about
the
cube.
Api
server
has
a
server
search
and
a
client
search,
so
that
I
can
talk
to
the
cubelet.
Xcd
has
a
peer
certs
and
a
server
search.
A
Klein
starts
to
talk
to
you,
the
API
server
and
the
xcd
health
check,
client
and
the
front
proxy
client
has
a
server
search.
Your
cubelets
are
also
going
to
need
certs,
but
those
aren't
actually
part
of
that
initial
setup.
I'd
also
really
recommend
checking
of
the
Cabernets
documentation
here.
There's
a
really
nice
table
on
all
of
these
certs,
including
what
they're
signed
by
if
their
client
or
server
starts
and
where
the
shorts
actually
live
in
your
cluster,
the
sed
starts
and
keys,
like
no
surprise
live
in
ED
CD.
A
So
this
was
still
a
little
bit
confusing
to
me
when
I
was
learning,
this
I
was
like
I,
think
of
a
CA
as
being
online
and
something
I
can
just
like,
send
requests
to
and
make
things
happen
right
like
and
just
sign
stuff.
So
now
that
I
know
where
all
the
certs
and
keys
are
stored,
how
do
I
actually
sign
things
and
that's
a
certificate
signing
request.
A
So
that's
when
a
component
goes
to
us
goes
to
a
CA
and
requests
a
site,
a
cert
signed
for
itself,
if
you
think
to
our
Kerberos
components
when
and
why
would
that
actually
happen?
Well,
very
commonly
it
would
happen
when
a
new
node
joins
a
cluster
and
the
cubelet
needs
a
cubelet
cert
to
talk
to
the
API
server.
So
luckily,
crew
Briones
provides
a
certificates
API,
which
lets
you
provision
TLS
certs,
signed
by
a
certificate
authority
that
you
control.
A
This
allows
for
certificate,
signing
request,
s'
or
CSRs,
and
so
the
CAA
certificates
can
be
used
by
your
workloads
to
establish
trust.
This
features
in
beta
and
kubernetes
note,
however,
that
the
Korean
certificate
authority
doesn't
work
out
of
the
box.
You
can
configure
an
external
signer
to
be
used
with
this
or
you
can
use
a
built-in,
signer
and
tied
to
an
existing
CA,
such
as
the
cluster
root
CA.
A
That's
a
fairly
common
set
up
to
activate
the
built-in
signer
you
may
have
to
pass
the
cluster
signing
cert
file
flag
and
the
cluster
signing
key
file
flag
to
cube
controller
manager
in
the
design
to
take
the
certificate
signing
request
to
stable,
there's,
also
a
proposal
to
allow
multiple
CA.
So
that's
something
you're
interested
in.
Please
go
comment
on
github.
So
to
use
this
API
Creed.
First,
you
have
to
create
a
certificate
signing
request.
A
It's
a
file
with
a
particular
format
that
is
used
to
meet
pkcs
10
and
you
can
generate
you
generated
using
a
tool
like
CloudFlare,
SSL
or
CF.
Ssl
then
send
the
request
as
an
object
to
the
API
server
to
trigger
the
kubernetes
certificate
signing
request.
You
can
then
also
check
its
status
in
the
API
server
and
then-
and
this
is
the
critical
bit
you
need
to
get
their
certificate
signing
request
approved.
A
This
is
either
done
manually
like
bian
and
min,
or
it
can
be
automated
if
you've
configured
an
approver
to
meet
certain
criteria
automatically
that
works
well.
I
really
say
that
this
is
critical,
because
this
is
a
back
to
lead
delegating
a
trust
for
your
entire
cluster
or
anyone
or
any
any
system
that
can
approve.
These
requests
is
able
to
decide
who
can
be
in
your
cluster
and
how
they
can
talk
to
each
other.
A
The
approver
needs
to
authenticate
the
client
asking
for
the
cert
and
be
really
stringent
in
validating
that
request,
and
you
should
only
grant
those
permissions
carefully
and
finally,
once
that
csr
is
approved,
you
can
download
the
certificate,
distribute
it
to
the
different
components
that
need
it
and
then
use
it.
You
can
also
configure
certificate
rotation
for
these
certs.
Your
cubelet
certs
are
valid
for
one
year
by
default.
A
That
means
that
they
have
a
one
year
expiration
so
that
they
don't
have
to
be
renewed
too
frequently,
and
also
because
your
nodes
are
probably
going
to
change
anyways
right,
like
cattle,
not
pets,
right
to
check
the
validity
of
a
certificate
in
cube
ADM,
you
can
check
expiration
with
an
appropriately
named
flag
to
renew
your
starts
well.
Q
medium
also
takes
care
of
this.
For
you
there
will
be
new
starts
as
part
of
a
control
plane
upgrade.
It's
not
recommended
to
only
rely
on
this.
A
However,
if
you
have
more
complex
needs,
but
if
you're
regularly
updating
upgrading
anyways-
and
you
should
be-
then
this
is
an
OK
solution.
You
can
also
manually
renew
certs
by
explicitly
triggering
a
renewal
with
Q
ADM
certs.
Renew
to
renew
starts
using
the
Search
API,
just
specify
use
API
as
part
of
the
request,
and
if
you
want,
you
can
also
just
generate
this.
A
The
CSR
is
yourself
like
if
you're
using
external
CA-
or
you
can
also
just
renew
single
starts,
it
doesn't
have
to
be
all
the
certs
that
you
have
and
introduced
in
criminales
1.8
in
beta,
cubelets
or
rotation
allows
you
to
set
an
automatic
rotation
period,
which
is
even
better
at
which
point
a
new
key
will
be
generated
and
a
new
certificate
requested.
Before
the
current
certain
expires.
Once
the
new
sort
is
available,
it
can
then
be
used
to
authenticate
connections
from
the
culet
to
the
api
server,
just
as
it
was
doing
before
so.
A
To
set
this
past
the
Quba
process,
a
rotate
certificates
flag,
which
will
cause
it
to
request
a
certificate,
one
Coast
expiry
and
past
the
cube
controller
manager
and
experimental
cluster
of
signing
duration
flag,
which
controls
how
long
the
certificates
are
used
for
again,
something
like
a
year.
It's
reasonable,
but
it
really
depends
on
your
environment
in
this
case.
A
Going
back
to
our
diagram,
I
want
to
highlight
a
couple
of
flows
through
the
API
server
cube,
Linette
CD
that
we're
going
to
focus
on
one
from
the
API
server
to
the
cubelet
two
from
the
cubelet
back
to
the
api
server,
three
between
the
api
server
net
CD,
four
between
two
instances
of
that
CD
and
then
five
from
an
administrative
these
one
by
one.
So
first
from
the
API
server
to
the
cubelet,
the
API
server
might
communicate
with
the
cubelet
to
fetch
logs
attached
turning
pods
and
provide
port
forwarding
by
default
in
criminales.
A
Your
connection
from
the
api
server
to
the
cubelet
is
protected
with
unauthenticated
TLS.
These
connections
terminate
on
your
cubelets
HTTP
endpoint.
The
API
server
does
not
verify
the
qubits
serving
certificate
which
makes
the
connection
subject
to
a
person
in
the
middle
attacks
and
therefore
unsafe,
to
run
over
untrusted
networks.
It
might
be
fine
right
like,
for
example,
if
you're
running
all
of
this
on
your
own
private
network.
That's
not
necessarily
a
concern.
A
You
can
force
authentication
here
if
you
wanted,
if
you
want
to
using
the
cubelet
certificate
authority
flag
with
the
api
server
so
that
you
specify
a
root,
cert
bundle
which
is
then
used
to
verify
the
cubelet
server
certificate.
You
can
also
use
SSH
tunneling
to
avoid
connecting
over
an
untrusted
Network.
However,
SSH
tunneling
is
actually
deprecated
in
criminales,
so
it
doesn't
address
the
actual
authentication
issue,
so
maybe
not
the
best.
A
The
best
way
of
solving
that
issue,
no
matter
what
you
still
verify:
authorization
for
request
to
the
cubelet
api
for
other
node
components
from
the
api
server,
two
nodes,
pods
and
services
running
on
that
node.
The
connection
uses
plain
HTTP
connections
by
default,
they're
not
authenticated
or
encrypted,
but
actually
those
kind
of
communications
shouldn't
really
be
happening
anyways.
So,
but
if
you,
if
you
are
for
some
reason
doing
that,
you
can
specify
an
HTTP
endpoint
to
encrypt
the
connection
but
that
still
doesn't
validate
their
certificates,
so
there's
no
authentication
or
integrity
for
those
flows.
A
Okay,
let's
go
in
the
reverse
direction.
From
the
cubelet
to
the
API
server,
a
cubelet
will
communicate
with
the
API
server
to
watch
for
pods
that
it's
supposed
to
run
and
report
its
own
health.
The
API
server
listens
for
connections
on
secure,
HTTP,
port
443,
for
connections
from
other
master
components
from
the
cubelet
and
from
admittance,
so
from
cluster
components
to
the
API
server
requests
are
over
TLS.
A
There
are
many
kinds
of
authentication
to
the
API
server
available
and
you
can
enable
many
of
them,
but
including
specifically,
a
mode
called
node
authorizer,
which
is
meant
for
these
requests
that
are
coming
from
cubelets.
The
cubelet
will
use
whatever
it's
specified
in
its
own
cube
config,
which
can
be
no
two
authorizer
or
other
authentication
methods
like
if
you
wanted
to
use
bearer
tokens
or
something
instead
in
order
to
be
to
be
authorized
by
the
node
authorizer
cubelets
have
to
use
a
credential
that
identifies
as
them
as
being
in
the
system
nodes
group.
A
This
is
provisioned
in
the
cubed
cert,
which
specifies
the
identity
of
the
cubelet.
When
you
use
new
authorizer,
this
connection
is
protected
with
m
TLS,
since
this
connection
has
authentication
integrity
and
encryption
by
default.
The
master
can
run
on
on
trusted
network,
so
that's
kind
of
exciting
from
the
pot
to
the
API
server.
A
The
client
certificates
use
the
bearer
tokens.
This
protection
also
has
authentication
integrity
and
encryption,
which
is
great
ok
now
from
the
API
server
and
at
CD,
the
API
server
net
CD
might
communicate,
for
example,
to
to
ask
for
and
return
a
secret
for
a
particular
workload
from
the
API
server
to
at
CD.
The
communication
is
entirely
over
local
host
HTTP
port
80
and
it's
not
authenticated
or
encrypted.
Now
that
sounds
scary,
but
it's
actually
probably
fine.
These
components
are
in
practice
running
on
the
same
node,
so
protecting
the
local
connection,
more
significantly,
probably
isn't
needed.
A
So
this
can
is
authenticated
using
and
this
connection
is
authenticated
using
one
of
the
SED
certs.
So
again,
despite
being
a
local
connection-
and
this
is
not
unusual-
you
just
by
being
a
local
connection-
you
have
authentication
integrity
encryption,
but
that's
not
unusual
as
all
the
requests
to
the
API
server
are
authenticated
in
such
a
way
or
in
some
in
some
way
now,
from
between
two
different
instances
of
sed
an
instance
of
a
TD
might
communicate
with
another
instance
of
sed
to
keep
state
updated
since
XE
D
is
a
distributed
datastore.
A
So
when
an
instance
of
sed
sends
a
request
to
another
instance,
that
request
has
authenticated
and
encrypted
using
mutual
TLS.
Exedy
uses
those
peer
certs
that
we
talked
about
earlier,
which
are
both
client
and
server
certs,
specifying
the
host
identity
and
then,
lastly,
from
the
administer
now
the
admit
is
another
main
components.
I
was
talking
about
before,
but
it's
one
of
the
most
commonly
asked
about
questions
so
I'll
just
quickly
summarize
some
options
that
you
have
here,
the
admin
might
connect
to
the
API
server
to
say,
manage
the
cluster.
A
The
omim
can
also
connect
to
the
cubelet,
which
requires
cubelet
authentication,
but
we're
not
going
to
go
into
that.
The
amenities
in
TLS
right
there
are
many
different
kinds
of
authentication
methods
available
here,
including
both
tokens,
for
example,
using
Open,
ID,
Connect
x5,
and
on
Clyde
certs
basic
auth,
using
static
passwords
enough
indicating
proxy
and
and
more
so,
if
you're,
using
the
client,
cert
method,
you
can
actually
set
this
up
using
the
certificates
API
and
from
the
certificate
signing
request.
Api
that
we
talked
about
earlier.
A
If
that's,
what
you
want
to
do
and
a
pod
might
communicate
with
another
pod
as
part
of
a
specific
workload
again,
no
kubernetes
components
require
this
particular
pod
pod
communication,
when
the
pod
sends
a
request
to
another
pod,
that
request
is
neither
authenticated,
gnorm
cryptid.
However,
you
can
restrict
pod
to
pod
traffic
with
like
a
network
policy,
and
you
can
encrypt
that
traffic
using
like
a
service
mesh
like
SDO
or
otherwise,
I'm
amending
application
layer,
encryption
all
right
so
pulling
it
all
together.
A
How
is
traffic
within
your
cluster
then
protected
and
how
do
credenza
components
communicate
securely?
In
summary,
for
the
five
flows
that
we've
looked
at
one
from
the
API
server
to
the
cubelet,
this
uses
an
authenticated
TLS
by
default,
so
your
communications
are
encrypted,
but
not
necessarily
authenticated
you'll
need
to
specify
at
cubelet
certificate
authority
and
specify
the
HTTP
endpoints
for
your
notes
from
the
cubelet
to
the
api
server.
This
uses
mutual
TLS
by
default,
so
you
get
encryption,
integrity
and
authentication,
checkmark
we're
good
3
between
the
API
server
and
@cd.
A
The
api
server
talks
to
at
CD
/
localhost,
which
is
not
protected
but,
like
we
said,
that's,
probably
ok
and
you
can
set
up
m
TLS
if
you
prefer,
and
at
CD
talks
to
the
API
server
/
/
HTTP
and
is
authenticated
encrypted
and
has
integrity
between
two
instances
of
at
CD.
These
peer
instances
communicate
using
mutual
TLS
and
from
the
admin
to
the
API
server.
You
have
lots
of
options
depending
on
your
authentication
method,
just
be
careful
with
anonymous
odd,
so
I
reckon
I'm
going
to
call
it
a
couple
of
additional
optional
controls.
A
You
have
there
what
might
be
different
from
we'll
just
described
by
default.
Gke
sets
up
the
cluster
root
CA
in
the
NCD
CA.
For
you,
these
are
single
cluster
CAS,
so
that
if
any
one
clusters,
private
key
were
to
be
compromised,
no
other
cluster
is
affected.
Certs
are
issued
by
the
cluster
roots,
asserts
that
are
issued
by
the
cluster
roots.
You
have
a
validity
of
five
years
right
now.
It's
your
job
to
rotate
those
certs,
given
that
you
might
have
other
dependencies
on
that
CI
and
those
sorts
in
your
cluster.
A
You
can
do
that
by
completing,
what's
called
a
credential
rotation
which
it
creates
a
new
sort
waits
for
you
to
update
anything
that
needs
that
cert
and
then
completes
the
rotation
by
invalidating
the
old
sir
note
that
today,
that
also
requires
a
migration
to
new
IP
addresses,
starting
in
gke
113
and
higher.
The
traffic
from
the
API
server
to
the
Keyblade
is
authenticated
by
default
in
G
KETV
at
CDC.
A
is
fully
managed
for
you
and
Google
manages
rotation
of
those
shirts.
There's
nothing
there
for
you
to
do
and
the
certificate
signing
request.
A
Api
is
also
already
set
up
for
you
and
a
set
up
to
use
the
cluster
route
CA.
So
there's
an
automatic
approver
that
you
can
use
for
those
certs.
If
you
use
a
an
option
that
we
have
called
shielded
gke
nodes,
the
biggest
difference
is
how
the
cubelets
bootstrap
credentials
are
protected.
That's
what's
actually
used
when
submitting
that
certificate
signing
request
the
certificate.
Signing
request
is
bound
to
the
machine
identity
which
is
protected
by
a
V
TPM
element.
So
you
can't
use
that
shared
secret
from
anywhere.
A
But
that
note
so
basically
what
I'm
trying
to
say
is
that
no
one
else
can
pretend
to
be
that
node
and
go
get
a
certificate
for
that
node.
That's
actually
really
powerful.
If
you're
running
on
gke,
please,
please,
please
enable
shielded,
GK
nodes,
it's
a
great
security
feature
and
also
ensures
secure
boot
for
your
notes
and,
lastly,
for
user
authentication.
A
Basic
auth
and
client
certs
are
not
on
by
default
in
GK
since
112,
but
if
you
have
an
older
cluster
that
you've
been
updating,
please
disable
those
for
cases
where
you
don't
need
them
and
if
you
want
to
read
more
check
out
the
documentation
on
cluster
trusts.
That
explains
how
some
of
these
things
were
conjugate.
A
Some
of
you
might
also
be
familiar
with
the
Center
for
Internet,
Security
or
CIS
benchmarks
for
kubernetes.
These
were
recently
updated
about
a
month
ago
to
version
151.50
of
the
benchmarks
which
apply
to
kubernetes
version.
1.15
I
know
that's
confusing.
The
numbers
are
not
the
same.
You
might
be
using
these
in
your
environment
to
verify
that
your
environment
is
meeting
the
recommended
best
practices
for
security,
so
I
want
to
call
out
some
of
the
benchmark
elements
that
specifically
relate
to
cluster
trust
and
the
element
that
we've
been
discussing
for
the
API
server
12.6.
A
You
should
set
up
a
cubelet
certificate
authority
to
verify
the
cubelet
for
communications
from
the
api
server
to
the
cubelet
1.2
29.
You
should
set
up
a
TLS
connection
between
the
api
server
net
CD
1
dot
2.30
ensure
that
the
api
server
only
sir
traffic
to
encrypted
HTTP
endpoints
for
the
controller
manager,
1,
not
3.6,
make
sure
to
rotate
the
cube
load
certs
and
for
SPD.
A
A
The
pack
line
start
off
should
not
be
used
for
users
instead
use
OAuth
or
another
such
method,
and,
finally,
on
the
cubelet
4
to
10
is
to
ensure
that
the
cubelet
only
serves
traffic
on
to
HTTPS
and
for
2012
are
about
setting
up
automated
cert
rotation
for
cuba,
client
certs,
all
in
all,
nothing
unexpected
here!
If
you
follow
the
recommendations
that
we've
been
talking
about
the
whole
time,
you'll
be
in
a
good
spot
on
the
CI
smudge
marks
and
to
give
you
kind
of
a
one
slide,
take
away
the
best
practices
from
today.
A
If
you
have
to
remember
something
from
this
talk,
first
set
up
your
cluster
using
cube
ATM
if
you're
not
using
a
managed
service
that
does
it
for
you.
This
will
be
significantly
simpler
for
you
than
doing
it
yourself,
and
you
can
still
use
an
external
CA.
Please
rotate
your
certs.
This
doesn't
have
to
be
super
frequent,
but
either
set
this
up
to
do
it
automatically
or
do
it
manually
on
a
regular
enough
basis,
so
that
minimally,
you
know
that
you
can
do
it
and
you
can
do
it
if
you
want
to
have
to
do
it.
A
You
can
use
the
cubelets
of
the
korean
certs
api
and,
finally,
if
you
running
on
GK
specifically,
please
use
shielded
GK
a
nodes
to
protect
qulet
bootstrap
potentials
and
perform
a
credential
rotation
again
on
a
regular
enough
basis.
So
you
can
rotate
the
certs
in
your
cluster
routes.
Yet
so
that's
it.
I
have
a
couple
of
links
here.
If
you
want
to
learn
more
about
some
of
the
stuff
that
we
talked
about,
and
thank
you
very
much.