►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
So You Want to Run Vault in Kubernetes? - Seth Vargo, Google
Kubernetes is great for running applications, but can it run secure workloads like HashiCorp Vault, a popular open source secrets management tool? This two-part, demo-driven talk explores the answers to that question. The first part showcases how to run Vault securely on Kubernetes. We walk through different deployment architectures and strategies for making sure Vault is run in the most secure manner on Kubernetes. The second part focuses on how services deployed in Kubernetes interact with Vault. We discuss the implementation details and tradeoffs for authenticating pods and services to Vault to retrieve dynamic credentials like database passwords and Google Cloud IAM credentials. Attendees we leave with an understanding of how to operationalize better run secure workloads like Vault inside Kubernetes and how to expose secure workloads to other services in the cluster.
To learn more: https://sched.co/GrZu
A
Welcome
to
so,
you
want
to
run
vault
on
kubernetes
or
in
kubernetes,
depending
on
your
preposition
choice.
I
know
some
people
are
still
filtering
in
if
you
have
empty
seats.
Next
to
you,
please
help
people
find
a
seat.
Thank
you
all.
So
much
for
coming
I
realize
that
this
is
an
odd
time
for
some
of
you,
you're
very
hungry,
and
for
some
of
you
you're,
very
tired,
because
you
just
ate
so
it's
the
ideal
conference
talk
spot.
It's
not.
A
A
You
can
find
me
on
Twitter.
My
DMS
are
open,
I'm,
pretty
good
at
responding
to
people,
although
I've
been
drowning
in
messages
at
cube
con.
Thank
you
all.
So
much
for
reaching
out
my
twitter
handle
will
be
on
all
of
the
slides.
If
you
have
a
question
or
if
you
want
to
DM
me,
because
you
don't
feel
comfortable
asking,
it
publicly
feel
free
to
do
so.
So
we
have
a
decent
amount
of
content
to
cover
today
and
I
promise
live
demos
so
we'll
get
there.
First,
we're
gonna
talk
about
vault
I.
A
Imagine
that
because
most
of
you
were
in
this
room,
you
have
a
good
understanding
of
what
vol
day's
for
those
of
you
that
don't
I'm
gonna,
give
you
kind
of
a
very
quick
crash
course,
but
the
majority
of
this
talk
is
going
to
focus
on
vault
in
kubernetes.
I
just
want
to
set
a
really
strong
base
ground.
Then
we'll
start
with
really
part
one,
which
is:
how
do
you
run
vault
on
kubernetes,
maybe
you're
a
security
team
or
security
architect,
and
you
want
to
provide
vault
to
your
team
as
a
service.
A
How
do
you
run
this
thing
on
kubernetes,
then
we're
going
to
switch
gears
and
we're
gonna
put
our
developer
or
operator
hat
on
and
we're
gonna
say:
okay,
I'm,
a
person
who
needs
to
get
secrets
from
vault
and
I'm
running
my
applications
and
services
on
kubernetes.
How
do
I
authenticate,
how
do
I
get
access
to
the
secrets?
A
I
need
so
two
different
trains
of
thoughts
there'll
be
a
very
clear
logical,
switch
where
we
take
off
our
security,
architect,
hat
and
put
on
our
developer
hat,
and
how
do
we
configure
vault
best
to
run
in
that
architecture
and
how
do
we
connect
to
it
in
kind
of
the
the
best
secure
way
or
the
most
secure
way?
So
with
that,
let's
go
ahead
and
get
started
part
one.
We're
gonna
talk
about
volt
and
again:
I'm
gonna.
A
Do
this
very
quickly,
so
I
apologize
if
you're
already
familiar
with
vault,
you
can
fall
asleep
for
two
minutes.
If
you're
not
familiar
with
the
ball,
gonna
go
very
quickly
just
to
get
everyone
on
the
same
baseline.
So
what
is
a
secret?
We
have
kubernetes
secrets,
that's
not
really
what
I'm
talking
about
here,
what
what
is
a
secret
and
there's
kind
of
two
ways
that
we
can
think
about
secrets?
The
first
is
that
anything
that
can
be
used
to
authenticate
or
authorize
most
obvious
example
username
and
password
an
API
key
TLS
certificate.
A
But
then
there's
this
kind
of
third
or
second
classification
of
secret
things
like
a
credit
card
number,
a
social
security
number,
a
passport
number
and
then
there's
a
third
classification
of
secret
that
are
context
dependent
things
like
a
phone
number,
an
email
address,
a
home
address
and
all
of
these
are
really
dependent
on
your
business.
So
your
definition
of
a
secret,
if
you
work
for
the
federal
government,
might
be
different
than
a
startups
definition
of
a
secret.
A
If
you
run
the
world's
largest
online
cat
meme
website
email
addresses
are
probably
not
something
that
you
protect
and
encrypt
right.
That's
just
a
byproduct
of
the
service
you're
offering.
But
if
you're
a
company
like
MailChimp
or
a
mailing
list
server,
that's
critical
to
your
business,
do
you
likely
treat
those
email
addresses
with
a
little
bit
more
scrutiny
and
a
little
bit
more
security?
So
when
we
accept
the
fact
that
every
organizations
definition
of
a
secret
is
slightly
different,
we
need
a
tool,
that's
kind
of
flexible.
A
So
there
are
a
lot
of
questions
that
come
up
when
we're
choosing
those
tools
like
how
does
an
application
get
access
to
a
secret
and
he's
an
API
key
to
talk
to
some
third-party
service,
or
it
needs
a
database
username
and
password,
but
at
the
same
time,
humans
also
need
access
to
those
things
right.
A
local
developer
might
be
trying
something
out
on
their
console
or
doing
local
development.
They
need
access
to
the
staging
or
the
testing
database
to
make
sure
that
their
query
is
going
to
operate
successfully.
A
So
how
do
we
have
both
applications,
services,
computers
and
humans,
access,
those
same
secrets
and
then
secrets
are
not
static.
I
know
this
might
be
a
shock
to
some
of
you,
but
they're
dynamic,
I
like
to
say
that
secrets
are
like
IP
addresses
they
come
and
go
with
the
tides.
So
how
do
we
update
them
over
time?
Maybe
we're
updating
daily
weekly,
you
know
by
annually,
but
how
do
we
update
them
and
how
do
we
ensure
that
that
update
is
propagated
across
all
of
our
systems
when
it's
leaked?
How
do
we
revoke
it?
A
Someone
accidentally
pushed
something
to
a
source
control
repository,
that's
publicly
accessible.
How
do
we
were
credential
and
then,
given
all
of
that,
how
do
we
know
who's
accessing
your
crunching
credential,
where
they're
accessing
it
from
how
they're
accessing
it
is
it
still
in
use?
And
what
do
we
do
in
a
compromise?
What
is
our
break
glass
situation?
A
A
So
volt
aims
to
be
a
single
source
of
secrets
that
provides
both
machine
and
human
based
access,
so
machines
can
authenticate
via
things
like
the
API
and
humans
can
interact
via
things
like
the
CLI
or
the
API
if
they
prefer
and
most
importantly,
it's
designed
to
be
cloud
friendly.
So
there
are
a
lot
of
Secrets
management
solutions
out
there.
A
Vault
happens
to
be
open-source
and
built
for
the
cloud
era,
meaning
it
doesn't
require
like
a
hardware
source
of
entropy
like
an
HSM,
so
you
can
run
this
in
any
major
cloud
provider
or
on-premise
if
you'd,
like
data
is
encrypted
in
transit
with
TLS,
1
2
or
later,
and
it's
encrypted
at
rest
with
the
latest
cipher
suites
and
it
operates
on
a
hierarchical
key
value
store.
So
if
you're
familiar
with
like
GCS
or
s3,
where
you
know
you
have
folders
but
they're,
not
really
folders,
that's
kind
of
how
volt
is
structured
as
well.
A
So
here's
a
quick
example
of
interacting
with
vault
on
the
CLI.
This
is
an
example
of
putting
some
data
into
vault
and
retrieving
that
data
back
out.
This
is
using
volts
key
value
store,
it's
just
think
of
it
like
encrypted
Redis
or
encrypted
memcached
I
put
plaintext
data
in
vault,
encrypts,
it
and
trans
it
with
TLS
and
at
rest,
and
then,
when
I
want
that
data
back
out,
I
authenticate
and
authorize
to
retrieve
the
plaintext
data.
A
Everything
that
you
do
with
the
vault
CL
is
also
available
via
the
API.
That's
going
to
be
really
important
when
we
start
talking
about
how
pods
and
services
are
going
to
get
access
to
these
credentials.
So
here's
that
same
thing,
but
done
via
the
API
using
curl.
So
here
you
can
see
that
same
exact
thing
slightly
less
of
a
nice
human
experience,
because
this
is
designed
for
machines
and
computers.
A
So
what
happens
in
these
break
glass
scenarios?
These
are
real
situations
that
people
have
encountered.
You
know
an
operator
accidentally
copies
passwords
text
from
the
remote
server
onto
their
local
machine
during
debugging,
an
application
logs
its
database
credentials
to
disk
as
part
of
a
routine
debug
or
a
stack
trace.
You
have
a
rogue
package
like
I,
don't
know,
maybe
in
the
NPM
ecosystem
that
dumps
your
entire
applications,
environments
to
some
HTTP
web
server
to
do.
Bitcoin,
mining
or
a
server
includes
an
API
token
in
a
diagnostic
output.
A
Right
because
a
lot
of
these
things
are
kind
of
insecure
by
default,
we
want
to
do
our
best
to
prevent
this
from
happening,
but
we
also
have
to
accept
the
reality
that
it's
going
to
happen.
The
unfortunate
reality
is
that
these
are
very
real
things
that
happen
every
single
day,
so
we
also
need
a
plan
for
recovering
for
when
these
bad
things
happen,
and
we
like
to
call
this
like
a
break
glass
procedure.
A
A
Like
IP
addresses
just
like
when
your
laptop
or
your
mobile
phone
connects
to
this
conference
Wi-Fi,
you
get
an
IP
address
from
a
DHCP
server,
that's
unique
to
you
when
applications
or
humans
connect
to
volt
and
asked
for
a
credential,
they
get
a
credential,
that's
unique
to
them,
and
just
like
no
two
of
you
were
sharing
the
same
IP
address.
No
two
instances
of
your
applications
are
staring
sharing.
The
same
database
credential,
so
it's
a
broker
in
a
sense
for
secrets.
A
A
Instead,
we're
sharing
nothing
essentially.
So,
with
this
dynamic
secrets,
what
happens
is
a
human
or
a
machine
authenticates
to
vault
and
request
some
credentials?
Vault
then
acts
as
a
broker.
It
audits
that
request
and
then
connects
to
that
third-party
service.
That
could
be
something
like
a
Postgres
or
my
sequel
database.
It
could
be
an
Oracle
database.
It
could
also
be
something
like
Google
Cloud.
We
can
generate
iam
credentials
or
Amazon.
A
We
can
generate
access
key
pairs
dynamically
on
the
fly
it
connects
to
that
third
party
system,
whatever
it
might
be,
generates
the
credentials
and
returns
them
to
the
requester,
assuming
they've
been
authorized,
and
it
audits
that
request
each
time
the
requester
requests
new
credentials.
They
get
a
different
set
of
credentials.
So,
if
you
imagine
your
application
boots,
the
first
thing
that
does
is
authenticates
to
vault.
Vault
then
goes
out
to
whatever
services
you
need.
Postgres,
my
sequel
creates
the
user,
for
you
returns
a
unique
username
and
password.
Your
app
then
uses
that
for
its
lifetime.
A
So
here
are
some
of
the
examples
of
what
we
call
the
secrets
engines
or
the
dynamic
secrets
engines
that
allow
us
to
generate
these
dynamic
credentials.
So
there
are
major
cloud
providers:
google,
AWS
asher,
pretty
much
every
sequel
standard
database
as
well
as
things
like
Oracle
and
MongoDB,
and
then
I
have
this
category
of
other,
because
I
don't
know
where
to
put
them
right
things
like
rabbitmq
TLS.
What
can
actually
act
as
your
TLS
root
or
intermediary?
A
It
can
also
manage
SSH
access
for
you,
so
instead
of
copying
around
authorized
keys
files
and
stuff
like
that,
you
can
use
and
delegate
that
to
vault
things
like
Active,
Directory,
etc.
So
I
understand
this
was
very
quick.
So,
if
you're
new
to
vault,
this
is
really
the
slide.
That
matters
is
that
vault
provides
authentication,
authorization
and
auditing
and
it
is
a
broker
for
secrets
management,
so
vault
clients,
those
gonna,
be
humans
or
machines
provide
some
sort
of
identity.
That
identity
is
mapped
to
policy.
A
That
policy
then
authorizes
them
to
access
as
euro
or
more
secrets
and
from
there
every
secret
that
they
access
in
every
operation
they
perform
is
audited
and
that
audit
log
is
a
rich
audit
log,
that's
by
default
under
JSON
format.
So
you
can
parse
that
and
feed
that
into
any
logging
or
anomaly
detection
software
that
you
have
so
that's
a
very
quick
overview
of
vault.
So
how
do
you
run
this
thing
on
kubernetes
right,
so
I'm
sold
on
this
idea
of
dynamic
secrets,
I'm
sold
on
a
bunch
of
vault
features?
A
We
didn't
talk
about
like
Shamir,
secret
sharing
or
my
boss,
just
told
me
that
I
have
to
figure
this
out
and
I
came
to
this
talk.
So
how
do
I
run
followed
on
kubernetes?
Well,
vole
operates
in
a
client-server
architecture,
so
clients
are
the
the
local
machines
and
the
humans
that
are
accessing
the
server
and
then
volt
itself
runs
as
a
stateful
service
in
high
availability
mode
or
not
high
availability
mode,
but
you
should
run
in
high
availability
mode,
so
this
architecture
diagram
shows
how
we
would
do
it
on
Google,
particularly
google
kubernetes
engine.
A
But
you
could
replace
these
and
we'll
talk
about
the
individual
components
in
later
slides
here.
So
you
run
vault
in
high
availability
mode.
You
put
it
behind
a
load
balancer.
It's
your
choice
of
whether
that
load
balancer
is
public
facing
or
private
facing,
and
then
from
there
you
use
a
highly
available
storage
back-end
and
you
use
a
vault,
auto
unsealing
for
extra
availability
and
all
of
that
runs
on
top
of
kubernetes,
which
gets
us
really
cool
things
like
Auto
restarting
in
the
event
of
failure.
A
A
So
what
are
some
highlights
from
this
architecture?
So
the
first-
and
this
is
where
I
get
contention
is
a
dedicated
kubernetes
cluster.
So
vault
should
always
run
in
a
dedicated
kubernetes
cluster
which
access
is
tightly
controlled,
ideally
by
your
security,
architects
or
your
security
team.
No
other
apps
or
services
run
in
this
cluster
now
I
know
some
ones
are
like
what
about
namespaces
I'm
very
adamant,
and
this
is
my
opinion
that
you
should
run
into
difficulties.
A
Am
space
--is,
but
it's
up
to
you
it's
my
recommendation
that
you
run
in
as
a
dedicated
cluster
and
I'll
talk
about
that.
A
little
bit
more
in
a
bit,
you
want
to
use
a
highly
available
storage
back-end
for
volts,
so
vault
has
a
number
of
different
places
that
it
can
store
data
they're,
all
pluggable,
so
you
have
things
like
Google
Cloud
spanner
Google,
Cloud,
Storage,
Amazon,
s3,
Amazon
DynamoDB,
the
file
system
in
mem,
and
each
of
those
storage
backends
has
different
functionality
that
they
expose.
Some
are
faster
than
others.
A
Some
have
this
ability
to
support
transactional
interfaces.
Some
have
support
for
high
availability,
the
one
that
you're
looking
for
particularly
is
support
for
high
availability.
You
want
to
be
able
to
run
vault
in
highly
available
modes,
even
though
you're
running
under
kubernetes.
You
want
to
have
multiple
nodes
running
simultaneously.
A
Additionally,
you're
going
to
want
to
use
a
canvas
based,
auto
one
sealer
or
if
you're
running
on
prem
kubernetes,
you
can
use
an
HSM
based
auto
on
sealer
as
well.
This
is
used
to
be
an
enterprise
feature.
This
used
to
be
a
paid
feature.
It's
now
open
source
and
vault
1.0.
You
can
use
the
open
source
version
of
vault
to
do
auto
on
ceilings
via
KMS.
A
A
Absolutely
under
no
situation
should
you
ever
communicate
the
vault
over
anything
other
than
TLS
1.2
or
higher
I
generally
use
a
custom
CA
because
it's
flexible
it
gives
you
an
additional
level
of
trust
because
you
can
say
never
trust.
The
system
certs
only
trust
this
certificate.
So
you
get
kind
of
client
and
server-side
validation.
You
could
replace
this
with
your
own
trusted,
PKI
infrastructure
or
you
could
delegate
to
something
like
let's
encrypt,
but
you
definitely
want
to
use
TLS
you're
gonna
want
to
use
l4
for
load
balancing
instead
of
l7
for
load
balancing.
A
The
primary
reason
for
this
is
that
it
delegates
the
TLS
termination
to
volt
and
one
of
the
ways
in
which
machines
can
authenticate
default
is
with
TLS,
so
they
can
present,
what's
called
a
client
certificate
and
we
can
do
mutual
TLS
authentication
if
you're
doing
TLS
termination
at
an
l7
load.
Balancer
it's
going
to
terminate
that
connection
and
then
that
client
certificate
is
not
going
to
be
passed
along
to
vault
or
if
it
is
it's
going
to
require
a
lot
of
additional
configuration
in
that
load
balancer.
A
You
don't
actually
want
to
expose
vault
or
it's
my
recommendation
that
you
don't
expose
vault
through
the
standard,
kubernetes
service,
discovery
interfaces
or
any
of
the
ecosystem
tools
that
help
discover
services
and
route
to
services
in
kubernetes.
The
reason
for
this
is
twofold:
it
makes
signing
those
certificates
very
difficult
because
again
you're
not
delegating
to
the
load
balancer
you're
delegating
TLS
to
volt.
So
now
you
need
to
know
every
possible
DNS,
address
and
IP
that
the
cert
needs
to
respond
to
to
put
it
in
the
San
list.
A
But
at
the
same
time,
this
practice
really
keeps
vault
flexible
in
the
future.
Your
company
might
want
to
migrate
vault
to
a
different
service.
They
might
move
from.
You
know
kubernetes
to
some
crazy
new
cluster
scheduler
that
exists
in
ten
years
and
if
you're,
relying
on
the
kubernetes
specific
features
that
migration
path
is
more
difficult.
So
instead
I
like
to
tell
people
hey,
think
of
kubrick
Inc.
A
If
you're
running
on
a
recent
version
of
kubernetes
network
policies
etc,
if
you're
running
vault
in
a
shared
cluster,
you
may
not
be
able
to
enable
these
things
because
you
might
have
applications
that
depend
on
these
insecure
functionalities.
So
that's
another
reason
why
it's
beneficial
to
run
it
in
a
dedicated
cluster.
A
So
when
you
run
vault,
you're
gonna
want
to
run
all
want
to
run
it
as
a
deployment
or
a
stateful
set.
Previous
previously,
I
was
kind
of
advocating
for
a
stateful
set.
We've
been
able
to
make
it
work
very
well
with
a
deployment,
so
just
a
little
bit
faster
to
do
it
via
a
deployment,
but
you
can
use
a
stateful
set
as
well.
A
We're
definitely
gonna
want
to
use
anti
affinity
by
default.
Kubernetes
is
gonna,
the
scheduler
is
gonna
bin
pack,
all
of
our
nodes.
So
if
we
say
hey
run
five
vault
nodes,
it's
gonna
put
all
five
of
them
on
the
first
VM
and
that's
really
not
what
we
want
right,
because
if
that
VM
dies,
we've
taken
our
entire
cluster
offline,
so
we
can
leverage
pot,
anti
affinity
and
this
example
I've
used
preferred.
But
you
could
also
have
required
where
we
want
to
spread
those
pods
across
as
many
VMs
as
possible.
A
A
We
want
to
make
sure
that
we're
giving
the
container
IPC
lock
capabilities
if
you
give
vault
the
ability,
if
you
give
the
the
container
the
ability
to
do
memory,
locking
vault
will
take
advantage
of
that.
This
is
again
just
an
added
layer
of
security.
Vault
will
do
memory,
locking
when
doing
certain
write
operations
in
case
someone
else
is
on
that
process,
and
it's
trying
to
page
the
memory.
A
We're
gonna
make
sure
that
standbys
return
an
okay
status
by
default.
When
you
ask
for
the
health
of
a
volt
standby
it'll
say
it
will
return
a
non
200
status
code
and
we're
also
going
to
allow
uninitialized
nodes
to
join
our
cluster
and
that's
solely
so
that
we
can
route
to
them
for
the
initial
bootstrapping,
which
I'll
show
in
a
minute,
and
then
we
want
to
load
balancer
and
I'm,
not
very
opinionated.
On
this.
This
is
really
up
to
you
and
your
security
team.
A
In
this
example,
I'm
using
a
public
facing
load
balancer
with
the
load
balancer
service,
you
could
be
using
an
internal
load
balancer
or
you
could
be
using
annotations
to
get
this.
You
should
be
using
a
load
balancer
though,
at
no
point
should
you
ever
talk
directly
to
a
vault
pod.
You
should
always
be
going
directly
through
a
load
balancer,
whether
that's
private
or
public,
that's
entirely
up
to
the
security
policies
of
your
organization.
A
So
what
does
this
vault
configuration
look
like,
so
this
is
where
people
get
confused.
There's
two
adder
configurations
involved.
There's
this
thing
called
the
API
adder
and
this
thing
called
the
cluster
adder.
The
API
adder
is
set
to
the
load.
Balancers
address
the
API
adder
is
what
volt
advertises
that
things
should
connect
to
it
as
so.
This
is
the
load
balancer,
because
you
should
always
go
through
the
load
balancer.
The
cluster
adder
is
where
volt
advertises
to
it,
piers
where
it
lives-
and
this
is
the
pods
IP
address.
A
So
this
is
a
key
thing
if
you're
gonna
take
a
picture
of
any
slide,
you
want
to
take
a
picture
of
this
slide
and
the
way
you
get
the
pods
IP
address
is
using
fuel
graphs,
so
you
can
actually
get
that
directly
from
a
gamble.
Spec
and
I'll
show
an
example
of
this
in
a
second
you're
gonna
want
to
bind
to
the
pods
IP
address
when
you
listen
as
well,
and
then
optionally,
you
might
want
to
listen
on
local
host
without
TLS.
A
This
is
useful
if
you're
kind
of
getting
started
learning
about
vault,
because
you
can
exec
it
into
the
pod
and
run
volt
commands
directly
without
having
to
set
a
bunch
of
environment
variables
or
specify
flags
or
disable
TLS
verification,
as
I
said
before,
you're
going
to
use
a
chase
storage.
So
here's
two
examples:
there
are
many
storage
backends
that
support
high
availability
and
you'll
want
to
leverage
vault
one
datos
Auto,
one
stealing.
A
If
you're
running
a
version
of
vault,
that's
pretty
well
known:
oh
I
had
that
recommend
you
upgrade
there's
a
lot
of
new
features,
but
this
is
one
of
the
best
ones.
So
what
this
will
do
is
it'll
store
our
auto
or
unseal
keys
in
vault
encrypted
by
kms
and
when
a
new
node
joins
the
cluster,
assuming
it
has
access
to
that
KMS
key.
It
will
automatically
join
so
now,
we'll
do
a
demo
who's
excited.
B
A
Right
so,
let's
take
a
look
at
some
Hamel,
so
this
is
the
full
expanded
kind
of
PMO's.
What
I
just
showed
you
there's
our
load
balancer.
Obviously
the
IP
is
different.
One
of
the
things
I
don't
talk
about
in
the
slide
is
that
I'm
binding
kind
of
mapping
port
443,
which
is
a
standard
ssl
port
to
port
8200
I'm
up
in
the
air
of
whether
I
want
to
call
this
a
best
practice.
The
reason
I
do
this
is
that
most
hotel
and
conference
Wi-Fi
doesn't
allow
access
on
random
ports,
so
443
is
a
safe
one.
A
The
second
is
that
I
work
with
a
lot
of
enterprises,
and
it's
actually
really
hard
for
them
to
open
up
ports,
and
four
four
three
is
also
always
open.
So
especially
if
you're
doing
a
hybrid,
called
multi
Club
thing
443
is
like
less
likely
to
run
into
just
weird
security
issues
that
you
might
pop
up
or
weird
firewall
rules.
A
A
Some
memory
requests,
we
give
it
our
TLS
certificates,
here's
that
field
graph
I
talked
about
before
and
then
here's
how
you
reference
that
field
raft
by
deploying
applying
the
configuration
directly
in
the
pods
back
so
kubernetes
will
actually
interpolate
that
for
us
and
we
don't
need
you
to
use
helm
or
another
like
template
provider
for
this
GCS
tcp
listeners
and
again,
I'm
also
listening
on
localhost
just
for
debug
ability
and
then
at
the
very
bottom.
Here
we
have
our
readiness
probe
and
we
mount
our
TLS
cluster.
This
is
all
open
source.
A
B
A
Well,
we'll
move
on
so
hopefully
that
will
be
done
when
we
get
back,
but
the
next
section
is
all
about
connecting
develop
from
kubernetes.
So
hopefully,
when
I
come
back,
that
we'll
be
running,
and
let's
imagine
now
like,
let's
completely
forget
that
volt
is
running
under
kubernetes
like
it
could
be
running
on
Heroku
or
on
premise
like
it's
just
a
thing
in
an
IP
address
again.
This
is
why
the
load,
balancer
and
dedicated
cluster
is
so
important.
So
forget
everything
again!
A
Imagine
that
it's
like
a
SAS,
that's
provided
to
you
and
the
kubernetes
is
just
an
implementation.
Detail
and
I
really
think
that
this
is
the
best
way
that
organizations
should
think
about
running
vaults,
don't
expose
it
through
kubernetes,
don't
use
kubernetes,
specific
things,
leverage,
kubernetes
functionality
to
make
vault
more
available
and
more
resilient,
but
it's
just
a
thing
in
an
IP
address.
A
So
let's
talk
about
this
authentication
thing.
So
in
order
for
humans
or
applications
and/or
to
communicate
with
vault,
they
need
this
thing
called
a
vault
token.
You
can
think
of
it
like
a
session
ID
or
a
cookie
for
a
website.
So
just
like
on
a
website,
you
give
a
username
and
a
password,
and
that
gives
you
a
cookie,
a
session
ID
which
is
stored
in
a
cookie
in
your
browser.
We
need
to
do
the
same
thing
to
vault,
so
I'm
gonna
provide
vault.
A
Some
information
might
be
a
user
name
and
password
vaults,
gonna
verify
that
with
a
third
party
system
might
be
Active
Directory.
Assuming
that's
successful.
We're
gonna
do
kind
of
a
lookup
and
the
database
to
see
what
permissions
I
have
am.
I
have
an
admin
and
my
general
user
and
then
I'm
gonna
get
back
a
token
that
has
all
of
those
policies
map
to
me
in
the
case
of
kubernetes.
We
actually
use
the
pod
service
account
specifically
the
jot
token,
the
JWT
token
for
that
service
account
and
the
token
reviewer
API.
A
So
we
configure
vault
to
have
permissions
to
talk
to
the
token
reviewer
API,
a
pod
launches
grabs
its
service
account
from
the
filesystem,
usually
post.
That
service
account
Jah
token
to
vault.
Vault
verifies
the
identity
of
that
pod,
as
well
as
its
namespace
in
the
service
account
with
the
token
a
reviewer
API,
assuming
that's
successful.
That
is
then
mapped
to
policy.
A
vault
token
is
then
returned
to
the
service
account
to
the
pod
that
requested
it,
and
they,
the
pod,
uses
that
token
for
further
requests.
A
The
same
way
that
you
would
use
like
a
session
idea
or
a
cookie
to
make
further
requests
through
the
browser.
Okay
demo,
one
didn't
even
finish
in
my
slides,
tell
me:
I
have
to
do
demo,
two
unable
to
find
server.
Can
you
all?
Do
me
a
favor
and
make
sure
you're
not
on
the
Wi-Fi,
pulling
like
a
Steve
Jobs
here,
I
need
everyone
off
the
Wi-Fi.
B
A
A
B
B
B
B
A
Okay
and
they're
ready
now
so
now.
What
I
need
to
do
is
in
three
minutes.
Do
the
rest
of
my
demo
all
right,
so
we're
gonna
initialize
this
thing
so
make
sure
that
we
can
talk
to
it
will
run
volt
status
and
it'll,
come
back
and
say:
hey,
it's
not
initialized
and
it's
currently
unsealed.
So
what
we'll
do
is
we'll
run
vault
operator
in
it,
and
this
will
give
me
my
initial
root
token
and
five
on
the
steal,
keys,
I'm,
not
gonna
dive
into
exactly
what
those
mean
but
I'm
happy
to
talk
about
them.
A
Export
that,
as
my
volt
token
and
source
that
again
and
now,
when
I
run
vault
status,
we
should
see
that
it's
initialized
and
unsealed,
so
initialized
is
true
and
sealed
is
false.
Thanks
for
double
negatives.
So
what
I'm
gonna
do
now
is
I'm
going
to
put
a
secret
in
I'm
just
going
to
call
the
secret
cube
Khan,
and
the
message
is
hello.
A
A
And
there's
nothing
in
here,
hopefully
there's
nothing
in
here
and
be
weird.
Alright,
so
there's
nothing
in
this
cluster
and
what
I'm
going
to
do
is
I'm
gonna
use
this
fancy
script,
that
I
wrote
because
I
suck
at
typing
and
I'm
just
gonna
walk
you
through
it
very
quickly.
So
the
first
thing
I'm
gonna
do
is
I'm,
creating
a
service
account
and
then
I'm,
creating
an
AR
back
binding
for
that
service
account
to
be
able
to
talk
to
the
token
reviewer
API.
A
Here's
the
important
part
we
give
vault
the
kubernetes
host
the
IP
address
to
our
apps
cluster,
the
CA
cert,
to
communicate
with
that
cluster
and
the
jot
token
for
the
service
account
which
can
actually
communicate
with
it,
took
under
viewer,
API
vault.
We'll
use
those
three
pieces
of
information
to
validate
any
pod
service
account
that
it's
given
against
the
token
reviewer
API
next
we're
just
creating
a
simple
policy
in
volt
that
lets
the
secret
read-only
policy,
the
ability
to
read
from
secret,
and
then
we
create
a
role
in
vault
where
we
were
in
this
example.
A
I'm
just
binding
to
the
default
servers
account
in
the
default
namespace,
but
you
can
create
multiple
roles
and
create
multiple
mappings
here
and
then
anything
that
matches
this
policy.
This
role
will
get
the
policy
secret,
read-only
and
I'm
also
up
letting
our
TLS
secret
for
real
this
time,
and
then
I'm
I
have
to
give
the
vault
address
to
these
pods
because
they
don't
actually
know
revolt
lives,
because
it's
just
a
thing
in
an
IP
address.
So
I'm
gonna
run
this
and
hope
that
the
bandwidth
holds
out.
A
A
B
A
B
B
B
A
A
You're
welcome,
I,
put
all
of
that
functionality
into
a
pod
that
you
can
put
in
an
init
container
that
automatically
pulls
that
service
account
and
drops
it
in
an
in-memory
volume.
And
then
all
you
have
is
that
volt
tokens
you
don't
actually
have
to
do
all
of
the
stuff.
I
just
did
I
vomited
it
for
you,
but
I
wanted
to
show
you
the
hard
way.
The
easy
way
is
to
just
use
this
container,
it's
open
source,
so
you
can
audit
it.
You
can
build
it
yourself.
A
If
you
don't
trust
me,
I,
wouldn't
another
tool
that
I
want
to
mention.
Is
this
tool
called
console
template
console
template
will
actually
help.
You
extract
multiple
secrets
from
vult.
Maybe
you
have
a
TLS
key
and
a
couple
database
passwords,
and
you
have
some
legacy
java
application
that
just
really
wants
to
consume
a
pom.xml
file
or
a
properties
file.
You
can
use
console
template
to
render
out
that
properties
file,
meaning
your
app
doesn't
have
to
be
vault
aware
and
then
console
template
can
actually
communicate
with
your
application
when
those
secrets
change.
A
So
it's
just
writing
out
a
file
in
a
shared
volume
that
your
app
is
consuming,
which
always
brings
up
a
big
question
which
is
like
how
do
I
know
if
my
credentials
change?
How
will
my
app
know
to
reread
its
config
there's
a
Kol
beta
feature
called
shared
process
namespaces
in
kubernetes
110.
If
you
enable
this,
this
will
allow
multiple
pods
to
communicate
with
one
another.
So
they're
not
all
running
as
pin
1
they're,
actually
sharing
a
process.
A
Namespace
you
give
P
trace,
which
allows
you
to
call
pitov
and
then
basically
console
template
is
sharing
a
process
namespace.
So
it
has
the
ability
to
signal
your
application
when
it
has
updated
that
configuration
file
and
then
your
app
can
be
restarted,
respond
to
a
signal
whole-heartedly
blow
up
whatever
your
choice
is:
whenever
that
config
file
changes,
here's
more
information!
Thank
you
all.
So
much
sorry,
I
went
overtime,
I'll
be
around
for
questions.