►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Scaling SPIRE for Performance and Availability - Tyler Julian, Uber
SPIRE, the community-supported implementation of SPIFFE, enables users to take advantage of workload identity primitives like X.509s and JWTs without needing a deep understanding of complex topics like trust bootstrap, secure introduction, and credential provisioning/rotation. But implementing the SPIFFE standard is not without its difficulties. SPIRE must scale to meet the needs of hundreds of thousands of workloads in today's hybrid cloud architectures. And, despite a requirement for high, efficient throughout, the system must remain resilient in the face of failure. In this deep dive, we will study the challenges encountered during the implementation of SPIRE, design considerations and philosophy, and production use cases.
https://sched.co/VGpg
A
A
I
thought
it
might
begin
with
a
discussion
of
trust.
What
that
word
means
the
word:
fidelity
in
English
comes
from
the
Latin
root
fides,
the
Roman
goddess
of
truth
and
honesty,
and
to
trust
someone
in
ancient
Roman
times
was
to
say
a
prayer
to
the
goddess
Thetis
to
to
bless
you
with
good
faith
and
to
to
watch
over
your
trust.
A
Now
this
hasn't
really
stood
the
test
of
time
and
today,
when
we
think
about
trust,
we
need
a
better
definition,
one
that
I
quite
like
goes
like
this
Trust
is
meant
to
convey
an
expectation
of
behavior.
However,
predictable
behavior
does
not
necessarily
constitute
behavior,
that's
worthy
of
trust.
For
example,
we
expect
a
bank
will
behave
like
a
bank,
and
we
expect
a
thief
will
behave
like
a
thief,
trust
as
expectation
of
behavior.
Look
at
a
practical
example.
A
Imagine
that
you
and
I
were
developing
a
service
to
deliver
fresh
produce
to
people
right,
and
so
we
would
start
with
a
front-end
service
that
serves
orders
and
say
we
have
Alice
come
and
she
wants
to
take
a
look
at
some
of
her
prior
orders.
Well,
we
could
trust
that
Alex
would
only
look.
Alice
would
only
look
at
her
data.
We
would
have
this
expectation,
but,
as
we
know,
that's
not
a
very
safe
expectation.
We
can't
really
trust
alice
in
this
regard.
So
what
do
we
do?
A
We
set
up
an
identity
provider
to
provide
identity
to
Alex
Alice,
some
kind
of
cryptographic-
token
right,
a
set
of
a
policy
engine
that
defy
what
Alice
can
actually
access.
Alice
can
look
at
her
data.
She
can't
look
at
Bob's
data
as
we've
taken
this
expectation
of
behavior
and
we've
turned
it
into
an
enforcement
of
behavior.
This
is
what
it
means
to
be
zero
trust
to
be
trustless,
to
remove
this
expectation
of
behavior,
but
we
haven't
removed
trust
completely,
we've
sort
of
just
displaced
it
we've
taken
the
trust
that
we
had
an
Allison.
A
We've
moved
it
to
these
components:
the
identity
provider
in
the
policy
engine.
But
why
don't?
We
trust
those
components?
Where
does
that
trust
come
from?
Well,
we
had
a
developer
set
up
the
identity
provider.
We
had
a
developer
set
up
the
policy
engine
and
the
the
front-end
that
glues
these
pieces
together
right
takes
that
identity
does
authentication,
takes
that
policy
and
does
authorization.
A
So,
ultimately,
the
trust
is
coming
from
our
organization,
and
you
start
to
look
at
this,
and
this
whole
idea
of
trustless
is
kind
of
just
an
exercise
in
us
learning
to
trust
ourselves.
But
that's
not
the
full
story
right,
because
Trust
has
to
come
from
somewhere
and
if
we
we
talked
about
earlier
trust
as
an
expectation
of
behavior.
Where
is
the
expectation
come
from
us
as
developers
and
who
sets
that
expectation?
A
So
when
Alice
or
Bob
good
old
Bob
hit
with
it
when
they
give
us
their
data
when
they
they
set
up
an
order
with
us,
they're
handing
us
their
data
and
they're
saying
protect
this
secure,
my
data
keep
it
safe,
that's
the
expectation
on
us
as
developers,
that's
where
the
trust
comes
from.
So
we
talked
a
lot
about
building
zero
trust
networks
and
we
think
that
we're
removing
trust
from
the
equation,
but
we're
not
we're
really
building
trust.
Okay,
and
so
how
do
we
get
there?
A
I
want
to
talk
about
spiffy
spiffy
is
the
secure
production
identity
framework
for
everyone,
and
it's
a
specification
that
talks
about
workload
identity
when
we
think
about
workloads,
we're
not
talking
about
Alice,
and
this
front
end
we're
talking
beyond
the
front
end
to
this
myriad
of
back-end
services
and
how
those
communicate
with
each
other.
We
want
to
secure
that
workload,
identity.
It
can
be
an
x.509
certificate,
it
can
be
a
JSON
web
token
for
the
purpose
of
this
talk,
I'm
really
just
gonna
be
talking
about
x.509,
but
there's
sort
of
interchangeable.
A
In
some
sense,
spiffy
introduces
these
trusted
components.
The
idea
of
a
trust
domain
and
a
trusted
authority
at
the
root
of
that
domain,
and
if
you
have
workloads
that
are
kind
of
loosely
coupled
inside
the
stress
domain
in
organization
a
and
they
want
to
talk
with
workloads
and
organized
organization,
be
spiffy
introduces
this
concept
of
Federation
to
solve
this
problem.
That's
been
traditionally
and
historically
very
hard
to
solve.
So
a
specification
is
nice
and
implementation
is
nicer.
That's
where
spire
comes
into
play,
spires,
the
spiffy,
runtime
environment,
first
and
foremost.
A
What
spires
trying
to
do
is
solve
this
problem
of
bootstrap
of
trust.
We
saw
in
the
in
the
earlier
slide.
Trust
comes
from
your
organization
and
you
want
to
allow
that
to
permeate
into
all
the
components
in
your
system.
So
what
spire
does
is
it
allows
that
trust
to
permeate
into
your
infrastructure?
It
needs
to
be
portable
needs
to
be
cloud
native.
It
needs
to
run
on
the
public
cloud.
Private
cloud
bring
your
own
hardware
anything
and
we'll
talk
about.
What
that
looks
like
in
a
minute
is
to
be
reliable.
A
Performant,
observable,
identity
and
authentication
to
me
needs
to
be
built
into
the
very
foundation
of
your
infrastructure.
It's
a
it's
a
core
component
like
like
routing
or
persistent
storage,
because
of
that
needs
to
be
really
really
available
and
when
something
goes
wrong,
we
need
to
know
about
it.
That's
the
the
essence
of
observability.
We
need
to
have
the
tools
to
fix
those
problem
at
problems
in
recover
effectively.
A
So
what
I'm
gonna
talk
about
today
is
a
little
bit
of
a
high
level
architecture
of
spire
in
case
you're,
unfamiliar
the
essence
of
short-lived
credentials,
which
sort
of
formed
the
base
level
of
security
for
all
of
spire,
and
then
the
various
components,
the
agent
the
server
and
what
registrations
are
all
about.
So,
for
the
purposes
of
this
diagram
we
can
assume
we
have
a
multi-tenant
system.
We
have
a
host
call
it.
Let's
call
it
host
X
with
workloads,
a
B
and
C,
and
so
these
workloads
are
heterogeneous.
They're.
A
So
ultimately,
spire
server
is
the
root
of
trust
in
our
system.
And
it's
what
allows
delegation
of
trust
into
these
agents
so
spire.
It
needs
to
serve
some
part
or
some
policy
information.
It
does
that
in
a
database
it
needs
to
get
policy
information
and
it
does
that
through
this
component
called
a
registrant,
and
that
thing
writes
the
this
policy
information
and,
if
you're
not
familiar
as
you
can
see,
workload
a
we
want
to
run
on
host
X,
but
we
don't
want
workload
a
to
run
on
host.
A
Why
we
need
this
policy
this
this
thing
that
tells
us
exactly
where
a
workload
can
run,
because
you
can
imagine
if
you
have
a
secure
group
of
hosts
and
you
want
to
run
secure
workloads
on
that
host.
You
need
to
be
very
specific
about
what
should
run
where
and
so
this
component.
This
registrant
is
very
coupled
to
your
scheduler.
A
Ok,
so
before
I
dig
into
these
components
a
little
bit,
it's
important
to
get
a
little
bit
of
background
on
short-lived
credentials
that
we
use
in
spire.
The
big
question
everyone
has
is
why
you
use
short-lived
credentials.
Why
don't
you
have
a
good
revocation
strategy
and
spoiler
alert?
Revocation
is
a
very
hard
problem.
There's
a
little
bit
of
background
on
revocation
strategies
that
have
been
used
historically,
there's
this
thing
called
a
crl
certificate.
Revocation
list
I
receive
a
certificate
and
I
want
to
know.
Is
it
valid,
I
consult
with
my
CRL
have
some
problems.
A
The
list
doesn't
really
scale
that
well
there's
problems
with
stale
data
implementation.
Difficulty
of
this
thing,
so
there
were
some
improvements.
The
online
certificate
status
protocol,
instead
of
consulting
with
a
list
I,
actually
talked
to
a
service.
The
OCSP,
responder
and
I,
say:
hey?
Is
this
x.509
certificate
still
good?
This
has
a
lot
of
the
same
problems.
What
do
you
do
if
that
servers
down
the
implementations
outside
of
browser
software?
A
The
imitations
are
actually
not
that
great,
and
so,
when
we're
thinking
about
using
revocation
in
back-end
services,
we
don't
have
the
same
of
implementations
from
browsers.
We
have
you
know
x.509
implementations
in
in
every
language,
Under
the
Sun,
and
we
to
think
about
that.
That's
why
there
is
a
third
method
of
revocation
that
is
sort
of
implicit
revocation.
Every
certificate
has
a
finite
lifetime
a
beginning
date
and
an
end
date.
When
you
pass
that
end
date,
the
certificate
expires
and,
what's
really
nice
about
this
approach
is
when
I
receive
an
x.509
certificate.
A
All
I
need
really
I,
don't
need
a
list
or
a
service.
The
only
thing
I
really
need
is
a
somewhat
accurate
clock,
that's
a
powerful
construct,
but
with
the
old
methods
of
revocation.
You
might
have
these
certificates
that
last
years
or
months,
and
we
need
to
rethink
that
if
we
want
to
be
able
to
revoke
quickly-
and
this
is
the
essence
of
what
it
means
to
have
short-lived
credentials.
A
These
are
credentials
that
only
you
know,
instead
of
years
or
months,
it's
weeks
days,
hours,
okay
and
because
they're
turning
over
so
quickly,
we
have
this
implicit
revocation,
that's
always
happening,
and
so
further,
if
you
look
at
x.509
implementations
across
the
board,
this
is
not
hard
to
get
right.
You're
just
checking.
You
know
against
a
a
specific
point
in
time
and
so
revocation
becomes
really
really
reliable,
but
one
of
the
problems
is
that
rotations
still
hard.
A
So
if
you've
ever
used
a
server
that
implements
TLS
you've
seen
configuration
that
looks
like
this,
you
have
to
provide
a
certificate
and
a
key
from
a
file,
and
when
you
have
long-lived
certificates,
this
is
kind
of
fine
right.
You
know
once
a
year
you
go
in
and
you
update
this
file,
it's
not
the
greatest
solution,
but
you
know
you
don't
have
to
do
it
that
often
so,
why
do
you
care
about
automating
it?
When
you
have
short-lived
credentials?
A
A
Instead
of
reading
your
credentials
from
a
file,
you
make
a
G
RPC
call
to
a
local
agent
and
that
agent
gives
you
credentials
and
uses
a
push
model
to
continually
update
you
every
time
that
that
credential
is
rotated,
and
so
this
this
is
part
of
what
I'm
talking
about
in
terms
of
solving
rotation
as
a
first-class
citizen,
but
a
file
read
is
so
reliable
right,
I
mean
if
the
file
is
there,
you
get
it
and
we
want
this
API
to
be
just
as
reliable,
so
file
reads.
So
how
do
we
achieve
that
aspire?
A
Agent
does
is
optimistic,
credential
fetching
we
talked
earlier
about
how
workloads
need
to
be
in
a
way
assigned
to
certain
hosts,
and
so
spire
server
knows
this
policy.
It
knows
every
workload
that
could
conceivably
run
on
a
host,
and
so
the
agent
does
this
optimistic
fetching
and
by
the
time
the
workload
starts
on
that
host.
It
already
has
that
thing
cached
locally,
and
so
the
workload
hits
this
workload
API
and
the
response
is
very
fast.
A
A
Another
cool
kind
of
consequence
of
this
is
that
you've
actually
decoupled
the
traffic
between
the
workload
and
the
agent
and
the
agent
and
the
server,
so
you
can
scale
them
independently
all
right,
so
you
can
have
a
hundred
calls
from
workloads
to
the
agent
and
then
it
doesn't
at
all
impact.
The
agents
upstream
calls
to
the
server
the
request
cycles
are
disjoint,
and
so,
as
the
number
of
workloads
on
a
single
host
scales
up
it
actually
doesn't
the
server
doesn't
feel
that
load
at
all.
A
It
says
this
is
really
important
for
a
capacity
planning,
as
you
start
to
scale
up
the
number
of
workloads
in
your
fleet
so
doing
this
kind
of
caching.
Obviously
you
run
into
the
the
age-old
problem
of
consistency
versus
availability.
When
the
agent
can
no
longer
contact
the
server.
What
do
you
do?
Do
you
serve
state,
potentially
stale
data,
or
do
you
sacrifice
being
available
again?
We
talked
about
this,
we
want
to
be
very,
very
available.
A
We
want
snappy
responses,
we
have
to
decide
to
serve
stale
data
and
because
the
the
nature
of
these
x.509
they're-
not
rotating
super,
often
so
this
is.
This-
is
actually
not
very
problematic
unless
the
length
of
unavailability
gets
to
such
a
point
that
your
certificates
expire
and
so
we'll
talk
about
some
strategies
that
we
use
to
address
that
one.
A
A
A
So
some
cool
knobs
that
we
can
turn
turn
there
as
users
to
kind
of
address
that
scaling
problem
and
beyond
that.
The
server
is
managing
this
agent
and
workload
policy,
the
which
work
loads
can
run
on
and
the
way
that
happens
is
through
this
process
called
registration
so
backing
up
a
bit.
We
talked
earlier
about
how,
in
kubernetes,
you
have
a
scheduler
cause,
a
workload
to
to
come
out
to
boot
right
and
so
somehow
spire
needs
to
be
made
aware
of
this
placement
decision.
So
you
need
some
kind
of
integration
to
allow
this
to
happen.
A
So
we
have
this
thing
in
the
open
source
called
the
kubernetes
registrar.
That
actually
does
this
kind
of
integration.
So
we
get
a
really
tight
integration
with
kubernetes
out
of
the
box,
but
one
of
the
problems
here
is,
as
you
might
notice,
is
in
order
for
a
workload
to
get
its
identity.
It
first
needs
to
be
registered
with
spire.
A
This
process
can
be
kind
of
racy
in
terms
of
you
know,
if
the
workload
coming
up
before
the
registration
actually
propagates
through
the
system,
we
got
to
be
aware
of
that
thing,
but
not
only
that
the
registrant,
the
registrant,
is
now
a
bottleneck
for
workload.
Boot,
that's
a
little
bit
of
a
problem
from
a
reliability
perspective,
because
now
you
have
this
extra
component
that
that
needs
to
always
be
up
in
order
for
we're
close
to
launch
so
want
to
talk
a
little
bit
about
the
thought
process.
A
A
That
means,
when
you
launch
workload
a
you
need
to
wait
for
that
registration
to
propagate
through
your
system
and
beyond
that
registrations
are
going
to
scale
with
the
number
of
workload
instances
that
you
have.
So
if
you
have
a
kind
of
a
disaster
event
where
you
need
to
reboot
all
of
your
workloads,
you
might
run
into
a
bottleneck
here,
and
so
this
idea
of
per
cluster
registrations
has
sort
of
come
about,
which
is
that
you
register
a
workload
against
an
alias
of
clusters,
and
then
you
only
need
to
do
one
registration.
A
This
is
really
nice,
because,
once
that
cluster
registration
happens,
you
know
you're
not
really
blocked
on
on
this
registrant
being
up,
so
it
gives
you
kind
of
a
two-pronged
benefit.
The
the
main
kind
of
difference
between
these
two
approaches
in
terms
of
security
is,
is
really
about
limiting
blast
radius.
If
you
only
have
workloads
they're
allowed
to
run
on
one
host
and
that
host
is
compromised,
it's
just
the
credentials
on
that
host
being
compromised.
A
But
if
you
have
you
know
many
many
workloads
running
on
a
little
bit
of
a
larger
cluster
and
one
of
those
hosts
is
compromised
kind
of
in
a
way.
All
of
that
entire
cluster
is
compromised.
So
you
really
got
to
think
about
what
do
you
need
from
a
business
perspective?
Is
it
okay?
What's
what's
your
threat
model?
If
it's
okay
to
to
think
about
these
concerns,
maybe
in
some
cases
you
really
do
need
that
really
tightness
of
policy,
and
sometimes
maybe
you
don't
so.
A
This
is
a
really
important
knob
that
spire
allows
its
users
to
tune
and
because
it's
an
important
decision-
and
we
can't
necessarily
be
opinionated
about
how
people
do
registrations.
One
thing
I'll
allude
to
earlier
was
this
idea
of
a
certificates
TTL
time
to
live
it's
their
lifetime,
the
distance
between
the
beginning
and
end
time
on
the
certificate.
A
So
one
thing
to
note
is
that
a
low
TTL
means
that
you're
revoking
very
quickly.
You
can
imagine
if
you
have
a
certificate
with
a
one
hour
time
to
live,
and
you
accidentally
log
that,
as
part
of
your
logging
infrastructure,
it's
kind
of
okay
in
a
way,
because
within
an
hour
that
certificates
no
longer
useful.
But
if
your
TTL
on
your
certificate
is
a
day
and
you're
accidentally
log
it
you
know.
A
Now
you
have
to
wait
a
full
day
while
that
certificate
is
potentially
out
there
being
used,
and
so
a
lower
TTL
means
these
things
are
constantly
turning
over
and
constantly
being
revoked
and
it
in
a
way
reduces
the
risk
of
compromise,
but
there's
a
but
an
hour.
Ttl
means
you
have
a
one
hour
maximum
time
to
recovery.
A
If
your
control
plane,
if
your
spire
server
goes
down
and
it
can
no
longer
rotate
these
certificates,
a
clock
starts
ticking
and
you
have
about
an
hour
to
fix
that
outage
before
kind
of
the
whole
system
in
a
way
breaks
and
you
have
to
reboot
strap
all
of
your
trusts.
It's
it's
not
a
good
place
to
be
in
so
we
sort
of
have
to
balance
this.
You
know
if
you
have
24
hours
to
fix
an
outage.
A
A
So
a
couple
of
takeaways
here,
because
I
do
want
to
leave
a
good
amount
of
time
for
questions.
Caching
is
kind
of
the
secret
sauce
of
distributed
systems,
but
you
got
to
use
it
tastefully.
You
can't
you
can't
overdo
it.
I
think
horizontally,
I
think
we're
starting
to
see
some
users
in
the
community
as
they've
started
to
scale
up
these
things,
we're
starting
to
see
some
of
the
pain
points
of
having
these,
like
multi
thousand
node
deployments
in
terms
of
how
the
server
reacts,
that
kind
of
load
some
trade-offs.
A
A
And
finally,
you
know
if
you're
interested
in
doing
this
kind
of
thing,
you
know
come
set
up.
A
POC
join
us
on
slack
I'm,
always
on
slack
super
available
so
happy
to
answer
any
questions.
I'd
be
remiss
if
I
also
didn't
mention
that
my
team
of
you
retiring.
So,
if
you're
interested
in
working
on
this
stuff
come
and
talk
to
me,
and
thank
you
very
much.
A
A
So
this
is
one
of
the
things
that
eluded
to
a
little
bit
earlier
on.
Is
that
OCSP,
while
it
is
used
actually
pretty
prevalently
in
web
browsers,
the
implementations
on
the
back
end
server
side
are
actually
kind
of
fuzzy
at
best
it's
hard
to
get
right,
and
so
a
lot
of
implementations
don't
do
it
correctly,
and
the
other
problem
is
that
when
the
OCSP
responder
is
down
a
lot
of
these
implementations
fail
open
and
they
say
oh
I
couldn't
get
the
couldn't
get
the
thing.
A
I
guess
certificate
looks
okay
to
me
and
a
lot
of
times.
That's
the
exact
time
that
you
need
that
you
know,
and
so
yeah
it's
it's
I
think
it
was
probably
considered
in
in
the
early
days
in
the
spiffy
spec.
But
ultimately,
the
sort
of
implicit
revocation
of
short-lived
certificates
is
the
easiest
to
get
right.
A
Yeah,
so
the
response
here
was
that
OCSP
stapling,
which,
instead
of
talking
to
an
OC
SB
responder
in
line
with
with
your
validation,
you
actually
have
your
server
who's,
presenting
the
certificate,
talked
to
the
OCSP
server
and
and
grab
a
response,
a
signed
response
from
that
OCSP
responder
and
then
send
it
in
line
with
the
certificate.
This
has
a
couple
other
problems
in
the
implementation.
What
do
you
do
if
that
stapled
response?
Isn't
there
again
a
lot
of
times
they
fail
open?
A
Sometimes
it's
not
even
implemented
at
all,
correctly
in
certain
languages
and
again
on
the
server
side,
you
have
to
decide
what
to
do
if
you
can't
get
assigned
OCSP
response
to
staple.
So
there's
a
lot
of
failure
modes
there,
and
it's
just
this
idea
of
adding
that
extra
component
into
into
your
trust
model,
necessarily
complicates
the
reliability
story
down
in
front
here.
B
Hi
I
was
reading
the
installation
instructions
this
morning
and
saw
you
mentioned
the
example
of
getting
an
AWS
say
the
identity
document
from
the
metadata
server.
I
hope
you
can
maybe
explain
the
threat
model
more.
Does
it
matter
if,
if
any
other
say
pod
or
program
running
on
that
machine
could
also
access
the
identity
document?
Does
it
matter
who.
A
It's
a
great
question:
that's
a
great
question,
so
the
thing
about
AWS
identity
documents
and
same
with
GCP
is
that
you
retrieve
that
document
by
essentially
doing
a
curl
in
HTTP
request
to
the
local
system
in
the
cloud,
and
so
potentially
anything
with
access
to
the
host
network
can
retrieve
this
instance.
Identity
document
that
you're
trading
in
for
this
very
trusted,
credential,
and
so
the
model
with
spire,
is
a
trust
on
first
use
for
the
agent,
and
so
we
assume
that
the
agents
like
are
kind
of
the
first
thing
coming
up
in
your
system.
D
Trust
on
first
use
is
kind
of
like
the
primary
mitigation
for
the
concern.
You're
talking
about
I
think
the
the
flipside
is,
of
course,
it's
a
race
and
of
course
it
can
be
lost,
and
in
that
event,
because
it's
trust
on
first
use
that
the
legitimate
agent
will
fail
to
come
up
and
you
can
page
on
that
kind
of
a
thing
so
at
the
very
least,
is
certainly
detectable.
That's
that's
that's
about.
D
As
far
as
we
can
go
from
like
the
spire
implementation
and
mitigating
that
that
kind
of
a
problem
there
there,
of
course
other
mitigations
you
can
put
in
place
as
a
user
like
iptables
rules,
and
things
like
this
that
can
like
limit
access
to
those
but
like
from
the
project's
perspective.
We
can't
really
expect
that
all
users
will
do
that,
and
so
we
take
as
many
steps
as
possible
to
kind
of
mitigate
this
replay.
This
show.
D
So
the
question
was:
does
a
survivor
reboot
it
can?
The
agent
can
be
configured
to
persist
it
like
the
result
of
that
whole
process
of
validating
this
identity
document
is
and
next
I
have
an
identity
issued
to
the
agent.
So
the
agent
can
be
configured
to
persist,
that
key
and
that
certificate
and
RAM,
or
on
desk
or
against
the
TPM,
or
something
like
this,
if
need
be
so
it
can
be,
it
can
survive
reboot
if
configured
appropriately.
E
Yeah,
this
is
maybe
a
poll
to
you,
but
also
the
rest
of
the
audience
in
the
wild.
What's
the
kind
of
bare
minimum
that
you've
seen
for
that
certificate
expiry
time,
if
you're
really
interested
in
keeping
that
down
to
a
tight
loop,
you've
got
very,
very
you've
got
a
network,
you
don't
trust
at
all,
and
you
want
to
keep
that
as
small
as
possible.
What's
what's
the
smallest
realistic
window,
you've
seen
work.
A
A
F
But
certainly
there
are
implications
that
Tyler
alluded
to,
for
example
like
that
that
becomes
like
the
minimum
time
that
you
can
withstand
an
agent
being
down,
for
example,
if
it
can't
get
that
rotation,
then
it
it
has
to
figure
out
how
to
like
reboot,
strap
trust
into
the
system.
Inspire
itself
like
we
have
integration
tests
that
run
with
like
really
low
TTLs.
F
You
were
talking
like
15
seconds
or
10
seconds
you
know
and
that
like
spire,
can
handle
that
low
of
a
TTL
but
yeah
your
your
room
for
failure,
your
tolerance
for
downtime,
it's
really
low
at
that
point,
so
yeah
I
think
it
had.
You
want
to
go
with
it
as
high
as
a
TTL
is
reasonable
in
terms
of
like
downtime
tolerance,
but
you
know
as
low
as
you
as
you
can
go
in
terms
of
like
revocation,
you
know
mitigation,
so.
G
A
A
Okay,
yeah
so
I'm
guessing
using
some
kind
of
like
a
note,
selector
pattern
here
where
you're
registering
against
like
a
cluster
or
a
group
of
hosts,
and
so
as
those
hosts
are
sort
of
Auto
scaled
to
come
up.
You
actually
have
this
registration
against
that
group
of
foes.
So
as
long
as
the
the
host
is
coming
up
inside
this
group
that
you've
already
created
a
registration
for
you
actually
don't
really
need
any
kind
of
interaction
with
the
spire
from
a
registration
perspective,
because
the
spire
agent
comes
up
on
that
server.
It
kind
of
queries.
A
B
Not
so
much
a
conceptual
question
is
implementation,
but
I
saw
also
that
workload
is
intended
to
communicate
with
the
agents
over
domain
sockets
mounting
a
host
path.
It's
common
to
have
pod
security
policies
that
won't
allow
mounting
of
hosts
paths.
So
do
you
recommend
white
listing
that
path
or
what
is
there
an
alternate
choice
for
the
workload
talking
to
the
agent.
D
Yeah
the
host
path-
we've
heard
this
a
little
bit
now
and
the
host
path
is
kind
of
the
route
that
most
people
are
taking,
there's
not
really
a
good
alternative.
At
the
moment,
we've
discussed
the
possibility
of
opening
like
a
management
API
on
the
agent
which
would
allow
you
to
build
like
a
flex
flow
and
plugin,
or
some
similar
integration
that
can
poke
the
agent
and
say
hey
I'm.
Turning
this
thing
on
create
a
new
socket
over
here.
D
H
A
Good
question
so
when
you're,
when
you're
kind
of
like
upstream
CA
or
your
spire
CA,
so
this
speaks
to
a
little
bit
to
the
strategy
of
what
is
your
root
of
trust
in
spire.
There's
a
couple
of
different
strategies:
you
can
have
spire
create
its
own
key
self
sign
it.
You
can
also
refer
to
another
spire
server
and
which
is
called
nested
spire
to
allow
these
things
to
sort
of
form
a
tree,
but
you
could
also
just
use
a
key
pair
that
you
provision
yourself,
and
so
what
do
you
do?
If
that
thing
is
compromised?
A
It's
pretty
difficult
right
now,
I
think
we're
planning
to
spend
a
lot
of
time
on
revocation
coming
up
on
the
middle
of
next
year
of
2020.
There
are
some
strategies
right
now.
You
essentially,
you
know,
have
the
new
key
to
your
trust,
bundle
and
then
rotate
out
the
old
one,
but
yeah
rotating
a
root
key
is
much
more
difficult
than
rotating
the
work
load
keys.
That
sort
of
have
this
thing,
this
rotation
mechanism
built
in
I.
D
So
there's
not
there's
not
so.
First
of
all
the
workload
API
it
does
have.
Support
for
delivering
an
x.509
crl
to
a
workload
aspire
has
not
implemented
implemented,
x509
crl,
but
the
workload
api
has
support
for
it.
This
Divi,
Federation
API,
is
that
we've
currently
defined
don't
really
have
any
concept
of
revocation
like
to
communicating
revocation
information
across
that
wire
right
now.
The
the
you
know,
the
implication
of
a
compromise
is
usually
hey.
I'm
gonna
yank,
this
root
from
the
bundle
inspire
propagates
routes
as
well.
D
We
can
yank
a
root
from
the
bundle
we
can
propagate
that
trains
pretty
rapidly
and
bring
a
new
key
in
I.
Think
that
there's
probably
a
similar
story
with
Federation,
where
you
would
expect
the
foreign
trust
domain,
which
is
compromised
to
nuke,
whatever
keys
out
of
their
bundle,
were
affected
or
chained
up
to
chained
down
to
you
know
the
affected
compromised
and
that
would
come
across
that
change
will
come
across
pretty
quickly
but
from
from
a
you
know,
from
a
consumer
point
of
view,
I
always
have
the
option
to
sever
that
Federation
relationship.