►
Description
Kubernetes Storage Special-Interest-Group (SIG) Object Bucket API Design Meeting - 15 April 2021
Meeting Notes/Agenda: -
Find out more about the Storage SIG here: https://github.com/kubernetes/community/tree/master/sig-storage
A
So
today
we
we
should
continue
the
conversation
on
credential
rotation,
so
I
went
back
and
looked
at
some
of
the
some
of
the
topics
we've
been
discussing
about,
and
the
first
thing
I
found
was
the
kms
based
credential
rotation
is
only
for
encryption
keys
and
not
for
access
keys,
so
that
wouldn't
apply
to
creation
of,
or
rotation
of
access
keys
like
we
originally
thought.
A
A
What
it
is
is
you
create
a
token
with
a
predefined
expiry
and
a
a
small
set
of
or
a
very
constrained
access
on
it
or
access
provision
for
it
or
given
to
it,
and
that
token
can
be
used
to
assume
the
role
of
a
certain
user
and
make
requests
on
behalf
of
the
behalf
of
that
user.
A
So
I
believe,
as
of
right
now
that
seems
like
the
best
approach
in
order
to
create
short-lived
credentials,
because
because
by
default,
the
access
keys
and
secret
keys
do
not
have
an
expiry
on
them,
so
so
vyani.
So
so
can
you
can
you
talk
about
sts,
more
you're,
proposing
it
last
week?
Did
you
have
anything
else
to
add.
B
No,
so
I
I
can
explain
for
those
who
don't
know
so
you
you
first,
you
need
to
prove
you
are
the
person
you
claim
to
you,
you
claim
to
be,
and
you
use
it,
you
do
that
sorry,
typically
with,
for
instance,
odc,
so
you
come
with
an
identity,
your
an
email
address
and
when
so,
typically,
when
iem,
the
amazon,
the
amazon
service,
validates
your
oidc
token.
B
It's
bound
to
a
policy
which
defines
the
actions
you
can
do
on
the
service,
so
typically,
for
instance,
you
can
write
in
this
packet,
for
instance,.
B
This
would
go
to
the
workload,
so
you,
the
user,
so
once
you're
you're
authenticated.
B
Typically,
by
calling
assume
all
I'm
calling
what
so
it's
a
specific
op
code
in
iem,
it's
called
assume
role
and
you,
you
simply
call
assume
row
and
your
return
temporary
controls.
So
basically
the
triplet
access
key
secret,
key
and
sts
tokens
just
like
you
would
have
an
access
key
or.
B
Throughout
yeah,
so,
instead
of
returning
a
what
we
call
static
access
keys,
so
access
key
secret
key,
it
returns
a
triplet
access,
key
secret
key
token,
but
it's
exactly
the
same
and
the
applications
they
know
how
to
consume
it
as
well.
Just
you,
you
just
have
a
new
parameter,
which
is
token
right.
A
Yeah
yeah
and
also
also
also
other
object.
Storage
languages
have
habit
okay,
so
so.
A
Is
so
this
is
purely
for
this
is
purely
advertised
as
temporary
credentials.
The
the
advertised
use
for
this
is
to
give
you
an
upload
link,
for
instance,
that's
active
for
a
day
or
hack
to
for
a
week.
C
I
get
the
concept
like
there
is
an
api
that
you
know
using
a
special
credential
lets
you
do
stuff.
The
question
is:
how
standard
is
it
and
do
we
want
to
depend
on
it
right
right?
Because
if,
if
anybody
doesn't
implement
it,
then
you
lose
out
on
portability
and
we'd
have
to
figure
out
like?
Is
this
something.
A
I've
I've
done
the
research
all
three
have
it.
I
can
actually
show
you
what
it
looks
like
so
so
I
was
just
looking
up
if
sts
can
be
revoked,
it
is
possible.
So
that's
good.
So
sts
is
the
temporary
credentials,
whereas
yeah
so
in
iem.
We
know
it's
sds
on
gcp,
it's
security
token
service
on
azure,
secure
token
service
yeah.
They
also
have
it.
C
C
And
so
like,
we
would
have
to
make
it
optional.
Somehow
right,
implementations.
C
But
the
other,
the
other
problem
is,
is
a
bigger
problem.
In
my
mind,
which
is
this,
this
seems
to
just
push
the
problem
up
a
level
right.
If
what
we're
giving
you
is
a
token
that
you
can
use
to
obtain
tokens
like
that's
all
well
and
good,
you
know
you
can
rotate
your
own
token,
but,
like
the
whole
reason,
you
rotate.
The
token
is
out
of
fear
that
your
token
got
stolen
and
we're
giving
you
the
token
that
you
use
to
make
the
token.
So
what?
C
A
Oh,
you
don't
use
it
to
make
secrets.
So
so
that's
the
other
thing.
It's
it's
a
session
key.
You
just.
C
B
Not
really
you,
for
instance,
you
can,
you
can
set
your
you
can
say
that,
for
instance,
your
driver
has
got
a
specific,
specific
oadc
role
within
kubernetes,
and
so
tpk's
driver
is
a
as
a
is
authenticated
and
has
a
right
to
generate
temporary
credentials
for
users
for
bucket
access.
A
Yeah
yeah,
that's
what
I
was
getting
so
last
week
when
we,
when
we
were
kind
of
finishing
up
the
meeting
one
of
the
one
of
the
discussions.
I
think
one
of
the
things
that
ben
brought
up
was
for
the
simple
case
of
provisioning
access,
keys
and
secret
keys.
We
don't
do
rotation
because
it
just
makes
the
life
cycle.
A
It
doesn't
really
add
much
value
like
like
just
like
the
case.
You
was
talking
about
right
now,
but
where
we
do
credential
rotation
or
where
we
have
more
advanced
access
control
is
when
we
start
doing
the
iem
style,
access
im
style
access
is
where
a
service
account
a
kubernetes
service.
Account
is
associated
with
the
cloud
service
account
and
based
on
that.
A
Based
on
that
cloud
service
account
you
you
or
on
that
cloud
service
account
you
will,
you
will
set
the
type
of
access
that's
allowed
for
the
service
account
and
the
kubernetes
service
account
will
be
given
to
the
pod
and,
and
so
the
part
will
will
be
able
to
use
that
and
then
make
calls
on
behalf
of
the
cloud
service
account
and
and
that's
where
we
can,
we
can
do
more.
A
I
mean
that's
the
im
style
of
access
that
that
happens
in
cloud
providers
where,
but
is
that
is
that
strictly.
C
A
It's
not
it's
it's
where
it
gets
the
access
from
so,
if
you're,
in
an
amazon
instance,
if
you're
in
an
aws
instance.
If,
if
the
instance
has
permissions
to
talk
to
a
bucket
without
needing
any
credentials,
you'll
be
able
to
just
talk
to
that,
you
know
get
it,
but.
C
What
will
actually
happen
inside
the
pod,
so
the
pod
has
access
to
a
token
that
it
can
use
to
to
get
the
bucket
but
like
in
order
to
design
an
https
header.
It
still
needs
an
access
key
and
a
secret.
So
at
some
point
so
like
does
it?
Is
there
just
a
process
to
take
your
toke,
your
your
access
token
and
and
go
to
the
s3
service
and
say,
give
me
a
access
key
and
secret
that
I
can
use
to
perform
my
signature.
C
A
Let's
just
yeah:
let's
take
a
look
right
now,
so
I'm
thinking
about
where
the
code
is.
I
think
it
is
here.
Oh
no
here!
So
no,
not
here
it's
under
bench.
Okay,
so
I
think
here
so
this
is
this.
Is
this?
Is
a
you
know,
drop-in
replacement
for
s3
cli,
I'm
just
aware
of
how
this
works.
I'm
taking
a
look
here.
A
So
there
is
one
type
of
axis
where
I
am
aws
access,
where
just
depending
on
where
you,
okay
yeah,
so
what
it
does
is
it
talks
to
the
link
local
ip
address,
which
is
the
metadata
service
that's
available
on
again
all
vms
and
and
it
uses
that
to
get
an
sts
token
and
then
uses
that
to
talk
and
get
the
credentials.
C
Okay,
so
this
is
just
this
is
basically
what
we
were
already
planning
on
doing
with
im
style,
authentication,
yeah.
Okay,
we
just
need
to.
I
guess
I
I'm
not
familiar
enough
with
the
what
it
looks
like
when
you
do
the
the
the
var
and
the
ba
for
this
type
of
access,
because
we
don't
have
any
data
to
to
convey
right.
It's
just.
Basically,
we
just
need
to
express
that
trust
is
exists.
A
Right
so
so
here's
what
is
needed
so
so
in
order
to
in
order
to
make
the
call
to
the
backend
so
it
needs
these
are.
This
is
the
these
are
the
things
that
it
needs
so
either
a
web
identity
token
file,
which
is,
which
is
the
file
that
contains
the
sds
token,
along
with
that
it
needs.
The
arn
is
amazon
resource
number.
It
needs
the
role
for
that
or
the
arn
for
the
role.
A
C
A
This
is
all
for
x3,
so
so
what
cozy
needs
to
do
is
just
have
a
mechanism
to
pass
just
one
of
these
types
of
credentials
all
the
way
to
the
part
there
is
a
concept
of.
So
what
I
was
talking
about
here
was
a
web
identity.
Token
file,
that
is,
there,
is
a
concept
of
a
relative
uri
container
credentials,
relative
container
credentials
full
uri.
I
think
this
is
what
we
should
be
looking
at.
A
C
A
C
Yeah,
if
there's
multiple
implementations
that
are
literally
indistinguishable
from
each
other,
then
yeah.
It
seems
like
a
the
kind
of
thing
where
you
have
an
extension
and
say:
oh,
you
know
there's
this
extension
that
vendors
have
tacitly
agreed
on
and
multiple
vendors
have
implemented
it,
and
so
we're
gonna
have
a
capability
bit
that
lets.
You
say
you
want
it
and
lets
you
find
out
if
you've
got
it
etc.
You
know
that
that's
the
whole
optional
extension
and
that
would
have
to
be
from
top
to
bottom.
C
You
know
all
the
way
from
the
storage
class
down
to
through
the
the
side,
cars
and
the
rpc
basically
have
some
negotiation
to
say.
Is
this
feature
available
or
not,
but
but
yeah
I
I
thought
I
thought
that
what
we
were
focused
on
was
like
a
portable
mechanism
for
doing
secret
rotation
when
using
s3
yeah.
I
still
haven't
heard
of
that.
C
A
Let
me
put
it
this
way,
so
if
we
provided
a
generic
mechanism
for
drivers
to
return
some
details
back
a
bunch
of
files,
bunch
of
environment
variables,
actually
that's
it
and,
and
whatever
is
there,
gets
put
into
the
part
workload
part.
We
don't
know
what
these
files
are.
We
don't
know
what
these
are
completely
opaque
to
cozy.
We
don't
know
what
this
environment
variables
are.
We
just
set
them,
as
provided
by
the
driver.
C
But
but
if
it's
opaque
we're
not
adding
any
value
right,
the
the
whole
point
of
cozy
is:
we
have
a
clearly
defined
downward
api
that
you
can
consume.
If,
if
the
definition
of
the
downward
api,
is
you
get
a
bunch
of
opaque
text,
that's
not
useful
to
anybody,
because.
A
C
C
C
C
Unless
you
have
some
sort
of
explicit
negotiation
that
says
like
here's,
the
value
that
tells
you
whether
you're
going
to
get
the
extra
values
or
not,
and
if
it
says
yes,
then
the
extra
values
are
here
and
if
it
says
no
next
values
are
not
there.
So
then
you
at
least
know
as
a
user.
Like
okay,
I
go
read
this
variable.
I
see
if
the
extra
variable,
if
the
extra
values
are
set
or
not,
and
if
they
are,
then
I
know
where
to
go.
Look
for
them.
C
A
B
I
understand
that
so
today,
if
I,
if
from
the
so
today
for
the
lido
I
I
wrote,
I
only
returned
access
to
secretly
right
but
if
I
add
the
third
parameter
yeah,
it
won't
be
written
in
the
volume.
A
Someone
is
typing,
and
you
know
if,
if
you
can
mute
yourself,
so
be
good
yeah.
B
So
today
we
return
an
access
key,
a
socket
keeper
right.
If,
if
we
return
a
triplet,
what
does
it
change?
B
The
same
value
is
the
token.
C
So
so,
as
long
as
we
document
what
it
is,
and
we
say
that
it
cannot
be
there
like,
then
people
can
optionally
leverage
it
when
it's
there
and
not
when
it's
not,
but
it
can't
be
opaque.
That's
my
whole
point
is
whatever
it
is.
We
have
to
write
in
the
doc
that,
like
these
are
all
of
this.
This
is
what
this
value
may
contain,
and
these
are
the
possible
set
of
values
you
might
get
because
applications.
B
C
C
E
So
we
have
something
in
the
story
class
where
we
have
those
secret
keys
that
they
are
in
the
inside
the
storage
class.
Can
we
do
something
similar
for
this.
E
C
E
C
A
I
know
I
know
what
you
mean:
it's
in
the
csi
documents
right.
A
Yeah
it
it
has
a
it
has
a
signature
right.
It
looks
like
something
that
secret
dash
name
and
yeah.
It.
E
C
The
purpose
of
those
is
is
so
that
so
that
csi
drivers
can
be
handed
secrets
down
from
above
that
they
can
use
to
to
authenticate
to
some
storage
controller.
C
So
that's
those
are
secrets
coming
from
the
top
down
we're
talking
here
about
secrets
coming
from
the
bottom
up,
because
the
whole
thing
is,
we
need
to
provide
a
secret
to
the
pod.
If
it's
going
to
do,
s3,
right
and
and
yeah.
The
thing
I
want
to
emphasize
is
like
we
can't
have
a
situation
where,
like
you
either
get,
you
know
the
access,
key
and
secret
or
something
else,
and
that
forced
workloads
to
deal
with
both
situations
like.
C
A
C
Credentials
but
that
begins
to
sound
like
a
different
protocol
to
me,
like
you
know,
there
was
one
protocol
which
is
s3
with
access
key
and
secret
key,
which
you
know.
If
that's
what
you
want
to
do,
that's
what
you'll
get
and
there's
another
protocol
where
you're
not
given
those
things,
it's
more
like
a
configuration
of
a
particular
protocol,
but
maybe
I
mean
you
could
go
either
way,
but
but
yeah
it's
like
it's
a
decision.
C
You
have
to
make
up
front
where,
if
my,
if
my
particular
workload
only
knows
how
to
consume
access
key
secret
key,
then
I
need
to
know
what
I
can
ask
for
that
will
guarantee
that.
I
will
get
that
and
then
somebody
else
who
wants
to
do
something
better
can
ask
for
something
else
and
get
the
better
thing
or.
C
There's
always
the
possibility
that
the
particular
environment
you're
running
in
just
doesn't
support
what
you
want
and
we
want
to
minimize
when
that
occurs.
But
if
it's
inevitable,
then
it
is
inevitable
and
you
just
return
an
error
or
you
fail.
The
request
like
in
the
case
of
provisioning,
a
bucket.
I
guess
you
would
create
a
br
and
it
would
just
never
bind
and
you'd
see
a
bunch
of
events
on
your
br
saying
we
can't
do
what
you
want.
B
Yeah
for
for
just
for
info
for
all
amazon
compatible
applications,
they
know
how
to
transparently
pass
the
config
file
right.
B
No,
but
the
point
is
so
the
point
is,
is
different,
so
let's
say
we
we
use
a
cozy
as
it
is
defined
today
and,
for
instance,
the
driver
generates
temporary
tokens
and
returns
the
triplet,
so
access
key
secret
key
token
written
inside
the
pod.
The
application
knows
how
to
consume
the
triplet
and
that
that's
fine
right
and
it's
especially
let's
say
it's
a
special
vendor
or
driver
setting
yeah.
Now
what
happens
if
the
token
expires?
B
C
B
B
To
do
that
in
the
first
place,
can
we
just
say:
okay,
it's
impossible
and
the
only
I
don't
know
if
it's
possible.
B
C
C
Mean
if
the
answer
is
you
just
can't
do
it,
then
we
say:
okay,
we
can't
do
key
rotation
within
this
particular
framework
of
of
you
know,
cozy's,
giving
you
an
access
key
and
a
secret,
and
just
say
that
that's
it
and
you
know
we
get
key
rotation
through
these
other
protocols
or
with
these
other
options.
In
other
ways,.
A
A
So
I
mean
I
don't
know
if
this
complicates
it,
but
we
could
have
the
concept
of
key
rotation
request.
What
that
would
do
is
it
would
go
provision
the
new
keys
and
then
do
a
rolling
upgrade
of
the
parts
that
are
using
it.
D
Sync,
some
chrome
job
or
whatever,
which
issues
key
rotation
requests
and.
A
C
D
C
Well,
I
I
I
don't
know
what
the
options
are.
Has
anyone
ever?
Has
anyone
investigated
whether
it's
possible
to
simply
update
the
access
key
in
secret
for
a
given
ba
and
have
that
propagate
into
cubelet
and
into
the
running
pod,
in
a
way
that
the
workload
could
just
re-read
that
old
file
and
get
new
values
that
the
last
part.
C
A
C
D
It
is
the
very
same
as
a
config
file
that
you
keep
in
a
config
map
or
some
generic
secret
that
you
keep
in
a
secret
and
that
you
mount
as
a
volume
in
a
pod.
Then
detecting
changes
in
that
config
file
is
something
you
have
to
code
up
in
your
application
and
then
re-read
the
config
file
or
just
exit
or
whatever
you
want
to
do.
When
you
detect
the
situation.
C
So
I
I
like
that
solution
because
wait.
What's
the
solution,
the
solution
of
just
periodically
mutating
the
ba
to
have
new
values
when
and
and
then
relying
on
the
pod
to
notice
that
the
values
changed
either
with
some,
I
notify
events
or
detecting
errors
and
re-reading
files
on
errors
or
whatever.
A
I
mean
I
notify
events
will
have
to
be
listened
on
by
the
by
the
workload
but
yeah.
Some
other
mechanism
is
possible,
but.
C
A
Should
use
the
new
ones?
Why
can't
we
just
say
it's
it's
going
to
be
a
rolling
upgrade
so
and
see.
The
idea
is
if
you're
using
object,
storage.
It's
already
stateless.
A
There
will
be
manual
and
it'll
be
too
much
effort,
but
I'm
saying
if,
if
we
were
to
say
we
will
manipulate
the
we'll
mentilate
the
access
and
secret
keys
in
place
in
the
ba.
I'm
just
saying
the
way
it's
put
into
the
pod.
Is
we
don't
expect
the
application
to
do
anything
special,
we'll
just
restart
the
part
and
when
the
application
re
re-reads
that
when
it
starts
up
and
reads
the
new
conflict
or
config,
it's
it's
the
new
value
right,
but
people
don't
like
their
pods.
Restarting
that's
what
I've
learned.
It's.
C
A
C
A
C
Would
be
at
any
step
where
the
sidecar
is
is
minting
your
credential
through
you
know,
satisfying
a
bar.
In
addition
to
getting
back
to
the
credential,
you
would
get
back
a
time
stamp.
That
would
say
you
know
refresh.
A
C
You
know
you
refresh
by
date
and
maybe
an
additional
expiry
date
that
is
at
a
later
time.
You
just
get
back
an
additional
time
stamp,
but
then
the
idea
would
be
the
the
controller
would
have
a
queue
of
you
know
of
eas.
That
need
refreshing
at
some
point
in
the
future
and
periodically.
It
would
look
at
its
cue.
C
We
would
the
the
idea
would
be
to
to
make
some
fairly
conservative
time
guarantee
like,
within
one
hour
of
of
the
time
stamp,
that
the
driver
returned.
We
will
get
a
call
to
go
refresh
it
so
like
you
need
at
least
a
buffer
of
an
hour,
but,
like
these
controllers
aren't
going
to
go
down
for
more
than.
A
An
hour:
well,
no,
that's
that's
where
the
problem
is
so
I'll.
Tell
you
why
I'm
concerned
about
this?
When
an
outage
happens,
let's
say
a
cozy
controller
goes
down,
but
but
the
object
storage
parts
are
working.
Fine.
Now
what
we're?
And
if
we
have
an
expiry
on
the
tokens
suddenly
object.
Storage
will
also
start
failing.
It'll
lead
to
a
cascading
failure
faster
than
we'll
know.
A
C
The
60-day
mark
you're
gonna,
refresh
it,
but
that's
the
choice
of
the
drivers.
Well,
I
mean
you
could
pick
different
days,
but
the
point
is
like
in
a
regime
like
that
you're
going
to
refresh
your
keys
every
60
days
and
if
you're
late,
for
whatever
reason
you
can
be
up
to
30
days
late
and
you
still.
C
C
A
At
about
the
two-thirds
point
right
right,
that's
what
I
mean
like
it's
a
best
practice.
What
you're
talking
about
we
can't!
We
can't
expect
everyone
to
use,
follow
the
best
practice,
but
we
can
put
in
fail
safes
in
the
sense
we
can
make
it.
You
know,
so
it's
better
to
put
the
responsibility
on
the
user
than
on
the
driver.
A
Yeah
yeah
like
expiry
duration,
or
something
like
that,
something
that
says
how
long
it
will
be
alive
or
how
long
will
be
valid
and
then
a
refresh
duration,
refresh
period.
C
I'm
more
comfortable
with
that
kind
of
that
kind
of
scheme
where
yeah
the
the
sidecar
just
calls
the
driver
and
the
driver
returns
back.
You
know
call
me
again
by
this
time
and
and
we
have
to
have
some
buffer
for
the
you
know,
controller
downtime,
but
you
know
you
do
a
best
effort
to
call
back
before
the
or
shortly
after
the
the
time
expires.
As
long
as
you
have
enough
of
a
buffer
between
your
refresh
time
and
your
expired
time,
you
can
withstand
pretty
large
outages
and
then
you
just
go
with
it.
A
C
C
C
A
Consider
this
scenario:
if
if
it
gives
you
back
and
a
key,
that's
valid
forever,
but
then
it
sets
the
expiry
date
to
90
days
and
then
says
refresh.
You
know
refresh
it
on
the
60th
day,
we'll
go
refresh
it
we'll
go
we'll
go
ask
the
driver
to
give
us
a
new
token,
and
that
implicitly
tells
the
driver
to
delete
the
old
token.
So,
even
though
it
was
valid
for
60
days,
access
can
still
be
rotated.
Credentials
can
still
be
rotated.
C
At
that
moment,
it's
going
to
say:
okay,
I
have
my
new
key,
the
old
one,
it's
still
valid
for
another
30
days,
but
it's
not
I'm
gonna
forget
about
it,
because
it's
not
my
problem
and
it
becomes
the
driver's
problem,
then
to
remember
to
expire
it
30
days
later,
when
it
originally
said
it
was
going
to
expire
it.
So
I
mean.
A
C
A
I
see
and
and
how
do
we
do
the
refresh
then
like?
So
if
you
say
it's
entirely
up
to
the
driver
to
say
call
me
back
in
60
days:
yeah
then
we'll
just
call
them
back
in
60
days.
That's
what
you're
seeing.
C
Yeah
and
if
it's,
if
the
driver
said
that
this
key
isn't
going
to
expire,
and
you
call
back
anyways
and
ask
for
the
new
key,
presumably
the
driver
will
just
give
you
the
old
key
again
and
say
it
hasn't
changed.
Here's
the
key
and
the
expiration
is
still
infinity
and
and
it'll
be
item.
Potent
right.
You
can
call
it
over
and
over
again
you'll
keep
getting
back
the
same
key
and
the
same
expiration
date
of
nil
or
infinity
or
whatever
you
want
it
to
be.
A
C
Yeah,
so
so,
if,
if
the,
if
it
becomes
refresh
time
and
you
have
an
old
key,
you
go
call,
you
get
the
new
key
and
the
driver
gives
you
a
new
one.
Then
the
sidecar
would
have
to
update
the
ba
with
the
new
value
immediately
and
update
the
expiration
time
of
in
the
ba
to
the
new
date.
A
A
C
C
A
C
A
Okay,
so
so,
if
the,
if
the
keys
do
get
leaked,
then
until
the
expiration
time
you
know
it
can
be,
it
can
be
used
for
you
know
malicious
reasons.
That's
what
you're
saying
yeah
that
that's
the
point
of
an
expiration
time
I
see
and
and
but
not
at
the
point
of
rotation
time.
So
it's
valid
until
the
point
of
expiration
I
see
and
then
for.
A
Don't
support
expiry,
it's
it's
infinity,
we
don't.
We
don't
even
call
refresh
right.
We
can't
do
that.
A
C
Right
because
he
only
ever
knows
about
the
most
recent
valid
one
and
the
driver
knows
about
two
of
them
and
then
presumably,
if
someone
comes
in
and
tries
to
delete
the
bar
and
the
side
card
deletes,
the
ba,
the
driver
would
have
to
be
smart
enough
to
clean
up
both
of
those
keys
because
they're
both
associated
with
the
same
access
request.
Just
one
of
you
know.
So,
if
somebody
explicitly
said
revoke
access
by
deleting
the
bir
you'd
want
to
revoke
all
of
the
access
associated
with
that
here's.
Another
problem.
D
D
C
A
The
driver
itself
necessarily
wouldn't
hold
that
state
it'll
be
in
the
api
in
the
storage
system.
Right,
that's
what
you're
saying
right!
Benny.
C
A
C
A
So
so
here's
another
problem
I
see.
Let's
say
the
driver
gives
you
a
new
key
and
then
we
try
to
update
the
ba
with
it.
So
the
sidecar
tries
to
update
the
ba
with
it
and
updates
to
the
bf
fails
for
whatever
reason,
let's
say:
apso
is
down
at
that
point
or
whatever.
C
A
So
it
needs
to
retry.
C
Yeah
because
you'll
fail
to
update
the
time
stamp
too
right,
you
would
make
the
update
to
the
access
key
and
secret
key
and
refresh
timestamp
all
at
once,
and
if
and
if
an
update
to
any
of
them
fails,
nothing
to
all
of
them
would
fail,
and
then
next
time
you
restart
the
sidecar,
you
would
see
that
it
still
needs.
It
still
is
due
to
be
updated
and
you
would
try
again
so.
C
A
A
So
the
same
answer
is
where
I'm
getting
to
so
it
could
be.
I
mean,
like
it's
not
reasonable,
to
expect
the
driver
to
give
you
back
the
same
set
of
new
credentials
right
yeah.
It
is
no
because,
let's
say
let's
say
you
know
you
you
refresh
the
refresh
period
is
60
days
and
expiry
is
90
days
and
then
and
then
on
the
59th
day,
the
sidecar.
A
C
A
Yeah
but
the
problem
becomes
if
the
amount
of
time
left
is
less
than
you
know,
expiry
time
minus
refresh
period.
C
A
C
You
just
get
whatever
the
latest.
I
I'm
presuming
the
existence
of
some
some
process,
that
is,
you
know,
refreshing
keys
every
60
days
and
and
at
any
time,
there's
either
well
at
any
time,
there's
either
one
or
two
valid
keys.
C
You
always
get
the
latest
one.
Whenever
you
ask
and
and
whatever
it's
expiry
time
is
it,
but
it's
so
it's
not
guaranteed
to
be
valid
for
60
days
from
the
moment
you
ask
for
it.
A
You're
saying
this
there's
an
external
mechanism,
I
mean
not
external
mechanism,
some
mechanism
by
which
credentials
are
rotated
periodically,
and
we
expect
this
driver
to
go
get
the
latest
version.
C
Yeah
and
hopefully.
A
A
A
It's
it's
like
you
know,
that's
being
run
either
as
a
you
know,
within
the
cluster,
by
the
driver
or
as
a
part
of
the
driver
or
or
somehow
it's
it's
enabled
in
the
back
end.
I
would
think
it's
more
a
part
of
the
driver.
C
But
the
point
is
it
doesn't
matter
and
it's
to
my
mind
it's
it's
fully
general
right,
because
then
it
doesn't
matter
what
you're
doing
down
there.
You
know
cozy
will
call
you
when
you
want
to
be
called,
and
you
only
have
to
return.
One
answer
and
cozy
will
always
give
that
answer
to
the
pods
and
then
everything
else
isn't
really
relevant.
C
Well,
I
mean
you're
saying
that
you
might
call
it
and
every
time
you
call
you
get
a
new
secret
yeah
like
yeah.
I
don't
see
any
harm
in
that
other
than
that.
It
creates
a
nightmare
for
the
for
the
driver,
because
because
every
time
you
return
a
secret
with
an
expiration
date,
it
has
to
be
valid
until
at
expiration
date.
And
so,
if
someone
calls
this
api
every
five
seconds
for
a
month
and
you
return
80
000
tokens
like
they
all
have
to
be
valid
for
the
time
period
that
you
say,
yeah.
C
A
So
this
is
why
this
is
why
I'm
thinking,
maybe
if
we
were
to
you,
know
instead
of
automatically
you
know,
instead
of
the
driver,
saying
give
me
new
credentials
or
driver
even
having
a
concept
of
expiry.
A
What
if
we
left
it
to
the
user
entirely,
and
we
said
hey,
you
know,
create
a
create
a
credential
rotation
request
put
in
a
cron
job.
If
you
want
do
whatever
you
like,
but
whenever
your
credential
rotation
request
comes
in,
we
go
talk
to
the
driver,
we
get
new
credentials,
put
it
into
the
ba
and
the
rest
of
the
workflow
remains,
as
is
so.
That
way,
what
happens?
Is
it's
not
an
automated
create
a
million
token
kind
of
thing.
A
It's
more.
You
know
you
call
it
when
you
need
it.
C
A
C
C
C
A
C
But
it
it,
it
seems
like
if
you
want
to
do
anything
at
all
like
this
is
sort
of
maximally
flexible
because
it
basically
says
whatever
the
driver
wants
to
do
is
valid.
The
sidecar
is
just
going
to
call
it
back
and
get
the
new
credential
when
the
driver
asks
it
to
and
that's
the
end
of
the
story
and
and
the
default
is
the
drivers
never
want
to
be
called
back
and
everything's
valid
for
infinity
or
until
deleted.
A
The
abstraction
screens
like
there's
something
here,
I
can't
quite
put
my
finger
on
it.
Maybe
maybe
you
know
I'm
thinking
of
it
wrong,
but
now
this
idea
of
having
the
workload
or
having
the
driver
automatically
ensure
that
the
secrets
that
they
get
back
is
is.
Is
that
important?
Then
the
new
rotated
credentials
are
very
important.
That's
what
that's!
What's
like
tough.
C
I
think
it
would
just
be
a
matter
of
like
the
way
one
would
implement
it
on
top
of
the
system
that
didn't
naturally
do.
This
would
be
every
time
you
get
that
call.
You
would
go
look
at
the
current
set
of
access
keys
that
have
been
set
up
for
that
ba
on
or
that
principle
on
that
bucket
and
if
there
was
one
that
was
created
within
the
last
x
amount
of
time
just
return
it,
and
only
if
a
certain
minimum
amount
of
time
had
passed.
C
But
since
anyone
had
asked
about
it,
you
could
go
ahead
and
generate
a
new
one,
and
that
would
put
a
cap
on
how
many
could
get
generated,
but
it
would
still
ensure
that
you'd
have
a
you
know.
If
somebody
called
called
back
very
very
late,
they
would
still
get
a
new
one.
That
was
good
for
a
healthy
amount
of
time.
A
So
so,
okay,
let
me
put
it
in
a
different
way.
So,
given
that
aws
doesn't
have
the
concept
of
or
any
of
the
cloud
providers,
they
don't
have
the
concept
of
expiring
tokens.
If
we
were
to
put
in
you
know,
there
is
a
likelihood.
B
They
do,
for
instance,
in
in
amazon,
eks.
D
C
C
So
so
yeah,
so
if
there's
no
way
on
the
actual
storage
back
backend
to
like
set
a
time
stamp,
that
says
this
one
expires
you
know
on
may
30th
may
31st
then,
like
that
that
metadata
has
to
get
stored
in
another
place
and
there
has
to
be
another
something
watching
a
clock.
And
then,
when
may
31st
comes
it's
going
to
have
to
go
talk
to
the
back
end
and
delete
that
token,
because
it's
time
and
so
yeah.
A
A
D
D
A
Right
right,
jwt
is
pretty
great
that
way.
Yeah
you,
you
you,
you
know
it
signs
the
payload
and
then
nobody
can
go
and
change
the
payload,
because
it's
signed
with
the
private
key
that
only
you
have
indeed
and
the
expiration
time
etc,
is
stored
in
the
token
yeah
it's
stored
in
the
token
itself,
yes,
and
nothing
in
the
token
can
be
changed,
but
the
token
itself
can
be
plain
text.
C
S3
requires
somebody
stores
this
access
key
in
secret,
at
least
where
the
workload
can
find
it
and
where
the
back
end
can
get
it,
and
it
has
to
at
least
pass
through
the
driver
you're
right
that
the
driver
probably
shouldn't
be
storing
these
things.
But
you
don't
need
to
literally
store
the
the
secret.
You
could
just
store
the
access
key
and
the
time
associated
with
that
access.
Key.
A
Right
right
that
that's
that's
fair
enough!
That's
all
should
be
yeah,
so
just
stores
the
access
key
and
the
time
associated
with
the
expiry,
and
when,
when
cosy
asks
for
a
new
key,
it
should
give
the
old
access
key.
A
C
A
So
so
what
I'm
saying
is
when
refresh
time
happens,
cozy
would
so
in
order
to
make
sure
that
the
back
end
can
be
added
important.
What
we're
saying
is
we'll
give
you
the
previous
access
key.
You
gave
us
use
this
as
a
reference
to
make
sure
that
the
next
one
you
give
me
you
know
it's
based
off
of
this.
In
the
sense,
let's
say,
there's
a
consistent
hashing
or
something
they
do.
A
So
that's
so
that
if
the
previous
access
key
is
like,
if
you
give
the
same
access
key
back
to
me,
I'll
give
you
the
same
new
key,
but
you
have
to
give
the
secret
to
what
what,
if,
if,
if
it's
going
to
create
new
access.
A
C
Well,
then,
you
need
a
separate,
then
the
function
signature
needs
to
look
there.
If
you
need
to
have
a
a
a
return
code,
that
says
nothing
changed
and
I'm
not
telling
you
what
the
secret
is
right
like
if
the
return
value
typically
says:
here's
the
access
key
and
here's
the
secret
and
here's
the
expiration
date.
C
Then,
when
it
didn't
change,
you
would
need
a
special
return
code.
That
said,
nothing
changed
and
I'm
not
giving
you
any
data
yeah.
So
hopefully,
then
it
becomes
the
responsibility
of
the
of
the
sidecar
to
just
not
do
anything.
Just
not
change.
What's
already
to
remember
the
value
and
if
it
ever
wanted,
a
new
value
you'd
have
to
pass
in
an
empty
string
for
the
last
access
key.
A
Yeah
there
are
time
over
actually
two
minutes
extra.
So
let's
continue
the
discussion
on
monday.
I
think
I
think
today
was
good,
especially,
I
think
the
first
time.
C
We
made
some
progress
on,
but
do
you
really
want
to
be
spending
this
much
time
on
key
rotation?
I
I
feel
like
we're
pretty
deep
down
around
hole.
I.
E
Should
where
are
you
going
to
schedule
this
api
review
meeting
again
so.
A
Tim
is
busy
this
week,
we've
reached
out
to
him.
I'm
gonna
send
another
email.
Today,
let's
see
what
he
says.
He
said
he
could
meet
us
last
friday,
but
then,
but
then,
when
we
tried
to
set
up
a
time
he
he
was
busy.
So
maybe
this
week,
maybe
you
know
friday
this
week
or
maybe
sometime
next
week.
Do
we
know
when
the
code
freezes
or
the
freeze
for
for
this
is.
E
E
I
I
I
there's
some
email
saying
they
want
to
get
that
merge,
but
I
don't
know
that
is
actually
merged
yet
it
may
not
be
modulated,
but
there
are
some
discussion
about
that.
E
No,
no,
no,
no,
not
that
I'm
talking
about
the
changing
the
the
cycle
from
three
months
to
four
months,
and
there
is
a
type
about
that.
I
don't.
I
think
that
way-
it's
probably
still
not
merged.
It's
been
there
for
very
long
time
so,
but
I
suspect
it's
going
to
be
changing
for
by
now
22
but
another
100
sure,
and
then
that
release
schedule
is
not
out
yet.
C
D
C
E
A
Okay,
all
right,
so,
let's
continue
on
monday.
I
want
to
finish
like
let's
make
a
decision
on
monday
about
how
we're
going
to
go
with
the
credential
rotation
and-
and
you
know,
reach
a
conclusion.
There
are
a
lot
of
other
things
to
talk
about
so
we'll
start
talking
about
the
next
priority.
Soon,
okay,
cool
all.