►
Description
Kubernetes Storage Special-Interest-Group (SIG) Object Bucket API Design Meeting - 24 June 2021
Meeting Notes/Agenda: -
Find out more about the Storage SIG here: https://github.com/kubernetes/community/tree/master/sig-storage
A
So
hello,
everyone
for
those
of
you
who
missed
it.
I
was
just
saying:
we've
come
to
a
point
now
where
our
major
architectural
concerns
have
been
addressed.
There
are
still
some
details
to
be
worked
out,
but
nothing
nothing
that
you
know,
there's
nothing
that
that
that
is
an
open
question
that
can
affect
the
overall
architecture
so
so
going
forward,
at
least
until
we
finish
the
api
review.
A
A
A
A
So
so,
if
someone
were
to
come
in
and
ask
us
how
they
would
go
about
writing
a
driver,
we
can
point
them
to
the
sample
driver.
If
someone
were
to
come
in
asking
how
they
can
start
contributing,
we
can
help
them
do
that
or
if
a
vendor
is
looking
to
figure
out
how
to
add
a
certain
feature.
We
can
talk
about
that.
However,
in
terms
of
architecture
itself,
at
least
for
the
core
of
it,
we
are
in
consensus
about
how
we
want
to
move
forward.
A
A
Okay,
yeah
yeah
no
worries,
okay,
so
the
so
one
of
the
things
I
wanted
to
discuss
today.
So
chris
came
up
with
a
good
idea
to
write
down
whatever
we
want
to
discuss
in
in
a
document
like
for
that
meeting,
so
I've
shared
the
document
in
our
sig
storage
cosi
channel,
so
that
would
be,
for
instance,
a
place
to
put
our
meeting
agendas
for
the
day.
A
It's
also
been.
The
same
document
is
pinned
to
the
six
storage
cosy
channel.
So
anyone
who
wishes
to
bring
up
something-
please
add
your
comments
or
please
add
whatever
you'd
like
to
talk
about
as
an
entry
in
that
document
and
and
to
start
off,
jiffin
and
jeff
brought
up
an
issue
that
I
think
needs
to
be
worked
out.
Jeff.
This
is
about
bucket
ownership
that
you
brought
up.
Did
you
want
to
explain
that
or
different?
Did
you
want
to
explain.
C
So,
while
developing
the
safe
driver,
one
of
the
main
concern
mean
question
was
who
wants
the
bucket
like
whether
we
have
to
like
in
a
bucket
request
when
it
comes
to
a
driver?
Should
we
create
a
user
owner
for
the
bucket,
or
should
we
create
a
like
one,
single
user?
Who
will
create
all
the
buckets?
C
So
that's
the
question
which
was
faced
and
there
was
some
scalability
issue
from
I
would
say,
maybe
from
hw
point
of
view
like
having
a
lot
of
uses
for
a
single
bucket
may
not
work
like
having
a
lot
of
buckets
for
you.
So
works
fine.
But
if
you're
having
a
lot
of
users
point
in
the
same
bucket,
it
can
like
it's,
the
general
conservatives
are
faster.
You
can
leave
about
that,
so,
but
basically
the
under.
So
they
were
a
bit
concerned
like
who
owns
the
bucket
in
a
bucket
request.
C
That
was
idea
and
whether
can
we
have
store
save
that
information
in
the
bucket
request
or
not
in
the
bucket
request,
at
least
whether
it's
usable
to
fetch
or
something
like
that.
So
when
a
bucket
tax,
the
quest
comes
so
that
the
bucket
attacks
supposed
to
have
an
idea
who
is
sound
of
the
bucket?
Something
like
that.
C
So
those
are
two
concerns
which
were
raised
during
our
like
still
on
the
development
or
like
when
I
pushed
the
pr
the
circus,
which
was
waste,
and
I
think
jeff
had
did
a
like
nice
write-up.
On
the
I
most
of
the
dogs,
most
of
the
things
jeff
mentioned
the
dog,
which
I
am
also
interested
like
about
the
opaque
field
and
yeah
even
bucket
ownership.
Then
green
field
brownfield
scenarios.
I
am
still
like.
C
A
Okay,
can
I
summarize
what
you
just
said
and
let
me
know
if
I'm
getting
it
right,
so
the
first
concern
you
brought
up
was
you're
trying
to
figure
out
if
there
is
a
way
to
have
a
user
per
bucket
and
or
have
a
single
user
for
all
the
buckets
or
a
set
of
users
for
a
group
of
buckets,
I
guess
and
in
your
context
you're
saying
it
is
much
more.
It
might
be
a
scalability
concern
if
you
had
lots
of
users
for
a
single
bucket
right.
C
C
D
C
E
C
A
A
A
If
we
were
to
start
having
parameters
in
there,
we
would
we
would
lose
portability,
which
is
a
very
important
concern
for
us
now.
That's
why?
But
you
know
we
understand
that
people
still
need
portability
and
that's
why
we
give
that
ability
through
the
bucket
class.
I
mean
sorry,
we
understand
that
people
still
need
the
ability
to
set
opaque
parameters.
That's
why
we
we
have
the
parameters
field
in
in
bucket
class.
A
Now
that
adds
the
admin
step
and
because
bucket
classes
bucket
class
is
a
cluster
scoped
object,
and
we
say
that
you
know,
because
it's
an
admin
step.
It
is
reasonable
to
expect
the
admin
to
to
work
with
two
different
systems
and
and
not
expect
you
know
out
of
the
box
portability
like
you,
it's
it's!
Okay,
to
expect
the
admin
to
configure
parameters
specifically
for
whatever
driver
they're,
running
they're
running
the
current
system,
with.
C
C
A
A
C
A
C
I
don't
know
like
what
I
want
to
say
like
the
opaque
feels
like
it
should
not
be
what
what
I'm
trying
to
say
was
like
like
it
should
not
be
like
it
should
not
depend
on
the
workload
like
workload
or
like
if
the
user
can
define
the
field.
Apart
from
that,
like
it
should
work
as
like,
if
aws
inaudible,
if
it
is
working
s
like
this,
it
should
work
similar
like
I
what
I
mean
the
application
should
not
be
aware
of
that
or
play
field
right
like.
A
A
They
request
an
s3
bucket
and
how
it
happens
in
the
back
end
is
something
that
the
admin
knows
about.
The
admin
knows
that
you
know
it's,
the
the
actual
buckets
are
being
provisioned
by
either
s3
itself,
aws,
three
itself
or
ceph
for
something
else,
so
that
that's
that's,
where
we're
coming
from
the
the
laws
of
portability
happens
when,
when
we
go
from
one
provided
to
another
for
the
same
protocol,.
C
C
C
C
D
A
Yeah,
so
bucket
class
is
a
clustered
resource,
so
you
can
specify
custom
fields
in
the
bucket
class.
All
you're
saying
is
for
user
defined
resources.
You
know
they
should
just
expect
portability
out
of
the
box
they.
The
idea,
is,
we
don't
expect
users
to
have
to
know
about
whatever
the
back
end
is
or
they
should
just
be
able
to
take
their
workload
from
one
cluster.
Let's
say
it
fails,
so
they
just
take
the
ammos
put
another
cluster
and
it
should
just
work.
C
Okay,
so
I
understand
depends
so
if
I
understand
correctly
so
this
is
another
I
like
like,
which
part
which
I'm
not
fully
understand.
So
the
bucket
request
is
user.
Different,
like
it's,
not
admin
driven,
I
thought,
like
admin,
have
the
power
to
do
the
packet
request
and
the
user
can
only
access
the
customer
bucket
access
request.
A
Yeah,
the
bucket
request
and
bucket
access
requests
are
user
definable.
If
a
user
wants
to
request
a
new
bucket,
they
can
do
it
through
the
bucket
request.
We
have
had
conversations
just
very
recently
actually
to
make
bucket
creation
a
user.
Sorry
admin
only
operation,
but
you
know
in
with
the
current
way
things
are
done
that
would
fit.
However,
one
of
our
responsibility
is
also
to
push
the
needle
forward.
You
know.
A
A
C
A
C
C
A
A
C
Yeah
yeah,
so
currently
the
bucket
acts
as
a
bucket
class
will
be
created
by
that
mean
and
bucket
request
and
bucket
access
request
can
be
created
by
the
user.
So
and
another
question
like
why
do
we
need
a
bucket
taxes
class?
That's
another
thing
which
I
always
wondered
like:
do
you
really
need
a
bucket
access
class
or
something
like
that,
like
the
bucket
access
request
can
offer
the
bucket
request
right.
A
C
A
D
F
D
F
Me
see
like,
like
you,
could
imagine,
having
one
bac
for
full
control
of
a
bucket
and
another
bac
for
read-only
access,
and
the
admin
could
set
those
both
up
ahead
of
time
and
then
a
user
could
have
some
bars
for
full
control
and
some
bars
for
read-only.
Without
without
bacs.
There
would
be
no
difference
from
one
bar
to
the
next.
They
would
all
just
refer
to
the
same
bucket.
They
all
get
different
credentials,
but
they
would
all
result
in
the
exact
same
thing,
so
bacs
allow
you
to
have
different
levels
of
access.
F
Yeah
you
could,
but
but
there's
expected
to
be
opaque
stuff
in
there
that
that
you
know
that
the
admin
might
want
to
set
up
at
the
bac
level,
and
this
just
gives
you
sort
of
a
menu
selection
way
of
doing
it
like
this
is
one
of
the
the
parts
of
cozy.
That's
relatively
poorly
defined,
I
think,
is
what.
D
C
Yeah,
okay,
but
the
higher
level,
like
so
from
higher
level
of
point
of
view,
like
the
first
admin
need
to
create
the
bucket
class
bc,
then
user
will
create
a
via
which
will
create
a
b.
Then
add
me
need
to
create
the
bac
like
that's
the
workflow
right.
Currently
we
are
committing,
then
only
user
can
use
like.
I
feel
like
a
bit.
A
F
A
B
Yes,
I
think
we
didn't
like
the
idea
of
referencing
another
object
from
the
bac,
and
we,
I
think
the
consensus
at
one
of
the
meetings
was.
We
should
just
embed
what
we
wanted.
The
reason
we
origin,
you
know
directly
in
the
pacs,
have
them
as
fields,
but
the
reason
we
didn't
do
that
originally
was
because
we
wanted
to
give
the
admin
the
flexibility
that
you
can
get
with
a
kid
configmap.
B
So
essentially,
it
is
opaque
in
the
sense
that
the
config
map
can
define
any
key
values,
and
then
we
would
just
reference
that
config
map
right
and
if
that's
not
a
good
idea,
we
can
we
can.
We
can
either
try
to
find
commonality,
which
has
always
been
a
challenge
in
this
project
between
the
different
object
stores
or
we
can
have
a
config
string
string
map
in
the
bac
that
would
be
equivalent
to
the
config
map.
F
F
My
main
objection
to
the
config
map
was
just
that:
config
maps
are
namespaced
and,
and
these
objects
are
not
name-spaced
and
and
it,
and
there
was
some
documentation
that
suggested
like
a
certain
format,
for
what
should
go
in
the
config
map,
and
if
it's
supposed
to
be
opaque,
we
should
just
leave
it
completely
opaque
and
say:
look
you
put
what
you
want
there.
We
don't
have
an
opinion
as
a
community
about
what
goes
in
the
opaque
fields.
D
F
And
and
a
config
map
in
particular
just
having
a
non-namespace
object,
point
to
a
namespace
object
felt
very
strange
to
me,
because
what
namespace
is
that
object
in
and
if
a
user
has
access
to
read
bacs
and
they're
going
through
and
looking
at
them
like?
Can
they
read
the
things
that
the
bacs
point
to
or.
A
F
Well,
you're
deleted
yeah,
I
mean
there's
all
those
questions,
so
it's
like
the
vac
should
be
self-contained.
I
think
if,
if
there
is
a
legitimate
reason
why
a
simple
string
map
of
opaque
parameters
doesn't
give
doesn't
give
driver,
authors
and
administrators
enough
power
to
do
what
they
want
to
do,
then,
then
we
could
have
a
problem,
but
we,
I
think
we
should.
We
should
find
the
gap
before
we
try
to
to
solve
it
with.
C
So
this
is
like
I
just
want
to
mention
like
if
you
are
offering
a
policy
kind
of
stuff
like
having
some
policies
say
I
can't
read
the
contents
or
something
like
that.
It
makes
sense
to
have
it
on
via
then
on.
I
mean
on
the
bucket
than
the
bucket
access,
of
course
like,
so
the
driver
have
more
control
over
that
like
if
it's
on
the,
if
you're.
A
E
A
F
F
F
A
F
Yeah
yeah,
that's
where
I
was
going,
is
many
use
the
same,
but
not
all
right.
So
we
have
to
make
the
decision.
Do
we
want
to
force
everyone
or
do
we
want
to
accept
the
ones
that
don't
conform?
And
if
we
do
then,
then
any
kind
of
you
know
standard
is
off
the
table.
We
just
have
to
say:
look
you
pick
from
a
menu,
wouldn't
that
be
up
to
the
driver
to
convert
that
yeah.
But
I
mean
the
important
thing:
is
the
end
user
experience?
F
A
Okay,
so
there
are
two
issues
we
saw:
one
is
the
different
formats
or
different
types
of
access
control
mechanisms
that
different
drivers
have.
Another
is
just
the
complexity
of
access
control
that
these
drivers
provide
take
aws,
for
example,
they
allow
very
complex
styles
of
access
definitions
using
the
using
the
iam
policy
definition
for
examples.
A
A
F
Well,
we're
removing
the
removing
the
power
from
the
hands
of
users
to
administrators
right
by
putting
it
in
the
class
right.
So
so
in
I
mean
and
there's
plenty
of
kubernetes
clusters
where
the
end
user
is
the
administrator,
and
so
he
has
the
power
to
do
both
things
and
he'll.
Just
if
he
cares,
he'll
go
set
up
the
classes
with
all
the
special
rules
he
wants
and
then
use
the
classes
like
he
wants
inside
his
cluster.
But
none
of
what
he
does
will
be
portable,
because
he'll
rely
on
his
bucket
access
classes
or
bucket
classes.
A
F
D
F
D
F
A
So
I
want
to
hear
from
jeff
for
jiffin,
so
you
know,
given
that
the
access
policy
in
in
ceph
is
very
different
from
other
standardized.
S3
stores,
like
like
mineio,
for
instance,
allows
the
premium
style
access
policy
definitions.
So
whatever
amazon
supports,
we
support,
but
that's
not
the
case
with
ceph.
The
access
control
is
done
differently.
C
A
F
So
so
sid
we,
we
also
support
amazon
style
access
control,
but
the
I
think
that
the
underlying
issue
here
is
that
everyone
who
supports
it-
I
think
they
do
so
by
this,
like
reading
the
amazon
docs
and
copying
what
they
did.
I
don't
think
that
there's
a
standard
written
down
anywhere
that
you
can
conform
to
and
and
the
other
problem
is
nothing
stops
amazon
from
changing
the
way
they
do
things
tomorrow
and
everyone
being
non-conformant
and
also
wouldn't
work
on
gcs
or
azure,
which
have.
F
Yeah,
but
this
is
this-
is
this
is
the
same
problem
we
had
a
decade
ago
with
you
know,
microsoft
and
mozilla
fighting
over
javascript
and
like
what
is
javascript
and
they
eventually,
someone
just
invented
a
standard
called
like
ecma
and
then
like
that.
Just
became
the
standard
because,
because
you
need
something,
that's
not
proprietary
and
that's
written
down
to
to
have
a
you
know
a
common
mode,
a
common
compatibility.
F
D
A
Do
you
do
you
suppose
that
if
we
came
up
with
a
new
we're
not
going
to
work
on
the
data
part
of
it,
we're
just
we're
going
to
work
on
the
the
provisioning
and
access
parts
of
it?
So
so,
given
that
and
given
that
others
for
the
data
parts
at
least
others
are
standardizing
around
s3,
do
you
still
think
there's
a
need
for
us
to
standardize.
F
F
D
F
This
is
not
the
point
of
cozy,
it
was
just
something
that
came
up
in
the
context
of
cozy
discussions
until
someone
decides
to
go.
Do
that
we're
stuck
with
you
know,
just
whatever
amazon
says
is
the
standard
is
the
standard
and
we
have
to
decide
if
cosy
wants,
to
also
inherit
that
you
know
blind
obedience
to
whatever
it
says
in
the
aws
docs
or
if
we
want
to
you
know,
take
draw
our
own
line
in
the
sand
and
say
this
is
this
is
what
we
do.
F
A
F
What
I'm
saying
is
we
if
we,
if
we
leave
it
all
opaque,
the
problem
goes
away
right
because
then,
if
then,
if
the
s3
standard
were
to
fragment
in
any
way,
it
would
be
somebody
else's
problem
right.
It
would
be
a
vendor
by
vendor
fragmentation.
We
wouldn't
be
a
cozy
problem.
If
we
try
to
say
whatever
the
aws
docs
say
is
say,
is
a
it's
a
access
policy.
F
That's
not
a
good
situation
to
have,
I
don't
think
so
like
making
it
opaque.
Just
says:
hey
it's
up
to
the
admin
to
go
figure
out
how
to
implement
a
read-only
policy
or
write-only
policy
or
full
control
policy,
and
it
might
happen
to
be
the
same
great
game
across
the
board
and
maybe
in
cozy
2.0,
we'll
standardize.
That.
G
We
just
need
to
have
enough
understanding
of
what
what
policies
are
interesting
for
kubernetes
to
define.
Not
just
you
know
it's
true
that
customers
want
to
define
a
lot
of
things,
but
the
the
reason.
The
question
is,
what
matters
to
defining
inside
kubernetes,
to
be
able
to
to
be
able
to
manage
due
to
orchestrate.
G
F
A
A
A
F
I
mean
you
could
have
an
opaque
field
that
was
vendor
specific.
That
was
like
the
name
of
a
config
map
in
a
specific
name
space,
and
then
the
driver
could
go
read
that
if
it
wanted
the
full
flexibility
of
a
config
map-
but
I
wouldn't
want
to
institutionalize
that
in
the
formal
api,
because
you
can't
remove
it.
A
Right
and
if
they
still
want
to
do
that
they
can,
they
can
specify
it
in
parameters
as
something
they
understand.
You
know
like
like
it
could
be.
Like
you
know,
in
csi,
in
the
in
the
in
the
parameters,
you
can
specify
something
that
you
know
like
kds,
dot,
io
dot,
csi,
slash
secret
name
and
secret
name
space
and
then,
and
then
the
csi
driver
or
the
csi
provisioner
goes
and
fetches
these
two
things,
the
secret
and
secret
name
and
then
and
then
gives
it
to
the
driver.
A
F
A
F
C
Yeah,
but
if
there's
a
parameter
for
it
like,
if
it's
a
like
what
I
want
to
say,
yes,
at
least
this
should
be
user,
can
request
a
user
need
to
like
if
it's
a
bucket
access
class
parameter
the
opaque
field,
then
admin
has
the
control
right.
If
I
need
to
if
from
a
user
point
of
view,
if
I
need
to
record
something
that's
bit
difficult
to
do
at
the
moment
right,
you
can
only
just
follow
what
admin
says.
F
Could
choose
to
have
no
flexibility
whatsoever
and
drivers
could
just
say:
hey.
Every
bar
is
going
to
be
full
control,
we're
going
to
ignore
all
the
parameters
and
that's
all
you
get,
whereas,
whereas
a
different
vendor
could
decide
to
make
a
very
powerful,
you
know
abstraction
in
the
bac
where
you
can
specify
all
kinds
of
policy
details
and
then
they
just
it
just
becomes
driver
dependent.
C
C
So
one
of
the
main
major
difference
is
this:
no,
so
here
this
bucket
taxes
and
bucket
access,
sorry
bucket
request
and
bucket
access
right
for
obc,
we
don't
have
that
difference.
It's
only
like
object
market
like
it's,
maybe,
like
you
can
say
it's
equivalent
to
bucket,
because
at
some
point
maybe
it
can
be
equivalent
to
bucket
access
request
at
some
point.
So
let
me.
G
A
A
F
The
netapp
driver
can
have
all
kinds
of
special
options,
but,
like
someone,
who's,
creating
a
pvc
should
never
know
what
those
are,
because
they
they
want
their
their
well
in,
maybe
not
everyone,
but
at
least
a
lot
of
users
would
want
their
pvc
to
be
portable
to
a
non-netapp
storage
controller
for
for
the
ones.
That
really
do.
F
A
H
A
If
you
wanted
to
specify
some
parameter
for,
for
you
know
for
the
bucket
owner
for
each
bucket
that
you
create
right
now,
you
know
either
either
the
driver
can
do
it
using
some
form
of
agreement
with
with
the
with
the
user.
So
something
like
if
the
br
starts
with,
say
xyz
then
make
it
a
user
make.
It
belong
to
user
xyz.
D
A
A
C
C
A
Yeah,
if,
if
there
is
any
of
these
designations,
preventing
you
from
moving
forward
like
you
know,
if
it
ends
up
being
something
that
that
is
completely
unworkable
for,
say,
ceph
or
any
other
driver,
then
that's
something
we
have
to
address.
However,
in
this
situation
we
do
have
a
few
workarounds
and-
and
you
know,
if
there
are
still
issues,
please
please
bring
it
back
up
and-
and
you
know
we'll
resolve.
A
C
A
B
C
A
A
Yes,
okay,
cool
all
right,
so
this
is
this
is
what
the
architecture
looks
like
and-
and
you
know
please,
you
know-
let
me
know
if
you
see
any
inconsistencies
here
from
what
we've
been
designing.
But
you
know
the
ten
thousand
feet
picture
looks
like
this:
I've
yeah
I've
messed
up
some
connection
on
my
machine.
A
Okay,
so
there's
a
cozy
controller
manager,
and
you
know
that
interacts
with
the
kubernetes
api
and
on
each
node
we
have
the
node
adapter
and
we
have
the
provisional
site
car
that
that
runs
alongside
the
driver
and
the
driver.
Provisional
sidecar
talks
to
the
kubernetes
api
and
the
driver
talks
to
object
storage
provider.
It's
it's.
A
simple
connection
here
makes
sense
right.