►
From YouTube: Ceph RGW Refactoring Meeting 2023-05-31
Description
Join us every Wednesday for the Ceph RGW Refactoring meeting: https://ceph.io/en/community/meetups
Ceph website: https://ceph.io
Ceph blog: https://ceph.io/en/news/blog/
Contribute to Ceph: https://ceph.io/en/developers/contribute
What is Ceph: https://ceph.io/en/discover/
A
B
Yeah
I
I
can
start.
Maybe
you
can
fill
up
other
things,
so
the
idea
is
to
try
and
tackle
the
S3
attribute
names.
So
attribute
names
are
the
the
like.
The
strings
like
x,
dash
AMZ
Dash,
something
something,
and
the
idea
is
that
those
are
relatively
long
strings.
B
B
There
exists
in
any
any
kind
of
intermediate
structure
that
we're
holding
this
information
inside
the
redis
Gateway,
and
we
also
put
them
inside
the
objects
so
and-
and
the
idea
is
that,
even
though
those
are
relatively
long
strings,
we
can
actually
hash
them
quite
easily
into
just
numbers,
because
there's
a
very
small
number
of
them,
I,
don't
know
in
a
in
a
setup
to
be
surprised
if
there
are
more
than
100
different
attribute
names.
B
B
So
there
could
be
a
couple
of
levels
for
this.
The
the
basic
one
would
be
to
just
do
that
inside
the
rgw,
so
that
the
client
would
continue
to
send
us
the
same.
The
same
messages
they
sent
before
and
we'll
just
do
the
translation
at
the
entry
point
and
use
those
numbers
from
now
on.
B
I
mean
after
a
pretty
short
period
of
time.
The
the
will
probably
like
we'll
see
all
the
different
attribute
names.
B
We'll
probably
see
all
the
different
attribute
names
that
exist
and
we
would
save
on
memory
allocations
and
and
so
on,
from
the
Raiders
Gateway
and
on,
and
so
that's
this
could
be.
One
level
of
optimization.
Second
level
of
optimization
would
be
to
actually
send
a
translation
table
to
the
clients
and
so
that
they
could
use
it
if
they
want.
C
B
So
if,
if
we
do
this
translation
at
the
entry
at
the
entry
of
the
rgw,
then
we
can
optimize
that
inside
the
rgw
memory
allocations
and
optimize
that
in
the
greatest
objects.
So
that's
that
could
be
one
level
optimization.
What
I'll
describe
a
second
level
would
be
to
actually
send
that
to
the
S3
client
and
I
mean
we.
We
don't
really
have
to
change
or
break
the
protocol.
For
that.
B
One
client
that
we
do
control
is
in
the
case
of
of
multicide.
So
when
the
client
is
another
rgw,
then
we
do
control
the
implementation
of
the
client,
and
we
may
optimize
that
in
this
case,
but
as
Gabby
said,
even
even
if
we
kind
of
put
aside
the
like
this
enhancement
to
S3,
which
is
always
kind
of
a
a
tricky
thing,
then
we
can
optimize
that
inside
the
rgw
and
in
addition
in
the
in
the
redis
objects
as
well.
C
On
our
side,
the
reason
we
wanted
to
try
and
optimize
the
art,
the
attribute
names
is
because
the
set
of
attributes
that
we
use
is
pretty
small.
The
objects
that
we
have.
C
We
could
have
many
millions
of
objects,
and
if
you
check
how
many
attributes
you
got
probably
a
few
dozens,
let's
exaggerate
and
say
1000,
separate
attributes
and
so
you'd
see
every
attributes
being
repeated
thousands
time
and
each
time
it's
Dynamic
allocation
of
a
short
string
and
when
the
object
node
is
brought
into
memory,
object
string
need
to
be
allocated
when
you
go
out
of
memory.
Deallocated.
C
It's
make
encoding
and
decoding
more
expensive
because,
of
course,
including
the
coding
strings
is
expensive,
and
it
also
means
that
the
data
that
the
attribute
name
is
not
continuous,
with
the
object
with
the
object
node,
because
inside
the
object
node,
we
have
pointers
into
separate
addresses
if
we
are
able
to
replace
them
with
short
integers,
and
we
are
thinking
about
u
in
16,
just
two
bytes,
which
would
allow
us
to
use
c4k
different
attributes
names
which
is
really
unrealistic
to
ever
get
in
this
direction.
C
So
to
buy
it
instead
of
16
or
maybe
more.
In
some
cases,
it
could
be
as
large
I
think
as
128
byte,
but
I.
Think
16
byte
is
a
more
reasonable
attribute
names,
but
what
we
have
been
saving
is
not
just
16.
Byte
goes
down
to
two
byte
is
move
away
from
dynamical
location
memory,
fragmentation
and
con,
and
the
cost
of
encoding
decode
foreign.
C
That
we
process
attribute
name,
so
we
search
it's
a
string,
compare
instead
of
just
doing
an
integer
compare,
which
is,
of
course
much
faster
and
internally
in
the
OSD.
We
really
don't
care
about.
Your
attributes
from
our
perspective
attribute
is
just
it
should
match
something,
and
if
everything
that
we
are
processing
it,
we
only
see
two
byte
integers,
that's
much
faster
alteration
on
our
side
and,
of
course
we
need
to
preserve
the
original
name
in
case
somebody
needed.
C
C
B
A
C
B
Gabby
I
think
what
what
Casey
is
asking
is
saying
the
API
to
redis
doesn't
allow
that.
D
D
C
D
C
B
So
the
main
question
is
I
mean
it
would
require
changes
both
to
the
OSD
and
the
rgw
and
the
API.
Do
you
see?
Do
you
see
the
value
here?
Do
you
think
this
is
something
we
should
actually
pursue
in
implementation?
Okay,.
A
So
I
mean
it
might
make
more
sense.
Instead
of
changing
all
of
the
keys
to
un16,
adding
a
separate
API
that
supports
the
16-bit
keys
and
start
converting
things
to
use
that
instead,
but
still
I,
think
there's
complexity
around
mappings
between
strings
and
numbers
and
also
supporting
things
essentially
like
user
specified
keys
for
rgw
S3
has
user-defined
metadata
and
we
map
those
directly
into
attribute
names.
B
Metadata
strings,
but
is
this?
Is
this
really
the
case
I
mean?
Have
we
seen
thousands
of
usually
defined
S3
attributes
I,
don't
think
so.
B
I
mean
the
current
number
of
attributes
is,
is
probably
under
a
hundred,
and
if
we,
if
we
I
mean
if
we,
if
we
take
the
user
defined,
it
could
maybe
go
to
a
couple
of
hundreds,
maybe
a
thousand.
So
if
we
have
64k,
this
is
kind
of
way
way
beyond
what
we
should
have,
but
even.
B
B
C
Whenever
you
see
a
string
for
the
first
time
you
access
hashtag
looking
for
it
if
it
exists,
the
hashtag
is
going
to
give.
So,
let's
say:
if
it
doesn't
exist,
you
allocate
an
index
or
running
incrementing
index
and
assign
it
to
it
saying
the
key
is
going
to
be
the
string
name
and
the
data
is
going
to
be
the
allocated
index.
C
C
D
C
I
was
thinking
using
rocksdb
object.
C
E
My
other
thought
on
that
is
like
how
frequent
are
the
user
defined
ones,
and
is
it
really
worth
keeping
Dynamic
table
for
that
reason,
or
does
it
make
sense
just
to
have
a
predefined
statically
defined
set
of
mappings
for
the
common
ones
that
are
fine
defined
for
S3
and
leave?
The
other
ones
still
have
string
keys
since
probably
a
marginal.
B
Case
so
the
user-defined
ones,
I
mean
the
the
cold
user
defined,
but
in
fact
they're
used
by
applications
right.
So
it's
not
like
every
end
user
of
the
system
is
defining
like
any
end
user
or
millions
of
the
users
of
the
system
is
going
to
Define
their
own
attributes
nobody's
doing
that.
So,
if
you
have
an
application
using
the
system,
they
they
can
add
their
own
attributes.
So
they
can
store
some
some
extra
information
that
they
need,
and
so
in
a
given
system
they
use
a
defined
one.
B
A
That,
though,
I
mean
the
the
protocol
itself
lets,
you
give
arbitrary
metadata
names
and
values.
Different
applications
might
use
different
ones
hard-coded,
but
a
given
cluster
could
have
all
kinds
of
different
applications
running
at
the
same
time.
So
I
really
think
we
need
the
flexibility
of
our
arbitrary
strings
without
the
cost
of
a
globally
consistent
table.
A
B
B
C
So
so
we
we
could
do
a
few
things.
So
first
we
could
just
create
a
very
simple
hostable
encount,
just
to
see
how
this
thing
behaves.
We
could
just
load
some
testing
utility
and
verify
the
actual
number
of
different
attributes
used
by
client.
Another
option:
if
we
want
to
have
unlimited
attributes,
then
use
four
byte
we
four
byte
your
set
is
virtually
unlimited.
I
cannot
imagine
somebody
having
4
billion
separate
attributes
unless
somebody
is
doing
something
extremely
crazy,
but.
C
Sorry,
why
are
you
racing
if
it's
something
that
the
OSD
is
going
to
determine
and
you
guys
just
going
to
get
the
answer
from
the
OSD,
so
the
first
access
to
the
OSD
you're
going
to
use
the
string
and
then
the
OSD
is
going
to
to
tell
you
you
know
what
from
now
on
on
the
reply
message.
Please
use
this
integer
instead,
that's
easier,
so
you
guys
never
have
to
worry
about
persistency.
That's
something
the
OSD
need
to
manage.
C
Persistency
consistency,
race:
any
of
these
concept
should
be
internal
to
theocity.
C
C
B
B
A
F
C
Because
the
rgw
definitely
have
to
see
the
full
stream,
so
there's
no
way
to
move
around
it.
If,
anyway,
you
have
the
full
stream,
then
go
to
the
hash
table
and
get,
and
then
you
can
get
rid
of
it.
Otherwise
you
got
it
anyway
and
you
need
to
do
string,
compare
and
every
operation,
but
you
also
require
the
OSD
to
do
this.
C
R2W
is
going
to
get
from
the
OSD
trusted
translations.
C
C
No,
they
should
just
they
should
replicate
applicator,
but
the
RSW
can
maintain
a
local
cache
which
doesn't
need
to
be
replicated
and
it's
okay
to
lose
it
it.
You
crush
the
next
time.
You
you,
you
restart
you'd,
get
the
full
tablet
from
the
OST,
so
the
OSD
should
maintain
the
persistency
rtw
should
should
have
a
shadow
copy,
but
it's
not
it's
not
the
owner
of
the
table.
It
just
have
a
copy
of
the
table.
A
C
C
B
Right
but
we
said
that
globally,
we're
going
to
have
maybe
a
thousand
different
attribute
names
right
right,
so
so
when
an
rgw
first
comes
up
or
whenever
it
has
a
cache
Miss
in
the
local
in-memory
hash,
which
is
kind
of
inexpensive
they're,
going
to
fetch
the
table
from
the
OSD
right
and
assuming
the
I
mean
the
the
change
rate
is
really
small
like.
If
we
have
a
thousand
attributes,
then
they're
not
never
going
to
change
after
the
initial
ramps.
F
I
mean
my
understanding:
is
it's
not
going
to
fetch
the
actual
table
from
the
OST?
It's
going
to
send,
say:
I,
don't
know
the
string.
It's
going
to
send
the
string
down.
The
OSD
is
going
to
do
a
lookup
which
involves
a
synchronization
event
and
pass
back
a
number,
and
this
is
going
to
happen
once
for
each
attribute.
No,
that.
C
Was
how
you
describe
the
API
is
working
okay,
let
me
redescribe
it
when
the
RW
doesn't
have
the
integer
it's
going
to
do
the
I
O,
with
a
string
the
OSD
going
to
either
find
the
if
the
if
the
string
exists
in
the
OSD
tables
is
going
to
find
it
act
on
it.
There's
no
synchronization
event
here
and
on
the
replay
to
the
rjw
is
going
to
piggyback
the
value
and
say
from
now
on.
Please
use
this
value
if
the
value,
if
the
string
doesn't
exist
on
the
OSD,
the
OSD
will
need
to
synchronize
with.
C
This
information,
sorry,
this
table
entry.
So
that's
a
synchronization
event
which
going
to
happen
on
every
time
an
OSD
sees
a
new
value.
The
question
is
how
many
unique
value
exist
in
the
system
if
the
number,
if
every
rjw
you
have
a
unique
set
of
attributes,
then
your
right
and
it's
maybe
one
thousand.
G
F
F
G
A
B
A
G
If
we
did
want
to
do
something
like
with
this,
this
within
the
rgw,
like
I,
think
we
could
do
I
think
we
could
do
an
interesting
split
potentially,
where
the
OSD
does
its
own
version
of
compression
behind
the
scenes
without
telling
us
that,
similarly,
the
OSD
could
have
its
own
in-memory.
Basically,
its
own
in-memory
table
of
interred
symbols,
put
things
out
when
they
get
older
and
just
don't
bother.
G
G
If
we
want
to
avoid
having
to
store
and
do
a
bunch
of
string
comparisons
in
the
in
the
rgw
per
se,
we
could
simply
have
an
in-memory
table
of
strings
that
we
populate
while
we're
running
just
by
what
we're,
seeing
and
cut
out
the
and
cut
out
the
synchronization
between
the
rgw
and
the
OSD.
G
That
way
like
we'd,
still
ingest
strings
as
we
see
them,
and
if
we
are
seeing
a
constant
workload
that
should
all
get
populated
relatively
soon
after
startup,
we
might
still
be
able
to
get
the
benefit
of
replacing
string
comparisons
with
integer
comparisons,
and
we-
and
if
we
did
something
like
that,
since
it's
not
a
global
shared
resource,
we
could
simply
do
an
lru
type
system
where
we
evict
the
where
every
now
where.
If
we
start
pushing
our
numbers,
we
evict
the
least
the
least
recently
used
1
000
and
go
that
way.
G
I
would
write
the
strain
to
rados
the
idea.
That
was
simply
the
idea
of
how,
since
you
had
mentioned
taking
since
you
had
mentioned
the
time
in
comparison,
use,
I
was
just
trying
to
think
if
there
was
a
way
to
to
still
get
that
Advantage
without
necessarily
having
to
without
having
to
pay
for
the
rate
of
synchronization
costs.
G
H
F
D
C
The
motivation
here
was
that,
if
you
guys
could
do
this
work
or
part
of
the
work
and
from
my
perspective,
the
work
means
the
translation,
not
the
persistency
of
the
table.
Then
you
could
say
all
the
effort
needed
by
OSD,
because
the
OSD
wouldn't
have
to
deal
with
streams,
since
you
anyway
are
translating
them.
B
So
I
think
I
think
what
Casey
said
before
regarding
the
limitations
of
return
value
from
right
operation
is
is,
is
a
true
problem
here.
So
maybe
maybe
we
go
because
you
understand
Gabby
they.
What
we
described
as
the
API
between
the
the
client
and
the
OSD
replying
with
couple
of
values
is,
is
a
problem.
If
this
number
of
values
is
too
high,
there's
a
there
are
some
split
limitations
on
the
on
the
right
operations
to
the
OSD.
B
H
Do
the
folks
from
Bloomberg
on
the
call
know
how
many
user-defined
attributes
their
cluster
or
clusters
have
or
have
any
idea
if
the
numbers
low
like
in
the
four
digits
or
if
it's
much
higher.
I
So
the
user
defined
attributes
are
those
a
matter
of
like
X,
EMZ,
yeah.
F
A
So
I
mean
this
would
come
up
with
CFS,
also
right
because
as
I
understand
that
they're
mapping
the
posix
X
adders
and
X
editors
in
the
same
way,
which
could
have
arbitrary
names
and
values.
B
Well,
I
guess
they
can
take
a
similar
approach
if
the
I
mean
or
the
non-user
defined
ones,
for
the
ones
that
are
I
know.
If
they
have
such
attributes
that
always
come
with
every
file,
then
maybe
they
can
just
hard
code
them
and
then
just
pass
them
as
numbers
to
the
OSD.
The
obviously
doesn't
have
to
really
care
about
that.
A
A
A
B
Well
for
the
non-user
defined
ones,
we
for
sure
we're
doing
like
we're
doing
lookups,
which
has
the
cost
of
hashing
those
strings.
And,
of
course
we
have
the
memory
allocation
calls
and
for
the
non-use
defined
ones.
It's
the
same
string
for
each
and
every
request.
A
C
Maybe
if,
if
we
decide
to
go
on
this
project,
then
we're
probably
going
to
be
some
step
in
which
the
background
process
would
start
converting
conversion,
but
that's
still
in
the
future,
I
would
start
with
brand
new
or
on
a
new
release
that
everything
on
the
new
release
for
new
installs.
Sorry,
not
just
news
just
new
installs
and
then
you
can
add
a
scrub
process
which
should
go
back
in
time,
should
go
back,
should
go
over
existing
object
and
fix
them
can
also
be
done
lazy.
So
every
time
the
object
is
brought
into
memory.
I
B
Oh
there's
no
tracker:
this
is
just
to
kind
of
brainstorming
the
idea,
but
for
sure
we
can,
we
can
I
mean
we
can
summarize
that
and
maybe
an
email
send
that
to
the
community
and
put
a
tracker.
Sorry.
The
information
could
be
aggregated
in
this
tracker.
Yes,.
A
So
I
mean
has
this
been
discussed
with
the
rados
team
at
all
I
think
yeah
I
would
want
to
get
their
feedback
on
the
the
OSD
side.
Maybe
a
self-developer,
monthly
or
similar
meeting
would
be
a
good
way
to
erase.
A
A
Foreign
to
the
pr
that
he
opened
about
it,
but
I'm
still
not
clear
on
on
the
motivations
there.
J
Kind
of
following
up,
we
had
a
discussion,
probably
a
couple
of
weeks
ago
or
maybe
three
weeks
ago,
about
the
notification
retry,
where
you
all
was
talking
about.
Should
we
come
about
away
on
how
to
retry
the
like?
Also
during
our
current
notification
testing,
what
we
observed
was
the
Gateway
is
very
aggressively
stand,
so
we
had
a
persistent
notification
setting
and
I
I
our
broker
Kafka
broker
was
on,
but
somehow
it
was
not
able
to
send
him.
J
It
was
kind
of
aggressively
sending
all
the
notification
and
crashing,
and
every
time
we
brought
up
the
Gateway
it
kept
on
sending
so
I
was
just
hoping
an
opinion
here.
Like
is
not
there
a
way
where
the
the
notifications
or
the
processing
notification
acknowledgments
are
retried
for
some
time
and
then
kind
of
give
up,
instead
of
just
doing
it.
Every
time
like
we
did
discuss
this
last
time,
but
I'm
not
sure
if
he
reached
the
conclusion
on
this.
B
Yeah
I
mean
you
have
to
bear
in
mind.
This
was
because
of
a
bug
that
we
fixed,
so
you
know
we
we
shouldn't
design
a
software
on
bugs.
We
should
design
a
software
around
kind
of
system,
behaviors
or
network
behaviors
or
environment
Behavior.
So,
putting
aside
the
specific
bug
that
was
causing
the
crash,
do
you
want
to
not
see
retries
of
notifications?
Let's
say
after
the
Kafka
broker
is
brought
up
again
or
the
network
recovered,
or
something
like
that.
J
B
The
current
behavior
is
that
eventually
you
will
get
pushback,
so
there
is
a
queue
of
retries
and
when
this
queue
fills
up,
then
it
pushes
back.
If
you
define,
if
you
define
that
those
notifications
need
to
be
sent
for
a
specific
bucket,
then
eventually,
when
the
queue
fills
up,
maybe
they
do
take
a
couple
hours
I,
don't
know,
then
they'll
be
pushback
on
the
client
saying
like
busy
or
something
like
that,
and
you
would
see
that
the
clients
need
to
slow
down
now.
B
J
B
B
Right
at
all,
right
right,
so
so
let
let's
take
like
scenario
where
there's
a
network
disconnect
or
the
Kafka
broker
is
down.
So
the
current
behavior
I'm
not
saying
this
is
the
the
best
thing
or
the
only
one
that
we
should
have,
but
the
current
behavior
is
that
they'll
be
retired
indefinitely,
because
I
I
don't
know
when
to
give
up
and
eventually
when
the
queue
fills
up,
then
I
would
push
back
on
the
S3
client.
B
It
might
take
some
time,
but
eventually
I'll
push
back
on
the
S3
clients.
Now
I
mean
one
one
design
option
could
be
to
add
like.
J
B
On
the
notifications
and
then
eventually
I
would
give
up
and
throw
them
away,
and
then
maybe
the
queue
won't
won't
fill
up
because
I
will
be
deleting
stuff
from
there.
This
is
one
option
or
I'm
open
to
other
options,
as.
J
Well,
just
a
good
couple
of
questions
before
we.
What
is
it
the
size
of
the
queue
that
it
decides
when
it's
full,
like
format,
do
we
is
it
configurable.
B
So,
currently
it's
a
single
single
radius
object
right.
So
it's
for
mags,
which
means
notifications
are
usually
a
couple
of
hundreds
of
bytes,
depends.
B
Yeah
and
depends
on
the
the
the
length
of
the
bucket
names
and
so
on.
So
it's
not,
but
it's
usually
not
more
than
one
kilobytes.
It's
really
that
that
big,
so.
J
B
B
No,
no!
No!
No!
No!
No!
It's
before
it's
before
you!
They
put
operation.
So
before
the
put
operation,
I
I
put
like
the
queue
works
with
a
kind
of
a
reservation
system.
So
whenever
I
see
a
put
operation
that
needs
a
notification,
I
make
a
reservation
on
the
Queue
and
if
I
cannot
make
the
reservation,
then
I
immediately
reply.
J
B
And
if
I
can
make
the
reservation
that
I
know
that
after
I
do
the
actual
put
when
I
try
to
send
a
notification,
which
means
to
put
something
in
the
queue
I
know,
I
have
a
reservation,
so
I'll
have
space
in
the
queue.
So
that's
how
the
system
works.
Okay,.
J
B
B
B
Option,
sorry,
it
was
one
second.
Another
option
was
to
to
have
like
instead
of
this
message:
Cube,
which
is
kind
of
a
limited
side
to
have
like
an
infant
size
or
use
the
the
message
fifo
and
then
it
concatenates
multiple
objects.
So
you
can
have
much
much
larger
size,
I'm,
not
sure.
J
So
again,
one
way
to
tackle
this
is
multiple
things.
Iphone
is
one
option
to
increase
there,
but
another
option
is
to
delete
after
a
few
replays
yeah.
So
again,
that's
that's
the
reason.
I
kind
of
open
this
I
don't
know
what
appropriate
solution
would
be
to
this,
but
it
would
be
nice
to
brainstorm
on
this.
B
Yeah
I
can
start
an
email
thread
on
the
on
the
user.
Mailing
list
kind
of
I'm
just
gonna
list
a
couple
of
options
and
would
be
great
if
you
can
kind
of
reply
with
definitely
I
think
and
then
maybe
other
people,
like
other
users
of
saffron
notifications,
can
can
chime
in
and
say
what
is
the
best
option
for
them.
I
mean
we
can
Implement
all
kinds
of
things,
but
I
would
like
to
implement
something
that
is
most
useful
for
most
users.
So.
J
Yeah
I
mean,
of
course,
we
could
always
have
an
option
of
configuring
as
well.
So
this
way
you
can
cater
multiple
people
as
well
like
yeah.
We
can
discuss
more
on
this
yeah,
so
that
was
another
thing
and
then
yeah.
The
follow-up
for
this
was
the
cleanup
thing
so,
as
you
said
right
like
if
the
Kafka
broker
is
down
or
they
usually,
if
you
consider
doing
it
a
5-4
object,
the
this
space
seems
to
be
there.
So
if
the
tenancy
is
deleted,
should
we
go
and
delete.
B
So
so
currently
the
I
mean
currently
when
you
delete
the
topic
from
the
Venice
Gateway
acli.
Let's
get
my
admin,
then
the
actual
queue
is
not
deleted,
but
this
is
when
you're
deleting
from
like.
After
we
fix
the
bug.
Oh,
if
you're
deleting
from
the
rest
API
from
from
yeah
three,
then
the
queue
should
be
deleted.
I
mean
this
is
tested.
Maybe
there's
a
bug
there
that
doesn't
work,
but
this
is
tested.
So
when.
G
B
Delete
the
queue
then
I
don't
see
any
more
retrials
or
anything
in
this
case.
Your
question
regarding
the
tenant
I
think
this
is
kind
of
a
wider
question.
Do.
H
B
Have
like
Cascade
deletions
of
anything
when
you
delete
the
tenant,
I'm,
not
sure
yeah,
okay,.
A
Tenant
is
not
actually
a
thing,
so
you
can't
delete
it.
I
mean
you
could
delete
all
users
under
attendant,
but
nothing
knows
nothing's
keeping
track
of
whether
a
tenant
name
is
used
or
not
they're
just
arbitrary
strings.
We.
J
J
Isn't
that
part
of
user
as
well
you
remove
user
and
Purge
data
that
will
remove
all
the
data,
all
the
buckets
Associated
to
that
user,
correct
right.
So
in
that
case,
First
Data
should
also
go
ahead
and
delete
the
the
queue
as
well
right.
B
J
The
confusion
here
is
I
believe
from
what
I
understand
here:
there's
one-on-one
mapping
between
a
tenant
and
S3
user.
They
could
be
some
users,
but
then
a
queue
is
specific
or
you
create
a
notification
system
specific
to
a
tenant
right
and
then
Associated
to
a
bucket.
But
if
that
tenant
itself
does
not
exist,
that
Q
doesn't
make
any
sense.
A
There
tenant
is
a
thing
that
originally
came
from
Swift
and
it's
essentially
just
a
namespace,
that's
isolated
from
other
tenants,
so
you
create
users
under
one
tenant
and
they
can't
view
anything
outside
of
that.
So
you
can
create
several
users
and
several
buckets
under
a
tenant
right.
A
J
B
So
so
I
mean
there's
a
distinction
here.
The
notifications
are
tied
to
a
bucket,
so
they
could
be
tied
to
a
user
and
also,
if
you
delete
a
bucket,
then
all
the
notification
configuration
is
automatically
deleted.
The
topics
are
not
tied
to
a
bucket
or
to
a
user
at
all,
the
namespaced
by
tenant,
but
they're,
not
tied
to
a
user
or
a
bucket.
B
Yeah
yeah
just
a
namespacing
thing
for
the
for
the
topics
but
they're
not
tied
to
a
specific
user
or
to
a
specific
bucket
I
mean
you.
Can
you
can
have
many
buckets
sending
notifications
to
the
same
Kafka
broker
so.
J
B
True
so,
but
that
would
be
okay,
so
that's
kind
of
a
different,
that's
kind
of
a
different
thing,
so
you
have
to
have
user
credential,
because
this
is
how
S3
works,
but
there's
no
association
of
this
topic
to
the
user.
B
Any
any
user
can
use
this
topic.
We
need
probably
to
add
some
kind
of
a
capability
mechanism
to
make
sure
that
only
administrators
can
create
and
and
delete
or
modify
topics,
because
this
is
really
a
system-wide
definition.
It's
not
a
it's,
not
a
user
or
a
bucket
specific
definition.
It's
only
that,
because
it's
done
over
an
F3
interface
or
to
be
honest
over
kind
of
an
SNS
like
interface,
then
then
you
have
to
have
some
kind
of
credentials,
but
other
than
that,
there's
no
tying
of
the
topic
created
to
the
user.
B
D
B
We
probably
need
to
add
kind
of
a
capability
verification
mechanism
to
make
sure
that
this
user
is
allowed
to
create
topics
but
other
than
that.
It's
like
an
administrator.
So
when
a
diminisher
does
something,
then
that
they're
not
tied
to
the
user,
which
is
the
administrator
okay,
I.
J
D
A
A
Thanks
a
lot
and
Shilpa,
are
you
on
the
call
yep
hi
I
was
just
wondering
if
you
had
a
quick
update
on
multi-site
testing
for
Reef?
Do
we
have
results
from
Mark
on
that.
I
Do
a
400
million
sync
workload
successfully
on
relief,
but
I
I,
don't
know
if
that
I
think
we
would
want
to
be
able
to
integrate
it
in
some
way
Upstream
but
yeah.
We
do
have
results
for
leave.
J
I
This
is
just
for
the
The
Reef
release.
It's
not
specific
to
anything.