►
From YouTube: Ceph Developer Monthly 2021-08-04
A
Okay,
so
hello,
everyone,
my
name,
is
shreya
sharma
and
I'm
a
gsoc
intern
with
the
dashboard
team.
For
the
past
two
months,
I've
been
working
on
the
project
reporting
sufficient
from
the
safe
dashboard.
The
problem
statement
was
that
when
a
safe
user
encounters
an
issue,
they
gathered
all
the
relevant
information
and
reach
out
to
the
developers
using
either
the
mailing
list,
or
they
file
an
issue
with
the
self
issue
tracker.
A
A
A
So
this
would
widen
the
funnel
as
it
would
be
convenient
for
users
to
report
issue.
So
we
expect
more
people
to
report
issues
now.
This
will
also
provide
less
engines
while
working
since
they
don't
have
to
you
collect
the
information
from
terminal
and
you
like
go
to
gmail
or
they
prefer
or
send
an
email
file
in
issue
tracker.
They
can
simply
report
the
issue
from
the
cli
itself
or
the
safe
dashboard.
A
This
would
also
mean
ultimately,
a
better
experience
and
to
report
an
issue.
We
take
the
bare
minimum
information
required
to
report
it.
That
is
the
project
name
where
they
encountered
the
bug
or
the
feature
request
they
have
for
the
project.
The
tracker
type,
which
is
a
bug,
feature
the
subject
and
the
description
of
the
issue.
A
A
So
for
this,
our
initial
approach
was
to
use
the
python
redmi
library,
but
after
the
research
we
found
out
that
it
is
not
available
in
saint
os.
It's
not
a
package
there,
so
we
decided
to
go
for
thor
requests
but
even
like
the
selfish
tracker
is
based
on
red
mine
and
is
well
documented,
but
even
the
curl
calls
that
were
given
in
the
documentation
did
not
work
when
I
tried
them
using
the
current
calls.
A
All
I
got
was
server
and
I
was
not
able
to
track
down
the
problem,
so
I
so
our
mister
suggested
me
to
use
wireshark
what
we
did
was
we
create
a
center
python
redmine.
We
use
this
library
to
send
a
request
to
create
an
issue
and
we
try
to
fire
fireshark
to
lock
the
request,
but
python
redmine
library
does
not
let
us
use
the
http
request
and
https
request
would
be
encrypted.
So
we
were
not
able
to
use
viral
shark
either,
but
ultimately
we
found
some
python
logging
libraries
that
we
used.
A
A
Then
a
prerequisite
for
filing
an
issue
is
an
account
on
the
safe
issue
tracker
like.
Ideally,
it
would
be
better
if
we
do
not
ask
the
users
to
create
an
account
on
the
safe
issue
tracker,
but
the
main
reason
that
we,
authentic,
like
we
verify
the
account,
is
to
avoid
spamming.
So
we
require
the
user
to
give
issue
tracker
api
key
to
avoid
spamming
and
to
authenticate
the
user
when
the
request
is
sent,
so
they
must
have
an
account
of
the
sufficient
tracker.
A
Well,
so
we
have
two
options
to
report
issues
the
cli
and
the
dashboard
user:
the
interface
using
this
fcli.
We
first
set
the
api
key
using
the
command
set
dashboard,
set
safe,
tracker,
api
key
and
then
giving
the
file
name.
Where
the
api
key
is
stored.
We
can
then
create
an
issue
using
this
command
safe,
dashboard
feedback
create.
We
would
then
give
project
name,
tracker
type,
subject
and
description.
This
is
an
example
and
we'll
be
having
a
look
at
the
live
demo
soon.
A
Then
we
have
the
dashboard
user
interface
in
which
the
user
can
go
to
settings
and
he
would
have
an
option
to
raise
an
issue
when
he
clicks
on.
There
is
an
issue
he'll
be
presented
with
this
model,
so
he
has
to
select
the
project
name
from
the
list
of
drop
down
project
names.
He
will
select
the
tracker
name
from
the
list
of
drop
down
trackers,
which
would
be
bargain
feature
he
has
to
get
the
subject
and
the
description
also.
A
A
A
A
A
A
A
Let's
say
the
description
is
mighty
liner,
so
it
would
be
easier
for
the
issuer
to
pass
the
file
name,
as
it
would
be
easier
for
user
to
pass
the
file
name
as
an
argument
to
take
the
description
for
there,
and
we
can
also
add
the
duplicacy
to
it
using
the
python.
What
do
you
mean
plugin,
which
is
a
plugin
for
that
mine?
A
B
C
A
D
B
We
could
provide
an
anonymous
or
a
public
api
key.
Let's
see
that
that's
it's
linked.
Probably
we
will
start
having
the
you
know,
classical
spam
attacks
that
we
had
in
the
past,
but
maybe
we
can
update
that.
I
mean
on
a
regular
basis,
just
away
that
or
but
definitely
I
think
it
would
be
great
for
every
user
to
be
able
to
report
this
without
and
having
to
go
through
the
process
of
creating
an
account
getting
approval
and
so
on
so
yeah.
C
E
E
G
G
D
I
wonder
if
obfuscating
the
key
at
some
level
would
be
sufficient.
I
mean
I.
I
would
assume
that
most
of
these
places,
where
they're
trying
you,
when
they're
spamming
to
redmine
they're,
probably
like
searching
for
any
redmine
instance,
that's
open
or
they're
searching
for
red
nine
keys.
But
if
it's
obfuscated
such
that
they
actually
have
to
decipher
and
reverse
engineer
some
code
instead,
specifically
in
order
to
spam
to
specifically
the
set
tracker
that
might
be
sufficient
to.
G
D
Yep
I
have
like
two
small
comments.
This
looks
great,
but
the
dialogue
where
you
actually
submit
the
issue.
The
description
should
probably
be
a
text
box
instead
of
just
a
single
line,
so
you
can
actually
have
a
virtuous
description
yeah
that
one
and
then
also
once
it's
successfully
submitted,
instead
of
just
showing
the
issue
number,
it
could
just
link
to
the
tracker.
C
H
Yes,
so
at
the
back
end
of
redmine,
we
can
create
a
unique
source
for
this
specific
type
of
issue
creation,
which
means
it
comes
either
from
the
cli
or
the
dashboard
in
an
automated
way,
and
then
we
can
pre-populate
it
in
your
project
and
when
the
issue
is
created,
you
can
see
and
in
red
mine
issue
itself
the
source
field
populated,
and
we
can
choose
a
name
for
it.
I
don't
have
a
good
idea
now.
A
E
H
H
B
B
B
I
have
a
question
and
well
it's
not
only
for
serious
for
everyone
based
on
the
reception.
From
this
feature,
I
was
wondering
if
it
would
make
sense
to
move
part
of
the
functionality
to
a
separate
module
or
something
because
right
now,
this
would
require
the
dashboard
to
be
running
and
I'm
wondering
if
it
would
make
sense
for
users
not
running
the
dashboard
to
still
be
able
to
report
issues
so
maybe
having
some
of
this
functionality
in
a
standalone
module
and
the
dashboard
just
consuming
the
cli
interface
or
something.
I
Okay,
so
my
name
is
aryan
and
I'm
I
I've
been
working
with
steph
under
google
summer
of
code
for
the
past
three
months,
and
I
was
my
project
was
visual
regression
testing
of
dashboard,
so
today
we're
gonna
discuss
like
so.
This
is
what
we
are
going
to
discuss
today.
We
are
going
to
discuss
types
of
testing
and
we
are
using
in
this
what
we
are
already
using
in
the
safe
dashboard
and
even
even
after
we
have
like
all
these
kind
of
testing.
Why
do
we
need
a
more
yes.
I
And
yeah,
so,
even
if
we
have
all
these
already
existing
testing,
why
do
we
need
more
tests
and
the
work?
And
what
exactly
is
visual
impression,
testing
the
workflow
for
it
and
the
different
tools
available
for
it?
The
criteria
I
use
to
select
the
best
tool
for
this?
Have
dashboard
and
we'll
discuss
the
most
popular
tools
and
why
they
are
not
why
some
of
them
are
not
useful
and
we'll
discuss.
Pixel,
graphics,
bitmap
matching
the
algorithm
how
it
works.
I
Yeah
so
currently
in
the
safe
dashboard,
we're
using
three
kinds
of
testing
we're
using
at
the
bottom
we're
using
unit
testing,
and
it's
really
good.
If
you
want
to
test
small
functions
and
small
components
and
unit
tests,
should
focus
only
on
the
described
purpose
and
shouldn't
try
to
test
other
things
in
the
same
blocks
and
yeah
we're
using
on
the
second
level
we're
using
integration
testing
where
we
combine
all
these
unit
tests
and
see
how
they
work
together.
I
Also,
the
test
should
verify
that
certain
items
are
displayed,
as
a
user
would
see
them
when
clicking
through
the
ui,
but
this
is
where
it
fall
sometimes
falls
behind
yeah.
So,
instead
of
dashboard,
we
already
have
this
whole
testing
therapy
setup
and
running.
But
the
question
is:
why
do
we
need
visual
regulation
testing?
I
B
B
I
Now
sorry
for
that
yeah
yeah
I'll
just
begin
from
here,
so
instead
we
already
have
this
whole
testing
limit
that
we
see
here.
So
the
question
is:
why
do
we
need
visual
regression
testing?
So
the
first
thing
is
visual.
Bugs
are
changing
issues
and
rendering
violation
is
not
what
functional
testing
tools
are
designed
to
catch
function,
testing
measures,
functional
behavior
and
that's
what
it
does
so.
Visual
regression
testing
also
helps
us
in
picking
up
css
bub,
which
we
don't
have
any
testing,
for.
I
It
also
helps
us
in
it's
also
where
sometimes
user
is
not
able
to
see
critical
buttons
or
are
they
or
they
are
in
on
unclickable
positions
and
stuff.
The
other
thing,
the
other
very
useful
feature
of
this
kind
of
testing
is.
It
helps
us
in
making
very
informed
decisions
on
breaking
changes
like
when
you
have,
when
you're
doing
large,
refactoring
or
just
upgrading
frameworks
or
libraries
which,
when
you're
moving
from
bootstrap
v4
or
to
these
bootstrap
files
and
stuff.
I
Let's
look
at
some
of
the
examples
here
in
the
first
image
we
have,
we
can
see
that
the
login
page
is
awkwardly
positioned.
This
is
a
css
buck,
resulting
in
a
user,
possibly
not
being
able
to
access
the
elements,
but
this
will
pass
all
our
all
our
testing
pyramid,
like
all
the
tests,
because
the
elements
are
still
existing
and
the
second
image
is
of
the
official
material
gui
website.
Material
ui
is
a
very
well
very
popular
ui
framework
developed
by
google.
I
I
This
is
the
breaking
change,
so
it's
really
good
for
that.
So
to
understand
now,
let's
move
ahead
and
understand
what
visual
regression
testing
is
so
to
understand
what
visual
diction
testing
is.
We
need
to
first
understand
what
regression
testing
is.
The
regulation
testing
is
used
to
verify
that
any
system
changes
do
not
interfere
with
the
existing
features
and
our
code
structure
is
the
reason.
Recreation
testing
is
a
part
of
almost
every
two
suite
in
software
development.
I
It
is
common
for
devs
to
change
and
change
or
add
a
section
of
code
that
and
having
it
unintentionally
disrupt
something
that
was
previously
working.
Just
fine
visualization
testing
applies
the
same
logic,
but
confines
testing
to
the
visual
aspects
of
the
software.
So
only
the
visual
aspect,
in
other
words
it.
It
checks
that
the
code
changes
do
not
break
any
aspect
of
the
code
of
the
software's
visual
interface.
I
A
visual
regression
test
checks
also
checks
that
the
user
see
will
see
what,
after
any
code,
changes
have
been
executed
by
comparing
screen
shots
taken
before
and
after
the
code
changes.
This
way,
yeah.
Okay.
So
now,
let's
talk
about
how
visual
the
regression
tests
generally
work,
so
the
work
of
the
workflow.
For
that
usually
looks
like
this.
I
So
now,
let's
talk
about
the
different
tools
you
can
use
for
visualization
testing.
There
are
plenty
of
tools
you
can
find
on
the
internet
to
implement
these
tests
and
to
select
one
two
for
the
step
dashboard.
I
have
to
come
with
come
up
with
a
robust
criteria,
so
this
is
the
criteria
I
came
up
with
there,
so
the
criteria
is,
it
has
to
have
like
minimum
manual
effort,
easy
cyprus
integration,
because
that's
what
we're
using
entirely
for
our
end-to-end
testing
it
should
be.
I
It
should
have
a
very
good
open
source
support
and
it
should
be
able
to
ignore
browser
platform
and
india
leasing,
corporates
which
we'll
discuss
in
detail
in
the
next
slides.
It
should
also
support
responsive
testing,
because
we
do
use
some
kind
of
responsive
testing
like
we.
We
do
have
some
responsive
features
and
functions.
I
I
I
I
We
don't
have
a
lot
of.
We
don't
have
a
lot
of
tools
that
passed
all
these
criteria,
but
so
we'll
just
discuss
some
of
the
most
popular
tools
and
then
we'll
discuss
the
tools
that
work
with
the
best
and
text
all
the
boxes.
For
us,
the
first
in
our
list
was
phantom
css,
which
is
also
the
first
visual
test
testing
tool
based
on
javascript,
and
this
is
where
all
the
visualization
testing
tools
started
to
come
up
with.
I
I
I
I
It
also
is
really
good
on
blogs
and
stuff,
and
it
also
allows
authors
to
test
various
viewport
bits
like
very
good,
with
responsive
and
also
has
this
really
cool
feature
that
it
will
ignore
all
these
platforms
platform,
specific
nkl,
using
differences.
It
is
the
most
part
of
the
false
positives,
and
ever
since
it
was
written
by
the
bbc
it
it
received
quite
a
bit
of
publicity
and
yeah.
I
So
these
are
the
pros
and
cons
here
it
doesn't.
It
has
a
really
good
nti
leasing.
Support
like
it
won't
have
a
lot
of
false
positives,
but
even
this
was
not
really
good
for
us,
because
it
wasn't
able
to
handle
the
dynamic
content
well
and
no
reporting
tools
and
was
only
able
to
test
full
page
screenshots
where
we
needed
to
test
small
components
as
well
individually.
I
I
Also,
the
biggest
reason
was:
I
talked
about
india
losing
offsets
and
all
that,
so,
let's
talk
this
is
one
of
the
most
east
will
have
like
the
same
problem
with
them.
The
problem
is
that
it
all
uses
pixel,
matching
and-
and
one
does
not
simply
do
with
map
comparison.
It
uses
pixel
by
pixel
matching
with
not
comparison,
we'll
understand
why.
I
Let's
see
how
bitmap
comparison
actually
works,
pixel
by
pixels
with
map
comparison
work
in
bitmap
comparison,
a
bitmap
of
screen
is
captured
at
various
points
of
a
testing
and
its
pixels
are
compared
to
a
baseline,
bitmap
and
comparison
stays
and
they
hydrate
through
each
pixel
pair
and
then
check
of
the
color
hexors
color.
Hex
codes
are
same
if
the
color
codes
are
different.
It
raises
the
visual
work
and
if
they
are
same
it
passes
them.
I
I
The
elements
don't
overlap.
The
pixel
comparison
tools
can't
test
for
these
cases
and
dynamic
content
is
probably
not
the
biggest
downside
of
this
pixel
by
pixel.
Matching
the
biggest
choice
was
this:
this
happens,
as
you
can
see
in
this
picture
like
you
can
see,
they
look
almost
identical
to
human
eye,
but
two
pixel
by
pixel
matching
algorithms
cannot
generates
a
lot
of
tips,
so
this
happens
because
of
a
lot
of
reasons,
and
these
are
not
even
dynamic.
I
As
a
result,
a
tool
that
expects
exact,
expect,
exact
pixel
matches
between
two
images
can
always
be
filled
with
pixel
differences
so
to
our
rescue.
We
have
ai
for
virtual
regression,
testing
and,
unlike
pixel
by
pixel
comparison,
ai
powered,
automated
visual
networking
tested
tools
do
not
need
special
environment
that
remains
static
and
that
remains
starting
to
ensure
accuracy
and
hence
have
a
high
degree
of
high
degree
of
accuracy
and
even
with
dynamic
content,
because
the
comparisons
are
based
on
relationship
and
not
simply
on
pixels.
I
I
I
I
Some
of
the
constraint
was
since
it
wasn't
open
source.
Some
of
the
constraint
was,
it
has
5000
screenshots
per
month
and
screenshot
from
a
screenshot
is
rendering
of
a
page
or
component
in
an
individual
browser
over
different,
responsive
widths,
combinations
and
everything.
And
if
you
want
different
browsers,
it
counts
as
one
screenshot
too,
and
it
has
30
bit.
It
only
has
30
day
build
history,
which
is
probably
fine,
but
no
I'm
not
really
so
in
our.
I
It
has
really
intuitive
dashboard,
intuitive
ui
and
very
easy
to
understand,
but
I
was
still
logging
in
by
lagging
behind
in
a
lot
of
things
that
apple
tools
was
able
to
fill
those
gaps
so
yeah
it
is
a
bit
more
powerful
than
percy.
It
has
exhaustive
documentation
and
easy.
I
see
the
integration,
but
some
of
it.
Even
it
has
some
of
the
constraints
it
happened.
It
only
supports
like
100
100
checkpoint
screenshots
per
month.
I
I
Had
the
minimum
number
of
false
positives
and
it
has
really
good
thinking
some
integration
that
we're
using,
so
this
is
one
of
the
sample
tests
that
it
is
merged,
actually,
not
always
so
it
is
really
simple.
You
only
have
three
methods
in
amplitude,
you
have
cy
dot,
eyes,
open
and
see
why
dot
check
window
and
then
eyes
close.
So
if
you're
for
starting
the
app
tools
you
use
eyes
open
and
to
check
window.
This
is
how
I
check
the
content.
I
I
It's
really
simple
to
just
add
ignore
regions,
and
this
is
the
articles
config
like
you,
can
see,
you
can
give
test
concurrency,
so
it
and
in
free
plan
they
support
like
10
test
concurrency,
and
you
can
just
provide
like
an
array
of
different
browser,
widths
and
just
different
browsers
and
different
responses
with
different
viewport
width
and
orientation.
You
can
even
test
it
on
mobile
devices
and
everything.
I
So
we
have
a
lot
of
dynamic
content.
I
already
talked
about
it,
but
in
our
safe
dashboard,
things
are
supposed
to
keep
moving,
and
this
is
one
of
the,
and
this
is
one
of
the
dynamic
element
we
have
in
our
dashboard
canvas
element,
a
chart
element
so
to
stabilize
these
screenshot
baseline
screenshots
and
mitigate
false
positives.
There
are
two
methods
that
we
can
handle.
This
one
is
that
we
create
an
ignore
regions,
ignore
region
that
I
already
use
in
that
we
are
already
using
in
the
dashboard
component
and.
I
And
you
can
create
these
ignore
regions
by
either
coding
them,
like.
You
can
just
add
a
css
selector
or
you
can
just
give
the
position
where
the
ignore
region
should
be,
or
you
can
go
in
the
testing
and
using
qe.
You
can
just
marquee
mark
the
ignore
region
and
the
next.
The
other
method
is
by
changing
the
match
level.
I
Now,
by
changing
the
match
layer
to
layout,
we
can
switch
to
different
match
levels
within
components
and
tests.
Instead,
it
can
check
you
can
check
content,
font,
layout,
color
and
position
of
elements
and
content.
It
can
check
the
and
it
is
very
similar
to
restrict,
but
can
check,
also
check
for
colors
it
can.
It
will
ignore
color
changes.
I
I
Let's
discuss
what
visual
testing
is
not
it.
What
visual
testing
is
not
is,
it
does
not
verify
the
logic
functionality
and
you
can't
replace
it
with
functional
tests.
Functional
tests
are
needed
and
which
should
then
can't
be
used
to
stick
them
and
they're.
Also
not
typical
fully
automated
test
suite.
You
don't
always
want
the
passing
of
failing
value.
I
I
Let
me
show
you
in
the
input
version
5.
Our
login
page
will
look
something
like
this,
but
all
our
tests
are
passing
here.
You
can
see
so
this
is
really
hard
to
catch
using
already
existing
solutions.
So
we
have
to
use
visual
testing
and
you
can
see.
Visual
testing
was
able
to
actually
check
the
devs
easily.
I
I
I
So
next
I'm
gonna
write
more
tests
for
the
different
dashboard
components
and
I'm
gonna
and
I'm
gonna
be
writing
contribution.
You
can
help
in
that
as
well
and
I'll
be
writing
documentation
for
that.
So
you
can
documentation
to
write
how
to
write
more
tests
and
you
might
have
seen
and
next,
as
we
also
I'll,
also
be
working
on
the
ci
cd
pipeline,
integrate
that
in
integrated
well
with
the
jenkins
pipeline.
I
There
are
a
lot
of
bugs
like
one
of
the
biggest
thing
that
it
can
help
in
is
detecting
new.
When
you're
upgrading
new
frameworks
like
you
have
in
since
it's
a
javascript
project,
you
have
to
keep
moving
and
you
have
to
upgrade
it
really
fast,
like
you're,
using
cyprus
version,
4
and
cypress
version
5,
and
now,
with
subscribers
version
7,
and
also
we're
using
bootstrap
version
4
and
since
we'll
have
to
constantly
upgrade
our
framework
in
libraries.
I
B
C
B
B
Yeah,
the
dissolved
ones
are
really
hard
to
catch.
So
that's,
I
think
this
addition
is
going
to
help
us
a
lot,
especially
currently
we
have
tried
not
to
migrate
to
the
latest
stuff,
like
the
latest
angular,
the
latest
bootstrap,
ui,
etc.
Just
to
avoid
this
kind
of
regressions,
but
with
having
this
kind
of
safeguards,
I
think
we
can
more
safely.
B
I
mean
keeping
in
updated
with
these
libraries
because
every
time
that
there's
a
new
twitter
and
senior
bootstrap
ui,
I
think
we
have
to
perform
a
massive
refactoring
and
check
everything
that
everything's
okay,
so
yeah,
that's
that's,
really
tricky
and
also
it
sometimes
works
well
with
some
devices.
But
then
you
get
a
report
from
someone
running
some
strange
browser,
that's
very
broken,
so
yeah,
that's.
Hopefully
at
least
we
will
cover
firefox
and
chrome
with
this,
and
that
will
be
a
starting
point,
but
I
mean.
I
C
I
No,
it
is,
it
won't
be
able
to
replace
the
functional
tests
at
all.
It
is
more.
Visual
tests
are
more
compliment
to
that
complement
to
functional
tests,
because
visual
test
visual
education
tests
only
as
for
the
visual
aspects
of
it
and
not
the
elements
and
everything,
and
if
even
if
they
are
clickable
or
not,
and
the
different
states
of
the
elements
and
stuff,
it's
not
gonna
more
of
complement
to
the
function,
type.
I
Yeah,
so
we
did
calculate
on
the
number
of
screens,
screenshots
we'll
be
catching
like,
based
on
the
number
of
pull
requests
we
already
receive
on
the
dashboard
and
after
tools.
Open
source
plans
gives
us
10
000
screenshots
per
month,
which
is
pretty
liberal
and
like
previously,
we
thought
there
was
5000
scene
shots
for
one,
but
even
then
it
was
really
good.
I
So
yeah,
I
don't
think
we'll
be
hitting
that
very
often
still
seeing
like
we'll
check
this
month
as
well
like,
if
I
don't
think
so,
will
be,
and
even
if
you
want
to
migrate
to
different,
let's
say
you
want
to
migrate
to
percy
from
aptitudes.
It's
just
changing
two
lines
of
code
in
every
test.
So
it's
really
easy
to
migrate
to
other
tools.
I
B
Yeah,
let's
see
we
did
a,
I
mean
a
prior
calculation,
more
or
less
the
amount
of
screenshots
that
we
would
take
and
it
might
work.
We
will
probably
need
to
tune
that
a
bit,
but
let's
see
in
any
case
well,
I
had
some
conversation
with
the
appleton's
team
and
they
I
mean
initially
granted
the
open
source
plan,
let's
check
if
they
might
be
flexible
in
the
future.
I
know
that
at
least
the
ansible
shower
team.
B
B
F
So
we're
going
to
present
what
we
have
been
exploring,
analyzing
and
developing
developing
in
the
last
four
months.
We
started
early
april,
I
believe
so
with
caching
and
yeah,
we
going
through
you
what
we
came
up
with
the
ways
we
have
done
it
and
how
how
we
did
it
so
why
we
have
started
this
issue.
First,
we
have
been
receiving
feedbacks
of
some
users.
F
F
F
What
we
did
was
manager
well
pressing,
the
manager
model,
testing,
injecting
fake
data,
osd
map
or
pg
maps
and
so
on,
and
we
have
work
on
shared
dtf
cache
from
the
c
plus
side,
because
we
have
thought
it
would
be
better
to
have
a
shared
caching
than
caching
each
module
and,
for
instance,
and
civilizing
data.
We
have
been
using
file
format
and
just
information.
We
have
tested
both
and
came
up
with
different
results.
F
F
So
if
we
expose
the,
if,
if
we
expose
the
osd
the
or
the
get
method
osd
map,
we
can
receive
this
in
a
json
format,
as
you
can
see,
with
manager,
api
git
osd
map,
and
now
I'm
going
to
talk
about
different
types
of
benchmarks
based
on
and
performance,
diversified
performance
component
level
and
system
level.
Benchmark
component
level
is
distinguished,
specific
component,
which
we
have
used
and
system
level
benchmark,
evaluate
the
overall
performance
of
running
application.
F
F
Regarding
this
we
have
we
have.
We
have
seen
some
design
flows
well,
as
I
said
before,
how
to
debug
the
manager
modules
for
the
developers.
Safe
managers
take
quite
long
time
that
you
take
quite
long
time.
The
cpu
reached
the
peak
and
we
have
realized.
There
is
a
missing
of
pagination
to
filter,
unnecessary
data
from
the
maps.
J
Hey
so
for
now,
where
we
will
talking
about
the
results
that
we
got
from
caching,
the
monitor
module
and
if
you
don't
mind
having
a
headache
or
you're
immune
to
it,
you
can
look
at
the
pr
that
you
can
see
on
top,
and
so
basically,
this
is
the
structure
that
we
currently
have
for
the
manager
demon.
And
basically,
you
can
see
that
we
have
some
maps
on
the
left
side
in
the
manager.
J
J
So
that's
that's
going
to
cause
some
problems
to
solve
this.
We
decided
to
go
with
a
cache
in
the
ziplockplus
site,
so
we
can
share
the
cache
between
every
module,
so
we
don't
have
to
store
every
call
in
every
module
and
basically
we
put
the
title
guys
there
and
to
do
this
we
had
to
solve
some
problems.
The
first
one
was
that
the
some
monitor
modules
were
modifying
the
pi
objects
and
as
an
example,
there
were
some
modules
that
deleted
a
value
and
then
another
module
tried
to
get
that
module.
J
J
Then
we
cast
it
and
it
was
31
faster
than
the
current
implementation,
which
is
a
bit
better,
but
not
good
enough.
So
what
what
we
thought
about
was
changing
the
formatter
to
adjacent
formatter.
We
we
went
with
the
json
formatter,
basically
because
we
already
have
the
implementation
and
it
was
faster
to
implement
this
case.
J
J
Also,
we
try
to
implement
a
pineal
serialization
into
what
I
like
another
formatter,
that
used
binary
civilization,
and
here
you
can
see
there
are
tons
of
them
and
we
went
for
message
pack
since
it's
it
has
a
support
for
a
centos
and
m
federer
and
piccolo
didn't
have
4th
century's.
So
we
didn't
go
with
that
at
the
end,
we
made
it
work
with
death,
but
it
didn't
go
fast
enough.
J
J
Inject
some
load
to
the
monitor
module
and
without
cache,
and
also
with
the
json
formatter,
and
so
basically
we're
going
to
run
this.
It
will
generate
some
load
on
the
monitor
module
and,
as
you
can
see,
this
takes
a
while
to
load
and
yeah.
They
will
finish
now
and
now
we
are
going
to
try
the
same
with
the
same
load
but
with
the
gaussian.
J
J
As
a
conclusion,
at
first,
we
saw
that
json
formatter
with
occasion
is
71
percent
slower,
but
if
we
cash
it
with
3
000
like
this,
we
get
a
boost
in
with
56
percent
faster
than
the
pi
formatter
and,
moreover,
we
we
can
decouple
the
modules
if
we
use
json
formatter,
for
example,
as
we
we've
seen
some
problems
with
interpreters
and
to
finish
the
I.
We
think
that
the
question
elevates
the
heavy
load
problems,
but
it
doesn't
solve
the
entire
problem.
J
K
K
C
J
C
L
Josh
we've
seen
a
similar
thing
in
the
rgw
cls
classes
in
the
osd,
where
we
need
to
filter
based
on
a
very
small
set
of
fields,
and
we
end
up
decoding
a
much
larger
data
structure
and
then
re-encoding
after
we
filter
on
those
different
fields.
It
might
be
a
good,
it
might
be
beneficial
for
us
if
we
can
re-structure
the
way
we
think
about
these,
so
that
we
really
focus
on
specific
fields
that
we
need
and
figure
out
how
to
kind
of
avoid
dealing
with
lots
of
extra
data.
C
D
B
Yeah
yeah,
the
thing
here
is
that
you
cannot
prevent
in
the
future
this
from
happening.
So
that's
the
issue
with
with
python
that
you
cannot
really
create
immutable
objects
and
it
has
to
be
a
best
effort
from
the
developers
to
you
know,
modify
these
structures.
Otherwise
the
consequences
will
be
catastrophic
here.
D
B
Yeah,
the
thing
is
that,
as
you
are,
are
returning
a
nested
structure,
anything
within
that
structure
would
be
modified.
So
if
you
want
to
do
some
that
kind
of
thing,
you
need
to
traverse
the
whole
structure,
which
is
smaller,
the
same
penalty
as
you're
paying
when
you
are
doing
a
copy
that
you
need
to
traverse
the
whole
structure.
It's
deeply
nested,
so
perry
was
trying
that
using
immutables
and
also
copying
right
approach.
But
in
the
end
I
mean,
if
someone
is
accessing
a
a
list.
D
B
Yeah,
they
could
think
of
the
json
approach
and
it
can
be
replaced
with
a
b
zone
or
binaries
on
or
whatever
some
other
more
compact
interfaces
that
we
can
decouple
the
I
mean
at
last.
We
can
decouple
the
manager
and
the
sub
interpreters
thing,
so
we
can
have
modules
running
outside
the
manager
maybe
remotely,
so
that
might
be
a
first
step
towards
that
yeah.
Okay,
coming
back
to
the
previous
question
from
mark
and
just
one
possibility,
the
api
is
not
very
fine-grained,
so
we
are
just
returning
these
big
chunks
of
data.
C
B
Let's
see
that
you
just
want
the
first
level
in
next
dictionary,
so
you
just
return
the
first
level
of
data,
so
maybe
we
can
use
some
syntax
just
to
say
that
or
maybe
you
just
want
to
access
some
specific
property
in
a
dictionary
and
if
we
can
agree-
and
we
can
use
some
existing
language
for
that-
and
there
are
a
couple
of
them,
but
we
can
use
our
custom.
We
just
want
to
extend
this
behavior.
G
I
think
you
mentioned
that
this
is
a
ttl
cache
which
I
assume
stands
for
time
to
live,
and
I
think
you
mentioned
10
seconds.
That's
what
I've
heard,
but
you
know
structures
like
hd
maps,
don't
usually
change
that
often
does
it
mean
that
we,
you
know,
do
the
work
of
recaching
every
10
seconds,
or
am
I
missing.
D
G
That
might
be
yeah.
These
are
versioned
data
structures.
Each
of
them
has
a
unique
version,
and
sometimes
they
like
there
is
a
burst.
So
you
could
see
you
know
in
five
seconds.
You
could
see
like
five
new
versions,
but
under
you
know
normal
cluster
operation.
The
same
map
would
persist,
for
you
know.
M
B
Well,
yeah,
actually,
the
object
won't
be.
Map
won't
be
recalculated
after
10
seconds
only
if
someone
actually
request
that,
so
the
cache
entry
will
become
still
and
only
some
some
module
is
requesting
that
data
will
be
recalculated
or
reconstructed
but
yeah.
I
agree
that
we
might
improve
this
further.
C
C
M
M
M
One
is
at
compile
time,
and
this
is
for
people
that
would
build
without
jaeger
and
or
for
systems.
That
cannot
support
that.
For
some
reason
and
the
more
interesting
level
is
to
disable
at
runtime,
so
the
runtime
disabling
inside
jaeger.
They
have
something
like
a
no-op
tracer,
so
it
does.
It
has
all
the
apis
that
regular
tracer
has,
but
it
does
nothing.
M
M
Then,
currently
in
in
the
rgwd,
we
started
from
tracing
on
the
operations.
So
all
the
operation
puts
object,
the
departure
that
object.
All
these
operations
are
going
to
create
a
trace,
and
then
they
can
have
one
or
more
spans
in
them
for
some,
which
means,
if
they're
branching,
then
then
this
could
be
supported
inside
and
that's
going
to
be
the
basic
implementation.
M
We
the
we're
going
to
add
some
tags,
which
is
something
that
is
supported
in
the
jager
those
tags
later
on
they're
stored
in
elasticsearch,
so
they
could
be
used
for
searches
like
if
you
want
to
get
all
the
traces
for
a
certain
bucket
or
a
central
user
or
object.
Let's
start
with
something
or
whatever
condition
you
can
get,
you
can
get
that
a
more
interesting
use
case
that
we're
going
to
have.
M
The
first
use
case
we're
going
to
have
for
this
is,
is
in
the
case
of
multi-part
upload,
so
in
case
of
multiple
upload
you
have
like
the
in
it
and
then
you
have
each
chunk.
You
have
an
object
put,
and
then
you
have
the
complete
so
we're
going
to
use
those
serialization
dc
relations
so
that
in
the
ui
it
would
look
like
just
one
big
trace
all
the
different
operations,
although
they
can
happen
on
different
latest
gateways
at
different
times,
and
so
on.
M
Another
step
that
we're
probably
going
to
use
the
serialized
industrialized
function
is
passing
the
trace
to
liberators
and
then
sending
that
over
to
the
osd,
and
then
you
can
have
a
trace.
That
starts
from
the
front
end
operation
in
the
redis
gateway
and
trickles
all
the
way
down
through
the
three
liberators
to
the
osd.
M
M
M
This
is,
for
you
know
just
explaining
how
we
expect
the
agents
and
the
collectors
and
so
on,
to
be
deployed
mainly
pointing
to
the
existing
documentation.
So
there's
not
another
title
deployment
would
be
cfdm
based
one,
and
this
is
something
that
I
think
deepika
started,
but
it's
still
working
progress.
M
As
far
as
I
know,
and
the
the
last
option
of
deployment
would
be
openshift
based
or
rook,
based
where
that
would
be
actually
from
jaeger
perspective,
that
would
be
the
easiest,
because
jaeger
is
actually
geared
toward
kubernetes
or
openshift
deployments,
so
they
already
have
an
operator
and
everything.
So
that
would
be
actually
the
easier
part
to
do.
That's
that's
another
direction
that
we
would
need
to
progress
in,
and
there
were
also
thoughts.
C
M
I
don't
think
so,
because
sampling
is
really
saying:
okay
from
from
the
entire
tracing,
we
don't
need
all
the
traces
we're
going
to
just
sample
10
and
that's
gonna
reduce
the
load,
but
I
think
that,
usually,
when
you
come
to
debug
a
system,
then
then
I
mean,
for
example,
if
you,
if
you
want
to
use
tracing
in
order
to
get
some
kind
of
estimates
on
your
on
the
different
latencies
of
the
different
steps
in
your
process.
So
samsung
would
be
sufficient
and
great
solution
in
this
case.
M
But
if
you
want
to
debug
multi-site,
because
you
have
problems,
let's
say
one
of
the
buckets
is
not
syncing
and
you
want
to
debug
multi-site.
Then
sampling
won't
help.
You
you'll
have
to
actually
disable
tracing
for
everything,
so
it
won't
stand
in
your
way
and
just
enable
tracing
for
for
bucket
syncing,
ideally
for
a
specific
bucket
and
then
there'll
be
low
impact
on
the
system.
But
you
would
get
all
the
information
that
you
need
for
debugging,
so
yeah.
C
A
Maybe
now
in
general,
as
josh,
suggested
that
there
are
sampling
strategies,
we
can
also
have
sampling
strategies
from
what
I
saw
that
they
would
collect
the
spans
locally,
but
they
won't
transmit
it
and
they
would
only
transmit
the
ones
that
we
choose
to
and
in
case
of
failure.
We
can.
We
also
have
those
spans
collected
in
the
local
demon,
but
not
transmitted,
but
still
available
for
us
to
look
into
if
needed,
while
also
giving
us
the
thorough
picture
of.
A
A
A
A
Convert
those
traces
in
json
format
and
those
json
format
based
traces
could
be
rendered
anywhere,
as
it
is
for
even
a
developer
to
take
a
look
into
what
happened
at
with
the
system.
At
that
particular
point
of
time,
I
added
a
blog
link
that
might
help
somebody
to
get
started
with
using
tracing
and
would
love
the
contributions
there
as
well
feel
free
to
add
any
suggestions.
M
Yeah,
I
think,
from
from
the
thinking
the
two
efforts
there
is
one
small
thing
would
be
to
to
have
like
one
common
wrapper
and
everything
for
the
tracer
instead
of
having
one
in
rjw
and
one
in
under
common,
but
that
should
be
pretty
easy
because
they're
almost
identical
the
two,
the
two
rappers
and
for
the
serialized
disciplines.
I
think
that
one
approach
could
be
that
you
you
try
to
to
deserialize.
M
So
if,
if
anyone
is
sending
you
a
parent
span,
then
you
can
digitalize
it
and
use
it
as
your
parents,
pen
and
then
in
the
ui,
then
you
would
see
the
entire
flow
end
to
end,
and
if
you
don't
get
it
then
well,
maybe
nobody
sent
it
to
you.
Maybe
there's
no
need
or
whatever
and
and
then
you
just
start
your
own
parent
span
and
continue
from
there.
And
hopefully
we
will
do
that,
of
course
on
our
side
as
well
in
the
rgw,
but
that
could
be
any
client,
not
necessarily
rgw.
A
A
E
M
C
You
can
get
them
from
the
monitor,
like
you,
can
ask
the
monitor
for
the
configuration
from
like
any
attitude's
perspective.
But
I
guess
I
would
push
in
my
mind
would
be.
Is
this
something
that
we
want
to
control
directly
with
like
a
self-configuration
option
or
some
other
kind
of
format
of
configuration.
M
I
think
it
should
be
set
configuration
just
something
that
somebody
goes
in
the
command
line
or
I
don't
need
to
have
that
in
the
dashboard
and
just
set
it
to
true
or
false
in
runtime
I'll.
Try
again,
I
don't
know
a
lot
about
how
this
configuration
mechanism
works,
so
I'll,
try
to
sync
up
with
and
see
if
we
can
have
like
a
convert
solution.
C
In
terms
of,
I
think
that
what
you
described
sounds
like
a
great
plan
and
like
the
deployment
aspect,
is
especially
key
to
making
this
usable
and
for
both
developers
and
for
actual
users.
Of
course,
I
think
those
are
the
other
kind
of
the
first
steps
in
my
mind
as
well.
I
just
getting
that
the
basics
there,
I
think
longer
term
movie
fantastic
to
expand
this,
to
include
like
background
operations
too,
not
just
the
I
o
path,
so
you
can
use
ladybug
and
and
understand.
M
Yeah,
once
we
have
a
good
infrastructure
in
place,
then
if
we
like
you
know
anyone
that
writes
a
feature
or
fixes
a
bug
and
want
to
add
tracing
there,
because
it
helps
them
to
do
better
work,
then
they
can
do
that.
So
I'm
going
to
be
kind
of
scale
out
the
work
to
other
people.
That
would
just
follow
the
examples
that
we
said
at
the.
A
A
C
As
another
aspect,
that's
missing,
I
guess
from
the
osd
side
today-
is
the
integration
with
crimson,
since
that's
going
to
require
a
bit
of
modifications
for
how
we
use
bigger,
to
avoid
locking
and
whatnot
and
blocking
and
I'd.
Imagine
that
we
could
probably
keep
the
same
interface
so
that
wouldn't
affect
this,
like
the
general
common
flow
for
how
you
define
the
traces,
but
the
implementation
might
need
to
be
different
for
like
rooms
internally.
M
So
the
way
that
we
try
to
approach
that
on
the
rgw
because,
like
we
have
like
the
front
hand,
has
multiple
threads
and
they're
using
we're
using
core
routines
there.
The
way
that
we
approach
that
hopefully
that
will
be
useful
is
that
the
so
the
traces
are
just
objects,
so
there's
no
there's
no
blocking
or
anything,
but
the
tracer,
which
is
what
sends
the
tracers
the
to
the
agent
is,
is
something
that
has
a
lock
in
it.
M
It
just
has
a
lock
when,
when
it
does
descending
at
the
at
the
end,
so
when
you
call
finish
on
a
span,
it
goes
to
the
tracer
and
send
the
spin
over
so
to
avoid
contention
on
those
logs.
So
we
cannot
avoid
the
lock,
because
unless
we
want
to
change
the
the
jager
code
but
to
to
avoid
contention
logs,
what
we
do
is
that
we
define
the
tracer
instead
of
just
being
a
global.
M
It's
a
thread:
local
global
variable,
which
means
that
there'll
be
a
log
taken,
but
I
mean
there'd
be
no
contention
because
there's
only
one
thread:
that's
gonna
use
it
at
a
time
and
different
threads.
That's
going
to
use
it!
We're
going
to
use
a
different
client,
so
it'll
be
a
little
a
little
load
on
the
system.
So,
instead
of
one
client
per
daemon
connects
to
the
agent
you
can
have,
I
don't
know
100
clients,
but
that
they
they
shouldn't,
cost
too
much
and
it
would
save
on
the
on
the
locking.
So
maybe.
C
C
G
C
H
Let
me
share
a
link
to
the
ether
path
yeah.
So,
as
some
of
you
know,
users
who
opt
into
telemetry,
sometimes
they
usually
also
opt
into
the
crash
channel
that
we
have
and
they're
sending
crash
information
about
all
the
crashes
that
occurred
in
their
clusters
and
we
have
a
crash
dashboard.
Maybe
I'll
share
my
screen.
C
H
So
whenever
a
user
sees
a
crash
in
their
cluster,
they
can
go
and
search
tracker
for
the
crash,
signature
and
they'll
be
able
to
see
the
status
of
that
crash.
Maybe
it's
not
even
a
bug.
Maybe
it
is
a
real
issue
and
there's
a
fix
for
it.
So
for
that
we
have
a
new
telemetry
crashes,
bot
that
tries
to
find
a
corresponding
issue
for
each
crash
signature.
H
So,
for
example,
let's
see
yeah,
we
do
have
maybe
I'll
say
just
a
few
words
about
the
search
page
here
I
I
guess
most
of
you
have
already
seen
that,
but
just
in
case
so
you
can
see
the
crash
signature,
which
is
a
new
signature
that
we
created
on
the
back
end
that
groups
more
crash
events
together,
you
can
see
once
the
first
time
we
encountered
that
crash
in
telemetry
once
last
time.
How
many
crashes
in.
H
Yeah,
okay
cool,
so
so
the
crash
crash
count
corresponds
to
the
time
frame.
Here,
that's
the
total
crash
count
and
that's
just
in
time
frame.
So
now
we
just
chose
new
fingerprints
in
this
time
window.
But
if
we
change
it,
you
can
see
that,
for
example,
this
signature
has
total
number
of
206
crash
counts,
but
in
the
last
30
days
we
saw
that
only
13
times.
H
You
can
see
the
number
of
clusters
that
were
affected
by
this
crash
and
all
the
all
the
versions
for
the
demons
that
are
reporting
this
crash
yeah
and
if
you
scroll
sideways,
you
can
also
see
the
assert
function
and
the
assert
condition
if
they
exist-
and
here
we
have
here,
we
have
a
signature
that
the
original
signature
is
reported
by
the
user.
So
that's
the
client-side
signature,
and
here
we
have
status
of
that
signature.
H
So
we
can
click
on
that
and
for
for
the
signature
page,
we
have
more
detailed
information.
You
can
see
the
pressure
currents
by
diversion.
You
can
see
the
the
sanitized
back
trace
that
helps
to
create
the
signature
itself
by
the
way.
If
you
click
on
on
each
of
these
frames,
you
can
see
all
the
crashes
that
occurred
that
had
this
frame
in
their
back
trace,
so
that
can
be
very
useful
as
well.
H
H
You
can
see
all
the
affected
clusters,
their
their
size,
all
sorts
of
other
information
about
them.
You
can
also
click
on
on
the
specific
cluster
and
see
more
graphs
for
that
specific
cluster.
Like
a
cluster
x-ray-
and
here
you
can
see
an
actual
crash
example
where
you
you
have
the
exact
the
exact,
maybe
that's
too
small,
too.
The
exact
crash
dump
here.
H
We
want
for
each
one
of
these
signatures,
a
corresponding
tracker
issue,
so,
for
example,
this
signature,
the
bot,
opened
a
new
issue
for
it.
It
populated
all
the
affected
versions.
So
far,
all
of
the
signatures
that
were
received
from
the
client,
the
signature
that
we
generated
on
the
server
side
and
then
it
populated
the
description
with
a
link
to
the
dashboard
and
with
all
the
other
details.
So,
for
example,
the
asserts
the
sanitize
the
vectorize
and
I
crushed
crashdown
for
example.
H
So
we
encountered
new
crash
events
in
telemetry
that
were
of
a
newer
version
than
the
one
that
was
fixed.
For
example,
you
can
see
the
15
version
here
so
since
that
issue
was
classified
as
resolved,
it
means
that
it
was
a
real
self
bug.
So
we
decided
to
open
a
new
issue
and
to
relate
that
to
the
close
one.