►
From YouTube: IPFS Testing/Benchmarks Census Meeting
Description
## Testing Census Meeting Notes
Benchmarks of js-ipfs up again!
All benchmark data relatively level - no observed performance hit
Running this within network testing pipeline could trigger this api, and spawn this job and input into unified metrics database (in addition to here)
Want more control in our own UI to display relative info in a comparative fashion (maybe that allows you to drill down in grafana)
Grafana good for time series, not for compare against baseline we choose
Meta objective: every group should have an MVP with ZERO dependencies on other groups - the stage 2 or 3 can dovetail and merge with a shared vision - however the first step that will block development should be separable from everyone else’s work
Doesn’t give nice overall view - but can be tweaked
Nice thing this *doesn’t* have is go-nightly or something like that to use in tests
Test infra - want specific test cases being run to extract metrics of interest
Want to see greens, yellow, and reds to make it clear where we should put attention to
Want it to be that any community engineer can see health on various go and js commits against other implementations (hard with grafana)
Alan will put a baseline of work into js benchmarks to make it useful for go, and then go from there
What does everybody actually need?
- project operations and package mangers need benchmarks they can use for their specific testing / feedback loops (and need to be able to start running this prior to test infra work completing
- Gateway team wants to be responsible for spinning up nightly mirrors and similar
- Testing pipeline that will hopefully be an orchestrator and aggregator
Using, adapting, and adopting benchmarks using js test suite to cover all the things we want
Alan get go tests running, steven add go tests
Kubernetes repo also has a whole series of go tests written in yaml-based dsl
Package manager tests will likely be shell scripts to import real world data
Current benchmark format looks good for v1 is good, also ability to run locally so can do it on branch
C
A
D
A
D
Did
okay
I'm
my
own
worst
enemy
share?
This
screen,
okay,
cool!
Let's
try
and
move
this
thing
out
of
the
way,
a
little
bit.
Okay
cool.
Can
you
all
see
my
screen
now,
yeah,
okay,
cool
I
managed
to
get
this
up
and
running
again.
It's
just.
It
was
just
due
to
some
the
server,
but
we
started
and
there's
a
component
of
it
that
isn't
set
up
to
automatically
restart
itself.
So
so,
unfortunately,
there's
there's
been
a
lapse
in
collection
of
benchmark
statistics,
but
we
still
have
historical
benchmarks
to
look
at
right
now.
D
So,
let's
quickly
take
you
through
this.
So
this
is
the
graph
on.
This
is
like
a
dashboard
where
we
see
the
results
of
the
performance
metrics
that
we've
been
running
and
collecting
and
as
it's
set
up
now,
these
performance
metrics
get
run
on
a
nightly
basis.
The
again,
and
what
happens
is
the
JSI
PFS
repo
gets
updated,
so
we
get
the
new
new
code
every
night
and
then
these
performance
metrics
gets
get
done.
It's
quite
daunting.
D
We
have
quite
a
lot
of
tests
that
are
here,
but
we
can
filter
them,
filter
them
down
to
just
this
things
that
we
care
about
to
make
it
a
little
less
less
daunting
and
a
little
more
easier
to
inspect,
and
then
you
can.
Obviously
you
can
change
the
dates
and
pounds
of
stuff
in
standard
curve
graph
on
so
thinks.
D
So
what
are
we
testing?
We
have
this
big
matrix
that
I
made
of
the
things
that
we'd
like
to
test,
so
they
are
like
adding
files,
they
are
cutting
files
and
like
yeah,
em,
FS
things
and
pub/sub
things
and
tests
in
the
browser.
But
it's
also
it's
also
variations
in
like
dag
strategy,
as
well
as
like
variations.
D
D
So
that's
totally
typical
I'm
sat
in
the
kitchen
because
anyway,
so
yes,
variations
in
like
deck
strategy
in
like
the
transport
we're
using
in
the
multiplexer
that
we're
using
and
whether
you're
using
CIO
or
not.
So
it's
variations
in
that,
but
also
variations
in
the
types
of
nodes
were
using
and
the
number
of
nodes
we're
using.
So
we've
got
like
the
idea
of
like
a
test
that
is
just
for
one
node,
so
like
adding
a
file,
that's
just
a
test,
adding
it
to
one
particular
nodes,
those
like
zero
to
zero.
D
But
then
we
have
like
zero
to.
One
is
like
from
one
node
to
another
node
and
they
are
two
J's
nodes
in
this
case
and
then
we
have
like
J
s
to
go,
and
then
we
have
five
j
s
nodes
to
one
jeaious
node,
and
then
we
so
we've
got
this
old
matrix
of
test
that
what
we'd
like
to
see
the
green
apples
are
the
ones
that
exist.
The
red
apples
are
the
ones
that
don't
rather
obviously,
and
we
have
like
a
bunch
of
tests,
not
clickable,
but
yes,
so
we
have.
D
D
Take
the
end
time
the
result,
and
that's
just
about
it
what's
kind
of
nice
about
this-
is
that
the
any
tests
that
you
edit
or
or
add
when,
when
they
get
merged
to
master
the
the
there's
tests
that
run
on
this
repo
that
get
run
and
if
they're
successful,
then
it
will
actually
be
deployed
to
benchmarks
or
ipfs
the
IO,
and
so
your
tests
will
automatically
start
running
against
whenever
nightly
happens
or
or
the
other
trigger
which
we
haven't
talked
about.
Yet
what
does
this
look
like
so
infrastructure
wise?
D
D
One
is
the
controller,
so
this
is
in
charge
of
receiving
the
fact
that
the
benchmarks
need
to
be
run
actually
scheduling
in
them
in
and
getting
the
run
and
recording
the
results,
putting
the
results
on
ipfs,
and
then
we
have
another
bare-metal
machine
which
is
responsible
for
actually
running
those
benchmarks
and
then
once
they
are
done,
it
will
use
these
clinic
tools
that
these
are
a
bunch
of
tools
that
near
form
the
company
who
set
up
this.
This.
D
These
benchmarks
and
have
created
to
help
us
on
a
DP,
debug,
know
and
what
happened
during
this
module.
So
that's
kind
of
cool.
Yes,
so
the
interesting
part
here
is
that
there
is
a
HTTP
endpoint
a
trigger
and
we
have
a
on
the
machine.
That's
like
a
nightly
a
cron
job
which,
and
every
night
will
trigger
the
benchmark,
the
controller
to
run
the
benchmarks.
D
There
is
also
this
so
this
could
this
doesn't
exist,
but
the
trigger
can
be
used
by
CI
to
run
the
benchmarks
for
a
particular
commit
and
there's
a
swagger
end
point
with
the
D
and
the
config
for
doing
that,
and
it
takes
like
a
young,
an
API
key
that
you
have
to
know
about
how
to
be
able
to
to
trigger
them.
So
we're
we're.
We
right
there,
okay,
so
these
are
some
of
the
interesting
things
about
the
benchmarks
we've
got
so
here.
This
is
just
an
example
of
like
we
are.
D
The
variation
here
is
between
our
M
flex
marker
and
speedy
waxer,
and
we
can
see
here
that
which
is
kind
of
interesting.
So,
what's
kind
of
cool
about
these
benchmarks
is
like
looking
over
the
that's.
What
the
captured
data
we
have
they're
all
fairly
fairly
level,
there's
no
big
kind
of
performance
hit
that
that's
observable
from
these.
So
that's
kind
of
reassuring
in
some
way.
I
guess.
D
But
so
that
there's
that
that
is
the
you
know
the
big
thing
that
we
can.
We
can
look
at
with
the
the
benchmarks.
You
know,
because
we
can
see
instantly
if
there
is
overnight
some
particular
dropping
for
four
months
for
a
particular
test,
so
either
either
we're
doing
okay
or
we
don't
have
the
right
tests.
So
so
that's
interesting.
So
this
is
just
one
of
them
with
our
variation
days
between
FX
and
speedy
and
it
looks
like
speedy
is
no
wait.
M
Plex
is
performing
ever
so
slightly
better.
D
On
the
left
hand,
side,
the
the
y-axis
is
time
in
milliseconds
for
this
test
to
happen,
and
but
so
that's
just
jst
J.
Yes,
what
we
can
see
on
this
one
is:
we've
got
adding
a
file
in
we're,
adding
a
64
Meg
file
and
go,
and
yes,
I,
don't
know
how
to
get
rid
of
this
from
down.
So
we
can
see
that
go
is
a
little
bit
faster
free,
that's
like
three
seconds
versus
two
seconds
which
is
kind
of
cool
and
then
like,
then
we
can
change.
D
We
have
that
we
have
different
types
of
files,
but
there's
four
megabyte
files.
We're
a
little
bit
closer
together
to
200,
odd
milliseconds,
to
kind
of
cool
other
ones.
We've
got
different
transport
here,
so
TCP
and
WebSockets.
These
are
these.
Are
we
can
see
here
that
you
know
there?
Actually
there's
not
a
lot
in
it
and
they're
pretty
fast,
then
it
was
kind
of
interesting
that
WebSocket
seemed
to
be
a
bit
faster
than
the
anticipate
all
right.
So
the
other.
The
final
kind
of
interesting
thing
about
the
benchmarks
is
that
we
also
have
this.
D
This
ability
to
use
the
clinic
which
I
talked
about
earlier
in
the
in
the
infrastructure
of
these
clinic
tools.
So
what
happens
here
is
that
once
the
benchmarks
are
in
run,
these
tools
go
over
the
kind
of
debug
files
will
in
a
Detroiter
file
or
whatever,
is
there
to
kind
of
pick
out
the
interesting
stuff
and
do
and
show
us,
like
slim
graphs
and
things
so
I
need
to
look
into
why
we
don't
have
things
for
all
of
them.
D
I
don't
know
if
they're
meant
to
be
there
for
all
of
the
tests
that
we've
ever
done
or
but
I
know
that
the
swagger
endpoints
for
triggering
them.
You
can
tell
it
to
clinic
enabled
you
can
tell
it
to
run
clinical
need
on
this
particular
and
benchmark
so
I.
Don't
I
need
to
figure
that
out,
but,
like
I
said
just
you've
got
this
running
so
so
yeah
but
yeah.
So
I
should
be
able
to
click
on
this
guy,
and
so
then
we
see
all
of
the
tests
we
have
here
and
social
extracting.
D
So
this
is
local
extract,
since,
like
a
file
has
been
added
to
your
node,
and
this
is
the
time
it
takes
to
cat
it
back
out
and
to
you
just
locally,
so
this
isn't
involved
in
any
kind
of
network.
But
we've
got
these
three
things
and
we
can
pull
up
the
flame
graph
for
that,
and
then
we
get
this
nice
kind
of
visualization
as
what
the
node
was
doing.
D
E
A
A
D
Right
so
when
you
first
get
there
you
will,
you
will
arrive
just
you
will
arrive
here.
There's
no
data
points,
because
we
last
seven
days,
though
this
wasn't
running,
but
we
can
change
that
our
videos
in
the
way
get
out
of
the
way
we
can
change
it
out
to
the
last
90
days
and
then
you
can
see
all
of
the
data
that
was
collected
and
then
just
underneath
that
grants,
let's
just
expand
it
here,
a
little
bit
just
on
an
h-back
grass.
These
are
all
the
tests
that
exist.
D
So
these
are
all
the
lines
on
the
graph,
and
so
you
can
click
on
one
or
more
of
these
to
just
show
this
particular
this
particular
test
and
then
yeah
you
can.
You
can
like
drag
over
a
particular
date
period
to
zoom
in
on
that
date
period.
So,
if
you
see
like
there's
a
big
drop
of
in
one
of
the
graphs
compared
to
the
others,
then
you
can
drill
in
take
a
look
at
that.
D
There
is
really
only
this
one
page
that
is
configurable
to
look
at
the
results,
but
you
can
like
things
that
I
often
do
are
like
changed.
The
file
set
that
was
used
to
see
if
that
had
like
different
performance
characteristics,
but
we
also
we
also
record
CPU
and
memory
as
well
as
iteration
test
and
it's
kind
of
cool
and
yeah.
You
can
filter
by
particular
and
yeah.
There's
there's
any
one
real
page.
D
A
D
Thank
you
can
there
is
I,
think
Hugo
went
in
and
and
they'll
they
different.
You
can
create
the
grass
like
it's
like
Johannes,
just
like
this
generic
kind
of
tools
for
graphing
things,
so
you
can
create
different
dashboards
and
login
as
an
admin,
and
it's
it's
really
possible.
Yeah
I
have
a
questions
wrong.
Yeah!
That's.
B
Universal
radius,
walk
is
I
need,
have
a
look
at
the
benchmarks.
Ripple
and
I
could
see
kind
of
a
theoretical
side
of
things,
which
is
really
it's
been
like
we
even
got
it
was
great
to
see.
You
know
there
is
a
bunch,
a
lot
of
anointment
between
between
one
test
and
from
testing
and
prod
the
design
are
we
working
on
and
what
is
the
moving
components
and
in
this
particular
system?
So
it's
like
at
the
end
of
the
day.
You
know
when
you're
doing
Penta
fire
game.
B
Like
you
know,
most
systems
aren't
going
to
be
concerned
or
the
same
contest
they're
going
to
be
concerned
with
the
same
time
right.
So
it's
good
just
to
gather
confirmation,
I
think
money,
Bree
said
and
like
basically
hey
getting
access
like.
We
don't
want
to
have
a
myriad
tools
out
there
that
we
need
to
check
one
by
one
to
get
a
sense
of
how
things
are
performing
across
the
board.
B
Ideally
we'll
be
having
a
single
dashboard
which
provides
a
unified
entry
point
and
experience
to
the
engineer
and
but
with
the
architecture
that
we
conceptualize
for
for
the
testing
running
this.
So
this
has
a
front-end
and
it
has
a
map
right.
So
what
what?
What
Alan
showed
us?
The
the
architecture
diagram
with
with
a
coordinator
or
your
I?
Remember
we
hold
it
in
the
like
that
components
in
the
middle
and
then
the
minions.
This
is
kind
of
like
the
packet
and
this
poverty
is
driven
by
an
API.
B
B
You
know
that
the
spawning
of
this
job
in
the
in
this
infrastructure
under
a
test
Runner,
because
we
have
the
concept
we'll
be
building
the
concept
of
pluggable
test
runners
out
of
which
you're
not
canary
testing,
is
one
is
one
strategy
for
testing
my
defense
code,
but
the
Jas
IVFs
benchmarks
could
be
another
test
run.
So,
potentially
we
would
be
triggering
the
job
and
then
the
test
runner
has
an
input
and
it
produces
an
output
where
the
outputs
are
basically
a
bunch
of
metrics
and
those
metrics
are
in
a
particular
data
model
such
that.
B
What
we
could
do
is
pick
them
up
from
grandfather
if
they're,
sorry
from
Prometheus,
if
they're
being
posted
there
or
I,
remember
in
flux,
TV
I
think
it
is,
if
they're
being
posted
there,
and
then
you
know
bring
them
import
them
into
the
identified
metrics
database
of
the
testing
pipeline,
so
that
that
you
know,
is
effectively
one
way
of
integrating
this
piece
into
the
pipeline.
Because
this
is
you
know
what
we're
building
always
heading
out
to
build
is
really
a
pot
like
it's
the
unified
view
of
the
pipeline.
B
The
hey
something
happens,
change
happen
in
the
idea
that
score
basic,
a
people
or
a
BFS
a
so
you
could
be
going
J,
yes,
and
we
want
to
see
how
things
are
interoperating
performing
whether
basic
slope
tests
are
running,
how
they're
performing
against
the
life
net,
how
they're
performing
in
private
you
know
in
private
networks,
with
a
particular
composition,
composition,
mimicking
the
public
network.
So
you
know
a
bunch
of
things.
F
Good
question
about
the
fauna:
can
you
see
this
this
table
like
time
for
each
run
and
then
I
press
file
uploads
a
is
it
possible
to
get
like
in
a
table
of
like
here?
Are
all
the
you
know.
It
smells
like
here
all
the
different
metrics
we
have
and
then
a
bunch
of
time
runs
and
then
like
cells
with,
like
the
individual
stats.
We
have
from
them
something
we're
like
I
can
easily
browse
through
all
the
benchmarks
is
it.
My
concern
here
is
like
just
like
trying
to
like
find
Oh
like
here's.
F
D
A
A
F
Nothing
like
with
this
link.
We
can
actually
change
this
without
having
to
go
into
into
the
backend
we're
like,
with
the
current
set
of
the
world.
Closing
we'd
have
a
fixed
UI,
which
yes,
we'd,
have
graphs
on
it,
but
like
we're
not
going
to
get
this
wonderful
interactive,
like
query
system
that
we
have
here
and
I
think
this
may
actually
like
just
give
us
everything
we
need
most
of
the
dogs.
It
turns
out
for
final.
F
Authors
have
work
for
buttons
Oh
like
we
can
inject
arbitrary
HTML,
so
we
could
do
that
for
even
like
triggering
tests
or
like
looking
to
various
things.
So
I
think
this
is
pretty
like
hey
that's
why
I
want
to
look
at
it,
see
like
what?
Basically,
how
can
here
to
create
our
system
into
this?
Do
you
do
the
king
as
much
work
as
possible
and
then
make
it
interactive
as
possible.
B
So
I
mean
I,
definitely
see
you
know
lots
of
links
in
that
particular
dashboard.
That
would
send
you
off
to
a
particular
profile,
on
instance,
making
with
a
slice
of
time
or
you
pre-selected
or
whatever
that
allows
you
to
like
zero
in
to
a
particular
event
that
you
know
has
happened
from
this.
Like
top
level
digest
view
right.
F
I
wonder
if
we
can't
just
make
that
dashboard
here
it
looks
like
we
can
I'm
just
not
entirely
sure,
because
if
we
already
have
a
dashboard
of
like
simple
times
now,
we
want
is
like
a
some
kind
of
dashboard
that
well
so.
The
look
thing
we're
missing
here
is
like
me,
yes
or
no,
like
they
did.
This
run
faster
in
the
cutoff,
restore
in
the
cough
I.
Think
like.
B
This
as
good
as
yeah,
it's
good,
it's
gonna,
be
super
tricky
I
think
to
model
this
one
profiler,
because
at
the
end
of
the
day
we
do
want
to
pick
be
able
to
pick
a
particular
baseline
that
we
want
to
compare
you
know.
Specific
metrics
against
and
graph
on.
Ax
is
really
good
to
show
to
show
time
series
you
know
as
a
series
of
time
type
of
data
points
over
time,
but,
like
I,
don't
think
that's.
F
I
think
that's
a
pretty
similar
click,
we'll
probably
have
is
like
one
row
per
benchmark
and
then
each
like
each
column
will
be
one
run
of
each
benchmark,
and
for
that
we
can
just
like
have
like
graphs
going
across
the
screen,
one
for
each
benchmark.
I
with
like
that,
each
point:
we'd
have
the
different
things
happen
and
we
could
use
bar
graphs,
so
they
could
clearly
see
like
this.
Is
this
run
this
run
and
we
can
zoom
in
all
the
time?
So
that's
what
I'm
thinking
like
the
nice
thing
about
this?
F
Is
we
get
this
nice?
You
are.
We
can
just
mess
with
things
ourselves
or
like
anything.
We
build
will
give
you
the
overview,
but
if
you
want
to
message
that
you'll
have
to
jump
to
Kohana
I've
concerned
like
that,
will
take
extra
effort
or
like
if
we
have
it
all
good
fun
already,
then
it's
just
like
they're
I.
A
G
B
Just
just
one
final
remark:
there
Rehana
is
great
for
a
contiguous
time
slices,
but
when
you
have
multiple
branches
that
are
evolving
independently
and
you
want
to
test
one
like
you
want
to
compare
one
against
the
other
and
another
snow
like
it,
there
are
different
timelines
that
are
happening
in
parallel.
That
would
be
very
difficult
to
model
a
graph
on
that
from
whom,
from
my
experience
with
grip
on
earth-
and
this
might
be,
you
know-
might.
F
Have
changed,
we
can
tag
each
one
of
these
with
like
master
or
whatever
branches
on
and
like
the
the
graphs
can
show
it
faster
or
nightly
or
whatever,
and
then
like.
We
can
have
like
a
separate
list
of
like
runs.
Male
actually,
I
think
this
is
important.
Discuss
like
this
business
skill.
People
work
so
I
want
a
composition
like
are
we
gonna,
just
use,
go
fauna,
or
are
we
going
to
like
build
up
it
on
the
system?
On
top
of
it?
Are
we
going
to
reuse
the
same
databases
that
javascript
is
using
right
here?
F
A
Yeah,
so
that
leads
leads
nicely
into
I.
Think
a
meta
point
of
how
to
think
about
these
things.
A
key
point
here,
given
that
we
have
really
problem
focus
groups
that
need
to
be
able
to
sprint
forward
on
specific
tasks
that
are
blocking
each
of
these.
These
different
projects
is
to
try
and
remove
cross
dependencies
where
possible.
A
So
if,
if
there
is
a
thing
that
is
going
to
block
this
team's
progress
and
is
otherwise
like
critical
for
the
next
stages
of
development,
having
an
MVP
of
of
that
stage,
that
has
zero
blocking
dependencies
on
other
groups
is
super
important.
Otherwise,
we're
gonna
end
up,
inevitably
in
the
place
where
one
group
is
blocking,
the
other
has
more
important
things
on
their
plate
than
unblocking
the
other
group
and
we're
sitting
around
spinning
our
wheels,
because
one
group
can't
get
to
the
other
thing
now.
A
An
MVP
doesn't
mean
that
we
don't
eventually
want
these
things
all
to
end
up
in
the
same
place
and
I
think
we
do,
but
that
could
be
like
you
know.
One
website
that
has
different
web
UI
is
that
you
know
boom
boom
boom
hit
these
same
things
and
by
the
way
we
embed
them
all
into
like
one
master
page,
you
can
like
really
easily
click
between
them
and,
like
that's,
a
fine
world
to
live
in
as
long
as
we're
making
it
really
accessible
and
not
adding
like
dev
overhead
for
checking
deep.
A
This,
like
what
now
looks
like
the
same
page
but
initially
was
goes
as
three
separate
you
eyes
when
we,
when
we
first
started
out
to
remove
this
cross
dependency
across
loops,
and
so
that's
the
method.
I
think
that
is
significantly
more
important
than
reducing
like
overlap.
If
we
have
some
amount
of
overlap
that
we
can
later
kind
of
reuse,
that's
okay,
as
long
as
we
end
up
using
our
time
really
wisely
now
in
terms
of
getting
to
a
place.
So
that's
that's,
like
my
meta
point,
for
how
we
make
these
decisions.
The.
F
The
kind
of
point
is,
it
would
be
nice
to
reduce
time
and
effort
in
building
a
new
system,
so
I
don't
want
to
build
a
second
dashboard
if
we're
to
have
something
you
can
just
use
yeah
and
like
Ethier,
has
already
have
the
test
Orchestrator
here.
That
appears
to
do
a
lot
of
what
we
need
to
do
like,
and
this
is
all
done.
So
it's
not
like
we're
waiting
on
anyone
else.
So
I
are
thinking.
If
we
can
just
like
open
this
up
the
testing
team
and
then
to
start
building
on.
F
E
E
B
B
It
is
morphed
a
long
time
initially
we
wanted
the
IP
invest
project
on
it,
like
the
ground
vision
is
to
create
a
dynamic
system
that
is
public
where
people
can
plug
in
compute
power,
essentially
and
an
infrastructure
and
a
node
anywhere
around
the
world.
That
could
then
be
used
to
run
a
series
of
tests
across
a
decentralized
network
right.
This
is
like
one
vision.
Then
there
have
been
other
visions
which
are
a
bit
more
constrained
and
more
like
with
with
a
type
beyond
applicability.
There
says
that
what
we
want
to
test
as
ipfs
and
if.
B
So
things
like
think
of
this,
as
kind
of
like,
first
of
all,
a
bunch
of
visions
which
have
made
you
know
which
have
iterated
over
time
and
also,
you
know,
created
like
various
efforts
a
long
time,
but
then
also
think
of
this,
as
like
in
micro
service,
oriented
organization,
right
people
just
build
the
tools
that
they
need
for
the
time
that
they
need
them.
So
this
is,
for
example,
the
way
that
I
see
that
the
J
is
benchmarks
actually
came
to
life.
I,
don't
but
I,
don't
know
what
the
like!
D
Yeah
I
just
wanted
to
add
that,
like
these
particular
benchmarks
were,
then
it
really
died.
There's
probably
been
nearly
a
month
since
they
they
have
been
off.
That
we've
got
that
historic
data
that
the
problem
was
just
there
is
the
machine
was
restarted
and
something
didn't
come
back
up
when
it
was
when
it
came
back
on
so
like
this
particular
put.
It
didn't
die.
There
is
a
previous
jeaious
benchmarks,
but
this
project
took
inspiration
from
that.
Wasn't
based
on
it
because
it
was.
D
It
was
based
in
the
web
browser
and
we
wanted
to
be
able
to
to
run
go
nodes
as
well
as
as
jeaious
nodes,
so
that
we
could
compare
between
the
two
implementations.
So,
like
I
was
saying,
this
has
been
a
kind
of
iteration
on
an
idea
for
a
while
and
the
other
thing
I
just
wanted
to
really
quickly
say
it
was
like
crow
fauna.
Is
it's
like
one
of
those
applications
that
you
can
kind
of
do
anything
with
and
when
I
was
asking
about
it,
when
this
was
being
worked
on,
the
guys
were
like
yeah.
D
What?
Basically,
whatever
you
want
to
do,
we
can
do
it
might
not
be
pretty,
but
it
can
be
done
with
Gravano,
but
it
is
a
bit
clunky
and
it
doesn't
give
you
that
nice
overall
overall
view
that
I
think
you're
talking
about
roll
it
can
be
created
in
a
way
and
I.
Think
that's
what
hugo
was
trying
to
do
initially
with
with
with
the
dashboard
that
I
was
and
I
quickly
demonstrated,
yeah.
D
But
actually
one
more
thing,
sorry
really
quickly
and
the
thing
that
this
doesn't
have
that
we
would
need
if
it's
like
90%
there,
then
that's
great
for
the
one
thing
that
this
doesn't
have
is
like
it's
taking,
JSI
VFS
nightly
and
updating
JSON
give
us
and
using
that
particular
version
in
the
tests.
It's
not
doing
so
what
we'd
have
to
do
is
and
change
it
so
that
it
took
go,
might
be
and
built
back
and
use
that
in
the
test
as
well.
So
that's
that's
the
Delta
there.
B
B
We
could
the
the
testing
the
testing
pipeline
could
also
be
performing
intelligence
to
extract.
You
know
metrics
of
interest
out
of
those
profiles
and
post
them
here,
as
well,
so
kind
of
like
you
know
we
want
this
to
be.
This,
is
god
this
would
be
kind
of
like
the
the
view
that
we've
been
conceptualizing.
It
is
very
like
this.
B
This
comes
from
from
inspiration
as
well
from
one
and
a
few
references
that
that
he
gave
us
at
the
beginning
to
you
know,
for
example,
if
you
look
at
the
other
chrome
benchmark
tests,
if
you
look
at
the
note
mean,
for
example,
that
kind
of
like
matrix
for
you
would
be
kind
of
like
the
main
source
of
inspiration
right.
We
want
a
matrix
where
we
can
see
means
yellows
and
reds,
and
it
becomes
immediately
obvious.
B
You
know
where
we
should
be
putting
our
attention
to
without
having
to
parse
it
right
right
when
you,
when
you
parse
and
try
it
like
I,
can
imagine
sorry
I
could
imagine
like
if
you,
if
you
model
this
up
or
finer,
then
it's
probably
going
to
be
looking
like
a
huge
either
a
huge
chart
with
a
lot
of
lines.
So
you
information
is
gonna,
get
lost
right
or
it's
gonna.
Look
like
a
dashboard
where
you
have
to
scroll
a
lot
and
down
right,
so
I
mean
I'm,
not
like
100%
sold
on
weather
graph.
B
Inge
is
in
the
sense
that
house
that's
performing
against
the
public
network
plus
is
performing
in
a
scenario
with
1,000
nodes
or
whenever
how's,
this
performing
against
another
implementation,
how
they
enter
operating
a
bunch
of
smoke
tests.
If
we
deploy
a
particular
release
in
like
a
soft
release
in
the
Gateway
or
a
bootstrapper,
what
are
the
metrics
right
that
we
capturing?
Is
there
looking
better
or
worse
than
yesterday?
Like
you
know
this
unified
view
that
is
intelligible
from
as
soon
as
you
as
you
go
in
as
intelligible?
E
Scientific
notebooks,
like
Jupiter,
for
sort
of
like
one-off
experimentation
and
I,
don't
like
that's
not
a
dashboard.
Obviously,
but
I
would
like,
like.
If
I
did
a
scientific
notebook
exploration,
I
want
to
publish
that
somewhere,
where
it's
visible,
so
I
like
that
to
be
linked
from
a
dashboard.
So
I
don't
really
know
how
that
will
work,
but
hopefully
have
some
prototypes
in
next
few
months.
A
I
want
to
continue
my
effort
to
bring
these
topics
meta,
to
give
us,
like
general,
guiding
principles
about
how
we
do
this
because
I
think
in
many
cases
that's
going
to
be
more
important
than
like
specific
decisions.
I
think
Stevens
point
from
before
that,
like
this
has
a
lot
of
what
we
need
already,
which
is
awesome
and
like,
let's,
let's
start
playing
around
with
the
benchmarks
data,
we
have
and
use
this
as
a
place
where
we
can
push
stuff
and
try
and
pull
it
from
different
directions
and
see
where
this
doesn't
meet.
Our
needs.
A
That
being
said,
principle
of
not
blocking
endeavors
across
each
other,
like
I,
see
a
lot
of
value
in
this
node
Green
style
thing.
I
also
agree
with
making
it
accessible,
but
to
the
extent
that
we
can
have
an
MVP
milestone
of
like
hey.
Let's
start
by
just
pushing
these
things
to
the
place
that
already
exists
and
hey
cool.
A
A
It's
like
great
I
can
get
this
thing
up,
but
I
also
want
to
do
this
other
thing,
because
in
this
air
for
this
slice
of
testing,
it's
gonna
be
really
critical,
but
no
other
team
is
dependent
on
me
doing
that
to
start
adding
their
own
things
to
something
that
already
exists.
So
I
guess
my
I
met
a
proposal.
There
is
like
this
seems
really
awesome.
A
Let's
start
pulling
around
with
it,
using
it
like
for
one
I
would
love
to
go
play
around
with
the
test
visualizations
because
to
make
it
more
accessible,
because,
let's
imagine
this
is
the
thing
that
we
are
using
for
test
visualizations
for
at
least
the
next
month.
We
probably
want
some
better
things
there,
but
a
month
from
now,
maybe
we
have
a
better
system
that
we
can
also
import.
This
data
into
I
think
that'd
be
grand
and
we
probably
have
value-adds
for
both.
A
D
F
From
go
side,
the
most
useful
things
we'll
be
able
to
run
a
go
test
or
go
benchmark
so
either
like
helper
functions
or
just
an
example
where
I
get
calling
from
a
program
and
then
to
some
example
like
there,
you
extracting
more
information
is
just
like
time
runs
me
and
with
all
the
good
stuff,
that's
somewhere.
That
would
be
awesome.
Yeah.
E
I
was
looking
at
the
the
machinery
that
runs
in
there.
It
would
be
totally
possible
to
just
run
that
stuff
with
cold
test
lab
inside
the
test
lab
environment.
If
we
want
to
scale
up
because
I
think
it's
only
like
maximum
5
nodes
right
now.
So
it's
like
we
wanted
to
try
bigger
things
than
do
that
across
the
cluster.
It
was
designed.
A
A
That
would
be
great
if
it
doesn't
then
say,
lovey
but
that'd
be
super
cool,
even
as
like
a
system
that
helps
us
unblock,
go
ipfs
releases
a
month
earlier
would
be
very
much
appreciated,
alright
to
go
back
through
the
document.
That's
even
created
yesterday,
there's
a
whole
ton
of
stuff
in
here,
which
is
great.
E
So
I
think
there's
some
low-hanging
fruit
in
terms
of
just
actual
test
cases,
I
was
playing
around
with
my
little
sweet.
Three
peer
in
testing
test,
lab
simulation
and
playing
around
a
bit.
Slop
and
I
could
just
tell
between
three
peers
that,
like
good
swab,
was
doing
some
bad
things.
So
I
think
that
would
be
an
obvious
one
to
focus
on
I
think
you
could
get
like
a
good
like
tears
here.
20
30
%
speed-up
would
be
reasonable
to
expect
to
be
able
to
see
like
the
graphs
move.
E
B
Yeah,
this
is
I
think
this
is.
This
is
something
we
need
to
go
through
synchronously,
because
there's
a
lot
of
brainstorming
to
do
here
in
general,
it
is
the
first
time
that
we're
getting
kind
of
a
consensus
of
what's
available
across
code
bases
across
to
the
layman
across
you
know,
like
all
the
menus
in
the
same
place,
and
also
during
the
brainstorming
of
what
kind
of
tests
we
want
in
the
same
place.
As
you
know,
that's
listing
any
more
tools.
B
We
already
have
I
think
we're
just
gonna
have
to
go
through
this,
so
synchronously
and
I
suggest
that
we
set
a
date
for
everybody
to
make
sure
that
it
gets.
You
know
that
everybody
gets
to
contribute
the
kind
of
tests
that
we
think
should
happen
and
any
others
there's
a
deadline
for
that
to
force
it
so
maybe
end
of
this
week
to.
A
But
if
you
don't
know
it,
you
can't
reuse
it
or
take
advantage
of
it,
and
so
that's
like
the
level
of
like
these
are
the
things
that
this
group
is
doing
from
a
testing
perspective
and
making
that
visible
to
all
of
the
other
groups
is
really
useful.
Otherwise,
everyone
continues
off
on
like
their
own
separate
threads
and
isn't
gonna
get
exposed
until
a
later
point.
That's
you
know,
hey
wait.
We've
been
doing
duplicate
work,
yeah.
B
So
I
think
in
terms
of
scope
of
what
this
particular
group
is,
has
set
out
to
focus
on
that
was
defined
in
the
OPR,
so
that
was
a
production,
canary
testing
and
different
levels
of
priority
private
network
testing.
But
having
said
that,
this
blue
is
building
test
in
front
for
all
of
my
BFS,
and
this
is
you
know
where
I
wanted
to
gather
requirements
from
all
of
ipfs
right,
because
we
can't
build
an
infrastructure
that
potentially
evolves
to
work
for
everybody.
B
If
we
don't
have
this
place,
where
we
connect
when
everybody
actually
that's
right,
so
this
is
like
I
think
the
school,
at
least
from
my
point
of
view
of
the
scope
for
testing
in
progress.
You
know
you
want
to
push
back
on
it
or
you
think
we
should
be
focusing
on
other
things.
I
think
the
focus
for
testing
fraud
in
this
quarter
is
sort
of
like
sealed
and
the
O'hara's
that
are
presented
a
few
weeks
ago
and
I
think
there
should
be.
It
should
be
gear
what
this
group
is
going
to
be
working
on?
E
Wonder
what
would
make
sense
to
put
like
the
the
IPS
benchmarks
project
under
the
infra
test
umbrella,
just
to
clarify
like
we're,
trying
to
decide,
decide
on
interfaces
here
things?
So
it's
like
having
like
islands
in
Stevens
group,
running
this
benchmark
system
and
then
Rho
and
me
building
in
Cole
building
another
system
and.
A
A
I
think
this
thing
that
exists
is
a
very
good
somewhere
to
start
with,
because
it
exists
and
that's
great
and
you
can
just
start
running
the
benchmarks,
so
I
think
this
should
be
our
default.
This
is
where
we
stick
the
thing
once
it
exists
and
if
you
then
take
that
thing
and
put
it
inside
test
in
front
wonderous
and
give
us
like
even
more
magical
things
on
top
of
it,
but
I
want
to
unblock
like
Stephen
Alan
focus
on
the
package
managers
team
under
Andrew
like
those
humans
from
like
writing
the
benchmarks
they
care
about.
A
One
I,
don't
think
that
the
two
of
you
have
enough
bandwidth
so
write
all
of
those
benchmarks.
I,
don't
think
that
we
want
the
testing
in
four
team
to
be
like
responsible
for
all
benchmark
writing
across
the
project.
I
think
that's
a
giant
load
of
work
and
you're,
not
gonna,
be
the
best
team
to
do
it,
and
so
there
is
like
tactical,
like
these
are
the
benchmarks.
A
We
need
that
I
think
needs
to
happen
on
other
teams
and
needs
to
be
unblocked
to
to
like
start
getting
progress
because
you'll
write
a
benchmark,
you'll
run
it
you'll
realize
you
actually
may
be
tuned
version
of
that
benchmark
and
you
want
to
be
able
to
iterate
without
being
blocked.
For
you
know
on
a
new
system
being
created
and
then
there's
the
the
gateway
work
that
Hector's
working
on,
which
is
like
hey.
We
should
have
nightly
mirrors
hey.
A
Wonderful
but
I,
don't
their
first
milestone,
shouldn't,
require
testing
in
from
being
done
or
require
the
testing
and
for
team
to
do
something
in
order
that
they
can
start
getting
value
from
that
and
using
that
to
iterate
on
on
gateway
production,
readiness
and
such
so
that's
my
understanding
of
the
way
of
the
world.
Jim
had
a
hand.
Roll
has
a
hey,
I,
gotta
run.
That's.
E
B
About
it,
so
let
me
I
think
I
see
your
point,
but
I
was
thinking
of
like
a
different
paradigm
and
a
mental
model
here
I.
Think
of
this
as
a
I
think
of
an
organization
and
enterprise
that
it's
building
my
groceries
right,
but
we've
got,
and
you
know
somewhat
Conway,
so
what
we
seen
here,
people
that
building
you
know
the
soffit
that
they
need
to
be
able
to
fulfill.
You
know
their
goals
and
fulfill
the
goals
of
the
group.
B
B
Does
that
continue
need
to
look
like
for
them
to
be
able
to
market
tests
and
in
terms
of
in
terms
of
gateway?
It's
got
like
the
the
same
thing
I
when
I
see
is
that
this
testing
pipeline
is
gonna,
be
the
orchestrator
right,
so
a
1-point
will
have
a
commit,
and
it's
gonna
be
a
night
me
and
it's
gonna
be
a
schedule.
B
A
They're
not
gonna,
invest
I,
don't
think
at
this
point
a
lot
of
time
in
creating,
like
you
eyes
and
fancier
orchestrators,
and
things
like
that,
with
the
assumption
that
this
baseline,
that
we
have
is
good
enough
for
our
use
cases
right
now.
It
sounds
like
it
can
do
all
of
the
things
we
want
for
the
benchmarks
we
know
about,
though
TBD
whether
Griffin
is
gonna.
B
Just
to
clarify,
like
these
two
sort
of
like
strands
of
work,
the
using
and
adapting
and
adopting
Jas
benchmarks
for
these
for
the
use
cases
that
we
want
to.
We
want
to
run
and
try
to
like
advocate
for
EJ
as
benchmarks
to
like
cover
the
minimal
thing
that
will
allow
is
to
into
a
block
idea.
Festivai
DFS
releases,
know
who's
gonna,
be
leading
this
particular
effort
like
this.
Looking
it
is
this
Alan.
Do
you
have
the
bandwidth
or
is.
F
D
Yeah,
so
the
takeaway
that
I
had
from
this
meeting
was
that
I
would
spend
some
time
making
this
less.js
focused
and
so
more
go
focus
but
bring
them
up
to
the
same
level
so
that
it
it's
not
something
just
for
Jess
and
I
might
need
some
help
with
that.
But
yeah
I,
like
I,
don't
have
a
lot
of
time
to
do
that
and
even
less
so
this
particular
period
I
have
a
baby
coming
soon
that
will
I'm
going
on
paternity
leave.
So
that
takes
up
a
chunk
of
my
time
as
well.
D
F
F
H
E
A
H
The
ability
to
run
them
locally
as
well
is
really
helpful,
because
we
know
at
the
point
where
we
like
we're
tracking
of
worrying
about
what
is
happening,
your
master
and
how
that's
affecting
it.
It's
much
more
like
when
you
insight
into
if
this
thing
that
takes
half
an
hour
to
run
on
a
regular
basis,
I
want
to
make
changes
on
a
branch
and
see
what's
doing
because
it's
too
much
to
for
a
developer,
to
be
running
it
and
then
going
to
make
a
coffee.
If
we
have
now
kind
of
thing.