►
From YouTube: Diablo: A Benchmark Suite for Blockchains
Description
A virtual meetup hosted by the Hyperledger Australasia Chapter on February 16, 2023 featuring Andrei Lebedev, a PhD student at the University of Sydney, speaking about the Diablo benchmarking framework. The focus of his research is evaluation and improvement of distributed systems.
With the recent advent of blockchains, we have witnessed a plethora of blockchain proposals. These proposals range from using work to using time, storage or stake in order to select blocks to be appended to the chain. As a drawback it makes it difficult for the application developer to choose the right blockchain to support their applications. In particular, the scalability and performance one can obtain from a specific blockchain is typically unknown. The claimed results are often obtained in isolation by the developers of the blockchain themselves. The experimental conditions corresponding to these results are generally missing and the lack of details make these results irreproducible.
In this paper, we propose the most extensive evaluation of blockchain to date. First, we show how the experimental settings impact the performance of 6 state-of-the-art blockchains and argue for more detailed experiments. Second, and to cope with this limitation, we propose a unifying framework to evaluate blockchains on the same ground. The framework includes a suite of 5 realistic Decentralized Applications (DApps), helps deploy the blockchain nodes at different scales and evaluate their performance. Finally, we show that selecting a particular virtual machine or weakening guarantees can help handle computationally demanding workloads but that none of the tested blockchains can yet support the load of these realistic DApps.
More about the Diablo benchmarking framework can be found at: https://diablobench.github.io/
----
Hyperledger – Open Source Blockchain Technologies
Hyperledger is an open source community focused on developing a suite of stable frameworks, tools and libraries for enterprise-grade blockchain deployments.
It serves as a neutral home for various distributed ledger frameworks including Hyperledger Besu, Fabric, Sawtooth, Indy, as well as tools like Hyperledger Avalon, Cactus and libraries like Hyperledger Aries, Ursa. Learn more about Hyperledger projects: https://www.hyperledger.org/use
Discord: https://discord.gg/rVCn4j7UFd
Case Studies: https://www.hyperledger.org/learn/case-studies
Training & Certification: https://www.hyperledger.org/learn/training
Tutorials: https://www.hyperledger.org/use/tutorials
Webinars: https://www.hyperledger.org/learn/webinars
Events: https://www.hyperledger.org/events
Vendor Directory: https://www.hyperledger.org/use/vendor-directory
Subscribe to the Hyperledger Newsletter: https://www.hyperledger.org/newsletter
Follow-us on Twitter @Hyperledger
Learn about Hyperledger Membership: https://www.hyperledger.org/about/join
#Hyperledger #Blockchain
License
Creative Commons Attribution license (reuse allowed)
A
A
We
are
now
Live
on
YouTube,
so
welcome
hello,
everyone
on
YouTube
and
as
everyone
here
in
in
the
in
in
the
meeting.
If
you
could
tell
us
where
you
are
that
that
would
be
great
in
in
the
in
the
chat
and
keep
the
chat
going.
So
Ali
was
going
to
run
it
today,
but
unfortunately
he
got
pulled
into
an
important
meeting.
A
Who
is
a
PhD
student
and
he's
going
to
be
talking
about
his
Benchmark
Suite?
As
you
can
see,
a
Diablo
I
have
just
one
or
two
housekeeping
items
which
I
had
to
mention
first
first,
this
is
a
Linux
Foundation
meeting.
So
that's
Foundation
hyperledger
is
one
of
the
projects
with
the
Lance
foundation.
So
we
have
to
talk
about
antitrust
policy.
So
if
you
want
to
know
more
about
that,
please
look
at
our
website
and
also
you
know.
A
B
Thank
you
again.
Yes,
hello,
everyone,
yes,
so
today,
there's
our
topic
is
measuring
the
performance
of
blockchains
and
so
first
of
all,
yeah.
We
are
dealing
with
this
well
idea
that
well,
let's
say
lately:
there
have
been
a
lot
of
different
blockchain
protocols
announced
by
different
companies
and
they
all
clean,
impressive
performance
like,
for
example,
algorand
claims
that
they
have
throughput
of
up
to
46
transactions
per
second
yeah
in
some
setting
Avalanche
and
other
example.
B
And
another
point
is
we
have
no
idea
what
was
the
content
of
transactions
that
they
sent
to
measure
such
impressive
performance?
And
so
here
we
on
this
slide.
We
see
the
overview,
so
the
one
of
the
points
that
they
claim
is
the
throughput
in
terms
of
transactions
per
second,
that
the
their
Network
can
process,
but
other
metric
is
latency.
So
that's
at
the
time
between
clients
sending
a
transaction
to
the
network
and
getting
a
confirmation
that
it's
been
committed.
B
So
it
is
an
open
source,
benchmarking
framework,
it's
written
in
Gold
Lane,
so
in
addition
to
unknown
workloads
that
that
were
mentioned
before
so
we
use
realistic
workloads
based
on
real
world
traces
from
applications
and
for
our
experiments.
We
geographically
distributed
the
machine,
so
we
used
10
different
regions
in
AWS,
which
I
will
show
in
detail
later
in
our
work.
We
evaluated
six
different
protocols
and
the
idea
was
that
we
try
well.
B
We
there's
different
well
like
protocols
with
different
consensus
properties
and
different
virtual
machines,
so,
for
example,
l,
Grant
and
Avalanche.
They
have
probabilistic
consensus.
So
when
the
transaction
is
considered
to
be
committed,
we
have
high
probability
that
it
won't
go
anywhere
from
the
blockchain
for
demon
Quorum.
We
have
deterministic
business
info
and
consensus
algorithms,
so
in
that
case
the
finality
is
guaranteed
and
also
we
have
ethereum
and
so
on
and
with
eventual
eventual
consensus
protocols
where
we
have
to
wait
up
until
several
confirmations.
B
So,
for
example,
several
blocks
to
be
committed
to
save
with
high
probability
that
the
transaction
won't
go
anywhere
and
as
well.
So
as
we
can
see,
they
have
different
virtual
machines
in
terms
of
smart
contracts.
So
for
our
work,
we
had
to
implement
our
workloads
in
different
programming
languages
to
to
be
able
to
compare
all
those
different
blockchain
protocols.
B
Here
you
can
see
the
architecture
of
Diablo,
so,
first
of
all,
it's
important
to
say
that
the
setting
that
we
used
to
evaluate
the
protocols-
it's
it's
realistic.
So
it's
not
a
simulator.
We
actually
deployed
the
binaries
that
are
compiled
from
the
source
code
available
well
on
GitHub
from
those
blockchain
protocols
and
for
for
Diablo.
B
We
have
a
primary
secondary
architecture
where
we
have
a
primary
which
orchestrates
the
the
whole
experiment
and
we
have
secondaries
which
send
the
actual
transactions
to
the
blockchain
network
and
so
for
the
primary.
We
have
two
different
configuration
files,
so
we
have
Benchmark
configuration
which,
which
tells
how
many
secondaries
do.
We
have
how
many
threads
are
used
to
send
the
transactions.
What
is
the
load,
distribution
and
the
transaction
parameters,
and
also
we
have
the
blockchain
configuration
so
what
type
of
blockchain
it
is?
B
B
And
in
general
the
idea
is
that
in
the
beginning,
we
externally
set
up
a
blockchain
like
a
private
blockchain
Network
on
the
machines
that
we
allocate.
Then
we
deploy
Diablo
with
possibly
multiple
secondaries
in
different
regions
and
so
on
the
primary.
We
only
have
the
descriptions
of
the
transactions
to
be
sent
then
on
the
second
there
is,
we
generate
the
actual
transactions
we
send
them.
We
we
note
the
time
when
we
actually,
when
we
send
the
transactions
from
the
client,
then
we
wait
for
the
confirmation
from
the
blockchain
network
on
the
secondary.
B
For
the
performance
comparison
like
I
mentioned,
so
we
in
our
experiments,
we
used
geographical
distributed
topology,
so
we
used
10
a
subset
of
10
AWS
regions
in
different
countries,
so,
for
example,
United
States,
Brazil
so
up
to
up
to
CD,
and
so
with
this
setting,
we
wanted
to
still
to
still
have
a
have,
an
idea
of
like
a
real.
Let's
say
what
can
be
a
real
Network,
so
we
have
different
countries
and
we
have
varied
latency.
B
From
the
so
and
also
from
these,
then
we
10
regions
we
used
diff
well
for
different
experiments.
We
used
different
configurations
so,
for
example,
for
data
center,
we
used
only
a
single
region
in
the
United
States,
so
there
is
Center
in
Ohio,
which
is
a
powerful
machine
with
a
lot
of
memory
and
CPUs
and,
for
example,
this
the
notes
like
a
data
center
of
a
single
company.
B
B
Then
we
have
a
devnet
where
we
used
all
the
10
regions
that
I
showed
in
the
previous
slide,
and
this
can
be
used
by
the
developers
to
test
the
realistic,
latency
so
well
to
check
how
the
protocol
operates.
When
it's
not
in
a
single
Data
Center,
then
we
have
a
community
configuration
which
mimics
the
real
world
mainnet
in
a
way
that
it's
commodity
machines
run
by
actual
individuals
and
we
have
well
so
we
have
200
blockchain
nodes
in
total,
that's
20,
blockchain
nodes
in
each
of
the
10
regions
and
as
well.
B
So
to
describe
the
realistic
workloads
that
we
developed
to
evaluate
these
blockchains,
the
idea
is
like
the
the
main
key
Point
here
is
that
we
took
traces
of
the
real
applications
like,
for
example.
Well,
you
can
see
the
names
and
well
before
going
into
the
detail,
just
to
mention
that
these
workloads
have
different
characteristics
so,
for
example,
NASDAQ
which
well,
which
shows
how
a
how
Stock
Exchange
works
in
the
beginning
of
the
day.
B
So
we
have
a
high
peak
when
the
trading
starts,
and
then
we
have
much
less
workload
throughout
throughout
the
day
and
so,
for
example,
this
characterizes
the
burst
load.
Let's
see,
we
can
test
how
how
the
blockchain
can
handle
the
burst
load
then,
for
example,
Uber
which,
which
is
about
well
like
running
a
taxi
service.
So
here
the
idea
is
that
we
have
to
calculate
some
distances
in
the
city,
and
so
this
is
a
characteristic.
B
B
To
start
showing
the
results
first,
we
want
to
overview
all
the
different
workloads
and
different
protocols.
So
here
you
can
see
in
the
columns.
We
have
different
smart
contracts
that
we
used
and
you
can
also
see
the
average
workload
in
terms
of
transactions
per
second
on
top,
so
it's
ranging
from
168
transactions
per
second
up
to
30,
almost
38
000.,
for
the
rows.
We
have
the
average
throughput
measured
during
the
whole
experiment.
B
Also,
what
is
important
to
mention
is
some.
In
some
cases
there
are
no
results
at
all
for
for
the
ratio
and
for
the
or
you
cannot
see
what
are
the
results
for
the
ratio
and
also
no
throughput.
So
it
might
be
the
case
that
the
blockchain
didn't
handle
the
workload
at
all.
So,
for
example,
with
Uber,
we
have
quite
highly
High
intense
workload
and
some
blockchains
they
in
the
transaction.
They
limit
what
is
called,
usually
gas
or
the
the
number
of
computations
that
can
be
done
in
single
transactions.
B
So
in
some
cases
we
couldn't
make
the
blockchain
to
execute
a
transaction
at
all.
So
there
are
might
be
different
reasons
for
not
having
no
results
for
a
particular
blockchain
and
experiment,
but
going
to
the
individual
results.
We
can
see
that
for
Nasdaq,
all
of
the
blockchains
managed
to
commit
something
and
maintain
some
throughput
with
quorum
having
the
highest
average
throughput
throughout
the
experiment,
and
the
idea
here
well
is
that
the
workload
is
not
high
and
Quorum
with
deterministic
consensus
algorithm.
So
well,
let's
see
by
implementation,
deterministic
consensus,
algorithms.
B
B
But,
however,
if
you
look
at
Dota,
2
and
YouTube,
we
have
no
results
for
Quorum
and
that's
because
during
the
if
the,
if
the
workload
is
constant
and
high
well
in
this
case,
we
we
might
have
the
over
well
like
the
cues
or
the
transaction
cues
or
transactions
we'll
go
overfill.
And
so
we
don't.
We
cannot
measure
any
results.
B
So
that's
the
idea
for
the
deterministic
consensus
algorithm,
so
we
can
also
note,
for
example,
ethereum
and
Avalanche,
which
have
quite
High
latency.
So
in
this,
in
this
case,
we
can
also
have
some
idea
that
those
two
protocols
they
they
have
a
fixed
block
rate
and,
in
this
case
the
email
to
optimize
the
latency,
but
still
to
have
some
throughput
so
not
to
so
to
be
to
be
available
in
general.
B
So
this
is
the
overview
of
different
results.
Now
we
can
go
into
the
detail
of
different
experiments,
so,
for
example,
here
we
have
also
the
average
throughput
and
latency
for
for
the
workload
of
my
constant
Trend,
constant
workload
of
1000
TPS
of
usual
transfers,
so
not
smart
contracts,
and
here
what
we
wanted
to
test
is
how
the
blockchain
scales
in
terms
of
clients
and
deployment.
B
Topologies
that
I
mentioned
before
so
either
one
region
or
10
regions
and
also
in
terms
of
different
Hardware
and,
as
we
can
see
here
so
the
the
blockchains
perform
more
or
less
the
same
in
terms
of
when
we
scale,
for
example,
from
commodity
Hardware
up
to
powerful
Hardware.
So,
let's
test
net
and
data
center
and
well
yes.
So
what
we
can
see
here
is
that
we
have
relatively
low
throughput
and
high
latency
of
ethereum
and
Avalanche,
which
can
be
also
explained
by
having
a
consensus
algorithm
with
a
fixed
block
rate.
B
But,
however,
what
is
interesting
to
note
if
we,
if
we
go
from
one
data
center
to
10
geographically
distributed
data
centers
for
GM?
We
can
see
that
the
throughput
and
drastically
lowers
from
almost
1000
TPS
to
around
200
DPS
or
even
lower,
if
we
add
up
more
machines
and
what
we
can
like.
The
conclusion
that
we
can
make
from
this
is
that
DM
was
optimized
for
low
latency
networks
and
probably
wasn't
optimized
for
for
geographically
distributed
Network.
B
We
can
also
see
that
Solana
manages
to
like
Solana
is
one
of
the
would
say
the
only
blockchain
that
manages
to
handle
the
throughput
in
different
in
different
topologies,
and
the
idea
here
is
that
Salama
has
a
verifiable
delay
function
which
helps
it
not
to
depend
on
the
number
of
machines.
So,
for
example,
if
we
increase
the
number
of
machines
from
10
to
200
for
some
blockchains,
we
see
we
see
the
increase
in
latency
and
decrease
and
throughput.
However,
for
Solana,
that's
not
the
case.
B
What
we
want
to
test
is
how
the
blockchain
is
robust
against
like
denial
of
service
attacks,
so,
for
example,
how
it
can
handle
such
high
workload
if
someone
wants
to
run
such
an
attack,
and
so
here
in
different
columns.
First
of
all,
you
have
we
have
different
blockchain
protocols
and
we
have
the
configurations
in
which
they
perform
the
best,
and
we
also
have
the
average
throughput
and
latency
what
we
can
note
here.
C
B
For
DM
it
doesn't
so
it
doesn't
crash.
However,
the
throughput
goes
down
almost
10
times
so
from
1000
TPS
to
100,
TPS
and
the
latency
goes
up.
However
yeah
it
still
doesn't
crash,
so
in
this
case
the
transaction
cues
that
it
that
it
has
so
it
helped
it
helped
to
not
to
crash
and
still
able
to
commit
some
transactions
for
Solana.
We
have
well.
We
see
that
the
latency
increases,
so
it
takes
more
time
to
process
the
transaction.
B
However,
also
the
the
network
doesn't
stall,
but
in
case
of
quorum
we
we
see
that
with
high
workload,
it
crashes
and
we
cannot
get
any
results.
So
it's
it
cannot
handle
the
the
what
we
mimic
as
a
denial
of
service
attack
for
Avalanche
and
El
Grant.
It's
also
well
hard
to
hard
to
give
more
details
on
this,
so
they
will.
B
I'm
sorry,
the
number
of
the
guess,
the
amount
of
guests,
so
the
number
of
computations
that
we
can
have
in
a
single
transaction
is
limited.
So
that's
why
we
couldn't
execute
this
workload.
However,
for
the
three
other
blockchains,
all
of
them
use
ethereum,
Virtual
Machine,
and
so
we
can
see
that
in
this
case
we
don't
have
the
art
capacity,
hard
upper
limit
for
the
number
of
computations
that
can
be
done
in
the
transaction.
B
So
we
have
and
we
have
the
results
for
the
three
blockchain
and
use
ethereum,
Virtual,
Machine
and
again
here
with
quorum,
because
you
know
because
of
the
deterministic
consensus
algorithm
and
the
idea
that
it
tries
to
commit
to
all
of
the
clients.
So
we
see
that
it
maintains
the
throughput
of
600
transactions
per
second
with
relatively
what
was
acceptable
latency
of
25
seconds
for
ethereum
and
Avalanche.
B
So
Google
has
800
EPS
Peak
loads.
Apple
has
10
000,
so
that's
the
highest
in
this
case
and
Microsoft
has
4
000
transactions
per
second
Peak
load
in
the
beginning,
and
here
we
can
see
that
quorum
manages
to
commute
all
the
transactions,
so
the
CDF
goes
up
to
one
and
it
manages
to
commit
them
under
10
seconds.
So
that
is
given
the
the
feature
described
before
was
the
deterministic
consensus
algorithm.
B
If
we
look
at
another
blockchain
with
deterministic
Concepts
algorithm
so
DM,
it
manages
to
also
commit
all
of
the
transactions
for
Google
and
Microsoft,
so
800
and
4000
TPS
with
quite
low
latency.
However,
if,
if
we
go
like,
if
we
scale
the
load
up
to
10
000
TPS
in
the
peak
it
doesn't,
it
only
manages
to
commit
around
well
like
higher
than
around
75
percent
of
the
transactions?
B
What
we
can
say
here
is
that,
while
the
transaction
pool
might
not
be
limited,
we
still
have
the
fixed
block
rate
and
that's
why
it
takes
a
lot
of
time
to
commit
all
of
the
transactions,
and
we
can
also
note
or
like
a
similar
Behavior
in
the
in
the
other
experiments
for
ethereum.
So
technically,
if
we
would
have
waited
long
enough
time,
we
could
have
seen
that
more
transactions
were
committed.
However,
we
limited
our
experiment
to
up
to
when
the
transactions
were
actually
sent.
B
However,
the
previous
results
might
have
been
quite
pessimistic,
so
not
that
we
in
the
first
slide
for
the
experiments
we
showed
that
none
of
the
blockchains
managed
to
commit
all
of
the
transactions,
so
the
the
ratio
was
well.
The
ratio
of
committed
to
civilian
transactions
was
less
than
one.
We
like
a
colleague
of
mine,
another
student
at
the
University
of
CE,
used
Diablo
to
evaluate
the
smart
red
belly
blockchain.
So
that's
the
so
first
there
was.
B
There
is
a
red
belly
blockchain
and
a
smartraid
blockchain
integrates
ethereum
virtual
machine,
and
so
that's
why
now
we
can
test
the
smart
contracts
together
with
other
blockchains
and
in
this
particular
experiment.
We
so
the
the
quality
views
the
get
fam
workload.
So
that's
also
the
NASDAQ
smart
contract
with
the
peak.
B
However,
the
peak
the
peak
load
here
is
even
higher,
so
that's
around
20
000
transactions
per
second,
and
what
we
can
see
here
is
that
the
smart
Trade
building
blockchain
manages
to
commit
all
the
transactions
under
20
seconds
and
during
the
peak
we
have
quite
high
throughput
of
well
around
3
000
transactions
per
second
here.
So
we
still
have
hope
that
all
there
is
still
work
to
do
to
to
be
able
to
execute
the
real,
the
real
workloads
and
realistic
applications
in
blockchain.
B
All
right,
yes,
so
machines
with
such
configuration,
and
we
have
several
aisles
and
in
general,
what
we
wanted
to
show
here
is
how
we
can
reproduce
the
the
geographically
distributed
scenario
in
a
local
setting,
because,
on
the
one
hand
in
the
in
AWS
setting
you
have
a
real
Network
so
that
the
latencies
can
also
fluctuate.
During
the
day.
B
B
Well,
adding
artificial
latency
in
this
case
through
the
network
links
and
so
here
what
we
did
is
that
we
have
several
aisles
and
in
each
aisle
we
have
yeah
also
as
an
aw
as
an
AWS.
We
have
Diablo
and
the
blockchain
nodes
and
in
different
experiments
we
add
artificial
latency
and
so
the
actual
results
we
can
see
in
the
following
slides.
B
So,
first
of
all,
we
also
wanted
to
test
the
scalability
now
not
in
terms
of
clients,
but
in
terms
of
adding
machines,
and
in
our
case
each
aisle
had
five
blockchain
nodes.
So
in
this,
so
we
go
up
to
from
five
blockchain
nodes
up
to
35
blockchain,
and
now
we
just
use,
we
don't
add
any
latency.
So
this
can
be
understood
as
a
single
Data
Center.
B
Here
we
can
still
see
that
the
measured
throughput
is
stays
relatively
the
same
for
for
all
of
the
protocols.
However,
in
terms
of
latency,
we
can
note
and
the
difference
and
like
what
is
quite
important
to
look
at
is
quorum,
so
here
we
have
not
the
average
latency,
but
we
have
the
box
plot,
and
so
here,
on
top,
we
have
the
maximum
measured
latency
for
all
of
the
for
all
the
committed
transactions
that
we
measured.
B
B
While
it
works
well,
we
can
see
that
we
still
have
some
measured
where
you
can
see
for
for
well
for
some
topologies.
Basically,
the
throughput
is.
We
cannot
see
the
throughput,
so
it
doesn't
manage
to
handle
this
workload
also,
as
we
have
seen
in
the
previous
experiments
with
AWS
and
for
other
blockchains
for
all
of
them.
We
see
how
now
the
latency,
the
average
latency
and
also
the
maximum
latency
increases
almost
up
to
100
seconds.
That's
the
duration
of
our
experiment,.
B
And
now
we
can
look
at
the
the
results
where
we
emulate
the
delete
and
see
so
here
we
use
all
of
the
seven
aisles,
so
that's
35
blockchain
nodes,
and
here
we
change
the
delay
from
zero
milliseconds.
So
that's
the
original
setting
up
to
300
milliseconds
between
each
aisle
so
and
one
aisle
consists
of
five
blockchain
nodes
and
here
in
the
first
experiment,
with
100
transaction
per
second
workload
of
native
transfers.
B
We
can
already
see
how,
for
example,
for
GM
the
so
the
maximum
and
minimum
a
measured
latency,
so
the
difference
between
them
increases
and
also
we
as
well
see
the
increase
in
average,
measured
latency,
and
so
with
this
we
can
and
well.
We
also
see
the
the
corresponding
drop
and
throughput-
and
this
confirms
our
previous,
our
previous
idea
as
well,
that
DM
was
optimized
for
low
latency
and
aiding
the
delay
between
the
nodes
and
negatively
it
affects
it,
performs
its
performance.
B
However,
all
the
other
protocols,
many
well
like
managed
to
also
maintain
maintain
the
constant
throughput
and
latency
more
or
less,
during
the
experiment,
with
different
delays
with
this
Oracle
and
then
we
also
increased
the
workload
to
1000
TPS,
and
here
we
already
see
the
different
behavior
in
terms
of
throughput,
so
with
GM.
Well,
we
see
the
same
behavior
but
well.
The
the
effect
is
impacts
the
performance
more,
so
we
have
much
more
drop
throughput
and
also
higher
increase
in
latency.
B
B
B
B
B
However,
for
we
see
a
much
better
performance
for
Native
token
transfers,
which
so
and
with
this
we
can
have
an
idea
that
Solana
manages
well,
it
handles
the
native
transfers
better
than
the
smart
contracts,
so
possibly
because
of
lower
instruction
count,
and
also
some
other
optimizations
built
in
the
protocol
and
also
as
before.
We
see
that
l
Grant
manages
to
commit
some
of
the
transactions
as
well
as
DM.
B
So
it's
also
important
to
mention
that
this
is
not
well
like.
This
is
not
the
first
attempt
to
actually
measure
the
performance
of
the
blockchain,
so
other
protocol,
well
other
benchmarking
Frameworks
were
proposed
before
so.
There
is
also
now
an
active
project
from
hyperledger
called
hyperledger
caliper,
with
which
you
can
also
measure
the
performance
of
hyperledger
fabric.
However,
it
doesn't
like
in
in
suit
it
doesn't
feature
any
realistic
workloads
and
only
features
synthetic
workloads
and
with
synthetic
workloads.
B
Well,
you
you
can
measure
some
performance,
as
we
have
seen
for
the
constant
workloads
with
Native
token
transfers,
but,
for
example,
we
can
we
cannot
account
for
or
we
cannot
compare
the
performance
in
terms
of
Peaks
or
High
computers.
We
have
seen
in
NASDAQ
or
Uber
workloads
that
we
have
developed.
B
There
is
also
a
book
bench
which
was
Well
published
earlier,
so
it
features
also
synthetic
workloads
for
databases
which
are
important
but
as
well.
It
doesn't
reflect
the
real
world
workload
as
in
our
work,
and
there
is
also
chain
Hammer,
which
is
specialized
for
ethereum,
and
it
only
features
a
continuous
High
load.
So
we
cannot
vary
the
workloads
during
the
experiment
like
having
Peaks
and
so
on.
B
And
yes,
of
course,
at
last,
it's
I
would
like
to
say
that,
yes,
as
I
mentioned
in
the
beginning,
Diablo
is
an
open
source
project.
So
all
of
the
code
is
available
on
GitHub
and
we
are
very
welcome
to
the
contribution.
So
we
would
like
to
support
other
protocols.
Then
then
we
tested
so
that
we
can
have
more
results
and
have
a
fair
comparison
of
all
the
different
blockchain
protocols.
B
So
you
can
access
the
web
page
on
the
link
now
displayed
in
the
slide,
and
in
this
website
we
have
all
of
the
instructions
how
you
can
well
like.
How
can
you
either
repeat
the
same
experiments
as
we
did
in
the
paper
or
how
you
can
make
your
own
experiments
locally,
and
also
we
provide
a
virtual
machine
image,
so
you
can
even
try
running
some
experiments
like
without
doing
some
complicated,
setups
and
configurations
on
your
own
machine.
A
A
Yeah,
so
we
have
a
bunch
of
we
have
a
bunch
of
questions
right.
Can
you
see
the
questions
in
the
in
the
chat,
so
the
normal
way
I
think
we'd
like
to
do
this?
Is
it's
people
to
put
your
hand
up,
and
then
maybe
you
can
answer
the
questions
or
we
can
just
go
through
the
maybe
I'll
leave
it
under
it
to
go
through
the
questions,
but
if
anyone
asks
a
question
put
your
hand
up,
of
course
we
have
so
many
people.
I
have
lots.
A
D
A
D
Years,
I'm
recently
moving
to
this
blockchain
platform
and
learning
a
lot
of
things
and
currently
focusing
mainly
on
the
Enterprise
applications
and
that's
the
reason
why
I
asked
for
this
hyperledger
part
because
I
haven't
found
good
blogs
and
good
articles
on
hyperledger
I.
Don't
know
why
it
is
so
when
I
go
in
ethereum
part
I
can
see
a
lot
of
things,
a
lot
of
contribution
by
different
developers,
but
hyperledger
is
still
maybe
not
mature
enough.
To
be
honest,
that's
what
my
limited
understanding.
B
So
yeah
as
because
I
met
as
I
mentioned,
so
we
we
didn't
really
test,
for
example,
hyperledger,
Fabric
or
hyperledger
Sawtooth
or
euroha
so
well.
There
are
different
reasons
for
it
like,
for
example,
hyper
Ledger
fabric
involves
quite
a
let's
say,
sophisticated
deployment,
and
so
we
would
really.
We
would
really
like
to
support
it
as
well
to
to
measure
the
performance
and
to
compare
it
with
others.
A
So
I
think
it's
a
future
right,
so
I
think
I
think
just
outside
the
answer
to
your
to
your
your
queries.
There
Nishant
yeah.
Definitely
if
you
can't
find
hyperledger
stuff
there's
there
is
a
lot
of
stuff.
It
is
mature
right.
It's
been
around
a
long
time
right,
so
I'm,
happy
I,
don't
know.
Have
you
looked
at
the
hyperledger
wiki
at
all,
wiki.hyperledger.org.
D
A
A
Okay,
so
we
have
a
I'll,
tell
you,
what
send
me
a
link
and
I
can
connect
you
with
the
Indian
chapter
there
we
have
a
whole
Community
right
that
you
can
connect
that
can
help
you
in
in
India,
right
and
globally
right.
So
so
also
another
thing
we
did
mention
that
canapa
we
have
caliper
that
already
does
benchmarking
for
hypolator
fabric.
A
E
Hi
hi
everyone
thanks
for
the
great
presentation,
I.
Imagine
a
lot
of
work
went
into
this
as
just
by
the
sheer
number
of
plots
he
showed.
So
congratulations
on
the
good
work.
Hi,
everyone,
I'm
Attila,
Clinic
researcher,
at
The,
Budapest,
University
of
Technology,
and
also
maintenance
of
high
pressure,
calipers,
so
I
think
there
is
an
interesting
overlap
here.
So
yeah
my
I
posted
four
questions
in
in
the
chat
in
order
of
importance.
My
first
would
be.
E
E
C
Yeah
hi,
thanks
for
the
presentation,
Android
really
thanks,
so
I
remind
my
time
at
the
University
of
New,
South
Wales,
just
because
I'm
a
little
bit
jealous
of
where
it's
sitting
right
now.
But
here's
a
question
about:
oh
okay,
my
background
I'm,
professor
of
computer
science
or
program
languages
at
the
University
of
Copenhagen
I'm,
also
here,
research
of
a
company
Dion
digital,
it's
working
on
Capital,
Market
infrastructure
based
on
DLT.
So
first
question
is
so
none
of
the
systems.
C
If
you
correct
me,
if
I'm
wrong
here,
is
it
right?
Not
a
systems
actually
have
sharding
or
you
know,
they're,
basically,
all
of
them
solving
the
problem
of
putting
all
the
transactions.
The
messages
that
you're
submitting
into
a
single
sequence
is
that
correct
I
mean
essentially
right,
like
a
blockchain.
B
C
B
B
So
the
the
idea
is
that
well
like,
while
we
have
quite
a
simple
smart
contract,
so
the
the
workload
Trace
like,
for
example,
having
having
a
peak
value
at
the
particular
time
Point
in
the
experiment
that
is
gathered
from
like
from
a
let's
say,
a
real
Trace
like
so
we
took
a
single
day
in
the
NASDAQ
and
then
like
the
beginning
of
the
day
and
then
map
those
values
into
the
experiment.
So
at
a
particular
time
point
we
send
a
particular
a
particular
number
of
transactions.
C
B
B
However,
we
might
expect
some
blockchains
to
well
like
to
handle
the
contention
a
bit
easier
in
the
sense
because
they
might
be
accessing
different,
well
different,
different
key
value
pairs
in
the
map,
so
it
will
be
so.
The
blockchain
will
still
have
to
handle
the
high
peak
workload
so
the
burst,
but
we
might
also
see
how
it
works
in
terms
of
accessing
different
key
value
pairs.
B
A
A
Well,
it's
a
seasonal
thing.
It's
a
seasonal
thing,
all
right,
I
think
we've
got
a
bunch
of
questions
in
the
in
the
chat
right
still,
so
everyone
put
your
hands
up.
You
want
a
lot
of
people
are
saying,
you
know
what
a
great
presentation.
Thank
you
Jerome's
asking
and
a
lot
of
people
asking
you
doing.
Are
you
going
to
try
test
on
other
other
networks?
So
we
have
Jerome's
asking
do
you?
Do
you
try
tests
on
cardano,
too
Andrea.
B
Yeah
polygon
and
cardano
yeah,
so
we,
yes,
we
didn't
test
those
those
two
protocols,
so
we
like,
let's
say
in
terms
of
in
terms
of
apis-
well,
not
well.
I
can
yes,
I,
give
more
details:
how
of
for
the
implementation
of
the
of
Diablo
and
particular
experimental
settings.
So
the
idea
here
is
that
in
Diablo,
what
we
have
is
a
like
an
extensible
API,
so
you
can
just
write
a
module.
B
You
can
implement
the
interface
like
a
simple
interface
of
practically
four
main
methods
like
creating
the
transaction
and
encoding
some
values
into
protocol
specific
format
and
also
sending
it
well
with
the
network
so
then,
onto
the
blockchain
network,
with
the
specific
protocol
used
and
in
this.
In
those
terms,
we
have
implemented
the
interaction
with
so
we
use
the
web3
interface,
which
is
used
by
ethereum,
also
used
by
Avalanche
in
C
chain
and
also
used
by
quorum,
so
Quorum
reuses,
the
so
all
of
them.
B
We
use
the
ethereum
virtual
machine
code
base
in
this
sense
and
we
can
interact
them
with
a
single
API
and
technically.
If
you
want
to
test
some
blockchain,
which
already
supports
web
3
API,
you
can
basically
plug
and
plug
it
in
right
away
and
only
write
your
deployment
scripts
or
like
deploy
your
own
private
Network
as
you
want
to
and
then
just
point
the
Apple
to
it
and
you
will
be
able
to
run
the
same
workloads.
B
And
yes,
if
you
would
like
to
add
the
support
to
other
for
the
other
protocols,
then
yes,
there
are
several
examples
and
the
code
base
like,
for
example,
we
have
so
we
have
the
web3
interface
that
I
mentioned.
We
have
the
DM,
that's
the
interface
for
DM
and
interface
for
Solana
and
also
for
algorithm.
So
you
can
look
at
those
different
implementations
and
just
we'll
implement
it
for
for
the
protocol
that
you
want
to
test,
but
yes
like
to
and
specifically
to
answer
those
questions.
B
B
B
So
now
going
to
the
question
from
so
the
reason
for
workouts
for
Coach
unification
is
like
that.
That's
what
we
answered.
So
the
idea
was
to
show
different
different
features
of
centralized
applications
and
how
blockchain
can
handle
it.
Failure
types,
yes,
so
that
we
also
mentioned
tuning
tuning
the
networks.
Yes,
that
is
a
good
question.
B
It's
yeah
it's
important
to
mention
this
as
a
limitation
in
the
in
the
experiment
for
the
experiment.
So
yes,
what
what
we
did
is
that
we
took
the
default
configuration
that
is
provided
by
by
the
protocol
and
also
what
they
recommend
and
so
also
not
to
make
any
custom
any
custom
tuning
from
our
site.
B
We
just
went
with
the
default
configuration
that
they
provide
and
wanted
to
make
sure
that
we
compared
the
different
protocols
on
the
same
ground
as
fairly
as
possible,
but,
as
you
like,
as
you
say,
is
it
possible
to
tune
it
to
have
to
achieve
higher
performance?
Yes
like,
for
example,
in
some
settings,
for
example,
in
a
private
network
of
not
so
many
nodes
and,
for
example,
in
the
same
data
center?
B
For
the
scalability
of
the
workload
generation,
with
a
single
instance
primary
so
also
going
into
the
implementation
here,
it's
important
to
say
that
the
the
primary
like
the
single
instance
of
primary
only
distributes
the
configurations
between
the
set,
and
there
is
so
that
and
it's
a
minimalistic
descriptions
without
any
key
pairs.
So
it
doesn't
have
any
fully
signed
transactions,
so
the
the
used,
the
bandwidth,
is
quite
low,
and
it's
so
in
this
same.
In
this
sense,
the
the
workload
is
generated
on
the
second.
B
There
is
either
during
the
experiment
or
before
the
experiment
and
the
scalability
is
well.
It
totally
depends
on
the
hardware
that
you
use
so
like
how
many
vcpus
does
the
machine
have
how
much
memory,
and
also
in
this
sense,
how
many
secondaries
do
you
want
to
run?
Do
you
have
to
so
that
you
can
run
how
many
machines
and,
like
all
of
these
different
parameters,
affect
the
scalability
performance,
so
in
our
experiments,
so
we
we
used.
B
Then,
second,
there
is,
where
possible.
So,
for
example,
it
wasn't
possible
to
do
it
in
the
data
center
setting.
However,
with
the
data
center
setting,
we
had
already
quite
powerful
machine,
so
even
less
secondaries
were
enough,
so
we
have
seen
that
they
were
not
fully
saturated,
but
for
the
other
scenarios
having
only
10
secondaries
in
the
whole
network,
so
having
a
single
secondary
nature
of
the
regions
that
proof
well
like
we
have.
We
have
also
seen
that
it
is
enough.
It
was
enough
to
support
our
workload.
B
Yes,
so,
as
Nishant
asked
before,
yes
for
hyperledger,
we
would
really
yes
appreciate
to
or
your
contributions
to
have,
the
support
for
it.
So
why?
Why
didn't
we
evaluated?
Why
why
we
didn't
evaluated?
Yes,
yes,
because
of
the
higher
complexity
in
terms
of
in
terms
of
deployment,
and
we
also
wanted
to
focus
more
on
on
the
protocols
that
had
main
Nets
or
which
were
more
like
a
public
blockchains.
B
A
B
B
We
can
have
a
Fair
setting
in
this
sense
because
actually
different
the
different
programming
languages
for
evm,
they
introduce
different
limitations,
so,
for
example,
solidity
and
DVM
might
have
like
have
one
set
of
instructions
and
the
move
and
drawing
reduce
the
ethereum.
Oh
sorry,
in
in
El
Grant.
Yes,
they
have
their
own
limitations.
B
Yes,
so
there
is
a
question
to
you
know:
to
learn:
creating
a
hyperledger
network
from
scratch.
Yes,
I
would
recommend
High
vs2
first
of
all
mention
yeah
like
have
an
idea
which
particular
hyper
Ledger
blockchain
you
would
you
want
to
use
like
Fabric
or
or
something
else
and
then
go
to
the
corresponding
wiki
page
and
documentation
and
yeah.
So,
oh
well,
Julian
answer
that.
Thank
you.
A
Andrea,
that
was
great.
Actually
just
one
general
question
about
you.
This
is
outsourced
Diablo
right.
What's
the
kind
of
plans
for
that?
What's
the,
how
do
you
you
know
people
get
involved?
Is
it
gonna?
You
know
we've
got
caliper.
Do
you,
you
know,
there's
all
kinds
of
what's
what's
the
kind
of
thoughts
moving
forward
for
the
platform.
B
E
B
B
Well,
let's
say
the
same:
module,
the
same
interface
we're
sending
the
transactions
so
that
we
can
all
support
this
like,
for
example,
if
you
implement
the
blockchain
support
only
once
you
can
use
it
with
Diablo
and
caliper,
but
yeah
so
in.
In
this
sense,
we
would
just
yes
like
to
have
these
two
different
projects,
so
they
have
their
their
own
contributors
and
well
just
like
keeping
it
open
sourced
and
that's.
A
Great,
this
variety
is
good
right,
so
I
yeah,
exactly
and
ultimately
some
of
these
things
merge
and
all
kinds
of
things
happen
in
the
open
source
community.
So
yeah,
that's
great,
so
I
think
we've
got
I.
Think
I
think
we've
had
a
lot
of
questions
and
lots
of
compliments
to
you
Andre,
so
I
think
that
was
a
great
presentation
right.
Thank
you
great
material.
You
know.
Obviously,
a
lot
of
effort
went
into
that
right
and
really
good
outcome.
A
So
I'd
like
to
say
thank
you
to
you
and
thank
you
to
everyone
listening
here.
This
will
be
on
YouTube,
so
people
be
able
to
watch
it
again
and
again
and
I
will
try
and
put
some
of
your
links
and
below.
So
if
you're
watching
Youtube
will
get
that
and
I
think.
Unless
and
maybe
you
want
to
say
a
few
last
words
Andrea
and
then
I
think
we
can
wrap.
B
A
D
A
B
Yes,
so
I
am
available
on
LinkedIn
and
also
probably
I'm
also
available
on
yeah
on
this
hyperledger
Discord,
so
I
I
was
also
like
previously
I
was
a
maintainer
for
hyperledger,
so
there
I
think
I'm
listed
as
maintainer
still
and
another,
probably
the
short
way
to
communicate
this
email.
So
in
the
in
the
Diablo
paper
that
is
on
the
website,
there
is
my
email
and
you
can
use,
or
people
can
use
that
as
well.
A
A
Yeah,
it's
it's
still
going
I'm
very
much
in
contact
with
this
excellent.
So
actually
just
the
question:
do
you?
How
do
you
think
people,
maybe
sorry?
How
do
you
think
people
should
contribute
or
how
do
people
come
and
maintain?
Have
you
got
any
advice
on
that
people
want
to
contribute
back
to
these
open
source
programs.
B
In
general,
oh
for
yeah,
for
open
source,
of
course,
I
mean
first
of
all,
I
think
it's
important
that
the
people
try
to
run
the
software
like
locally
or
somehow.
Just
try
to
well,
let's
say
interact
was
the
was
the
program
so
like
try
to
compile,
try
to
get
the
whole
thing
running,
that's
the
first
step
and
then
for
the
contributions.
I
think
it's
well!