►
Description
You have seen the impact of observability in enhancing the understanding and pace of development of complex systems. With the latest innovation in managing logs through the OpenTelemetry project, we can now combine together multiple blockchain systems to report fine-grained information into a comprehensive consortium view.
In this talk, we will show you a reference architecture to integrate best of breed technologies: Kubernetes, OpenTelemetry Collector and Hyperledger Fabric.
We will show how information such as traces, metrics and logs is captured and aggregated at the level of each member of the consortium. We will then show how this information can be shared to form a consortium view between multiple industry participants.
Presenters:
Antoine Toulme, Engineering Manager @Splunk
Dave McAllister Sr, Technical Evangelist @Splunk
A
All
right
we're
gonna,
go
to
go
ahead
and
get
started
now.
So,
first
of
all,
I'd
like
to
thank
everyone
who
is
joining
us
and
welcomes
today
our
cnc
webinar,
implementing
culinary
release
on
kubernetes,
with
spinnaker
sto
and
from
the
fields.
So
my
name
is
danielle,
I'm
working
for
redhead
as
a
principal
technical
marketing,
major
and
as
well
as
I'm
responsible,
cncp
investor.
So
I
will
be
moderating
today's
webinar
and
we
would
like
to
welcome
our
president.
Today,
antoine
tulmay
engineering
manager,
splunk
and
david
mcclass,
mcallister
senior
technical,
evangelist
s
blog.
A
There
are
a
few
a
couple
of
things,
a
housekeeping
item
today
so
before
we
get
started
now.
So
during
the
webinar
you,
you
are
not
able
to
talk
as
attendee,
but
there
is
a
clinton
box
at
the
bottom
of
your
screen.
So
please
feel
free
to
drop
your
question
in
there
and
we
will
get
to
as
many
as
we
can
at
the
end.
So
this
is
the
official
webinar
and
cncf.
A
A
B
B
So
a
little
bit
of
a
round
of
introductions
for
us,
my
name
is
antoine
tulumi,
I'm
an
engineering
manager
at
splunk
like
daniel
mentioned,
I'm
part
of
the
distributed
ledger
technology
team
at
splunk.
We
work
on
all
aspects
of
blockchain
industry,
ledgers,
previously
city
at
whiteblock,
working
on
blockchain
protocols
at
consensus
and
before
that
iot
at
acer
and
I'm
joined
today
by
with
dave,
go
ahead.
C
Thanks
antoine
hi,
I'm
dave
mcallister,
I'm
the
senior
technical
evangelist
for
splunk.
I
focus
on
observability
monitoring,
platforms
and
and
practices,
especially
around
application
performance
monitoring.
Here
I've
worked
for
a
number
of
different
vendors,
scaler
and
solace,
but
I
worked
for
you
stanford
university
for
red
hat
as
well,
hi
daniel
as
well
as
adobe,
among
others.
For
this
I
started
with
open
source
back
in
linux,
0.93
time
frame,
so
it's
very
near
and
dear
to
my
heart.
Next
slide,
please.
C
C
So
observability
is
obviously
the
hot
buzz
word
these
days,
but
it
can
have
lots
of
meanings
inside
of
here.
It
actually
comes
from
control
theory
and
the
control
theory
basically
is
defining
or
designing
the
exposure
of
state
variables
within
a
manner
to
allow
the
inference
of
what's
happening
inside
of
the
activity.
You
can
think
of
this.
C
In
some
ways
like
your
car,
your
car
will
show
you
that
it's
got
a
problem,
but
inside
of
there
there's
a
number
of
different
variables
that
are
being
provided
that
let
you
get
more
understanding
about
what
happened
at
a
particular
moment
inside
of
the
car
when
we
come
to
software
or
services
platforms
or
products
in
particular.
We're
actually
looking
at
this
as
a
quality
of
the
software
that
lets
us
understand
how
the
systems
are
behaving,
and
this
is
a
big
difference.
C
That's
changed
over
the
last
few
rounds
of
software
design
going
from
the
old
black
box
monitoring
style,
where
we're
looking
at
nagios
and
asking
the
box
whether
it's
up
or
down
to
where
we're
actually
looking
at
white
box
style,
where
we're
now
getting
information
specific
to
and
related
to
the
activities
of
choice
inside
of
each
of
these
pieces
slide.
Please.
C
So
with
that,
you
probably
have
heard
these
things
called
the
three
primary
pillars
of
observability
and
you
can
think
of
these
really
as
classes
of
how
we
bring
data
in
from
our
applications
and
from
our
infrastructure
to
be
able
to
answer
those
unknown
questions.
Observability
is
different
from
the
old
style
of
monitoring,
because,
with
the
old,
solid
monitoring,
we
knew
what
questions
we
wanted
to
have
answered.
Now
we
in
our
complex
world,
where
we
have
micro
services,
where
we
have
service
messages
where
we
are
in
public
clouds,
private
clouds,
hybrid
clouds.
C
We
now
have
a
huge
amount
of
information
and
we
may
find
issues
that
we
never
even
thought
could
occur.
This
is
not
unusual
to
come
into
here
and
what
a
reservability
by
bringing
in
all
of
this
data
into
a
common
file
format
a
common
way
of
looking
at
this.
We
can
now
start
looking
into
answering
questions
for
things
that
we
didn't
expect
could
ever
have
happened.
C
C
A
much
maligned
microsoft
study
says
that
human
today
have
eight
seconds
of
attention
span,
and
a
google
study
shows
that
if
you
wait
for
a
web
page
to
refresh
for
3.7
seconds,
you're
likely
to
leave
observability
not
only
shows
us
our
problems,
but
also
can
show
us
our
performance.
Slow
is
the
new
downtime
next
slide.
Please.
C
If
you
had
to
wait
five
minutes
to
see
the
results
of
this
think
about
how
many
lambdas
could
have
started
in
that
single
time
frame
to
to
bring
in
this,
bringing
in
all
the
data
bringing
in
the
data
in
an
open
format
lets
you
then
pick
the
right
tool
for
the
back
end,
so
that
you
can
actually
answer
those
questions
that
you're
going
to
be
faced
with
next
slide.
Please.
C
Metrics
is
actually
really
close
to
being
there
as
well,
but
metrics
is
a
little
bit
farther
behind
here,
a
new
one
that
we've
added
to
this
with
the
approval
of
the
logging
specification
is
logs
now.
The
next
thing
inside
of
this
from
a
dimensional
viewpoint,
is
we
need
to
look
at
how
we
instrument
how
we
bring
in
the
apis,
and
each
of
these
pieces
is
language
specific.
What
you
build
for
go
may
not
work
for
for
java
or
rust,
and
therefore
we're
also
at
the
same
point
in
time,
building
out
across
this
entire
model.
C
Here
you
need
canonical
implementations
to
be
able
to
work
with
those
pieces.
You
need
a
data
infrastructure
that
meets
the
needs
of
those
unique
data
categories,
but
at
the
same
time
provides
a
unified
viewpoint
here
and
then
you
wanted
to
interoperate
with
other
things.
This
is
not
where
we're
trying
to
rip
and
replace
what
people
have
already
done
distributed.
Tracing
has
been
very
successful
and
around
for
quite
a
while
application.
C
Let's
go
slide
please,
and
so
I
want
to
spend
just
a
couple
of
slides
talking
about
the
new
entry
for
this,
and
this
is
the
logs
and
the
logs
are
incubation,
but
the
log
data
specification.
The
model
specification
is
now
approved
and
you
can
go
to
check
it
out
on
the
github
open
telemetry
inside
outside
of
the
open,
telemetry
enhancement
proposals
the
oteps.
So
you
can
see
what's
going
on
here.
C
C
C
This
is
an
interesting
challenge.
Inside
of
here.
There
are
lots
of
data
formats
in
here.
It's
also
designed
to
map
between
existing
formats.
So
if
you
have
a
log
format
coming
into
log
a
you
map
it
into
to
open
telemetry
logs,
and
then
you
map
from
open
telemetry
logs
into
log
b,
it
should
be
no
worse
than
you
writing
code
to
map
between
log,
a
and
log
b
directly
and
so
again
retain
the
data
information.
That's
really
important
here
this.
C
There
are
three
types
of
logs
and
bins
that
have
been
identified:
the
first
one
system
locks-
these
are
logs
and
events
that
are
generated
by
the
operating
system
and
of
which
we
have
no
control.
For
here
we
can't
change
the
format.
We
can't
affect
the
information.
That's
really
included
here.
You
can
think
syslock
as
an
example
of
this,
so
this
log
has
been
around
and
yet
it
does
change.
C
C
Is
third
party
applications
and
third-party
applications
obviously
are
generated
by
third-party
applications
here,
and
we
may
have
certain
levels
of
control.
We
may
be
able
to
customize
some
of
the
output
that's
inside
of
here,
but
in
general
we
don't
have
the
ability
to
make
changes.
We
don't
have
the
ability
necessarily
to
include
new
information
and
we
don't
usually
have
the
ability
to
change
the
log
aspects
inside
of
here
and
then
finally,
first
party
applications.
These
are
applications
that
we
write
we
develop
these.
C
We
have
control
over
how
the
logs
and
events
are
generated
and
what
information
we
include
in
the
logs.
Having
said
that,
let
me
caveat
that
we
have
some
control
over
the
logs,
because
if
you've
got
a
long
running
log
files
or
long
running
applications,
you
may
not
want
to
arbitrarily
go
in
and
change
your
log
information,
because
then
you
won't
be
able
to
compare
to
historical
data
inside
of
here.
C
C
Now,
inside
of
this
logging
information,
we
also
identify
two
kinds
field
kinds
inside
of
here.
The
first
one
is
top
name
fields
here
you
can
think
of
the
first
one,
the
top
level
fields
as
things
that
are
mandatory
or
frequently
present
in
every
well-known
format.
Think
of
this,
for
instance,
timestamp.
If
you're
writing
log
files,
you're
writing
a
timestamp.
Now
the
timestamp
has
specific
sets
of
information.
We
know
what
that
information
is
going
to
be.
C
We
know
how
it
is
represented
and
therefore
it's
a
top
level
feel
this
gives
us
the
ability
to
efficiently
represent
those
views.
We
don't
have
to
recreate
or
parse
them
on
the
fly.
We
already
have
that
information
in
place
inside
of
here
and
some
of
the
fields,
as
you
can
see
here.
Inside
of
this,
we
are
talking
about
about
open
telemetry,
so
we
have
traces
and
spans
trace,
ids
and
span.
Ids
are
going
to
be
involved
in
every
one
of
these
things.
C
We
also
are
working
with
the
w3c
side,
so
there'll
be
a
w3c
trace
span
inside
of
here
as
well,
and
you
can
continue
on
down.
The
second
side
is
the
key
value
pairs
and
here
we're
getting
into
less
frequent
data
or
data
that
you
may
want
to
uniquely
represent
inside
of
here.
It
may
not
be
standardized,
so
this
could
easily
fall
into
that
first
party
application
space,
where
you're
writing
off
data
that
is
unique
to
your
application
or
unique
to
your
needs
and
it's
stored
in
key
data
pairs.
C
C
C
You
know
if
you're,
if
you're
looking
at
this
from
the
model,
somebody
who's
using
your
system
in
a
request-based
system
cares
about
their
specific
request,
how
long
it
took
and
whether
it
was
successful-
and
you
can
think
of
this
in
in
literally
in
the
set
of
metrics
traces
and
logs
as
well.
So
you
start
with
a
high
level
metric.
What's
going
on
in
my
system?
Is
my
system?
Happy?
Is
my
system
doing
the
things
I
expected
to
be?
Are
my
numbers
right?
C
I
have
spent
a
lot
of
time
working
with
radio
duration,
red
monitoring
patterns
for
this,
and
it's
a
quick
way
of
looking
at
this
and
making
sense.
The
next
thing
you
want
to
be
able
to
do
is
drill
down
and
detangle
those.
So
now
you
want
to
look
and
say:
oh
something
doesn't
look
right.
What
doesn't
look
right?
B
Thank
you,
dave
great
presentation
and
love
to
understand
a
bit
more
about
open,
telemetry
and
we're
going
to
show
in
our
demo
how
we
can
shore
up
those
concepts,
so
we're
going
to
talk
a
little
bit
about
multi-part
computation,
first
to
kind
of
understand
how
this
applies
well
to
this
complex
field.
So
in
our
world
multi-party
competition.
Here,
what
we're
going
to
talk
about
enables
multiple
actors,
different
companies
departments,
even
silos
inside
the
same
companies
to
execute
code
without
revealing
their
input
data.
B
The
applications
of
these
technologies
are
far-reaching.
A
simple
example
is
to
imagine,
creating
and
applying
a
machine
learning
model
to
determine
if
you
will
get
cancer
by
analyzing
millions
of
individuals
dna
without
ever
revealing.
Your
dna
multiparty
computation
has
been
a
topic
of
extensive
research
for
decades
and
has
recently
experienced
tremendous
progress.
B
B
We
start
with
a
fax
machine
and
then
legacy
systems
have
been
relying
forever
on
edi,
for
example,
to
pass
on
pos
and
invoices
over
time
systems
have
evolved
to
embrace
rpc,
so
they
could
send
and
acknowledge
orders
now
we're
seeing
the
use
of
web
services
which
create
a
tapestry
between
companies
that
expose
apis
to
each
other
with
blockchain.
We
can
further
enable
trustless,
multipatch
computation.
B
So
there
are
some
challenges
that
come
with
that
multi-party
computations
in
invite
blind
spots
whenever
data
or
computation
is
handed
off
distributed,
systems
have
complexity
as
they
introduce
a
synchronicity
and
retries.
It
is
difficult
to
capture
the
concrete
life
cycle
of
data
across
the
actors
put
another
way.
Those
systems
are
not
observable.
B
B
B
Well
now,
let's
see
what
happens
right
when
we
expose
the
systems
to
observability
and
going
back
to
what
they
said.
These
are
all
the
elements
of
observability.
We
are
aware
of
metrics
traces
and
logs
we're
able
to
understand
in
detail
how
the
synchronous
nature
plays
a
role
and
how
it
allows
invisibility
of
events,
meaning
one
transaction
into
the
system
will
issue
several
events
that
we
can
witness
and
cascade
into
multiple
business
events.
B
That's
the
strengths
of
such
distributed,
blockchain
oriented
systems,
the
challenges
of
monitoring
and
understanding
the
origin
of
an
event
are
huge
building
on
observability
and
our
private
slides.
We
cannot
interpret
metrics
traces
and
logs
and
marries
them
with
ledger
data
to
gain
a
complete
picture
of
the
behavior
of
the
system
and
then
ledger.
Data
itself
is
quite
complex.
You
have
block
intellectual
data,
you
have
the
events
being
stored
by
the
code
itself
as
they
execute.
B
So
now
I'm
going
to
show
you
a
little
bit
of
the
demo,
show
you
how
the
systems
operate
and
how
they
manage
safely,
to
execute
gene
code
for
this
demo,
we'll
be
showing
hyperledger
fabric
and
we'll
be
executing
some
chain
code
and
we'll
be
looking
at
transactions
coming
into
our
splunk
instance.
A
showcasing
the
transaction
dashboard
we're
also
going
to
show
how
traces
are
produced
by
the
chain
code
and
are
sent
to
a
remote
server
for
exploration
and
how
it
allows
much
faster
feedback
to
the
development
team.
B
All
this
blends
into
great
dashboards,
allowing
the
team
to
participate
in
continuous
deployment.
So,
let's
jump
in
I'm
going
to
show
you
here,
first
and
foremost,
a
little
bit
of
an
overview
of
what
is
possible
today
to
understand
how
hyperledger
fabric
itself
runs.
This
is
our
spanking
stance,
we're
running
this
against
a
simple
hyperledger
fabric
deployment
that
has
something
like
six
channels
and
two
organization.
B
Well,
what
does
that
mean?
Well,
we
have
two
different
organizations
that
do
not
trust
each
other
and
they
have
six
different
channels
which
are
private
in
which
they
can
exchange
information.
We're
seeing
here
at
a
level
a
little
bit
of
information
about
our
network,
architectures
and
channels.
We
can
also
see
all
sorts
of
information
for
coming
from
metrics
around
health
and
monitoring
to
understanding
exactly
how
the
system
behaves
if
we're
seeing
a
spike
in
cpu
or
hard
drive.
B
So
all
the
traditional
item,
metrics
that
you
are
using
today
to
monitor
your
machines,
are
still
there.
They
even
are
more
exacerbated,
as
you
run
on
the
cloud
that
you
do
not
control,
but
for
a
business
analyst
trying
to
understand
how
the
system
behaves
from
the
point
of
view
of
its
use
case,
we
like
to
concentrate
on
the
transaction
analytics
dashboard.
It
allows
us
to
understand
exactly
what's
going
on
inside
the
chain
and
what
data
is
being
transferred.
B
Last
but
not
least,
I
would
be
remiss
not
to
mention
that
we
have
full
capacity
to
search
any
data
on
the
chain,
so
we
can
solve
all
sorts
of
computation
and
we
can
always
go
through
each
and
every
one
of
the
metrics
and
dive
in
and
understand,
better,
what's
being
exchanged
so
back
to
our
mantra.
Metrics
traces
and
logs
we're
now
seeing
metrics
very
well
exposed
here
and
now
we're
going
to
understand
a
little
bit
what's
going
on
inside
the
system
at
this
time.
B
So,
as
I
click
here,
I'm
being
transported
to
a
transaction
analytics
dashboard
that
shows
us
that
we
are
seeing
a
number
of
bids
which
are
being
executed
on
our
chain
and
this
dashboard
refreshes
every
minute.
It's
showing
the
last
60
minutes
of
activity,
we're
seeing
all
sorts
of
metrics
being
exposed.
B
B
You
can
also
see
all
the
blocks
in
each
of
the
the
blocks
transactions,
so
we
can
really
understand
how
things
are
being
batched
in
and
whether
we
see
a
lot
of
load
or
not
now
to
dive
in
a
little
deeper.
Our
developers
will
want
a
different
view
into
this,
and
for
that
we're
going
to
go
over
to
a
different
view
called
signalfx.
B
It
allows
us
to
see
a
real-time
view
and
see
in
the
last
10
seconds
a
resolution
of
what
is
being
exchanged
on
the
network.
We
can
even
see
the
different
versions
of
our
chain
code
as
we
execute,
and
this
has
been
built
by
a
developer
strangely
for
their
own
use.
They
can
create,
however,
how
many
dashboards
they
want
using
this
type
of
metrics.
So
this
is,
if
you
introduce
data
that
is,
live,
allowing
you
to
really
quickly
react.
B
We're
also
able
using
traces
to
understand
how
those
systems
transact
with
each
other,
and
I
can
see
that
we
have
a
little
bit
of
a
of
a
map
here,
where
we
can
see
that
there
is
an
auction
app
which
is
talking
to
a
b
contract.
So
we're
now
able
to
correlate
together
different
services
and
make
sense
of
a
blockchain
in
the
realm
of
the
rest
of
the
system
and
not
just
by
itself.
B
The
blockchain
itself
is
executing
the
chaincode
twice
because,
as
you
remember,
we
have
two
organizations
which
are
sharing
this
code
base,
so
both
of
them
do
not
trust
each
other
they're
going
to
want
to
run
the
contract
on
their
own
environment.
So
they
can
make
sure
this
is
working
in
this
case.
They
both
trust
the
monetary
environment.
So
they
both
report
back
the
traces.
So
we
can
understand
exactly
what's
going
on
now,
I'm
going
to
make
this
a
bit
more
concrete
for
the
developers
in
the
room.
B
Here
is
the
actual
chain
code
that
we
execute,
and
this
is
very
straightforward-
we're
creating
as
part
of
this
chain
code,
a
number
of
spans,
and
these
are
the
spans.
The
traces
that
dave
was
mentioning
here-
we're
recreating
the
parent
span
context
from
parameters
being
passed
into
our
chain
code.
Here
we
pass
the
trace
id
and
the
span
id,
and
we
will
create
this.
B
B
So
I've
was
showing
you
this
also
for
a
reason,
as
we
collect
the
value
we
now
have
that
available
to
us.
So
just
like
they
mentioned
all
those
key
value
pairs
become
available
to
us.
So
we
can
now
do
things
by
understanding
the
value
of
being
exposed,
and
I
can
now
show
you
the
average
value
of
a
bid
of
a
time
for
each
of
the
options
right
and
this
being
splunk.
I
can
change
that
can
say
I'd
like
to
get
the
max.
Please.
B
B
Now,
going
back
to
our
slide,
you've
seen
operator
working
with
a
hyperledger
fabric
system,
the
operator
is
able
to
quickly
understand
the
data
exposed
by
the
system
in
react
situations
through
observability,
with
the
help
of
metrics
traces
and
logs.
The
vision
of
an
operator
is
truncated,
though,
to
what
they
can
see
as
part
of
their
environment
right.
They
cannot
see
further
than
them.
So
I'm
going
to
talk
to
you
about
a
little
bit
of
a
daring
concept
in
a
second
we're
going
to
talk
about
constructions
and
to
resolve
the
issues
of
largely
distributed
systems.
B
Consortiums
are
needed
to
work
across
organizations
in
silos,
and
here
is
atypical
type
of
different
course
solutions.
Now
there
can
be
associations
of
companies,
so
it
can
be
a
students
of
departments
inside
the
same
companies
in
our
typical
experience.
They
form
to
allow
greater
automation
and
collaboration
across
shared
critical
processes,
they're
highly
valuable
right.
It's
not
just
small.
No,
these
are
the
the
heart.
The
blood
flow
of
this
company
depends
on
these
type
of
processes.
B
Those
consortiums
are
massive
bets
made
by
innovative
businesses
that
discovered
that
melding,
their
businesses
with
their
partners
had
immense
ramifications
for
the
bottom
line,
and
each
of
the
members
of
the
consortium
have
access
to
their
own
view
of
their
data,
their
transactions,
their
past
history,
their
private
data,
everything
is
preserved.
However,
consortium
members
need
to
collaborate
to
operate
the
ensemble
of
the
system,
and
this
is
how
I'm
I'm
going
to
dive
into
my
new
demo,
we're
going
to
show
you
a
rendering
of
a
dashboard
showcasing
a
blockchain
deployment
across
multiple
cloud
providers.
B
Where
you
can
see.
Each
participants
have
developed
a
complete
deployment
of
the
system
that
they
then
themselves
host,
but
it's
not
enough.
The
operators
of
the
consortium,
this
congregation
of
blockchain
members,
are
mandated
to
ensure
that
consortium
can
operate
and
evolve
over
time.
You
need
to
be
able
to
add,
remove
members,
or
you
need
to
be
able
to
amend
the
logic
of
the
code
running
as
part
of
the
consortium.
B
They
really
help
us
to
pinpoint
the
different
members
and
we
can
see
that
our
infrastructure
health
check
does
not
seem
to
be
completely
okay,
so
we
can
go
as
an
operator
in
drill
down
understand
exactly
where
the
data
is
coming
from
how
the
insight
has
been
captured.
We
can
understand
how
that
that
is
coming
together.
B
Observability
has
changed
completely
how
we
operate
our
businesses
and
data
once
again
moves
well
now.
Let's
talk
about
this
from
a
technical
perspective,
it's
a
breeding
ground
for
insightful
beds.
It
allows
us
to
understand
the
second
order
effects
of
our
decisions
in
microservices
deployments.
In
distributed
systems
in
vast
complex
systems,
it
has
become
imperative
to
understand
how
the
systems
as
a
whole
performs
from
the
level
of
the
machines
the
system
operates
on
to
the
code
logic
for
businesses.
It
significantly
reduces
mistakes
and
fraud,
while
increasing
speed
and
improved
auditing.
B
It's
a
win
for
everyone
except
the
bad
actors.
We
know
the
complexity
of
data
now
resides
on
the
edge.
Businesses
have
done
a
good
job,
understanding
the
cooperations
and
reporting
on
them.
We
are
now
facing
challenges
through
the
operation
of
innovative
consortiums
that
allow
businesses
to
automate
the
collaboration.
B
So
please
join
us
and
learn
more
like
if
you
do
one
thing
today:
click
on
open,
telemetry.io
and
go
check
us
out.
This
is
a
great
project
and,
as
dave
mentioned,
it's
a
very
active
project,
lots
of
stuff
to
keep
up
with
a
lot
of
collaboration
between
different
companies.
If
you
were
intrigued
at
all
by
hyperledger
fabric
and
what's
possible
right
now
is
blockchain
check
out
hyperledger.org.
B
I
would
be
remiss
if
you
didn't
mention
this.
Last
webinar
was
made
by
steve
flanders
from
splunk
as
well.
I
would
definitely
recommend
that
you
watch
it
if
you
want
to
learn
a
bit
more
about
the
ignorance
about
how
open
telemetry
is
getting
the
world
and
if
you're
a
developer
like
me
or
I
try,
I
try
to
pass
as
one
check
out
some
cool
code
under
the
open,
telemetry
collector
releases.
I
think
version.5
is
out.
B
A
A
A
A
C
To
add
to
that
just
real
briefly,
there
are
a
number
of
companies.
A
lot
of
this
work
around
observability
started
with
the
e-commerce
sites
and
and
heavily
into
social
media
activity
sites.
Many
of
them
use
the
other
display
boards
grafana
cabana
those
various
pieces
to
display
information
coming
from
multiple
locations.
C
What
open
telemetry
brings
in
play-
and
we
do
know
the
ones.
Unfortunately,
I
can't
mention
names-
we
do
know
that
there
are
ones
that
have
already
moved
over
to
open
telemetry
as
their
data
acquisition
model
to
do
this,
but
it's
still
being
able
to
roll
that
data
out
to
a
group
hana
type
of
dashboard.
B
It
is
open
source
and
it's
all
made
of
standard
components,
so
a
hyperledger
fabric
exposes
most
of
its
metrics
of
a
prometheus
or
statsd.
For
this
particular
example
we're
using
prometheus
and
we're
using
the
signalfx
agent
to
collect
this
data,
we
could
use
also
the
open,
telemetry
collector.
That's
now
to
a
point
where
we
can
easily
collect
those
metrics
and
some
of
the
some
of
the
blog
data.
B
All
the
later
data
is
leveraged
and
instrumented
by
our
team's
work
through
the
fabric
logger,
which
is
an
open
source
apache
licensed
tool
which
is
specific
to
hyper
ledger
fabric
data
gets
all
the
protobuf
using
a
standard
peer
connection,
so
yeah
all
this
is
coming
together
and
you
can
connect
that
data
to
any
source.
You
want
at
this
point.
A
C
So
so
the
short
answer
is
no,
in
fact,
most
of
the
the
modeling
for
all
of
these
things
is
to
bring
in
already
existing
solutions.
This
is
this
is
not
a
rip
and
replace
structure.
So,
for
instance,
again
I
have,
I
have
a
we
have
a
customer
who
literally
brings
in
fluent
d
into
the
instrumentation,
combines
it
into
the
tracing
data.
C
At
the
same
point
in
time
there
are
some
advantages:
fluentd
doesn't
have
quite
the
same
data
granularity,
that's
possible
through
open
telemetry,
but
it's
not
a
replace
it's
not
trying
to
do
away
with
fluency.
We
actually
use
fluency
internal
internally
as
well.
We
track
things
with
fluency
ourselves,
so
again
not
replacement,
but
it
is
an
expansion.
Let's
put
it
that
way,.
A
B
Oh,
that's
a
great
question.
Thank
you
for
asking
so
we're
in
early
days
we're
trying
to
first
and
foremost,
prove.
This
is
the
right
approach
and
we
have
a
lot
of
time
on
our
hands
to
make
this
right
because
we're
first
and
foremost
applying
ourselves
to
success.
So
people
want
to
to
try
this
approach,
the
hyperledger
foundation,
so
I'm
a
committer
on
some
of
the
projects.
B
I
believe
that
we
could
very
selectively
add
tracing
in
some
of
the
elements
in
terms
of
metrics,
however,
to
slam
dunk,
because
all
the
information
is
already
available
to
use
for
performance
metrics.
So
it's
just
a
matter
of
making
that
more
apparent
in
leveraging
those
metrics
forward
right.
We've
had
just
one
contact
with
the
hyperledger
foundation
about
making
that
maybe
more
part
of
the
framework,
and
I
think
this
is
going
to
accelerate
in
the
coming
months
as
our
maturation
and
our
use
cases
get
crisper.
A
Okay,
I
think
so
we
think
it's
almost
done.
Okay,
thanks
antoine
and
dave
for
a
great
presentation
and
a
fantastic
demo
and
queen
a
all
answers
all
right.
That's
all
the
questions
we
have
time
for
today
and
thanks
for
joining
us
today
again
and
though,
once
again,
the
webinar
recording
and
slice
that
will
be
online
later
today
and
you
just
go
to
www.cncf.ioslash
webinar,
so
you
can.