►
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2023 in Amsterdam, The Netherlands from April 17-21. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Why Large-Scale Observability Needs Graph - Richard Benwell, SquaredUp
How do we make use of the increasing volume of observability data that we collect? Observability was inspired by control theory, but current observability solutions are missing a key element of that theory: the system model. We can’t understand the state of the system, or ‘answer unknown unknowns’ if we don’t know how the system works. We’re drowning in data but starved of answers. Graphs (think social network graph, not line graph) allow to understand how things are connected, and a new trend of graph databases including neo4j, AWS Neptune and Azure Cosmos have taken graph technology mainstream. This talk looks at early signs of how the observability community are beginning to leverage graph concepts – including Jaeger service graph, Backstage service catalog and eBay's Groot (root cause analysis) tool – and some of the opportunities and challenges around how observability data can reveal more answers with graph.
A
All
right,
a
few
people,
drifting
back
from
break
that
I'll
I'll
get
going.
My
name
is
Richard
benwell,
founder
and
CEO
of
squared
up.
You
may
not
have
heard
of
us
before,
but
we
are
a
data.
Visualization
company,
that's
been
working
in
the
Enterprise
I.T
space
for
the
last
10
years
and
just
this
month
released
a
new
visualization
platform
that
correlates
data
across
your
observability
tools,
your
development
tools
and
your
business
tools.
A
It's
great
to
be
here
and
today
my
talk
is
on
why
large-scale
observability
needs
graph
and
by
large
scale
I
mean
not
necessarily
the
volume
of
data,
but
the
complex
messy
kind
of
systems
that
we're
trying
to
observe,
as
we
scale
up
our
organizations
and
by
graph
I
mean
not
the
line
graph
type
of
graph,
but
the
graph
that
is
the
connected
things
type
of
graph.
So
in
the
talk
today,
I
hope
to
cover
why
I
think
large-scale
observability
needs
graph
I'll
cover.
A
What
sort
of
graph
is
how
it's
used
today
and
maybe
some
challenges
for
adoption
going
forward.
When
we
talk
about
observability,
we
talk
about
metrics,
logs
and
traces
and
I.
Think
bartek
was
saying:
there's
maybe
more
than
those
three
pillars:
maybe
there's
six
pillars,
maybe
maybe
there's
a
maybe
there's
a
lot
of
pillars,
but
I'll
use
metrics,
logs
and
traces.
A
The
other
challenge,
though,
is
how
we
use
that
data
collecting
that
data
doesn't
necessarily
actually
offer
any
value
to
us
just
sitting
in
a
database.
It's
only
valuable
if
we
can
actually
use
that
to
answer
questions.
So
how
do
we
find
and
use
the
right
data
to
answer
questions
like
where's
the
problem
for
this
incident?
How
reliable
are
my
components:
where's,
the
weak
spot,
how's
my
application
going
to
scale
with
its
performance
and,
if
I
make
a
change
to
one
component,
how
it's
going
to
impact
other
components?
A
How
do
I
plan
change-
and
these
are
a
very
different
kind
of
Challenge
and
I-
think
the
latter
challenge
using
the
data
is
still
an
unsolved
challenge,
we're
still
finding
that
our
data
is
massively
underused
and
we're
still
finding
it
hard
to
answer,
even
maybe
sometimes
some
straightforward
questions
and
so
I
think
this
is
because
we've
been
missing
something
when
we've
been
talking
about
observability
and
what
it
needs
and
to
explain
this
I'm
going
to
bring
up
I.
Think,
probably
the
Wikipedia
page
that
everyone
here
has
read.
A
It's
you
know
it's
where
there's
sort
of
inspiration
for
observability
came
from
was
from
the
control
systems
domain,
and
so
you
know
you
go
Google,
observability
and
you'll,
find
this
Wikipedia
page
and
up
right
at
the
front
at
the
top.
Is
the
definition
of
observability
that
we're
all
very
familiar
with
observability
is
a
measure
of
how
well
internal
states
of
a
system
can
be
inferred
from
knowledge
of
its
external
outputs.
So
how
do
we
understand
what's
happening,
given
what
we
are
observing
from
the
external
outputs?
So
that's
great:
that's
the
bit!
A
We've
taken
from
this
Wikipedia
page
and
we've
put
on
every
single
blog
about
what
is
observability.
It's
where
every
single
observability
Talk
starts
with
that
definition
and
if
you
have
been
to
this
page,
you've
probably
done
like
me.
First
time,
I
got
here
which
was
you,
you
read
that
and
you
think
that's
pretty
cool.
A
You
maybe
read
some
of
the
rest
of
the
paragraph
and
then
you
start
to
scroll
down
and
your
eyes
glaze
over
and
your
your
scroll
finger
sort
of
kicks
into
action,
and
then
you
just
scroll
down
the
bottom
of
the
page
and
move
on,
but
I
think
there's
more
to
be
inspired
from
from
the
control
systems.
Observability
and
that's
further
down
the
page.
So
so
the
first
first
instant
we
have
is
collect,
collect
the
data,
but
actually
we
scroll
down
the
page.
A
We
hit
some
of
these
equations
and
again
most
of
us
kind
of
glaze
over
and
move
on
and
I'm
not
going
to
talk
in
depth
about
equations.
This
isn't
going
to
be
a
maths
session,
but
I
did
want
to
just
talk
about
what
those
equations
actually
mean
and
what
we
can
learn
from
them.
So
as
we
scroll
down
the
page
from
control
theory
and
observability,
it
talks
about
whether
a
system
is
observable,
and
these
two
equations
broadly
mean
the
first
one.
A
Is
the
the
x
dot,
so
kind
of
the
new
state
of
the
system
is
a
function
of
you
know
the
previous
state
of
the
system,
plus
plus
the
the
inputs,
so
you've
got
x,
dot
equals
ax
plus
b.
U
so
U
is
the
inputs
and
X
is
the
current
state
of
the
system.
So
that's
the
first
equation.
How
does
the
system
change
over
time
with
inputs?
And
the
second
equation
is
what
our
outputs
look
like?
Our
outputs.
A
Our
signals
are
some
function
of
the
current
state
of
the
system,
plus
the
the
inputs
that
going
into
it.
That's
what
control
theory
talks
about.
It
has
has
the
idea
of
the
current
state
inputs
and
and
those
outputs,
and
then
it
talks
about
this
observability
Matrix.
So
when
in
control
theory
they
say
is
a
system
observable.
You
need
the
output
sure
you
need
the
signals,
but
you
need
something
else.
You
need
those
those
matrices
that
Define
how
the
system
works
right.
A
How
does
system
change
over
time
how
the
output's
a
function
of
the
state
of
the
system
so
in
control
theory
they
have
these
matrices
that
Define
the
model
of
the
system,
and
that
raises
the
question
where's
our
system
model,
so
we're
talking
about
collecting
the
data.
We
have
the
the
signals,
the
outputs
but
where's
our
system
model.
How
do
we
work
out
the
internal
state
of
the
system
so
just
putting
this
sort
of
into
context
if
we
take
a
control
system
observability
case
this
is
like
a
textbook
case.
A
What
is
the
internal
state
of
the
system?
I?
What's
the
voltage
across
this
capacitor,
and
we
know
these
these
signals,
we've
got
a
voltage
across
the
inductor
there
and
an
input
voltage.
So
what's
what's
the
voltage
of
C?
Now
it's
pretty
obvious.
We
have
absolutely
no
idea
because
we
don't
know
what
circuit
it's
talking
about.
So
in
a
control
systems
case
you
have
a
circuit
diagram.
That's
then
modeled
using
those
matrices
and
you
can
find
out
whether
we
can
in
fact
understand
what
the
voltage
across
that
capacitor
is.
A
If
we
have,
these
output
signals
right,
and
so
you
put
those
into
that
observability
Matrix
equation
and
say:
yes,
it
is
available,
given
those
outputs,
we
can
figure
out
what's
happening
within
that
system.
So
very
obviously
we
need
a
model
to
interpret
those
voltages.
Those
voltages
are
meaningless
without
that
model.
A
A
We
can't
use
those
unless
we
have
a
model
of
the
system
right.
Just
we
if
we
had
a
database
full
of
metrics
logs
and
traces,
we
wouldn't
know
what
we're
looking
at
and
often
that
is
the
case
in
in
organizations
that
the
data's
collected,
but
not
many
people,
actually
know
how
to
interpret
that
data.
So,
of
course
we
need
a
model
as
well
right.
We
have
a
a
model,
maybe
of
our
microservice
architecture
or
model
of
our
infrastructure
and
our
clusters,
so
we
absolutely
need
a
model
to
interpret
the
observability
data.
A
The
signals
that
we
have
so
we
clearly
do
have
some
sort
of
model.
Well,
just
just
as
a
recap
there,
sorry,
you
know
in
terms
of
sort
of
asking
the
question:
is
our
system
observable
the
proposition
here?
Is
that
actually
we
need
the
signals
plus
the
model
right.
We
can't
just
say
our
system's
observable
because
we're
collecting
the
data.
We
need
to
put
that
data
into
context
with
the
model.
A
So
clearly
we
do
have
a
model
of
some
sort,
because
we
do
use
this
data,
it's
not
absolutely
useless
to
us.
So
where
is
the
model
typically
well?
The
model
typically
is
in
someone's
head
or
in
our
in
our
heads
in
terms
of
informal
knowledge,
maybe
it's
written
down
in
a
Confluence
page,
that's
out
of
date
or
an
out-of-date
diagram,
but
it's
probably
just
informal
knowledge
shared
within
a
team
and
across
teams.
A
A
You're
you're,
a
system
of
systems
where's
the
model
of
the
system
of
systems
who,
in
their
head,
understands
how
it
all
fits
together
and
that's
when
you
start
to
see
the
pattern
of
something
like
an
architect
whose
kind
of
job
it
is
to
work
out
how
everything
fits
together.
Everything
comes
to
them,
or
maybe
the
Unlucky
SRE
who's
like
really
trying
to
figure
out
how
everything
fits
together,
and
so
all
questions
come
to
them.
A
So
that's
scales
so
far
right,
you,
you
get
maybe
a
few
people
who
have
this
knowledge
and
understand
the
model,
and
so
nowhere
to
to
look
and
interpret
that
data,
but
at
some
point
in
a
large
scale,
application
you
get
to
this
level
of
complexity.
So
this
is
the
the
infamous
AWS
Death
Star,
so
when
they
mapped
out
their
microservices,
they
ended
up
with
with
a
diagram
that
looks
something
like
this
where's.
The
knowledge
of
that.
A
How
do
we
interpret
all
of
that
data
and
put
it
into
context
when
we
have
a
system
of
that
sort
of
complexity
and,
of
course,
we're
not
all
necessarily
like
that?
We
don't
have
sort
of
massively
complex
systems
like
that.
But
if
you
have,
if
you
work
in
an
organization,
that's
been
around
for
a
few
years
and
has
gone
through
technology
platforms,
it's
gone
through
developers
gone
through
scale.
A
A
So
that's
why
I
want
to
talk
about
graph
I,
think
the
model
is
a
graph
and,
like
I
said
in
the
intro,
when
I
say
graph,
I
don't
mean
the
sort
of
line
chart
type
graph
I
mean
the
graph
of
connected
things.
These
two
diagrams
are
actually
just
from
the
Wikipedia
pages,
so
line
chart
on
the
left
and
the
social
graph
on
the
right.
So
if
you
look
up
Wikipedia
for
social
graph,
that's
what
you
see
and,
of
course,
we're
all
quite
familiar
with
that
idea
of
a
graph
The,
Social,
Network
graph.
A
You
know
interconnected
things
and
very
clearly
this
Maps
very
well
on
to
our
domain
with
our
services
that
may
be
cool
another
service,
or
maybe
even
that
service
puts
a
message
onto
a
Kafka
topic,
so
publishes
to
a
Kafka
topic,
and
maybe
another
service
subscribe
to
that
topic.
So
they
are,
they
are
relating.
They
have
those
relationships.
Maybe
that
service
is
hosted
on
a
particular
cluster.
Maybe
those
clusters
have
particular
hosts
in
them.
A
It's
all
a
lot
of
connected
data
and
graph
is
the
way
to
model
that
data,
so
before
I
go
on
I
wanted
to
do
a
couple,
quick,
slides
on
some
graph
101
just
so
we
kind
of
all
understand
what
what
graph
is.
It's
a
technology
I
think
that's
trending,
quite
a
lot
of
moments
and
we're
seeing
it
more
and
more
and
I'll
talk
about
how
we
are
starting
to
see
it
in
our
observability
platforms
today.
A
So
the
graph
concept
is,
this
idea
of
you
know
the
two
dots
and
a
line
right
the
boxes
and
lines.
So
we
have
two
nodes
with
a
link
in
in
sort
of
mathematics,
graph
terms,
they're
called
vertices
and
edges
which
I
never
get
used
to
so
I
kind
of
often
call
just
some
nodes
and
links,
and
so
we
have
something
like
service,
a
called
service,
B.
A
Okay,
so
we
can
capture
that
relationship
as
a
as
a
graph
of
two
two
nodes
there
and
then
just
so
slightly
academic,
but
just
to
demystify
some
terms.
You
hear
you
then
start
to
hear
about
something
called
a
property
graph
and
a
knowledge
graph,
and
so
property
graph
is
again
something
very
familiar.
A
It's
when
we
have
those
nodes
with
relationships
so
service,
a
call
service
B,
but
we
start
to
add
properties
to
both
the
nodes
and
potentially
the
edges
as
well.
So
we
can
add
the
label
of
that
service.
It's
it's
called
service,
a
it's
owned
by
team,
Team
Alpha
it's
in
this
particular
repo,
and
we
can
put
properties
on
the
the
links
if
we
want
to
as
well.
A
So
we
can
mix
properties
in
their
gears
and
just
sort
of
kind
of
easily
express
what
we
want
to
express
and
then
just
to
as
I
say,
demystify
a
term
if
you've
heard
it
there's.
Also
this
term
called
Knowledge
Graph,
that's
being
used
more
and
more,
and
you
may
have
heard
about
it
from
Google's
Knowledge
Graph.
So
now,
when
you
Google
something,
instead
of
just
getting
web
pages
as
results,
you
get
that
sidebar
that
knowledge
panel.
A
That
actually
knows
what
you're
interested
in
and
starts
to
give
you
facts
about
what
you're
interested
in
it
could
be
a
you
know,
a
person,
a
famous
famous
person,
or
it
could
be
the
the
results
of
the
the
football
match
last
night,
so
Google
used
something
called
a
Knowledge
Graph
and
it's
it's
slightly
academic,
but
it
the
idea.
Really.
A
Is
you
sort
of
blow
apart
that
graph
to
mean
that
everything
is
expressed
as
this
subject
predicate
object,
so,
instead
of
having
properties
on
a
on
a
node,
you
say
well
that
node
has
repo
and
the
the
identity
of
the
repo
is
a
separate
node
and
it's
owned
by
a
team,
and
that
team
is
a
separate
node
and
it
also
calls
you
know
another
service
and
that
service
is
a
separate
node.
It's
a
little
bit
like
how
we
normalize
a
relation,
a
relational
database.
A
You
blow
everything
apart
and
put
everything
in
its
own
space
and
So
within
this
sort
of
idea
of
Knowledge
Graph
everything
has
a
URI
a
unique
resource
indicator
and
those
identifiers
can
be
then
shared
across
different
domains
and
that's
how
you
start
to
link
data
together.
So
when
Google
build
their
Knowledge
Graph,
they
ingest
all
this
data
from
different
places,
but
they
correlate
it
all
using
these
these
Uris,
this
sort
of
is
very
exciting.
This
idea
of
like,
could
you
express
every
fact
in
the
world
as
a
graph?
That's
pretty
cool.
A
What
could
you
do
with
it?
If
you
could,
unfortunately,
though,
it's
typically
very
academic,
has
the
feeling
of
XML
web
services
and
and
sort
of
soap
have
you
ever
ever
worked
with
those
battles
kind
of
20
years
ago,
so
it
feels
a
little
bit
too
academic,
not
very
practical,
which
is
why
the
property
graph
is
is
generally
how
graphs
are
used
today
and
just
sort
of
another
couple
of
aspects
of
graph.
A
So
if
you
come
across
those
databases
and
talk
about
graph,
that's
what
they
do
and
you
get
something
like
this
as
an
example,
query
a
very
powerful
if
you've
got
all
this
data
connected,
you
can
ask
questions
like
what
Upstream
front
ends
use
this
particular
microservice
So
within
larger
organizations.
I've
heard
this
several
times
via
a
development
team
developing
a
particular
component.
You
don't
necessarily
even
know
how
that
component
is
being
used
by
all
the
applications
within
your
organization.
A
V
is
those
vertices
and
then
you
just
say:
hey
I'm,
looking
for
a
service
that
has
the
name
service,
X
and
I'm,
going
to
iteratively
look
at
the
incoming
links
to
those
services
with
a
cause
relationship
until
I
find
a
a
node
that
has
Type
front
end
and
then
I'm
going
to
map
those
out
so
super
powerful.
Once
all
this
data
is
connected,
so
that's
graph.
How
can
graph
be
used
to
model
our
systems
that
we're
trying
to
observe
the
good
news
is?
A
Is
that
it
already
is
not
necessarily
in
the
form
of
those
databases
but
in
terms
of
Concepts
graph
is
already
being
used
within
many
observability
tools,
and
it's
kind
of
one
of
the
reasons
why
I
wanted
to
to
talk
today
about
this
topic,
because
it
is
just
a
a
concept
and
a
technology
that
is
seeing
a
sort
of
increased
adoption
and
I
think
as
a
as
a
community.
We
should
look
at
how
we
can
formalize
that
and
really
accelerate
its
its
adoption.
A
So
one
system
I
wanted
to
quickly
talk
about,
is
probably
the
most
familiar
Jaeger
when
we
perform
our
tracing.
Ostensibly
that's
to
allow
us
to
identify.
You
know,
performance
issues
with
a
particular
transaction
or
type
of
transaction,
but
typically
I.
Think
one
of
the
greater
benefits
is
when
it's
first
run.
Is
you
understand?
A
What's
talking
to
what
and
you
end
up
with
this
graph
and
go
aha
I
didn't
realize
that,
actually
you
know
this
microservice
is
actually
being
called
by
so
many
other
different
services,
so
in
Jaeger
they
have
a
graph
View,
and
this
is
the
graph
you
get,
but
it's
just
a
visualization
at
the
moment.
That's
not
stored
anywhere,
but
it
is
one
of
the
most
sort
of
useful
outputs
of
these
traces.
A
The
other
use
of
graph
that
you
may
be
less
familiar
with
is
in
a
tool
called
backstage.
There's
anyone
here
familiar
with
backstage
yeah,
just
just
maybe
a
handful,
so
backstage
actually
have
backstage
con,
just
a
few
doors
down
the
hallway.
For
the
first
time,
it's
a
Spotify
project
that
was
open
sourced
a
few
years
ago
and
is
now
in
the
incubator
phase
of
the
cncf
and
it
it
grew.
It
came
into
life
within
within
Spotify
by
some
platform
engineering
team.
A
So
those
those
definitions
are
defined
normally
in
the
code,
repo
backstage
pulls
in
the
yaml
files
for
different
code,
repos
stores
in
a
centralized
catalog
and
enables
you
to
to
map
out
your
your
entire
environment.
It's
not
specific
to
any
particular
infrastructure
technology
or
or
sort
of
deployment
style
or
application
Style.
It
is
just
a
service
catalog.
A
You
might
hear
backstage
also
called
a
development
developer
portal
I
used
to
I
think
it's
got
more
use
as
a
service
catalog
than
a
developer
portal,
but
again
the
the
guys
within
Spotify
weren't
quite
sure
what
to
call
it,
which
is
why
they
call
it
system
Zed,
but
but
it's
a
very
powerful
concept,
which
is
a
service
catalog
that
allows
you
just
to
dump
all
this
metadata
about
your
services
and
what
they
talk
to
into
in
a
single
place.
And
so
this
is
starting
to
provide
people
with
a
with
a
model
of.
A
So
you
could
actually
spot
anomalies
or
discrepancies
between
what
your
developers
were
declaring
as
their
dependencies
and
actually
what
the
dependencies
were
when
you're
looking
at
the
the
code
running
for
real,
that's
kind
of
an
interesting
use
case
right
so
so,
do
your
developers
really
know
what's
calling
them
and
and
what
applications
they're
part
of
so
graph
is
already
being
used
to
to
model
systems.
Certainly
conceptually
I
just
wanted
to
also
present
on
a
couple
of
examples
of
where
it's
being
used
to
combine.
You
know
those
signals
with
the
model.
A
I
think
backstage
is
is
very
exciting
and
you
know
so.
It's
just
just
down
the
hallway.
There
I
think
there
is
great
potential
to
combine
what
backstage
you're
doing
with
with
what
we're
seeing
here
in
the
open,
observability
community,
but
a
couple
of
examples
of
how
we
can
then
correlate
observability
data
across
services
using
tools
like
this
one
is
kiali
on
top
of
istio
service
mesh,
because
this
is
specific
to
twistio.
A
But
this
is
a
really
nice
example
where
they
use
the
Prometheus
data
to
understand
what
services
are
connecting
to
what
other
services
with
the
with
Envoy
there
just
tracing
all
in
incoming
and
out
are
going
calls.
They
can
map
create
that
map,
and
then
they
overlay
on
top
of
that
other
Prometheus
data.
So
you,
including
including
alerts,
so
you
can
see
the
data
in
the
context
of
your
your
service
map
and
the
performance
metrics
as
well
and
I.
Think
bartek
also
mentioned
about
Prometheus,
maybe
having
something
similar
as
well.
A
I
haven't
seen
that
yet,
but
I'm
looking
forward
to
seeing
that
so
starting
to
combine
the
model
with
the
observability
data,
I
think
really
produces
some
some
powerful
results
and
I
kind
of
one
very
far
end
of
the
the
scale
of
how
we
could
use
it.
I
just
wanted
to
highlight
a
project
that
was
internal
to
eBay,
something
called
Groot,
there's
white
paper
and
I
think
I
think
it
may
have
been
open
sourced,
there's
an
event
graph
based
approach
for
root
cause
analysis,
it's
a
little
small
on
the
screen
there.
A
But
this
idea
is
saying:
okay,
if
we
have
on
the
left
our
dependency
graph,
that
we've
got
from
either
sort
of
declarative
or
or
tracing,
and
then
we
overlay
on
top
of
that,
the
events
that
are
going
on
within
our
system,
then
we
can
actually
understand
causality.
We
can
understand
that
an
issue
or
an
event
that
occurred
on
some
service
has
impacted
some
Upstream
service.
A
That's
impacted
a
customer
by
correlating
events
across
across
different
tools,
so
there's
definitely
definitely
scope
for
for
using
this
in
some
very
powerful
ways
and
starting
to
to
make
make
useful
observability
data.
We're
collecting
I
wanted
just
to
finish
with
some
ideas
around
like
how
can
we
you
know?
How
can
we
use
this
this
sort
of
concept
further?
How
can
we
adopt
this,
and
how
can
we
do
more
with
our
data?
A
The
first
one
is
about
extending
the
scope
of
observability
so
as
I
say,
we're
we're
talking
a
lot
about
metrics,
logs
and
traces,
and
typically
that's
because
we're
focused
on
the
system
as
we
Define
it.
As
our
software
like.
What
is
the
software
doing?
What
is
the
code
doing
right
now,
we'll
perhaps
the
infrastructure
that
code
is
running
on,
but
really
that
software
system
is
just
running
within
a
larger
system
that
we
really
do
care
about?
A
I've
called
it
here
that
the
business
system,
the
kind
of
two
elements
of
that
one,
is
when
we're
looking
at
how
our
software
is
performing.
We
shouldn't
be
looking
at
that
in
isolation
of
what
the
business
is
actually
doing
and
cares
about
or
what
our
customers
are
experiencing.
If
we're
just
looking
at
the
performance
of
the
code,
then
we're
we're
already
blinkered
to
the
bigger
picture
so
really
as
a
system
in
terms
of
modeling
the
system,
we
should
be
thinking
more
broadly
than
just
our
code.
A
We
should
be
thinking
about
the
business
that
the
code
is
running
within
so,
for
example,
is
the
business
hitting
certain
kpis
like
conversions
or
adoption
or
Revenue
or
other.
You
know,
Customer
Events,
something
that
we
need
to
observe
as
well
like
log
on
purchases
service
tickets?
Even
that's
what
we
should
really
be
thinking
about
as
our
system
that
we're
trying
to
observe
not
just
the
code
and
on
the
other
side.
Very
importantly,
as
we
all
know,
things
break
often
because
of
changes.
A
So
when
we're
talking
about
the
system
that
we're
trying
to
observe,
we
really
need
to
include
in
that
the
development
teams
like
pipeline
deploys
code,
changes
the
issues
that
they're
they're,
tracking
and
fixing
and,
of
course,
our
third-party
dependencies,
our
external
dependencies.
The
cloud
providers
that
we're
running
on
some
external
API
that
we're
that
we're
depending
on
and
we
need
to
understand
the
status
of
that
potentially
even
cost
as
well.
That's
another
another
aspect
of
observability.
So
when
we're
thinking
about
observability
in
the
model,
the
system
we're
trying
to
model,
we
really
should
think
Beyond.
A
Just
the
software.
We
should
look
at
it
in
the
context
of
the
broader
system
and
again,
that's
where
graph
can
really
help
us,
because
we're
trying
to
find
answers
really
by
connecting
the
dots
across
everything
that's
going
on.
So,
for
example,
if
we
have
a
customer
impacting
incident
that
might
be
recorded
in
a
our
service
desk
tool
and
we
might
know
which
customer
it
is
and
and
has
some
information
about
that
that
ticket
and,
of
course
you
know,
the
first
question
might
ask:
where
do
I
look
where's
the
problem,
but
often
within
a
large
organization?
A
It's
not
so
much
about.
Where
do
I
look
it's
like
who
do
I
get
involved.
I
don't
want
to
now
a
have
to
go
and
get
you
know
some
architect
who
understands
how
all
pieces
together
and
B
have
to
try
and
pull
in
multiple
Dev
teams
and
take
them
off
what
they
should
be
doing,
which
is
delivering
you
new
features
and
value
to
try
and
resolve
this
incident.
So
by
correlating
that
data
across
multiple
different
tools
and
domains,
we
should
more
easily
be
able
to
relate
something.
A
A
And
then
just
the
sort
of
final
point
in
terms
of
adoption,
I've
talked
about
a
number
of
different
tools
that
are
really
either
defining
graphs,
creating
graphs,
perhaps
consuming
data
and
graph
form.
But
right
now
we
don't
have
a
way
of
connecting
these
things
and
that's
very
frustrating,
given
that
the
whole
idea
here
is
that
we
should
be
able
to
extend
that
graph
from
one
tool
and
one
domain
across
multiple
tools
and
multiple
domains,
and
that's
when
really
we
start
to
see
the
big
picture.
A
So,
for
example,
you
know
taking
a
our
service
Catalyst
in
backstage
with
the
data
we're
seeing
from
our
observability
platforms,
with
our
Dev
systems
like
RCI,
CD
pipelines
and
jira,
and
then
even
sort
of
platform
resources.
We've
got
Azure
resource
graph,
we
have
AWS.
How
can
we
stitch
this
all
together,
so
potentially
at
a
future
open
observability
day?
Hopefully
we
can
be
talking
about
a
an
open
standard
for
something
we
might
be
calling
the
observability
graph
that
allows
us
to
to
really
start
to
make
sense
of
all
this
data
that
we're
collecting.
A
So
that's
all
I
had
just
to
recap:
observability
needs
signal
and
model
models
are
graph
s
already
in
use
today
across
a
number
of
tools,
but
we
should
be
thinking
about
extending
the
scope
of
what
we
think
about
as
observability.
We
can
use
that
to
connect
the
dots
to
find
those
answers,
and
do
we
need
an
open
standard
for
for
the
observability
graph.
So
thank
you
very
much
for
for
listening.