►
From YouTube: Dynatrace Sponsored Session - Observability Where are We Headed? - Alois Reitbauer, Dynatrace
Description
Join us for Kubernetes Forums Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Dynatrace Sponsored Session - Observability Where are We Headed? - Alois Reitbauer, Dynatrace
Observability is helping us to move beyond the traditional paradigm of monitoring. Companies are looking for more answers and gathering more data than traditional alerting can provide. On our journey we learned that simply having more data has just as many challenges. Ultimately, what you do with those insights from the data provides the value. Let’s look at some of these challenges and how they can be addressed.
A
Yeah,
hello,
everyone
today
briefly,
I
want
to
speak
a
bit
about
observability
and
where
we
had
it,
because
I
believe
observability
per
se,
so
being
able
to
monitor
assistance
and
ask
questions
is
not
really
the
final
goal.
It's
a
starting
point
on
a
journey
and
there
are
a
couple
of
things
that
come
out
of
it.
A
My
point
is
we
really
actually
need
both
there's
a
lot
of
discussions,
and
sometimes
you
hear
people
saying
that
well,
observer
ability
is
great,
it's
actually
replacing
monitoring
and
we
should
move
to
observe
ability
from
traditional
monitoring.
I
think
the
story
is
a
bit
more
complicated
than
that.
If
you
look
at
what
monitoring
can
do
for
you?
First,
you
monitor
well-defined
metrics,
which
is
super
helpful,
which
is
still
what
a
lot
of
people
want
to
use.
The
typically
wants
response
times,
error,
wait,
load
and
so
forth.
A
You
define
arrows
on
those
metrics
very
often,
because
you
understand
them
well
and
you
know
how
to
define
so
difficulty
fine
alerts
on
them,
because
you
know
how
to
understand
them
or
you
have
a
system
learn
how
to
have
errors
defined
on
them.
Obviously
using
anomaly
detection,
and
then
you
can
build
up
with
the
pre-built
dashboards,
because
only
they
were
only
way
it
can
be
to
build
people
dashboards,
as
if
you
know
what
you
want
to
put
on
them.
Why
this
is
important,
because
this
approach
actually
scales
very
well
for
large
organizations.
A
So
if
you
really
roll
out
monitoring
to
a
lot
of
teams
which
micro-service
architectures
kind
of
force
us
to
do,
if
you
have
hundreds
of
teams-
and
we
want
to
have
a
consistent
picture-
we
build
everything
on
a
consistent
basis,
and
we
know
that
we
understand
somebody
else's
system
when
we
look
in
it
and
last
but
not
least,
they're
great
for
operations
teams.
So
if
you
have
to
train
an
Operations
team
on
applications,
you
want
to
have
some
standardization.
A
What
they're,
looking
for
and
for
fault,
isolation
we
had
today
and
cause
before
we
talked
about
root,
cause
analysis
and
finding
what
what
the
problem
is
to
fix
and
how
to
remediate.
Also,
we
had
a
notorious
differentiate
between
fault
reservation,
knowing
which
component
is
actually
broken
versus
then
doing
the
root
cause
analysis
so
figuring
out
what
the
exact
reason
for
something
brokenness
and
last
but
not
least,
once
you
build
automation
on
top
of
your
problems.
A
A
Super
abilities
often
referred
to
as
monitoring
the
unknown.
So
let's
put
it
that
way,
it's
monitoring
or
looking
at
all
the
other
metrics
in
your
environment
that
are
not
part
of
the
core
monitoring
strategy.
That's
part
of
your
fault
domain
isolation.
What
you
want
to
do
is
you
want
to
provide
flexible,
multi-dimensional
analysis
of
the
data,
so
you
want
to
ask
it
questions,
that's
technically
what
you
do.
You
have
tags,
you
have
dimensions
and
cardinality,
so
sometimes
it's
just
called
multi
cardinality,
while
those
are
multi-dimensional
and
you
want
to
provide
a
flexible
data
visualization.
A
Based
on
the
questions
you
ask,
you
want
to
look
at
the
data
in
different
ways.
This
is
great,
but
this
is
also
a
totally
different
use
case
and
I.
Think
that's
the
important
piece
here.
It's
great
for
its
hoc
analysis,
it's
great
for
res
the
reason
they
want
to
go
into
root
cause
analysis,
it's
great
for
capacity
planning,
it's
also
great,
for
if
we
learn
and
what,
if
analysis,
so
that's
the
key
bits
and
pieces
that
you
want
to
look
into
and
I
think
you
need
to.
A
A
If
we
talk
about
unknown
unknowns,
we
obviously
only
talk
about
humans,
because
you
can't
expect
to
ask
your
system
about
something
that
it
doesn't
know.
So
unknown
unknowns
only
exist,
for
us
is
the
human
user,
but
not
for
the
system.
What
this
actually
means
is
you
have
to
bring
all
the
information
into
your
system,
because
otherwise
you
cannot
ask
questions
about
it.
So
the
first
thing
you
have
to
do.
A
You
have
to
understand
a
very
power
to
build
a
very
powerful
system
or
a
model
in
your
system
that
helps
you
to
understand
all
of
the
complexity,
and
what
you
have
here
is
a
model
I'm
taking
this
out
of
Thanet
words,
but
technically
what
we
do.
We
take
all
the
data
and
put
it
into
consistence
easily
the
query
model
which
in
this
case
it's
a
five
dimensional
data
presentation-wise
it
five-dimensional
it
reports
in
ten.
A
It
will
present,
obviously
the
lower
level
the
infrastructure
it
represents,
the
application
level,
which
is
the
services,
but
also
the
user
in
the
user
and
business
level
like
which
transaction
do
certain
services
belong
to
you.
Just
looking
at
a
login
service
won't
tell
you
whether
this
is
a
part
of
like
an
uber
example,
whereas
this
currently
maybe
somebody
logging
in
as
a
driver
or
somebody
who
wants
to
consume
something
and
even
less
it
won't
help
you.
If
this
is
one
of
your
premium,
customers
or
other
customers,
then
the
fourth
dimension
is
time.
A
In
order
to
get
all
of
this
data,
you
can't
rely
on
manual
instrumentation,
because
then
you're
back
to
your
known
knowns
people
will
only
instrument
what
they
know
about
and
very
often
they
forget
something.
That's
why
we
believe
that
having
learning
agents
as
part
of
your
infrastructure
is
key
to
monitoring.
So
it
means
you
deploy
an
agent
on
the
system
that
automatically
detects
what
is
running
on
these
environments
and
installs.
All
the
monitoring
automatically
that's
available.
A
Think
of
it
more
like
like
like
a
big
Service
Catalog,
they
would
install
things
automatically
rather
than
relying
on
a
human
to
install
something
somebody
decides
to
use
Redis
ensure
that
the
very
scarce
automatically
monitors
somebody
decides
to
use
a
certain
AWS
service
insurer
gets
more
into
it
and
very
often
not
there
is
nobody
in
the
company
who
knows
about
like
all
of
the
ADA
health
and
she's
loosed
in
a
system.
This
is
just
a
very
small
system
and
you
already
see
all
the
different
types
of
technologies
that
are
used
in
there.
A
So
rather
you
want
to
have
a
big
catalog
of
monitoring
it's
automatically
applied
and
have
this
adaptive
learning
agents
running
on
the
system
that
will
turn
out
to
magically.
Do
it
then
there's
another
problem
which
we
need
to
solve,
that
we,
unfortunately
that
don't
talk
enough
about
which
is
human
buyers
and
observability.
A
If
it's
up
to
us
to
ask
questions
that
also,
the
quality
of
the
answers
depends
on
the
questions
that
are
asked
and
just
to
give
you
a
number
of
a
couple
of
examples
here
why
humans
need
assistance
in
analysis
because
we're
actually,
if
you
look
at
human
buyers-
and
you
can
google
human
bias,
then
you
see
how
bad
the
humans
are
in
analyzing
data.
The
first
bias
that
usually
run
into-
and
this
is
very
true,
especially
for
performance
and
problem
analysis-
is
the
confirmation
bias.
A
So
usually
we
target
our
analysis
to
what
we
already
think
might
be
the
problem
and
a
lot
of
the
questions.
We
will
be
asking
a
system
target
towards
okay.
We
assume
it
is
a
WS,
that's
the
problem,
so
we
will
ask
a
lot
of
questions
and
the
more
data
we
will
look
for
data.
That
confirms
our
initial
assumption.
This,
however,
might
have
lead
in
the
wrong
direction.
A
A
If
this
seems
to
be
an
obvious
solution,
we
are
preferring
it
over
a
highly
complex
solution
and
the
last
one
is
biases
confounding
variables.
Let
me
give
you
an
example:
people
who
are
using
umbrellas
are
more
likely
to
break
their
legs
than
people
who
don't.
This
would
be
a
valid
study.
What
did
you
forget
about
this?
There
is
a
so
called
confounding
like
Stewart
variable
in
it,
which
is
like
it's
obviously
raining,
so
people
might
be
slipping
on
the
street.
A
That's
why
they're
wearing
an
umbrella
in
the
first
case,
and
that's
why
they
might
be
slipping
more
often
than
others.
So
sometimes
we
correlate
variables
together
that
Coralie
depart
himself,
but
as
a
certain
force
variable
in
between
which
we
forget
about,
and
that's
even
where
a
great
master
Yoda
said
that
humans
are
very
biased,
and
we
have
to
be
aware
of
this
whenever
we
do
data
analysis
and
lead
to
these
results.
A
A
What's
the
potential
hood
curse
and
you
also
see
the
dimensions
that
were
analyzed
plus
what
we
call
explainable
AI,
so
using
AI
algorithms
that
analyze
all
the
data,
because
once
we
use
algorithms
to
analyze
data,
we
can
analyze
everything
and
overcome
a
lot
of
that
bias,
leading
us
in
the
right
direction,
while
still
keeping
this
explained
ability
in
here
and
having
it.
The
system
explained
back
to
us.
A
It
also
helps
us
to
scale,
because
we
are
simply
dealing
with
a
complex
problem
here.
I
know
that's
a
funny
statement.
If
you
look
at
this,
these
are
the
number
of
interdependencies
that
you
have
in
a
system,
so
this
is
just
I'm
taking
one
service,
two
services,
three
services
and
providing
an
order
of
how
these
events
affect
each
other.
This
is
where
our
brains
are
that's
the
lower
piece-
and
this
is
a
logarithmic
scale
by
the
way,
so
actually
that
scale
in
reality
looks
more
like
this.
A
So
you
see
that
there
is
like
this
breaking
point
at
the
very
end,
where
it's
totally
impossible
for
human
to
analyze
it.
Unfortunately,
we
talk
about
micro
services
that
have
way
more
events
that
come
together
and
where
we
have
to
build
in
some
type
of
observability
in
there,
so
we're
good
at
analyzing
small
amounts
of
data.
A
And
here
we
go,
so
you
see
a
problem
here
and
the
sister
front-end
errors
that
you
would
see
unexpected
and
of
input
cannot
say
property
of
undefined.
Something
like
this.
Usually
this
would
result
in
a
front-end
problem,
because
this
is
obviously
a
front-end
problem,
because
these
are
all
Chavez
would
air,
and
somebody
forgot
to
include
something
in
this
analysis,
so
you
would
go
to
your
front-end
team.
You
will
tell
them
what
they
were,
but
what
they
did
wrong
and
what
they
deployed.
A
Last
yesterday,
they
might
come
back
to
you
now
telling
you
that
they
did
not
deploy
anything
yesterday
or
even
worse.
They
would
tell
you
that
they
deployed
something
yesterday
and
you
would
look
for
an
error.
The
interesting
thing
here,
however,
is
when
you
look
back
into
the
actual
problem,
and
this
is
what
I
mean
by
AI,
assisted
that
it
will
tell
you
that
the
problem
lies
actually
somewhere
completely
different
and
gives
it
a
reason
for
this
actions.
Failure
and
the
reason
this
case
was
that
the
couchdb
was
failing.
A
The
CouchDB
was
failing
because
there
was
an
error
in
the
configuration
of
CouchDB
which
resolved
it
in
CouchDB
crashing
and
not
returning
any,
not
returning
any
JavaScript
anymore
to
the
front-end,
which
obviously
resulted
in
the
front
and
a
lot
of
JavaScript
variables
that
we're
relying
on
JavaScript
to
be
broken.
So
this
is
a
very
simple
problem.
It
already
shows
you
how
bias
is
actually
putting
us
into
a
situation
where
we
as
humans
take
the
wrong
decision
and
follow
all
of
these
practices
and
just
take
this
as
some
feedback.