►
Description
Why Understanding Data (not just collecting it) is Necessary to Deliver High Performance Dynamic Applications.
When new technology platforms appear, new application monitoring solutions also emerge to provide application and platform visibility. Dynamic containerized microservice applications are no different – monitoring performance first requires visibility, which is why it’s nice that open source standards like OpenTracing and Jaeger have appeared. Unfortunately, just collecting the tracing data is no longer enough. These new applications are so complex and dynamic that the most important aspect of monitoring is how pieces fit together, correlating information from multiple data sources. We’ll explore the pros and cons of different monitoring technologies, look at the challenges of monitoring new microservice applications, and dig into how to best use the vast monitoring data now available to deliver optimum application performance.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
Join us for KubeCon + CloudNativeCon in San Diego November 18 - 21. Learn more at https://bit.ly/2XTN3ho. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Right,
let's
go
ahead
and
get
started.
We've
got
quite
a
few
attendees.
A
good
topic
for
everyone
today
today
is
cognitive
computing
foundation.
Webinar
is
monitoring
micro
service
applications,
it's
about
visibility
and
it
isn't
I'm
Lee,
Cal,
Coates,
I'm,
head
of
Technology
Strategy
at
SolarWinds,
and
the
CMC
F
and
after
I'd
like
to
thank
everyone
who
is
joining
us
today.
We'd
like
I,
send
out
a
warm
welcome
to
Chris
Farrell
from
Astana.
Chris
will
be
our
presenter
today.
A
Interesting
fact,
Chris
joined
in
Astana
around
about
a
year
ago.
I
think
he
started
his
first
day
at
dr.
Khan.
So
what
what
an
appropriate
first
day
for
him?
We
certainly
appreciated
Astana
I'm,
sharing
their
experience
on
monitoring
micro
services
with
all
of
our
listeners.
Today,
a
few
housekeeping
items
will
cover
just
before.
Chris
comes
on
note
that
during
the
webinar,
you
will
not
be
able
to
speak
as
an
attendee.
B
A
B
Thank
You
Lee
and
thanks
to
CN
CF
for
inviting
us
here
today,
I'm
quite
excited
to
be
talking
about
managing
micro
services,
and
specifically,
this
topic
was
amusing
for
me
to
put
together
that
managing
micro
services
about
visibility.
And
it's
not
take
you
through
the
agenda.
Real,
quick,
we're
gonna
start
with
a
bad
joke
that
wasn't
it
I
guarantee
you,
the
one
I'm
gonna
tell
is
going
to
be
worse.
B
I
might
show
you
some
screenshots
or
might
even
show
you
some
live
product
if
it
makes
sense
to
kind
of
tie
in,
but
I
really
wanted
to
keep
this
at
a
level
of
talking
about
concepts
and
solutions,
as
opposed
to
show
you
product.
But
you
will
see
a
couple
of
glimpses
of
the
in
sana
product
along
the
way.
B
So
a
little
bit
of
fuse
ik
physics
humor
here
to
get
some
people
at
the
two-speed
Schrodinger's
cat
is
a
famous
physics
experiment
about
observability
and
essentially
Schrodinger
did
an
experiment
where
he
to
put
a
cat
in
the
box
and
put
something
lethal
in
the
box
with
the
cat.
But
it
was
on
a
time
thing,
or
maybe
it
was
something
that
was
fragile
and
might
break.
And
the
question
is:
is
the
cat
dead
or
alive?
And
of
course
you
can't
really
know
without
opening
the
box
and
ruining
the
experiment.
B
So
Schrodinger
came
up
with
the
answer
that
the
cat
is
both
dead
and
alive,
while
it's
in
the
box
and
you're
not
seeing
it.
Many
of
you
probably
already
know
that,
but
one
of
my
favorite
jokes
around
this
is
that
Schrodinger's
cat
walks
into
a
bar
and
it
doesn't
and
the
reason
I
love
this
joke.
I
hope
some
of
you
are
laughing.
B
Some
of
you
were
probably
groaning,
but
the
reason
I
love
it
is
that,
because
managing
micro-services
and
specifically
micro
service
applications
is
about
having
the
right
level
of
visibility
into
all
the
things
that
are
part
of
those
applications
and,
at
the
same
time,
it's
not
so
very
much
like
Schrodinger's
cat.
It
is
both
critical
thing
and,
at
the
same
time,
it's
not
critical
to
really
understand.
B
What's
going
on
in
the
micro-services
world
to
understand
why
we've
gotten
to
that
point
and
why
that's
important
I
wanted
to
talk
a
little
bit
about
application,
performance
management
specifically
and
what
that
has
meant
over
the
years
to
organizations
of
many
sizes.
So,
let's
talk
about
the
first
generation
of
APM
I
actually
was
part
of
the
first
generation
of
APM.
B
And
yes,
what
that
means
is
that
I'm,
really
really
old,
when,
when
the
first
APM
solutions
came
out
around
18
to
20
years
ago,
the
critical
business
applications
at
some
of
the
world's
largest
companies
were
just
then
moving
to
a
three-tiered
monolithic
java
application
structure
and
I
literally
just
mean
Java.
There
were
no
other
languages
to
write
these
things
in.
They
were
done
in
Java.
B
There
was
actually
a
lack
of
confidence
and
pretty
much
everything
about
this
architecture.
The
app
servers
were
flaky.
The
technology
itself
was
kind
of
new
and
flaky.
You
weren't
really
sure
what
you
were
gonna
get
from
your
programmers
and,
of
course,
what
you
were
doing
was
migrating
a
set
of
legacy,
applications
written
on
mainframes,
written
in
COBOL,
written
in
running,
on
a
s
400s
and
in
these
client-server
environments,
and
trying
to
put
them
into
this
integrated,
middleware
type
of
application,
and
there
were
all
kinds
of
funky
things
that
might
happen.
B
B
So
that's
why
the
first
generation
of
APM
tools
was
all
about
providing
visibility
of
code
in
production,
and
there
was
some
a
little
bit
of
understanding
of
what
transactions
were
going
on
there
and
then
the
idea
was
to
try
to
get
an
understanding
of
business
logic
too,
and
so
each
of
the
tools
that
came
out
during
this
four
to
five
year
period
had
some
different
ways
of
manually
handling
what
transactions
looked
like
and
what
business
logic
looked
like
now.
This
was
required.
B
This
was
the
visibility
that
was
quired
by
the
the
tool
users
to
monitor
things,
to
detect
problems,
to
see
where
they
problems
might
be,
and
to
get
to
the
real
heart
of
why
you
have
an
APM
solution
which
is
to
solve
production
problems.
So
that's
first
generation
and
then
things
shifted.
A
little
bit
more
languages
came
around.
It
started
with
dotnet
becoming
a
more
prevalent
used
language.
Well,
not
a
language
c-sharp,
but
a
use
platform
for
developing
and
deploying
these
applications.
B
Well,
let's
fast
forward
a
little
bit
about
another
ten
years
and
we
start
to
see
the
evolution
in
of
the
application
space
itself
into
containers
and
in
fact,
I
was
just
talking
with
someone
a
few
weeks
ago
about
how
quickly
containers
were
not
only
adopted
by
early
adopters,
but
how
quickly
these
same
companies
that
were
so
reluctant
20
years
ago
to
even
have
one
java
application
are
really
testing
playing
with
and
deploying
containerized
applications.
They
have
been
just
the
so
quick
to
be
adopted
and
used
in
critical
business
applications
for
many
different
reasons.
B
They
enable
a
faster
velocity
on
application
delivery
because
they
enable
every
single
developer
to
run
their
favorite
and
best
specific
platforms
that
they
need
for
their
services
that
they're
responsible
for
so
containers
really
help
us
enable
this
fully
distributed
development
team
where
every
developer
is
operating
on
their
own
as
well
as
part
of
the
broader
team.
So
they
are
the
basis
of
many
micro
service
applications
and,
as
I
said,
it's
been
an
extremely
fast
adoption
cycle.
As
far
as
how
large
companies
and
small
companies
are
deploying.
B
Production
level,
containerized
applications,
so
with
most
switches
and
platforms,
the
monitoring
tools
have
to
switch
as
well
and
I
will
tell
you
that
monitoring,
micro-service
applications
requires
a
whole
new
set
of
visibility.
Why
there's
a
couple
of
reasons?
One
is
we're
not
dealing
with
one
language,
we're
not
dealing
with
two
languages,
we're
dealing
with
a
dozen
languages
or
more
so
you
have
some
applications
that
are
completely
written
in
things
like
Scala
or
nodejs,
or
we
have
graphic
pieces
UI
pieces
written
in
Ruby.
We
have
things
they're
written
in
Python.
B
We
have
business
logic
being
put
in
Python,
we
have
PHP
making
a
comeback
and
we
have
customers
that
are
running
full-blown
transactional,
PHP
applications.
The
the
reason
for
this
is
that
every
problem
that
developers
are
trying
to
solve
every
function,
you're
trying
to
build
now,
has
some
very
specific
needs
and
they
are
no
longer
tied
to
well.
Our
our
standard
is
Java,
so
I
gotta
go
figure
out
how
to
do
this
in
Java
they're
going
to
use
the
language.
That's
easiest,
most
robust,
maybe
the
the
the
fastest
to
code
and
get
out
there.
B
As
an
example,
when
I
was
part
of
that
APM
Gen
1
again,
we
were
all
excited
because
you
know
we
had
this
production,
Java
visibility,
and
then
you
built
connectors
onto
that
essentially
to
watch
specific
back-end
systems.
But
there
were
maybe
three
or
four
of
those
today
we're
dealing
with
over
a
hundred
different
micro
service
platforms
and
I've
mentioned
what
some
of
them
the
way
that
some
of
them
fall
into
the
architecture,
world
of
messaging
systems
and
storage
systems.
We
have
cached
specific
things.
B
We
have
databases
specific
to
search
like
Cassandra
and
elastic
search,
so
there's
all
kinds
of
specialization
that
developers
can
use
to
make
it
easier
to
develop
easier
to
architect
and
easier
to
deploy
their
micro
service
applications.
So
what
does
that
mean
from
the
visibility
perspective?
So
here's
the
here's,
the
first
half
of
our
premise,
monitoring
micro
services,
is
about
visibility.
B
The
first
is
the
same
as
any
other
APM
solution,
code
level,
code
level,
visibility,
but
it's
code
level,
visibility
for
all
the
different
languages,
the
polyglot
of
languages
that
are
out
there
and
rather
than
having
that
visibility
to
help
build
an
application
picture.
Instead,
it's
there
for
when
you
need
it.
When
the
problem
ends
up
showing
that
it's
inside
a
Python
component,
then
it's
good
to
have
the
Python
code
level
visibility.
B
Secondly,
is
understanding
the
technology
platforms
that
are
being
used
and
specifically
there's
performance
data
going
on
both
inside
the
platform
and
in
its
response
to
application
requests.
There's
transaction
information
to
understand,
what's
going
on,
there's
the
configuration
itself
of
what's
in
that
system
and
if
it's
a
layered
system,
so,
for
example,
if
it's
a
Cassandra
system
running
in
a
docker
container
in
a
kubernetes,
orchestrated
environment
or
even
better
a
de
kubernetes
engine
from
OpenShift
or
IBM
or
Google.
B
Now
you
have
these
layers
of
different
configuration
and
performance
pieces
that
you
want
to
be
able
to
see
so
that
you
truly
understand.
What's
impacting
application
performance,
there
are
two
more
pieces.
The
next
one
is
dependency
mapping
between
all
the
different
pieces.
Now
remember,
we
said
at
Jen
to
a
p.m.
so
acree
ated
this
distributed
application
environment
where
it
was
important
to
map
individual
application
components,
but
they
were
application
components.
They
were
two
pieces
of
Java
that
dealt
with
each
other
and
that
they
would
be
reused.
B
What
we're
talking
about
here
is
dependency
mapping
between
all
the
different
micro
services
and
technologies
that
you
use
and
how
they
band
together
to
create
the
broader
applications
and
that
dependency
of
course,
changes
over
time.
We're
going
to
talk
about
that
in
a
minute.
The
last
piece
of
visibility
that
you
want
is
the
ability
to
trace
all
the
requests
that
come
through
in
the
end
across
the
entire
distributed
application
and,
of
course,
there
are
many
different
ways
to
trace
these
days.
B
There's
open
tracing,
here's
a
Jaeger,
but
also
you
have
tracing
built
into
the
tool,
some
of
the
tools
themselves,
for
example
at
Insogna.
We
auto
trace
100%
of
requests
ourselves,
while
also
supporting
the
third-party
tracing
systems,
because
there
are
sometimes
there's
things
that
you
want.
The
developer
or
the
developer
feels
like
should
be
exposed
other
than
service
level,
KPIs
or
performance
information.
B
So,
but
it's
not
now
we're
gonna
start
talking
about
it's,
not
just
about
visibility
with
all
that
comprehensive
performance
configuration
and
trace
data.
What
else
would
you
possibly
need,
and
the
reason
that
you
need
more
is
because
of
a
dirty
little
word
and
micro
services,
or
maybe
an
exciting
little
ward,
depending
on
where
you
sit,
and
that
word
is
dynamism
or
dynamic.
It's
the
dynamic
aspect
of
these
applications
that
make
just
pure
visibility
a
difficult
to
achieve,
but
also
difficult
to
you
and
be
the
only
thing
that
you
need.
Why?
B
Because
you're
spinning
up
in
destroying
containers
and
microservices
all
the
time
now,
sometimes
you
have
an
application
that
it's
not
destroying
these
things
all
the
time.
But
the
idea
is
that
these
these
technologies
are
built
to
allow
you
to
be
dynamic
and
how
you
deploy
and
how
you
utilize
your
resources
that
you
have
available
to
you.
B
So
the
truth
is
is
that
over
the
last
few
years,
what
we've
seen
at
customers
is,
the
average
lifespan
of
a
container
has
shrunk
by
about
80
percent
and
I
can
I
expect
that
to
continue
to
shrink
as
we
get
into
more
agile,
faster
velocity
development
teams.
There's
also,
the
idea
of
those
agile,
faster
velocity
development
is
you're
trying
to
get
to
that
continuous
integration.
B
Continuous
delivery
aspect-
I
was
just
talking
to
a
customer
last
week
where
they
used
to
have
two
updates
a
month,
and
now
they
have
about
200
updates
a
day
with
20
of
those
updates
plan,
essentially
being
production
updates
on
an
average
basis.
20
production
updates
a
day
in
that
environment.
Just
having
visibility
isn't
enough.
You
have
to
be
able
to
deal
with
this
dynamic
aspect
and
then
the
last
piece
of
the
dynamism
is
all
those
fluid
relationships
and
dependencies.
That
is
the
reason
we're
going
to
these
environments.
B
No
single
transit,
no
two
transactions
necessarily
follow
the
same
path.
So
the
ability
to
understand
exactly
where
things
go
is
very
important
as
you
get
to
that
problem-solving
aspect
of
what's
causing
issues
in
my
microcircuits
application.
So
dealing
with
this
requires
visibility,
because
you
have
to
have
all
that
information,
but
it's
just
raw
data
if
you
can't
do
anything
with
it.
So
what
we
found
is
that
it
requires
really
programmatic,
AI
and
automation
to
effectively
monitor,
not
just
the
fact
that
these
things
are
running,
but
also
the
overall
performance.
B
So
let's
talk
about
what
these
things
are
doing,
if
we
think
about
the
really
really
old
days,
even
before
the
the
20
year,
rewind
I
talked
about
earlier,
we
had
mainframes
and
every
application
was
essentially
contained
on
one
place,
so
you
only
had
to
monitor
the
one
thing
and
then
client/server
came
along
and
a
set
of
tools
came
along
to
up
to
monitor
it.
Then
we
got
to
the
monolithic,
treats
your
applications.
We've
got
to
SOA
a
few
years
later.
Then
we
started
talking
about
micro
services.
B
Then
we
started
having
containerized
micro
services
and
where
are
we
going
we're
going
to
orchestrated
containerized
micro
services?
Now
all
of
these
things
are
creating
a
kind
of
enabling
development
to
move
their
velocity
to
quicken
how
they
deploy
to
get
closer
to
continuous
integration.
Continuous
delivery.
B
Now
the
interesting
thing
and
what
I
always
talk
about
is
the
every
tool
that
makes
it
easier
and
faster
for
developers
to
deploy,
makes
it
more
complex
and
more
that's
difficult
for
IT
operations
or
DevOps
to
actually
monitor,
configure
and
manage
the
performance
of
these
applications
that
are
running
and
because
of
the
way
that
these
are
going.
Automation
ends
up
being
a
key
part
of
being
able
to
actually
handle
all
this,
and
before
we
talk
about
what
we're
gonna
automate
III
like
to
tell
you
this.
B
This
comes
right
from
our
CTO
and
his
talks
about
AI
and
why
AI
is
important,
and
so
there's
this
uncertainty,
determination,
quadrant
and
this
is
actually
from
a
financial
planning
system.
So
it's
it's.
It's
actually
kind
of
neat
to
apply
these
aspects
of
complex
system
management
from
non
IT
into
our
IT
environment.
So
there's
this
matrix
of.
What's
known,
what
can
we
known
what's
unknown,
and
what
do
we
not
know
about
that?
So
there
are
things
that
we
are
aware
of,
and
we
understand
it.
B
This
way
is
the
is
what
you
have
to
think
about
from
a
DevOps
perspective,
when
you
consider
these
complex,
dynamic
systems
that
are
being
deployed
and-
and
we
talked
about
the
polyglot
of
technologies-
I
described
this
new
world-
is
kind
of
the
wild
wild
west
of
Technology
you
essentially
in
the
past,
when
you're
three-tiered
you'd
sit
down
as
a
group
with
some
senior
architects
and
you
decide
well.
This
is
our
web
server
technology
we're
going
to
use.
B
This
is
our
app
server
technology
we're
going
to
use,
and
this
is
our
database
technology
that
we're
gonna
use,
and
maybe
you
had
a
transaction
system
or
some
messaging
that
you
did,
but
you
picked
the
technology
that
was
going
to
be
standard
and
everybody
would
use
well
in
the
microservices
world
that
that
goes
out
the
door
you're
like
hey
our
database,
that
we
use
is
sequel,
server
and
three
developers
say
well.
The
best
database
I
can
use
what
I'm
doing
is
my
sequel.
B
So
that's
what
I
programmed
in
and
oh
by
the
way,
it
was
so
easy
to
deploy
a
container
that
it's
actually
running
and,
and
so
DevOps
has
to
be
able
to
pick
up
and
monitor
and
manage
these
things
that
they
didn't
necessarily
pick
as
part
of
it
and,
of
course,
the
more
complex
the
system,
the
broader
the
set
of
tools
that
you
use.
So
when
you
think
about
the
classic
APM
tools,
whether
gin,
one
or
gin,
two
and
what
they're
doing
to
to
troubleshoot
and
you
apply
it
to
the
quadrant.
B
They
understand
the
known
unknowns
right,
hey
I,
know,
there's
a
problem
but
I'm
not
really
sure.
What's
going
on
with
the
business
logic
or
with
the
requests,
and
they
can
all
solve
the
already
known
knowns
all
by
focusing
in
on
the
application.
But
they
can't
really
get
beyond
those
things,
because
they
can
only
monitor
the
things
you
told
them
to
monitor
and
the
things
that
you
knew
about
ahead
of
time.
And
so
that's
why
he
trying
to
get
to
the
next
set
of
things,
and
that
requires
AI.
B
Now,
interestingly
enough,
AI
has
been
a
big
buzzword.
Of
course,
not
just
in
IT
and
monitoring,
but
across
many
of
the
other
systems
we
use.
I
get
I,
probably
get
a
dozen
emails
a
day
from
marketing
technology.
Vendors
tell
me
how
AI
is
gonna
help
me
find
people
that
are
ready
to
buy
or
find
better
leads
within
my
own
database
or
be
able
to
set
messages
at
a
show
that
are
going
to
attract
more
of
the
audience.
I
want
all
by
applying
AI
and
we're
not
talking
about
Cyberdyne
systems.
B
Here
we're
talking
about
a
couple
of
different
axes
of
different
types
of
functionality
that
we
all
recognize
as
part
of
the
AI
base.
But
what
we
want
to
talk
about
what
what
we
like
to
talk
about
is
how
each
and
every
one
of
these
pieces
go
together
to
create
a
more
complete
system
to
help
you
deal
with
these
complex
environments.
So
the
first
thing
that
you
have
as
part
of
any
AI
is
a
knowledge
base
approach
and,
of
course,
there's
different
ways.
You
can
have
a
knowledge
base.
B
B
This
is
where
you
start
to
get
a
little
bit
of
intelligence
at
it
in
there,
but
in
in
reality,
as
things
change
the
rules
and
the
situations
that
that
you
have
this
knowledge
base,
for
they
start
to
shift
and
all
of
a
sudden,
the
the
the
questions
about
an
app
server
don't
apply
the
same
way.
They
do
anymore,
because
your
app
server
isn't
being
used
in
the
same
way
anymore,
and
so
you
can
set
these
free
defined
rules
but
in
reality
they're
always
shifting
and
changing.
B
B
The
next
thing
that
people
talk
about
is
doing
deep
statistical
analysis,
another
branch
of
the
AI
axis
so
to
speak,
and
whether
you,
whether
you
call
it
advanced
statistics
or
deep
statistical
and
you're,
essentially
doing
a
lot
of
correlation
and
applying
statistics.
But
when
you
apply
statistics
you
actually
are
kind
of
at
the
whim
of
the
data
and
and
if
some
of
you
have
had
to
make
reports,
maybe
you've
noticed
this
and
I
again.
I
do
this.
B
In
my
day
to
day
job
myself,
I
get
asked,
you
know,
what's
the
what's,
the
average
life
cycle
of
a
lead
to
go
from
one
step
to
the
other
and
while
that's
an
interesting
statement
and
an
interesting
piece
of
data,
maybe
the
median
is
more
interesting
because
that's
directly
in
the
middle,
because
you
could
get
skewed
data
and
what
I?
What
I
love
about
these
pictures
on
the
right
is
that
they're,
they
actually
all
show
the
same
averages
for
x
and
y,
but
obviously
they're
a
whole
different.
B
Each
one
is
a
different
data
set,
but
it
shows
you
how
data
can
be
skewed
and
and
if
you
ever
get
some
time,
go
out
and
look
for
the
dinosaur,
statistical
distribution
and
it's
actually
fun
I
could
I
could
create
all
kinds
of
distributions,
including
pictures,
fun,
pictures
of
dinosaurs.
It
all
end
up
with
the
same
mean
and
the
same
standard
deviation.
B
Let's
talk
a
little
bit
about
why
structuring
your
statistics
versus
having
a
broader
understanding
is
important.
This
is
actually
a
screenshot
from
the
product
and
one
of
the
interesting
things
here
and
and
I
know
it
I'm
about
to
say
something.
You're
gonna
go
no
you're
crazy
Chris.
This
is
actually
a
very
simplified
view
of
the
application
that
this
this
that
this
particular
instance
is
monitoring.
B
Why
is
it
set
up
like
this?
It's
set
up
like
this
because
we
as
a
IT
group,
we're
accustomed
to
looking
at
a
tier
based
map
of
our
applications.
We
create
tear
based
architectures,
we
think
of
them
in
steps
of
this
server
to
that
server
and
so
on
back
to
storage.
So
we
create
this
view
to
kind
of
help.
People
understand
a
little
bit
about
how
things
are
built
and
what's
going
on.
B
This
is
a
structured
view
of
the
world,
but
if
I
were
to
start
to
dive
into
what
the
services
actually
looked
like
that
make
this
view,
you
start
to
get
a
little
bit
less
structured,
and
now
you
start
to
see
that
it's
more,
it's
more
like
a
just,
a
random
collection
of
services
that
tie
together
but
believe
it
or
not.
This
is
actually
still
structured
a
little
bit.
B
We
wanted
to
create
a
view
that
allows
you
to
understand
a
little
bit
more,
that
of
the
complexity
of
what's
going
on
in
your
application,
without
essentially
giving
you
overload
of
the
brain.
What
does
this
really
look
like?
This
is
a
simple
application
in
a
pure
micro-services
mode,
because
every
micro
service
has
the
ability
to
come
up,
get
destroyed,
communicate
with
any
other
micro
service
when
and
where
at
once
and
have
very
loosely
connected
connections.
So
you
end
up
with
essentially
this
blob
of
things
that
tie
together.
B
So
one
of
the
things
that
you
have
to
do
as
a
monitoring
tool
in
this
world
is
to
help
not
only
find
all
this
data.
Remember
that's
the
visibility
part,
but
also
be
able
to
structure
it
in
such
a
way
that
it's
understandable,
at
least
from
a
from
a
organizational
perspective
to
the
ops
team.
No
individual
is
going
to
be
able
to
understand
everything
about
every
application
anymore.
That's
just
not
going
to
be
possible,
but
also
just
because
I
have
this
view,
and
this
is
the
way
the
data
looks.
B
You
don't
want
to
just
throw
this
at
people,
so
hence
coming
back
to
this
structured
view
of
the
world.
So
now
that
we
have
this
ability
to
kind
of
structure
everything
together,
we
can
start
applying
the
next
layer
of
AI
to
this
and
that
next
layer
is
one
really
most
organizations
that
start
talking
about
having
AI
as
part
of
their
tool,
whether
it's
a
marketing
tool
or
an
IT
tool.
That's
where
machine
learning
starts
to
come
into
place
right.
Machine
learning
starts
with
some
deep
statistical
analysis,
but
it
has
to
go
on.
B
It
has
to
start
taking
an
understanding
of
okay.
So
this
is
my
standard
deviation.
This
is
my
mean,
but
what
are
some
of
the
other
things
we're
looking
at?
What
model
do
we
want
to
apply?
Can
can
the
Machine
actually
learn
what
the
model,
what
the
distribution
looks
like
and
start
applying
different
statistical
models
to
it?
B
So
now
you
start
to
get
into
some
knowledge
base
concepts,
not
just
knowledge
base
about
technologies,
not
just
knowledge
base
about
error
codes,
or
things
like
that,
but
now
starting
to
apply
statistical
information,
the
knowledge
about
how
things
are
modeled,
how
data
is
distributed
and,
depending
on
that
model
and
distribution,
what
should
you
do
with
it?
What
how
are
you
going
to
determine
whether
things
are
good
or
bad
now?
Why
is
that
important?
Because
in
an
application
that
looks
like
this
and
again,
this
is
actually
a
simple
application.
This
is
our
demo
application.
B
There
are
applications
that
that
in
real
life
are
ten
times
a
hundred
times
a
thousand
times
more
complex
than
this.
So
how
can
you
apply
that
in
a
way
to
help
the
end
user,
that's
operations
and
DevOps
understand
what
they
need
to
do
next,
because
in
this
world
you
can't
configure
monitoring,
you
can't
go
set
thresholds,
you
can't
go
say
this
is
a
piece
of
metric
that
I
want
to
look
at
for
this
type
of
technology.
That
is
a
non-starter.
B
You
will
spend
all
the
time
you
have
and
then
still
not
be
done
with
setting
up
and
configuring
your
monitoring.
So
you
have
to
start
applying
these
layered
aspects
of
automation
in
AI,
so
machine
learning
concepts,
but
on
the
statistical
analysis
of
things
and
then
so
that
the
nice
thing
is
that
once
you
start
training
your
system
on
algorithms
and
distributions
and
statistics
all
of
a
sudden,
now
those
unknown
unknowns
and
start
to
move
to
unknown
knowns.
B
And
so
you
start
to
move
around
that
quadrant,
which
means
that
you,
the
people
and
now
take
better
actions
or
or
even
start
to
understand
what
things
are
going
on
that
need
your
intervention
to
do
stuff.
So
now
that
we've
talked
about
some
of
the
basic
premises
of
AI
and
a
p.m.
specific
types
of
situations.
Now,
let's
talk
about
what
it
means
to
practically
implement
this
in
an
APM
environment.
Well,
you
start
with
this
combination
of
the
knowledge
base,
but
it's
a
multi-tiered
knowledge
base.
It
is
technology
based.
B
It
does
have
information
that
it
knows
about
what
you
should
do
with
a
Cassandra
database
or
an
elastic
search
or
what
you
should
do
with
Redis
database
or
with
Kafka
message
queue.
And
then
you
add
in
the
aspect
of
the
statistical
understanding
knowledge
base
of
those
things,
and
then
you
start
to
apply
the
correlation
and
start
to
actually
put
the
machine
learning
in
place.
So
the
idea
is
it's
it.
B
We
wanted
to
behave
similar
to
humans
in
the
classic
APM
space.
A
problem
has
occurred.
How
do
I
see
where
the
problem
is
and
where
do
I
start
debugging
to
figure
out
where
things
are
going
so
and
of
course,
the
nice
thing
about
having
a
computer
do
it?
Is
the
computer
doesn't
get
tired
the
computer
at
2:00
in
the
morning?
B
The
idea
is
to
essentially
make
it
so
that
your
operators
can
focus
their
attention
on
the
specific
problems
that
are
going
to
impact
service
and
not
worry
about
the
thousands
of
change
events
and
warnings
that
come
up
during
the
day
or
even
issues
that
come
up
with
an
instance
of
a
container
of
something
or
even
even,
if
it's
in
an
app
server
and
it's
it's
a
something:
that's
going
to
deplete
resources.
If
it's
not
impacting
the
end-user
service,
then
you
don't
necessarily
have
to
worry
about
it.
B
So
what
can
you
automate
with
AI
and
the
in
the
APM
environment?
All
these
things
on
the
right
and
I
could
read
them,
but
my
favorites
are
issue
recognition
and
and
correlation
of
events
to
issues
to
service
incidents
and
specifically
the
idea
of
understanding
anomalies
from
the
perspective
of
they
are
a
an
impact
on
end-users
and
be
something
unexpected
anomalies
occur
all
the
time.
But
in
the
world
of
computers
and
when
I
say
unexpected,
I
don't
mean
that
the
operations
team,
so,
like,
oh
hey,
why
did
the
behavior
change?
B
We
didn't
expect
that
I
mean
the
system
didn't
expect
that,
because
the
system
understands
when
anomalies
are
actually
part
of
the
process
and
part
of
the
applications
itself,
so
AI
is
the
essential
piece
of
APM
automation
that
allows
you
to
deal
with
these
dynamic
environments,
so
what
we
bring
to
the
table.
This
is
the
way
that
we
apply
this.
We
actually
personify
our
AI
withstand
the
robot
and
we
apply
it
in
this
way.
B
We
automatically
discover,
what's
going
on
otherwise
you're,
going
to
end
up
with
too
many
holes,
we
continuously
align
with
real-time
changes
right,
as
as
the
lifecycle
of
containers
shrink
as
more
and
more
things
get
put
into
the
application
and,
more
importantly,
as
you're
developing
team
starts
to
approach
continuous
delivery.
How
do
you
deal
with
the
fact
that
all
these
changes
are
going?
The
monitoring
tool
has
to
be
a
real-time
adjusting
tool
to
what's
going
on.
You
want
high
fidelity
data
now
X.
Here
is
the
immediate
analysis.
You
want
one-second
granularity.
B
You
want
three
seconds
to
understand
a
problem
again
that
goes
back
to
the
idea
of
continuous
delivery.
If
you
have
again
bringing
up
the
customer
I
talked
about
earlier,
if
you
have
twenty
production
updates
a
day
waiting,
five
minutes
or
thirty
minutes
to
understand,
it's
a
there's,
a
problem
with
the
Tran,
with
one
of
the
one
of
the
systems
that
just
got
changed,
Wow
how
many
users
are
going
to
be
impacted?
How
many
transactions
are
going
to
fail?
B
How
much
business
are
you
going
to
lose
it's
critical,
especially
for
the
developers
part
of
the
DevOps
team
to
understand?
Hey
I
applied
this
change.
This
update
and
five
seconds
later
I
got
told
there
was
a
problem
that
means
I
can
roll
it
back
immediately.
Go
back
to
my
testing
my
drawing
board
and
see.
B
What's
going
on,
nice
thing
is
not
only
do
I
have
that
information
I
have
it
historically,
so
I
can
go
back
to
my
testing
system
and
see
why
it
missed
what
came
out
in
production,
applying
that
over
time
and
being
able
to
help
you
figure
out
where
to
start
to
debug.
Unexpected
problems
is
the
kind
of
my
favorite
piece
so
for
any
instant.
It
occurs.
B
So
in
reality,
in
these
dynamic
applications,
AI
isn't
just
a
cool
thing
to
have
it's
required
to
get
the
stuff
done.
A
big
part
of
doing.
That
is
what
we
call
our
dynamic
graph,
but
in
reality
it's
modeling.
Now
there
were
some
systems
almost
all
of
the
tools,
the
Gentoo
tools
had
modeling
as
part
of
dealing
with
us
away.
So
the
industry
and
the
operations
team
know
that
modeling
is
a
critical
piece
of
understanding
how
distributed
applications
work
and
how
things
go.
B
But
if
you
don't
have
a
full
model,
then
you're
gonna
struggle,
and
especially
if
you
don't
have
a
full
model.
Your
program,
your
AI
tool,
can't
make
the
right
decisions
right.
So
having
a
model
that
is,
has
the
ability
to
adjust
to
different
architectures
has
the
ability
to
apply
different
statistical
analyses
based
on
what
it
knows
in
the
past
and
includes
configuration
information
so
that
it
can
understand
exactly
how
everything's
supposed
to
be
stuck
together
and
then
how
it
actually
is
put
together?
B
And,
lastly,
I
want
to
talk
about
a
concept
and
if
you
get
a
chance
and
and
maybe
we'll
what
we'll
do
is
get
everyone
a
copy
of
this
white
paper.
Just
because
you
signed
up,
we
have
a
white
paper
called
the
six
pillars
of
AI.
Here
it's
called
the
six
pillars:
application
management,
but
it's
really
not
about
application
management.
It's
really
the
things
that
you
need
to
be
able
to
effectively
apply
AI
and
it
starts
with
that
always
have
the
right
model
of
what's
going
on,
so
it's
auto-discovery
in
real
time.
B
It's
that
one-second
granularity
in
three
seconds
to
know
what's
going
on,
because
you
want
to
be
able
to
make
decisions
quickly
and
the
better
your
data,
the
better
your
AI
right
or
another
way
you
to
think
about
this.
Is
you
want
to
avoid
garbage
in
garbage
out?
You
have
an
application
model
that
you
can
ground,
both
your
operations,
team
and
the
aian,
so
that
everyone
can
speak
a
common
language.
B
You
want
to
be
native
to
whatever
environment
you're
in
that's
the
only
way
and
the
best
way
to
really
have
understanding
of
what
the
applications
are
doing
and
then
the
last
few
pieces
are.
What
do
you
do
with
the
AI?
And,
of
course
we
all
want
to
get
to
the
idea
of
predictive
incidence
or
predicting
incidents
are
going
to
happen,
and
this
is
this
is
one
of
the
reasons
that
you
build.
B
A
correlated
model
of
events
and
change,
events
and
warnings
and
how
those
relate
to
issues
with
a
specific
instance
of
technology
and
how
all
those
issues
come
together
to
create
service
incidents,
and
that
way
you
can
start
to
do
things
like
predict,
based
on
a
set
of
specific
change.
Events
that
you're
gonna
have
a
service
problem
with
your
Shopp
service,
or
you
have
a
service
problem
with
your
payment
service
right
and
in
the
last
is
now
that
a
problem
has
occurred.
How
can
we
correlate
the
data
and
understand?
B
A
B
A
B
Surrealists
has
both
the
an
opportunity
to
to
create?
Remember
we
talked
about
velocity
of
development.
Server
list.
Is
that
next
thing
that's
going
to
make
it
even
faster
and
easier
to
deploy
things
for
developers
and
even
more
complicated
and
more
of
a
headache
for
Operations
teams
almost
to
the
point
of
maybe
bringing
some
of
them
to
tears
and
an
in
reality
how
you
monitor
server
lists,
we
talked
a
little
bit.
I
touched
on
the
different
tracing
technologies
that
are
available
for
Microsoft,
a
open
tracing
jäger
in
Stano
zone,
auto,
trace
capabilities.
B
The
reason
that
we
want
to
support
all
the
different
types
of
tracing
that
are
out
there
is,
first
of
all,
if
you,
knowing
that,
there's
auto,
trace
and
available.
If
you
are
having
your
development
team,
put
specific
information
in
to
to
an
open
tracing
instrumentation,
then
that's
data,
that's
important
to
you
and
if
it's
important
to
you
it's
important
to
us,
so
that's
one
part
of
it,
but
the
one
of
the
other
reasons
that
we're
we
know
we
have
to
do
this.
B
Is
it
serverless
today
really
requires
you
to
be
able
to
support
the
different
methods
of
monitoring
built
into
the
platforms
itself?
So,
for
example,
if
you
were
to
run
Amazon
lamda,
you
would
have
to
have
x-ray
support
in
your
monitoring
tool
to
be
able
to
pull
that
data
out
and
the
nice
thing
about
the
way
we
think
about
it.
B
Is
that
that's
that's
a
specific
component
of
the
system,
which,
of
course
has
a
very
limited
lifespan,
but
we
also
want
to
be
able
to
include
it
as
we
tie
into
our
KPIs
as
we
tie
into
the
maps
etc.
So
we
want
to
tie
our
x-ray
data
in
with
the
data
that
we
get
that's
x-ray.
So,
overall,
as
as
more
serverless
technologies
come,
they
will
start
to
include
their
own
monitoring,
but
also
this
is
where
developers
instrumenting
their
apps,
also
help
out
and
can
tie
together
with
auto
tracing
capabilities.
A
A
B
But
so
that's
so.
This
is
a
a
really
good
operational
aspect
in
question
and
and
what
tools
try
to
do,
especially
from
a
monitoring
perspective.
We
we're
focused
on
being
able
to
provide
as
broad
of
support
as
we
can,
which
means
that
we're
going
to
get
all
kinds
of
messages
and
all
kinds
of
errors
and
all
kinds
of
different
scenarios
that
come
up
and
one
of
the
things
that
you
want
to
have
happen.
B
Of
course,
is
actions
get
taken
so
when
you
think
about,
for
example,
an
orchestration
engine,
the
kubernetes
open
source
or
an
open
shift
version
of
it
or
one
of
the
other
kubernetes
engines.
One
of
the
reasons
you
have
kubernetes
in
place
is
to
start
to
do
things
like
provisioning,
new
servers
when
you're
running
out
of
resources
and
that's
great
if
the
environment
you're
running
in
matches
exactly
everything
that's
going
on
and
you
don't
start
to
get
resources
depleted
in
ways
other
than
the
containers
asked
for
them.
B
And
another
aspect
of
that
is:
how
does
application
performance
come
in,
like
you
might
have
all
these
resources
allocate
and
you
you
allocate
them
based
on
cost
analysis
and
and
what
you
have
available,
but
you
also
want
to
start
thinking
about.
Are
we
meeting
our
service
levels
with
this
allocation,
and
so
all
that
ties
together
to
the
ability
to
take
actions
based
on
events
based
on
performance
and
based
on
other
things
you
might
see
by
integrating
with
the
orchestration
in
Jim
first
and
then
with
other
runtime
tools
that
you
might
have
in
place?
B
So
that's
a
it's
actually
one
of
the
more
interesting
aspects
of
probably
started
with
Gen
2
tools,
but
now
with
with
all
the
modern
tools,
the
idea
is,
how
can
I
take
actions
either
myself,
sometimes
or
through
the
management
tools
that
already
exist
like
a
container
orchestration
tool
or
like
an
AI
ops
tool
that
are
that's
already
out
there?
So,
yes,
we
can
do
that.
That
will
continue
to
grow
and
to
become
a
even
faster
and
more
complete
set
of
actions
to
take.
A
B
That's
that's
a
really
specific
thing.
The
this
short
answer
is
yes.
Now
I
have
a
demo
set
sitting
here,
but
let
me
see
if
I
have
any
I
in
our
demo
system.
Our
demo
system
is
very
specific
and
in
the
way
that
it
throws
incidents
out
there,
so
I
don't
have
any
incidents
to
show
you
that
I
can
actually
show
you
the
the
auto
trigger.
But
let
me
in
answering
this:
let
me
real
quick
I'll
share
the
screen
just
so
you
guys
can
kind
of
understand
the
way
that
things
might
work
from
that
perspective.
B
So
when
you
deal
with
applications,
of
course,
in
a
microservices
environment,
it's
important
to
actually
understand
how
your
infrastructures
map
down.
So
we
have
this
nice
big
picture
that
can
be
organized
by
containers
or
it
can
be
organized
by
hosts
and
each
each
specific
place
is
each
specific
picture
depends
on
what's
running,
so
these
are
actually
bare-metal
servers.
So
each
slice
that
you
see
in
the
in
these
are
actually
processes
so
over
here.
Here's
a
spring
boot
process,
there's
a
rabbitmq
process
now
down
here.
B
This
is
my:
these
are
all
my
docker
containers,
so
every
slice
here
is
a
container.
Remember.
I
talked
about
running
native,
so
we're
actually
monitoring
each
parts
of
these
in
the
way
that
makes
sense
for
them.
So
over
here.
The
agents
running
is
a
host
process
over
here
the
agents
running
as
a
docker
container
so
that
it
can
actually
work
specifically
within
the
framework.
That's
there
and
then
you
know.
Actually
we
can.
We
can
gather
these
all
up
and
look
at
individual
containers
or
we
can
actually
dig
in
to
the
kubernetes
cluster
itself.
B
Now
the
we
set
all
this
up,
the
infrastructure
to
start
a
course
is
because
we
want
to
get
to
the
application
map,
but
remember
I,
said
we.
We
want
to
make
things
a
little
bit
more
understandable.
So
from
this
perspective,
this
is
the
idea
of
making
things
a
little
bit
more
understandable.
We
put
it
in
that
familiar
service
map
and
then
so
you
can
look
at
how
different
services
are
reacting.
B
Now
here's
your
end
user
out
here
you
get
the
end-user
response
times,
and
so
you
can
start
to
see
how
the
end
users
themselves
are
actually
impacted
or
what's
going
on
there
and
then,
if
I
dig
in
on
any
specific
service,
I
actually
can
open
it.
A
dashboard
on
a
specific
service
and
I
can
start
to
see
what
the
performance
is
over
time.
B
But,
more
importantly,
you
can
see
there
are
some
spikes
at
here
that
go
on,
so
I
can
actually
dig
in
and
look
at
traces
themselves,
and
this
is
where
you
start
to
get
into
that
idea
of
in
the
end
and
being
able
to
to
drill
into
something
specific.
None
of
these
are
particularly
long.
Like
I
said
we
don't
have
any
any
big
big
problems
out
there,
let's,
let's
actually,
let's,
let's
go
back
to
the
map
and
let's
see
if
we
can
pick
a
different
service,
maybe
down
here,
that's
I'm.
Looking
for
a
shop
service.
B
The
winners
will
go
back
to
the
product
search.
The
the
interesting
thing
with
the
traces
is
that
the
nice
thing
is
that
they
tie
into
the
specific
data.
That's
there.
So,
for
example,
here
we
have
an
elastic
search
call
and
you
can
get
a
little
bit
more
detail
about
the
elastic
search
column
and
what
it
did.
So
we
actually
have
the
spam
for
every
trace
and
I
researched
this.
B
These
are
product
searches,
so
of
course,
you're
getting
an
elastic
search,
Kozik
that
come
here,
but
you
see
the
span
of
how
where
each
step
was
spent
and
so
in
a
longer,
broader,
bigger
service
it
you
actually
would
have
some
code
pieces,
you
could
drill
into
the
code,
you
might
have
some
JDBC
queries,
you
could
drill
into
selects
and
and
and
the
and
the
queries
there.
So
that's
how
you
go
from
the
front
end
user
back
right
to
the
the
system
and
then
down
into
the
specific
traces
themselves
to
solve
problems.
A
B
So
the
answer
to
those
questions
is
yes
and
kind
of,
so
we
do
offer
trial
licenses
when
you
just
feel
free
to
come
in
and
sauna
calm
and
sign
up
for
a
trial
when
we,
when
we
communicate
with
you
after
this
event,
you
can
just
respond
to
the
email
and
say
you
would
like
a
trial
and
we'll
get
you
guys,
connect
anyone
connected
and
set
up,
and
the
nice
thing
is
that
things
happen.
It's
it's
really
easy.
It's
it's
surprise,
surprise
its
automated!
We
care
about
automation,
that's
why
we
talk
about
it.
B
As
far
as
detailed
demos
of
the
product
online.
There
are
some
interesting
videos
that
are
out
there.
They
don't
some
of
the
older
ones
showcase.
Some
of
the
things
we
talked
about
today.
We
just
released
a
nice
little
four
minute
or
two
and
a
half
minute
video
about
that
showcases,
our
new
kubernetes
monitoring,
and
it
actually
shows
all
the
things
that
you
care
about,
that
we
talked
about
today,
discovery
and
drill
down
and
and
automatic
monitoring,
but
specifically
to
the
environment
of
how
it
also
includes
kubernetes
monitoring.
As
part
of
that.