►
From YouTube: Lightning Talk: Debugging microservices applications with Envoy + Squash - Idit Levine, Solo.io
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Lightning Talk: Debugging microservices applications with Envoy + Squash - Idit Levine, Solo.io
The mainstreaming of containerization and microservices is raising a critical question by both developers and operators: how do we debug all this?
Debugging microservices applications is a difficult task. The state of the application is spread across multiple microservices, and it is hard to get a holistic view of the state of the application. To meet this challenge we developed Squash, the microservices debugger. Squash integrates with Envoy to attach a debugger to any application in the service mesh.
In this talk, we will describe the Envoy filters we wrote to integrate with Squash to facilitate distributed debugging.
A
So,
let's
go
so
I
everybody.
My
name
is
Edith
Levine
I'm,
the
founder
and
CEO
of
so
I
own,
and
this
is
your
father-
is
the
chief
architect
of
the
company,
and
we
want
to
talk
to
you
a
little
bit
about
about
debugging
in
anyway.
So,
let's
start
so,
first
of
all,
it's
in
the
standard
problem
just
to
make
sure
that
we
are
best
lining
it
right.
So
what
is
the
problem?
So
the
problem
in
debugging
microservices?
It's
the
fact
that
everything
is
distributed
right
back
on
the
day.
A
It
was
really
really
simple
to
take
one
monolithic
application,
attach
it
debugger
and
see
the
state
of
the
application,
but
that's
not
the
case
right
now.
Right
because
now
we
have
a
lot
of
little
little
bit
of
micro
services
and
you
kind
of
like
lost
the
big
state
of
the
application
and
I
pick
an
application.
You
know
here
we
have
like
seven
micro
services.
Usually
it's
much
more
right.
It
looks
something
like
that
right
and,
and
then
you
see
a
lot
of
people
tweeting,
something
like
that
right,
because
this
is
a
real
problem
right.
A
So
so
good
question
is
how
to
fix
it
and
we
will
look
at
it.
We
decided
to
put
to
create
a
squash.
So
let
me
show
you
a
quick
demo
about
how
it's
actually
working
at
then.
We
will
understand
how
it's
tied
to
and
boy
okay,
so
spinning
it
in
AWS
over
take
a
sec
yeah
I!
Don't
worry
about
that!
That's
not
the
yep!
That's
like
skip
the
script
that
I'm
running.
That's
setting
up
the
environment,
but
I
will
make
that
bigger.
A
A
A
It
again,
really
really
dumb
thing
Joe.
So,
let's
see
what
we
did
okay,
so
this
is
really
simple.
This
calculator
is
actually
two
micro-services
right.
This
is
the
first
one.
As
you
can
see,
this
is
Visual
Studio
code,
and
it's
really
it's
really
really
simple.
It's
basically
serving
manage
my
HTML.
It's
really
neat
go,
but
it's
serving
the
HTML
and
the
other
one.
It's
a
juggling
micro-services,
but
basically
getting
the
topogrator
europe
ed
up,
sub
snack
and
return
the
result
of
the
door
micro
services.
So
simple
right!
A
A
Now.
Now
we
did
the
same
thing
for
the
other
one,
so
we're
coming
here.
This
is
AIA.
This
is
IntelliJ.
So
it's
a
different
idea.
The
s
quash
debug
container,
give
me
the
list
of
all
of
them.
I
will
take
the
second
one,
because
it's
the
Java
micro
services,
then
it
will
run
the
only
code
error
that
exists
there
and
then
I
will
attach
the
Java
debugger,
because
it's
reading
in
Java
okie-dokie.
A
So
we
have
two
microservice
running
in
the
cluster
different
language
and
then
the
only
thing
that
I
actually
need
to
do
another
to
debug.
Let's
go
back,
click
again
and
I
want
to
get
it
right
now,
debugging
in
life
right,
so
it's
running
on
AWS
I
can
see
I'm
getting
all
the
farmer
there.
It's
good
right.
What
will
happen
we'll
do
next?
Should
we
jump
to
the
other
one
right,
because
I
put
a
brakeman
there
so
now,
basically
I
write
them
here.
It's
great.
A
Let's
see
that's
actually
working
so
I'm
stepping
into
you
know
and
I
see
up
one
week.
Well,
forty
four
downstairs
it's
great
as
to
the
next
one:
thirty-three
that's
exactly
what
we
wrote
and
then
we
see
that
I
introduced
a
bug
here
is
the
back
that
I
introduce
right.
Basically,
we
see
that
other
parameter
is
goodbye.
A
What
I'm
doing
is
that
if
it's
true
I'm
actually
doing
a
minus
instead
of
class,
but
that's
simple:
this
is
actually
a
regular
debugger,
so
we
can
actually
own
life
just
change
the
value
to
false
right,
so
it
will
render
and
then
I
will
just
do
next.
What
will
happen?
It
will
jump
to
the
other
microservices.
B
B
B
A
A
Messed
up
here,
it's
probably
a
click
or
not
need
to
show
you
that
again
later,
I
guess:
okay,
nevermind!
You
trust
us!
It's
working!
It
seriously
does
I,
think
I
messed
it
up,
I
think
I
messed
it
up.
So
I
will
take
the
responsibility
here
and
your
legs
menu
how
it's
working
right
now
and
then
we'll
tied
it
up
and
I'm
sorry
I
hornier
them
once
a
complete
a
little
yet.
C
Right
so
really
quick.
How
is
squash
working
when
we
deploy
squash?
It
deploys
the
squash
server
and
then
also
deploys
squash
client
that
are
deployed
as
a
demon
set
to
your
kubernetes
cluster.
Now,
the
interaction
with
squash
so
is
through
a
vs
code
extension.
So,
just
like
you
just
saw
easier,
we'd
go
to
vs
code.
This
code
will
talk
to
the
squash
server
and
it
will
give
it
the
name
and
name
space
of
the
pod.
We
wish
to
debug.
The
school
server
will
find
the
relevant
node
and
contact
the
squash
client
ornated.
A
Yes.
So
now,
let's
talk
about
the
end
point
now,
where
everybody
so
that
suppose
she's
not
working.
Let's
talk
about
it,
so
servicemen
brain
squash.
So
this
is
service
fish
right,
we
all
know
about
it.
I
mean
I!
Think
that
to
talk
about
so
specifically,
the
integration
between
envoi
and
in
and
squash
she's,
using
each
have
to
use,
as
Sarah's
mentioned,
also
I
think
that
it's
really
not
responsible
for
us
to
the
word
and
we're
not
talking
about
service
much.
So
this
is
service
managed.
There
was
a
few
of
the
out
there.
A
The
beautiful
of
it
is
that
everybody
is
using
anyway,
so
here's
the
big
vision
of
us
right.
So,
let's
say
that
I
saw
that
there
is
a
lectin
see
somewhere
between
two
microservices.
That's
a
problem.
We
need
to
fix
it
and
why
we
want
to
do.
It
is
basically
zoom
in
right.
Zoom
in
with
two
debugger
see,
what's
going
on,
what's
causing
this
latency
and
that's
basically
the
big
vision
in
the
service
mesh.
A
Now,
in
order
to
do
that,
and
where
is
the
major
piece
that
we
need
to
deal
with
it
right,
that's
actually
the
most
important
one.
So
that's
what
we
did.
We
basically
created
a
filter
in
order
to
allow
us
to
do
that,
and
we
will
talk
about
one
second
about
the
flow
a
little
bit
about
the
filter.
We
do
one
all.
C
Right
so
we
wrote
an
invoice
extension
to
help
support
squash
in
a
service
mesh
context.
So
the
problem
with
this
context
is
two
problems:
one
we
don't
know
which
part
we
are
going
to
debug
and
the
other
is
that
we
wanna
debug
the
relevant
put
before
the
requests.
Hits
it
and
we'll
just
show
the
flow.
How
we
use
any
way
to
do
that.
So
first
thing:
it's
a
service
mesh,
so
you
have
NY
running
as
a
sidecar
configuration.
C
The
user
is
before
it
talks
with
the
ID,
but
now
it
doesn't
provide
ID
with
a
specific
pod,
but
rather
a
container
image
that
it
wants
to
do
back
and
that
contain
image
can
be
deployed
in
multiple
locations
across
the
cluster.
Then
the
user
simulator
request
and
appends
header
so
that
our
filter
will
know
to
engage.
The
request
goes
through
the
ingress
and
it
doesn't
matter
which
body
just
hits
any
of
them
gets
to
the
end.
We
filter
that
we
have
that
filter
sees
that.
C
It
has
a
certain
header
that
says:
oh
I
need
to
see
talk
with
a
score
server
and
see
if
I
need
to
debug
this
request.
It
talks
with
the
score
server,
which
coordinates
again
with
a
squash
client
and
with
the
IDE
see
if
this
is
indeed
the
image
that
the
user
is
waiting
to
debug
and
if
that's
the
case
facilitates
the
debug
session
for
the
users.
C
A
So,
basically,
it's
really
simple.
It's
very
simple!
Basically,
the
idea
is
that,
in
the
end
of
the
day,
the
request
is
coming
and
we
come
to
squash
and
said:
debug
me,
that's
really
simple,
right,
even
like,
if
I'm
trying
to
kind
of
like
up
it
up,
okay,
so
what
next?
Okay
we're
doing
this?
This
is
pretty
cool,
but
here
is
the
sake.
My
client
said,
as
is
right,
that
the
most
simple,
interesting
stuff
in
the
echo
system
right
now
wait
to
see
what
people
building
on
top
of
the
service
mesh
so
anyways
commodity.
A
This
is
the
way
that
everybody
would
learn
to
talk
as
a
proxy,
but
there
is
a
lot
of
service
match
and
we
don't
know
yet
what
will
happen
and
a
lot
of
people
will
use
a
difference
specifically
if
they're,
using
different
clouds,
for
instance,
you're
running
on
AWS,
you
most
likely
want
to
go
with
a
degree
less
up
mess,
because
it's
free
you're
running
on
Prem,
maybe
Sto
right.
So
you
definitely
wouldn't
want
to
do
that.
Okay,
so
we
agree
that
it's
everybody
is
using
a
tango
and
I.
A
Think
no
one
will
think
that
and
then,
on
top
of
it
there
is
some
service
mesh
and
they're
going
to
be
more
and
more
and
more,
and
the
thing
is
that
eventually
I
don't
want
to
build
integration,
which
is
quite
for
each
of
them.
That's
just
too
expensive,
very
dunking,
ready
and
also
we
can
actually
simplifies
the
service
quite
a
lot
service
much
by
Allah.
This
is
where
we
build
the
super
glue.
It's
also
an
open-source
project.
The
idea
with
that
is
basically
to
simplify
it.
For
us
it
was.
A
We
don't
want
to
integrate
it
with
every
match,
but
mainly
it's
an
abstraction
layer
to
make
sure
that
it's
very
easy.
So,
like
it's
easy
as
I
tell
me
which
mess
you
want
to
install
and
install
it
then
flip
a
coin.
You
want
observability
on/off,
you
want
retries,
on/off,
very,
very
simple
one,
so
you
have
that
and
then
the
beauty
of
it.
You
can
do
this
right.
We
can
actually
plug
stuff
in
so
this
is
a
summer
example
ingress
right.
That's
we
have
glue,
but
there
is
other
ingress
we're
really
hoping
that
people
were
integrated.
A
It
squash
so
integrated
once
to
superboots.
We
work
on
every
mesh
that
we
support,
which
is
most
of
them
and
then
same
thing
with
K
native
right.
There
was
a
little
bit
of
buzz
about.
Is
that,
okay,
that
they're
only
using
gas?
Do
they
will
need
to
abstract
that
and
we
can
help
with
that
and
I
think
tetra
is
building
something
on
top
of
the
mesh
and
so
on.
A
I
think
that
that
should
be
the
way
that
the
community
looking
and
the
big
the
end
goal
is
that
eventually
you
suck
the
meso
style
right
a
lot
of
node.
We
will
use
ever
all
of
this
as
a
computer.
A
lot
of
mesh
people
use
all
of
them
as
one
big
match,
so
I
can
just
say:
policy
boom.
It's
apply
to
every
mesh
I,
don't
really
care
each
right
so
yeah,
so
we
I
think
we're
on
time
on
fully
and
I'm,
really
sorry
that
didn't
work
everyone,
but
we're
open
source.
This
is
what's
always
doing.