►
From YouTube: eBPF & Gloo Mesh - What's New
Description
SoloCon 2022:
eBPF & Gloo Mesh - What's New
Speakers:
Lawrence Gadban
Field Engineer, Solo.io
Session Abstract:
eBPF has exploded in popularity recently, and for good reason, as this quickly maturing Linux technology has unlocked tons of potential in just about every aspect of Linux computing. When looking at eBPF from the lens of the service mesh, it is clear that eBPF can have a major impact given the core tenets of connectivity, observability, and security. In this session we will explore the possibilities of eBPF and service mesh as well as how Gloo Mesh is in a unique position to provide users with a battle-tested service mesh deployment that can be extended and augmented with the burgeoning power of eBPF.
Track:
Service Mesh and Application Networking
A
My
name
is
lawrence
gagbon,
I'm
a
field
engineer
here
at
solo.io,
and
this
is
a
summary
of
what
we'll
be
talking
about
today.
So
first
we'll
set
the
stage
kind
of
talk
about
what
ebpf
actually
is
and
what
it
can
be
useful
for
and
we'll
talk
about,
glue,
mesh
and
istio
and
where
they
fit
into
the
picture,
and
then
we'll
start
going
into
some
specific
topics
related
to
that.
A
So
we'll
talk
about
a
little
bit
about
mesh
and
seo
networking
then
we'll
talk
about
kubernetes
networking
and
then
we
will
focus
on
security
and
observability
so
to
get
started.
What
actually
is
ebpf,
so
it's
a
linux
technology
which
enables
users
to
run
custom
programs
in
the
kernel,
basically
in
their
own
sandbox
and
ebpf
itself,
stands
for
extended
berkeley,
packet,
filter
and
the
reason
it's
called.
That
is
because
it's
kind
of
an
extension
or
an
evolution
of
classic
bpf
classic
berkeley
packet
filter,
which
you've
probably
used
if
you
ever
have
used
tcp
dump.
A
A
A
Most
of
the
time,
at
least
in
modern
settings,
can
be
used
interchangeably.
So
at
least
in
this
on
these
slides
and
kind
of
going
forward.
If
you
see
evpf
or
ppf,
they
can
be
used
interchangeably,
going
back
to
the
actual
programs
and
technology.
The
programs
that
you
write
when
you
write
an
ebpf
program,
they're
event-based
programs,
and
they
can
be
attached
to
various
hook,
points
in
the
kernel,
and
so
that
means
that
your
program
will
essentially
fire
whenever
an
event
that
you've
you've
attached
to
is
triggered.
A
It
will
trigger
your
program
to
run,
and
so
one
example
of
that
is
something
that's
called
the
k-probe.
So
a
k-probe
program
is
something
that's
attached
to
a
kernel
function
and
that
can
be
attached
to
pretty
much
any
kernel
function.
There
are
a
few
that
it
can't
be
put
for
the
most
part.
Any
kernel
function
can
have
a
probe
program
attached
to
it
and
then
anytime,
that
function
is
called
your
program.
A
That's
attached
will
actually
be
executed
and
you'll
be
able
to
see
things
like
the
parameters
or
arguments
that
are
calling
that
were
used
in
calling
of
that
kernel
function.
Another
really
important
part
about
bpf
is
that
the
ebpf
and
dpf
programs
are
verified
to
be
safe.
So
when
you
load
a
program
into
the
kernel,
there
is
a
verifier
that
will
do
things
like
check
that
it's
guaranteed
to
return
or
doesn't
have
any
info
infinite
loops.
A
One
of
the
main
use
cases
is
networking,
so
remember
that
classic
epf,
which
kind
of
is
the
basis
for
ebpf,
was,
was
built
for
efficiently
filtering
packets,
so
again
kind
of
the
tcp
dump
example,
and
so
because
of
the
fact
that
it's
kind
of
it
comes
from
that
world.
It's
a
natural
fit
to
use
ebpf
to
do
various
things
in
the
networking
kind
of
space,
and
one
of
the
you
know
kind
of
say
it
another
way.
Basically
what
evp
does
and
it
allows
for
programmatic
processing
of
packets
at
different
locations.
A
That's
there
are
a
lot
of
different
locations
in
the
linux.
Networking
stack
that
you
can
attach
an
ebpf
program
to
and
we'll
kind
of,
look
at
this
in
in
more
depth.
Another
really
good
use
case
for
evpf
is
security,
so,
since
you're
operating
at
the
system
level
at
the
kernel
level,
it's
a
really
good
fit.
It's
a
really
good
place
to
do
things
like
enforce
security
policies
or
even
monitor
sensitive
operations,
because
you're
operating
essentially
at
the
kernel
level.
A
So
again,
we've
already
talked
a
couple
times
about
attaching
to
kernel
functions,
since
you
can
attach
to
pretty
much
anything
in
the
kernel,
it's
possible
to
extract
data
from
that
and
get
some
really
rich
and
low
level
data
and
metrics.
There
are
other
hook
points
other
than
kind
of
the
k
pro
that
we
talked
about.
A
So
another
important
part
is
that
ebpf
is
super
efficient
because
it's
operating
inside
of
the
kernel,
and
so
that
gives
you
the
opportunity
to
process
events
as
they
happen
in
the
kernel
which,
if
you're
operating
in
the
user
space
is
not
really
doable.
So
it
kind
of
has
unlocked
a
whole
world
of
observability
opportunities
with
evpf.
A
So
now,
let's
talk
about
istio,
so
istio
is
pretty
popular
service
mesh.
At
this
point,
it's
a
sidecar
based
service
mesh.
It
uses
onboard
proxy
as
a
data
plane
and
it
builds
on
top
of
that
and
really
you
know
if
you're
not
familiar
with
service
mesh,
it's
a
technology
that
intends
to
solve
several
of
the
technical
challenges
that
come
with
a
micro
services
architecture.
A
A
You
can
see
basically
a
sample
architecture
diagram
that
shows
that
you
know
you
have
your
services
deployed
and
alongside
that
is
a
proxy
that's
deployed
as
a
sidecar,
which
is
where
all
of
the
data
flows
through
and
then
you
have
a
control
plane
which
configures
those
proxies
based
on
the
state
of
the
cluster,
for
example,
so
glue
mesh
builds
on
top
of
that
blue
mesh
is
our
service
mesh
platform.
You
can
kind
of
think
of
it
as
an
application.
A
Networking
platform
that
builds
on
top
of
istio
and
kind
of
is
intended
to
provide
like
a
one-stop
shop
for
your
application
networking
concerns,
so
it
handles
your
service
mesh
using
istio.
We
also
can
do
api
gateway
functionality
with
istio
managed
by
glue
mesh.
It
also
provides
a
management
plane.
So
if
you
have
more
than
one
istio
installation
glue,
mesh
makes
that
easy
to
manage,
and
then
on
top
of
that
it
provides
kind
of
a
higher
level
api.
A
So,
let's
go
ahead
and
dig
into
mesh
networking,
specifically
istio,
we've
already
kind
of
seen
in
the
previous
diagram
how
the
how
this
is
sidecar
proxies,
which
are
envoy
kind
of
live
alongside
your
application,
and
so
those
network
proxies
handle
all
of
the
traffic
that's
flowing
to
and
from
the
application
container
and
the
traffic
that's
intended
to
either
go
to
the
application
application
container
or
leave.
The
application
container
is
transparently
redirected
to
that
proxy
and
that
redirection
occurs
with
iptables
rules
kind
of
the
the
standard
way
of
doing.
A
That
is
that
whenever,
when
every
pod
starts
up,
when
the
proxy
is
injected
to
that
pod,
there's
an
init
container
that
runs
that
will
create
some
ip
tables
rules
on
the
given
node
that
it's
running
on
to
handle
that
redirection
and
so
anything,
that's
really
intended
for
the
the
pod.
The
main
container
in
that
pod
will
be
redirected
to
the
envoy
proxy
and
the
same
would
apply
for
outbound
traffic
as
well,
so
the
traffic
would
get
redirected
to
the
proxy.
A
A
I
don't
call
it
magic,
but
there's
a
lot
of
iptables
configuration
that
happens
to
to
handle
all
of
the
inbound
and
outbound
redirection,
and
so
this
diagram
kind
of
goes
into
depth
of
you
know
all
of
the
different
places
where
iptables
is
making.
You
know
the
iptables
rules
are
being
used
to
determine
okay.
This
packet
actually
needs
to
be
redirected
to
go
to
envoy
and
there's
a
specific
port.
A
That
envoy
is
listening
on,
for
example,
for
the
inbound
traffic
to
the
application
container
and
there's
also
a
specific
port
for
outbound
traffic
from
the
application
container
and
those
ip
tables
rules
are
kind
of
controlling
to
make
sure
the
packets
go
where
they
need
to
go
so
it
works
great.
It
is
just
a
little
confusing
if
you're
not
familiar
with,
what's
going
on.
So
if
you
want
to
look
at
that
kind
of
like
zoomed
out
what
happens,
is
you
know
you
have
your
pet
store
application
in
this
case
and
you
have
your
istio
proxy.
A
So
if
some
traffic
were
to
leave
the
pet
store
app,
maybe
talking
to
another
service,
you
know
it
would
use
a
socket
and
it
would
send
out
a
packet
and
then
what
would
happen?
Is
the
network
stack
here
which
has
you
know
everything
in
the
linux
kernel,
that's
related
to
networking
ip
tables?
Would
the
ip
tables
would
come
into
play
and
that
packet
would
be
redirected
to
this
socket
here
on
the
envoy
proxy?
But
then
you
have
other
ip
tables
rules
that
say:
okay.
A
This
is
intended
for
you
know
this
destination,
really
that
needs
to
go
to
the
onboard
proxy
and
so
you're
kind
of
traversing
through
the
network
stack
on
both
cases
to
get
to
where
you
correctly
need
to
be
so,
as
I
mentioned,
it
works
well,
it's
battle
tested.
You
know
it
steals
running
production
and
has
been
for
years
now,
but
there
can
be
concerns
that
you
know
the
sidecar
injection
model
and
traffic
redirection
model.
A
You
know
you
are
doing
that
network
traversal
in
the
kernel
for
pretty
much
you're
doing
that
twice
for
every
flow,
and
so
there
are
ways
that
we
can
improve
that
and
that's
one
of
the
places
that
bbpf
can
help
so
istio
data
path.
Acceleration
is,
is
a
way
of
accomplishing
that.
So
again,
you
can
attach
a
bbpf
program
to
pretty
much
anywhere
in
the
network
stack.
You
know
there
are
very
specific
places,
but
there's
there's
a
network
location
for
a
bbpf
program
like,
for
example,
a
network
socket.
A
I
actually
want
to
redirect
that
directly
to
another
socket
so
you're
kind
of
keeping
a
map
of
from
this
socket
to
this
socket,
and
when
you
do
that,
you
can
skip
all
of
the
rest
of
the
traversal
that
we
were
looking
at.
So
you
can
point
directly
to
where
you
need
to
go.
So
what
that
kind
of
looks
like
is
again.
We
have
this
application,
the
pet
store,
and
we
have
our
istio
proxy.
A
We
have
the
sockets
used
for
communication.
So
in
this
example,
right
is
steel.
Proxy
is
receiving
communication
from
the
pet
store,
because
the
pet
store
is
sending
out
a
sending
out
packets
and
so
for
these
sockets.
What
we
can
do
is
we
can
have
glue
mesh
help
and
attach
an
evpf
program
to
each
one
of
those
sockets
and
then
what
happens
is
since
we're
keeping
a
map,
and
we
can
do
this
programmatically,
so
we
can
say
okay
in
a
given
pod.
You
know
this
is
the
application
container.
A
This
is
the
envoy
proxy,
the
sidecar
proxy.
What
I
want
to
do
is
that
whenever
a
packet
is
sent
to
this
socket
the
real
destination
is
this
socket
over
here
for
envoy
and
that
that
happens
by
the
programmatically?
We
can
keep
that
map
up
to
date
and
we
can
skip
all
of
the
ip
tables
magic
that
needed
to
happen
to
make
that
happen.
A
So
we'll
be
able
to
do
that
with
blue
mesh,
we're
doing
things
like
benchmarking
and
confirming
that
it's
ready
for
production,
etc.
We
have
seen
improvements
to
throughput
and
latency,
so
there's
more
to
come
on
that
there
is
a
caveat
here,
I'm
linking
to
a
blog
article
that
we
published
at
solo.
We
don't
think
that
envoy
is
going
anywhere
this.
This
functionality
is
really
about
shortening
the
path,
basically
to
go
from
application
container
to
the
sidecar,
but
we
still
think
that
envoy
as
a
sidecar
is
here
to
stay.
A
So,
if
you're
interested
in
that,
I
highly
recommend
checking
out
this
blog
okay.
So
let's
talk
quickly
about
kubernetes
networking,
so
kubernetes
networking.
What
we're
going
to
do
is
we'll
take
a
kind
of
a
this.
This
is
kind
of
foundational
for
some
of
the
other
topics,
so
kubernetes
networking
itself.
It
relies
on
an
underlying
network
plug-in
to
implement
cluster
networking
and
cluster
networking
is,
for
example,
the
pod
pod
communication.
A
A
That's
called
eat,
zero
in
a
specific
container
right
and
another
really
interesting
thing
about
the
cni
spec
is
that
it
allows
for
chaining,
so
you
can
have
a
chain
of
cni
plugins,
so
you
so,
like
we
just
said
the
example
is
create
network
interface
et
0
and
container
bar,
so
every
plugin
will
get
to
operate
at
that
time.
So,
if
we're
doing
the
creation
of
the
network
interface,
one
of
the
cni
plug-ins
will
will
operate
on
it
and
create.
Maybe
it
creates
the
interface.
A
Then
the
next
plugin
will
be
able
to
modify
that
interface
or
add
some
additional
functionality.
You
can
kind
of
think
of
it
as
like
composition
of
different
networking
features
and
then
lastly,
I
call
out
that
the
container
runtime
interface
or
the
cri
is
what
actually
delegates
the
control
to
the
cni.
A
So
the
cri
is
responsible
for
doing
things
like
creating
the
containers
or
creating
like
the
linux
name,
spaces
that
define
a
pod
boundary,
but
when
it
gets
to
actually
creating,
for
example,
the
network
interfaces
that
your
containers
would
use.
That
is
the
job
of
the
cni
and
the
cri
delegates
that
to
the
cni
so
kind
of
seeing
that
in
action.
A
We
have
a
note
here,
kubernetes
node,
you
know
it
has
some
network
interface
that
allows
the
actual
node
to
make
network
requests,
but
we
also
have
the
cubelet
and
then
the
cri
and
the
cni,
and
so
when
we
do
some
when
we
have
to
do
something
like
create
a
pod,
so
the
cubelet
is
trying
to
create
this
pod.
That's
named
bar
what
it
will
do
is
delegate
to
the
cri
to
actually
create
the
containers.
You
know
create
the
the
pod
boundary
whatever
it
needs
to
do.
A
So,
in
this
case,
we
have
one
container
we'll
create
that
container
the
cri
plug-in
will
and
then
the
cni
gets
called,
and
that
is
where
the
the
networking
interfaces
in
that
container
get
created.
So
now
the
cni
plug-in
is
going
to
actually
create,
for
example,
eat
zero
interface,
that's
available
to
this
bar
container
and
it's
kind
of
in
the
network
namespace
of
this
bar
pod
and
then
again
with
chain
plugins.
During
that
step,
where
we're,
for
example,
creating
this
e0
interface,
all
of
the
plugins
in
the
chain
get
to
do
some
work.
A
They
get
to
operate,
for
example,
on
the
interface
and
then
once
that's
done,
we
have
the
pod
created.
We
have
the
container
created,
we
have
the
networking
interface
created,
and
now
the
pod
and
application
container
inside
of
the
pod
can
operate
as
usual.
You
know
as
normal
right
so
to
the
applications
container.
It's
making
a
network
request.
Maybe
it's
calling
another
pod
in
the
cluster
and
the
basically,
the
routing
is
set
up
to
where
that
request
can
leave,
and
then
traffic
can
come
back.
A
So
this
is
kind
of
a
zoomed
out
view.
Basically,
if
we
have,
you
know,
let's
say
we
have
three
pods:
we
have
a
pet
store,
foo
and
a
bar
pod.
You
know
the
the
foo
in
the
bar
pod
can
talk
to
pet
store
and
they're
kind
of
going
out
their
own
network
interface
and
then
on
the
pet
store
side,
the
server
side,
it's
going
to
the
pet
store
application,
and
it's
going
through
the
network
interface
on
the
pet
store
side.
A
A
Istio
has
a
very
powerful
set
of
features
related
to
security,
kind
of
one
of
the
core
concepts
in
istio
security.
That
was
the
concept
of
the
service
identity,
and
that
means
that
that
that
basically
is
how
each
workload
instance
in
the
service
mesh
has
a
cryptographically
verified
identity
and
that's
done
using
x,
509
certificates.
A
This
pod
is
what
is
used
as
the
basis
for
the
kubernetes
istio
identity
and
then
the
way
that
this
works
is
that
the
istio
agent,
which
is
another
process
running
in
that
pod,
will
will
submit
a
csr
or
certificate
signing
request
to
the
control
plane
and
depending
on
what
ca
or
certificate
authority.
The
control
plane
is
set
up
to
use
that
ca
will
then
sign
the
csr,
and
that
is
then
results
in
a
signed
certificate
and
then
the
private
key
is
available
to
the
proxy
and
that's
kind
of
the
basis
for
the
identity.
A
That
means
that
for
every
request,
basically,
as
long
as
it's
using
mutual
tls,
you
will
have
a
client
side
and
a
server-side,
well-known
identity.
That's
cryptographically,
secure,
so
kind
of
just
to
solidify
that
say
we
have
a
deployment
which
is
the
pet
store
deployment.
It's
running
in
the
default
namespace
and
it
uses
a
service
account
named
petstore.
So
here's
the
service
account
the
end
result
of
that
csr
kind
of
flow.
The
provision
certificate
flow
is
we
get
a
certificate
and
key
that
correspond
to
this
identity.
A
Cluster.Local
is
a
trust
domain.
We
won't
go
into
details
and
let's
talk
about
that,
but
really
it
just
identifies
the
mesh
more
or
less.
That
is
a
part
of
so
in
this
example,
the
identity
would
be
cluster.local,
ns
default
for
default,
namespace
and
then
slash
sa
pet
store
for
the
pet
store
service
account
so
authentication
we
already
kind
of
touched
on.
A
Authorization
is
built
on
top
of
that,
where
authorization
allows
you
to
do,
there's
essentially
an
authorization
engine
that
runs
in
the
envoy
proxy
in
the
istio
proxy
and
when
you're
using
mutual
tls.
Since
you
have
the
client
side
identity,
you
can
also
write
operation
rules
based
on
the
identity
of
the
service.
Talking
to
you
and
then
on
top
of
that,
since
envoy
works
at
layer
7,
you
have
access
to
all
of
the
http
request
attributes.
A
So
the
end
result
is
that
this
example
here
on
the
right
gives
you
a
authorization
policy
that
will
say
that,
for
the
pet
store
application,
I
will
accept
requests
from
the
cluster.local
default
foo
identity
from
that
client
I
will
accept
requests
that
are
gets
to
slash
bar,
and
so
this
is
a
really
powerful
feature
that
allows
you
to
write,
really
complex
authorization
rules
and
again,
that's
all
enforced
in
proxy.
You
can
kind
of
do
that.
You
basically
get
it
for
free
when
you
use
istio,
so
glue.
A
Mesh
kind
of
again
builds
on
top
of
that
by
providing
a
single
api
called
the
access
policy
that
controls
both
the
authorization
and
authentication
configs,
and
that
it
also
can
do
that
on
a
multi-cluster
level.
So
when
you
define
an
access
policy,
you'll
define
in
this
example,
we
only
have
authorization
but
we're
doing
it
based
on
the
service
account
just
like
the
istio
identity
is
based
on,
and
then
this
can
be
applied
to
say
multiple
clusters
in
your
mesh.
A
A
A
A
So,
ultimately,
istio
is
not
a
network
firewall
and
it
never
wasn't
intended
to
be
so
even
the
istio
best
practices,
security
documentation
calls
out
that
you
know
to
further
secure
your
mesh.
It's
probably
worth
you
know
basically
exercising
defense
in
depth,
and
you
can
do
so,
for
example,
using
a
network
policy.
A
The
identity
for
this
policy
is
based
on
the
name,
space
or
pod
labels
that
you
want
to
either
allow
or
deny
traffic
to.
So
in
this
example
here
on
the
right,
we're
saying
that
for
pet
store
application,
I'm
going
to
accept
traffic
from
a
pod
that
has
a
label
of
access
is
true,
and
so
what
that
would
look
like
is
going
back
to
our
kind
of
three
pod
setup.
We
have
foo
bar
and
pet
store.
Foo
has
a
label
of
access?
A
A
When
you're
talking
about
network
security,
we
can
do
things
like
this
example
is
allowing
to
the
pet
store
only
a
pod
that
has
a
service
account
that
is
essentially
labeled
with
account.
Foo,
then
that
that
client
is
allowed
to
talk
to
pet
store,
but
anything
else
is
not,
and
so
what
that
means
is
that
the
fupod,
which
is
running
with
a
service
account
of
foo,
that's
allowed
at
the
network
level,
and
it's
also
allowed
at
the
istio
or
service
mesh
level.
A
A
A
On
top
of
that,
istio
has
its
own
metrics
that
are
generated
in
proxy
and
that
kind
of
provides
service
level
style
metrics.
So
you
can
think
of
like
what
client
is
talking
to
what
service
you
can
get
metrics
for
that
from
istio
and
those
are
exported
by
prometheus
as
prometheus
metrics
by
default.
And
so,
when
you
combine
the
envoy
metrics
as
well
as
the
istiometrics,
you
get
a
really
rich
set
of
data
to
help,
monitor
and
understand
your
mesh.
A
So
this
is
just
an
example
of
kind
of
like
this
is
the
istio
dashboard,
and
you
can
see
that
this
is
specifically
for
a
service.
That's
the
detail
service
and
you
can
see
in
this
screenshot.
You
know
the
type
different
types
of
data
you
get.
You
can
think
of
it
as
like
kind
of
the
golden
signals.
That's
intended
to
be
supplied
out
of
the
box
when
using
istio
and
envoy
metrics,
so
on
top
of
that
glue,
mesh,
again
kind
of
builds
on
top
of
it
and
what
glue
mesh
can
do.
A
Since
you
have
that
management
plane,
it
can
help
with
the
aggregation
of
the
metrics
and
statistics
for
all
of
the
meshes
that
make
up
your
your
you
know,
virtual
mesh
or
whatever
you
want
to
think
of
as
the
various
components
that
make
up
your
mesh
glue
mesh
can
help
with
the
aggregation
of
that
and
then
on
top
of
that,
since
it's
aggregating
from
different
clusters,
we
can
augment
that
data
with
cluster
level
data.
So
you
can,
you
can
know
which
metric
is
coming
from
which
cluster
and
then
the
end
result
of
that
is.
A
We
can
do
things
like
our
service
graph,
which
allows
you
to
do.
You
know
basically
track
your
service
flow,
your
network
graph,
but
it
also
is
cluster
aware.
So
in
this
example
right
we
have
cluster
one
and
cluster
two,
and
so
you
can
see
the
traffic
flowing
across
both
of
the
clusters
in
this
mesh.
A
That
being
said,
you
know
there's
a
lot
of
data
there,
but
there
are
a
few
gaps
so
again
kind
of.
As
we
talked
about
in
security,
it
all
relies
on
the
proxy
being
there.
So
if
you
have
workloads
that
say
have
not
been
onboarded
to
the
mesh,
you
don't
have
any
of
this
data
because
you
don't
have
a
proxy
living
alongside
the
application.
A
You
also
are
limited
to
what
envoy
and
istio
gather
as
far
as
metrics
go.
So,
as
I
was
saying,
there's
a
bunch
of
data
there,
but
one
example
of
where
it
might
fall
short
is
that
envoy
by
default
captures
to
the
endpoint
level.
So
if
you
have
a
service,
that's
usually
made
up
of
different
pods
right,
different
replicas,
that's
considered
an
endpoint
and
there's
a
lot
of
really
good
endpoint
level
statistics
in
envoy,
but
because
of
the
cardinality
concerns
when
you're
using
prometheus.
A
That
is
not
exported
by
default
or
even
exported
at
all
for
prometheus
style
metrics.
So
if
you
need
very
low
level
endpoint
per
endpoint
level
data,
you
can't
get
that
through
prometheus
metrics
with
envoy
and
then
additionally,
you
may
have
environmental
issues
that
are
kind
of
even
lower
level
than
envoy
right.
They
may
be
node
level
or
so
on,
and
you
can't
get
that
through
envoy
or
istio.
A
So
this
is
where
evpf
can
help
out.
Ebpf
is
a
natural
fit
for
tracing
observability.
For
you
know,
several
of
the
reasons
that
we
talked
about
already.
I
do
have
a
link
here
to
performance
analysis,
superpowers
with
linux
evpf
by
brennan
gregg.
It's
a
fantastic
talk.
I
recommend
you
check
it
out.
It
goes
into
way
more
detail
than
I
can
here,
but
really
since
bbpf
is
working
at
the
kernel
level,
you
can
get
a
ton
of
low-level
data
on
pretty
much
anything.
That's
happening
on
your
system,
and
that
is
done
at
the
kernel.
A
So
you
don't
need
a
proxy.
You
don't
need.
You
know
you
don't
need
to
have
a
sidecar
running.
You
can
get
that
directly
from
the
kernel,
and
so
it's
a
really
nice
fit
for
kind
of
getting
data
from
you
know
if
from
workloads
that
don't
have
a
proxy
or
if
you
do
have
a
proxy
and
you
need
some
specific
custom
data,
you
can
use
bbpf
to
do
that
as
well,
and
so,
when
you
combine
istio
and
envoy
and
bbpf
powered
observability,
you
get
a
really
really
powerful
set
of
observability
data.
A
So
to
kind
of
look
at
that,
you
know
again.
In
this
example,
we
have
the
three
pods
in
this
case,
they're
running
on
two
different
nodes,
and
only
the
pet
store
in
this
example
has
the
istio
proxy
injected.
Only
this
is
the
part
of
the
service
mesh.
The
foo
and
the
bar
pods
are
not
they're.
Still,
they
haven't
been
onboarded
yet,
and
so,
when
we're
using
a
prometheus
instance
to
scrape
some
data,
you
know
from
from
the
pet
store
app.
We
can
go
ahead
and
get
the
app
data.
A
We
can
go
and
get
istio
and
envoy
metrics.
So
we
have
a
pretty
good
set
of
information
that
we
need
from
this
workload,
but
then
the
foo
and
bar
pods.
We
can
only
get
application
level
metrics,
so
you
you
know
is
that
enough
data
for
troubleshooting,
you
know,
maybe
bbpf-
can
help
to
get
more
a
richer
set
of
data
for
a
richer
set
of
metrics
from
these
applications.
A
So
what
we
can
do
is
using
glue
mesh
and
then
we
also
have
another
open
source
project.
We
have.
We
have
a
talk
at
solocon
as
well
on
bumblebee.
Really.
What
that's
intending
to
do
is
provide
some
a
nice
workflow
for
for
observability
based
bbpf
programs,
and
it
makes
it
to
where
you
can
deploy
those
programs
from
any
oci
registry,
so
glue
match
can
utilize
that
kind
of
functionality
and
what
we
can
do
is
say
we
have
a.
A
We
have
an
ebpf
program
that
can
get
some
network
level
statistics
from
the
node,
regardless,
if
you
have
an
istio
proxy
or
not
glue
mesh
can
kind
of
pull
that
program
as
an
oci
image
from
your
from
your
registry,
you
know
where
you're,
storing
your
other
container
images
it
can
deploy
that
to
the
nodes
in
the
cluster
and
then
now
we
can
scrape
the
istio
proxy
for
the
the
for
the
pet
store
which
is
injected
with
istio
proxy.
So
we
can
get
the
at
istio
and
onboard
metrics
from
pet
store.
A
We
can
get
the
app
metrics
from
foo
and
bar
which
are
not,
which
don't
have
istio
proxy,
and
then
we
can
scrape
the
evpf
programs
to
get
the
really
low
level
metrics
that,
for
example,
for
flu
and
bar
there
they
don't
have
any
network
statistics.
We
can
get
those
from
the
node
from
our
ebpf
program
and
then
we
can
also
get
you
know
some
custom
metrics.
Maybe
we
need
some
higher
fidelity
data
about
the
pet
store.
A
A
Okay,
so
to
summarize,
evpf
is
a
very
exciting
technology.
It
unlocks
a
lot
of
really
cool
stuff
that
hasn't
really
been
possible
and
also
allows
us
to
improve
a
lot
of
things
and
glue
mesh
intends
to
be
able
to
leverage
edpf
and
do
so
in
a
way
that
you
can
adopt
it.
You
know
you
can
adapt
with
what
you
need
without
having
to
rely
on
specific
implementations.
Bluemesh
can
help
kind
of
manage
that
as
you
need
it
also
with
the
speed
that
this
area
is
changing,
you
know.