►
From YouTube: Debugging with eBPF on Arm Platforms - Leo Yan, Linaro
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Debugging with eBPF on Arm Platforms - Leo Yan, Linaro
This session will discuss the in-kernel virtual machine to extract information from a running kernel based on eBPF. Firstly this session will quickly give brief introduction for eBPF working mechanism so that allow developers to gather information for debug and tuning by attaching eBPF programs using kprobes or tracepoints and is reinforced with a roll-your-own example program that illustrates how userspace can exploit these mechanisms. After that we move from theory to practice by looking at several userspace tools that allow developers to write high level eBPF programs (or run other peoples pre-canned ones) and process the resulting data.
https://sched.co/Nrtw
A
B
A
B
A
Now
we
applied
it
to
arm
platform
and
what
are
the
challenges
and
what
are
the
potentials
that
we
can
deploy
and
make
three
years
of
it,
and
we
can
give
you
several
quick
examples,
only
20
to
35
minutes.
This
is
very
tight,
so
we
have
to
I'm
afraid
I
had
to
skip
the
last
two
with
examples.
We
can
see
the
graph
to
the
right.
The
dotted
line
above
that
line
is
fly,
BCC
and
other
tools
and
the
other
role
buildings
below.
A
We
have
lots
of
the
kernel
machines
and
these
machines
are
all
running
codes
from
EBP
F
and
we
have
to
check
transfer
these
codes
to
almost
64
and
x86.
We
have
to
do
encoding
into
local
codes
for
operation
of
processing
and,
starting
from
here
the
chapter
1
will
be
about
EP
PF
general
introduction.
What
is
a
PDF,
and
currently
you
can
see
this
is
a
very
complicated
topic
and
lot
of
details.
A
If
you
want
to
go
in
and
if
we,
if
you
used
TCP
dump,
then
you
probably
have
really
used
it
and
TCP
dump
beach
and
BPF
hackett
ceasing
to
the
right.
We
can
see
very
shocked
and
if
you
have
really
into
debugging
for
the
basic
hardware
or
drivers,
then
you
will
be
very
familiar
about
EPB
F
and
we
can
also
catch
data
from
network
and
EPB
F.
We
can
see
that
a
lot
of
bad
codes
shown
on
the
windows
to
the
right
and
B
PPF.
A
It's
an
expansion
and
classic
EPF
mainly
focused
on
the
very
fires,
so
we
improve
the
performance
of
EPF
on
EP
PF
greatly,
and
previously
we
have
really
focused
on
the
dapeng
of
online
patch
packet
and
we
have
managed
to
expand
it
to
have
trays
and
the
other
platforms.
Mainly
now
we
focus
on
the
performance,
debugging
not
only
focused
on
network
and
also
we
expand
the
struts,
and
we
hope
that
we
can
do
lots
of
them.
A
Data
transit
between
host
and
your
clients,
for
example,
high
C,
and
we
have
a
specific
order,
which
is
for
functional,
functional
core,
which
is
uncle,
and
if
you
have
some
functional
or
in
CPU
design,
orders,
then
et
PF
can
extract
the
other
functions
from
the
kernel
and
you
run
a
code.
Then
that
is
available
for
functional
calls
in
kernel
and
we
can
further
expand
its
functions,
a
PPF.
A
Some
of
you
might
not
be
very
familiar
about
that,
but
this
is
a
very
good
example.
This
overall
data
flow
and
structure
to
the
left
you
can
see.
This
is
your
space
to
problems
which
is
through
currency
and
food,
/
user
dot,
C
one
is
for
Colonel.
One
is
for
clients
for
the
colonel
one.
We
have
codes
user,
we
have
data
and
we
will
form
a
target
document
for
each
EPF.
The
yellow
arrow.
The
blue
arrow
is
where
the
data
flow
and
we
can
see
in
the
kernel
there's
a
bad
code
in
the
PPF.
A
This
is
one
of
the
command
lines
from
EPB
F
and
we
have
to
transfer
it
into
a
function
either
it's
x86
or
I'm
64
and
the
code
it
had
to
be
triggered.
They
cannot
run
on
themselves,
for
example,
trace
point,
keep
prop
or
have
trace,
I
have
to
trigger
the
order,
and
the
code
then
will
collect
the
data
and
ready
yeah
I'll
red
arrow.
A
We
can
see
the
epp,
F
map
and
transfer
data
to
read
data
maps
so
internally
the
struct
is
pretty
complicated,
but
we
have
to
go
through
that
quickly
and
if
you
have,
code
will
have
to
be
compared
with
C
library
and
you
have
to
use
VM
Kong
while
we're
not
using
GCC,
because
GCC
will
not
support
the
encoding
of
e
PBF.
So
it
will
be
very
complicated
whether
it's
kernel
or
use
case
Colonel.
B
A
B
A
A
We
can
now.
Also.
We
cannot
let
the
code
lead
to
a
deadlock
in
the
system
and
all
the
visiting
of
the
data
will
have
to
be
written
and
then
read
so
we
have
to
guarantee
the
circle
functioning
very
well.
I
need
to
be
of
cold
first
expand
to
the
network
and
later
tracepoint
make
a
problem.
The
other
platforms,
any
PDF
will
also
need
to
support
map
which
is
customized
and
kernel
with
that.
That
will
be
a
generic
structure
for
days
of
transition
and
a
PDF.
A
The
methodology
of
debugging
is
different
from
EPS
and
let's
see
this
graphic,
if
you're
using
a
stress,
then
you
will
know
without
a
PDF.
If
you
are
doing
debugging,
then
you
have
to
do
a
lot
of
trace,
lock,
transfer
it
from
kernel
to
customer
side
and
the
trace
data
is
in
large
quantity.
We
cannot
do
this
manually.
If
you
wonder
and
analyzed
it,
we
have
to
run
it
offline.
B
B
A
B
A
Pf,
thus
library
and
be
very
convenient
for
you
to
write
a
problem
and
will
provide
you
with
hyperfunction,
including
how
can
you
transform
the
code
of
EVPs
from
customer
to
colonel
side?
And
how
can
you
manipulate
the
map
structure
and,
for
example,
we
provide
functions
such
as
EPF,
slash
notes,
function
and
for
the
map
have
the
other
functions
like
that
you
had
to
schedule
or
you
have
to
call
this
function.
If
you
want
to
write
a
code
for
kernel
and
if
you
are
customer,
then
you
have
to
do
systems
both
in
this
function.
A
A
A
If
both
sides
are
using,
this
struct
then
have
to
you,
then
the
customers
will
have
to
use
the
describers
to
describe
the
function
and
basically,
this
wolf
function
has
a
value
of
the
command
line
and
we
transform
it
into
the
kernel
and
for
Apple
era,
customers,
it's
transparent
and
in
the
kernel
we
modified
this
map
command-line
somehow
and
upon
loading.
This
function,
and
still
we
have
a
attached
K
probe
and
we
have
an
trays
trays
pointing
to
the
right
key
prop
event
will
be
opened.
A
Finally,
we
have
to
put
the
describe
of
tress
point
and
PMD
this
driver.
We
have
to
combine
it
together,
so
k
prop
event,
and
this
program
have
to
be
integrated
and
bound
it
together
and
if
we
triggered
a
prop
event
and
then
we
will
use
this
code
to
run
it.
So,
finally,
we
are
writing
or
imposing
value
into
this
code.
So
basically,
this
is
a
very
simple
process
of
UPF
codon.
We
will
not
actually
write
any
conclusion
code
in
real
world
and
so.
A
We
had
to
focused
on
the
task
case
of
the
sample
and
with
PPF,
not
Darcy,
and
it
will
help
you
to
create
a
map
loading.
The
data
from
customer
to
Colonel
and
K
prop
point
had
to
combine
with
the
program.
This
is
all
done
in
this
process.
In
this
program
we
have
two
to
stutters,
one
is
Colonel,
one
is
customers,
and
if
we're
using
this
method,
so
the
meaning
is
only
two
lines.
First,
we
have
to
install
some
headers
documented,
also
LLVM.
A
Changing
parameter
and
to
the
left:
this
is
how
we
run
it
in
kernel
and
on
the
upper
side.
This
is
a
virtual
structure
and
first
this
is
the
definition
and
second,
like
I,
said:
name
vpf
library.
Compared
with
the
BPS
library,
now
see,
we
have
to
analyze
all
these
sections
and
make
preparations
for
you.
For
example,
this
is
a
map
and
we
have
to
assign
a
map,
assign
a
structure
and
Cape.
A
Here
we
use
the
free
node,
3
node
will
be
added
and
every
time
we
release
this
node
and
we
were
cleared
the
value
in
the
math
and
they
would
know
that
as
release.
But
it
will
be
noted
as
a
located
but
not
released,
and
we
read
the
map
data
in
client,
side
and
we'll
know
how
many
node
is
allocated
and
for
how
long
and
it
hasn't
been
released.
Yet
it
will
marked
clearly
in
the
system
and
so
plant.
This
is
live,
waited
way
of
using
it.
A
We
provided
a
walk,
simple,
simple
descript
distributors
for
a
trace
point.
They
probe,
and
the
most
convenient
way
is
that
we
are
now
based
on
now,
IBM
and
clang.
We
have
a
buildin
compiler
and
based
on
this
buildin
controller,
we
can
do
I
run
isolated
orders
into
the
right.
It
is
a
very
simple
order,
and
this
is
what
happened
when
we
have
a
cyst
code
and
if
the
value
is.
A
A
Even
if
you
want
to
signed
a
memory
to
the
whole
functional
stagnant,
you
can
still
use
that
the
creators
of
the
Vantage
apply
kisti.
You
can
use
them
embedded
systems,
the
dependency
on
other
systems,
pretty
light
pretty
low,
and
first
day
you
had
to
combined
with
certain
and
probe
and
had
the
filter
determine
on
what
kind
of
events
they
were
capturing.
What
are
not,
and
we.
A
A
A
We
want
to
make
it
easier
for
you
to
write
a
code
in
particles
and
easier
compelling
the
internal,
a
PDF
code,
and
you
have
to
using
the
client.
You
have
to
bound
it
with
other
codes
and
we
have
a
script
and
that
the
script
will
do
it
for
you
and
if
there's
a
mistake,
it
will
be
easier
for
you
to
find
the
mistakes
and
ok
the
mistakes
and
to
the
right.
You
can
see
the
pc,
the
flan.
A
A
A
B
Actually,
a
very
complicated
that
you
can
see
the
whole
process
because
it
lies
a
lot
of
issues,
including
the
dependent
it
I'm
being
busy.
So
while
I'm
trying
this
I
have
actually
spent
a
lot
of
time-
and
this
is
some
conclusion
easily
is
own
compilation.
It
demands
a
lot
of
elaborate,
including
the
semi
kind
of
labor
and.
B
Because
we
have
a
lot
of
perversions
for
this,
so
sometimes
you
cannot
use
the
distribution.
The
installed
version,
so
you
have
to
have
our
new
population
lives,
so
Rav,
Emma
and
CC.
You
need
to
spend
a
lot
of
time
on
it
and
it
has
a
higher
current
score
system,
because
our
platform,
you
can
see
that
one
TDI
is
one
GB
storage
it,
but
for
1.0
completion
is
not
enough.
That's
why
I
need
to
switch
to
a
larger,
and
it
has
high
requirements
called
the
fire
system
like
the
PC.
B
B
A
B
B
A
B
A
B
A
B
B
Is
the
internal
integration
spirit
complicated
s
based
obesity
and
BM
BM,
so
it
has
its
own
high-level
language
and
you
need
to
turn
it
into
SD.
Yes,
the
abstract
once
so,
you
have
to
get
the
petticoat,
so
the
biggest
challenge
is
that
it's
very
flexible
it
can
expand
on
its
own
language,
and
it's
code
is
very
strong
so
because
it's
based
on
PCC
and.
B
Built
it's
very
complicated.
It's
based
on
the
business
environment.
You
need
to
add
there
be
the
code,
so
if
it
doesn't
have
the
PCC,
yes,
the
be
easy.
You
have
to
perform
all
the
tasks
for
P
fixing
them
to
do
this,
but
in
this
debian
environment,
if
we
have
the
lip
every
city
have
a
death.
Then
you
have
this
environment.
You
just
need
to
compile
the
PP
of
choice,
and
this
is
perfect.
B
B
B
Perfect
race,
this
main
function
is
that
we
use
extras.
We
need
to
schedule
all
the
parameters
in
the
assistant
who
need
to
extract
all
the
parameters
for
the
viewers
to
see
it.
But
the
biggest
good
point
is
that
it
doesn't
cost
too
much
for
the
system,
because
you
just
need
to
extract
all
this
data
and
for
the
perfect
it
can
do
the
profiling
for
the
program.
B
B
Fifty
PPF,
you
have
a
new
version,
for
this
is
the
aerial
vm8
in
the
2004
channel,
so
the
biggest
benefit
is
that
is
like
to
Annette
annotation
annotation
of
the
system
and
to
see
how
many
percentage
of
there's
a
program
you've
either
supports
PDF.
You
can
also
it's
show
the
source
code
of
the
dot
sync
file.
B
B
B
B
So
you
can
find
the
memory
leak
issues
in
the
system
very
conveniently.
The
second
one
is
based
on
PCC,
because
your
system
has
different
modules
and
they
have
this
relationship
of
scheduling.
If
you
want
to
schedule
our
program,
we
need
to
use
the
predicate,
but
in
this
one
you
tell
us,
basically
we
don't
need
to
add
the
some
scheduling
codes
for
our
server.
You
cannot
just
report
it
at
the
end
again
and
also
for
the
remote
client
and
for
them
to
schedule.
B
B
B
What's
the
rate
value
of
it
is
and
for
this
single
command
we
can
see
this
name
of
the
function
and
the
plus
the
character
string,
and
what
kind
of
the
argument
you
need
to
print
out?
Yes,
you
can
print
it
out
very
quickly
and
we
can
still
see
the
later
part
of
this
the
function.
Yes,
this
is
a
later
part
of
it.
Whether
the
parameter
is
the
correct.