►
From YouTube: Lightning Talk: BTFGen: one Step Closer to Truly Po... Mauricio Vásquez Bernal & Rafael David Tinoco
Description
Lightning Talk: BTFGen: one Step Closer to Truly Portable eBPF Programs - Mauricio Vásquez Bernal, Microsoft & Rafael David Tinoco, Aqua Security
Many cloud native projects started using eBPF to provide OS and application observability, networking and security. Kubernetes deployments run on many different kernel versions and currently there is a big challenge on portability, as the eBPF programs depend heavily on the kernel version. BPF CO-RE (Compile Once – Run Everywhere) is a mechanism to solve this issue. It requires the kernel to expose information about its types by using BTF (BPF Type Format), which is not always available. BTFHub aims to solve that by providing BTF files for released kernels that don't support BTF. However, it’s not possible to ship the BTF information for many different kernels with the application because of size limitations. BTFGen generates very small BTF files that can be shipped with the application, making it easier to run eBPF programs in different kernel versions. Mauricio will introduce the challenges of creating portable eBPF cloud native ready applications and how BTFGen helps to solve those challenges. The talk covers BTFGen implementation, in bpftool, and discusses our experience in integrating it to the open-source eBPF powered projects Inspektor Gadget and Tracee.
A
This
is
this
has
traditionally
been
solved
by
compiling
the
program
at
each
target
machine.
This
is
what
is
done
by
bcc
and
the
python
tools
were
mentioned
before.
But,
of
course,
this
is
not
an
ideal
solution,
because
in
order
to
compile
the
tools,
we
need
a
compiler
in
the
target
machine,
so
it
means
that
we
have
to
spend
some
hundreds
of
megabytes
of
extra
storage
there.
Also
compiling
the
programs
is
a
heavy
process.
It
means
that
we
are
running
on
a
very
tight
schedule.
A
So
the
few
events
that
we
are
getting,
the
the
first
information
that
we
are
getting
from
the
ebpa
program
could
take
a
bit
longer
to
arrive,
and
the
last
problem
is
a
very
annoying
one
is
that
in
order
to
compile
those
programs,
we
need
the
linux
kernel
heaters,
but
those
heaters
are
now
available
on
many
different
distributions.
So
we
we
have
to
tell
the
user
to
install
an
additional
packages
to
have
the
kernel,
the
linux
kernel,
headers
available
and
so
on.
A
A
The
idea
in
this
case
is
that,
instead
of
shipping
source
code,
we
ship
the
compiled
bpf
programs,
but
in
order
to
make
those
programs
to
work
in
different
machines,
what
we
have
is
a
mechanism
that
allows
us
to
update
the
offset
of
the
structures
of
the
member
of
the
structure
that
those
programs
access
around
time.
So
this
is
like
a
patching
mechanism,
so
we
relocate
the
program
before
injecting
the
program
into
the
kernel.
A
The
whole
technology
behind
this
is
btf
is
a
stands
for
btf
type
format
that
this
is
a
representation
of
the
kernel
data.
So,
as
we
can
see
in
the
picture,
we
have
the
compiled
bpf
program.
We
have
some
btf
information
that
is
exposed
by
the
kernel
and
then
the
loader
library
takes
both
of
them
together
and
patches
the
program
according
to
the
kernel
that
we
are
running
so
more
information
about
btf.
So,
as
I
mentioned
before,
this
is
a
metadata
type
that
describes
the
different
kernel
types.
A
A
As
you
can
see
here,
we
have
information
that
there
is
a
task
structure.
We
have
information
about
the
size
of
the
structure,
all
the
members
and
on
the
bottom
we
can
see
that
there
is
a
pid
member
and
we
can
see
the
offset.
This
is
the
actual
information
that
the
loader
library
uses
to
perform
the
relocation
of
the
program,
but
everything
is
so
good
so
far,
but
there
is
a
problem
in
the
problems.
Btf
is
now
available
on
all
the
kernels.
A
This
is
only
available
in
some
releases
and
when
this
compilation
flag
is
enabled,
this
is
the
reason
why
bta
hub
was
created.
So
this
is
a
project
that
exposes
btf
information
for
different
distributions,
that
don't
support
btf,
so
bta
hub
takes
that
information
from
different
debug
packages
that
are
provided
by
different
distributions,
performs
on
data
processing
and
exposes
those
btf
files
from
the
different
kernel
versions
of
different
distributions.
A
We
can
see
there,
for
instance,
for
ubuntu.
We
have,
for
the
version
204
a
lot
of
different
kernels
for
different
btf
files
for
different
kernel
versions.
So
this
is
how
we
can
use
core
together
with
bta
hub,
so
the
loader
library,
the
compiler
bpa
program,
is
the
same.
What
is
changing
here
is
that
we
are
not
taking
the
btf
information
from
the
kernel,
but
we
are
taking
that
information
from
bthub.
So
what
we
usually
do
is
that,
before
loading
our
bpf
program
into
the
kernel,
we
go
to
a
site.
A
A
Again,
there
are
some
limitations
with
this
approach.
Each
btf
file
is
around
five
megabytes,
so
what
it
means
is
that
this
is
not
possible
to
cheap
all
the
btf
files
for
so
many
different
kernels.
Together
with
our
application,
it
will
require
some
gigabytes
of
data
and,
of
course,
this
is
not
possible.
So
the
alternative
is
to
download
the
file
from
the
current
kernel
from
the
internet,
but,
of
course,
taking
a
file
from
the
internet
requires
some
time
and
even
worse.
A
It
turns
out
that,
in
order
to
perform
the
relocations,
we
don't
need
to
have
the
full
information
of
all
the
kernel
types.
We
only
need
information
of
the
kernel
types
that
our
ebpf
program
is
using
so
by
using
btfg.
What
we
are
able
to
do
is
to
take
a
file,
a
bt
file
describing
all
the
kernel
types
and
we
are
able
to
generate
a
small
file
with
only
the
kernel
file,
the
types
that
are
used
by
our
program.
A
What
it
means
is
that
we
can
generate
various
more
apt
files
and
we
can
ship
those
together
with
our
program.
So
this
is
how
btf
gen
works.
In
general,
we
have
different
btf
object,
files,
bts
programs.
We
have
btf
files
for
different
kernels.
Those
could
be
from
bt
hub
or
of
different
source.
Please
notice
that
each
of
them
is
around
five
megabytes
big.
Then
we
put
that
all
together
into
btf
gen,
and
then
we
generate
small
files
for
each
kernel.
A
We
did
an
integration
of
bts
with
the
bcc
tools
and
we
found
out
that
in
order
to
support
something
like
1000
kernels,
we
only
need
like
something
like
100
kilobytes
per
application.
So
this
is
a
there
is
a
really
small
overhead
there.
So
again,
everything
is
the
same.
The
difference
here
is
that
the
btf
information
is
not
on.
The
internet
is
not
provided
by
the
operating
system,
but
in
this
case
we
ship
the
btf
information
for
many
different
kernels,
together
with
our
application
so
yeah.
A
According
to
the
running
linux
distribution,
the
kernel
version,
we
take
the
right
file
and
then
we
provide
that
file
to
our
loader
library,
so
yeah.
I
know
this
is
a
lining
talk.
It
was
very
fast
a
lot
of
details
that
probably
were
not
able
to
cache,
but
the
conclusion
that
I
would
like
you
to
take
from
this
is
that
if
you
are
developing
an
ebpf
project,
if
you
are
using
ebpf-
and
you
want
to
cheap
those
programs
to
run
those
programs
in
many
different
distributions,
you
could
use
btf
gen
in
able
to
do
so.
A
There
is
a
small
size
overhead
that
we
have
to
pay,
but
is
really
small
and,
of
course,
if
you
want
more
more
information,
please
go
to
our
blog
post,
where
we
provide
the
full
details
about
how
we
implemented
btf
gen
about
how
can
you
use
btf
gen
in
your
a
project
and
so
on?
And
of
course,
if
you
have
any
questions
please
reach
out,
I
will
be
also
on
his
luck.
Replying
the
question
for
the
people
that
is
watching
the
screen,
and
before
finishing
I
would
like
to
thank
you,
the
people
that
made
this
possible.
A
This
was
a
joint
project
by
different
people
from
different
companies,
especially
to
rafael
from
aqua
security.
This
is
the
guy
behind
the
bts
project.
He
helped
a
lot
with
implementation
of
this
tool,
also
lorenzo
fontana,
leonardo
donato
from
elastic,
provide
valuable
feedback
and,
finally,
to
the
maintainers
of
bpf
tool
that
make
this
possible
to
have
the
support
available
there
and
that's
it.
Thank
you.