►
From YouTube: Think eBPF for Kernel Security Monitoring- Falco at Apple- Eric Sage & Melissa Kilby, Apple
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Think eBPF for Kernel Security Monitoring- Falco at Apple- Eric Sage & Melissa Kilby, Apple
In this talk, Apple infrastructure engineers talk about why they’re excited about eBPF, and how Apple is using eBPF to supercharge its security monitoring. Eric will highlight options for better eBPF introspection and benchmarking tools and Melissa invites you to the world of cyber security featuring a hands-on Falco capability demonstration for runtime security insights & detections. Join us to learn what possibilities exist to evolve eBPF and Falco.
A
Good
afternoon,
everyone
and
welcome
to
falco
at
apple,
I'm
melissa,
kilby
from
the
apple
services,
security,
engineering
team,
no
hybrid
conference
without
a
hybrid
format.
We're
already
good
at
this,
so
I'll,
be
virtually
joined
by
eric
sage.
A
member
of
the
linux
kernel
team
at
apple
today,
we're
going
to
show
how
we
deployed
falco
as
one
of
our
security
monitoring
solutions.
A
B
B
Ebpf
programs,
on
the
other
hand,
are
limited
to
a
set
of
curated
function,
calls
called
ebpf
helper
functions
depending
upon
the
epf
program
type
ebpf
programs
allow
us
to
categorize
the
types
of
programs
loaded
into
the
kernel
so
that
each
program
be
given
only
the
data
and
functionality
needs
to
complete
its
task.
The
new
falco
ebpf
probe,
for
example,
is
specifically
a
tracing
program
and
only
has
access
to
tracing
helper
functions
and
data.
B
B
For
example,
there
are
many
packet
processing
frameworks
designed
to
circumvent
the
cost
of
the
kernel
network
stack
to
process
packets
at
line
rate.
Ebpf
programs
alleviate
the
need
for
these
drastic
solutions
by
allowing
user
space
to
provide
different
types
of
epf
programs
to
solve
specific
problems
and
gives
the
eppf
code
access
to
very
low
level.
Processing
pipelines
such
as
tying
directly
into
a
network
driver
before
the
network
stack,
is
invoked
in
the
case
of
xdp
programs.
B
B
The
linux
kernel
team
at
apple
is
always
improving.
Our
testing
and
deployment
strategy.
Ebpf
has
added,
compile
once
run
anywhere,
which
allows
us
to
write
ebpf
programs
which
can
be
targeted
to
many
different
kernel
versions.
This
is
a
drastic
improvement
over
kernel
modules,
which
must
be
compiled
and
tested
against
specific
kernel
versions,
compile
once
from
anywhere
uses,
btf
type
information
to
adjust.
Epf
programs
on
the
fly
to
work
on
a
variety
of
kernel.
B
Which
is
a
feature
that
cannot
be
oversold.
Many
of
the
solutions
that
eppf
replaces
do
not
have
direct
support
from
the
normal
kernel
development
channels
and
must
work
to
maintain
compatibility
with
the
direction
of
kernel
development.
Having
ebpf
be
a
part
of
the
kernel
means
having
the
peace
of
mind.
The
eppf
will
always
be
supported
by
the
primary
kernel
developers.
B
B
A
Thanks
eric
now
we
dive
into
how
an
organization
can
leverage
falco
to
supercharge
its
runtime
security
monitoring.
We
will
cover
a
lot
of
content
very
fast,
such
as
insights,
detections
or
data
modeling
implications,
and
we
are
in
la
this
year.
Hollywood
magic
happens
just
around
the
corner,
so
there
will
be
a
magical
offensive
security
demo
with
special
effects,
and
then
we
show
how
you
can
use
file
code
to
detect
such
attacker.
A
One
could
say
that
system
calls
hold
the
potential
to
capture
entire
classes
of
vulnerabilities
and
attacker
behavior,
often
chained
together
to
a
nice
skill
chain.
If
there
was
one
takeaway
message,
then
it
would
be
syscalls
never
lie.
Consider
this
your
open
invitation
to
run
off
and
make
it
a
false
statement
when
a
system
call
is
made,
it
is
the
metadata
around
the
system
call
most
notably
the
process,
execution
path,
command,
args
or
file,
descriptor
name
that
is
essential
to
determine
if
the
observed
behavior
is
unusual
or
even
malicious
for
your
environment.
A
Here's
a
list
of
the
most
important
system
calls
from
a
security
monitoring
perspective.
They
are
important
because
they
cover
behavior
observed
in
many
exploitations.
Falca
rules
are
written
in
a
basic
expression
language
how
it
works.
The
rules
specifically
built
for
linux,
kubernetes
or
cloud
native
are
instantly
evaluated
on
the
host
falco
ships
were
standard
rules
created
by
the
community
that
you
can
easily
build
upon.
A
Falco
can
also
lock
container
events
or
has
a
kubernetes
integration.
Today,
however,
we
continue
to
focus
on
system
calls.
Only
a
subset
of
falco
rules
falls
into
the
category
of
very
specific
security
detections.
For
instance,
ssh
outbound
connections
could
channel
a
shell
over
non-standard
ports.
Perhaps
configurations
are
such
that
this
should
never
happen
in
this
case,
such
a
rule
can
also
serve
as
assurance.
If
the
alert
fires
we
can
say
with
high
confidence
that
this
is
unusual,
a
larger
subset
of
falco
rules
falls
into
the
category
of
less
specific.
A
They
are
derived
from
high-level
constructs
that
are
known
to
be
security,
critical,
for
example,
spawning
a
shell
by
a
non-shell
application
or
modifying
files
below
a
set
of
system
standard
binary
directories.
These
type
of
falco
rules
generate
a
whole
bunch
of
signals,
and
it
is
hard
to
map
them
to
attacker
behavior.
A
A
It's
now
time
for
a
story
worth
sharing.
Once
upon
a
time
there
was
a
web
application.
What
can
possibly
go
wrong
over
the
next
couple
of
slides?
We
will
show
some
of
the
ways
you
can
exploit
the
three
top
three
consistently
exploited,
vulnerabilities
and
misconfigurations,
that
is
arbitrary
file,
read
bugs
remote
code,
execution
and
misconfigurations,
leading
to
privileged
escalation.
A
A
Interesting
here
is
that
we
can
use
such
an
lfi
to
trigger
an
arbitrary
file.
Read
we
just
successfully
read
the
world
treatable
at
the
password
file.
More
interesting
is
how
do
we
get
remote
code
execution
out
of
this?
The
way
we
can
achieve
this
is
to
upload
a
file
we
create
over
the
lfi
and
then
invoke
that
file
using
the
local
file
include
vulnerability
again.
A
In
this
example,
application
there
was
an
authenticated
endpoint
that
processes
uploaded
files.
So
we
simply
include
this
admin
upload
functionality.php
file
with
the
lfi
in
order
to
upload
our
script
to
the
web,
app
that's
sneaky,
and
we
also
know
where
that
file
ends
up
after
setting
up
the
listener
on
the
attacker
machine,
we
make
another
request
to
include
the
just
uploaded
file
with
the
lfi
and
that
way
invoke
the
payloads.
Now
we
have
remote
code
execution
on
the
box,
I
now
switch
roles
in
m.a
defense,
magician
one
by
one.
A
A
A
Falca
also
has
a
generic
rule
that
detects.
If
a
package
management
process
is
run
inside
a
container,
this
demo
made
it
look
very
easy
to
accurately
detect
all
malicious
artifacts.
In
reality,
systems
are
very
noisy
and
falco
logs.
Yes,
they
can
drive
you
crazy.
This
is
why
we're
now
going
to
dive
deeper
into
some
of
the
challenges
you
may
face,
once
you're
ready
to
get
even
more
value
out
of
falcon.
A
The
application
just
read
the
file
names
at
the
password
with
a
technique
called
directory.
Traversal
etsy
password
is
quite
literally
read
all
the
time
on
linux
systems,
so
generically
modifying
monitoring
this
file
would
not
be
very
helpful,
but
this
particular
artifact
is
pretty
suspicious
and
indeed
could
indicate
that
the
app
is
vulnerable
to
an
arbitrary
file
read
bug.
A
But
what
happened
to
the
falco
logs
here?
The
directory
traversal
signal
is
not
in
the
file
descriptor
name.
This
will
make
a
good
and
bad
etsy
password
read,
look
exactly
the
same
some
source
and
make
file
modifications
are
necessary
to
unlock
the
entire
debugging
engine.
So
we
start
our
debugging
journey
in
kernel.
Space
bpf
print
k
will
show
that
the
file
descriptor
name
is
captured
as
expected
following
the
control
flow
in
user,
space
will
show
that
the
fd
path
is
being
sanitized.
Modifying
this
sanitization
function
can
fix
this.
A
Ironically
user
input,
sanitization
function
or,
let's
say
better
lag
thereof-
are
often
the
root
cause
for
such
arbitrary
file.
Read
bugs
we're
trying
to
detect
here
next
making
sense
out
of
command
arcs
or
file.
Descriptor
names
is
the
tricky
part
that
can
drive
you
crazy.
This
is
because
one
you
rely
too
much
on
prior
assumptions
to
match
for
two
specific
strings
and
are
at
risk
of
missing,
potentially
malicious,
behavior
or
two.
You
have
a
hard
time
tuning
your
rule
and
constantly
play
catch
up
with
filtering
out
benign
activities
specific
to
your
environment.
A
The
hard
part
really
is
that
shell
input
is
touring
complete,
combined
with
the
inherit
variability
in
exploits.
There
is
an
unknown
amount
of
semantically
related
process.
Behavior
that
will
look
very
different
in
all
of
these
system
called
metadata
to
make
matters
worse.
Many
benign
commands
contain
exotic
elements,
think
of
hashes
ips
timestamps
that
can
make
normalization
and
profiling
a
daunting
task
in
and
of
itself.
A
A
A
A
Popular
tools
are
big
data
technologies
such
as
presto
spark
or
elasticsearch.
As
far
as
methodologies
are
concerned,
there
are
endless
ways:
one
simple
approach
could
be
to
simply
group
and
aggregate.
The
signals
grouping
could
be
done
on
a
per
container
basis
or
for
a
particular
subprocess
tree
when
performing
subprocess
tree
analysis.
There
are
a
few
tricks
that
can
help
reduce
computational
burden.
A
One
word
of
caution
here
again,
making
too
many
prior
assumptions
regarding
how
correlated
signals
are
expected
to
look
like
can
be
tricky
in
reality.
We
just
don't
know
things
always
look
very
logical
in
retrospect
and
puzzling
and
prospect-
and
this
is
especially
true
in
security
personally,
a
big
fan
of
first
focusing
on
high
value
single
events,
one
at
a
time
in
line
with
what
we
have
shown
up
to
this
point
before
performing
correlations.
A
A
A
A
One
projection
could
be
that
more
high
volume
kernel
signals
could
be
discovered.
This
is
also
in
line
with
what
we
heard
earlier
this
morning
that
perhaps
the
way
we
do
tracing
at
the
moment,
where
falco
is
not
what
we
should
be
doing
in
the
future,
and
this
also
includes
new
ways
of
deriving
high
value
kernel
signals
also
enhanced
encoding
of
the
enriched
metadata
around
process
collection,
which
would
address
all
the
challenges
we
touched
on
earlier,
when
making
sense
out
of
command
arcs
or
file
descriptor
names.
A
Finally,
advanced
pre-filters
in
the
kernel
could
be
in
enticing
proposition
advanced
here
could
mean
that
the
semantics
or
intent
of
a
process
is
encoded
as
a
state
vector
for
more
dynamic
ai
ml.
Like
reasoning
right
in
the
kernel-
and
I
am
aware,
we
are
very
very
far
away
from
this-
but
remember
a
lot
can
happen
between
now
and
never.
A
Wait:
what,
if
I
didn't
pay
any
attention
at
all,
it
happens
start
with
a
very
simple
rules
created
by
the
community,
customize
falco
and
falco
rules
to
suit
your
environment
and
threat
model.
Better
then
slowly
venture
out
eventually
into
deep
space
to
start
getting
even
more
value
out
of
the
tool
and
for
which
sophisticated
data.
Modeling
approaches
can
be
of
help,
perhaps
in
the
future,
we'll
be
given
the
opportunity
to
dive
even
deeper
into
modeling
implications.
For
now,
thank
you
for
having
us
here
and
big
shout
out
to
the
falco
developers
and
the
falco
community.