►
Description
Preventing Attacks at Scale [I] - Dino Dai Zovi, Capsule8
Security hardening for containers, clusters, and operating systems is a very important part of setting up infrastructure and always "Plan A". The world of "Plan A" defends the importance of making sure your cluster is set up securly. Dino comes from the world of "Plan B" and will focus on detecting when security boundaries have been breached. This is necessary for environments where you don't have ability to ensure base OS is fully patched, etc.
Step into the world of Linux kernel features such as seccomp, eBPF, kprobes and Kubernetes tunable security features and learn how to detect and defend against attacks at scale.
About Dino Dai Zovi
Dino Dai Zovi is the Co-Founder and CTO at Capsule8. Dino is also a regular speaker at information security conferences having presented his independent research at conferences around the world including DEF CON, Black Hat, and CanSecWest. He is a co-author of the books "The iOS Hacker's Handbook" (Wiley, 2012), "The Mac Hacker's Handbook" (Wiley, 2009) and "The Art of Software Security Testing" (Addison-Wesley, 2006). He is best known in the information security community for winning the first PWN2OWN contest at CanSecWest 2007.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
A
Whoever
basically
gave
all
that
to
so
I've
done
a
good
amount
of
this
stuff,
but
I
wanted
to
talk
about
how
to
scale
be
defend
against
attacks
as
you
grow
your
infrastructure
and
especially
in
kubernetes
environments,
because
what
we're
seeing
kind
of
a
in
a
lot
of
places
in
industry
is
that
we're
not
no
longer
being
held
back
by
the
amount
of
infrastructure
time
we
have
available
and
we're
just
able
to
scale
the
number
of
nodes,
basically
with
a
click.
Click
click
on
the
cloud
and
you're
done.
A
A
And
stop
breaches
as
being
kind
of
a
regular
part
of
daily
life,
and
the
problem
with
this
is
I
think
it's
just
you
know
fundamentally
trying
to
scale
the
wrong
way
and
so
saying
yeah.
We
need
10
gazillion
people
to
manually
look
through
alerts
to
manually.
Do
all
this
stuff
to
manually
audit.
All
these
all
this
code
to
manually
find
every
vulnerability
to
manually
patch
every
vulnerability.
It's
you
know
if
you've
ever
written
software,
which
hopefully
most
of
us,
have
you
realize
that
maybe
that's
not
the
best
way
to
go
about
it
and
security.
A
It
goes
without
much
explanation
that
that
is
a
pretty
antiquated
way
of
looking
at
things
now
and
sort
of
what
illustrates
this
I
think
the
most
clearly
our
statistics
of
number
of
servers
per
employee
at
different
types
of
firms,
and
so
you
know
the
average
company
it's
less
than
one
to
one
but
clearly
Facebook
and
Google
or
if
the
scale
much
more
significantly
through
that,
and
how
did
they
do
that?
Well,
they
did
it
by
treating
operations
like
a
software
problem,
and
this
pattern
is
what
goofed,
what
we
now
call
the
cloud.
A
So
that's
that's
how
I,
how
I
view
it
and
it's
basically
just
another
instance
of
software
eating
the
world.
So
first
software
a
tops
and
now
it's
eating
security,
and
it's
actually
gonna
be
really
interesting,
because
it's
actually
gonna
really
change
a
lot
of
how
how
I
typically
did
things
and
how
people
I
know
did
things
but
I
think
it's
for
the
better.
So
what
do
I
want
people
to
take
away
from
this
talk?
I
want
people
to
think
about
sort
of.
A
How
do
we
apply
a
lot
of
the
things
we've
learned
from
the
asari
and
DevOps
transformations
to
security?
So
kind
of
my
thought
exercise
for
a
lot
of
this
is
taking.
You
know,
if--
style
I
saw
that
you
know
office
was
very
similar
to
where
security
is
kind
of
still
today
and
what
what
what
the
principles
that
they
learned
and
which
of
those
apply.
So
we
can
actually
start
from
something
that
works.
A
A
You
help
secure
it,
and
security
can
only
scale
with
shared
responsibility
and
the
more
we
can
push
to
self-service
to
application
developers
and
infrastructure
teams.
The
more
this
scales
better
and
it
actually
doesn't
require
a
PhD
in
the
defense
against
dark
arts.
It's
all
very
simple
stuff
and
wanting
to
talk
about
today
is
how
to
build
a
continuous
security
pipeline
in
cloud
native
environments
and
understand
which
principles
of
SRA
and
DevOps
apply
to
security.
So
starting
with
a
little
background.
A
So,
if
you
have
a
root,
that's
just
you
only
wrote
the
system
can
detect
it
and
it's
pretty
cool
I
also
looked
at
how
Google
did
a
lot
of
their
security
monitoring
with
data
mining,
one
of
the
takeaways
that
they
learned
that
I
think
are
really
useful.
Is
that
purely
statistical
like
machine
learning
approaches
had
very
poor
results,
and
one
of
the
reasons?
Why
is
because
in
machine
learning,
a
80
percent,
true
positive
rate
is
actually
pretty
good.
A
I
got
so
that's
an
academic
result
in
security,
20
percent,
that's
equivalent
to
a
20
percent,
false
positive
rate,
so
in
security,
20
percent,
false
positives
is
really
bad,
and
what
that
also
means
is,
if
you,
as
you
scale
more
like
the
workload
and
the
number
of
servers,
that
an
amount
of
false
positives
basically
becomes
a
just
a
data
problem
beyond
the
human
attention.
You
have
now
just
saturated
your
servers
and
you
the
entire
pipeline.
So
that's
some
of
experience
that
they
had.
A
That
I
think
is
pretty
useful
to
look
to
also
the
most
recently
Netflix
had
a
presentation
about
how
they
approached
security
monitoring
and
they
are
using
e
BPF,
which,
if
you
haven't,
started
looking
at
eb
PF.
It
is
really
awesome,
but
it's
also
really
complicated
and
requires
a
really
new
kernel.
I'll
talk
a
little
more
about
that
in
a
little
bit,
but
I
think
some
of
their
requirements
were
a
pretty
illustrative
one.
They
needed
the
system
to
be
event-driven.
You
can't
pull
around
the
network
and
be
like
hey.
A
Are
you
hacks
now
nope
I'll
come
back
in
ten
minutes?
How
about
now
you
hacked
now
how
about
now
and
then
you
know,
attackers
it's
pretty
easy
to
come
in,
come
out
and
be
done
because
you
know
in
the
desktop
world
kind
of
a
this
like
when
you're
thinking
about
security
of
especially
advanced
attacks.
You
might
watch
an
attacker
for
60
days
before
you
begin
to
evict
them.
A
How
many
of
you
want
to
let
an
attacker
roam
around
your
production
environment
for
60
days
I
mean
you
should
probably
looking
for
a
new
job
like
after
like
day,
one
I
think
60
minutes
is
too
long.
I
think
60
seconds
is
too
long,
because
it's
very
easy
to
get
a
lot
of
data
out
fast.
So
we
have
to
really
change
the
cadence
here,
and
so
that's
why
it
must
be
event
driven.
Also,
the
system
must
be
lightweight
when
you
have
a
lot
of
nodes.
A
You
can
run
in
the
kernel
on
Linux,
trace
events
and
I'll
talk
a
little
more
about
that
a
little
bit,
but
at
the
broadest
scale.
What
we're
looking
at
to
do
to
secure
our
systems
is
apply
the
five
factors
to
secure
systems
that
I'm
ripping
off
from
Mugu,
and
basically
you
start
with
one
response:
make
sure
that
you
like
be
able
to
know
about
a
threat
and
respond
to
it
effectively
seconds
make
sure.
A
There
were
significant,
there's
a
good
balance
there
between
that
and
performance,
and
also
focus
on
containment
that
you
can
limit
the
impact
so
there's
typically
a
spectrum
between
prevention
and
response,
and,
like
kind
of
the
bottom
three
or
the
more
prophetic,
the
more
prevention,
the
hardening
you
set
up,
your
but
your
environment,
you
basically
make
sure
you've
configured
it
correctly,
and
then
you
walk
away
right
and
you
can't
have
just
that.
It's
sort
of
like
okay,
we
built
a
good
bank
vault
we
put
in
the
Middle
Central
Park
and
we've
done
our
job.
A
It
should
be
fine
and
will
does
not
look
at
it.
A
whole
security
program
requires
monitoring
your
response
because
we
know
there
are
known
unknowns
like
we
know
there
are
new
vulnerabilities.
We
know
there
are
new
attacks,
and
so
we
need
to
actually
be
on
the
lookout
for
for
when
those
happens
so
that
we
can
actually
respond
effectively
and
the
cool
thing
about
online
services
environments
is
that
the
attacker
is
on
your
turf.
So
you
know
we
think
about
kind
of
a
game
of
attack
versus
defense.
A
The
attacker
gets
to
choose
the
timing
and
the
method,
whereas
the
defender
gets
to
prepare
the
battlefield,
so
you
can
carve
a
nice
little
Valley.
That
looks
really
tempting
and
you
can
wait
for
them
to
walk
down
it
and
just
watch
them.
That's
your
that's
your
prerogative
as
the
defender,
and
so
what
I
like
doing
is
you
know
I
like
playing
chess
I
like
thinking
about
strategy.
Well,
that's
how
I
do
that's,
how
I
do
defense
to
I
basically
set
traps.
A
I,
basically,
look
make
really
easy
avenues
that
I
can
watch
really
effectively
and
respond
really
quickly
to
because
the
attackers
no
way
of
knowing
that
and
that's
where
I
can
turn
a
information
asymmetry
to
my
advantage.
So
that's
that's.
What
I
think
why
I
think
kind
of
a
response
game
is
a
lot
more
fun
and
I'll?
Hopefully
it'll
kind
of
get
you
all
interested
in
it
as
well.
But
then
we
look
at
prevention,
make
sure
you're
patching
and
all
this
other
stuff,
because
don't
rely
on
a
safety
net.
A
If
you
don't
have
to
and
elimination
like,
can
you
innovate
your
way
out
of
an
entire
class
of
vulnerability
and
I
think
this
is
a
really
good
way
of
breaking
down
kind
of
this
process
sort
of
like
the
12
fact
for
stateless
apps.
So
what
should
we
call?
This
thing
like
this
is
like
actually
a
common
joke
ever
heard
in
the
last
talk
as
well.
Do
we
call
this
sec
DevOps?
So
we
call
it
dev
sec,
ops,
how
about
DevOps
sec?
It's
like
hey!
A
I
can
permute,
we've
all
been
in
programming
interviews,
or
maybe
we
call
it
seconds
what
I
think
of
this,
as
is
continuous
security,
because
the
real
technology
that
people
are
getting
out
of
the
cloud
is
continuous
delivery
and
when
you
start
think
about
everything
in
this
continuous
cycle,
you
start
thinking
about
Big
Data
differently.
You
think
about
fast
data,
not
Big
Data.
You
start
thinking
about
how
your
architecture
systems
differently,
and
you
start
thinking
about
security
differently
as
well,
and
so
that's
why
I
like
emphasizing
continuous
security,
and
so
what
is
that?
A
What
does
that
mean
to
me?
It's
a
software
driven
pipeline
to
securing
systems,
so
we
act
like
we're
benchmarking,
a
piece
of
infrastructure
or
software,
do
macro
benchmarks
and
then
optimize
and
optimize
that
optimize,
so
that
you
know
you're,
applying
your
defenses
in
the
right
or
your
effort
in
the
right
place
and
it
doesn't
require
having
a
dedicated
security
team.
So
I'm
gonna
show
you
a
really
really
simple
architecture.
A
Vet
is
basically
a
blog
post
and
it
should
look
a
lot
like
the
sort
of
case
studies
that
you'll
see
from
like
the
cloud
provider
on
how
to
build
a
streaming.
Iot
analysis
pipeline.
It's
pretty
much
the
same
thing
so,
but
in
order
to
kind
of
guide
how
we're
gonna
think
about
this,
let's
work
backwards
again,
ripping
off
or
under
of
all
tools.
So,
let's
talk
start
start
talking
about
server
breaches,
so
how
many
of
you
have
actually
breached
a
server
got
in
remote
access
all
right?
A
Well,
that's
I
lost
that
bet,
that's
great,
and
so
for
those
of
you
that
haven't
in
haven't
done
penetration
testing
and
things
like
that
I
think
it's
really
to
see
what
it's
like.
So
you
can
know
why
you're
defending
against,
because
they
pretty
quickly
fall
into
a
lot
of
common
patterns,
but
I'm
going
to
talk
only
about
kubernetes
environments,
there's
a
lot
of
ways
to
do
stuff,
but
kubernetes
is
the
new
shiny.
So
let's
talk
about
that
and
what
do
we
care
about?
A
The
most
we
care
about
remote
commands
execution
vulnerabilities
that
give
people
actually
breached
the
server,
and
these
are
vulnerabilities
that
were
you
know
how
about
some
high
profile
ones
in
the
last
few
years,
included
shell
shock
image,
tragic,
a
huge
number
of
ones
in
an
Apache,
but
notably
a
vulnerability
in
struts
that
affected
Equifax
and
the
entire
class
of
Java
deserialization
vulnerabilities
that
some
people
called
mad
gadget.
But
the
common
theme
of
all
of
these
is
they're
executing
a
shell
and
then
you're
like
wait
a
minute.
A
That's
I,
don't
really
run
shells
in
my
containers
that
often
now
you
have
a
signal
that
you
can.
You
know
hone
in
on
there's
other
things
you
want
to
might
want
to
look
at
like
people
SS
aging
into
a
shell
or
doing
some
other.
It's
alright
stitching
into
a
container
for
to
fix
something
in
production.
Not
this
isn't
always
malicious,
but
it's
probably
something
you
at
least
want
to
keep
your
eye
on,
and
if
people
are
doing
this,
it's
probably
a
signal
that
you
need
it
easier
way
to
do
their
job.
A
So
how
I
like
doing
this
is
I
like
building
monitoring
and
then
you'll,
see
insecure
practices
and
you'd,
say
hey
all
right
now,
I
know
the
use
case,
let's
figure
out
something
that
meets
that
use
case
and
is
safe
and
have
people
do
that
and
it's
sort
of
like
the
wisdom
of
the
iPod.
You
know
people
were
downloading
IP
mp3s
and
people
were
you
know
they.
People
clearly
wanted
a
digital
music
experience.
A
They
wanted
insta
gratification
of
grabbing
a
song
and
listening
to
it
on
the
go,
and
you
could
either
fight
it
and
call
it
piracy
or
you
could
build
the
largest
music
store
in
the
world.
I
think
we
know
which
one
actually
is
more
effective.
So
the
list,
let's
kind
of
go
over
like
a
generic
data
breach
scenario.
So
first,
when
an
attacker
is
going
to
do,
is
they're
either
gonna
scan
your
infrastructure
in
particular,
or
they're
gonna
scan
the
entire
Internet.
It's
really
cheap
to
scan
the
entire
Internet.
A
So
even
though
we
all
think
we're
special
snowflakes,
like
don't
always
think,
every
attack
is
actually
about
you,
it
might
just
be
opportunistic
and
then
they
might
come
back
and
see.
You
know
what
you
learn,
which
hosts
are
actually
interesting,
but
it's
pretty
inexpensive
to
just
scan
the
internet
for
types
of
these
types
of
vulnerabilities
and
actually,
when
I
set
up
our
cluster
and
I
add
you
know,
set
up
ingress
on
AWS
the
first
time.
A
A
Finally,
these
vulnerabilities
able
to
get
a
shell
in
a
container
and
what
I
find
interesting
is
a
lot
of
attention
is
paid
on
securing
the
container
or
securing
the
host
like
the
colonel
in
that
environment
from
the
container,
but
not
as
much
about
the
entire
cluster,
and
we
think
about
the
cluster.
Like
that's
what
matters.
An
individual,
no
you'll
burn
it
down
like
we
burned
ours
down
within
24
hours.
It's
just
gone
next,
so
compromising
a
node
is
not
all
that
interesting,
but
compromise
the
entire
cluster
and
persisting
that
way
is
way
more
interesting.
A
We
think
about
what
an
attacker
actually
is
trying
to
do
so.
We're
talking
a
couple
ways
that
are
trivial
to
compromise,
criminales
clusters
and
so
I
don't
feel
bad
talking
about
them
because
they're
not
vulnerabilities,
it's
just
there's
no
security.
There
they're
just
like
they
didn't
try.
This,
it's
not
there
so
I
consider
it
a
a
obligation
to
make
sure
people
know
the
limitations
and
make
their
own
risk
trade-off,
but
yeah,
and
then
they
establish
persistence
and
move
laterally
within
that
clusters,
basically
how
it's
gonna
work.
A
So,
let's
start
with
talking
about
shell
shock
shell
shock,
one
of
my
favorite
vulnerabilities
because
and
you'll
see
why
you
can
show
people
how
to
even
if
people
have
never
exploited
vulnerability.
If
they've
used
a
linux
shell,
a
little
bit
and
they've
used
curl,
which
many
people
have
you
can
show
people
how
to
exploit
it
in
two
minutes
and
they
can
start
like
playing
around
with
it
and
seeing
it
like
whoa
wait.
I
can
run
commands,
but
is
this
how
easy
it
is
you're
like
yeah?
That's
pretty
much
how
easy
this.
A
So
this
was
a
vulnerability
that
abused
some
functionality
in
bash,
where
bash
would
pass
exported
function,
definitions
and
environment
variables,
and
in
that
syntax
you
could
add
extra
commands
so
that
wind
bat,
the
next
the
bash
sub
shell,
was
parsing
and
adding
that
function
definition
to
the
environment.
It
would
also
execute
that
command.
A
So
that's
that's
the
base
of
shell
shock,
which
is
generally
fine
on
the
same
system,
but
sometimes
you
can
pass
environment
variables
across
a
security
boundary
so,
for
instance,
like
a
lot
of
HTTP
headers
get
turned
into
environment
variables,
and
this
happens
in,
like
you
know,
CGI,
but
it
also
happens
in
PHP
and
a
lot
of
other
application
environments
where
you
may
not
expect
it.
You
can
also
people
where
they
also
able
to
pop
DHCP
servers
with
this.
Or
do
you
see
P
clients
with
this
a
lot
of
fun
stuff.
A
Go
back
sorry,
that's
my
first
time
using
Google
slides
for
for
a
presentation
and
that
one
cool
all
right,
okay.
So
this
is
how
easy
it
is
to
exploit
shellshock
we're
going
to
connect
to
a
port
forward
to
a
vulnerable
shell-shocked
container
running
in
our
kubernetes
cluster.
It's
just
a
standard,
Apache
server,
but
kind
of
old.
These
came
with
a
test
CGI.
That
was
just
a
shell
script,
and
this
is
kind
of
you
know.
Historic
like
this
is.
This
was
true
when
I
was
a
teenager,
which
means
basically,
it's
been
true
forever.
A
A
So
what
we're
gonna
do
instead
is
we're
now
going
to
pass
some
environment,
especially
formatted
value
through
the
content
and
type
header,
because
we
can
see
that
content
type
is
one
of
those
variables
that
it
gets
and
we're
gonna
do
kind
of
magic
incantation
and
try
and
run
a
command.
So
let's
just
try
and
run
it
and
we
get
a
500
error,
doesn't
really
tell
us
very
much
so
I,
don't
know
if
it
worked.
I,
don't
know
if
it
didn't
so
now.
Let's
try
something
that'll
tell
us
whether
it
worked
like
a
sleep
one.
A
Two
three
four
five
I
have
a
pretty
good
idea
that
that
worked.
Let's
be
sure,
let's
try
with
two
did
you
do
pretend
I'm
typing
to
make
it
more
interesting,
one,
one
thousand
two
one
thousand,
oh!
So
now
we
now
we
have
this
deterministic
feedback
loop,
which
really
tells
us
as
an
attacker
that
you
have
some
element
of
control
and
the
process
of
attacking
systems
is
finding
some
element
of
control,
leveraging
it
for
moral
leveraging
it
for
more
and
kind
of
building.
A
Your
way
out,
just
like
writing
software,
like
I,
think
the
best
metaphor
that
I've
heard
people
describe
hacking
with,
is
it's
just
your
building
software
out
of
someone
else's
software
like
out
of
an
existing
ecosystem,
and
you
have
to
kind
of
invent
every
single
step
of
the
way
so
I
find
that
people
who
have
done
a
lot
of
programming
tend
to
like
it,
but
you
know
just
making
a
remote
process.
Sleep
is
not
that
interesting.
A
What
is
a
lot
more
interesting
is
getting
a
remote
shell,
so
we
can
use
the
Bosch
command
line
that
I
pasted
in
there
to
actually
use
functionality
built
into
bash
to
redirect
from
a
socket
and
back
to
a
socket
and
connect
out.
So
now
we're
you
know
now
we're
actually
hacking.
So
let's
go
to
next
one.
Yes,
so
as
so,
what
the
attacker
also
does.
They're
gonna
set
up
a
listening
like
just
like
a
net
cat
or
something
like
that.
A
So
I'm
gonna
bounce
to
my
AWS
jump
box
now
I
have
to
change
that
P
address
cuz
I
just
gave
it
away
to
all
of
you
alright,
and
so
you
can
just
listen.
You
know
this
is
what
we're
doing
in
a
different
window
and
that's
also
happening,
and
so
we're
just
gonna
run
that
command
and
we're
gonna
wait
for
a
connection
boom.
A
A
You
know
like
look
around
like
whoa,
where
am
I
what's
going
on
here,
and
you
can
see
the
host
name
that
we're
in
a
kubernetes
pod,
so
I
an
run
Linux
relatively
recent
CentOS
7,
and
that's
basically
where
the
game
starts
now,
where
it
gets
really
interesting,
is
privilege
escalation
from
here.
Ok,
so
these
work,
we're
going
to
talk
about
that
was
good,
good,
good.
A
Yeah
so
actually
frame
move
on
to
propose
escalation.
If
you
want
to
play
along
at
home,
we
published
this
container.
So
it's
really
easy
to
launch
a
shell
shock
and
then
actually
see
what
an
attacker
can
do
in
your
cluster,
which
I
really
recommend
but
pay
attention
for
the
red
warning
at
the
bottom.
Don't
expose
the
port
only
do
the
coop
CTL
port
forward,
because
this
is
a
remote
vulnerability,
but
it's
like
you
know
not
the
only
one
in
your
cluster
I'm
sure
so's.
A
It's
probably
fine.
I'm
about
your
normal
security
guy
I
used
to
jump
out
of
planes
know
most
security.
People
don't
do
that,
but
it's
not
safe
why'd.
You
do
that,
but
so
this
lets
you
see
what
an
attacker
sees
if
they
would
land
a
shell.
So
you
can
basically
do
that
and
kind
of
play
around
with
hardening,
but
now
you're,
just
an
unprivileged
user
and,
as
you
can
see,
I
followed
all
of
the
best.
A
You
know
recommendations
because
I
was
running
the
web
server
as
a
unprivileged
user,
but
that
doesn't
matter
because
what
is
root
in
a
container
really
mean,
and
now
the
problem
is
a
lot
of
kubernetes
configurations
are
credibly
weak.
They
don't
have
our
back
turned
on
by
default
and
even
some
tools
like
like
so
that
cluster
was
installed
with
cops
and
it
just
doesn't
enable
our
back.
So
basically,
every
container
is
full
cluster
root,
all
the
time.
A
So
from
that
shell,
you
can
download
coop,
CTL
and
just
start
executing
stuff
and
deploy
a
new
pod,
deploy
a
privileged
pod,
whatever
it's
great
and
if
that's
fixed,
if
you
install
if
you're,
if
you're
using
helm
in
your
cluster
helm,
also
doesn't
require
any
authentication.
So
you
can
just
talk
to
the
tiller
service
and
be
like
hey,
install
this
chart
and
you
have
full
cluster
root
again
so
for
kind
of
a
live
demo.
This
is
kind
of
going
a
little
bit
on
a
tangent.
A
But
if
you
want
to
see
more
about
that,
I
did
a
turbo
talk
at
kubernetes
NYC
on
his
youtube
link
and
I,
just
kind
of
went
through
it
in
ten
minutes
and
it
was
pretty
fun
but
we're
playing
defenders
today,
not
gonna
teach
you
all
how
to
cause
mischief,
so
we
want
to
work
backwards
from
the
breach.
So,
according
to
this
template,
how
do
we
build
a
system
to
monitor
and
detect
this?
So
we
won?
We
know
that
we
need
to
monitor
process
execution
within
containers
right,
we
need
and
we
need
it
to
be
fine-grained.
A
We
also
need
to
monitor
network
connections
because
we
want
to
know,
for
instance,
like
how
many
pods
do
you
have
that
actually
have
a
legitimate
need
to
talk
to
the
kubernetes
api
to
do
stuff.
Usually
it's
the
couplet,
it's
not
necessarily
your
pod,
so
that
might
be
a
good
thing
to
watch
for
and
you
can
figure
out
when
win
a
pod.
A
That
never
does
that
even
attempt
at
it
also,
which
pods
should
be
connecting
to
your
tiller
service,
almost
nothing
right
and
that's
another
thing
that
you
can
like
kind
of
set
a
a
guardrail
guard,
I
think
like
more
electric
fence
on
and
what
you
also
want
to
make
sure
that
your
system
that
you
set
up
does.
Is
it
not?
It
also
attempts,
because
even
unsuccessful
connections
are
a
good
signal.
A
So
if
you
have
strict
two
egress
filtering
so
that
no
one
can
get
out
of
your
cluster,
anything
trying
to
get
out
is
a
pretty
good
signal.
You
want
to
watch
out
for
and
will
tell
you
when
something
goes
wrong,
so
it's
either
softer
than
is
to
be
fixed
or
you
know
some
a
situation
that
needs
to
be
responded
to.
So
let's
talk
about
how
we
actually
build
a
continuous
security
pipeline
to
get
this,
so
our
strategy
is
first,
we
want
to
gain
visibility
into
the
activity
on
our
infrastructure.
A
We
want
to
make
sure
to
enable
investigation
into
past
activity,
because
whenever
we
see
something,
that's
the
tip
of
the
iceberg,
something
acts
out
of
place.
We're
gonna
want
to
drill
down
deeper
and
see,
what's
really
happening,
and
actually
what
led
up
to
that
point
and
in
order
to
scale
we
need
to
you
know
we
don't
look
over
the
logs.
A
So
if
something
does
something
weird
just
shut
it
down,
and
even
if
you're,
not
a
security
expert
like
just
shut
it
down
like
basically,
if
something
is
acting
anomalously
you
can
just
you
know,
kill
it
and
move
forward.
And
then,
if
you
have
enough
of
a
forensic
trail,
you
can
give
that
to
a
security
expert,
hire
a
security
company
and
be
like
hey
what
went
on
here,
because
I
don't
know,
but
at
least
you're
defending
yourself
and
also
build
the
automated
responses
to
alerts
and
do
this.
A
So
how
are
you
build
the
pipeline?
First,
we're
gonna
have
an
event.
Sourcing
architecture,
that's
gathered
from
various
sources.
Existing
data
sources
are
good,
like
logs
are
great,
but
it
usually
doesn't
give
you
the
most
as
much
information
as
you'd
want.
So
I
recommend
implementing
sensors
of
various
components
in
various
stages
of
your
environment,
so
monitoring
build
pipelines,
monitoring
production
as
they
you
know,
behavior
as
they
run,
and
can
so
that
you
can
get
that
get
more
data
to
feed
into
this
continuous
security
pipeline.
These
events,
we
analyze
generate
alerts
all
alerts.
A
You
should
always
prefer
automatic
response
where
possible,
and
given
that
we're
in
a
cycle,
we
can
change
the
way
that
our
hosts
are
deployed
to
make
automatic
responses
more
effective
and
more
and
more
possible.
We
can
change
the
software
and
you
work
with
the
teams
to
make
this
capability
as
effective
as
possible.
Just
like
you
would,
if
you're,
maintaining
performance
of
production
infrastructure.
So
what
you
do
is
you
reserve
your
human
time
to
monitor
the
alerts?
A
Do
a
manual
investigation
tune
the
sensors
to
get
more
or
less
data,
depending
on
what
you
need
and
automate
the
responses,
so
the
way
that
we
set
this
up
for
our
our
testbed
is
we
have
a
kubernetes
cluster,
it's
kind
of
auto-scaling
set
of
nodes
and
we
have
a
a
sensor
running
on
every
node
and
then
a
log
feeder
that
basically
sends
everything
to
AWS
Kinesis.
So
we
use
Kinesis
because
I
don't
really
want
to
manage
a
bunch
of
this
stuff
like
and
so.
Kinesis
is
like
simple
done
like
you.
A
Do
it
and
same
thing
with
lambda,
it's
like
cool.
You
scale
that
done
because
that's
not
really
what
I'm
good
at
so
I'll.
Just
let
you
do
it
and
I'll
focus
on
writing
the
logic
instead
and
and
then
one
of
the
outputs
you
can
configure
with
fire
hook.
Thesis
fire
hose
is
elasticsearch,
and
so
this
gives
us
a
Cabana
instance.
So
we
can
use
the
web
to
browse
and
go
through
this
data.
So
this
is
actually
like
pretty
simple,
it's
kind
of
a
you
know.
A
It
was
like
a
couple
days
of
work
to
actually
make
it,
so
it's
not
really
that
much.
It's
your
little
deeper
dive
on
how
the
fire
hose
connects
to
lambda,
so
I
Kinesis
fire
hose
lets.
You
use
lambda
for
transformation
functions
so
for
us
we're
just
going
to
use
the
the
the
identity
function.
Basically
is
a
transformation
just
return,
the
same
event
and
on
the
side.
If
something
is
interesting,
we're
going
to
publish
an
alert
to
an
sqs
queue
and
then
we
can
monitor
that
sqs
queue
for
those
alerts
and
actually
automatically
respond.
A
So
it's
actually
sort
of
super
easy
here
and
we
can
write
detection
logic
in
in
JavaScript
or
in
Python,
and
the
nice
thing
about
that
is
it
scales
pretty
well
with
the
complexity
of
the
logic.
So
it's
pretty
tempting
to
like
you
know,
make
a
DSL,
but
you
this
is
not
that
complicated.
These
are
JSON
records
and
you're
writing
code
to
look
for
a
string
match
verify
some
conditions.
Why
do
you
need
a
DSL
for
that?
A
But
when
you
have
a
full
programming
environment,
you
can
scale
to
more
complexity
as
you
need
it,
and
not
get
bounded
and
stuck
by
the
stuck
by
the
DSL.
So
the
event
sources
that
are
interesting,
I
should
look
at
your
environment.
I'm
talking
AWS,
just
cuz,
that's
the
one
that
I
speak
natively,
so
you
know
translate
that,
where
appropriate,
to
Azure
or
GCP
or
internal
infrastructure
cloud
trails,
good
get
all
the
API
activity
see
when
the
stuff
is
being
used.
A
You
want
network
monitoring,
so
you
can
see
all
your
VPC
flow
logs
for
all
your
network
activity,
but
there's
some
limitations
here.
You
don't
see
things
within
your
kubernetes
cluster
as
well.
Also,
you
definitely
don't
see
them
within
a
pod
and
then
system
monitoring.
So
in
order
to
get
like
system
level
behavior,
you
want
a
system
monitoring
agent.
I,
look
now
there's
five.
They
talked
about
here,
I'm
talking
about
capsule,
eight,
the
open
source
sensor
that
we're
releasing
right
about
now.
I
said
right
about
now.
Cuz
tried
to
do
it
today.
They'll
probably
happen
tomorrow.
A
Go
audit
is
a
pure
go.
Implementation
of
audit
using
the
kernel
audit
subsystem
go
BPF
is
part
of
the
IO
visor
project.
So
all
these
are
github
URLs.
You
can
just
go
there
and,
let's
you
actually
write
BPF
rules
that
that
run,
I
thought
I
had
an
hour
cool
and
oh
there's,
OS
query:
it's
also
a
really
popular,
really
cool
and
cystic.
So
all
of
these
are
good,
and
what
you
want
to
do
is
dive
in
deeper,
so
I've
got
five
minutes.
A
I
mean
it
kind
of
go
through
a
little
faster
here,
dive
in
deeper
and
see
what
actually
happened.
There's
a
couple
trade
offs
for
each
of
these.
What
going
by
kind
of
the
Netflix
criteria
must
be
event-driven
must
be
lightweight
and
give
kernel-level
inspection
and
be
kernel
version
independence.
Naturally,
the
one
that
we
wrote
from
the
ground
up.
We've
basically
checks
all
those
boxes,
because
that's
why
we
did
that
and
the
other
thing
is
odd:
it
has
terrible
performance.
A
So
anything
using
linux
tracing
performs
much
better,
so
capsulate
assisted,
go
audit
or
sorry
capsulated
Cystic
and
go
BPF
all
use
linux
tracing
under
the
hood,
so
they
perform
a
lot
better
and
shameless
plug
for
the
sensor.
It's
a
single
static,
go
binary.
We
don't
use
Segoe,
we
just
run
it.
It's
fine.
It
uses
all
userland
api's,
so
it
doesn't
actually
violate
a
signed
kernel
image
and
it
works
on
every
where
from
kernel
to
6
and
up
and
very
much
in
development,
because
we're
we're
still
building
it.
A
But
it's
alpha
you've
been
warned,
but
it's
Apache
2.0
license.
So
you
can
you
can
play,
but
this
was
a
piece
of
our
infrastructure
that
sent
all
his
logs
to
cabana
for
manual
search,
and
you
can
now
do
things
like
say:
hey
someone
launched
a
busy
box
container.
My
infrastructure
show
me
where
that
happened,
and
then
you
can
now
search
for
that
unique
container.
A
Id
say:
hey,
show
me
everything
that
happened
and
give
me
all
process,
execution
events
and
and
see
everything
there
and
yeah
I
talked
a
little
bit
about
lambda,
it's
easy
because
it
scales
up
and
the
language
scales
up
with
you,
whether
you're
familiar
with
JavaScript,
Python
or
Java,
and
when
you're
automating
responses,
you
can
take
events
off
that
sqs
queue
and
don't
overcomplicate
it.
You
can
just
write
a
simple
shell
script,
that
does
you
know:
AWS
sqs,
command-line,
pipe
JQ
and
pipe
that
coop
CTL
delete
and
boom.
A
You
have
an
automatic
thing
that
kicks
any
offending
pod
and
you
can
do
this
at
each
tier
of
your
infrastructure
as
you
develop
this.
The
thing
that
you
want
to
do
is
always
escalate
the
attacks.
You
can
simulate
and
your
responses,
and
you
can
keep
training
yourself
so
play
with
some
open-source
security
tools,
but
I
recommend
doing
things
like
if
you
have
an
open
source.
A
If
you
have
a
bounty
program,
try
and
find
the
researchers
before
they
report
the
vulnerability
hire
a
penetration
testing
team,
try
and
detect
them
before
they
are
before
they
report
and
when
you
get
really
advanced,
hire
a
red
team
that
tries
not
to
get
caught
and
try
and
catch
them
and
try
and
make
their
life
hell
and
that's
basically
and
once
you've
once
you're
at
that
level.
Pretty
much
most
attackers
are
gonna
have
a
really
hard
time
attacking
your
attacking
you
and
you'll.
Do
it
across
your
entire
environment
yay.
A
A
A
Sure
yeah
that's
another
thing
that
we
made
sure
to
design
around,
because
when
you
are
having
like
a
local
service
proxies
and
all
these
other
things,
what
you
want
to
monitor
is
who
the
container
thinks
it's
talking
to,
because
the
service
mention
all
those
other
stuff
is
going
to
change,
how
it
gets
there,
but
which
service
it
discovers
and
which
is
reaching
out
to
is
really
important.
And
so,
when
you're,
using
Linux
tracing
based
tracing
based
infrastructure
in
containers,
you're
monitoring
the
system
calls
and
so
you're.
A
B
A
The
events
with
where
that
service
was
at
that
time,
but
you
know
what
like
this
application
is
trying
to
talk
to
tiller.
Not
it's
talking
to
this
IP
address
and
then
three
days
later,
trying
to
figure
out
what
what
was
running
on
that
IP
address,
because
the
fit
the
kind
of
the
anti
pattern
of
security
monitoring
in
the
modern
world
is
looking
at
an
IP
address
and
then
be
like
okay,
cool
I
had
100
containers
running
on
that
IP
address.
You
know
falls
five
days
ago
or
30
days
ago.
What
do
I
do
now?
A
It's
just
no
longer
a
meaningful
identifier,
and
also
with
things
like
domain
fronting
and
like
works
really
well
in
the
cloud
domain.
Fronting
is
a
technique
where
you
connect
to
a
website
or
web
service
under
one
name,
so
your
DNS
lookup
is
for
one
name
and
your
HTTP
host
header
is
another
name,
so
like
Google
and
a
lot
of
other
services
do
this,
where
you
can
reach
any
service
from
any
any
entry
point
any
think
about
how
ingress
works.
You
kind
of
figure
out
how
that
stuff
works
to.
B
A
Yep
I
have
a
little
bit
so
one
of
our
one
of
our
engineers
installed
container
Linux
or
clear,
Linux
and
went
went
to
town
on
it,
and
then
everything
broke
and
found
a
little
more
work
for
his
desktop
than
he
wanted.
But
what
I
discovered
kind
of
being
on
both
sides
of
being
a
security
person,
saying
hey,
we
need
to
reduce
attack
surface,
and
so
initially
we
started
with
for
our
own
development,
using
from
scratch,
docker
containers
and
then
debugging.
Those
in
production
was
impossible
and
debugging
like
how
Winsett,
for
instance,
hey.
A
Why
is
this
container
not
able
to
resolve
this
service?
Everyone
else
can,
but
that
one
can't
I,
don't
know
what
does
it
think?
It's
two
connecting
two?
You
can't
you
can't
get
a
shell
in
there
and
there's.
There's
people
have
workarounds
for
that,
so
I
kind
of
how
I
resolve
this
myself
is
the
extra
two
Meg's
for
a
busy
you
know
doing
from
busy
box
or
from
Alpine
are
well
worth
it
for
that
debug
ability
and
what
you
really
care
about
is
the
security
boundary
outside
the
container.
So
have
a
shell
in
there.
A
It's
no
big
deal
but
focus
on
the
container
security
boundary
and
the
kind
of
the
cluster
security
boundary
and
one
of
the
limitations
of
a
lot
of
the
hypervisor
based
container
runtimes.
Is
they
don't
really
do
much
for
the
for
the
interface
to
the
carbonated
API?
So
you
still
have
cluster
root.
You
just
can't
do
anything
to
the
node,
so
I'm
like
alright
a
lot.
That's
a
lot
of
good.
That
does
you.
B
B
A
So
it's
actually
not
that
bad
yeah.
So
it's
basically
it's
fun.
It's
like.
If
you
looked
into
the
architecture
of
systick,
like
they
have
a
kernel
module
to
do
stuff
that
the
kernel
already
does.
So
you
already
have
the
ring
buffers
for
perf.
You
already
have
the
trace
events
accessible
through
perf
and
I.
Didn't
you
know,
I'd
like
to
indirectly
thank
Brendon
Greg
for
his
blog
posts.
For
that's
why
I
figured
this
stuff
out?
I
was
like.
Oh,
that's
cool
any
other
questions.
A
Similar
to
like
the
clear
containers
project
I
see,
a
lot
of
this
hardening
is
necessary,
but
not
sufficient,
like
I.
Don't
want
to
denigrate
this
work
it.
So
it's
all
really
cool
and
like
I
wrote
a
hypervisor
in
a
past
life
and
I
think
that
stuff
is
really
cool,
but
you
still
that's
a
really
good
way
to
have
that.
I
said
that
host
isolation,
but
there's
performance
downsides,
you're,
basically
committing
a
bunch
of
RAM
and
the
kernel
interface
is
generally
pretty
good
anyway,
but
they
don't
do
much
for
that.
A
Like
kind
of
the
the
orchestrator
level
security
and
when
I
work
backwards
from
a
breach
and
like
kind
of
think
about
as
an
attacker,
how
am
I
gonna
attack
this?
How
am
I
going
to
get
the
objectives
that
I
want?
How
am
I
gonna
achieve
the
objectives
that
I
want
I
see
once
I
see
any
security
projects
going
in
a
certain
direction?
You're
like
yeah,
just
don't
even
bother.
Looking
like
I
can't
member
who
had
the
quote,
they
said
something
about
cryptography
is
great.
A
B
B
A
Yeah
we
don't
yet,
and
so
for
our
internal
system
with
its
is
based
on
that's
a
big
blasts,
a
big
blank
spot.
So
until
we
fixed
that
no
one
go
hack,
our
build
servers,
please
yeah.
Oh
sorry,
yeah
he
mentioned
pointed
out
that
the
kubernetes
API
servers
have
an
audit
log
and
that
would
be
an
excellent
event
source
for
the
system,
so
that
you
could
see
when
a
pod
is
abusing
the
API
or
abusing
its
privileges.
A
That's
another
very
good
point:
I
should
have
made
that
a
signal
stopping
is
a
huge
signal
in
and
of
itself,
and
one
of
when
you're
really
getting
serious
about
designing
these
systems.
What
you
also
want
to
make
sure
to
do
is
tie
in
kind
of
a
the
monitoring
with
the
transaction
so
that
they
can't
turn
it
off
and
still
be
functional.
A
So
if
you
can,
if
you
can
implement
a
way
where,
if
you
don't
see
any
events
from
a
node
in
a
certain
period
of
time,
turn
off
its
port
there's
no
events
shouldn't
be
on
the
network.
It's
not
doing
anything
and
you
can
kind
of
experiment
with
this
stuff
so
that
turning
off
an
attack
or
so
turning
off
a
monitoring
agent
is
is
the
biggest
red
signal
you
know
biggest
red
flag
on
the
play
and
possible.
A
A
Yeah,
it's
a
good.
It's
a
good
point
definitely
apply
our
back
to
helm
our
to
tiller
so
that,
basically
it
is
so.
It
is
isolated
on
what
it
can
do.
Even
given
kind
of
a
lack
of
authentication
and
also
I
think
I
did
mention
is
if
you're
using
this
is
where
I'm
definitely
the
border
of
my
knowledge.
It's
using
things
like
calico
and
like
having
that
workload
separation
built
into
the
network
layer.
A
It's
going
to
be,
you
know
impossible
for
you
to
hit
tiller,
that's
running
in
a
different
name
space,
but
without
that
you
can
like
it's
a
flat
it.
So,
even
though
it's
if
it's
in
a
separate
kubernetes
namespace,
you
can
still
reach
it
over
the
network
and
you
can
still
resolve
it
through
Kubb
dns.
So,
no
matter
what
dude,
no
matter,
what
kubernetes
namespace
you're
in.
A
A
A
Honestly,
I,
don't
really
think
a
lot
about
that,
so
I
don't
want
to
give
advice
that
I
haven't
thought
through
I
know,
people
have
done
cool
stuff
with
vaults
I'll
sit
square
when
we
open
sourced
key
wiz
and
actually
ironically,
I
was
like
wait.
We're
open
sourcing,
the
thing
where
we
put
all
our
secrets
and
why
the
owned
one
of
things
is
a
bad
idea,
we're
telling
people
where
our
secrets
are
and
how
it
works,
but
turns
out.
You
actually
do
get
a
lot
of
benefit
out
of
it
and
making
the
system
stronger.
A
But
for
some
reason
it's
not
configured
by
this
tool
and
my
advice-
I
usually
give
people
is
there's
a
lot
of
knobs
that
you'd
be
surprised,
aren't
turned
on
by
default
and
different
tools.
So,
if
you
don't
want
to
go
through
all
this
stuff,
a
cloud
provider
or
a
commercial
distribution
of
kubernetes
tends
to
be
your
best.
Your
best
option,
yeah.
B
B
A
Sure
so
I,
don't
actually
believe,
like
my
philosophy
on
these
things
is
I,
don't
like
hard
blocks
because
has
a
high
potential
of
breaking
things.
What
I
like
is
a
low
latency
response,
because
when
you
are
out
of
line
but
have
a
low
latency
response,
you
can
use
have
a
lot
more
options
and
you
don't
want
to
do
pure
bulk
data
analysis,
because
that
increases
the
latency
significantly.
A
But
if
you
look
at
how
a
lot
of
people
do,
things
like
automated
trading
is
one
example:
you'll
do
a
lot
of
data
analysis
to
build
the
models
to
run
in
real
time
and
I
think
that
is
a
good
model
for
defending
your
infrastructure
because
you
are
penalized.
Similarly,
if
you
make
the
wrong
decision
fast,
that's
bad!
A
If
you
make
the
right,
you
know
if
you
miss
something
fast,
that's
also
bad,
but
you
need
data
analysis
in
order
to
inform
what
actions
you're
gonna
take
in
real
time,
and
so
what
I
focus
on-
and
you
know
we
focus
on
a
capsule-
a
is
a
low
latency
response,
a
low
latency,
automated
response
to
security
incidents
and
kind
of
that
architecture,
but
I,
don't
wanna,
go
too
much
into
the
product
plug.
So
he's
gonna
talk
about
the
open
source,
the
open
source
component,
and
that
doesn't
do
any
of
that
today.
A
Yes,
sir
beam
only
event
stream,
it's
a
uses
probes.
So
basically
it
is
a
you
know:
it's
you
know
you
can
call
it
like
the
analog
of
eventually
consistent.
You
know
it's
because
blocking
can't
tend
to
be
really
disastrous
in
production
environments.
So,
but
you
have
to
go
through
a
pretty
significant
system
load
to
start
dropping
events
with
with
perf
and
the
ring
buffers.
A
Incriminated,
environment,
yeah,
recommend,
Damon
said
it's.
Why
it's
one
it's
one
command
to
do
it
yeah
I
mean
I.
Think
the
if
you
for,
like
the
capsulate
sensor,
is
a
single
binary,
so
we
provide
a
docker
file
for
it
and
it's
pretty
easy
to
throw
that
in
a
daemon
set
and
and
you're
good
and
then
there's
a
G
RPC
API
to
communicate
with
it
to
gather
all
the
telemetry
that
you're
insted
in
and
provide
filtering
options
that
are
all
evaluated
in
the
kernel.
A
So
you
can
say,
and
it's
all
dynamic
subscriptions,
so
you
can
say
hey.
This
is
what
I
want.
So
you
can
have
one
level
agent
that
is
or
one
level
subscription.
That
is
monitoring
something
across
the
board
at
one
performance
profile
and
when
something
interesting
happens
you
can,
then
you
know,
increase
the
subscription.
You
can
say:
oh
wow,
that
container
is
interesting.
I
want
to
see
everything
that
it
does.
Alland
and
I'll
pay
the
performance
penalty
to
log
every
system
call
for
that
container
in
particular,
and
so
on.
A
You
know
doing
a
lot
of
similar
things,
but
one
of
the
things
I
notice
about
OSX
is
that
I
kind
of
like
using
cloud
api's,
because,
like
one
of
the
things
that
I
like
about
this
architecture
of
putting
detection
logic
in
AWS
lambda
is
the
attack.
Surface
is
minimal
right
because
you
don't
have
to
worry
about
because,
like,
for
instance,
alright,
let's,
let's
put
our
attacker
hats
on
okay.
So
what
is
the
you
know?
Where
of
that
detection
logic
is
I
want
to
know
what
the
detection,
czar
and
I
want
to
modify
them
right.
A
You
know
and
and
we're
also
where
all
the
logs
are
I
want
to
know
that
modify
it,
and
so
you,
if
you
make
that
more
difficult,
you're
you're
winning.
So
that's
why
I
like
lambda
cuz
I'm,
like
coughs
right
on
my
logic
in
there
and
let
Amazon
scale
it,
and
you
know
I'm
I
mean
if
Amazon's
infrastructure
is
compromised,
I
mean
I'm
pretty
much
hosts
anyway,
but
that's
one
piece:
I
have
to
worry
less
about
and
I,
don't
like
it's
not
really
that
complicated,
so
I
didn't
see
a
benefit
to
adapting
it.
A
Well,
because,
if
you're
using
a
cloud
provider
you're
already,
depending
on
the
security
of
of
their
environment,
plus
also
your
credentials
in
your
I,
am
isolation,
model
and
other
things,
and
this
is
an
easy
way
where
you
can.
If
you
also
care
about
sort
of
separation
of
concerns
among
staff,
you
can
even
do
things
like,
say,
alright
cool
well,
for
various
reasons.
A
B
A
Yes,
oh
right
now,
just
looking
at
like
things
like
health
check,
make
sure
that's
still
running
and
using
kind
of
kubernetes
health
monitoring
to
identify
wind
that
has
been
shut
down,
but
when
you
think
about
tamper-proof
a
or
making
tamper
resistant
software
that
runs
in
an
environment
that
could
be
compromised,
it's
bit
challenging,
but
it's
I
spent
several
years
doing
it
so
I
I
have
my
ways
that
I
like
doing
it
and
what
I
find
works
well.
Is
that
what
attacker
can't
do?
A
Is
they
can't
be
everywhere
at
once,
and
they
also
can't
go
back
in
time?
So
if
you,
you
figure
out
like
that
terrain
and
you're,
like
okay,
what
is
really
hard
for
them
like
if
I'm
gonna
modify
the
software
I'm
gonna
need
to
reverse
engineer.
It
I
need
to
know
how
to
do
it.
I'm
gonna
need
to
do
this
stuff
that
will
generate
signal
until
it
happens,
and
also
they'll
have
to
kind
of
you
know,
house
in
preparation,
and
you
could.
Those
are
pressure.
Points
that
you
can.
You
know
add
difficulty
on
that
side.
A
A
If
some
evasion
is
some
like
deception
into
your
infrastructure
makes
it
really
hard
for
the
attacker
in
that
infrastructure
and
so
that
they
can't
just
download
a
piece
of
open
source
software
and
figure
out
how
it
works
and
where
the
knobs
to
turn,
because
when
they
they
say,
oh
you're
running,
you
know
this
piece
of
software.
I
I
know
how
to
disable
that
it's
like
okay,
cool
well,
the
binary
image
is
totally
different,
so
try
and
find
that
for
try
and
find
that
piece
in
my
environment,
it's
a
little
different.
A
So
there's
a
lot
of
things
that
you
can
do
and
once
you
have
liked
one
of
the
things
I
realized
with
the
continuous
build
pipelines
as
it
lets.
You
do
a
whole
lot
of
cool
stuff
and
when
you
have
a
continues
delivery
pipeline
that
continuous
security
pipeline,
you
basically
don't
have
to
set
it
and
forget
it
and
get
hacked.
You
do
a
lot
of
cool.
You
know
a
lot
of
fun
stuff,
so
I
don't
wanna,
give
away
all
the
all
my
ideas
just
yet,
but
stay
tuned
anywhere.