►
From YouTube: SoloCon 2022 [Lightning Talk]: Log4Shell Meets Its Match - Remediation Approaches & Lessons Learned
Description
SoloCon 2022:
[Lightning Talk] Log4Shell Meets Its Match: Remediation Approaches and Lessons Learned
Speakers:
Jim Barton
Field Engineer, Solo.io
Will McKinley
Field Engineer, Solo.io
Abstract:
Review the anatomy of a Log4Shell attack. Discuss how Solo customers can protect against it and similar attacks with policy-based solutions that require no changes to protected systems.
Track:
Community and Open Source
B
So
my
name
is
jim
barton
and
I'm
a
field
engineer
in
north
america,
with
solo.I
o
joining
me
today
is
will
mckinley,
who
is
also
a
solo
field.
Engineer
today
will,
and
I
are
going
to
dive
into
the
log
for
shell
vulnerability
and
discuss
approaches
for
addressing
it
with
solo
products.
I'll
explore
blocking
malicious
requests
at
the
ingress
point
of
your
application
network
using
blue
edge
will
then
walk
us
through
defeating
log
for
shell,
by
blocking
egress
from
our
applications
to
an
attacker's
server
using
istio
and
glue
mesh.
B
Let's
start
with
some
background
on
the
log
for
shell
attack
itself.
So
in
late
november
of
2021,
the
alibaba
cloud
security
team
reported
a
zero
day.
Remote
code
execution
exploit
that
targeted
the
wildly
popular
log4j
logging
framework
well
by
early
december,
this
exploit
had
been
weaponized
and
deployed
widely
across
the
internet.
Shortly
after
that,
headlines
began
screaming
the
internet's
on
fire,
and
so
a
number
of
systems
from
prominent
tech
organizations
were
compromised
and
emergency
response
teams
from
around
the
world
scrambled.
To
respond
to
this.
B
Some
of
the
analogies
that
have
been
published
for
this
attack
are
pretty
entertaining
my
favorite.
Is
the
scotch
tape
analogy?
Imagine
they
suddenly
discovered
scotch
tape
spontaneously
explodes
you
throw
out
your
scotch
tape
rolls?
Then
you
realize
you
already
have
tape
deployed
everywhere.
So
this
analogy
highlights
one
of
the
most
insidious
characteristics
of
this
vulnerability.
B
B
So
how
does
a
log
for
shell
attack
work?
This
diagram
from
the
swiss
government's
computer
emergency
response
team
illustrates
the
anatomy
of
an
attack
quite
nicely.
The
key
to
the
attack
is
that
a
special
string
is
passed
to
an
application
via
a
common
mechanism.
Maybe
a
request
header,
maybe
a
parameter
on
a
request
path
or
maybe
via
a
posted
message.
Payload,
and
this
string
conforms
to
a
specific
syntax
that
log4j
interprets
as
a
request
for
a
lookup
substitution.
B
Now,
most
of
the
time,
those
substitutions
are
completely
harmless.
For
example,
log4j
supports
a
time
lookup.
So
you
add
a
message
to
the
log
saying
that
you
want
the
time
to
be
displayed
in
a
particular
format
and
log4j
will
replace
your
time
request
with
the
actual
time
that
the
message
was
logged.
B
But
what
happens
in
a
log
for
shell
attack
is
that
a
different
type
of
substitution
is
requested,
most
commonly
the
attacker
requests
a
jndi
substitution,
although
other
protocols
like
rmi,
have
been
exploited
as
well,
and
when
an
unpatched
version
of
log
for
j
sees
this
gen
d
substitution
string
in
a
log
message,
it
interprets
that,
as
a
request
to
look
up
a
java
object
on
a
remote
server.
Now,
if
that
server
happens
to
be
controlled
by
an
attacker,
then
it
can
serve
up
malicious
java
classes
that
will
be
executed
by
the
application.
B
When
that
happens,
the
attacker
can
potentially
exfiltrate
sensitive
data
from
the
application,
or
just
generally
create
a
lot
of
mischief
on
the
compromise
system.
So,
in
short,
if
number
one,
the
attacker
passes
in
a
malicious
string
through
a
common
web
application
parameter
and
number
two.
If
your
java
application
logs
that
string
or
passes
it
to
a
java
library
that
logs
the
string
and
number
three,
if
your
log4j
instance
interprets
that
malicious
string
and
executes
a
remote
request
say
to
agendy
server,
then
your
application
is
dangerously
compromised.
B
B
How
can
we
prevent
fires
like
this
in
the
future,
because
this
is
not
the
first
attack
of
its
kind
and
it
certainly
won't
be
the
last
sql
injection
attacks
have
been
around
for
almost
a
quarter
century
now
and
they
operate
on
exactly
the
same
principles
as
log
for
shell
attacks.
A
malicious
string
is
processed
through
a
valid
input
mechanism
and
it
causes
an
unintended
execution
sequence
in
the
application.
B
Javascript
injection
attacks
follow
the
same
pattern
as
well.
So
how
do
we
stop
it?
How
do
we
prevent
it
going
forward,
or
at
the
very
least,
how
can
we
lay
a
foundation
that
will
make
it
easier
to
stop
next
time?
Well,
at
a
fundamental
level,
we
have
three
opportunities
to
respond
to
these
attacks
number
one.
We
can
intercept
the
malicious
request
when
it
enters
our
system.
This
leads
us
to
technologies
like
web
application,
firewall
that
we'll
discuss
in
more
detail
shortly
number
two:
we
can
interrupt
the
processing
of
the
malicious
requests
in
our
application
servers.
B
We
might
do
that
by
patching
log4j
or
perhaps
reconfiguring
it
to
disallow
remote
lookups
number.
Three.
We
can
block
the
outbound
requests
that
attempt
to
reach
the
attacker's
server.
So
those
are
our
options,
let's
begin
by
considering
the
middle
option
for
blocking
the
request
processing
of
the
of
these
malicious
requests
internally.
B
B
So,
let's
talk
about
an
alternative
which
is
to
block
request
at
the
ingress
point
by
leveraging
the
enterprise
edition
of
the
glue
edge
api
gateway,
specifically
we'll
look
at
its
web
application
firewall
facility
for
applying
mod
security
rules
directly
on
board
the
envoy
proxy.
So
what
is
waf
web
application?
Firewalls
offer
a
standard
mechanism
to
inspect
incoming
requests
and
apply
a
suite
of
mod
security
rules
to
ensure
there's
no
malicious
content.
B
It
can
be
used
to
block
many
kinds
of
threats,
attacks
like
sql
injection
and
javascript
injection,
plus
requests
that
come
from
blacklisted
ip
addresses
and
much
more
excuse
me.
The
owasp
organization
even
offers
a
set
of
curated
popular
mod
security
rules,
called
the
core
rule
set
that
you
can
activate
directly
within
glue
edge
and
with
glue
edge.
You
confront
an
entire
suite
of
applications
using
these
rules
without
touching
one
line
of
code
in
the
underlying
applications.
B
B
First,
you
specify
a
set
of
mod
security
rules
to
block
malicious
input
patterns,
whether
they're
associated
with
log
for
shell
attacks
or
other
potential
risks.
Then
second,
you
apply
that
rule
set
to
a
suite
of
services
potentially
at
the
gateway
listener
level,
so
that,
with
a
single
policy,
you
can
protect
an
entire
application
network
without
touching
a
line
of
application
code.
That's
some
pretty
powerful
fire
prevention
right
there
for
more
details.
C
All
right
appreciate
that
jim,
so
jim
did
a
great
job
of
thoroughly
explaining
you
know
this
exploit
and
how
it
works
and
also
how
to
prevent
it
at
the
ingress
point,
and
so
what
I'd
like
to
focus
on?
Is
you
know
from
from
the
sense
that
you
know,
we
know
that
this
log
for
shell
exploit
had
been
sitting
there
for
a
long
time
before
it
had
been
discovered,
and
it's
not
going
to
be
the
last
one
that
you
encounter.
C
So
one
of
the
things
that
this
has
taught
us
is
that
you
know
these
exploits
can
be
pervasive
and
we
need
to
have
strategies
to
mitigate
these
exploits
from
actually
taking
effect
within
your
clusters
and
within
your
systems.
So,
in
effect,
what
we
want
to
do
is
make
sure
we
keep.
The
horses
in
the
corral.
C
You'll
also
see
a
point
out
here
to
a
blog,
that's
specific,
to
glue
mesh,
and
what
I'm
about
to
go
over
here
is
going
to
touch
on
this
subject.
But
if
you
want
an
in-depth
example
of
how
to
prevent
this,
and
also
how
to
observe
and
test
your
policies
to
make
sure
that
they
work
correctly,
then
take
a
look
there,
where
you'll
find
not
only
a
thorough
explanation,
but
you'll
also
find
a
code
repository
link.
C
What
you
do
see
is
that
you
can
very
easily
presume
a
posture
of
zero
trust
with
it
and
one
of
the
tools
in
your
in
your
tool.
Chest
is
this
ability
to
set
your
outbound
traffic
policy
so
that
you
only
go
to
the
prescribed
allowed
urls
that
you
create
with
service
entries
now
glue.
Mesh
also
has
an
abstraction
for
this,
for
any
external
destinations
that
that
you're
allowed
to
go
to
which
is
called
a
destination
and
that's
equivalent
to
a
service
entry.
C
So
if
you're,
looking
at
a
potential
malicious
agent
that
had
been
deployed
into
your
cluster-
and
it
was
coming
in
to
say
pet
clinic-
which
we
have
an
example
in
the
in
the
blog
there
and
was
trying
to
execute
this
jndi
to
some
nefarious
ldap
server-
that
you
could
see
that
getting
sent
to
the
black
hole
cluster.
But
this
is
not
the
only
thing
that
you
need
to
do
or
think
about
when
you're
looking
at
blocking
outbound
requests
or
making
sure
you
have
a
secure
posture.
C
Now,
last
of
all,
I'd
like
to
say
as
well
that
you
know
this
registry
only
is
a
client-side
policy.
So
if
you
want
to
enforce
a
cluster-wide
policy,
then
using
an
egress
gateway
is
the
best
solution
for
your
security
posture,
but
whatever
you
do
make
sure
that
you
have
a
system
that
you
can
test
these
exploits
in
a
in
a
safe
way,
be
able
to
reproduce
these,
perhaps
by
deploying
your
own
ldap
server
that
you
can
do
this
exploits
have
to
be
observable,
and
using
access
logs
is
a
great
way
at
a
minimum.
C
Our
conclusions
and
lessons
learned
you
know
obviously
log
for
shell
is
a
big
problem
here
and
we
need
more
than
just
a
software
solution.
We
need
platforms
that
are
going
to
create
policies
for
us
to
take
advantage
of,
so
that
we
can
remediate
unknown
vulnerabilities
and
having
the
capability
for
quick
remediation
is
key.
C
Glue,
edge
can
apply
policies
that
will
block
inbound
requests
and
make
sure
that
you're
not
at
risks
and
then
glue
mesh
can
apply
policies
as
well.
That
can
block
outbound
requests
from
within
your
cluster.
Should
the
attack
happen
from
within
and,
lastly,
observability
and
also
all
stress
testability
is
foundational.