►
Description
A classic model for risk management and control is something called “The Three Lines of Defense (3ODL).”
Line 1: Risk Owners - Front line staff and operational management
Line 2: Risk Oversight - Risk management and compliance functions
Line 3: Risk Assurance - Internal audit
However, with the advent of modern sociotechnical systems like Agile, Cloud Native, and Event-Driven architectures these legacy lines (3ODL) are at best blurred and at worst completely broken. With the modern patterns and practices of DevOps and DevSecOps it’s not clear who the front line owners are anymore. Risk management and organizational compliance teams struggle to adapt to new cloud-native models such as ephemeral containers, microservices, and event-driven architecture like serverless. Most organizations' internal audit processes today are highly toil based and have low efficacy.
A
We
have
all
kinds
of
topics
that
we
do
on
fridays,
but
today
I'm
really
thrilled
to
have
with
me
again
john
willis
who's,
going
to
talk
about
some
devsec
ops
topics
and
his
title
today
is
the
blurred
and
broken
lines
of
defense
and
I'm
going
to
let
john
introduce
himself
and
we
will
have
a
live
q
a
at
the
end
of
this.
So
please
type
your
questions
in
the
chat
and
we
will
have
a
conversation
afterwards.
So
take
it
away.
John
hey
thanks.
B
B
This
presentation,
I'm
calling
the
blurred
and
broken
lines
of
defense
so
and
it's
sort
of
as
it
turned
out,
is
it's
turning
out
to
be
a
little
bit
of
a
meta
piece
too.
So
you'll
see
what
I
mean
by
that.
But
one
of
the
things
I
would
love
is
you
know,
bots
galoop,
there's
twitter,
handle
or
j
will
sit
right
at.
I
would
love
your
feedback
right,
because
this
is
my
first
version.
B
A
B
Sort
of
any
type
of
feedback-
you're
negative
positive
you
could
have
done.
This-
should
have
done
that.
So
that
would
be
very
helpful.
All
right,
let's
get
started,
and
so
for
those
you
know
so
quickly.
I
waste
too
much
time.
There's
a
lot
of
information
about
me
out
there.
You
know
I'm
old
as
dirt,
so
I've
been
doing
a
lot
of
stuff.
B
B
Book
but
it's
called
beyond
the
phoenix
pack
with
gene
kim
and
then
some,
I
think,
really
important
working
groups
that
I've
been
on
and
I'll
talk
about
these
two,
the
the
the
green
one,
which
is
that's
what
I
meant
to
go
and
so
we're
about
to
start
the
second
version
of
that
and
something
called
the
automated
cloud,
governance
and
I'll
talk
about
that
anyway
and
done
a
whole
bunch
of
startups.
You
know
so,
and
I
do
play
guitar
too,
so
not
really
well,
but
I
do
play
guitar.
B
B
B
Right,
like
I
mean
like
big
time
like
in
both
of
those
spaces,
and
you
know
again,
devsecops
is
lagging
a
little
bit
behind
us,
our
all
the
things
we've
done
well
in
devops,
but
you
know
what
let's
face
it:
it's
still
a
mess
out
there
right,
I
mean
that's
the
bottom
line
right
like
we're,
still
struggling
we're
now,
calling
it
digital
transformation
and
that's
okay,
but
like
just
these,
are
sort
of
like
we're,
having
a
hard
time
that
the
complexity
is
growing
faster
than
our
ability
to
reason
with
it.
That's
a
whole
another
presentation.
B
Right,
you
know,
and-
and
you
know
we
can
talk
all
day
long,
even
today,
most
of
of
hard
and
security
people
say
we
still
are
running
on.
You
know:
20
30
year
old
security
models,
right
and-
and
you
know
in
devsecops,
we've
done
some
very
cool
stuff
right,
like
we
we've
sort
of
added
automation,
but
in
a
lot
of
ways
I
guess
you
know,
I'm
gonna
and
I'm
gonna
pick
one
particular
area
to
sort
of
explore,
which
is
the
we'll
talk
about
the
lines
of
defense.
B
But
but
I
guess
the
question
I'm
adding
is
like.
Yes,
it's
additive.
What
we've
done
you
know
devsecops
reference
architecture.
Is
you
know
putting
automation
of
things
that
were
done
manually
outside
the
pipeline?
Yes,
yes,
yes,
yes,
all
great
stuff,
but
are
we
significantly
making
things
better
from
a
security
perspective?
B
You
know
I
mean
I.
I
was
gonna
post
like
all
this
sort
of
the
security,
vulnerability
and
breaches
that
just
happened
within
the
last
three
days
and
it
was
like
it
was
gonna
fill
up
the
screen
anyway,
so
you
know,
and
then
we
you
know
we
put
into
sort
of
agile
security
and
we
we
have
those
conversations
so
hold
that
thought
for
a
second
and
then
I'll,
say:
okay,
let's
go
back
and
say
if
I
say
that
dev
psychops
is
five
years
old,
shannon
leads
over
it
into
it.
B
If
you
haven't
follow
her
work,
you
should
she's
brilliant.
You
know
she
coined
it.
You
know
I
think,
before
2015,
but
this
was
a
sort
of
seminal
posting
that
she
did.
You
know
and
she's.
You
know
she
owns
the
sort
of
coining
of
the
devsecops
term,
because
he
you
know
it
was
actually
just
you
know.
As
I
was
talking
to
last
night,
I
wanted
to
say,
like
I
showed
up
so
a
little
bit
about
presentation.
Like
am
I
going
to
be?
Is
this
nonsense?
Is
this
good
shannon?
B
You
know
make
sure
I
don't
make
an
idea
out
of
myself
right
and
you
know-
and
I
think
one
of
the
things
she's
great
at
is
you
know
she
sees
the
world.
You
know
way
different
than
everybody
else,
and
so,
if
you
go
back
in
2013-14
she's
an
intuit
she's
trying
to
solve
these
really
hard
problems
against
adversaries
and
she
sort
of
has
this.
You
know
it
sounds
simple.
Now
this
epiphany
of
this
idea
of
devsecops
at
that
time,
it
wasn't
you
know,
because.
B
No,
I
mean
we
have
we're
doing
a
lot
of
things,
but,
like
I
you
know,
I
I
do
a
lot
of
interviews
and
I
do
qualitative
data
analysis.
Some
of
you
might
see
my
presentation,
you
know
the
last
one
I
did
on
this.
This
forum,
where
I
interview
hundreds
of
people
in
organizations,
certainly
talk
to
risk
and
and
and
I
don't
get
the
sense
that
we're
you
know
we're
like
you
know
we're
not
even
in
the
bees
or
like
c
plus,
maybe
you'll
be
honest
in
this
and.
B
There
aren't
pockets
of
like
excellence
all
over
the
place,
but
globally,
I
think
we're
not,
and
then
you
know-
and
I
think
we're
still
trying
to
figure
out
who
should
do
security
right.
Like
you
know,
you
know
one
of
the
areas
I
I
spent
a
lot
of
time
is
something
you
know.
You
know:
sort
of
modern
governance
or
sort
of
a
new
branding
of
or
a
new
way
to
think
about
automated
covenants
and
and
a
lot
of
that
stems
for
sort
of
a
very
siloed
approach
between
risk,
I.t
sec.
B
B
B
B
In
2018
there
was
a
sort
of
tongue-in-cheek
their
auditors,
which
was
an
apology
letter
to
auditors,
but
it's
actually
more
than
that
right
because
if
you
actually
like
so
maybe
the
first
three
pages,
is
this
apology
letter
like
how
we
screwed
up
and
and
then
you
know,
the
next
sort
of
30
pages
is
a
lot
of
control,
regs
and
and
and
and
trolls
that
were
so.
We
promised
to
be
better
at
this
one.
B
I
think
historical
marker
for
me.
Thinking
about
this
whole
devsecops
for
the.
If
you
don't
know
me,
I
was
the
only
american
at
the
first
devops
days
in
ghent.
You
know
back
in
2019,
you
know,
andrew
myself
and
damon.
Edwards
ran
the
first
and
actually
mark
hinkle
ran
the
first
devops
days
in
the
us.
Together.
We
collaborated
on
that.
So
I've
been
in
the
devops
game
for
quite
a
while,
but
but
you
know
the
devsecops
thing
sort
of
started
sort
of
gnawing
at
me
in
about
2014
15.
B
You
know
about
like,
oh
my
god,
we
forgot
security
like
how
could
we
make
that
mistake?
Like
and
and
toppo
powell
had
written
this
article
in
2017
capital,
one
about
how
they
did
pipeline
design
right,
creating
better
pipelines
and
and
they
they
called
it
it's
kind
of
funny
for
you
sort
of
geeks
out
there.
B
They
said
the
ten
gates
that
they
they
had
set
up,
which
is
hex
right,
so
it
was
16
gates
right,
but
the
you
know-
and
they
said
that
so
the
the
thing
you
had
to
read
between
the
lines
was,
they
would
say
they
were
telling
the
organization
hey.
You
know
you
have
to
they
weren't
telling
the
audience
you
have
to
do
this.
You
have
to
just
have
to
do
this
and
then
sort
of
fight
that
battle.
B
Then
you
will
get
certain
privileges
in
how
you
deliver
your
software
like
one,
for
example,
you
might
not
have
to
go
to
a
wednesday
cab
meeting
right
and
and
at
that
time
you
know
I
was
I
was
hanging
out
with
toppo.
We
were
having
a
lot
of
discussions
about
like
what
could
we
apply
a
blockchain
model
to
this,
and
I
think
we,
you
know
the
industry
found
out
really
quick,
like
blockchain
is
probably
not
a
great
model
and
and
like
what
was
really
lacking
here
is
what
the
industry
was
doing.
B
Great
is,
is
gating
a
lot
of
things
like
you
know.
Everybody
was
sort
of
maturing
very
well
on.
How
do
we
stop
the
build?
If
you
don't
like
you
know,
if
it
doesn't
come
from
source
control
or
if
there's
not
a
pairing
on
a
pull
request
or
whatever
and
then
also
if
it
doesn't
have
you
know
a
clean,
build
or
even
furthermore,
in
the
devsecops
discussion
does
it
you
know,
does
it
pass
a
vulnerability
scan?
B
Digitally
signed
access
station
store
I'll,
come
back
to
that
a
little
bit,
and
so
the
two
projects
that
I
I've
worked
pretty
heavily
on
over
the
last
few
years
is
you
know
one
called
the
devops
automated
governance?
I
talked
about
we're
about
to
start
the
second
version
of
that
and
there's
some
really
fantastic
stuff
that
that
has
been
designed
and
developed.
Since
you
know
the
first
paper
in
2019,
a
couple
of
sort
of
organizations
have
really
taken
this
pretty
far
we're
doing
some
really
cool
stuff
in
red
hat
with
it.
B
So
I'll
show
you
that,
and
then
another
group
I
got
invited
to
the
beginning
of
next
year
last
year,
which
was
a
bunch
of
large
sort
of
healthcare,
helco,
healthcares
and
and
find
financial
organizations
that
wanted
to
do
something
around
cloud
loosely
related
to
the
work
that
we
did
in
this
paper.
So
I
chaired
something
called
the
automated
cloud.
B
Governance
and
I'll
talk
a
little
bit
about
that
too,
so
and
that
that
actually
we're
about
midway
through
the
second
working
group
on
that,
so
so
really
interesting,
stuff
and
I'll
put
it
in
context
as
we
move-
and
you
know
one
of
the
things
that,
as
we
think
about
the
sort
of
the
problem
statement
of
like
in
today's
world,
with
all
of
the
complexity
that
just
grows
and
grows
and
grows.
This
is
more
a
cloud
focus,
but
this
is
in
general.
B
B
You
know
like
15
years
ago,
right
like
insane,
like
you,
know,
colonel
stuff,
right
and
and
then
really
just
gone
into
a
lot
of
other
stuff,
like
you
know,
sort
of
development
and
was
pulled
back
into
a
deep
security
conversation
and
he
started
off
with
apologizing
that
hey.
You
know,
I
haven't
really
done
anything
in
security
for
15
years
and,
like
you,
hadn't,
missed
anything
buddy.
I
had
a
little
tongue
in
cheek,
but
but
again
I
think
the
the
sort
of
question
then
is
yeah.
I
mean
devsecops
reference.
Architectures
are
cool.
B
Right,
like
I
mean,
like
you,
know,
automating
a
lot
of
stuff
we
did
decoupled
and
manually
are
now
sort
of
in
pipelines
which
is
good.
You
know
sort
of
our
sas
stats,
automated
pen
testing,
but
then
you
know
the
the
real
question
is:
is
all
this?
You
know
how
much?
How
channel
is
a
term
called
secure
ability
like
that?
B
This
should
be
an
ability
and
our
can
we
ask
the
question
are:
are
we
actually
more
secure
and-
and
I
think
you
know
I
mean
like
some
presentations-
I
try
to
scare
everybody
to
death
and
show
you
all
the
stuff
right.
I'm
not
gonna
do
that
this
time,
but
like
take
it
as
default,
it's
pretty
freaking
scary.
You
know,
you
know
the
the
the
complexity
of
the
problem,
we're
trying
to
solve,
and
the
intelligence
of
the
adversaries
and
I'll
talk
about
that
a
little
bit
all
right.
B
B
Would
we
have
made
all
the
right
the
same
decisions
about
how
we
are
trying
to
deconstruct
or
maybe
just
inherit,
as
is
security
models,
as
opposed
to
like
what
shannon
tried
to
do
originally
in
2015,
which
is
breaking
blow
everything
up?
In
fact,
you
know
some
of
the
stuff
that
she's
going
to
be
publishing
shortly,
which
is
adversary,
analysis
and
adversary,
intelligence
she's,
going
to
blow
it
up
again
right.
B
The
second
line
is,
is
really
sort
of
sort
of
control
owners
and,
and
so
the
original
three
lines
of
defense
right,
which
was
you
know,
they're
sort
of
based
on
like
the
there
was
sort
of
it's
the
financial
control,
the
security
control,
the
quality
control
right,
like
you,
can
already
see
problems
here.
If
you,
if
you're
sort
of
thinking
devops
and
then
you
have
this
third
line,
was
internal
audit.
B
B
You
know
space,
space,
space,
ops,
pre-devops,
discussion
right,
you
know,
and-
and
so
you
know-
and
so
I
was
talking
to
somebody
else
about
this-
you
know
it's
great
to
have
smart
friends
right.
You
know
about
this
whole
thing
about
three
lions
defense
and
they,
you
know
sort
of
brought
something
that
should
have
been
obvious
to
me.
That
wasn't,
which
is
the
whole
conway's
law,
and
I,
like
I
started
thinking
like
maybe
there's
sort
of
a
new.
It's
called
the
called
the
tail
slo
taylor,
sloan
conway
law.
B
Right
like
you
know,
and
I
won't
go
in
front
of
taylor
and
and
sloan-
and
you
know
this
command
and
control
structure.
But
you
know
that,
if
you're
familiar
with
the
sort
of
obviously
over
used
in
presentations,
but
sorry
because
I
think
it
the
reason
probably
is
overused
the
presentations
because
it
actually
it
it
helps
us
define
some
of
the
constraints
we
have
in
modernization
opportunities
right.
So
the
conway's
sort
of
definitions,
any
organization
that
the
design
system
that
produces
design
structure
is
a
copy
of
the
organization's
communication
structure.
Right.
B
B
You
know
three
lines
of
defense
structure
right
and
then
and
like
I
said
earlier,
like
that,
you
know
in
all
honesty
like
that,
like
this
isn't
like
cavemen
mentality,
you
know
they've
like
in
2020.
The
iaa
has
updated
this
to
be
sort
of
more
modern,
but
it's
still
a
solid
approach
right.
So,
okay,
so
there's
a
governing
body
which
is
awesome.
B
It
actually
got
rid
of
the
defense
to
sort
of
promote
a
more
proactive
posture
as
opposed
to
everything
sort
of
wait
to
like
it
happened,
and
then
we
got
to
deal
with
it
but
again
like
I
will
argue
that
you
know
they
sort
of
collapsed.
You
know
the
the
the
first
line,
the
second
line
and
sort
of
this
stutter
quasi.
B
You
know
you
could
say:
okay,
give
them
the
benefit
of
doubt
that
it
isn't
silo
there.
I
would
argue
it
still
is,
but
you
are
definitely
siloed
from
internal
audit
and
by
the
way,
that's
one
of
the
biggest
problems
we
have.
Is
the
miscommunication
non-dialectic
opportunities
that
we
don't
have
with
internal
audit
or
risk
when
it
comes
to
I.t
right?
That's
that's
why
we
have
thousands
of
control
regs
that
nobody
understands-
and
I
was
you
know
I
wanted
to
dust
check
myself.
I
won't
name
the
organization,
but
it's
like
a
well-known
big
four.
B
You
know
consulting
company
who
was
heavily
involved
in
sort
of
external
internal
audits
and
all
that
and
in
their
sort
of
late
2020
banking.
You
know
banking
regulatory
observations.
You
know
part
of
the
our
response
to
the
final
suspect
isaac.
Basically,
you
know
I'll
try
to
read
this
quick,
but
you
know
another
common
issue
is
having,
so
your
capability
is
missing
or
in
the
category.
Basically,
what
they're
saying
I'm
gonna
read
it.
B
You
know
that,
for
example,
if
the
business,
the
first
line
of
defense
does
not
so
they're
still
using
defense
right,
even
though
the
the
ia
has
changed
that
sort
of
nomenclature,
it's
not
testing
capabilities
risk
and
the
client's
functions
to
the
second
line
might
perform
testing.
However,
if
the
second
line
does
that
anybody
remind
you
of
like
where
sort
of
early
devops
or
pre-devops
was
about
qa
and
and
development
right
like
this
idea
like
the
next
one,
will
catch
it
right
like
it's.
B
B
I've
been
doing
this
qualitative
data
analysis
stuff
for
quite
a
few
years
now,
where
I
basically
interview
hundreds
of
people
in
an
organization-
and
I
have
this
sort
of
presentations,
I've
done
in
days
gone
by
called
the
seven
deadly
sins
of
devops,
and
so
I'm
thinking
a
lot
about,
like
you
know-
and
you
know-
and
you
know
to
me
like
it
all
boils
down
to
seven
deadly
sin-
is
that
you
know
your
security.
You
have
security
compliance
theater,
like
your
audits.
Basically,
you
know
sort
of
create
incredible,
toil
and
have
like
terrible
efficacy
right.
B
Let's
and
that's
that's
even
before
you
get
into
you,
know,
sort
of
containers
and
platforms
and
clusters
and
and
even
worse,
like
you
know,
event,
driven,
architectures
and
and
serverless
and
stuff
like
that.
Right,
like
I'm
saying
you
you,
like,
even
in
your
sort
of
virtual
environments,
that
haven't
been
containerized
or
kubernetes
or
open
shift
eyes,
you're
terrible
at
this,
but
you
in
and
the
gap
gets
worse
and
worse
the
more
you
modernize
right.
B
You
know
you
know
the
the
ability
to
tie
some
of
the
ephemeral
activity
that
happens
in
pipelines
to
change
records,
of
somebody
describing
that
complexity
in
human
to
human
discussions
right
and
that's
what
orders
are
sort
of
going
by
notice.
Like
you
know,
what's
the
sort
of
common
theme
of
when
an
order
truly
doesn't
understand,
they
ask
for
screen
prints
like
that's
the
sort
of
you
guarantee?
No
you've
not
done
it
correctly,
and
so
the
other
thing
I've
been
thinking
a
lot
about.
So
I've
been
in
these
multiple
working
groups.
B
I
think
that
they
cover
a
lot
of
ground
like
it's,
not
everything,
one
of
the
hard
things
I
I
find
that
if
you
sit
down
a
bunch
of
people
and
say
okay,
we're
going
to
say,
we
want
to
have
sort
of
a
holistic
presentation
of
you
about
zev
techops,
and
I
I
would
tell
you
in
every
conversation,
that's
even
tried
to
do
that.
It
falls
apart
really
fast
because
there
are
so
many
security
related
things
that
are
hard
to
categorize
at
a
primitive
level,
and
I'm
not
trying
to
do
that
here.
B
I'm
just
saying,
like
I'm
really
only
going
to
cover,
probably
primarily
depending
on
time,
the
first
one,
maybe
for
another
day,
I'll
talk
about
sort
of
defense
and
trust,
but
I'm
very
been
very
focused
in
risk.
That's
that
automated
governance,
stuff
I
was
talking
about
which
is
you
know.
What
are
we
doing
to
reduce
you
know
to
to
reduce,
toil
and
increase
efficacy
when
it
comes
to
sort
of
our
our
risk
controls.
B
A
second
area
that
I
like
through
the
cloud
governance
project,
realized
that
all
these
people
are
highly
involved
in
the
sort
of
like
you
know
again,
overloaded
terms
but
they're
all
building
data
lakes,
cyber
data
lakes
right,
that's
the
thing
all
the
ultimately.
These
data
lakes
are
really
just
elk
stacks
right,
but
yeah
and
I'm
not
that's,
not
saying
that's
a
trivial
thing
right
to
build.
B
B
You
know
and
if
you've
seen
some
of
the
presentations.
So,
oh,
no
we'll
have
a
spring
conference
and
you'll
get
to
see
if
you
sort
of
pay
attention.
Just
google
own
and
ut
you'll
get
to
see
some
of
these
companies
and
the
complexity
that
they
have
they're
creating
on
their
own
great
data
and
then
the
last
area
just
quickly
just
because
I
I
you
know,
I
think
a
lot
about
this
and
I
do
less
work
in
in
the
third
category,
which
is
about
trust
right
and
that's,
that's
all
related
to
identity.
B
You
know,
and
I
had
a
cso,
tell
me
that
lisa
was
like
again
one
of
the
largest
defense
contractors
in
the
world
said
that
john,
it's
all
about
identity
right,
like
really
think
about
all
the
breaches
like
it's
some
form
of
sort
of
you
know
an
account
that
sort
of
shared
or
it's
a
server-side
request.
Forgery.
Like
you
know,
all
the
sort
of
big
ones
we've
seen
is
all
about
identity.
B
So
then
we
get
into
like
you
know,
and
it's
not
just
getting
in
like
it's
like
once
they're
in
how
do
they
find
secrets?
How
do
they
find
you
know
passwords?
How
do
they
get
into
systems?
How
do
they
get
to
the
amazon
metadata
server
and
and
and
do
a
fake
identity
right?
So
there's
a
lot
of
stuff
there
right
so
again,
I
just
wanted
to
get
that.
B
So,
let's
talk
about
the
automated
governance
like,
which
is
sort
of
that
first,
category
of
risk
right
and
and
so
one
of
the
things
that,
when
you
think
about
like
the
whole
idea
of
of
risk,
you
know
or
or
this
this
way
to
create
digitally
signed
evidence.
As
I
described
earlier
right,
you
take
what
what
capital
one
was
trying
to
do
with
the
enforcement.
B
What
if
we
could
turn
that
into
digitally
signed
evidence,
so
I
like
to
say,
think
blockchain
but
like
don't
use
blockchain
for
this
problem,
but
the
idea
like
is
that
instead
of
an
auditor
going
ahead
and
sort
of
having
to
go
back
to
a
bunch
of
change,
records
and
emails
and
give
me
this
log
give
me
that
log,
how
come
these
two
logs
don't
match?
Well,
they
never
will
they
they
they're,
not
the
same
right
and
give
me
screen
prints
that
what
if
we
could
just
say,
there's
basically
a
digitally
signed
event.
B
That's
all
immutable
can't
be
changed.
In
fact,
the
store
is
immutable
and
and
like
you
don't
like
okay,
I
mean
there's
like
no
discussion
like
we.
We
know
that
this
is
truth,
and
so
then
the
question
becomes
this
verifiable
data
audit
model,
which
is
how
do
I
prove
that
I'm
safe
okay
and
like
we
do
that
all
day?
B
Long
right,
I
have
arguments
with
cios
like
not
since
I've
been
a
red
hat,
but
when
I
was
independent
I
would
go
in
and
do
this
analysis,
company
and
I'd
go
to
and
I'd
start
pointing
out
that
seventh
deadly
sin
and
like
they
would
say
I
would
get
really
upset
when
I
would
tell
them
that
their
audits
are,
you
know,
are
basically
theater.
I
mean
it
really
matters,
they
defend
themselves,
and
then
I
give
them
proof
about
what
I
learned
and,
and
their
proof
would
be
well.
No,
no.
A
B
So
the
proof
is
this
subjective,
manifest
of
change
records
and,
and
you
know,
and
archer
databases
and
all
this
stuff
right
and
but
then
the
real
question
is:
can
you
like
when
they
ask
me
I'll,
say:
well:
okay,
here's
the
story,
I'm
going
to
tell
you
that
I
learned
tell
me
how
you
would
demonstrate
that
that's
secure,
even
though
in
your
subjective
model,
you
said
it
was
from
an
order
perspective
and
the
answer
was
you
can't
sorry
about
that?
And
so
the
question
is:
how
do
you
do
both
right
like?
B
B
How
is
how
do
we
change
the
subjective
nature
of
subjective
evidence,
or
we
just
call
them
attestations
into
objective
attestations
right
like
how
do
we,
instead
of
like
having
sort
of
a
telephone
game
of
you
know,
sue
says
it's
gonna
be
like
this
bob
said:
did
you
do
this
this
and
this
you
know
jane,
says?
Well,
it's
gotta
have
this.
If
it's
gonna
go
in
production
and
you
know,
and
then
we
spend
30
days
40
days
a
year,
sort
of
reconciling
these.
These
discussions,
as
opposed
to
we
already
have
the
automation.
B
Couldn't
we
just
you
know,
create
digitally,
signed
events
of
the
things
that
happen
with
no
human
intervention
and
have
that
be
the
evidence,
and
so
this
idea
of
like
sort
of
an
objective
evidence
and
close
feedback
loop
when
it
comes
to
security
right.
So
let's
reduce
audit
times
from
like
30
to
40
days
a
year,
so
I've
got.
I
know,
banks
that
basically
just
have
groups
that
just
reconcile
audit
issues
all
year,
long
right
to
like
zero
time
right.
It's
continuous
it's
hit
enter
like
anything
that
gets
deployed
the
evidence.
B
Is
there
in
immutable
format
and
or
and
then
also
because
it's
actually
coming
from
the
automation,
it's
not
tampered
by
humans,
the
you
know
the
efficacy
starts
to
get
increased,
so
those
the
scenarios
where
you
know
the
you
know
the
cs
says.
Well,
you
know,
john,
I'm
not
going
to
accept
that
and
I
said
well.
Let
me
tell
you
about
this.
This
sort
of
micro
service
that
uses
dynamodb
that
has
this
and
like
that
and
a
container
and
it
branches
up
to
that
and
use
a
service
mess.
B
Tell
me
where
the
evidence
is
for
that
like
and
I'm
not
saying
automating
government
solves
all
that,
but
at
least
you
can
have
web
hooks
have
the
opportunity
to
create
as
much
evidence
as
you
have,
and
in
fact
what
actually
gets
more
interesting.
Is
you
move
from
a
reactive
mode
where
most
sort
of
service
people
developers
are
in
a
sort
of
reactive
or
even
conflicting
mode,
with
internal
order
of
risk?
And
then
oh
geez?
B
They
always
want
us
to
do
this,
and
or
you
know
the
thing
I
always
hear
is
like
you
know,
we
don't
tell
order,
there's
things
they
don't
already
know
like
it's
a
common
thing.
I
hear
and
you
move
out
of
that
to
a
proactive
mode
where
you
know
one
of
the
telltale
signs
that
things
are
working
way
better.
Is
you
see
an
increase
in
self-identified
risk
controls?
B
That
means
the
not
only
are
these
sort
of
service
delivery
people
on
board,
with
this
new
way
of
doing
things,
they're
actually
starting
to
think
about
how
to
protect
the
brand,
and
is
that
a
latency
issue
that
might
be
a
that
might
be
a
you
know,
a
brand
opera.
You
know
brand
tarnish
opportunity
right.
So
all
those
things
you
know
shorting
feedback
loops.
B
You
know
moving
away
from
cabs,
enabling
shows
you
know,
and-
and
I
you
know
I
you
know-
I
do
these
loose
surveys,
but
90
percent
of
most
controls
and
most
banks
are
still
manual
today
right,
you
know,
and
so
what
we
did
is
we?
Oh
there's
one
more
thing
I
want
to
say
too.
I
think
this
is
interesting.
B
I
heard
a
charity
major
say
this
and
I
thought
it
was
sort
of
a
brilliant
observation
and-
and
I
I'm
using
it
to
extend
the
observation,
so
she
was
saying
in
one
one
of
her
sort
of
interviews.
She
wants
honeycomb
to
found
a
brilliant
woman.
She
said:
do
you
made
this
thing
about?
What's
if
there's
a
company
a
this
sort
of
a
devops
conversation,
I'll
sort
of
give
you
my
devs
sec,
I'll
spin
in
a
second?
B
You
know
what's
their,
what
is
the
real
answer
of
the
difference
between
those
cons?
The
important
answer
is
that
company
a
order
learns
capital.
L
learns
maybe
two
orders
of
magnitude
faster
than
company
b
right.
We
understand
that
we,
like
that's
everything
we
learned
from
lean.
It's
shift
left
it's.
Why
do
we
do
that
like
when
amplified
feedback
moves?
B
They've
been
on
like
four
more
projects
since
then
right
so
so,
then
the
question
is:
are
we
getting
immediate
feedback
loops
on
our
control
and
I
would
argue
that
this
type
of
automated
governance
structure
is
an
ability
to
increase
efficacy
through
that
process,
so
sort
of
quote
unquote
terribly
phased
devops,
seeing
security,
and
so
this
is
a
structure.
It's
in
that
creative
commons
books
that
you're
sort
of
welcome
to
get
an
I.t
revolution.
You
can
download
it
as
the
first
reference
group
that
I
talked
about.
B
We
tried
to
figure
out
what
what
are
the
stages
and
what
were
sort
of
the
common
attestations.
It's
a
really
good
book
for
just
understanding.
You
know
you
know,
control,
actors
and
and
common
gating
or
attestation
manifestations
and
and
again
I
won't
go
through
all
these,
and
actually
this
has
been
updated.
To
sort
of
this
is
not
act.
B
Was
there
a
pairing
on
a
pull
request
branch,
an
optimum
branching
strategy,
clean
dependencies,
build
performance,
build
version,
linting
sas,
then
the
build
stage
you
can
look
at,
and
these
are
just
examples
of
certain
places
that
we've
done
a
couple
of
internal
hackathons.
So
we
use
you
know
some
of
the
easy
kill
products
like
sauna
cube
and
you
know,
and
then
artifact
versions,
I
would
say,
maybe
come
to
nexus
meta
package
code
signing
you
know
I
gotta
I'll.
B
B
You
know
originally
we're
calling
policy
code,
but
that's
such
an
overloaded
term,
like
you
know,
for
our
purpose,
it's
really
more
like
a
risk
as
control,
and
I
don't
know
that
we're
going
to
stick
with
that
as
the
name
of
the
dsl,
but
this
is
very
specific
to
the
automated
governance
model
I
just
described.
In
other
words,
there
would
be
a
collaboration
between
risk
or
internal
audit
and
developers.
It's
like
sort
of
cut
out
the
infrastructure
people
like
cut
up
them.
It's
like
office
space
like
what
do
you
do
here?
B
B
There
was
like
some
really
awesome
projects
of
you
imagine
as
a
large,
open
source
company
that
we
have
and
something
that
is
called
the
trusted
software
supply
chain.
That
was
originally
done
as
some
of
the
government
software
factory
models
that
they
were
trying
to
sort
of
drive
or
a
red
sword.
They
called-
and
this
is
really
cool-
and
you
know
I
I'm
getting
a
little
tight
on
time
so
like
there
are
lots
of
really
cool
presentations
here,
but
this
is
just
sort
of
foundational.
B
This
is
an
automated
governance
by
itself,
but
it
is
I
you
know
it's
just
my
whole
thinking.
I
I
really
like
this
one,
because
it's
it's
an
opinionated
reference,
but
not
an
opinionated
implementation.
In
other
words,
it's
a
it's
a
composable
sort
of
yammel
based
structure
to
define
whatever,
with
which
sort
of
opinionated
structure
in
terms
of
stages,
but
completely
unopinionated.
On
the
choices
like
you
want
jenkins,
do
you
want
bamboo
whatever
like
it?
Doesn't
care
and
and
and
it
does
a
lot
of
cool
things
like
like?
B
Not
only
will
it
sort
of
you
know
sort
of
inject
like
sony
cuber
on
a
scan,
it
will
actually
take
the
artifact
and
scan
and
store
it.
Those
are
all
necessary
things.
You
need
for
automated
governance,
because
what
you'd
like
to
do
in
the
attestation
is
not
only
say
this
happen,
but
possibly
tah
and
then
shah,
that
that
lock
right
as
a
mutable
event
in
the
in
the
chain.
So
it's
this
is
brilliant.
B
It
doesn't
necessarily
have
to
be
with
kubernetes
or
openshift,
but
like
in
our
examples
we're
using
opa
and
rego,
and
then
this
is
the
tscc
project,
it's
actually
being
renamed
to
to
this
tsc
python
project
and
and
the
other
thing
again,
this
is
like.
What's
really
cool
about
working
renat
is
another
team
that
was
building.
B
Actually
a
lot
of
it
came
from
a
certificate,
transparency
and
and
key
transparency,
but,
like
you,
can
also
use
it
for
this
idea
of
verifiable
audit
data
structure.
It's
just
really.
You
put
all
this
together.
It's
really
cool,
so
here's
the
sort
of
obligatory
that
we're
going
to
talk
about
security.
You
got
to
mention
solarwinds,
but
I
I
have
a
real
sort
of
way
here
so
yeah.
I
I
had
the
opportunity
to
do
some
research
about
the
solar
panels
thing,
and
I
don't
talk
about
why
I
had
to
do
that.
B
But
you
let
your
imagination
be
your
guide
and-
and
I
you
know,
I
went
a
lot
of
places
and
like
kpmg
had
some
good
stuff,
and
you
know,
krebs
on
software
is
always
good,
in
fact,
he's
the
one
that
pointed
me
to
this,
the
crowdstrike
and
they
to
me
they
did
a
fantastic
job
of
how
the
original
kill
chain
happened
inside
of
solarwinds.
This
is
like
before
it
gets
out
to
the
wildness
everybody
and
it
it's
an
ingenious.
B
I
mean
it
like
the
at
the
shannon's
channel
lisa's
point
man,
the
the
adversary.
Analysis
is
overdue,
because
these
people
are
ruthlessly
intuitive
and
and
creative,
and
so
I
won't
go
through
the
whole
thing,
but
like
one
of
the
things
in
the
blog
article,
which
is
brilliant,
is
they
use
the
familiar
with
the
miter?
You
know
the
miter
attack
framework
like
which
is
we're
using
heavily
in
our
cloud
ornaments
governance
project.
B
So
we'll
go
through
a
couple
of
these,
but
like
the
two
metapoints
and
these
slides
will
be
variable,
so
you
can
have
fun
sort
of
find
the
original
article
and
see
how
I
said
you
know
this
attestation
would
have
been
like
incredibly
helpful
for
them,
but
one
of
the
things
that
this
crowd
got
strike
sorry
pointed
out,
which
was
there
were
all
sorts
of
hash
mismatches,
so
it
just
in
short,
they
hijacked
ms
build.
They
basically
went
ahead
and
sort
of
inserted
their
own
code
into
the
build
they
did
some
log
masquerading.
B
B
So
you
know
so
I
tried
to
sort
of
put
in
like
their
sort
of
observation,
and
you
know
in
their
tactic
and
id
from
a
minor
perspective.
So
I
thought
this
was
fun
and
you
know-
and
I
think
it
it
to
me-
you
know
again
I'm
taking
advantage
of
the
situation.
You
know
follow
and
say:
oh
pay,
attention
everybody,
but
but
I
I
really
wanted
to
make
the
point
that
that
you
know
had
they
had
automated
governance.
B
A
A
You
covered
more
than
everything
you
wanted
to
cover
there,
there's
enough
tips
and
tricks
here
and
leads
to
other
topics
that
we
could
probably
go
on
for
a
month
of
fridays.
So,
thank
you
very
much
for
this.
I
think
this
is
it's
one.
What
I'm
gonna
have
fun
with
is
getting
all
of
the
additional
resource
links
for
this.
B
A
Together,
I
think
so.
Thank
you
very
much
for
coming
john.
You
know
there
were
a
couple
of
questions
I
think
I
handled
mostly
about
you
know
how
do
I
get
a
certification
and
devsecops
from
out
of
red
hat,
and
you
know
it
that's
an
interesting
question
in
that
you
know
how
would
you
certify
someone
in
this
topic?
There's
there's
so
much
to
it.
B
You
know
I
have
dear
friends,
dear
friends,
that
I
really
like
that
actually
do
this
right,
and
so
I
I
try
to
be
sort
of
like
split
the
middle,
but
you
can't
you
can't
certify
it.
I
mean
again
if
the
if
the
goal
is
to
sort
of
you've
got
an
uninformed
organization
that
requires
it
and
and
you're
going
to
play
that
silly
game,
because
it
helps
you
achieve
the
things
that
you.
I
don't
know
why
this
keeps
happening.
B
B
Is
it
identity?
Is
it
what
they're
saying
is
it
you
know,
and
even
in
the
identity
space?
Is
it
secrets
management
is
it
is
it
certificates?
Is
it
you
know?
Is
it
encryption?
Is
it
christian
at
rest?
Is
it
you
know?
Does
it
include
discussions
about
vaults
is
in
the
discussions
about
spiffy
and
the
service
match,
I'm
like
like
right
and
then
okay.
So
let's
take
another
topic:
audit!
Okay,
you
know
like
forget
it.
B
We
could
spend
you
know
months
on
audit
like
do
you
have
no
archer,
I
would
say
no,
you
know,
like
that's
a
sort
of
old
way
of
thinking.
You
have
no
three
lines
of
defense.
If
we
talk
about
automatic,
if
we
talk
about
cloud
governance,
you
know,
then,
oh
god,
like
you
know
then,
like
you
know
like
what
are
all
the
sort
of
things
in
in
in
dims
or
or
soars.
Or
you
know,
how
do
you
create
cyber
data
lakes?
B
I
know
I'm
over
rotating
on
this,
but
I
want
to
just
make
the
point
why
certification
is
in
general
nonsense,
because
even
if
I
got
all
that
right
about
amazon,
what,
if
you're
going
to
use
google
and
microsoft
yeah,
because
that's
all
another,
so
I
mean
when
we
just
use
the
banner
I
mean
in
devops.
I
would
give
you
a
little
bit
of
wiggle
room
on
a
certification,
because
I
would
say:
okay,
if
you
understood
the
meta,
if
a
certification
test,
I
still
sort
of
fundamentally
disagree
with
these
certifications.
B
But
if
you
understood
the
sort
of
meta
around
certain
principles
that
I
think
were
very
important
to
have
in
devops,
you
know
sort
of
the
culture,
the
automation,
those
things
you
know
I
mean
damon
coined
something
called
cams
years
ago.
Of
course,
all
the
restaurants
like,
if
you
understand
those
things,
is
as
primitives
of
behavior
and
how
you
how
the
technology
and
and
the
social
social
social
technical
aspects
of
it
like.
I
think
you
can
answer
those
questions
to
do
a
pass
fail.
B
Maybe,
but,
but
I
think,
when
you
get
into
security
like
evidence
that
even
the
security
people
make
fun
of
their
their,
I
just
don't
see
how
you
I
mean
look
at
the
training
you
have
to
just
go
for
sans
institute
just
to
learn
traditional
security,
yeah,
all
right
yeah,
I
I
gotta
say
don't
get
me
started,
but
I've
already
started.
I.
A
I
know
I
I
love
getting.
You
started.
That's
like
you
know,
working
you
up
on
this.
I
think
the
the
short
answer
is
like
red
hat
certification
is
specific
to
products
for
the
most
part,
there's
some
you
know,
and
so
like
there
is
a
you
know,
certification
as
a
specialist
in
security
for
containers
and
openshift,
contain
on
the
open
and
the
openshift
container
platform,
where
you
learn
all
the
security
bits
of
that
platform
of
that
technology.
But
it's
not
the
cultural
thing,
it's
not
the
you
know,
automating.
B
B
Tell
me
what
you
know,
and
now
you
get
all
this
out,
because
I've
been
struggling
for,
at
least
on
the
security
question,
for
like
five
or
six
years
to
try
to
figure
out
what
to
and-
and
it
was
this
presentation
that
finally
got
me
to
at
least
create
those
three
boiler
plates
that
I
talked
about
like
and
that's
taken
like
five
years
for
me
to
get
to
even
build
that
slide.
To
say
in
my
mind,
it's
it's
risk,
defense
and
trust.
B
B
But
that's
gto
too
right.
We
didn't
spend
a
whole
lot
of
time.
I
know
our
team
is
on
you
know
on
this,
this
forum,
quite
often
right,
but
like
we're
sort
of
about
like
we
didn't
get
hired
because
we
were
like
red
hat
experts
or
we
knew
centos.
So
you
know
like
we
got
hired
because
we
you
know,
are
all
four
of
us.
You
know
andrew
clay
safer.
If
you
don't
know,
kevin
bear
has
caught
the
phoenix
project
and
jay
bloom
is
probably
one
of
the
smartest
guys.
B
You
know
one
of
the
top
three
smartest
people.
I've
ever
met
the
you
know
like
we're
all
you
know.
We
try
to
bridge
that
technical
and
social
right.
We
try
to
make
sure
that
we're
not
over
rotating
on
meta,
but
we're
also
like
you
know
so
we're
trying
to
hit
the
middle
ground
and
that's
that's
what
we
do
here
at
red
hat.
You
know
that's
our
sort
of
bull
horn
too,
so.
A
A
This
is
both
techno
technology
issue
and
problems
and
things
that
we
can
solve
with
some
training
and
some
certifications,
but
it's
also
a
meta
problem
and
a
cultural
problem
or
initiative
within
companies
that
we're
thrilled
to
have
you
guys
here,
helping
us
work
with
all
of
our
customers
and
ourselves
internally
too,
to
wrestle
with
these
things
and
help
drive
them
forward.
So,
thanks
again
for
coming
here
today
and
we'll
make
this
this
this
session
available
up
on
youtube
and
tell
us
again
john.