►
From YouTube: Mission Mobility Webinar : A DevSecOps Discussion
Description
Join Bill Nystrom, CTO of Air Combat Command Directorate of Communications, Harold Smith III, Co-Founder and CEO of Monkton, Pradeeb Chhabra, Director, Security Engineering at Capital One, and John Jeremiah, Product Marketing Leader at GitLab in a panelist discussion around improving your speed to market in a highly regulated environment.
Hear about upcoming webcasts: https://bit.ly/2U9ACfv
Read more about our product vision: http://bit.ly/2IyXDOX
Learn about FOSS & GitLab: http://bit.ly/2KegFjx
Get in touch with Sales: http://bit.ly/2IygR7z
A
So,
let's
get
going
and
I
want
to
dive
right
in
so
the
Geo
today's
landscape
of
rapidly
changing
priorities
and
threats
and
apps.
It
set
new
expectations.
You
know
the
citizens
that
we
support
this,
the
airmen,
the
soldiers,
the
sailors,
the
Marines,
our
customers.
They
all
expect
different
digital
services
and
capabilities,
and
we
are
forced
to
uncover
and
discover
new
ways
to
streamline
delivery,
to
deliver
applications
and
systems
that
are
secure
and
protected
and
and
they
want
it
faster
than
ever
before,
and
so
the
question
to
the
panel
maybe
start
with
bill
house.
A
B
I
think
for
us
it
is
a
very
much
the
pace
of
acceleration
and
and
the
urgency
is
predicated
on
the
new
operating
environment
that
we
are
in.
So
in
decades
past
we
had
the
luxury
of
having
economies
of
scale
and
systems,
approaches
and
platforms,
approaches.
They
are
able
to
dominate
our
adversaries
and
what
we're
seeing
with
cyberspace
operations
developing
and
in
an
area
of
persistent
engagement.
B
You
know
the
far
process
the
federal
acquisitions
processes
is
is
made
to
be
able
to
kind
of
achieve
those
economies
of
scale
and
software
and
software
capabilities
and
the
information
age
just
not
conducive
to
that
type
of
approach.
And
so
yes,
there
is
an
acknowledgement
that
we
must
be
able
to
fight
with
the
force
that
we
have
today.
B
C
A
D
And
you
talk
to
what
you
know
bill
was
talking
about.
There's
I,
think
when
you
look
at
Capital,
One
and
air
force.
The
speed
Domitian
is
critical
in
both
here
for
Capital
One,
it's
been
able
to
deliver
to
the
customer
features
of
their
customers
faster
to
outpace,
their
potential
competition
and
the
Air
Force.
It's
been
able
to
even
no
project
power
and
strike
quicker.
So
in
both
those
instances
it's
delivering
capability
to
users
faster,
cheaper,
quicker
and
more
secure.
So
you
know
it's
they're
they're,
not
really
misaligned
they're
they're.
D
Actually,
in
my
opinion,
that
those
what
both
organizations
are
trying
to
achieve
are,
you
know
very
much
aligned
and
as
far
as
you
know,
what
we've
seen
in
the
Air
Force
it's
it's
there's
a
lot
of
look.
There's
a
big
learning
curve
to
get
to
that
agility.
You
know,
I
think
is
Bill
can
probably
it
has
to
you
know,
moving
quick
is
not
comfortable
for
the
Department
of
Defense
as
a
whole
and
understanding
that
a
lot
of
that
risk
manager
and
framework
the
RMF
that
people
reference
and
a
lot
of
that
can
be
automated.
D
There
are
extensive
tools
out
there
that
can
be
plugged
into
a
continuous
integration,
continuous
development
pipeline.
At
this
point,
where
you
know
we
can
emit,
we
can
remove
human
error
in
rapidly
field
solution,
so
something
that
might
take
a
human
two
weeks
to
vet.
You
know
a
team
of
people
we
can
achieve
that.
You
know
in
10
minutes
in
a
build
process.
You
know
going
through
a
continuous
integration,
continuous
development
pipeline.
So
it's
a
lot.
You
know
from
my
opinion,
it's
a
lot
for
executives
to
get
over.
That
mindset
of.
A
A
A
All
right
I'm
going
to
keep
going
Agnes.
Maybe
you
can
chat
with
Pradeep
and
see
if
you
guys
can
work
out
the
audio
issues
and
we'll
just
keep
going
to
bring
pretty
fast
response
once
we
get
there.
Let's
keep
going,
though,
and
it's
interesting
Harold
you
talked
about
automation
is
a
key
part
of
it
and
that's
really
I
think
one
of
the
odds
that
people
have
as
they
start
to
learn
about
DevOps
and
learn
about
the
importance
of
CI
and
CD,
and
it's
not
just
about
automating
a
few
tests
here
or
a
few
tests
there.
C
A
A
C
Yeah,
so
it's
like
this
that,
just
like
any
other
financial
company,
we
are
facing
many
challenges
and
they're
different
times,
so
we
addressing
sub-sect
DevOps
by
standardizing
SDLC
processes
and
also
standardizing
the
tooling
that
is
used
for
those
processes.
So
SDS
a
should
be
based
on
consistent
to
set
for
developing,
deploying
and
delivering
code.
So
if
you
do
not
have
standard
tools-
and
you
do
not
have
standard
processes,
then
implementing
DevOps
and
hence
the
SEC
DevOps
on
top
of
it-
is
going
to
be
harder.
C
So
you
know,
I
lack
of
standardization
will
result
into
the
DevOps
and
multiple
versions
of
DevOps.
So
in
this
situation,
implementing
SEC
DevOps
is
a
big
challenge
and
just
like
any
other
large
organization.
So
we
also
go
through
mergers
and
acquisitions.
We
acquire
small
companies
or
businesses,
so
as
new
companies
are
integrated
and
the
IT
departments
are
onboard.
So
the
effective
ops
team
needs
to
work
with
the
DevOps
team
and
follow
their
schedule
and
make
sure
that
the
security
controls
and
students
can
and
security
tools
are
automated
into
the
new
IT
group
as
well.
C
So
the
second
challenge
that
we're
facing
is
the
speed
and
accuracy.
You
know
some
adjustments
are
done
on
the
weekly
basis
and
some
even
on
the
daily
basis.
So
think
like
this,
that,
if
development
team
is
taking
this
one
day
to
write
test
and
deploy
code
in
production,
you
just
cannot
run
a
scan.
That
will
take
five
six
or
seven
days
and
ask
tarpon
teams
to
stop
and
to
scan
is
the
best.
So
slowly
you
can't
need
to
be
incremental
and
hence
faster.
C
So
what
we
have
done
is
that
our
story
can't
detect
the
change
and
only
perform
the
incremental
scan.
So
this
way
we
are
able
to
measure
the
speed
of
a
job
and
another
challenge
that
any
other
company
can
face
is
the
false
positives.
So
you
know
these
tools,
they
need
to
be
optimized,
they
need
to
accustom
to
the
environment
and
they
need
to
understand
the
compensating
controls
that
you
have
before
they
produce
the
results.
C
So
high
rates
of
false
positive
is
another
nuisance
that
security
engineers
should
be
aware
of,
and
if
you
do
not
keep
your
false
positives
to
minimum
level,
you're
going
to
lose
confidence,
sub
developers.
So
I'll
say
that
do
not
go
with
a
big
bang
approach.
You
need
to
start
small
and
start
to
sound,
ensure
that
there
is
a
good
quality
in
your
results
and
developers
are
understanding
your
results
before
you
increase
the
scope
before
you
increase
the
depth
and
breadth
of
your
scans.
C
There's
three
solutions
that
you're
building
into
your
DevOps,
so
in
capital,
one
we're
shifting
left.
So
we
have
silicon
tools
built
into
earlier
life
cycles
relating
to
the
life
cycle
like
requirements,
design
and
coding,
so
be
a
fitting
security,
the
way
engineers
think
and
work.
So
our
security
controls
are
more
iterative.
The
incremental
they're
automated
repeatable
easy
to
use
and
we
are
integrating
security
controls
in
such
a
way
that
that
allow
engineers
to
fast
fail
fast
and
fail
early.
A
Alright,
I'm
back
and
I
apologize
for
the
distal
noise.
In
my
background
here,
I've
had
I
think
I
have
an
audio
issue
here,
but
Verdi.
Thank
you.
That's
really
interesting
about
shifting
left
and
in
grading
controls
and
allowing
developers
to
get
fast
feedback.
I
think
it
plays
into
automation
and
the
use
of
the
Python,
which
is
really
where
we
were
going
to
go
next,
and
so
you
know
just
nice
load
leads
bill.
How
do
you
see
that
automation
of
the
pipeline
and
what
pradeep
talked
about
it?
Capital
or
not?
B
C
B
Base
money,
and
so
so
within
the
air
force
and
within
the
DoD.
We
we
have
baselines
and
we
have
standards
right
and
a
lot
of
that
is
system,
centric
and
then
sauce
system
architecture
and
design.
On
the
business
side.
On
the
information
base
side,
there
doesn't
there's
not
a
cohesive
set
of
architectures
that
allow
for
the
merger
of
all
the
facets
that
we're
trying
to
convert,
and
so
as
we
look
at
dev
sec,
ops,
we're
looking
at
how
we
provide
that
baseline
level
platform
and
it's
an
interesting.
A
B
Because
it
really
starts
with
something
as
simple
as
standardizing
around
terms.
So
if
I
say
the
word
platform
to
the
secretary
of
the
air
force
of
the
chief
staff
of
the
air
force,
platform
to
them
actually
means
something.
That's
stove-piped,
a
single
platform
provides
a
single
effect
or
a
single
set
of
effects,
whereas
we
an
IT,
believe
that
the
platform
is
a
unifying
of
VI
and
so
really.
D
B
Repository
and
extrapolate
it
out
with
these
sets
of
applications
was
the
right
type
of
tempo
for
this
operation.
However,
that
might
not
meet
the
criteria
for
the
next
operation,
so
be
able
to
have
that
flexibility
to
be
able
to
do
that,
but
I
guess
it
for
us
as
we
go
down
this,
it
starts
with
the
definition
of
terms
and
beautifying
our
architecture,
so
we
can
start
standardizing
tools.
A
How
is
that
helping
to
address
the
challenges
of
working
in
regulated
environments,
to
working
where
there's
expectations
for
whether
it's
you
know
having
an
ATO
for
an
application
to
go,
live
or
meeting
the
compliance
and
regulatory
requirements
exist
that
pretty
talked
about
giving
developers
fast
feedback?
How
do
you
see
people
solving
that.
D
Would
at
the
top
one
I
think
it's
you
know
evaluating.
You
know
what
are
the
what,
if
the
desired
outcomes
you
want,
you
know,
do
you
need
code
scanning
in
there?
Do
you
need
dynamic
application?
Testing
fuzzing,
you
know
I,
think
it
comes
down
to
assessing
what
is
your
risk
of
what
you're
trying
to
deploy?
You
know.
There's
a
mobile
application
to
request
a
tee
time
at
a
golf
course
is
a
lot
different
than
a
maintenance
application.
That
has
you
know
technical
orders
about
the
f-35
aircraft.
D
So
you
know
what
in
you
know
what
outcomes
are
those
tools
give
you?
You
know
I
think
in
several
instances
you
know
it
might
be
appropriate
to
have
multiple,
dynamic
application,
testing
tools
or
multiple
code
scanning
tools
that
give
you
a
different
view
of
the
outcome.
So
you
know
I
think
it's
incumbent
upon.
You
know
people
like
Bill
to
say
you
know,
here's
the
outcome
we
want.
D
We
don't
necessarily
we're
not
necessarily
saying
this
exact
tool
that
you
have
to
use,
but
here
are
a
set
of
tools
that
could
give
you
that
outcome
of
you
know
we're
going
to
use.
You
know
I'm
going
to
harp
on
dynamic
application
testing.
You
know
in
the
mobile
space
you
have.
The
big
competitors
are
now
secure
encrypted
wire.
So
you
know,
maybe
one
of
those
tools
fits
your
mission.
D
Maybe
both
of
them
fit
your
mission,
but
you
know
I,
don't
think
it's
the
I,
don't
believe
that
the
DoD
should
be
saying
you
have
to
use
this
exact
tool,
but
here
are
a
set
of
tools
that
might
satisfy
your
mission
needs.
So
you
know
by
leveraging
those
sets
of
tools
by
building
your
pipeline
to
have
that
continuous
integration.
Canoes
delivery.
You
know
now
you
can
automate
that
step
from
code
check
in
to
automatic
deployment.
You
know
where,
where
we
like,
what
we're
trying
to
go
as
a
company
as
month.
D
It
is
you
know,
as
soon
as
I
check
in
code
in
the
gate
lab
somebody
reviews
it.
It
should
just
automatically
deploy
to
production.
You
know
I
know
that's
a
big
change
for
organizations
like
the
Air
Force,
but
as
you
progress
through
that
there's,
you
know
we're
going
to
see
a
lot
of
takes
from
industry
like
Capital
One,
like
those
big
earner
of
Silicon
Valley
organizations
that
ship
code
daily
I
know
Kessel
run.
You
know
as
bill.
D
No
I
think
somebody
posed
to
them.
Can
you
guys
update
the
web
application
ten
times
a
day
and
they
did
it?
You
know
that's
a
big
change
from
a
big
air
force
where
you
know
you
have
to
go
through
here's
your
assessment,
it's
going
to
take
a
month
to
deploy,
and
if
you
plan
out
your
deployment
later
or
you
know
that
just
doesn't
work
in
2019
it's
you
know,
our
adversaries
are
iterating
a
lot
quicker
than
we
are.
If
we're
stuck
to
a
one-month
deployment
cycle,
we're
already
behind.
In
my
opinion,.
A
Completely
agree-
and
you
know
the
velocity-
that
we
have
a
get
lab,
we're
deploying
to
our
website
120
times
a
day
on
average
for
the
last
30
days,
and
we
are
pushing
new
updates
to
the
get
lab,
comm
app
behind,
often
with
feature
flags
daily,
and
we
have
our
month
with
it
really.
So
we
know
that
feels
like
to
go
fast,
that
we're
doing
the
same
things
that
you're
talking
about
it's
about
the
pipeline
and
how
you
define
the
pipeline
to
give
the
developers
the
fast
feedback,
and
it's
just
like
what
Pradeep
was
describing
right.
D
You
know
to
capitalize
on
what
bill
said
earlier.
You
know
our
our
value
in
the
21st
century
is
going
to
be
how
fast
we
can
achieve
this
mission
success.
So
we
know
what
our
adversaries
can
just
go:
buy
an
iPhone
off
the
shelf
and
use
it
when
it
takes
six
months
to
acquire
that
through
the
far
like
you
know,
we're
already
behind
the
ball.
At
that
point,.
A
Precisely
so,
very
good,
any
other
thoughts
about
the
use
of
automation
and
how
automation
really
is
and
I
want
to
make
sure
we
don't
move
off
of
automation
too
fast,
because
it's
incredibly
important
to
understand
how
automation
really
is
going
to
be
the
key
to
enabling
us
to
go
faster
and
to
go
faster
and
manage
our
risk
by
giving
people
fast
feedback
by
having
the
right
tools
and
the
right
scans
that
give
that
feedback
quickly
by
taking
the
human
manual
error-prone
tasks
and
encoding
it
into
a
form
of
automation.
So
you
can
trust
pipelines.
A
C
John
I'll
say
that
we
can
choose
your
tools
wisely,
so
automation
is
one
thing,
but
what
you
are
automating
is
another
thing,
so
you
have
to
make
sure
that
whatever
you're
picking
up
is
special
to
breed
and
it
fits
into
your
environment
because
if
you
go
around
there
are
offenders
and
they
have
a
different
sales
pitch
and
they
all
win
does
think
that
their
tool
is
the
best.
But
you
have
to
take
a
decision
like
what
is
best
for
your
environment.
What
is
best
for
your
you
know
our
DevOps
culture.
C
So
in
this
case,
what
we
do
is
we
all
just
do
a
pilot
first,
so
we
do
a
busey
and
we
have
our
own
and
like
test
CI
CD
pipeline,
we
have
a
pilot
on
real
applications,
but
in
a
test
environment
and
we
compare
multiple
and
we
get
the
feedback
from
developers
and
then
we
implement
it.
So
we
go
slow
and
once
we
automate,
then
they
go
fast.
C
So
you
have
to
be
slow
in
picking
and
choosing
the
right
tool
for
your
dev
environment
and
make
sure
that
the
developers
understand
the
scanning
is
one
thing
and
making
sure
that
someone
is
looking
at
the
results.
Understanding
it
and
fixing
vulnerability
is
another
thing,
so
we
have
to
make
sure
that
developers
are
comfortable
with
the
guidance
they
are
comfortable
with
the
remediation
and
one
more
thing.
C
What
we
have
seen
is
that
if
you
implement
n
number
of
tools,
if
you
automate
n
number
of
tools,
there
will
be
n
number
of
guidance
and
there
will
be
n
number
of
remediation
advices,
so
that
makes
the
life
of
developers
harder.
They
don't
know
which
one
to
look
for
so
K
one
tool
is
saying
to
do
ABC
and
the
other
tool
is
asking
to
do
XYZ
so
having
a
standardized.
Remediation
guidance
is
also
very
important
that
no
matter
which
it's
can
you
perform
developers
understand
the
nomenclature
and
they
understand
what
needs
to
be
done.
A
Very
good,
thank
you
pretty
and
I'm
going
to
move
on.
You
said
something
I
think
that
was
really
interesting
and
while
I
agree
with
you,
we
could
spend
a
whole
lot
of
time.
Talking
about
that,
the
multiplication
of
you
know
n
number
of
tools
times
a
number
of
guidances
and
number
of
user
interfaces,
but
we
could
talk
about
that.
C
A
C
So
that's
it
it's
a
hot
pot,
so
that
cannot
be
done
overnight.
So
it's
a
gradual
shift
and
education
and
turning
and
play
an
important
role
here.
So
you
need
to
educate
your
developers
that
security
is
as
important
as
today
of
delivering
your
function
or
feature
into
the
market,
and
once
you
understand
the
culture
once
you
understand
that
mindset
of
then
deliver
study
controls
in
that
fashion.
C
So,
as
I
was
telling
you
be
a
shifting
left
because
developers
they
hate
to
wait.
So
if
you
are
performing
a
scan
and
then
asking
them
to
wait
for
a
day
or
two
to
get
results,
that's
going
to
piss
them
off,
so
you
need
to
fail
fast
and
fail
early,
and
also
the
security
scans
and
security
controls
should
be
available
on
demand.
So
whenever
I
need,
whenever
I
want,
I
should
be
able
to
scan
my
code
and
see
you
know
whether
there
are
any
issues.
So
it's
a
combination
of
multiple
elements.
C
One
is
the
business
need.
Then
business
is
dictating
its
own
terms
telling
you
that
hey!
You
need
to
go
to
the
market
fast
and
in
our
to
support
business.
That
DevOps
is
adopting
right,
so
they
are
changing
the
toolset.
Their
process
is
the
STR
C
so
that
they
can
meet
business
needs
and
then
we
have
to
meet
developers
need
the
development
teams
need
so
that
we
can
interlock
our
steps
and
as
they
are
moving,
we
need
to
move
along.
C
And
what
we
are
also
doing
is
that
if
you
tell
developers
not
only
what
needs
to
be
done,
but
how
it
has
to
be
done,
then
they
love
you
because
we
often
be
able
to
we
have
seen
is
that
this
security
scan
and
this
security
teams,
they
just
slap
security
requirements
on
developers
and
say,
hey,
go
and
fix
it,
but
there's
no
standardized
solutions.
There
are
no
standardized
security
design
patterns,
and
due
to
that,
you
know
these
teams
will
come
up
with
different
solutions.
C
So
there's
a
same
problem,
but
n
number
of
teams
are
coming
up
at
the
and
number
of
solutions,
because
there
is
no
clear
guidance.
So
if
we
can
change
the
culture
by
creating
skill
design
patterns
which
are
consistent
and
they
are
compliant
to
our
regulations,
they
are
compliant
over
standard
and
to
education
through
training
make
our
developers
know
about
those
design
patterns,
so
once
I
would
developers
adopt
and
use
those
design
patterns
they're
automatically
automatically
compliant
to
the
regulations
to
the
standards
that
we
have,
because
our
solutions
are
taking
care
of
it.
C
So
if
you
tell
developers
that
you
are
on
their
side,
you
are
part
of
a
team,
then
they
love
you
and
if
you
tell
them
that
hey
I'm,
going
to
make
you
like
easier
I'm
going
to
make
security
transparent,
but
in
return
they
need
it.
We
need
their
help,
then
that
helps
to
change
the
culture.
Because
then
you
are
blending
your
security
into
their
culture,
I
hope
it's
making
sense.
I.
A
Think
it
does
I.
Actually
I'm
really
interested.
Thank
you
pretty.
That
was
fantastic
and
it's
I
love
I
love.
The
idea
about
you
know
giving
them
the
path,
making
it
easier
for
them
to
do
the
right
thing
and
that
when
the
pipeline,
if
they
follow
the
pipeline
in
the
process
and
their
work,
is
secure.
C
C
You
know
liaisons
between
State,
Department
and
IT
department,
so
they
are
like
senior
developers
or
technical
leads
or
project
managers,
and
we
train
them
mentor
them
and
educate
them,
and
then
we
ask
them
to
perform
certain
security
tasks.
So
if
application
security
champion
is
kind
of
a
role,
so
a
senior
developer
needs
to
spend
like
five
to
ten
percent
of
their
time
in
performing
some
security
activities,
making
sure
their
security
scans
are
happening
on
their
applications
and
vulnerabilities
are
remediated
within
a
timely
manner.
C
A
Fantastic
advice,
I
think
having
champions
on
teams
is
a
one,
is
a
great
way
to
help
people
start
to
learn
and
they
they'll
follow
those
champions.
Local
leaders
yet
so
to
speak
bill
how
about
in
the
Air
Force?
How
do
you
see
the
shift
to
both
DevOps
dev
SEC,
ops,
the
cultural
transformation?
We
talked
about
Kessel
Ron
and
some
of
the
initiatives
that
the
Air
Force
has
been
doing.
How
do
you
see
this
transformation
happening
so.
B
B
The
ops
community
wants
it
stable
and
then
you
put
in
the
security
people
who
want
it
secure,
and
it
is
very
much
a
fractured
state
of
business
that
ends
up
having
an
impact
on
the
user
experience
and
so
for
us.
We
acknowledge
that
it
is
a
culture
change.
It
is
instead
of
having
three
distinct
processes
and
three
distinct
organizational
levels
that
we
need
to
bring
all
that
together
and
I.
B
Think
Kessel
run
is
an
example
of
how
how
we
bring
all
of
that,
together
with
the
focus
on
being
outcomes
and
making
sure
that
we're
delivering
outcomes
and
you're
able
to
look
at
mission
owners
and
stakeholders
and
be
able
to
provide
that
fidelity.
It's
not
all
just
about
it
being
the
most
secure
ever
it's
not
all
about
it
being
most
available
ever
or
the
most.
B
However,
it
is
a
dialogue
between
what
is
required
for
that
outcome
and,
as
we
continue
to
go
through
this
and
get
exposed
to
what
dead-set
cops
is
and
how
to
implement
it,
then
we
will
progressively
get
better
and
better
and
better
and
be
able
to
scale
scale
that
for
effect,
but
right
now
it
is
very
much
a
cultural
challenge
to
kind
of
level
level
set
those
silos
that
are
very
engrained
with
traditional
ways
of
doing
business,
with
with
long
lead
times
and
and
and
poor
outcomes.
You
sure.
A
Very
true,
very
good
insight,
as
well
as
to
how
those
different
three
different
groups
have
to
interplay
and
how
you
have
to
figure
out
how
to
work
across
them.
Harold,
hey.
What's
your
experience,
taught
you
a
note
of
you
seen
about
how
people
go
about
the
transformation
either
at
Munson
or
with
your
with
the
projects?
You've
worked
on
for
your
clients,
yeah.
D
I
think
a
lot
of
it
is
education
and
getting
people
out
of
their
comfort
zone.
You
know
people
have
been
doing
things
on
the
same
way
for
a
long,
common
and
Harcourt
is
getting
people
to
understand
that
there
is
a
you
know.
There
is
a
new
way
in
some
ways:
it's
better
in
some
ways.
You
know
it
might
be
the
singing,
but
if
that
to
me,
that's
the
hardest
problem.
I
think
you
know
from
the
Air
Force
I'd
in
the
DoD
side
in
general.
D
A
D
Know
a
lot
of
people
lean
on
policy
as
a
reason
you
can't
do
XYZ
I,
think
policies
are
crutch,
that
people
just
blame.
You
know
further
for
people's
unwillingness
to
move
to
a
different
methodology,
so
it
that's
what
I'm
seeing
I
there
were
a
lot
of
people
that
really
want
to
move
fast.
I
think
you
know
there's
a
lot
of
talk
of
you
know
in
the
DoD,
the
frozen
middle
as
a
lot
of
people
call
it.
But
it's
for
everybody's
edification.
That's
the
perception
of
you
know.
Middle
management
doesn't
want
to
do
XYZ,
I.
D
Think
from
my
perspective,
that's
the
end
users
or
the
functional
community,
not
engaging
with
those
middle
management
to
bring
them
in
the
discussion
to
see.
What's
the
value
of
that
proposition
that
they're
trying
to
do
if
I
can
you
know
the
project
we're
working
on
the
Air
Force
is
a
maintance
application
that
brings
maintenance
aid
in
a
web-browser
sitting
in
a
utility
shed
to
the
aircraft,
and
you
know
there's
a
lot
of
discussion
of
you
know.
Acquisitions
does
understand.
Well,
why
aren't
the
acquisition
officers
coming
out
for
the
line
and
seeing
how
that
impacts?
D
The
user,
why
aren't
the
AO?
Is
the
authorizing
officials
coming
out
to
see
how
that
impacts
users
on
the
flight
line?
If
there's
that
disconnect
I
think
we're
still
going
to
see
these
cultural
impasses?
But
if
we
can
get
to
see
the
value
of
how
this
you
know,
rapid
acquisition
of
technology
by
using
things
like
continuous
integration,
continuous
delivery
can
make
that
outcome.
Safe,
secure
check
off
your
policy
and
compliance,
checkboxes
I
think,
will
close
that
gap
a
lot
earlier.
A
Very,
very
good
and
I
think
the
idea
of
the
frozen
middle
of
the
middle
management.
You
are
somehow
left
behind
and
they're
they're,
often
given
goals
and
objectives
that
often
encourage
them
to
do
the
opposite
of
what
we
want
to
do
as
far
as
driving
a
transformation
I
I
would
I
would
suggest
that
goals
and
annual
how
you
goal
and
what
you
do
to
motivate
people
often
will
motivate
them
to
do
one
thing
or
the
other,
and
that
will
that
often
is
something.
A
Leaders
have
to
look
at
and
think
about
so
I
think
we're
at
a
point.
We've
had
a
couple.
We
have
at
least
one
question
is
come
in.
So
if
there
are
other
questions,
please
feel
free
to
post
a
question
in
the
Q&A
panel
we're
going
to
get
to
Q&A
in
just
a
few
minutes,
but
I
wanted
to
at
least
give
the
panel
a
topper
tunity
to
talk
about
and
to
offer
your
advice
and
your
insight.
A
Your
you've,
all
three
of
you,
have
been
involved
in
leading
transformation,
helping
people
adopt
DevOps,
SEC,
DevOps
dev,
suck
ops,
whichever
you
want
to
call
it
you've
been
involved
in
driving
that,
and
so,
if
you
were
to
be,
you
know,
meeting
with
our
attendees
in
an
elevator,
big,
elevator
and
you're,
going
to
give
them
your
advice
as
to
how
do
you
drive
at
range?
How
do
you
lead
this
kind
of
a
transformation
work
in
a
regulated
environment?
C
So
my
first
advice
is
that
hey
pick
a
place
to
begin.
You
know
start
by
fixing
an
important
problem
or
addressing
address
an
important
risk,
so
you
need
to
start
from
somewhere
so
start
with
something
simple
where
you
can
achieve
a
quick
win,
whether
it's
a
automation
of
a
security
scan
or
automation
of,
is
any
security
control
and
showcase
the
benefits
showcase
that
the
results
of
automation
and
build
momentum.
C
And
the
second
is
that
as
I
was
telling
again
I'm
repeating
in
like
a
broken
record
that
you
need
to
understand
the
corporate
culture
and
the
mindset
of
engineers
and
developers,
you
need
to
make
them
as
your
partners.
So
whatever
you
automate,
whatever
you
integrate,
it
do
see
a
CD
or
sub-sector
up,
make
sure
that
developers
are
fine
with
it
and
they're
able
to
understand
the
results.
C
And
third
thing:
I
will
say
that
hey
go
slow,
make
sure
the
quality
of
your
results
is
sound,
because
if
you
are
giving
lots
and
lots
of
false
positives,
then
you
will
lose
confidence
of
engineers
and
developers
on
you
and
ass
on
the
toolkit.
So
go
slow
and
go
some
and
make
sure
that
whatever
you're,
integrating
and
automating
automating
has
a
no
tangible
benefits
and
showcase
it
and
build
momentum
and
then
implement
more
and
more
I.
A
C
A
About
from
your
perspective,
what
advice
do
you
offer
to
other
CEOs
and
executives
who
are
faced
with
the
challenge
of
moving
their
organizations
faster
into
you
know:
SEC
DevOps,
DEP,
SEC,
ops,
DevOps,
which
ever
what
do
you
tell
them
that
they
need
to
think
about
and
do
and
how
do
you?
How
do
you
help
them
on
their
journey?
So.
B
Overused
buzz
work,
very
much
crew,
so
every
every
engagement,
every
meeting
that
I
go
to
is
predicated
by
that
kind
of
movement
happening.
It
is.
It
is
a
sea
change
of
of
what
our
processes
are
today
into
what
they
will
be
or
what
they
need
to
be
tomorrow,
and
so
what
we
in
the
Air
Force
like
to
do.
B
Do
a
lot
of
reorganization,
but
when
we
don't
necessarily
do
is
change
our
processes
after
that
reorganization,
so
you
have
a
lot
of
remnant
processes.
So
what
I
would
say
is
that
most
of
our
successful
ventures
when
we're
successful
with
change
are
based
on
outcomes,
and
so
here's
three
things
that
I
would
say
is
one
is
you
have
to
show
operational
value
and
you
do
that
by
starting
small.
B
You
start
small,
and
you
show
that
the
way
that
you're
implementing
something
or
the
the
tool
that
you
introduced
provides
a
tenth,
something
that's
tangible
and
has
operational
value.
Then
you
can
figure
out
what
the
trade
space
is.
There
is
a
more
expensive
does
it
require
more
oversight,
but
it
always
it
always
starts
with
operational
value
in
order
to
order
to
start
anything
second
part
is
it's
got
to
be
modular,
and
so
what
we're
seeing
is
we
used
to
have
platform
centric
and
you
buy?
The
whole
thing
is.
B
And,
depending
on
which
parts
of
the
proprietary,
it
became
a
whole
systems
problem
that
was
hard
to
solve,
and
so
what
we
see
is
a
lot
of
innovation
occurring
on
trying
to
solve
little
problems,
because
our
bigger
systems
aren't
flexible
enough.
So
we
need
to
look
and
be
modular
in
our
approach
to
be
able
to
say,
hey
this
works
for
today,
but
for
tomorrow
we
can,
we
can
we
can
throw
it
out
and
it
won't
compromise
the
entirety
of
the
architects
of
the
entire
system.
First
thing
would
be
transparency.
B
Open-Source
tools
are
the
de
jour
and
and
they're
readily
available
and
most
of
the
time
they're
they're
fit
for
fit
for
need.
But
if
you
don't
disclose
the
fact
that
you're
using
open-source
war
or
on
your
data
sources
are
aren't,
aren't
disclosed
rarely
available,
then,
when
the
security
people
come
in,
then
the
whole
initiative
kind
of
get
put
on
the
Shelf
right,
and
so
so
it
starts
with
that
it.
You
know.
In
order
to
be
successful,
you
must
have
that
transparency.
B
A
D
I
mean
I
think
you
books
are
the
other
gentleman
stole
a
lot
of
the
Thunder,
but
yeah
I
think
they
make
great
points
of
you
know:
don't
boil
the
ocean
if
you
try
to
tackle
the
whole
problem
at
once,
you're
just
going
to
fail.
It's
going
to
take
you
multiple
years
to
do
that,
but
you
know
adopt
the
agile
approach.
You
pick
off
one
piece
that
you
can
take
care
of
and
build
upon.
That
is
stand
up.
You
know.
D
Maybe
your
first
step
is
just
standing
up,
something
like
gitlab
and
getting
it
working
getting
code
checked
in
there.
Maybe
your
second
step
is
adding
code
scanning.
Maybe
the
third
step
is
that
dynamic
application
testing,
but
if
you
try
to
do
all
that
at
once
and
get
it
perfect,
your
your
you
know
the
outcome.
Success
probably
isn't
going
to
be
all
that
high
or
it's
going
to
take
you
years
just
to
get
that
first
capability
out
the
door
and.
C
C
D
Don't
think
failure
is
a
bad.
You
know.
Failure
is
a
good
thing
and
we
don't
get
to
where
we
are
without
failing
that,
the
first
jet
engine
airplanes.
You
know
a
lot
of
them
crashed,
but
we
wouldn't
be
flying
around
the
way
we
are
today
so
they're
going
to
look
for
how
you
fail
and
they're
going
to
use
that
against
you.
So
you
know
be
strategic
in
how
you
are
failing.
So
when
you
get
to
that
success,
you
can
build
upon
it.
D
I
think
one
thing
we've
seen
who
actually
saw
in
the
last
week
is
a
lot
of
these
services,
starting
to
use
or
being
more
interconnected.
You
know
we're
using
CTE
the
file
computing
environment,
we're
using
GCD
s
from
DISA
we're
using
D
mop
from
DISA.
You
know
these
are
multiple
different
services
that
are
helping
deliver
the
mission
and
I.
You
know
from
our
perspective,
I,
don't
think.
We've
all
tackled
the
interconnectivity,
the
configuration
management
changes
that
go
into
all
those
services
working
together.
D
D
If
you're
Capital
One,
you
know
we're
all
working
together
and
trying
to
achieve
the
same,
so
I
think
a
lot
of
it
from
my
perspective
is
getting
people
to
understand
that,
like
you
know,
we
all
may
have
different
competing
priorities,
but
our
end
goal
is
all
the
same,
and
you
know
the
security
people,
the
you
know
the
know,
people
you
know
they're
they're,
looking
out
for
their
programs
their
best
things,
but
you
it's
part
of
that
education
of
getting
people
to
understand.
There
is
a
new
way
to
do
this.
D
A
Thank
you
that
very
much
that's
useful
and
and
the
thing
I.
The
thing
I
appreciate
about
a
lot
of
what
you've
said
in
your
advice
and
it's
similar
advice,
I've
given
people
as
I've
given
presentations
and
talks,
it's
about
moving
fast
and
one
of
the
things
we
do.
It
get
lab.
Frankly,
as
we
believe
in
what
we
call
minimum
viable
change
and
it's
one
of
the
things
that
it'll
unlock
some
our
ability
to
go
fast,
I've
done
webinars
and
talked
about
what
we
do.
We
really
break
down
our
changes
into
the
smallest
incremental
change.
A
We
can
and
then
that
runs
through
our
pipeline
and
we
go
live
with
small
changes
all
the
time,
because
it's
easy
to
go
faster.
It's
easy
to
get
feedback.
It's
easy
aligned
around
that
and
it's
it's
it's
a
very
difficult
way
of
thinking.
But
once
you
get
your
head
around
and
I'd
never
go
back
to
writing.
You
know
doing
long-form
anything,
it's
all
small
and
incremental,
and
that
really
is
one
of
the
things
that
helps
us
to
go
to
go
so
much
faster,
so
I
think
that's
really
awesome,
advice
and
insight.
A
We
have
a
little
time
for
some
questions
and
and
I'm
going
to
read
the
questions
out
loud,
and
we
can
hear
your
insight
your
thoughts
early
on.
In
our
conversation,
we
had
a
question:
is
it
tech,
DevOps
or
sec?
Ops
or
you
know,
I've
heard
it
both
ways?
Can
it
please
be
standardized
and
we
talked
about
standardizing
terms
earlier?
What's
your
thoughts
on
the
terminology,
I've
heard
DevOps
steps,
I
cops,
I've
heard
all
the
different
varieties
that
we've
even
used,
some
of
them
on
this
call
bots
on
that
term,
I.
D
Think
people
are
going
to
call
whatever
they
want
in
the
end,
but
you
know
as
long
as
you're
from
my
opinion,
as
long
as
you're
getting
the
point
across
of
you
know,
as
you
know,
pretty
was
saying
shift
security
left.
It
doesn't
really
matter
as
long
as
you're
getting
to
where
you
need
to
go.
I.
Don't
think
that
the
acronym
really
matters.
A
Yeah
it's
funny.
I
was
talking
to
John
Willis
and
an
Alan
Chimel
DevOps
calm.
They
started
the
dev
SEC
ops
conference,
the
deficit
cops
days
and
their
comment
wasn't
until
they
called
it.
Dev
sack
ops
and
inserted
security
in
the
middle
of
it.
They
couldn't
get
anyone
to
apply
or
to
submit
papers
on
the
topic
of
security.
It
was
only
after
they
changed.
The
label
did
all
the
sudden
people
show
up
with
security
stories
that
was
there
they're
inside
of
that
yeah.
B
For
me
for
us
in
the
Air
Force
I
think
that
when
you
add
the
security
in
the
middle,
it
becomes
culturally
palatable
to
the
recipe.
Because
again
we
have
three
distinct
communities.
If
you
leave
one
out,
then
you
end
up
having
some
aversion
through
to
the
rest
of
the
process.
So
for
us
I,
think
Deb,
dance,
dev
says:
hey
speaks
to
the
need
to
be
able
to
actually
develop,
not
only
the
ability
to
produce
software
but
be
able
to
do
it
securely
and
then
to
operate
it
through
the
lifecycle.
B
C
So
I
also
prefer
difficult.
It
is
that
sometimes
the
ski
enthusiasts.
They
think
that
students
first
and
then
they
call
it
a
sec
develop
but
to
Harold's
point
it
doesn't
matter
at
as
long
as
you
can
convey
the
point,
I
think
it's
just
dumb,
then
so,
but
in
our
documents
in
our
lingo
we
use
deaf
psych
ops
in
Capital
One,
very
good.
Thank
you.
A
Very
much
and
I
use
the
terms
I
throw
I
all
use
them
interchangeably
depending
upon
the
audience.
Sometimes
I'll.
Do
it
to
make
a
point,
but
generally
I
use
them.
Interchangeably,
I
have
a
question
that
came
in
through
chat
and
it
was
automated
me
read
the
question
and
I'd
like
to
hear
your
thoughts.
Automation
is
great
for
security
compliance
assessment.
However,
working
in
the
government
with
required
ATO,
which
is
what
a
sort
which
is
authority
operate,
some
controls
still
require
manual
evaluation.
How
do
you
deal
with
both
manual
and
automated
controls?
At
the
same
time,.
A
B
B
For
us,
you
just
missed
us
some
gold
back
to
you,
I
apologize
so
for
us
really
there's
an
acknowledgment
that
the
the
RMF
process,
the
risk
management
framework
process
that
we're
going
through
from
a
systems
and
an
enclave
approach,
has
some
deficits,
and
especially
one
of
those
deficits
is
the
ability.
One
is
to
do
anything
with
speed
and
two
is
to
provide
real
time
status
or
real
time.
B
Indication
of
how
secure
a
system
or
Enclave
is-
and
so
yes
right
now,
because
culturally,
our
organizations
are
are
kind
of
organized
trained
and
equipped
to
be
able
to
produce
in
order
to
operate
manual.
Controls
are
very
much
part
of
that
equation.
Specifically,
it's
kind
of
like
case
law
is
once
once
something
gets
smooth.
B
Only
then
when
we
can
provide
that
real-time
feedback
that
will
see
some
of
those
manual
controls
kind
of
be
overcome,
but
we
do
have
to
deal
with
that.
We
do
have
to
deal
with
kind
of
legacy,
controls
and
kind
of
new
approaches,
and
that's
all
about
making
sure
our
mission
owners
understand
the
risk.
D
Think
to
build
on
what
bill
said.
Is
you
know
I
there
are
some
controls
to
still
need
a
human
involvement,
but
if
you
shift
to
the
left,
where
now
I
have
a
defined
process,
those
humans
should
be
very
vetting
and
validating
the
process.
They
don't
and
the
outcomes.
You
know
I.
My
vision
is,
you
know
you
give
the
factory
where
you're
putting
the
software
the
authority
to
operate.
You
audit
that
factory
and
then
you
randomly
alt
it
the
outcomes
of
that
factory,
so
the
people
should
not
be
like.
D
If
you
have
a
hundred
projects
that
you're
running
through
that
factory,
there
should
not
be
humans
assessing
every
single
project
for
every
single
release
that
you're
doing
they
should
be
evaluating
the
factory
you
built
their
continuous
integration
pipeline
and
then
randomly
auditing
the
outcomes
of
that
factory.
Where
you
know
it's,
where
has
the?
Where
has
the
process
failed
in
that
see?
Icd
pipeline
verses,
every
single
thing
we
have
to
manually
check-
and
you
know
that's
sort
of
that
shifting
to
the
left
automating.
That's
really
where
I
see
those
human
processes
that
need
to
occur.
D
Just
in
you
know,
maybe
there's
you
know
bill
is
saying
it's
because
this
is
the
way
we've
always
done
things
that
doesn't
mean
that
that's
the
way
they
should
be
done.
So
if
there's
a
you
know,
a
human
has
to
read
through
100
pages
of
you
know
the
text
that
somebody's
put
together
to
check
off
a
checkbox,
maybe
that
hundred
pages
of
text
doesn't
need
to
exist
because
your
process
has
check
that
box
off.
So
it's
you
know,
it's
support.
Help.
Excuse
me
it's
part
of
that
education.
D
B
Really
good
point
and
really
I
wanted
to
kind
of
reattach
that,
because
I
think
you're
exactly
right
for
from
up
from
Russ
and
art
in
our
perspective,
is
that
our
CIO
has
said
this.
Is
you
know
we
really
need
to
get
to
a
point
where
it's
like
the
FDA
meet
inspecting
plant?
Where
you
know
you
you're
validating
the
process,
not
every
piece
of
meat
needs
to
be
inspected,
but
every
piece
can
tell
you,
but
you're,
really
raising
the
level
of
what
you're
able
to
do
and
really
you're
driving
down.
C
B
Us
it
is,
it
is
cultural,
and
it
really
is,
speaks
to
how
do
we
make
sure
we
have
automation
that
we
need
and
get
better
at
it.
I'll
tell
you
that
I've
always
had
cable
television
until
I
had
a
viable
alternative
right,
and
so
you
always
went,
and
you
always
got
cable
TV
and
you
always
paid
the
price.
It's
always
sub
adequate,
but
there's
always
what
you
did
until
there
were
tools
available.
That
shows
you.
You
can
get
the
same
results
for
less
cost
less
friction.
B
C
And
we
also
perform
this
manual
assessments
and
menu
security
scan
says
our
band
assessments
so
think
like
this,
that
we
in
the
business
of
managing
risk
the
attackers
they
do
not
always
use
automated
scans
right.
So
we
need
to
emulate
adversary's.
We
need
to
see
what's
the
risk,
what
are
the
latest
threats?
What
are
the
latest
GTP's,
the
tectus,
the
techniques
and
procedures
used
by
attackers
and
whether
about
existing
automated
tools
can
detect?
It
can
find
those
kind
of
on
abilities
or
not,
and
for
that
extent
we
perform
some
Red
Team
operations
on
our
asset.
C
There
we
think
like
an
attacker
and
not
only
we
just
look
into
the
technology.
We
also
look
into
the
people,
processes
and
technology
to
see
how
an
attacker
can
get
into
the
bank.
So
they're
few
things,
you
cannot
automate
you,
you
cannot
automatically
havior.
You
know
we
try
to
perform
some
phishing
attacks
whenever
employees
and
see
if
they
can
fall
for
that
and
then
using
their
lack
of
knowledge
or
using
their
ignorance.
Can
we
get
into
the
bank
and
can
we
exploit
anything
so
we
perform
these
kind
of
drills
also
from
time
to
time.
C
A
Thank
you
all
so
very
much
and
we
were
really
at
the
end
of
the
hour.
We've
got
if
we
didn't
get
to
your
questions.
I'll
make
sure
we
follow
up
and
in
the
email
will
we'll
see
if
we
can
get
answers
in
the
questions
that
are
outstanding,
but
I
want
to
respect
everyone's
time
and
we've
had
a
I
think
a
fantastic
conversation.
I
want
to
I
want
to
start
by
thanking
everyone.
You
know
thanking
all
of
you
who
attended
this
session.
A
I
think
it
was
very
helpful
in
educational
I've
learned
along
the
way
and
more
specifically,
to
our
panel
bill
pretty
Harold.
Thank
you
so
much
for
sharing
your
insight
as
to
what
it
takes
to
develop
and
deliver
secure
applications
faster
at
the
speed
of
DevOps.
Today's
webinar
I've
been
happy
to
be
part
of
today's
webinar
to
host
today's
webinar
as
a
part
of
the
get
lab
team.
A
You
know
I
want
to
thank
everyone
for
your
participation
and
if
you
have
questions
and
want
to
learn
more
about
gitlab,
we'll
be
happy
to
follow
up
with
you
and
educate
you
as
to
what
gitlab
does.
But
the
purpose
of
today
was
really
let's
learn
about
DevOps
and
learn
together.
So
thank
you
all,
so
very
very
much
have
a
great
rest
of
your
day.
Thank.