►
From YouTube: Supply Chain Security with the Jenkins Templating Engine! - Steven Terrana, Booz Allen Hamilton
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Supply Chain Security with the Jenkins Templating Engine! - Steven Terrana, Booz Allen Hamilton
In this talk, Steven will provide a comprehensive introduction to DevSecOps. He'll help attendees get past the buzzwords and demystify the various kinds of software security scanning that teams can incorporate into their software delivery processes to shift-left security. Equally important - you'll then learn how to apply these principles at scale using the Jenkins Templating Engine to develop tool-agnostic pipelines that can be shared across teams.
A
A
So
before
we
start,
I
want
to
take
a
step
back
and
and
look
at
what
is
software
supply
chain
at
a
glance
where
it
covers
everything
from
trusting
who
is
contributing
to
your
code
base?
How
is
that
codebase
getting
pulled
into
your
automated
devsecops
pipeline?
What
dependencies
are
being
pulled
into
that
pipeline?
How
do
you
trust
the
infrastructure
of
that
ci
pipeline?
A
How
do
you
make
sure
that
the
artifacts
published
to
your
artifact
repository
are
the
ones
that
got
built
from
your
devsecops
pipeline?
And,
finally,
how
do
you
make
sure
that
the
artifacts
that
you're
running
in
production
are
the
artifacts
that
you
expect
to
be
running
in
production?
Each
one
of
these
arrows
could
really
be
its
own
conference
talk.
So
I
want
to.
A
So,
let's
start
with
a
overview
of
devsecops
one
of
the
privileges
of
of
my
role
is
I
get
to
talk
to
dozens
of
teams
and
hundreds
of
developers
about
devops,
and
the
first
thing
I
learned
is
that
no
one
has
the
same
definition
of
what
devsecops
actually
means
the
way
I
look
at
it.
Devsecops
means
incorporating
security
into
every
step
of
the
software
development
lifecycle
and
that
manifests
itself
in
a
couple
different
ways.
The
first
is
application
dependency
scanning
right.
A
So
as
we
build
applications,
how
do
we
make
sure
that
the
third-party
dependencies
that
we're
pulling
into
our
environment
don't
have
known
vulnerabilities?
There
is
lots
of
great
tools
out
there
in
the
industry
to
do
this
today
from
an
open
source
perspective.
You've
got
a
west
dependency
checker,
but
there's
also
a
ton
of
great
vendors
in
this
space
around
sonotype
nexus
with
their
firewall
and
lifecycle
products.
A
You've
got
artifactory
with
x-ray
black
duck,
there's
plenty
of
options
to
make
sure
that
the
applications
or
the
dependencies
that
you're
pulling
into
your
applications,
don't
have
known
issues,
and
these
dependencies
are
like
the
sort
of
the
raw
materials
you're
pulling
into
your
your
assembly
line.
That's
going
to
produce
your
your
artifact.
You
want
to
make
sure
that
these,
to
the
best
of
your
knowledge,
are
aren't
compromised
once
you've
pulled
in
those
dependencies.
A
We
can
actually
build
an
artifact
here,
so
that
would
mean
you
know
we're
at
day
zero
of
kubecon
building
a
container
image
so
with
the
container
image
comes
a
whole
new
artifact,
which
means
a
whole
new
set
of
scans
that
we
get
to
do
so.
This
is
where
container
image
scanning
comes
in
there's
tons
of
different
tools
in
this
space.
A
Once
you
scan
these
images,
you
know
vulnerabilities
really
come
in
two
different
flavors
you've
got
cves,
so
I
pulled
in
you
know,
compromised
code
into
my
ecosystem,
but
then
you
also
have
vulnerabilities
that
stem
from
misconfigurations,
and
this
is
where
continuous
compliance
comes
in.
So
continuous
compliance
means.
A
A
Most
of
those
policies
have
a
control
profile
associated
with
them
that
these
continuous
compliance
scanning
tools
can
validate,
making
sure
that
you,
you
know,
have
configured
your
infrastructure
in
a
way
that's
secure.
Once
you
have
a
container,
you
can
actually
go
and
deploy
it
to
an
environment
and
from
there
there's
still
more
testing
to
do
right.
You
can
do
penetration
testing
to
make
sure
that
your
application
is
not
susceptible
to
some
of
the
more
common
attacks.
A
Things
like
sql
injection
or
you
know
not
parsing
user
input
and
ending
up
with
a
remote
code,
execution
vulnerability
and
then
the
same
vein
of
compliance.
I
think
it's
important
to
mention
accessibility
assurance,
so
your
devsecops
pipeline,
once
you've
got
a
deployed.
Application
in
a
test
environment
can
also
scan
it
from
a
compliance
standpoint
to
see
if
it's
compatible
with
508
there's
no
tool
out
there
that
I've
seen.
Yet
that
can
tell
you
with
100
certainty
if
your
application
is
accessible,
but
it
can
definitely
tell
you
if
you're
not
right.
A
So
if
you're
missing
some
obvious
accessibility
features,
you
know
things
like.
I
don't
have
an
alt
text
on
on
this
image.
These
tools
will
catch
that
for
you,
which
really
gives
the
folks
that
are
doing
manual,
accessibility,
testing,
the
ability
to
focus
on
the
more
complex
areas
of
accessibility.
A
So
let's
take
a
simpler
example:
if
I've
got
a
java
application
versus
a
node
application,
my
java
application
for
unit
tests
and
for
building
the
application,
I
is
going
to
probably
use
gradle
or
maven.
If
it's
an
ultra
application,
you
might
be
using
something
like
ant.
If
I've
got
my
node
application,
it's
probably
using
npm
or
yarn
or
gulp.
A
So
what
let's
talk
about?
What
happens
in
in
practice
right
so
typically
there's
when
you're?
First
starting
your
devsecops
journey,
a
lot
of
organizations
will
choose.
You
know
what
us
consultants
would
call
a
champion
so
who's
your
team,
that's
going
to
be
the
first
to
try
this
out
and
you'll
build
a
pipeline
for
them.
It
will
be
moderately
successful,
there'll
be
some
bumps
in
the
road
as
they
experience
their
first
security
scans.
There's
a
ton
of
patching
that
always
happens
the
first
time.
A
You
start
in
incorporating
these
practices
into
your
teams,
but
you
did
it.
You
have
a
devsecops
pipeline
for
a
specific
team,
so
you
know
you
deal
with
that
learning
curve.
You
you
successfully
get
the
team
integrated
and
you
say
awesome
like
let's
scale
this
across
the
organization.
Our
pilot
was
successful.
So
what
happens?
Next?
A
You
wrote
your
pipeline,
presumably
in
one
of
the
more
popular
ci
cd
tools,
these
days
like
jenkins
or
gitlab,
and
to
do
that
you
needed
to
have
a
pipeline
artifact
that
defined.
You
know
what
what
is
our
software
delivery
process
as
code?
So
there's
a
there's.
A
file
in
your
source
code
repository
called
a
jenkins
file
or
called
a
dot
gitlab
ci
diamo
file.
That's
outlining
what
exactly
is
supposed
to
happen
when
developers
make
changes
to
this
repository,
so
we
want
to
take
that
artifact.
We
want
to
apply
it
to
multiple
teams.
A
So
what
do
we
do?
We
copy
and
paste
it
to
the
next
repository
that
we
want
to
onboard,
and
we
start
tweaking
it
right
where
we
were
doing.
I
don't
know
the
gradle
test
execution
before
we'll
swap
that
out
to
maven
or
to
npm
or
yarn,
and
you
know
that
that
works
for
a
while.
So
we
you
know,
if
we
look
at
the
two
axes
of
this
graph
here,
we've
got
scale
on
the
x-axis.
A
A
So
that's
you
know,
as
we
start
to
onboard
more
than
one
application
and
we're
copying
and
pasting
and
tweaking
our
complexity
starts
to
go
up
at
a
pretty
steady
rate.
As
we
start
to
onboard
things.
Eventually,
you
hit
a
point
where
you
start
to
realize,
like
I,
either
need
every
development
team
in
my
organization
to
understand
how
to
create
a
devsecops
pipeline
and
maintain
these
pipeline
artifacts
or
I
need
to
find
a
better
way.
You
know
we're
running
into
all
sorts
of
challenges.
Teams
don't
know
how
to
edit
these
files.
A
You
know
in
jenkins
that
would
mean
let's
create
a
jenkins
shared
library
where
we
can
pull
out
common
steps
and
that
will
really
simplify
our
pipeline
as
we
scale
it,
and
you
know
it
does
at
first,
you
do
simplify
your
larger
scale
pipeline
development.
That
way
over
time.
As
you
add,
more
and
more
teams,
you
find
yourself
adding
more
and
more
modules
to
those
shared
libraries
or
modularized
pipeline
implementations.
A
Eventually,
you
say
this
isn't
working
either.
You
know,
we've
probably
hit
50
to
75
percent
of
of
our.
You
know
organization's
application
portfolio,
but
things
are
getting
way
too
complicated.
This
this,
you
know
shared
library,
is
gigantic,
so
I
have
a
better
idea
and
I've.
You
know
it's.
It's
really
interesting
that
I've
probably
seen
this
pattern
six
or
seven
times
just
independently,
like
what
teams
end
up
doing
once
they
reach
this
scale.
A
Is
they
simplify
their
pipeline?
To
just
say,
let's
call
an
entry
point
method
inside
the
shared
library,
so
your
you
know,
for
example,
your
jenkins
file
would
just
say
pipeline.run
and
you'd
pass
it
a
bunch
of
arguments
and
that
would
try
to
abstract
away
all
of
the
complicated
bits
of
this
devsecops
pipeline.
So
you
know
you
simplified
the
configuration
for
an
individual
team.
They
had
to
just
copy
and
paste
this.
A
A
You're
saying
you
know
this
is
what
I
want
to
happen
when
right,
when
I
open
a
pull
request
to
this
particular
branch.
These
are
the
series
of
tests
that
I
want
to
happen
and
within
those
conditionals
you're,
also
putting
the
specific
technical
implementation.
Logic
of
you
know,
go
run
java
or
go
run
my
node
app
and
do
tests
or
do
scans
so
that
that
has
two
main
issues
with
it
right.
The
first
is
time
I
need
to
duplicate
my
pipeline
across
every
single
source
code
repository,
which
is
really
cumbersome.
A
A
There's
no
way
to
know
that
these
teams
are
actually
following
the
process
they're
supposed
to
right
from
a
supply
chain.
Security
perspective
like
what's
to
stop
a
developer
from
going
and
editing
their
jenkins
file
in
their
feature,
branch
and
saying
deploy
to
production
all
right.
We
would
like
to
think
that
the
developers
that
we
work
with
wouldn't
do
that,
but
you
know
sometimes
our
accounts
get
compromised.
A
I've
been
on
teams
where
we
have
to
maintain
the
devsecops
pipeline.
That's
used
by
you
know
a
bunch
of
different
development
teams
throughout
an
organization
and
trying
to
keep
up
with
that
complexity
is
just
not
sustainable.
You
either
have
to
scale
this
def
ops
team
linearly
to
support
you
know
the
number
of
apps
that
are
being
onboarded
or
you
need
every
dev
team
to
become
devops
engineers,
which
itself
is
not
super
sustainable.
A
So
let's
talk
about
how
we
can
make
this
a
bit
better
pipeline
templates
when
we
pulled
up
that
you
know
graph
earlier
I'll
shoot
back
to
it.
When
we
looked
at
this
devsecops
pipeline,
we
didn't
call
out
a
tool
here.
We
just
said
every
app,
regardless
of
the
type
of
app,
is
going
to
do
these
things.
A
To
simplify
this
picture
a
bit.
Everyone
is
going
to
run
some
tests.
Everyone
is
going
to
build
an
artifact,
hopefully
sign
that
artifact
and
publish
it
to
an
artifact
repository
they're,
going
to
scan
that
artifact
they're
going
to
deploy
it
somewhere,
they're
going
to
run
some
more
tests
and
then,
if
everything's,
good
they're
going
to
deploy
to
production,
nothing
about
what
I
just
said
calls
out
a
specific
type
of
application,
a
specific
build
tool,
specific
scanning
tool.
What
have
you,
however,
when
we
build
our
pipelines?
We
couple
them
to
specific
tools.
A
A
So
you
know
this
fancy
gif
here
is
stepping
through
we
can
apply
the
same
template
to
different
combinations
of
tools
without
having
to
create
a
new
pipeline
from
scratch,
and,
let's
dive
into
like
a
real
world
example
that
talks
through
what
that
looks
like
using
the
jenkins
templating
engine
so
step,
one
defining
our
software
delivery
process
in
the
upper
left
in
the
upper
right.
I
have
two,
admittedly
contract
pipelines
to
demonstrate
the
concept
here
on
the
left.
We
have
a
application
using
maven
as
their
build
tool
and
on
the
right.
A
A
We
are
going
to
do
our
build
with
gradle
and
we're
also
going
to
do
static
code
analysis.
You
know
the
big
aha
moment
here
is
that,
regardless
of
the
build
tool
being
used,
the
process
is
largely
the
same.
We're
going
to
do
a
build
now
we're
going
to
do
static
code
analysis.
So,
on
the
bottom
half
of
the
screen,
you
know
you
can
see
a
reusable
pipeline
template
that
calls
out
those
two
steps
build
in
static
code
analysis.
A
So
wouldn't
it
be
great
if,
instead
of
needing
to
define
my
pipeline
for
each
of
these
teams
individually,
I
could
put
you
know
in
a
central
source
code
repository.
This
is
our
reusable
pipeline.
Template
build
and
static
code
analysis
and
then
we're
going
to
plug
and
play
with
each
team
gets
to
pick.
Are
you
using
maven
or
are
you
using
gradle,
and
I
can
enforce
that
you're
going
to
use
sonarcube
from
a
static
code,
analysis
perspective.
A
So
how
do
we
make
that
possible?
You
know
the
first
step
on
the
left.
We
have
a
pipeline
configuration
repository,
so
this
is
a
central
location
where
you're
able
to
define
your
common
pipeline
template
and
some
libraries
so
that
pipeline
configuration
directory
has
a
jenkins
file.
This
jenkins
file
is
that
reusable
pipeline
template.
We
talked
about
on
the
last
slide,
where
it's
going
to
say,
build
and
static
code
analysis
in
the
center.
A
We
have
what
are
called
libraries,
so
we
have
a
build
step
in
our
template
that
has
two
implementations
from
maven
and
gradle,
and
then
we
have
a
static
code
analysis
step
from
center
cube.
If
we
look
at
our
library's
structure,
you'll
see
that,
just
as
you
would
expect,
there's
a
library
for
maven,
a
library
for
gradle
and
a
library
for
sonarcube.
A
The
maven
and
gradle
libraries
have
a
build.groovy
file
and
the
center
cube
directory
has
a
static
code.
Docker
ruby
file,
these
files,
the
base
name
of
the
file,
becomes
the
step
name
when
these
libraries
are
loaded.
If
we
look
at
the
center,
we
can
see
the
source
code
within
those
the
steps,
as
we
call
them.
A
This
pipeline
code
is
the
exact
same
pipeline
code.
You
would
have
written
before
just
modularized
in
a
way
and
wrapped
in
a
call
method
so
that
it
can
be
plug
and
played
right,
or
rather
we
can
pull
in
different
libraries
to
the
same
template
to
get
different
implementations
of
the
pipeline
in
the
bottom
right.
There's
pipeline
configuration,
so
it's
specifying
you
know
from
a
library's
perspective.
A
I
want
to
load
maven
and
center
cube
from
the
griddle
apps
perspective.
I
want
to
run
griddle
in
sonarqube,
so,
let's
step
forward,
so
these
libraries
become
the
building
blocks
of
your
dev
tech,
apps
pipeline
within
your
organization
and
they're
only
as
valuable
as
they
are
reusable
right.
So
how
do
we
make
that
possible
when
you
load
your
libraries
on
the
left
separate
from
your
pipeline
template?
A
A
A
Let's
talk
about
how
we
can
govern
these
pipeline
configuration
files
a
bit
more,
so
you
know
each
repository
needs
to
end
up
with
a
pipeline
configuration
that
says
which
build
tool
to
use
and
the
fact
that
they're
using
sonar
key.
We
don't
want
every
team
to
have
to
define
the
fact
that
they're
using
sonar
cube.
We
want
to
be
able
to
enforce
that
from
a
compliance
or
governance
perspective.
A
We
want
to
be
able
to
say
you
have
to
use
sonar
cube,
but
tell
me
what
build
tool
to
use
the
way
we
can
achieve
that
in
the
jenkins
templing
engine
is
through
hierarchical
pipeline
configuration
files.
So
the
way
that
works
is,
I
can
define
in
a
central
location.
This
is
the
pipeline
configuration
everyone
has
to
inherit.
A
In
our
example,
we're
saying
that
everyone
has
to
use
the
sounder
cube
library
so
by
default
in
the
jenkins
templing
engine.
That
means
that
teams
aren't
going
to
be
able
to
override
the
fact
that
they're
using
sonar
cube.
But
if
you'll
notice
in
the
global
pipeline
configuration,
we
have
an
annotation
there
that
says
at
merge.
A
So
what
that
does
is
it
says?
You
know
development
teams
are
going
to
be
allowed
to
add
additional
libraries
to
this
pipeline
configuration.
You
can't
change
the
fact
that
you're
using
sonar
cube,
but
you
can
add
your
build
tool
to
the
library's
block
at
runtime
when
the
pipeline
kicks
off.
We
aggregate
these
pipeline
configurations
together
and
we
end
up
with
the
resultant
aggregated
pipeline
configuration
that
the
application
needs
right.
So
one
team
is
using
maven
and
center
cube.
The
other
team
is
using
gradle
and
center
cube.
A
A
What
is
our
team's,
our
organization's
software
delivery
process
right
and
that
can
look
like
build
scan,
publish
deploy
in
practice.
These
can
get
more
sophisticated
to
call
out
branching
strategies
in
practice.
You
can
have
multiple
pipeline
templates
for
your
organization
and
let
teams
choose,
but
the
idea
is
that
we're
going
to
define
them
in
a
central
location
and
teams
will
inherit
these
templates.
A
So
what
types
of
things
can
we
templatize
right?
What
can
I
inject
into
that
pipeline
template
to
make
it
mean
something,
and
we
have
our
steps
like
build
and
static
code
analysis
and
there's
other
objects
that
can
be
created
that
in
our
framework
we
call
the
pipeline
primitives.
So
there
are
steps,
stages,
keywords
and
application
environments
that
can
be
injected
at
runtime
to
the
pipeline
template.
A
So
how
do
I
define
those
things
in
my
pipeline
configuration
file
and
in
the
bottom
right?
We've
got
a
couple:
the
different
blocks
that
you
can
configure
to
define.
What
is
your
template
going
to
do
and
then,
where
do
I
define
this
pipeline
configuration
and
where
do
I
define
this
template
that
lives
in
your
jenkins,
folder
settings?
A
So
you
can
have
libraries
and
pipeline
configurations
applied
globally
jenkins
wide
and
then
the
manage
jenkins.
You
know
settings
page
set
that
up
and
then
you
can
create
an
arbitrarily
complex
folder
structure
to
map.
Whatever
governance
hierarchy
you
need
so
in
every
folder
in
jenkins,
you
can
say:
here's
another
pipeline
configuration
that
applies
to
this
level
of
the
hierarchy.
A
The
teams
or
jobs
within
this
folder
are
going
to
inherit
and
then,
finally,
you
can
have
the
application
specific
pipeline
configuration
file
which
can
live
at
the
root
of
the
team's
source
code
repository.
So
through
this
approach,
you
can
set
up
as
complex
a
governance
hierarchy
as
you
have
a
permissions
hierarchy
within
your
organization.
A
If
some
applications
are
super
critical
and
they
shouldn't
be
allowed
to
change
anything,
you
can
create
a
folder
with
more
locked
down
permissions
on
those
pipeline
configuration
files
than
something.
That's.
You
know
a
rapid
prototyping,
not
production,
facing
application,
yet
that
might
need
some
more
lenient
permissions
at
first.
A
So
what
are
the
the
key
takeaways
here?
The
first
is
that
the
jenkins
tumbling
engine
is
a
framework
for
developing
tool.
Agnostic
templated
workflows
that
can
be
reused
by
multiple
teams
simultaneously,
regardless
of
the
tools
being
used
right
and
the
reason
that
provides
value
is
because
most
ci
cd
tools
in
most
organizations
are
defining
pipelines
on
a
per
application
basis,
which
leads
to
a
ton
of
complexity.
A
The
second
key
takeaway
here
is
that
this
approach
separates
the
business
logic
defined
in
your
pipeline,
template
from
the
technical
implementation
of
your
steps
of
your
pipeline
through
pipeline
libraries,
allowing
teams
to
configure
their
pipelines
instead
of
build
them
from
scratch
right
separating
business
logic
and
technical
implementation
is
a
software
best
practice.
We've
talked
about
for
years,
just
hasn't
made
its
way
to
pipeline
development,
yet
at
a
large
scale
for
some
reason
right
and
then.
A
Finally,
through
the
jenkins
sampling
engine,
you
can
apply
organizational
governance
by
defining
these
reusable
templates
that
incorporate
security
all
right,
while
making
sure
that
developers
can't
bypass
these
checks.
You
can
optimize
pipeline
code
reuse
through
these
plug-and-play
libraries.
In
a
way
that
is
scalable
and
then
finally,
you
can
simplify
pipeline
maintainability.