►
From YouTube: Protect the Pipe! A Policy-based Approach for Securing CI/CD Pipe... Shripad Nadgowda & Jim Bugwadia
Description
Protect the Pipe! A Policy-based Approach for Securing CI/CD Pipelines - Shripad Nadgowda, IBM Research & Jim Bugwadia, Nirmata
Modern applications are composed of hundreds of packages and delivered to production via automated CI/CD pipelines. With rapid delivery comes the growing risk of attacks, vulnerabilities, and misconfigurations. Protecting these critical assets requires policy-based controls for CI/CD pipeline composition, configurations and execution. In this session, Shripad and Jim will present a cloud-native security framework for Tekton pipelines using in-toto, Kyverno and sigstore. They will discuss the unique security challenges for CI/CD pipelines, and then demonstrate the use of open-source tools to attest and verify each pipeline resource and execution step using declarative policies.
A
B
B
B
Yeah,
so
why
we
are
talking
about
circuit
right,
so
if
we
revisit
our
software
supply
chain,
so
it
starts
with
the
developer
writing
code,
pushing
it
to
some
source
controller
systems
like
github
gitlab,
which
then
eventually
triggers
our
ci
cd
right
and
cf
pipeline.
Specifically,
we
run
multiple
devsecops
checks.
We
run,
we
have
git
client,
we
run
some
static
scan
we
generate
as
bomb.
We
build
our
artifact
and
finally,
we
push
them
right.
So
in
this
we
push
and
deploy
our
application
on
the
runtime
now,
in
this
whole
spectrum
right.
B
Why
it
is
critical
is
because
there
is
a
growing
open
source
ecosystem
around
ci
cd
pipeline,
where
multiple
ready-to-use
c
tasks
are
getting
basically
being
available
on
into
marketplace
like
tecton,
catalog
or
github
action
marketplace
and
just
like
we
are
securing
our
opens,
our
open
source
dependencies
for
applications.
We
need
to
pay
attention
to
basically
securing
these
ci
cd
open
source
element
as
well
right
and
the
other
aspect
is
our
ci
cd
pipelines.
B
They
have
access
to
a
lot
of
critical
user
credentials
right,
for
instance,
we
have
github
client
which
need
access
to
our
git
git
or
token,
so
it
can
basically
push
some
changes
or
create
a
pull
request
automatic
to
automate
some
of
the
tasks
we
have
our
build
system
which
essentially
has
access
to
our
registry
credentials.
So
it
can
push
images,
so
they
have
access
to
this
lot
of
credential
critical
information
credentials.
B
So
we
need
basically
right
way
of
securing
our
pipelines,
so
it
can
cannot
be
compromised,
and
now,
in
today's
talk,
we
are
going
to
primarily
talk
about
four
technologies
right
and
how
they
come
together.
We
are
going
to
talk
about
tech,
town
as
a
ci
cd
platform.
But
again
this
the
scope
of
the
stock
is
not
limited
to
tecton
right.
It
applies
equally
to
other
ci
platforms
like
jenkins
or
github
actions.
B
So
first
talking
about
tech,
town
right,
it's
a
cloud
native
city
building
blocks.
It
runs
natively
on
kubernetes.
It
has
a
bunch
of
crds
like
it
does.
We
have
pipelines,
we
have
tasks
pipeline
encode
the
task
layout
like
how
what
is
the
workflow,
that
you
want
to
execute
tasks,
encode
the
execution
logic
we
have
taskrun
pipeline
run,
which
encode
the
execution
trigger
properties
and
tecton
again
has
multiple
dimensions
right.
So
one
dimension
is
tektron
chains,
which
is
a
new
project
which
has
about
secure
the
supply
chain,
security
of
pipelines
themselves.
B
It
creates
an
automated
attestations
of
the
task
runs
as
they
execute.
Then.
As
I
said,
there
is
open
source
catalog
for
ready
to
use
text
on
task,
and
then
there
is
one
element
that
we
are
going
to
talk
about.
More
in
detail
is
tecton
bundle
like
it
allows
basically
us
to
package
and
distribute
this
tecton
task
and
share
them
as
an
through
an
oc
add
as
an
oc
artifact
the
other
project.
Again
in
total,
it's
an
open,
extensible
metadata
standard.
B
B
This
is
basically
emerging
as
this
public
infrastructure
for
signing
software
artifact
and
in
this
work
we
are
essentially
using
it
to
sign
various
tectonics
resources,
which
includes
our
pipeline
definitions,
configurations,
policies,
images
we
sign
them
using
cosign.
We
store
those
artifacts
into
artifact
store.
Well,
we
are
using
oci
as
the
artifact
store
for
storing
all
this,
and
it
does
the
six
store
it's
a
basically,
it
has
broader
scope.
It
has
record
for
transparency,
log
full
co
for
key
management,
and
then
we
are
qrno.
C
So
kiverno
is
a
policy
engine
that
was
designed
for
kubernetes
and
what
it
does
is
it
kind
of
uses
kubernetes
as
declarative
configuration
management
system
to
manage
policies
itself.
So
the
goal
of
kiverna
was
not
just
enforcement
or
validation
or
checking
of
resources,
but
also
to
automate
security
right
so
to
be
able
to
generate
resources.
Mutated
resources
in
a
first
class
manner
was
very
important
to
kiberno
from
the
beginning
and
as
you'll
see
in
the
examples.
This
gives
you
a
very
powerful
toolbox
to
use
similar
patterns
that
you
would,
with
any
kubernetes
controller
any
kubernetes.
B
B
Here
we
go
yeah,
so
we
essentially
came
up
with
this
deciduous
attack
threat
modeling
for
the
takedown
right,
and
I
think
we
are
probably
familiar
with
the
legend,
so
the
boxes
in
the
in
the
great
they
are.
They
basically
represent
the
facts,
the
pink
boxes
they
represent
the
attack
vector
and
then
the
blue
essentially
represent
how
we
can
secure
them
yeah.
So
for
tecton.
If
you
look
into
this
it
as
we
just
discussed
right,
it
has
the
starts
with
the
pipeline,
the
definitions
right.
B
We
have
the
pipeline
definitions
that
encodes
the
workflow
like
what
are
the
tasks
that
you
can
be
executed?
What
is
the
order
in
which
this
task
can
be
executed,
then
each
task?
It
has
a
certain
steps
like
set
of
steps
that
encode
the
execution
logic
like
in
this
particular
task.
I
want
to
execute
these
particular
steps
each
step.
It's
run
as
this
container
as
a
the
pod
and
on
the
kubernetes
cluster
right
and
in
this
particular
workflow
or
in
this
particular
spectrum.
There
are
various
attacks
that
are
possible.
C
So
what
we
did
over
here,
like
you
know,
sripad
mentioned.
We
have
three
major
facts
right,
so
you
have
your
pipelines,
you
have
your
tasks
and
then
you
have
your
steps.
All
of
these
are
operating
as
native
kubernetes
resources.
Each
one
is
going
to
you
know,
run
several
containers.
Each
step
runs
as
a
containerized
image,
so
every
possible
attack
you
can
imagine
on
kubernetes
also
applies
to
tecton
and,
of
course,
now
you're,
because
you're
building
your
software
itself,
it's
almost
exponentially
critical
here
right.
C
So
some
of
the
things
we
did-
and
this
is
not
a
complete
picture
right-
so
don't
take
this
as
a
representation.
You
can,
you
know,
kind
of
take
and
put
into
production,
but
the
idea
here
is
to
come
up
with
okay.
What
can
we
enforce?
What
should
we
check-
and
you
know
the
exercise
the
tripod
and
I'm
going
through
is?
Can
we
convert
these
into
policies
expressed
by
kiverno?
So
the
first
thing
we
check
for
is
to
make
sure
that
every
ev
anything
that
executes
is,
you
know
executing
from
a
pipeline
bundle
right.
C
So
that's
that's
one
policy.
The
next
thing
we
checked
is
to
make
sure
that
you
cannot
just
run
tasks
without
pipelines.
Now
tecton
like
kubernetes
again,
is
very
flexible.
It
lets
you
run,
you
know
just
like
in
kubernetes.
If
you
want
to
run
a
pod,
you
can,
but
it's
not
a
good
idea.
You'd
rather
run
a
deployment
or
some
other
controller.
So,
similarly
in
here
the
idea
is
instead
of
running
tasks.
You
run
a
pipeline
and
better
yet
run
a
pipeline
with
a
signed
bundle
which
is
securely
stored
in
your
oci
registry
right.
C
So
those
are
some
policies
up
there,
but
then
every
step,
like
I
mentioned,
runs
as
a
container.
So
at
that
point
you
want
to
make
sure
that
that
step
is
secured.
It
has
the
right
pod,
security
or
well,
in
this
case,
step
security
context,
or
you
know
the
same
definitions
that
you
would
apply
to
a
container
like
run
as
non-root.
C
Don't
elevate,
privileges,
things
like
that
you
want
to
enforce
for
that
step.
Similarly,
you
know
in
in
a
kubernetes
environment,
for
your
workloads.
You
would
run
each
workload
in
a
namespace.
Hopefully
we
all
know
that's
the
best
practice
to
use
namespaces
for
segmentation
so
similar
year.
What
we
did
is
we
applied
namespacing
to
say
every
time
you
execute
your
pipeline,
it
should
be
in
its
own
namespace
right
so
and
once
you
have
a
namespace,
you
can
generate
security
defaults.
So
going
back
to
what
I
mentioned
about
caverno
and
the
fact
that
it's
to
automate
security.
C
So
why
not
generate
those
defaults
instead
of
having
to
manually,
configure
them
or
you
know,
come
up
with
some
other
processes
for
them
right
and
then.
Finally,
you
know,
besides
checking
for
signatures,
we
also
came
up
with
policies
with
check
for
attestations
in
in
total
format,
and
so
this
is
to
verify
that
the
images
actually
have
a
scan
performed.
C
You
can
even
check
how
recently
the
scan
was
performed
or
what
type
of
vulnerabilities
you
might
want
to
allow
or
prevent,
right
or
or
block
from
executing
and
then
of
course,
signatures
for
all
the
images
was
the
final
policy
you
see
over
here
as
a
leaf
right.
So
that's
the
kind
of
attack
tree
that
we
came
up
with
now.
Of
course
again.
C
C
Yeah,
so
I'm
not
going
to
cover
this
policy
as
well.
I
already
spoke
about
them
briefly,
but
let's
go
to
the
demo
and
see
this
in
action
itself
right.
So
the
first
thing
we're
going
to
do
is
I'm
going
to
actually
kick
off
a
pipeline
because
it
takes
a
while
to
run
this,
and
you
know
I
guess,
with
different
wi-fi
speeds,
etc.
This
might
take
longer.
So
let
me
go
and
like
I
mentioned
on
my
cluster
right,
so
we
already
have
tecton
and
other
things
installed,
including
kiverno.
C
C
Outside
of
a
namespace
right,
so
instead
of
just
creating
a
namespace,
let
me
just
do
I'm
going
to
say
in
the
sign
pipeline
I'll
show
you
what
this
is.
If
I
go
into
all
of
the
declarations
and
by
the
way
we
have
this
available
as
a
git
repo,
so
you
can
look
at
this
later.
If
I
go
in
here
and
look
at
sign
pipeline,
it
has
a
run
step
and
the
run
step
is
very
simple
right.
All
it's
saying
is
in
this
pipeline.
C
I
want
to
execute
it
from
this
bundle,
and
you
know
it
doesn't
have
any
namespace
or
anything
defined,
so
I
can
just
run
it
in
any
namespace.
So
if
I
try
this,
you
know
what
I'm
expecting
to
happen
is
it
gets
blocked.
It
says:
no,
I
need
a
namespace
right,
so
let's
go
ahead
and
create
a
namespace
for
this,
and
I
think
I
already
have
something
called
run
so
I'll
just
call.
It
run
two.
C
So
I
see
that
there
are
already
some
things
populated
for
me.
Like
secrets,
you
know
there
should
be
a
role,
a
role
binding.
So
this
is
everything
this
pipeline
needs
to
execute
and
kiverno
has
automated
that
so
the
way
that
that
got
you
know
populated
if
I
go
into
my
policy
set
that
I
have
over
here
for
security.
C
C
Some
of
these
policies,
which
are
you
know,
actually
generating
even
things
like
pvcs
right
so
techton
shares
information
through
volumes
right.
So
you
can
configure
this
in
many
different
ways,
but
here
what
we're
doing
is
we're
creating
a
volume
and
we're
you
know
creating
the
pvc
in
there
to
make
it
easy
to.
You
know
to
now
execute
that
pipeline.
So
now
that
we
have
that
in
here,
if
I
I
haven't
executed
my
pipeline,
so
let's
go
ahead
and
do
that.
C
And
we
can
look
at
this
also
from
the
tecton
dashboard
right.
So
let's
go
back
here.
If
we
look
at
it
as
pipeline
runs,
we
should
see
there's
a
new
pipeline
run
and
it
has
started
executing.
So
these
are
the
different
steps
that
are
in
the
pipeline,
so
maybe
tripod.
If
you
want
to
quickly
explain
what
this
pipeline
does.
B
Yeah,
so
this
is
yeah,
so
this
is
essentially
a
template
pipeline
that
we
have
created
and
it
it
includes
a
typical
steps
that
we
see
in
most
common
pipelines
right-
and
this
is
again
shorter
version
of
it.
So
it
starts
with
cloning,
the
repo
which
essentially
is
getting
the
artifact
into
the
workspace.
B
Then
we
run
some
static
scans
on
this
like
gosek,
which,
if
it
is
a
go
application,
it
checks
for
it
statically
or
check
for
vulnerabilities
in
the
code.
Then
we
have
okay,
this
is
not
showing,
so
we
have
build
and
push
then
once
we
pass
those
it
essentially
build
and
push
the
registry
the
image
to
ocr
registry,
it
signs
them
with
cosine
and
it
also
add
the
attestations
right.
So
we
see
this,
there
is
generate
bomb.
We
generate
the
s
bomb
again,
attach
it
to
the
to
the
build
image.
B
It
runs
the
vulnerability,
scan
and
again
create
attestation
for
it.
And
finally,
if
it
sees
there
are
no
vulnerabilities,
all
the
test
stations
have
been
created
properly.
Then
it
goes
goes
ahead
and
deploys
it.
This
particular
application
on
onto
the
cluster
again.
This
is
not
meant
to
be
used
in
production,
but
this
is
just
to
give
a
context
and
some
scope
for
us
to
basically
play
around
with
this
demo,
all
right
so
yeah.
It
should
take
maybe
a
couple
of
more
minutes
to
finish
but
yeah
yeah.
C
So
typically,
I
think
the
last
time
I
tried
it,
it
took
about
five
to
seven
minutes
to
go
through
all
the
checks
and
obviously,
as
you're
adding
some
policies.
You
know
there
is
some
time
added
to
fetch
data,
because
some
of
this
data
comes
from
oci
registries
and
let
me
quickly
show
you,
while
that's
going
on
what
some
of
these
policies
look
like
right.
C
So
if
we
go
back
to
the
policies,
one
of
these
is
just
to
require
a
bundle
so
that
we
we
already
you
know-
and
we
can
try
this
by
running
a
task
directly
to
see
if
it
gets
blocked
or
not.
But
basically,
as
you
can
see,
there's
a
very
simple
policy
which
is
saying
you
know
everything
should
run
into
here
with
a
bundle
and
don't
allow
you
know,
bear
tasks
to
be
executed
right.
So,
let's,
let's
give
that
a
try
I'll.
We
don't
need
to
watch
this.
C
And
what
I
want
to
do
is
in
my
policies.
I
have
some
test
resources
and
if
I
look
at
that,
what
I
want
to
do
is
to
just
say
you
know
I'll
just
use
the
sample
test
run
to
try
and
run
that
right
and
again.
What
I'm
expecting
here
is
that
this
is
gets
blocked,
because
it's
saying
first
of
all:
well
it
got
that.
There's
no
signature
on
it,
so
that
policy
blocked
it
and
it
would
if
that
passed.
C
It
would
also
be
blocked
to
make
sure
that
you
know
it
has
to
come
from
a
in
a
sine
bundle
which
is
stored
in
our
oci
registry.
So
going
back
to
again
the
set
of
policies
there's
this
other
set
which
are
now
verifying
signatures
right
and
what
that
looks
like.
So
it's
a
pretty
interesting
scheme,
because
what
you're
trying
to
do
is
you're
enforcing
a
policy
so
by
the
way
this
pop-up,
that
shows
is
the
vs
code
integration,
because,
given
everything
everything's
a
crd,
you
get
your
help
right
in
here.
C
So
that's
a
nice
little
value
add
right!
So
we're
checking
signatures
here
and
the
one
thing
which
is
a
little
bit
complex
is
you're
not
really
running
a
pod
right.
So
it's
one.
It's
simple
enough
to
check
a
signature,
well,
not
exactly
simple,
but
it's
possible
to
set
a
say
or
check
a
signature
on
a
pod.
But
here
what
ki
verno
is
doing.
C
Is
it's
using
this
new
feature
which
we
introduced
in
1.7
to
extract
images
out
of
the
configuration
in
this
case
it's
looking
into
the
spec
of
the
pipeline
reference
and
it's
going
to
check,
for
you
know
a
value
with
call
bundle
and
the
name,
and
then
it's
going
to
check
that
that
bundle
was
signed
using
the
key
that
you're
declaring
over
here.
So
now,
this
key
could
be
in
a
config
map
or
somewhere
else,
but
ideally
what
you
now.
What
this
policy
is
doing
is
it's.
You
know
going
through.
C
All
of
the
you
know
the
actual
runtime
configurations
it's
looking
in
there
and
into
the
spec
and
checking
for
all
of
the
you
know
the
sign,
bundles
and
then
verifying
the
signature.
So
similarly,
here
this
one's
a
little
bit
different,
because
now
we're
looking
at
task
runs
so
remember:
there's
pipelines,
there's
tasks
and
there's
steps
and
each
one
needs
to
be
verified.
You
want
to
make
sure
each
one
can
be
trusted
and
executed.
So
similarly,
we're
checking
whether
the
task
reference
is
coming
from
a
signed
pipeline.
C
And,
finally,
you
know
for
the
steps
here
there's
a
check
to
say
that
for
each
step
make
sure
that
if
once
you
extract
the
image,
the
image
is
also
signed
and
verified
right
and,
in
addition,
we're
checking
for
a
predicate
one
of
these
things.
I've
commented
this
out
because
we
actually
found
that
one
of
the
step
has
a
high
severity
vulnerability,
so
that
was
blocked
and
there
are
you
know,
ways
to
kind
of
you
know
allow
that
exclusion,
but
what
we
did
for
this
demo
is.
C
We
just
commented
this
out,
but
we're
checking
that
every
you
know
step
here.
It
has
an
attestation
which
is
in
you
know,
which
is
actually
a
vulnerability
scan
created
in
this
case
by
gripe
right.
So
you
could
use
whatever
tool
you
want
and
then
you
can
write
if
you
were
to
kind
of
check
for
vulnerabilities
in
the
policy.
C
It
would
look
something
like
this,
where
you're
looking
inside
the
json
data
and
checking
for
that
payload
of
that
attestation
or
in
fact,
in
the
predicate,
to
make
sure
that
you
know
you're
in
this
case,
we
want
to
make
sure
that
there's
no
severity
higher
than
eight
available
or
in
that
report
itself
right.
So
those
are
all
of
the
things
that
would
be
enforced.
C
One
last
thing
I
want
to
show
and
we'll
check
on
our
pipeline
too
is:
I
want
to
actually
run
an
unsigned
pipeline
or
a
pipeline
which
is
not
from
a
valid.
You
know
repo
right,
so
what
I
did
was
I
just
copied
this
pipeline
that
shri
sripat
created,
and
I
pushed
it
into
my
registry,
my
private
registry-
or,
I
guess
my
github,
you
know
registry,
and
I
want
to
try
and
execute
that
just
to
see
what
happens
right.
So,
let's
go
ahead
and
create
another
namespace.
C
Is
we're
going
to
run
now
this
unsigned
pipeline
right
so
just
to
show
what
exactly
that
is.
So
if
I
go
back
to
here,
this
is
our
sign
pipeline
and
if
we
go
to
unsigned
what
we
should
see
is
it's
using
a
different
image
registry
to
run
execute
this
pipeline
bundle
right.
So
what
should
happen
is
again
because
of
that
you
know
missing
signature
check
that
should
get
blocked.
C
Yeah,
so
here
we're
seeing
one
of
the
you
know,
because
we
are
using
a
public
key
to
match
it,
saying:
there's
no
valid
signatures,
which
internally
kiverno
integrates
with
cosign
from
six
store.
It's
making
that
call
and
validating
that
pipeline
bundle
itself
right,
so
that's
more
or
less
how
you
would
go
ahead
and
secure
this
entire
pipeline,
as
you
can
see
here
where
it's
still
continuing
it's
towards
the
final
steps
of
it
completing
so
again,
depending
on
you
know,
because
it
pulls
a
lot
of
information
these.
C
C
B
Yeah,
so
I
mean
to
summarize
here
we
are
talking
about
techton
right
but,
as
we
mentioned
right,
it's
applicable
to
other
ci
systems,
but
techno
is
a
powerful,
ci
cd
solution
right,
it's
built
and
operated
natively
on
kubernetes
ci
system
needs
to
be
secure
right.
We
need
to
treat
them
just
like
our
production
workload
and
not
just
the
integrity,
but
also
the
configurations
we
need
to
make
sure
they
are
configured
properly.
B
Whatever
we
are
executing,
they
are
signed,
they
can
be
trusted
and,
of
course,
the
kiwano
policies
are
the
perfect
fit
to
achieve
this
right,
which
allows
us
to
perform
the
validation,
mutate,
generate
and
verify
images,
and
all
this
demo
it's
available
on
the
github.
So
if
you
want
to
try
it
out,
it
has
instructions.
All
the
policies
are
there.
You
can
try
any
closing
notes.
Jim.
C
Now,
let's
just
check
one
more
time
to
see
if
the
pipeline
completed,
but
otherwise
you
know,
I
think
we're
done
with
what
we
wanted
to
demo
and
happy
to
take
any
questions.
I
think
it
actually
completed.
Yeah
all
the
tasks
are
done.
The
pipeline
is
done
so
there
you
go
yeah
all
right.
Any
questions,
thoughts,
feedback.