►
From YouTube: Baking Security into your DevOps Supply Chain with Argo CD & OPA - Dan Garfield (DevOps Experience)
Description
Argo CD has become the defacto entry for engineering teams to deploy and manage their applications while Open Policy Agent has become one of the most reliable security policy enforcement engines. These two tools work perfectly together and yet there are very few articles and videos on the subject! In this talk, we look at the best strategies for getting these tools to work together to improve the software delivery supply chain. We’ll look at admission hooks, sync waves, and other ways to ensure your applications and clusters are always running in a secure and safe manner. Including example files and reference repos.
A
A
So
if
somebody
were
to
bypass
the
cicp
pipeline,
they
get
around
it.
So
this
is
going
to
be
a
default
security,
creating
a
default
security
using
these
really
cool,
Cool
Tools.
All
of
these
are
open
source
tools.
You
can
use
so
to
kick
this
off.
Let
me
introduce
myself.
My
name
is
Dan
Garfield
I
am
the
co-founder
and
chief
open
source
officer
of
code
fresh.
We
are
a
software
delivery
company.
We
are
the
Argo
Enterprise
company.
We
have
a
lot
of
expertise,
implementing
git
Ops,
implementing
Argo,
implementing
cicd
implementing
software
delivery.
A
We've
been
around
for
about
five
or
six
years
and
I've
been
very
proud
of
this
team
and
what
we've
built
so
definitely
check
out
our
platform,
because
we
have
some
really
great
tooling
for
you
to
use
to
deploy
software
and
deliver
securely
and
at
scale
making
your
developers
productive.
All
those
kinds
of
things
you
can
follow
me
on
Twitter
at
today
was
awesome
and
then
I
am
a
maintainer
on
the
Argo
project.
Now,
truthfully,
I
don't
write
that
much
code
every
once,
while
I
do
a
little
bit
of
code
contribution.
A
Most
of
the
work
that
I'm
doing
is
helping
organize
and
and
organize
contributions
from
our
developers,
helping
organize
in
the
community
helping
build
our
community
events
and
all
those
kinds
of
things
so,
while
I
do
contribute
every
once
in
a
while
in
the
code
base.
Most
of
the
work
that
I'm
doing
is
in
the
community
side
of
things
and
on
helping
organize
work
and
build
governance,
and
all
this
kind
of
stuff
I'm,
also
a
maintainer
co-founder
and
co-chair
on
open,
get
Ops
and
the
get
Ops
working
group.
A
A
So
with
that
when
I
was
a
kid,
I
had
a
little
bit
of
a
side.
Business
fixing
computers,
setting
up
networking
doing
these
kinds
of
things
and
one
of
these
jobs
actually
paid
me
in
a
laptop
and
so
for
the
first
time
as
a
teenager,
you
know
I
couldn't
drive
yet
I
had
a
laptop
I
had
mobile
Computing.
It
was
a
blew
my
mind.
Battery
would
last
about
40
minutes,
and
so
what
I
would
do
is
I
would
go
out
on
my
skateboard
and
allegedly
I'm,
not
saying
I.
A
Did
this
I'm
saying
allegedly
I
would
go
out
of
my
skateboard
I'd
put
in
my
headphones
with
my
laptop
and
I
would
have
it
ping
and
tell
me
when
there
were
Wi-Fi
networks
and
allegedly
I
would
use
my
laptop
to
see
if
there
was
a
way
that
I
could
bypass
these
and
a
lot
of
people?
If
you're
familiar
with
this
era,
there
was
WEP
encryption.
A
Basically,
what
you
could
do
is
you
could
get
in
the
middle
of
the
of
the
Wi-Fi
router
and
the
clients,
and
you
could
trigger
the
reauthentication,
the
reissuing
of
keys,
and
then
you
could
Snoop
on
that
traffic
and
you
could
use
it
to
steal
the
Wi-Fi
password
now
I'm,
not
saying
I.
Did
this
I'm
saying
allegedly
I
skateboarded
around
and
did
this,
so
this
is
not
an
admission
of
guilt
that
could
be
used
in
a
court
of
law.
I
want
to
be
very
clear
about
that.
A
But
what
this
taught
me
is
that
when
it
comes
to
security,
if
you
can
get
into
the
middle
of
a
process,
you
can
usually
own
the
process.
So
that's
that's
kind
of
some
of
the
stuff
that
we're
going
to
be
talking
about
today,
we're
going
to
be
trying
to
solve
a
couple
of
problems.
So
the
first
problem
is:
do
you
know
where
your
software
comes
from
it's
eight
o'clock
at
night?
A
Do
you
know
where
your
software
comes
from,
and
the
challenge
here
is
most
processes
that
we
have
involve
us
building,
artifacts,
pushing
them
to
Registries
and
then
later
referencing
those
artifacts
pulling
those
from
the
Registries
and
having
them
deployed.
Now?
If
someone
were
able
to
get
in
the
middle
and
put
something
to
replace
your
artifact,
you
might
not
know
about
it.
You
might
deploy
something
that
has
a
vulnerability
into
it.
At
the
same
time,
if
you
baked
the
image
originally
and
signed
it
with
vulnerabilities,
you
also
have
problems.
So
that's
problem
number
one.
A
So
there's
actually
a
way
that
we
can
make
sure
these
things
are
are
haven't
been
intercepted
and
there's
also
ways
that
we
can
make
sure
that
maybe
a
programmer
who
has
access,
maybe
their
their
machine
is
compromised.
We
can
make
sure
that
they
can't
necessarily
push
artifacts
that
are
going
to
be
valid
and
deployed
into
our
servers,
and
this
is
also
a
use
case
that
I've
seen
in
the
real
world
in
the
wild
I've
seen
a
company
that
had
software
engineer
had
their
machine
compromised,
they
clicked
on
a
phishing
email.
A
It
installed
a
little
bit
of
a
root
kit
and
then
suddenly
they
started
injecting
they
basically
just
started,
injecting
a
bunch
of
malicious
code
into
every
code
base
that
they
had
access
to.
The
engineer
had
no
idea,
so
you
can
actually
catch
stuff
like
this.
With
this
security
approach
that
we're
going
to
talk
about
the
second.
The
second
problem
that
we're
going
to
try
to
solve
is
how
we
enforce
rules
on
our
cluster.
So
a
lot
of
people
approach
this
with
training.
A
They
try
to
enforce
Automation
and
they
try
to
have
audits.
I
have
really
interesting
conversations
with
people,
sometimes
where
they'll
say
things
like.
Oh,
we
never
touch
things
in
production
by
hand
that
would
be
out
of
process.
People
know
not
to
do
that,
so
we
know
it
doesn't
happen
and
I
I
love
this
level
of
trust
that
we
have
for
each
other.
I
mean
it
just
must
be
a
wonderful
organization,
so
utopian
that
you
don't
have
anybody
bypassing
the
rules.
I
actually
saw
a
company
who
had
a
similar
approach
years
ago.
A
They
actually
Unfortunately.
They
had
to
go
through
some
layoffs
and
the
team
that
was
maintaining
a
core
piece
of
software
had
been
laid
off
and
suddenly
you
know
a
month
later
that
service
went
down
so
when
people
went
to
go
debugging
and
figure
out.
What's
going
on,
they
found
out
that
none
of
the
code
changes
have
been
checked
in
over
the
last
eight
months.
A
Suddenly,
suddenly,
the
debug
challenge
went
from
a
10
to
100.,
so
it's
really
good
to
have
audits
in
place
and
have
processes
in
place
that
literally
don't
allow
for
people
to
bypass
the
right
way
to
do
things,
and
certainly
you
might
have
some
sort
of
break
glass
procedure
that
involves
multiple
people
getting
involved.
But
you
want
to
make
sure
that
that
the
default
path
is
for
people
to
do
things.
The
correct
way:
that's
what
this
talk
is
about
doing
things
correctly
by
default
and
there's
another
interesting
data
point
for
you.
A
A
study
was
done
to
figure
out
how
they
could
get
more
people
to
opt
into
being
organ
donors.
That's
a
donating
organs
saves
a
ton
of
lives.
It's
super
important
and
if
more
people
donate
organs,
they
they
more
people's
lives
will
be
safe.
So
what
they
did
is
they
started
saying?
Do
you
want
to
opt
in
and
then,
when
they
said,
do
you
want
to
opt
out?
The
default
was
to
donate
your
organs.
A
Suddenly,
everybody
was
willing
to
donate
their
organs,
because
if
people
had
to
think
about
it
to
opt
in
they
were
they
were
more
hesitant
than
if
they
had
to
opt
out
so
doing
things
by
default.
It
works
and
the
third
thing
is
kind
of
related
get
Ops
is
really
powerful
and
get
Ops
if
you're
not
familiar
with
this.
This
is
the
ID,
and
there
are
principles
on
this
that
I
mentioned
open,
get
ops.dev.
A
You
can
go
through
and
review
these
principles
now,
for
those
of
you
that
are
not
aware
of
open,
get
Ops
or
the
get
off
standard
or
maybe,
or
maybe,
you've
heard
of
get
Ops.
It
basically
has
four
principles.
Your
your
state
needs
to
be
defined
in
a
declarative
format,
so
most
people
are
using
terraform
like
hey
I,
checked
that
box
I'm
feeling
great
I'm
using
yaml's,
manifest
things
like
that
feeling
great
two:
it
needs
to
be
versioned
and
mutable.
A
This
is
where
we
start
to
see
people
stumble,
because
maybe
they
use
something
like
a
floating
tag
in
their
manifest,
so
it's
no
longer
versioned
or
even
though
they
are
used,
they
are
checking
in
to
get
they're
not
following
best
practices
to
keep
those
things
version
mutable.
It
needs
to
be
pulled
automatically.
A
That
means
you
need
a
software
agent
that
is
consistently
checking
what
the
desired
state
of
Truth
is
as
defined
in
git,
and
then
it
needs
to
be
continuously
reconciled,
which
means
it
needs
to
be
aware
of
what's
happening
in
production
and
the
actual
State
and
then
constantly
comparing
that
against
the
desired
State.
And
if
you
could
do
those
two
things
and
have
a
Reconciliation
policy,
you
can
do
get
Ops
and
that's
fundamentally
different
than
the
way
that
we've
delivered
software
with
CI
CD
that
maybe
relies
on
a
web
hook.
A
Now
you
can
still
use
a
web
hook
as
part
of
this
process.
You
just
can't
rely
on
the
web
hook.
So
how
do
we
make
sure
that
people
are
doing
this?
There's
some
great
Tool
training
out
there
that
we
put
out?
That
is
a
get
Ops
certification
using
Argo
CD
course,
and
it
has
two
levels:
get
off
some
fundamentals
and
get
off
the
scale.
I
think
we
have
over
13
or
14
000
students
this
year
alone.
A
It's
the
most
popular
fastest
growing
training,
certification
in
the
world,
Bar
None-
and
it
is
quite
excellent
if
I
do
so
do
say
so
myself
so
go
to
learning.codefresh.io
to
check
that
out,
but
I
wouldn't
say
that
we
should
rely
entirely
on
it.
So
the
first
step
when
it
comes
to
security
training
is
the
first
step.
It
is
not
the
last
step.
A
Okay,
so
we're
gonna
go
into
that
those
other
steps
that
we
can
take
so
approaching
security.
We
want
to
verify
our
software
supply
chain.
We
want
to
enforce
security
policy
and
we
want
to
do
it
all
with
githubs,
so
I'm
going
to
show
you
how
to
do
this
using
Sig
store,
cosine,
six
store,
cosine
six
store
is
a
project
within
cncf
and
cosine
is
an
excellent
tool
they
put
out
which
allows
us
to
sign
images
to
verify
their
Providence.
A
We
actually
just
implemented
this
in
the
the
Argo
project,
so
all
of
those
images
starting
with
2.4
2.5,
inargo
CD.
Those
are
now
signed.
So
to
use
this,
it's
pretty
easy,
there's
a
command
line
utility.
So
you
run
cosine
generate
key
pair
and
in
this
case
I'm
specifying
that
it
go
into
my
kubernetes
cluster.
So
this
is
going
to
generate
a
key
that
is
going
to
be
in
my
kubernetes
cluster.
That's
available
for-
and
this
is
the
cluster
that
I
use
for
building
things.
A
So
I
can
verify
that
an
image
was
built
using
that
cluster,
because
the
keys
are
only
stored
and
only
created
on
that
cluster.
So
once
I
I've
got
that
key
created
I
can
then
sign
an
image
and
upload
it
to
a
registry.
So
I
could
do
that
manually
on
the
command
line
if
I
have
access
to
the
keys.
I
also
created
for
everybody's
benefit,
a
tool
in
the
code
fresh
workflow,
template
Library.
So
if
you're
not
familiar
with
this,
let's
just
go:
look
at
it
really
quick!
A
This
is
the
code
fresh
Argo
hub
or
this,
or
rather
it's
the
code
fresh
hub
for
Argo,
as
you
can
see,
and
in
here
I
created
this
workflow
template.
So
if
you're
using
Argo
workflows
to
build
images-
and
you
want
to
sign
them-
you
can
just
grab
this
workflow
template
off
the
shelf,
throw
it
into
your
workflow
and
you're
good
to
go.
So
this
is
a
template,
that's
free
and
available
for
you
to
use
at
codefresh.io
Argo
hub.
A
So
let's
go
back
and
once
you've
done
that
you
can
put
it
into
your
workflow
template,
and
you
can
see
here
an
example
of
how
that
was
done.
You
can
see
we
we
basically
just
pass
in
the
cosine
Keys.
These
are
pulled
from
a
secret
and
then
in
the
template.
It
just
references
which
image
needs
to
be
signed
and
the
tag
needs
to
be
signed,
and
then
we
need
to
make
sure
that
we
have
our
Docker
config
mounted,
because
this
is
going
to
allow
us
to
push
it
to
the
registry.
A
That
is
there
all
right.
So,
once
we've
done
that
we
have
another
tool
that
we're
going
to
use
called
Opa.
Now
many
people
are
familiar
with
Opa,
but
they're
not
familiar
with
Opa
gatekeeper,
so
Opa
is
stands
for
open
policy
agent
and
it
is
a
very
popular
general
purpose.
Security
policy
engine,
so
it
gained
a
lot
of
popular
popularity
off
of
istio
and
its
implementation
with
Envoy
proxy,
because
you
can
set
policies
for
how
Opa
deals
with
those
kinds
of
things.
A
So,
for
example,
you
can
basically
set
firewall
rules
on
the
service
level,
and
things
like
that.
This
is
a
graduated
cncf
project.
It's
very
well
baked.
It
has
a
ton
of
tooling
it's
it's
very
excellent.
Now
it's
a
newer
implementation
of
opa
is
Opa
gatekeeper.
This
is
a
kubernetes
controller
that
sits
on
your
kubernetes
cluster
and
you
can
give
it
policy
and
it
will
check
using
in
ingestion
web
injection
web
hooks
to
make
sure
that
those
policies
are
implemented.
A
So
you
can
do
things
like
say
make
sure
this
image
is
signed
before
it's
allowed
onto
the
cluster.
So
that's
going
to
be
a
tool
that
we're
going
to
be
using.
We
can
also
there's
a
ton
of
policies
available
at
github.com,
open,
policyagent,
slash,
gatekeeper
library,
and
they
include
things
like
protecting
specific
name
spaces,
preventing
privileged
containers
blocking
and
allowing
image
repositories.
Ensuring
requests
and
limits
are
set,
limiting
how
many
replicas
there
are
blocking
wild
cards.
A
All
kinds
of
stuff-
and
you
can
actually
write
your
own
in
a
language
called
Rego
which
I
wasn't
ever
familiar
with
Rego,
is
a
little
tricky
to
learn,
but
because
the
library
is
so
large,
I
feel
like
Opa
gatekeeper
is,
is
a
great
resource
and
a
great
one
to
use.
So
the
way
this
is
going
to
work
is
when
we
make
changes.
A
Argo
CD,
we
didn't
even
talk,
I,
didn't
even
introduce
you
to
Argo
CD
if
you're
not
familiar
with
Argo
CD
Argo
CD
is
a
tool,
that's
part
of
the
Argo
project
and
rocd
basically
lets
you
set
as
a
policy.
This
is
an
application.
This
is
where
I
want
to
deploy
to.
This
is
where
it's
manifests
are
from,
so
you
define
the
source
of
truth
from
git
and
the
destination
in
a
cluster,
and
then
Argo
CD
will
follow
the
policy
make
sure
those
are
kept
in
sync.
A
Argo
CD
is
going
to
apply
those
manifests
to
the
kubernetes
cluster.
That's
going
to
trigger
an
admission
web
hook
for
Opa
gatekeeper
to
review
policy
and
apply
constraints.
So
there
are
two
components
to
Opa
gatekeeper.
There
are
policy
templates.
These
are
things
like
like
I
want
to
check
namespaces
for
some
criteria
and
then
a
constraint
is
the
actual
policy
itself.
A
It
says
check
against
this
policy
template
which
checks
namespaces,
and
these
are
the
values
that
I'm
looking
for
and
then
once
it's
reviewed
those
things
if
it
passes
then
Opa
gatekeeper
will
pass
it
on
to
the
scheduler
and
kubernetes
will
continue,
deploying
it
if
it
doesn't
meet
policy.
A
It's
going
to
be
rejected,
so
I
mentioned
that
Opa
Library
exists,
which
is
a
huge
library
of
policy
templates,
and
these
imp
are
implemented
each
one
as
a
crd,
so
as
a
custom
resource
definition,
and
you
can
see
that
I,
actually,
sync
them
using
Argo
CD,
so
I
just
have
a
customization
that
references,
the
OPA
library
and
then
Argo
City
takes
that
and
renders
these
and
deploys
them
into
my
into
my
cluster
for
Opa
gatekeeper
to
use.
So
let's
do
some
demos
here,
let's
use
Opa
gatekeeper
to
apply
some
policies.
A
So,
let's
see,
if
I
remember
how
to
do
this
demo,
so
I've
got
a
couple
of
policies
that
I
want
to
apply.
No
floating
tags
only.
Allow
me
to
just
actually
before
we
do
this.
We
need
to
deploy
it
so
I'm.
A
Looking
at
my
cluster
here
and
you
can
see,
I
don't
have
anything
deployed
other
than
I've
got
Argo
CD
managing
itself
I'm
using
a
tool
called
Argo,
CD
autopilot,
Argo
CD
autopilot
provides
an
opinionated
straightforward
way
of
setting
up
Argo
CD
in
a
self-managing
way,
so
that
everything
is
recorded
in
git
and
it
goes
into
git
first
and
it's
self-managing,
and
it's
awesome.
A
So
let
me
let
me
bootstrap
my
cluster
and
when
I
do
this
bootstrap
of
all
my
components
by
default
in
my
repo
I,
actually
had
it
as
part
of
the
normal
bootstrap,
but
for
the
demo
I
wanted
to
bootstrap
it
for
you,
I,
don't
know,
because
I
wanted
to
be
special
I
guess
so
what
I'm
going
to
do
is
use
Argo,
CD,
autopilot
and
I've
already
passed
in
a
git
token
and
a
git
repo
in
my
environment
here,
so
that
what
this
is
basically
going
to
do.
A
Is
it's
just
going
to
make
git
commits
for
me?
So,
let's,
let's
actually
look
really
quick
at
the
repository.
So
you
can
see
this
is
my
repository
where
I
have
everything
defined
and
I?
Have
this
folder
called
apps,
which
is
currently
empty?
A
Anything
in
my
application,
folder
is
going
to
be
synced
to
is,
is
is
going
to
generate
applications
into
Argo
CD
and
what
I've
done
is
I've
set
up
an
application
I'm
going
to
be
setting
up
an
application
called
source
that
is
going
to
pull
in
the
Manifest
the
application
configs
from
that
repo,
so
I'll
show
you
what
this
looks
like
it'll
be
more
it'll,
be
more
obvious:
cellular,
app,
create
and
source
and
I'm
going
to
specify
the
app
is
at
this
Source
folder
I
just
showed
you
under
the
app
configs
directory.
A
So
this
is
where
I
put
all
my
application
configurations
and
specifically
the
ones
that
I
want
deployed
onto
OSS
secure,
which
is
the
name
of
the
cluster
I'm
operating
on,
and
the
project
is
also
OSS
secure,
so
I
do
this.
This
is
going
to
just
create
a
commit
for
me
and
that
commit
is
going
to
generate
this
configure
Json,
which
is
going
to
be
consumed
by
an
application
set
into
Argo
CD.
So
whoa
oh
shoot.
It's
happening
too
fast,
so
you
can
see.
A
This
is
now
triggering
a
bootstrap,
that's
happening
where
there
are
a
bunch
of
applications
that
are
getting
deployed,
and
this
includes
my
my
autopilot
sorry.
This
includes
my
Opa
Library.
Let's
look
on
this
page
here
are
my
Opa
constraints.
My
Opa
Library
I've
got
my
cosine
provider.
I've
got
gatekeeper
running
all
this
stuff
is
deployed
and
up
and
running,
and
then
you
can
see.
It's
also
deployed
some
of
the
stuff
for
my
application.
So
I
have
a
demo
accept
app
and
I.
A
Have
a
demo
reject
app
that
are
being
applied
Okay,
so
we've
got
everything
applied
on
our
cluster
and
now
we've
got
policies
that
we
can
actually
test
against.
So
let's
go
back
and
look
at
this
demo
and
what
we'll
do
is
we're
going
to
go
into
whoa
whoa
whoa
whoa,
oh
yeah,
it's
in
here,
okay,
so
I've
got
my
Opa
GateKeeper.
A
A
Let's
make
sure
that
I'm
using
foreign
make
sure
I'm
using
the
right
cluster
yep
I'm,
pointing
at
my
OSS,
secure
repo
that
has
been
generously
been
provided
by
our
partners
at
AWS
and
by
provided
I
mean
they.
Let
us
pay
for
it
and
demo
it.
So
what
I'm
going
to
do
is
in
my
example,
repo
here
I've
got
this
pod
not
allowed
so
I'm
going
to
do
is
I
created
a
couple
of
namespaces
in
here
and
I
made
one
that
I'm
going
to
apply
security
policies
onto
now.
A
I
could
apply
this
onto
all
of
them,
but
I'm
going
to
be
doing
the
secure,
namespace,
so
I'm
going
to
try
to
deploy
this
pod,
not
allowed.yaml.
So
I'm
going
to
do,
K
apply,
Dash
F
pod
not
allowed
in
the
namespace,
secure
and
I'm
going
to
just
do
a
dry
run
on
the
client
here
and
when
I
do
that,
you
can
see
it
was
created.
Oh
shoot
was
my
policy
not
applied?
Well,
remember:
policy
application
happens
on
the
cluster,
so
I
can't
do
a
dry
run.
Client
I
need
to
do
a
dry
run
server.
A
So
if
I
do
a
dry
run
server.
You'll
see
that
this
one
actually
violates
policy,
so
this
says
that
my
gatekeeper
denied
the
request
for
this
to
be
ingested,
because
this
image
is
using
a
latest
tag.
I
need
to
specify
a
specific
version,
so
I
have
a
policy
that's
set
to
reject
if
anybody
tries
to
deploy
something
using
a
latest
tag,
because
that's
a
floating
tag
that
would
violate
my
git
Ops
policies,
I
would
buy
my
gas
principles,
so
I'm
not
going
to
allow
people
to
do
that.
So
you
can
see
that's
pretty
straightforward.
A
It
works
very
well
when
we
do
it
server
side,
it
restricts
the
resource.
When
we
do
a
client-side,
it
doesn't
because
it's
not
aware
of
the
injection
web
hook.
That
means
that
if
I
were
to
actually
try
to
apply
apply
this
manifest,
it
would
be
get
rejected.
It
wouldn't
allow
it
to
deploy.
So,
let's
talk
about
cosine
with
CI
CD
now
this
is
the.
This
is
the
big
demo.
A
So
now
that
you
understand
how
Opa
policy
gatekeeper
is
working
and
it's
enforcing
policies,
what
we
want
to
do
is
go
through
a
process,
and
this
is
what
the
entire
process
would
look
like
and
we
can
trigger
all
of
this.
So
starting
with
we've
got
two
repositories
over
here.
We've
got
a
git
Ops
repo,
that's
the
one.
I
just
showed
you,
and
then
we
have
an
app
repo.
The
app
repo
is
where
we
Define
our
application.
A
So
if
we
go
back
and
look
at
this,
this
Repository
oopsie,
this
repository
just
contains
what
is
needed
to
run
everything
on
my
cluster.
So
all
of
my
if
I
just
ran
a
tree
on
this
really
quick.
Let's
just
do
that.
Let's
go
over
to
here.
A
So
you
can
see-
oh
there's
actually
quite
a
bit
in
here,
so
maybe
maybe
this
isn't
the
best
way
to
display
this,
but
the
way
that
it
is
organized
follows
the
principles
that
are
put
out
on
repo
organization
code.
Fresh,
has
a
great
blog
post
on
this,
and
basically
we
get
all
our
all
of
the
automatic
generated.
Apps
are
in
apps
and
then
we
have
our
source
repo.
That
has
two
different
sets:
app
app,
configs
and
manifests
all
of
our
manifests
are
in
there
anyway.
So
then
we
have
our.
A
Then
we
have
our
application
repo,
which
is
where
these
things
are
actually
being
generated,
I'm
using
this
color-coded
app
and
basically,
whenever
I
cut
a
release,
I
want
to
trigger
the
creation
of
my
resources
and
sign
them.
So
if
we
look
at
this
process,
we
trigger
CI
CD.
This
is
going
to
run
a
build
in
code
fresh
using
our
go
workflows
in
the
code
fresh
platform,
and
it's
going
to
sign
the
image
using
cosine
using
that
workflow
template
I
already
showed
you
and
then
it's
going
to
push
it
to
the
image
registry.
A
Once
it's
in
the
image
registry,
it's
available
to
be
deployed
to
the
next
step,
is
I.
Add
the
release
to
my
get
Ops
repo.
Now
this
could
be
done
automatically
here.
I've
actually
done
it
manually,
but
there's
a
great
tool
called
image
updater.
That
would
write
this
to
get
for
me
once
I
update
the
release
and
get
up
in
my
get
off
repo,
this
will
sync
it
using
Argo
CD
and
first
it
creates
the
applications
in
kubernetes
the
admission
controller
runs.
A
It
will
accept
it
in
our
resources
and
then
it
will
deploy
those
resources
and
Report
listening
status
back
right
if
it's,
if
it's
allowed.
So
what
does
that
look
like?
Let's
do
it
live
really
quick,
so
I've
got
in
here
my
my
app
repository
and
I
just
cut
I.
The
last
release.
I
cut
was
this
version
1.01
secure,
and
this
kicked
off
a
CI
pipeline.
Now,
I
can
show
you
what
that
looks
like
really
quick
just
proved
you
that
we're
doing
it.
A
A
And
I'll
pull
up
my
release
Here,
so
you
can
see
this
is
the
last
release
I
ran.
Was
this
101
secure?
Let's
just
look
at
this
workflow
really
quick,
so
you
can
see
what
it
did.
The
release
was
triggered
by
once
I
cut
the
release.
This
triggered
the
build
it
built
my
Docker
image.
It
signed
the
image
using
cosine,
there's
nothing
to
display
here,
because
everything
ran
properly
and
then
it
ran
some
report.
A
Information
did
some
image
enrichment,
which
is
just
native
into
code
fresh
and,
of
course,
all
that
was
pushed
into
GitHub
all
right.
So
the
first
part
of
the
demo,
for
that
is
complete.
The
next
part
was
to
trigger
the
release.
So
let's
actually
look
at
that
really
quick
so
to
trigger
the
release.
If
I
go
over
to
my
get
Ops
repo
and
I,
look
at
my
manifests
for
my
demo
accept
app
and
I
look
at
what's
supposed
to
be
deployed
for
OSS
secure.
You
can
see.
A
I've
got
this
customization
here
and
this
is
specifying
what
image
should
be
deployed
now:
I've
actually
already
updated
it
to
Colors
version
1.0.1
secure.
Now,
if
I
were
to
change
this
to
an
insecure
release
like
blue
purple
or
yellow,
this
would
be
rejected.
So
if
we
go
back
and
look
in
here,
really
quick,
you
can
see
that
we
have
a
demo
app.
That's
been
rejected.
A
Let's
find
out
why
this
was
rejected,
really
quick
and
if
we
look
at
the
events
in
here,
we'll
see
that
this
one
was
failed
because
we
try
to
deploy
an
image
that
was
from
a
repository
that
is
not
in
the
accepted
list,
so
we
verified
that
it's
coming
from
the
right
repository
and
this
one
wasn't
allowed
in
my
demo
except
application.
You
can
see
that
it's
currently
running
looks
like
it's
doing
a
canary.
It
has
done
a
canary
release
and
completed.
A
Let's
go
ahead
and
trigger
a
new
one,
and
let's,
let's
send
you,
can
already
see
that
it's
deploying
properly
when
we
do
a
secure
image.
Let's
do
an
unsecure
image.
So
we'll
just
change
this
to
Blue
go
Blue!
It's
not
secure
LOL
and
we're
just
going
to
commit
that
directly
for
the
sake
of
demo
speed.
So
this
will
there's
actually
no
web
hook
set
up
from
the
git
repo.
Here
you
could,
but
it
doesn't
rely
on
one.
A
So
within
you
know,
30
seconds
or
so
it'll
pick
up
and
see
that
a
change
has
happened
and
it
will
trigger
the
the
deployment
to
go.
So
this
approach
is
pretty
awesome
because
it
lets
you
do
things
like
make
sure
that
people
can't
deploy
images
that
weren't
built
where
you
expected
them
to
be
built.
So
here
you
can
see
this
has
triggered
the
resource
Group
now
because
I'm
using
Argo
2.5,
this
does
server
side
apply.
A
So
that
means
when
it
tried
to
create
this
Resource
Group,
we
got
the
event
right
away:
hey
this
is
not
allowed
in
this
case.
This
was
blocked
because
it
was
checking
for
the
image
verification
and
there
was
no
signature
on
this.
So
this
was
blocked
and
was
picked
up
now
because
I'm
doing
a
canary
release.
All
of
my
service
there's
been
no
service
disruption
here
and
in
this
case
this
one's,
where
he's
rejected
via
the
server
side,
apply
now
prior
to
Argo,
CD
2.5.
We
actually
did
client
apply.
A
So
that
means
that
it
would
have
applied
it,
but
then
it
would
have
failed
in
the
cluster.
Now
it
would
still
work.
You
would
actually
just
see
a
little
bit
more
spinning
on
this
resource
Source
because
it
would
be
failing
creation.
So
now
this
is
going
to
be
stuck
in
a
progressing
State,
because
this
is
never
going
to
progress,
so
this
doesn't
trigger
a
synchronization
fail,
whereas
it
would
prior
to
Argo
CD
2.5.
A
A
Let's
commit
that
we'll
go
back
to
Argo,
CD
and
and
now
I'm
just
going
to
hit
Refresh
on
it
to
trigger
the
refresh
immediately,
and
you
can
see
now
this
this
has
been
applied.
This
new
resource
looks
like
it's
been
applied.
Oh
wait
is
this?
A
Oh
well,
actually,
it's
already
deployed
it's
this
one.
Sorry!
So
that's
not
a
great
demo!
So
let's
do
since
we're
already
deployed
it.
Let's
switch
to
the
the
other,
secure
image
that
I
have,
because
it's
not
going
to
trigger
the
canary
release,
switch
to
diff
secure
release
because
I
already
had
it
deployed
before.
So
it
was
just
sitting
in
the
canary
release,
so
it
was
like
yeah.
A
Everything
looks
great
now
because,
because
you
already
had
it
deployed
so
you're
just
saying
you
want
to
go
back
to
the
one
that
we
already
have
deployed,
everything's
fine.
So
now,
when
we
sync
it
we'll
actually
trigger
a
new
resource,
a
new
rollout
to
trigger
with
a
new
replica
set-
and
you
can
see
these
these
pods
rolled
out
just
fine,
and
this
is
going
to
go
through
its
Canary
release.
So
this
will
be
tested
and
it
will
move
forward
as
it's
ready.
A
Okay,
so
we
just
demoed
that
so
using
Opa
with
Argo
CD,
a
couple
of
things
to
mention
here
so
I
tend
to
think
of
in
kubernetes
there's
kind
of
two
kinds
of
resources.
They
are.
These
aren't
official
but
I
call
some
fragile.
These
are
resources
that
once
they're
started
broken,
they
will
always
stay
broken
and
a
good
example
is
if
you're
deploy
an
Ingress
before
an
Ingress
provider.
It
will
get
the
wrong
information
on
it
and
it
will
never
be
fixed
and
then
I
think
there
are
resilient
resources.
A
These
are,
if
failures
occur,
they
will
eventually
correct.
So
a
great
example
of
this
is,
if
you
try
to
apply
a
resource
with
a
custom
resource,
but
that
custom
resource
definition
has
not
been
applied,
it
will
fail,
but
as
soon
as
the
crd
is
added
that
custom
resource
will
deploy
just
fine
and
that
will
just
take
place
automatically
based
on
kubernetes
retries,
it
might
take
a
long
time.
Maybe
you
should
have
better
policy
around
it,
but
it'll
work.
So
there
are
tools
within
Argo
CD.
A
One
is
called
sync
waves,
and
this
allows
you
to
set
the
order
in
which
resources
are
applied
and
the
other
is
retry.
Retry
basically
lets
us
set
a
policy
for
when
and
how
often
Argo
CD
should
retry
applying
a
resource.
So
within
our
group,
because
we're
relying
on
admission
controllers
or
Mission
web
hooks
through
open
policy
agent,
that
means
every
resource
is
potentially
fragile
when
it
comes
to
applying
security
policy.
Let
me
explain
that
differently.
A
If
I'm
trying
to
apply
a
security
policy
to
a
new
resource,
I
have
to
make
sure
that
security
policy
exists
on
the
cluster
before
I,
try
to
deploy
the
new
resource
right.
So
this
is
most
important
when
you're
doing
a
bootstrap
and
I'll
show
you
how
I
did
it
on
here
if
you're
using
so
oh
well,
just
one
other
thing
use
retry
on
policies,
because
the
policies
themselves
are
resilient.
They
only
require
that
the
templates
exist
for
the
policies.
A
So
we
talked
about
those
two
things
that
were
on
the
cluster
and
we
also
want
to
make
sure
that
the
policies
are
applied
before
any
other
applications.
So
to
do
this,
we're
going
to
need
to
monitor
app
sync
status,
and
this
does
work
with
multiple
providers.
So
if
you're,
using
Opa,
gatekeeper
or
kyberno
or
something
else
that
will
work
as
well
and
you
can
run
Audits
and
2.5
Argo
CD
works
best,
but
before
2.5
we'll
work.
A
Okay,
you'll
just
get
different
kinds
of
Errors
when
policies
are
enforced,
so
to
do
sync
waves,
I
recommend,
following
this
blog
written
by
Christian,
you
can
follow
him
on
Twitter
at
christian814,
but
this
blog
Argos
CD
application
dependencies
allows
to
set
the
order
in
which
applications
are
synced.
So
what
I've
done
is,
and
we
bootstrapped
this
earlier
I
have
a
app
of
apps
pattern
in
Argo
CD.
Basically,
this
is
an
application
that
can
that
references,
two
additional
applications
and
one
I
called
system
requirements
and
one
I
called
user
space.
A
Everything
in
system
requirements
has
to
be
deployed
before
anything
in
the
user
space,
because
system
requirements
includes
all
of
my
policies
to
enable
this
is
pretty
easy,
starting
in
Argo
1.8,
argocd
1.8.
We
removed
this
functionality
and
made
it
optional,
but
basically
what
we
can
do
is
we
can
say
that
when
we're
deploying
an
app
of
apps
by
default,
if
this
one
failed,
it
would
still
try
to
apply
this
one.
So
we
want
to
prevent
that.
A
A
So
you
can
enable
that
with
this
config
map,
which
is
explained
in
the
blog
post,
I
mentioned
a
second
ago,
if
you're
using
it
with
Argosy
data
pilot,
here's
an
example
of
what
that
code
would
look
like
to
inject
that
into
your
manifest,
and
you
can
also
find
this
in
the
OSS
secure
repo
that
I
was
showing
you
earlier
and
then
we
can
use
sync
waves
for
these
apps,
so
sync
waves
specify
the
order.
So
under
my
system
requirements,
application
I
have
sync
wave
one,
and
under
my
user
space
I
have
sync
wave
2..
A
That
means
everything
has
to
be
applied
in
my
system
requirements
group
of
applications,
which
includes
my
policies
and
all
that
stuff
before
my
user
space
stuff
can
be
applied,
which
is
my
demo
applications.
A
So
I
showed
you
what
that
looks
like
yeah.
We
just
did
this,
so
we
don't
need
to
do
that
again.
Let's
move
on
to
the
next
thing,
so
another
thing
you
can
do
is
you
can
prevent
resources
from
being
created
that
were
not
created
using
Argo
CD.
So
this
is
a
policy
that
I
created
inside
of
gatekeeper
and
the
policy
template
exists
out
there.
A
It's
it's
referenced
in
here
and
then
all
I
do
is
basically
I
say
check
for
these
annotations
and
The
annotation
I'm
looking
for
is
the
tracking
ID
that
is
set
by
Argo
CD.
Now
you
need
to
enable
this
level
of
resource
tracking
using
Argo
CD
here
I
set
it
on
service
and
pod.
This
is
actually
incorrect.
It
should
be
on
deployment
and
replica
set
and
things
like
that,
because
the
pod
doesn't
actually
set
a
tracking
ID,
but
by
doing
this,
I
can
make
sure
that
nobody
deploys
an
application
outside
of
Argo
CD.
A
So
since
I
already
demoed
what
it
looks
like
when
you
apply
a
resource
and
it
doesn't
work-
that's
pretty-
you
probably
don't
need
to
see
it
again,
but
there
you
go
bonus.
Tip
kyverno,
we
haven't
talked
that
much
about
kyberno
kyverno
is
a
competitor
to
Opa
gatekeeper,
and
it
has
some
additional
things
that
now
for
this
demo,
I
actually
used.
Opa,
gatekeeper
and
kyberno
I
actually
have
kyberno
checking
my
image
signing,
but
then
I
have
gatekeeper,
checking
my
other
policies.
A
You
can
use
both
you
could
use
just
one
I
just
found
that
I
wanted
to
do
a
demo
that
included
both
so
I
included
both
of
them
and
they
worked
fine
together.
Now
kyverno
does
work
a
little
bit
differently,
because
kyverno
actually
mutates
resources
when
it
applies
policy,
and
this
means
that
if
you
don't
do
something,
Argo
CD
will
get
in
a
fight
with
kyberno
Argo
CD
says
apply.
These
manifests
kyvera
says,
looks
great
I've
added
these
information.
Argo
CD
says:
where
did
that
information
come
from?
A
A
Basically,
we
can
add
and
ignore
differences
section
that
and
set
a
cluster
policy
that
ignores
all
of
the
components
that
are
auto-generated
by
kyberno,
so
those
will
not
trigger
resynchronization.
So
you
can
do
that
here.
The
instructions
are
in
the
blog
post
that
are
linked
at
the
bottom
of
the
slide,
and
the
slides
are
of
course,
going
to
be
provided
at
the
end.
So
what
do
we
accomplish?
Well,
we
verified
our
software
supply
chain
right.
We
made
sure
the
images
were
built
in
the
place
that
they
were
built.
A
We
made
sure
that
was
enforced
using
our
open
policy
agent
in
kyberno
and
Argo
CD.
We
implemented
it
all
with
Argo
CD
to
give
us
a
much
better
enforced
security
policy.
People
can't
get
around
these
things.
Now
it's
just
default.
It's
going
to
work
this
way
and
it's
going
to
reject
you.
It's
going
to
reject
your
release.
If
you
try
to
do
something
outside
of
process,
if
you
try
to
deploy
something
not
using
Argo
CD,
you
try
to
replace
something
manually.
A
We
can
actually
reject
that
and
we
can
reject
any
kind
of
resource
that
people
were
trying
to
deploy
that
way
and
we
did
it
with
100
get
Ops.
Everything
was
in
git,
and
so
you
can
follow
along.
If
you
want
to
go
and
implement
this
yourself,
you
can
take
my
repository
that
we
just
looked
at
this
OSS
secure
repository.
A
You
can
bootstrap
this
onto
your
own
cluster,
just
name
it
OSS
secure
and
you
can
bootstrap
everything
that
I
did
right
here
and
it
will
all
work
exactly
the
same
tons
of
resources
that
I
use
for
this
to
use
the
project
repo
artist
studio
I
mentioned
you
can
check
out
all
the
rest
of
these
great
resources
after
the
Talk
and,
of
course,
as
always,
go
get
certified
I.
This
slide
says
over
10K
students
it's
out
of
date.
A
It's
over
like
13
14
000
this
year
alone,
Argo
CD,
codefresh.io,
Argo,
get
Dash
certified
or
learning.codefresh.io
get
you
to
the
same
place,
and
we
have
github's
fundamentals
and
Argo
CD
at
scale
are
both
available
with
that.
I
would
take
questions,
but
I
think
we're
out
of
time.
So
hit
me
up
on
Twitter
at
today
was
awesome.
I
would
love
to
hear
from
you
also
go
check
out
code
fresh.