►
From YouTube: Introduction to Cloud Custodian...- Jorge Castro, Kapil Thangavelu, Sonny Shi & John Anderson
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe in Amsterdam, The Netherlands from April 17-21, 2023. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Introduction to Cloud Custodian, simple rules for your Cloud, Cluster, and Terraform - Jorge Castro, Kapil Thangavelu, Sonny Shi & John Anderson, Stacklet.io
Speakers: Kapil Thangavelu, Sonny Shi, Jorge Castro, John Anderson
Cloud Custodian (c7n) is an easy to use declarative policy engine for solving governance as code use cases for Kubernetes, Azure, GCP, AWS, and Terraform. Simplify rule sets around your entire stack from the cloud to Kubernetes and everything in between. In this talk the maintainers will cover use cases for using Custodian, cluster policy best practices, event driven enforcement of policies, and everything you need to get started running a well managed cloud and cluster.
A
B
Yeah
and
I'm
John
Anderson
I'll
talk
about
what
cloud
custodian
is
and
how
I
got
into
it.
I'm
one
of
the
contributors
on
the
custodian
project
and
so
I
got
into
using
custodian
as
an
SRE,
so
I've
been
managing
kubernetes
clusters
in
AWS,
Cloud
Estates
at
companies
like
server
monkey
and
zapier,
and
my
job
was
to
empower
developers
to
manage
their
costs
but
be
productive.
B
Make
sure
that
they're
in
compliance
with
any
governance,
things
that
we
have
like
sock
2
and
so
that
they
could
follow
any
of
the
best
practices
that
we've
set
out
within
our
organization.
And
so
what
because
Cloud
custodian
allows
you
to
do
is
do
all
of
this.
In
one
place,
you
can
set
up
policies
with
an
easy
to
use,
yaml
based
engine
and
control,
your
terraform,
your
kubernetes
clusters,
your
any
of
the
public
clouds,
Azure
AWS
gcp,
and
it's
all
the
same
yaml.
B
B
I
was
running
something
for
kubernetes,
something
for
AWS
something
for
terraform
and
when
you're
doing
that
across
all
these
different
things,
you
start
getting
a
different
experience
between
each
one,
so
you
might
be
preventing
some
sock
2
stuff
on
AWS,
but
you're,
not
preventing
it
on
the
terraform
layer,
and
so
Cloud
custodian
opened
all
that
up,
and
so
one
of
the
first
things
that
most
of
us
do
on
the
SRE
level
is
manage
cost.
We
want
to
get
those
costs
down
and
we
want
to
give
that
information
to
the
developers.
B
Saying,
okay,
you
have
underutilized
resource
or
you
have
resources
that
aren't
even
being
used.
So
when
I
was
at
zapier,
one
of
the
big
things
that
we
had
was
some
git
lab
Runners,
where
their
health
checks
were
running.
They
were
up
there
working
everyone
thought
we
were
utilizing
them,
and
then
we
got
a
report
from
the
developers
that
pipelines
were
taking
a
long
time
to
run.
We
started
investigating
it
and
it
turned
out
a
lot
of
them
were
just
sitting
there
doing
nothing.
B
So
we
were
just
burning
fifteen
thousand
dollars
a
month
on
get
lab
Runners
that
weren't
actually
running
crcd
jobs,
and
so
that's
where
we
use
cloud
custodian
today
where
we
can
run
these
policies
and
we
can
identify
underutilized
resources,
zombie
resources.
We
can
actually
enforce
tags
and
alert
the
owners
to
say
like
hey.
We
need
to
know
what
this
resource
is
and
who
it's
from
that
way.
B
And
then
the
next
thing
is
compliance.
Most
of
us
have
some
type
of
compliance
where
you're
going
to
be
running,
PCI,
HIPAA
CIS
any
of
those
and
you
just
when
you're
doing
something
like
sock
2
on
an
annual
basis.
You
want
to
be
checking
it
throughout
the
year
because,
if
you
just
say
oh
it's
out
of
time,
Let's
start
Gathering.
All
of
our
evidence.
It's
going
to
be
really
difficult
to
get
all
that
in
time.
B
For
your
auditor,
with
Cloud
custodian,
you
can
run
on
an
event
based
mode
or
a
pull
mode,
and
you
can
be
running
your
sock,
2
policies
all
the
time
and
then
finally,
the
like
probably
the
most
important
part
of
this
is
correctness.
You
have
you're
hiring
a
lot
of
experts.
You
have
them
there
because
they
know
what
the
best
practices
are.
So
you
want
them
to
codify
that
you
don't
want
developers
writing
code
without
tests.
You
don't
want
to
deploy
infrastructure
without
cloud
custodian,
because
Cloud
custodian
is
going
to
be
those
tasks.
B
A
A
So
here
we
have
a
built-in
notify
action
that
allows
you
to
send
notifications
to
the
resource
owner,
and
that
way
you
can
really
drive
that
behavior
change
at
your
organization,
telling
your
users
that
hey
you've
misconfigured
something
or
you
need
to
change
some
something
about
the
metadata
of
the
resource
and
get
it
so
that
your
users
are
constantly
basically
leveling
up
their
usage
of
the
cloud
and,
finally,
to
run
it.
You
just
run
Casino
run
with
the
policy
yaml
file
there.
A
A
Finally,
here
I'll
talk
a
bit
about
the
different
modes
that
we
have
so
with
Cloud
custodian.
You
have
a
pull
mode,
which
means
I
want
to
take
a
look
at
the
existing
Cloud
infrastructure
environment
so
querying
against
your
AWS
account
or
your
Azure
subscription
or
gcp
project.
But
additionally,
we
have
what
are
called
event
modes.
So
this
way
you
can
react
in
a
more
Dynamic
way.
A
So,
for
example,
if
you
were
to
have
a
developer,
create
an
S3
bucket
that
is
public,
you
can
actually
detect
that
right
away
when
it
happens,
change
the
permissions
on
that
bucket
to
make
it
not
non-public
as
well
as
do
things
like
notify
the
user
or
auto
tag
it
or
any
other
actions
that
are
available
on
the
resource
type.
So
really
this
lets
you
close.
A
The
gap
on
how
long
You're
vulnerable
for
as
well
as
prevent
things
that
are
headaches
down
the
line
when
you
know
you've
had
like
an
RDS
database,
that's
been
out
there
for
you
know
years
and
then
you're
telling
developers
like
hey.
You
got
to
go
and
fix
it.
You're
like
well
I
kind
of
wish
I
caught
that
earlier.
A
Finally,
just
a
quick
rundown
on
sort
of
like
the
software
development
lifecycle
policies,
so
one
way
to
do
this
is
you
can
keep
all
your
policies
in
a
git
repo
run,
CI
on
that,
and,
typically
what
that
means
is
run.
The
actual
policy
in
a
dry
run
mode
against
your
resources
and
then
deploy
them,
and
you
can
have
scheduled
policies
with
periodic
modes
or
run
it
on
a
Cron
job,
as
well
as
those
automated
policy
triggers
and
then
send
notifications
and
perform
remediations
on
top
of
that.
A
A
So
this
is
a
admission
controller
that
you
can
deploy
into
your
cluster
or
outside
your
cluster,
using
a
mutating
webflow
configuration
that
can
do
all
the
things
a
admission
controller
does,
which
is
you
know,
allow
denied
objects
based
on
filters
that
you've
defined
it's
easy
to
deploy
with
the
helm
chart
and,
for
example,
you
can
do
things
like
Auto
label
objects
as
they
come
in.
So
if
you
want
to
require
everybody
to
label
their
resources
as
I'm,
you
know
being
managed
by
sunny,
but
people
don't
seem
to
be
doing
it.
A
A
And
then
we'll
do
next
is
we
will
take
a
look
at
the
policy
that
we
have
here
and
here
we
have
a
policy
that
just
says
we're
going
to
require
this
to
be
managed
by
or
sorry
have
this
label
at
that
kubernetes
managed
by
and
if
it's
absent
to
filter
it
in
so
we
can
run
just
run
your
policies.
Oh
sorry,
gotta,
be
in
my.
A
A
So
if
we
run
this,
you
can
see
that
after
we
applied
it,
we're
getting
back
some
results
here.
Basically
all
of
these
results
show
up
in
our
output
directory
here,
so
we
can
see,
for
example,
in
this
managed
required,
managed
labels.
We
can
see
that
in
the
labels
here
we
only
have
the
foo
and
bar
labels.
So
basically
you
could
do
a
whole
lot
of
other
more
complicated
stuff
with
the
filter
syntax.
But
this
is
just
a
quick
look
at
how
you
can
get
started.
A
A
A
And
you
can
see
here
at
the
top,
we
have
a
missing
recommended
labels,
a
policy
that
says
all
pods
must
have
food
and
bar
labels.
So
we
actually
do
create
the
Pod
here,
but
you
get
that
you
get
that
warning.
The
second
one
here
is
requiring
deployments
to
have
more
than
at
least
three
replicas.
This
can
be
important
because
you
want
people
to
you
know,
deploy
things
in
a
way
that
are
more
AJ
and
you
can
see
here.
A
Right:
okay,
there
we
go
so
now
it's
it's
restricting
based
off
that
service
account
because
it
now
exists
the
policies
here.
If
we
take
a
look,
it's
the
same
policy
language
as
before.
The
only
thing
that's
changed
is
that,
instead
of
having
just
the
resource,
description
and
filters,
you
now
have
this
new
mode
section
where
you
can
Define
what
to
do
when
you
match
on
that
policy.
So
in
this
case
we're
saying
we're
going
to
deny
connections
and
what
operations
do
you
care
about?
A
A
A
B
C
B
Them
on
the
admission
controller,
we
have
the
same
type
of
functionality
when
you're
talking
about
terraform,
so
our
terraform
provider
actually
will
scan
at
the
it'll
do
static
analysis.
So,
where
you'll
see
a
lot
of
policy
agents
where
they'll
require
the
plan
to
actually
run
and
validate
that
your
terraforma
is
working,
we
do
static
analysis
on
that,
which
means
you
can
actually
have.
Your
developers
run
this
and
then
unprivileged
mode,
so
they
can
actually
check
on
their
local
computer.
B
B
B
That
say
we
require
KMS
encryption
instead
of
AES
for
an
S3
bucket
and
we
actually
do
want
encryption
enabled
on
any
sqs
cues,
and
so
this
allows
you
to
just
validate
any
of
the
terraform
that
you're
writing
locally
against
those
policies
and
those
can
be
correctness,
policies,
governance,
policies,
all
that,
and
so,
if
we
wanted
to
go
here
and
fix
this,
it
told
us
the
exact
lines
that
we
need
to
go
into
put
in
the
diff.
And
so
you
can
say
we
just
need
to
enable
KMS
encryption
and
then
that'll
fix
one
of
our
errors.
B
B
B
So
anyone
that
would
like
to
parse
terraform
via
python,
you
can
get
it
from
the
GitHub
Cloud
custodian,
tfr's,
Repository,
and
so
the
the
new
command
for
running
Cloud
custodian
against
airform
is
called
c7n
left
and
it
supports
actually
traversing
your
entire
graph
and
so
you'll
see
like
one
of
those
policies
that
we
were
showing.
It
was
saying
we
require
encryption
on
the
S3
bucket,
but
with
terraform
server-side
encryption
configurations
are
actually
a
separate
resource,
so
it's
actually
linking
those
resources
together
and
it's
seeing.
A
On
top
of
that,
it's
much
more
verbose,
so
here's
an
example
of
opa
requiring
encrypted
S3,
and
this
is
looking
at
the
terraform
so
purely
on
the
infrastructure
as
code
side,
not
looking
at
the
actual
resource
itself,
and
these
are
the
equivalent
policies
in
custodians.
So
on
the
left
here,
this
is
actually
checking
in
the
cloud.
What
your
bucket
encryption
is
on.
All
of
your
buckets
on
that
account
and
then
on.
The
right
is
the
the
terraform
equivalent,
so
basically
much
more
concise
and
and
ergonomic.
A
But
of
course,
Opa
has
a
lot
of
different
use.
Cases
like
if
you're
using
in
other
contexts,
there's
many
reasons
to
use
Opa
as
well
and
I.
Think
the
next
question
is:
how
are
you
different
from
kybrino?
So
California
was
great
because
it's
built,
you
know
natively
for
kubernetes,
and
it
has
a
lot
of
great
Power
and
features
in
the
kubernetes
space.
A
That's
touching
the
entire
stack
going
from
you
know
your
infrastructure
as
code
all
the
way
to
the
cloud
to
the
cluster
as
well
and
so
here's
another
example
of
what
a
policy
looks
like
inside
of
kyverno
and
here's
the
equivalent
inside
of
custodian
so
again
a
little
bit
more
concise,
but
for
the
different
use
cases
you
may
want
to
pick
kyrono,
but
for
people
that
are
using
custodian.
One
of
the
initial
goals
with
the
kubernetes
provider
is
to
provide
something
that's
very
native
and
familiar
to
existing
concerning
users.
A
Yeah,
that's
that's
a
good
question.
The
question
was:
if
there's
a
a
bucket
you
have
in
S3,
that's
required
to
be
managed
by
terraform.
Could
we
do
it
in
Cloud,
custodian,
yeah,
I
think
the
facilities
are
there
I,
there's
some
interesting
policies
that
you
could
write
so,
for
example,
with
the
event
driven
policies,
you
can
take
a
look
at
the
actual
underlying
event,
which
means
that
you
can
also
take
a
look
at
who
the
user
is.
That's
creating
it
so
any
sort
of
modifications
to
the
bucket.
E
D
Well,
I
was
thinking
is
because
you
can
statically
analyze
the
Tariff
form
and
like
presumably,
you
could
then
link
that
statically
analyze
terraform
to
an
actual
resource.
If
you
can
make
sure
that
for
each
resource
there
exists
like
some
terraform.
Basically,
of
course,
you
could
look
at
like
the
the
user,
agent
or
whatever.
That
seems
like
a
bit
of
a
hack
kind
of
because,
like
okay,
that
that
will
say
for
like
stuff
created
on
the
bucket,
but
then
like
okay,
for
like
modifications,
it
gets
a
bit
right.
A
I
mean
I,
I,
wouldn't
say
it's
a
hack,
I
mean
if
you
have
stuff
in
terraform.
You
really
should
not.
It
really
shouldn't
be
any
other
agent
going
and
modifying
the
resource,
and
that's
I
think
that's
something
that
I
would
want
to
know
about.
As
a
SRE
or
you
know,
Cloud
Engineers
like
are
people
coming
in
and
actually
manually
modifying
stuff,
especially
if
it's
supposed
to
be
managed
by
terraform
but
I.
A
Think
if
you're,
if
your
organization's
in
the
state,
where
you
have
widespread
infrastructures
code,
adoption
like
enforcing
those
kinds
of
standards
is
a
lot
easier
than
really,
if
you're,
in
kind
of
a
50
50
state
where,
like
some
developers,
are
really
using
IEC
and
terraform
cloudformation.
What
have
you
and
other
people
are
still
doing?
Click
Ops
well.
D
Well,
an
example:
I
guess
is
like
okay,
like
you
could
have
a
if
you
create
a
terraform
resource
like
okay,
you
write
some
telephone
code.
You
create
a
resource
using
like
terraform
locally
like
okay,
what
it'll
see
if
you
look
at
the
users
as
it
was
created
using
like
the
terraform
agent
or
whatever?
D
But
if
you
is
there
a
way
to
talk
like
okay,
like
let's
look
at
what's
in
git,
and
let's
look
at
what's
in
the
and
let's
look
at
what's
in
AWS
and
to
see
like
okay,
it's
like
everything
like
in
the
like,
for
instance,
could
you
check
like
okay?
Is
everything
that's
like
in
the
production
account
also
in
the
main
branch
of
like
the
IAC
repower
or
whatever,
like
the
productive
branch.
C
A
B
A
F
I
mean
Azure
policies
are,
are
powerful
as
a
you
know:
control,
plane,
primitive,
but
I
think
the
challenges
is
expressivity
and
readability,
whereas
these
are
potentially
operating
both
in
CI
pre-deployment,
much
more
expressive
and
can
be
operated
on
a
developer
station
as
well
and
can
also
do
analysis
post
deployment
so
to
speak,
whereas
Azure
policies
like
the
last
time
I
looked.
This
is
a
fairly
primitive
comparison
model
as
far
as
what
you
could
do
and
defining
exceptions
and
other
things
other
real
world
things
were
actually
a
little
bit
harder.
B
Yeah
and
I'll
add
like
one
of
the
big
benefits
of
custodian
is
the
fact
that
it's
operating
across
all
these
things.
It's
doing,
kubernetes,
terraform,
Azure,
gcp,
all
that,
and
so
a
lot
of
times
like
we
said
with
Opa
and
caverno
they're
great
in
the
place
that
they're
sitting,
but
when
you're
trying
to
govern
across
all
of
it.
It's
really
nice
to
have
it
one
policy
agent
that
everyone
can
learn
so
like
when
we
rolled
out
opa
zapier
the
people
who
were
writing
Rego.
All
the
time
knew
how
to
write
rhaega
anyone
else.
B
It
was
difficult
every
single
time
and
so
with
custodian
when
it
spread
across
all
of
these
different
environments.
People
are
writing
it
all
the
time
and
it's
yaml.
So
it's
much
easier
for
them
to
understand,
and
so
Azure
policy
is
a
great
policy
agent.
But
if
you
want
it
across
everything,
you're
going
to
want
something
like
custodian.
G
B
B
G
B
You
know
it's
going
to
be
generating
load
balancers
on
your
cloud
and
then
so,
if
you
want
to
identify
those
as
long
as
you're
having
whatever
tool
is
generating
those
resources,
labeling
them
or
some
way
to
identify
that
they're
created
by
this
tool
or
like
we
were
talking
about
earlier,
you
can
use
the
ability
to
scan
for
underutilized
resources
as
well
like
unused
resources,
and
you
can
just
run
policies
that
say:
hey
I've
got
this
unused
resource,
identify
it.
Remove
it
a
lot
of
times
what
we
recommend
is
actually
just
marking
it.
B
A
I
A
Yeah,
so
custodian
has
a
bunch
of
the
question
was:
are
we
able
to
basically
report
on
that
data
in
formats
like
CSV
yeah?
So
custodian
comes
with
a
built-in
report
command
where
you're
able
to
pass
in
the
policy
file
that
you
were
using
as
well
as
the
output
directory,
and
it
will
create
a
tabular
view
in
CSV.
A
On
top
of
that,
there's
a
whole
bunch
of
other
tools
inside
of
custodian
as
well.
It's
not
just
the
custodian
CLI
there's
the
mailer,
which
is
good
for
notifications,
policy
stream,
there's
Trail,
crater
and
stuff
like
that.
So
there's
there's
a
whole
lot
of
cool
stuff
in
the
repo
and
yeah
reporting
is
a
huge
part
of
that,
because,
ultimately,
the
people
I
care
about
compliance
are
typically
not
running
clis.
They
usually
want
like
an
Excel
sheet,
so
yeah.
H
It's
like
it's
based
on
going
off
his
questions.
Do
you
does
it
do
like
a
dependency
search
so
like
say
something
like
crazy
ec2
instance,
and
then
they
tell
a
security
group
to
that
and
then
maybe
they
want
to
do
a
ALB.
On
top
of
that,
if
I
delete
like
the
ec2
instance
or
before
deleted
we'll
say
well,
this
is
dependent
on
this
this
and
this
you
may
want
to
delete
this
or
not
delete
this.
A
Yeah,
the
question
was
around:
how
does
custodian
handle
basically
interdependencies
between
the
resources
so
with
custodians?
Really,
the
query
part
of
custodian
allows
you
to
query
on
relationships
as
well.
Your
question
about
like
before
you
delete.
You
know
a
security
group.
You
got
to
make
sure
that
it's
not
attached
to
anything.
I
would
have
to
look
at
the
specific
code
for
that,
but
I'm
pretty
sure
we
we
cut
that
and
handle
that
the
security
group
itself.
A
E
A
So
we
have
on
the
documentation
on
our
website.
We
have
a
bunch
of
sample
policies.
You're
right.
The
custodian
language
is
very
easy
to
kind
of
understand,
but
you
do
it
does
help
to
get
some
initial
traction
first,
so
I'll
take
a
look
at
those
and
it
covers
a
wide
array
of
use
cases
so,
like
simple
compliance
just
stuff
like
cost
efficiency
using
off
hours
and
stuff
like
that.
A
Sure
so
I
would
start
by
checking
out
our
recently
revamped
website
pakistanian.io.
You
can
also
check
out
the
GitHub
repo,
it's
Cloud
custodian.
We
also
have
a
slack
and
getter
which
you
can
get
access
to
both
on
that
on
the
website
and
if
we're
in
there
you
know
pretty
active,
so
the
slack
is
relatively
new.
A
Oh,
yes,
and
we
also
have
a
YouTube
channel
yeah.
So
we
recently
had
governance
as
code
day
which
had
a
bunch
of
different
talks
from
all
sorts
of
different
users,
including
like
Intuit
as
well.
As
you
know,
stacklight
as
well
was
there
so
I
would
definitely
go
check
out
those
videos
and
then
I
think
we
have
a
just
a
bunch
of
other
videos
as
well
so
I'm,
not
even
on
the
YouTube
channel
about
custodian.
So.
E
B
So
right
now
it's
through
config
map,
so
basically
the
the
service,
that's
running
it's
going
to
just
mount
it
into
a
volume
and
then
read
all
the
policies
from
there.
So
how
you
mount
that
volume
into
it
doesn't
matter,
but
the
helm
chart
itself
is
just
going
to
load
it
up
into
a
config
map
and
then
just
read
them
from
there.
B
Yeah
and
eventually
we
may
introduce
crds
for
policies
so
that
people
can
deploy
them
that
way,
but
right
now
we're
trying
to
work
with
people
who
are
using
it
across
many
different
resources.
So
we
expect
them
to
not
only
be
deploying
kubernetes
policies
by
terraform
and
AWS
and
all
that
as
well,
so
we're
assuming
they'll
have
a
repo
of
policies.
At
that
point,.
B
G
C
G
A
A
So
the
question
was:
how
can
we
basically
fix
tags
on
resources
that
are
already
out
there?
What
cloud
are
you
using
okay,
cool,
okay?
So
for
AWS
we
have
a
tool
called
a
trail
Creator,
it's
not
super
well
known,
but
they
basically
can
scan
your
cloudtrail
event
history
and
then
do
resource
attribution
based
off
of
that,
and
that's
a
like
you
know
what
you
described
right
now
is
basically
the
exact
scenario
that
many
many
large
companies
are
in.
A
It's
also
the
scenario
that
basically
caused
Cloud
strain
to
even
exist
in
the
first
place
is
there's
a
whole
bunch
of
stuff.
We
don't
know
who
owns
it.
We
got
to
figure
it
out
because
we
gotta
also
govern
it.
So
I'd
recommend
checking
out
it's
under
the
tools
directory.
So
it's
not
hey.
You
guys
go
like
one
one
directory
in
but
yeah.