►
From YouTube: Ensuring Kubernetes manifests validity & compliance - a tooling overview - Yann Hamon, Contentful
Description
In an Infrastructure as code / GitOps world, testing that your Kubernetes configuration is correct, secure, and compliant to your company's requirements & best practices is more important than ever. An increasingly large list of tools is there to help you - linters, validators, testing frameworks, admission controllers... each working in subtly different ways.
To help you navigate these waters, I will present some of the most common tools for Kubernetes manifests validation & compliance testing, detail their use, limitations and provide some usage examples.
I will also introduce Kubeconform, a new Kubernetes validation tool I authored.
A
All
right
so
thanks
everyone
for
joining
on
a
friday
evening,
so
there
is
a
lot
of
tools
to
validate
your
manifest.
You
commit
manifest
I'll
try
to
provide
like
a
quick
overview
of
the
different
levels
of
validation
we
can
do.
I
assume
that
you
all
have
like
a
little
bit
of
communities
background
other
than
that.
I
hope
that
everyone
will
be
able
to
to
learn
everything
something
from
my
presentation
tonight.
So
with
me,
thanks
for
the
intro
so
yeah,
my
stephanie,
I
got
10
fold.
I
moved
to
berlin.
A
A
Also,
what
I
do
at
contentful
listed
here
also
three
different
projects.
I've
been
working
on
in
the
last
three
years:
core
dns
node
cache
is
a
node
local
cache
system
based
on
core
dns.
It's
open
source,
and
it
is
a
part
of
contentful
cube
secret
syncer
is
a
kubernetes
operator
to
sync
secrets.
Manager,
secrets
to
kubernetes
and
cube
conform
is
a
cube,
vowel
alternative
which
we'll
be
talking
about
a
little
bit
later.
A
A
A
A
If
you
look
carefully
and
you're
very
used
to
that,
you
might
see
that
there
will
be
a
few
mistakes,
maybe
in
there
I'm
super
pretty
proud,
but
I
think
it's
not
perfect
so
before
deploying
that
I
want
to
be
sure.
Is
this
correct?
Can
I
validate
that
this
is
right
before
actually
having
a
pr
someone
review
it,
deploying
it
and
realizing
way
too
late.
This
is
actually
not
correct,
so
our
mission
should
you
choose
to
accept.
It
is
to
use
available
and
open
source
tooling
to
identify
all
the
errors
in
this
manifest.
A
A
A
A
Another
tool
that
you
can
use
is
called
yamo
lint,
it's
kind
of
more
in
depth.
It
will
really
like
detect
trailing
spaces.
Inconsistent
alignment.
If
you
use
two
spaces
in
some
places
and
screen
others
duplicate
keys
can
be
useful.
You
can
recursively
test
folders,
so
you
here
it
detects
the
same
error
that
yq
found
and
also
another
one
with
a
trailing
white
space.
A
A
A
So,
however,
like
again
most
of
the
high
level
tools
will
also
validate
yaml.
Consistent
formatting
is
nice,
but
failing
a
white,
a
test
for
a
white
space
that
can
can
be
frustrating.
So
the
recommendation
here
is
look
at
the
the
linters
that
you
can
use
with
your
editor
embed
that
it's
always
better
to
have
something.
That's
consistent
and
not,
you
know,
fail
a
failed
job
later
on
for
a
like
a
semicolon
or
column.
That's
kind
of
misplaced
as.
A
So
what
do
I
mean
with
conformity
so
kubernetes
publishes
schemas
for
the
different,
manifest
you're
gonna
use.
There
is
one
big
file,
that's
kind
of
checked
in
the
kubernetes
repository,
it's
like
five
megabytes
big
and
it
describes
the
whole
api,
like
all
the
manifest
that
you're
gonna
have
the
kind
of
properties
you're
gonna
put
in
there,
which
ones
are
required
which
one's
optional.
What
types
these
fields
should
be
mostly.
You
want
your
manifest
to
conform
to
that.
A
For
example,
if
you
have
like
a
port,
that's
supposed
to
be
an
integer,
not
a
string
and
some
some
some
keys,
some
properties
will
be
required.
Some
will
not.
So
it's
it's
a
requirement
for
you
like
for
your
file
to
conform
to
the
schema.
That's
not
sufficient
for
the
minus
manifest
to
be
accepted
by
the
api
server.
A
A
This
will
alert.
You
was
not
supported
anymore,
so
it's
a
good
validation
before
upgrading
a
new
version
of
kubernetes.
It's
a
default
tool.
I
think
nearly
everyone
uses,
it's
called
cube
vowel.
So
this
is
how
it
looks
like
you
just
pass
it
the
list
of
files
you
want
to
validate
or
a
list
of
folders.
A
It
will
tell
you
which
one's
actually
valid,
which
ones
are
not
in
this
case.
It
warns
us
like
that.
The
service
I
I
passed
to
it.
It's
invalid
kind
must
be
one
of
the
followings
a
service
with
a
capital
s.
I
didn't
have
a
capital
s,
that's
an
error,
while
the
the
deployment
which
we
copied
from
the
quintessence
documentation
is
valid.
So
it
exists
with
one.
That's
simple
enough:
we'll
go
quickly
how
it
actually
works,
so
cubot
actually
doesn't
work
directly
with
the
swagger
definition.
A
The
reason
behind
that
is
the
tooling
to
work
with
swagger,
slash,
open
api
files
and
go
is
bad.
There
is
no
good
library,
it's
pretty
complex,
to
write
one
as
well.
So
what
garrus?
Like
the
the
author
of
cube
val,
has
done.
He
wrote
a
tool
called
open
api
to
json
schema,
which
actually
will
take
the
swagger
definition
converts
that
split
it
into
parts
and
it
converts
that
to
json
schemas,
for
which
the
tooling
and
go
is
much
better.
A
A
All
of
that
then
is
published
on
our
website
with
all
the
files
that
qval
will
actually
download
at
runtime.
So
when
you
don't,
when
you
men,
when
you
validate
the
service
cube
file,
will
look
at
this
website
commences.
Json
schema.dev
try
to
download
the
appropriate
schema
for
the
service
and
try
to
validate
your
file.
That's
what
happened
in
the
background.
A
So
kubota
is
great.
It's
very
widely
used.
It's
pretty
fast.
It
iterates
recursively
over
folders.
You
can
validate
different
versions
of
against
different
versions
of
kubernetes,
but
there
are
two
gotchas
so
first
it
says
there
is
no
support
for
custom
resources
that
the
documentation
says.
Actually
it's
possible,
it's
a
little
bit
tricky
and
there
are
a
bunch
of
not
so
easy
to
fix,
bugs
sometimes,
like
warnings
will
cause
cause
the
tool
to
expose
one
sometimes
zero.
The
output
is
supposedly
json
or
tab,
but
some
is.
A
A
So
I'll
go
back
to
this
no
support
for
custom
resources
because
welcome
to
the
world
of
quintus
operators,
there
are
more
and
more
communities
operators,
prometheus
jaeger,
cube
secret
sinkers,
the
one
we
developed
there
are
like
many,
many
like
for
so
many
customer
resources.
Nowadays,
if
you
don't
validate
that,
like
you,
you
end
up
with
like
a
a
lot
of
potential
mistakes.
Contentful
like
what
there
was
one
we
did.
A
A
Tldr,
it's
not
easy,
so
I
end
like
I
required
a
lot
of
lots
and
lots
of
changes,
so
what
I
ended
up
doing
is
trying
to
write
like
an
alternative
to
cube
valve
that
contained
the
changes
I
wanted
to
see
in
cuba.
So
this
new
new
tool
is
called
cube
conform,
so
it's
very
very
similar
to
cuban.
It
solves
all
of
the
issues
I
had
with
it.
A
A
A
A
You
will
you
convert
all
your
open
api
to
json
schema
you
put
that
in
a
folder
somewhere
with
your
library
of
custom
resources,
schemas
and
you
add
that
to
the
schema
locations
like
the
the
libraries
of
schemas,
that
you
confirm
will
look
up
into
so
cube.
Conform
is
very,
very
close
to
cuban
it's
kind
of
has
future
priority.
It
does
the
same
thing.
I
use
the
same
test
suite
to
validate
that
it
works.
It
fixes
a
lot
of
the
outstanding
cube
valve
box.
A
There
are
some
quite
nice
performance
improvements
like
three
to
four
times
faster,
at
least
on
a
machine,
quite
nice
when
you
write
it
in
a
ci
or
you
want
to
run
it
locally
a
few
times
and
the
support
for
customer
resources
is
quite
flexible.
You
can
choose
how
you
want
to
put
them
on
your
disk.
You
might
make
some
that
are
specific
to
a
specific
version
of
kubernetes,
etc.
So
that
supposedly
works
nicely
cons
is
like
it's
mostly
a
smaller
project,
smaller
community,
it's
pretty
new,
so
it's
not
just
as
battle
tested.
A
A
A
new
newcomer
with
I
was
surprised
to
learn
it's
not
too
much.
Advertise
cubecattle
also
does
client-side
validation,
so
when
you
run
cubecut
always
apply,
validate,
equals
true
and
then
dash
dryrun
equals
client.
So
that's
a,
I
think,
a
property
from
cube,
console,
118
and
up
it
will
actually
try
to
validate
on
the
client
side
the
manifest
so
like
in
just
exists.
We
will
try
to
just
say
it's
like
try
to
create
and
exist
with
zero
if
it
succeeds.
A
In
this
case,
however,
it
didn't
detect
the
error
that
cube
val
and
cube
conform
found,
so
the
capitalization
error
didn't
work,
so
I
tried
again
with
another
file.
Like
I
tried
to
break
the
deployment
I
used.
I
changed
like
the
ports,
which
was
an
integer
to
a
string,
and
in
this
case
it
actually
did
detect
it.
So
it
works,
I
think,
slightly
differently.
It
has
a
different
validation
mechanism,
I'm
not
sure
if
it
actually
uses
just
the
swagger
file
or
something
else,
and
also
it
requires
a
connection.
A
So
the
pro
says
it's
a
kubernetes
upstream
project.
Everyone
has
cube
cattle
installed,
it's
very
fast,
but
it
still
requires
a
connection
to
the
cluster.
I'm
not
sure
why,
like
I
open
the
ticket
on
that,
I
think,
maybe
to
figure
out
what
version
of
the
cluster
he
needs
to
bring
it
against.
I'm
not
too
sure
and
it's
unclear
what
exactly
is
validated
and
what
the
logic
is.
I
would
have
expected
the
service,
which
was
not
valid,
to
be
actually
detected
as
non-valid,
so.
A
You
need
a
capital
s
and
learnings
is
you
need
to
use
cube
valve
or
cubeconform
to
valuation
manifest?
There
is
a
lot
of
mistakes
that
we
found
regularly
with
that,
if
you
misname
a
property
or
you,
you
put
the
wrong
type,
something
supposed
to
be
a
string
or
an
integer.
It's
much
easier
to
figure
this
out
during
uci
before
trying
to
apply
that
then
actually
trying
to
apply
it,
and
maybe
it
will
apply,
but
not
do
what
you're
expected
to
do.
A
For
example,
classically,
if
you
have
a
property,
you
misname
it,
maybe
it
will
apply,
but
you
will
think
it
does
something
and
it
doesn't
keep
cattle
is
doing
starting
to
do
the
same
validation
as
well
as
of
2020
at
least
out
of
now.
I
think
it's
fully
documented
it's
unclear
what
it
does.
I
would
keep
an
eye
on
it
like
always
next
years,
but
right
now
I
would
advise
to
use
cuba
could
conform.
A
All
right
enforcing
best
practices
and
compliance,
so
there
are
different
once
your
file
is
a
valid
manifest.
You
want
to
ensure
that
it
conforms
to
your
your
best
practices,
your
security
requirements,
different
kind
of
questions
you
want
to
ask
like,
for
example,
if
you
want
to
say
all
deployments
should
have
resource
request
sets.
A
No
containers
should
run
as
privileged
processes
and
containers
should
not
run
as
rules.
I
think
there
are.
There
is
a
lot
of
questions
of
policies
you
can
put
in
there.
I
think
all
the
policies
depend
on
what
the
company
requirements
are.
Someone
like,
if
you
require
very
strong
security
like
there
is
a
lot
of
things
you
can
test,
for.
There
is
a
lot
more
things
you
can
do
like
if
you
want
to
ensure
like
every
deployment
as
namespace
set
explicitly.
A
A
But
in
the
end,
it's
a
really
good
language
to
to
test
your
manifest.
For
example,
all
manifest
should
have
a
namespace
explicitly
defined.
That's
how
I
would
test
it
with
conf
test.
So
if
I
have
a
policy
called
namespace.rego
that
I
put
in
my
folder
policies-
and
I
want
to
test
my
deployment
and
my
service-
I
see
that
both
fail,
because
I
do
not
have
a
namespace
explicitly
set,
and
so
this
will
fail
in
my
case,
for
example,
unlike
the
namespace
to
always
be
explicitly
defined.
B
A
A
It
uses
the
same
language
used
for
admission
controllers,
so
that's
something
I
won't
go
too
much
into,
but
mostly
there
is
a
control,
an
admission
controller
you
can
deploy
to
kubernetes
and
use
the
same
rules
that
you
would
write
for
conflict
test
to
actually
deny
some
resources
at
deployment.
So
it's
like
resident
testing
and
uci.
You
can
also
deny
deployment
of
some
manifest,
depending
on
some
of
the
policies
you
write.
A
So
it's
the
same
language.
So
it's
quite
useful
to
have
this
both
in
uci
and
at
kubernetes
level.
So
documentation
of
rego
is
excellent
and
there's
a
lot
of
existing
rules
so,
like
I
linked
here
like
a
a
lot
of
rules
for
benito's
which
will
go
across
security,
best
practices
etc.
So
community
is
here
so
you,
if
you
have
questions
and
need
help,
there
is
always
someone
to
reply.
A
The
whole
cons,
the
only
cons
I
have
here
is
rigo
can
be
unfamiliar.
It's
kind
of
one
more
thing
to
pick
up:
it's
pretty
unlikely
that
you
like,
before
you
start
working
with
it
that
you
know
this
already,
so
it
requires
like
a
couple
of
days
at
least
to
kind
of
reach
through
segmentation
and
pick
up.
It's
not
the
easiest
thing
in
the
world,
but
yeah.
I
think
two
days
is
a
good
time
to
start
with
property
with
us.
A
A
I
don't
think
it's
such
a
great
tool.
What's
actually
pretty
good,
is
the
list
there's
a
website
around
it
and
the
list
of
checks
it
actually
has?
There
are
a
lot
of
tests
with
a
good
description
every
time
of
why
success
is
important.
What
is
the
the
consequence
of
this?
Why
you
should
kind
of
make
sure
that
your
manifest
respects
those
rules?
A
A
A
A
However,
it's
not
extensible,
you
can't
write
your
own
rules,
but
there
is
a
good,
documented
list
of
checks.
So
that's
kind
of
my
same
advice
is
all
of
this.
You
can
do
also
using
conf
test,
but
add
your
own
as
well.
So
look
at
the
list
of
checks.
It
does
see
if
you
can
rewrite
them
in
rego,
and
then
you
it's
a
really
good
starting
point.
A
Cube
linter
is
very
similar
to
keep
score.
It
reports
errors
instead
of
scores,
it's
also
not
extensible
and
in
early
stages
of
deploy
development,
so
the
both
are
very
very
similar.
My
take
is,
I
think
you
you:
can
you
better
off
with
a
tool
that
you
can
extend
and
write
your
own
policies
for
last
honorable
mention
cubetest
is
also
a
testing
framework
with
tests
written
in
skylark,
which
is
a
subset
of
python.
A
A
A
So
learnings,
I
would
say
comp
test
and
opi.
Opa
is
a
de
facto
standard
for
policy
based
testing.
I
think
it's
more
flexible,
testable
and
pretty
fast
as
well.
So
I
think
I
think
it's
more
interesting
than
maybe
the
other
solutions
and
while
most
people
are
not
familiar
with
rigo,
it's
not
too
hard
to
pick
up
again,
maybe
one
two
days
and
you're
you're
good
enough
to
start
to
write
your
own
policies.
A
A
A
B
A
A
If
is
not
the
same,
then
we
get
an
error
that
says
the
service
points
to
a
non-existing
deployment.
So
this
would
be
like
a
rule.
I
would
run
with
conf
test,
and
this
actually
fails
and
shows
me
that
the
selector
I
had
for
my
service
was
incorrect
and
you
need
to
use
nginx
instead
of
nginx
servers.
So
this
is
a
good
example
of
a
task
that
actually
tests
to
manifest
two
different
files
at
the
same
time.
A
A
Try
runs
equals
client
and
in
this
case,
for
example,
a
file
actually
works,
but
if
I
use
a
dash
type
run
equal
server,
they
will
actually
send
the
file
to
your
kubernetes
api
and
which
will
try
to
apply
it
in
a
dry
run
mode
and
in
this
case
we'll
detect
one
additional
error
that
we
haven't
caught
so
far.
It
will
say
the
protocol
should
be
tcp,
that
was
a
written
in
capital
letters
and
we
wrote
tcp
and
small
letters.
A
So
that's
our
final
manifest
was
highlighted
all
the
the
mistakes
I
made
and
as
a
comment,
the
different
tools
we
used
to
figure
them
out,
so
there
are
yellow
errors.
There
are
some
errors
we
figured
out
because
we
added
a
policy
for
it
with
conf
test.
Some
we
used
to
control
and
dry
run
client
and
server
mode,
and
then
we
had
a
final
count
test
across
multiple
files
to
detect
the
error.
With
the
selector,
so
that's
my
take
on
this.
A
I
think
there
are
many
more
tools
to
use,
so
it's
very
much
an
opinionated
list,
but
I
would
recommend
you
use
a
good
yaml
linter
integrated
with
your
editor,
rather
than
build
that
new
ci
that
can
be
frustrating.
You
can
use
cuba
or
conform
to
ensure
your
files
are
valid
when
it's
manifest.
This
will
usually
save
you
time.
A
A
So
errors
and
manifests
are
easy
to
overlook.
You
should
definitely
test
them.
It's
easy!
It's
pretty
set
up
and
you
save
time
communities
will
not
detect
all
mistakes.
Sometimes
you
would
like
I've
seen
regularly
that
we
used
properties
that
did
not
exist,
and
actually
we
expected
them
to
do.
Something
and
kubernetes
has
just
accepted
the
manifest
and
ignored
it,
and
just
wasn't
doing
what
we
thought
it
would
and
testing
your
manifest
will
ensure,
will
help
you
enforce
best
practices
and
ensure
you
are
compliant
with
your
security
guidelines.