►
From YouTube: Hardening the Kubernetes Software Supp... Adolfo García Veytia, Verónica López González, Nabarun Pal
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Hardening the Kubernetes Software Supply Chain Through Better Transparency - Adolfo García Veytia, uServers; Verónica López González, Digital Ocean; Nabarun Pal, VMware
Software supply chains are gaining increasingly complex nowadays, especially when it is about deploying cloud native environments securely. After refactoring the Kubernetes release process over the past years, SIG Release efforts have shifted towards three main areas of work. In this talk, Verónica, Nabarun, and Adolfo will cover all of them in-depth: * Starting with Kubernetes v1.22, every release includes an SPDX Bill of Materials describing the source code, binaries, and all published images. * Automatic verification of the integrity and consistency of release artifacts as part of the Kubernetes Release process. * Digital signing of released artifacts and signature verification of upstream images. In the final part of the presentation, the speakers will demonstrate some of the tools that SIG Release has created, which can be leveraged today by the community in other projects, too.
A
And
we're
part
of
the
big
release
engineering,
family
for
kubernetes
team
and
we're
presenting
today
a
little
bit
of
the
work
that
we
have
been
doing
for
the
past
few
years,
but
that
obviously
wouldn't
be
possible
with
the
whole
team
and
yeah.
Also
nabarun
will
be
joining
us
from
india
because
at
the
last
moment
he
couldn't
come.
So
I
hope
that
you
enjoy
this
weird
hybrid
format
and
let
us
know
if
you
have
any
questions.
C
C
C
So
until
february
last
year,
a
lot
of
the
tasks
done
in
a
release
were
performed
using
a
tool
called
enago
and
some
other
helpers
written
in
bash.
It
was
amazing
to
see
all
of
the
heavy
lifting
done
by
kodian
batch,
but
there
are
problems
as
well.
Although
the
tooling
worked
really
good,
really
great,
there
were
some
downsides,
a
particularly
big
one
was
testing.
It
is
comparably
harder
to
test
code
written
in
dash
using
this
set
of
tools
also
involved
some
really
nasty
war
stories
while
cutting
releases.
C
C
The
team
took
a
really
pragmatic
approach
into
moving
one
component
at
a
time
into
the
new
way
of
doing
things,
and
as
such,
there
was
an
iterative
development
process.
For
example,
in
this
tracking
issue,
you
can
see
what
all
micro
tasks
were
done
and
what
steps
were
taken
in
order
to
port
the
code
into
code.
D
D
C
C
Jobs
into
google
cloud
build
so
that
we
don't
have
to
run
it
on
our
own
machine
resiliency
and
not
just
at
a
few
places
all
across
the
board.
The
tooling
now
can
handle
situations
like
network
errors.
It
handles
them
with
appropriate
amount
of
retries.
If
things
fail,
we
can
start
from
the
same
place.
C
Security.
That
aspect
is
also
enhanced
a
lot.
We
can
now
reliably
filter
secrets
from
outputs.
We
can
also
keep
also
use
key
management
services
if
we
want
to
store
some
secrets
out
there
and
then
retrieve
them
in
the
pipeline
and
at
the
same
time
ensure
that
the
the
logic
works.
Well,
since
all
of
this
tooling
is
written
in
go,
we
also
get
like
faster
execution
types.
C
C
C
C
A
All
right
so
again
for
those
who
are
late,
veronica
lopez
and
release
manager
recently
promoted
from
associate
and
well.
I
have
been
a
part
of
the
release
family
for
a
while,
like
we
have
internal
jokes
about
it,
and
I
have
been
serving
in
different
roles
for
like
around
seven
releases
already,
and
the
change
has
been
real.
Like
change
has
happened
before
my
eyes,
like
in
real
time,
I
remember
as
part
of
other
roles
or
other
sub
teams
and
and
the
release
team.
A
How
sometimes
the
wrangling
and
fighting
the
tooling
was
more
time
consuming
and
labor-intensive
than
the
actual
tasks
that
we
were
trying
to
achieve.
But
you
know
like
that
is
systems,
programming,
and
that
is
kubernetes
programming,
so
yeah,
so
with
par
part
of
the
things
that
we're
presenting
today
and
that
navarro
already
mentioned
are
like
the
evolution
of
of
how
we
we
we
are
here
now.
A
That
is
the
part
that
I'm
going
to
talk
about
and
how
has
the
past
work
and
the
present
work
set
us
for
success,
hopefully
for
the
future
and
all
the
efforts
that
that
that
will
involve
okay.
So
we
we
have
to
go
a
little
bit
back
to
the
yeah
sorry.
I
cannot
see
okay
back
in
the
day,
with
all
of
this
description
that
I
told
you
of
the
tooling
that
we
had,
even
though
we
had
amazing,
teammates
and
and
who
knew
their
their
job
like
the
palm
of
their
hands.
A
Sometimes
these
processes
didn't
didn't
inspire
a
lot
of
confidence
to
like
people
outside
so
little
by
little
we
have
been
able
to
provide
tools
that
have
provided
the
community
with
more
certainty
and
guarantees
of
about
the
products
that
we
release.
You
know
some
examples
of.
This
are
artifact
by
verification.
A
This
is
just
like
an
example
of
how
we
have
been
adopting
or
or
inheriting
certain
parts
of
the
release
process,
but
in
the
past
we
needed
people,
for
example,
for
in
this
specific
case
from
google
to
like
when
we
were
done
with
a
process
like
we
needed
someone
from
google
to
to
do
the
next
step.
You
know-
and
that
was
I
mean
even
as
we
we
love
them.
Sometimes
they
are
super
busy
and
like
just
we
have
people
in
europe.
A
We
have
people
in
india,
we
have
people
in
america
and
the
american
continent,
and
just
it
was
madness,
yeah,
so
being
able
to
now
own
this
process
and
others
as
well
has
given
us
more
freedom
to
just
move
forward
yeah
and
so
one
of
the
main
things
personally
speaking
wins
of
of
our
team
and
stephen
is
here,
so
he
won't.
Let
me
lie.
I
was
stripping
basil
from
from
all
of
this
tooling.
A
A
You
know,
because
we
couldn't
find
anything
else
and
you
people
were
not
going
to
to
reinvent
the
wheel,
yeah
so
being
able
to
finally
achieve
a
phase
where
we
could
get
rid
of
it
at
different
points.
In
on
the
release,
it
was
amazing
and
a
huge
technical
effort
and
well
yeah.
We
have
other
other
elements
that
you
can
see
here,
but
that
I
wouldn't
like
to
repeat
or
or
that
adolfo
will
mention
at
more
detail
but
yeah.
A
So
all
of
this
amazing
work
from
all
the
team
and
like
we
explicitly
are
not
mentioning
names
except
for
like
a
few,
because
we
are
very
aware
that
we
will
be
missing
like
people,
so
it's.
This
is
a
love
letter
to
the
release
team
because,
like
it
definitely
wouldn't
all
these
huge
efforts
wouldn't
exist
without
the
coordination
of
a
lot
of
people.
A
B
Yeah
thanks
veronica
yeah,
my
name
is
adolfo
garcia.
I
am
a
technical
lead
with
kubernetes
release
for
the
release,
engineering,
sub
project
and
yeah.
If
you
saw
the
history
and
evolve
evolution
of
the
tooling
that
powers
the
kubernetes
releases,
it
basically
are
it
considers
steps
to
be
where
we
want
to
be
right
now,
so
this
this
lag.
B
So
I
want
to
break
it
break
this
this
effort
in
three.
Just
to
give
you
a
quick
update
on
what
we
are
working
on
right
now,
so
the
first.
Our
first
aim
is
to
sign
whatever
of
our
artifacts.
We
can
the
most
artifacts
and
the
actual
every
file
and
container
image
that
gets
consumed
by
someone
downstream.
B
We
want
to
sign
that
we
want
to
give
the
community
the
guarantees
of
authentication
and
and
on
repudiation
of
the
artifacts
that
we
were
producing,
and
this
involves
several
changes
that
we
need
to
put
into
place.
But
now,
as
veronica
was
saying
now
that
we
have
like
stripped
and
re-re-uh
re-owned
the
container
image
promoter-
and
it's
now
in
a
ready
state
too,
so
that
we
can
start
iterating
over
this
project
by
the
way
is
not
one
that
originated
within
sig
release.
B
B
We
ended
up
having
the
the
owning
the
project,
and
this
will
allow
us
to
implement
the
the
sign-in
code
and
necessary
features
to
have
have
a
send
containment
images
at
some
point
and
the
other
one.
We've
been
working
about
is
the
kubernetes
bill
of
materials.
That
effort
is
mainly
done.
I
mean
it's.
It's
still,
of
course,
like
every
evolving
software
project,
also
open
to
suggestions,
comments
whatever,
but
you
can
already.
B
We
are
trying
to
push
towards
salsa
compliance
in
our
in
our
part.
So,
as
I
said
before,
kubernetes
is
not
the
whole
supply
chain.
Software
software
supply
chain,
but
certainly
the
most
important
artifacts
happen
in
the
links
that
fall
under
our
responsibilities.
So
we
are
trying
to
push
and
secure
them
as
soon
as
we
can
so
that
we
can
provide
guarantees
to
the
community
so
salsa,
just
we're
not
going
to
deep
dive
into
the
what
it
does.
B
B
Simply
the
some
of
the
security
features
that
salsa
requires
a
game
already
were
already
done
from
the
beginning.
I
think,
and
so
we
are
just
trying
to
match
up
and
have
like
a
benchmark
to
see
where,
how
are
we
how
we
are
going
to
to
to
to
match
with
the
success
level
one
requirements?
Of
course,
we
would
like
to
push
forward
to
further
levels,
but
we
see
that
it
requires
a
little
bit
more
work
to
achieve
there.
So
we
currently
have
an
open
tracking
issue
where
we
are
gathering
ideas.
B
We
are
already
have
gold
to
reach
salsa
level,
one
and
as
a
prototype,
and
we
are
going
to
be
testing
this
in
the
coming
releases,
which
is
just
providing
the
basic
attestation
files
of
provenance
of
the
artifact,
so
provenance
in
the
south
speak
means
that
you
should
be
able
to
take
one
of
the
artifacts
and
trace
it.
All.
B
B
There
are
lots
of
aspects
of
this.
I
think
the
most
challenging
right
now
is
the
human
part
of
the
problem,
because
we
we
have
to
modify
if
we
were
to
achieve
higher
levels
of
salsa
compliance
for
releases,
we
would
have
to
actually
go
and
modify
some
of
the
practices
with
the
way
contributors
authorize
changes
to
the
code,
the
the
code
reviews
and
things
like
that.
So
I
think
the
main
message
here
is
that
we
need
to
have
the
conversation
we
need
to
have
some
discussions
about.
B
How
are
we
going
to
handle
the
kubernetes
release
process
within
a
world
that
is
constantly
demanding
more
and
more
security
around
the
software
supply
chain?
We
we
try,
we're
going
to
be
opening
the
caps
and
most
of
the
there
are
alternatives
for
all
of
the
implementations
from
the
attestation
files
to
the
signing
of
the
artifacts
to
the
way
we
store
them
inside
of
the
container
registries.
B
So
we
want
to
hear
from
everyone
we
want
to
hear
from
everyone,
and
we
want
to
collectively
reach
and
find
the
best
solutions
for
each
of
these
problems
and
cell
size.
Simply,
let's
say
like
a
bunch
of
check
boxes
that
we
that
guide
us
to
see
if
we
are
complying
with
some
of
the
the
things
that
are
expected
from
us.
So
I
I,
if
you
are
going
to
have
a
takeaway
from
this
talk,
is
we
need
involvement.
B
It's
the
people,
it's
the
community
that
drives
everything
behind
it
and
we
don't
want
to
appear
like
we're
pushing
into
one
direction
without
talking
to
everyone.
So
please
come
and
weigh
in
on
the
issues
on
the
caps
and
let
us
know
if
you
have
any
of
the
any
suggestions
and
we're
of
course
happy
to
to
to
hear
from
you
so.
B
And
I
guess
just
one
final
final
remark
for
me:
is
that
things
in
the
software
supply
security
chain,
the
security
side
of
the
of
software
supply
chains
is
evolving
very
rapidly
and
if
you
have
experience,
if
you
have,
if
you
want
to
help
the
project
in
this
rapid
evolving
space,
please
come
join
us.
We.
We
really
need
your
help.
So
we
are.
B
We
try
to
keep
things
as
current
as
we
can,
but
we
are
small,
a
very
small
team
of
volunteers
working
on
this,
so
please
come
and
join
and
from
either
just
making
your
voice
heard
or
actually
submitting
prs
to
help
a
release
process.
You
are
welcome.
You
are
encouraged
to
join.
You
are
absolutely,
we
absolutely
need
you
and
yeah.
So
if
you,
I
think
we
could
open
for
some
questions
if
there
are
any
and
and
thanks
again
for
coming
and
listening
to
her
to
our
experiences
yeah,
we.
A
B
A
D
D
B
You
can
reuse
some
of
the
code
that
we
used
to
launch
the
builds
inside
of
google
cloud,
build
and
the
bill
of
materials
tool
as
well,
and
some
of
the
some
of
the
provenance
at
the
stations
at
the
station
files
that
we
are
currently
producing
to
reach.
Salsa
one
are
going
to
be
built
into
the
blue
of
materials
tool
soon,
so
that
you,
whenever
you
run
the
developed
materials
for
your
project,
you
you
can
actually
output
the
separate
file.
B
The
the
predicate
needed
to
have
the
the
the
provenance
data
for
your
project
and
to
make
you
closer
to
reaching
someone
and
yeah.
So
yesterday
we
were
discussing
about
if
there
was
a
way
that
other
projects
could
come
and
reach
out
to
us
to
have
some
either
help
or
pointers
of
our
software
to
run
the
releases.
Absolutely
we're
open
everything
is
open
source
everything
tries
to
be
as
kubernetes
agnostic
as
possible.
B
We
even
have
now
a
new
repository
where
we're
trying
to
make
sure
that
everything
that
lives
in
that
repositories
has
zero
kubernetes
specific
content.
So
when,
if
you
go
to
kubernetes
release
engineering,
it's
still
kubernetes
specific,
but
we
have
another
repository
where
I'll
share
the
links
later
in
the
presentation
where
everything
there
is
ready
to
run
for
any
project.
So.
A
Yeah
as
an
example,
a
quick
example
of
this
like
without
going
like
super
deep
into
the
process,
someone
recently
told
us
that
they,
the
the
tool
as
it
is
right
now,
has
been
very
useful
for
some
teams
just
to
get
the
list
of
container
images
and
and
and
a
release
you
know
and
then
to
use
them
for
their
own
purposes
in
their
own
companies.
You
know,
but
like
it's
already
like
not
being
useful
for
people
without
the
full,
the
fully
fledged
features
you
know
so
definitely
like.