►
From YouTube: Notary v2: Redesigning the Secure Supply Chain for Containers - Justin Cormack & Steve Lasker
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon Europe 2021 Virtual from May 4–7, 2021. Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Notary v2: Redesigning the Secure Supply Chain for Containers - Justin Cormack, Docker, Steve Lasker, Microsoft, & Omar Paul, Amazon
The Notary v2 project was launched at Kubecon North America in 2019, as a joint community effort to resolve issues with the first generation Notary, which was launched five years ago. Since then we have learned a lot about how containers are used in practise and the security requirements, and Notary v2 builds on that experience from the whole community. The protocols are OCI registry native and designed to improve the supply chain security of the whole container ecosystem. This talk gives an overview of the progress, and the problems being solved, and then a deep dive into the state of the specification and implementations. We also cover current open issues and the road to production.
https://sched.co/ekIB
A
A
A
Who
are
you,
who
are
we
indeed,
I'm
justin
cormack,
I'm
an
engineer
at
docker,
I'm
also
a
naturally
maintainer
and
a
member
of
the
cncf
technical
oversight
committee.
I
work
a
lot
with
security
and
cnc,
so
I'm
really
interested
in
all
the
bits
of
how
security
fits
in
with
containers,
but
I'm
also
I've
been
working
for
you
know.
For
five
years
now,
at
docker
I've
been
very
much
embedded
in
the
whole.
How
do
we
do
containers?
How
do
we
make
them
better
space.
B
B
A
bit
adjusted
in
it,
so
I'm
steve
lasker.
I
work
at
microsoft
on
the
azure
container
registries
for
acr
and
mcr.
I
work
a
because
of
that,
because
registries
are
this
source
of
all.
You
know
cloud
computing.
At
this
point
I
get
involved
a
lot
of
the
end-to-end
workflows,
including
security,
and
as
part
of
that
I
work
a
lot
with
the
community
and
the
oci
technical
oversight
board
for
things
like
notary,
as
well
as
artifacts
and
other
things
that
we
do
under
oci.
A
A
The
way
I
like
to
tell
the
supply
chain
security
story
is
back
in
in
the
good
old
days
before
the
whole
cloud
thing
happened.
You
had
things
like
hardware,
firewalls
and
actual
cables
you
plugged
in,
and
you
knew
that
if
the
computer
was
plugged
into
this
cable,
it
would
get
this
network
and
straightforward
things
like
that.
But
everything
is
software
now
and
if
an
attacker
can
just
modify
your
software,
you
can
change
absolutely
anything
to
do
with
the
infrastructure
and
connect
the
computer
to
a
different
computer.
A
Rewire
things,
rewire
networks,
rewire
connections,
rewire,
tls
certificates,
rework,
you
know,
change
absolutely
everything,
and
so
your
code
suddenly
starts
to
really
matter,
and
it
really
matters
that
the
right
code
gets
into
production
and
someone
hasn't
come
along
and
made
any
modifications
to
it
and
that
your
code's
got.
You
know
security
checks
in
it
and
that
you
understand
how
everything
gets
from
source
code
into
production
and
make
sure
the
right
bits
get
into
production
supply
chain
attacks
have
been
growing.
Not
petro
is
a
famous
one
from
2018
caused
billions
in
damage.
A
You
know
brought
down
a
lot
of
companies,
a
supply
chain
attack
indirectly
via
a
you
know,
a
ukrainian,
a
piece
of
accounting
software.
I
mean
probably
a
state
level
attack.
You
know
these
things
are
important,
and
so
we
want
these.
We
want
protections
against
these
in
the
container
ecosystem
because,
like.
B
B
A
And
actually,
in
many
ways
we
have
opportunities
to
build
better
practices
than
with
vms.
I
think
that's.
What's
one
of
the
exciting
things
background?
Not
only
is
it
in
a
way
I
mean
like
there
are
potential
risks
because
everything's
software,
but
actually
the
fact
that
everything
software
means
it's
more
mutable
there's
a
lot
we
can
do.
The
artifacts
are
more
manageable.
We've
got
a
better
ecosystem
around
them
than
vm's
ever
ever
did
so.
A
A
A
There
was
a
bunch
of
security
issues
serving
up
fake
packages
and
claiming
that
your
package
is
nginx
when
it's
not
serving
up
replay
attacks
having
an
old
package
and
saying
it's
a
new
one,
just
deleting
packages
from
mirrors
and
saying
that
there
aren't
any
updates
this
package
to
keep
people
on
an
insecure
version,
changing
dependencies
between
packages,
mixing
matching
differences
and
and
so
on.
So
there's
a
whole
set
of
papers
around
what
these
threats
were.
And
so
the
idea
was
to
kind
of
take
these.
A
Take
these
ideas
and
models
and
apply
them
to
containers
was
the
original
idea
was
originally
a
docker
project
and
back
in
2015,
and
we
worked
at
darker
with
the
nyu
people
who
had
invented
tough
and
let's
join
cncf
in
2017
as
one
of
the
one
of
the
quite
early
projects
in
cncf,
along
with
the
tough
specification
itself,
which
is
also
a
cncf
project,
but
there
was
a
kind
of
a
bunch
of
issues.
I
think
2015
was
an
awful
long
time
ago
in
the
la
in
the
world
of
containers.
A
That
was
back
when,
basically,
when
I
started
working
in
this
space
actually
was
not
involved
with
this.
Originally
it
just
been
launched
when
I
joined
docker,
I
think
my
first
dockercon
was
in
barcelona
and
I
think
that's
where
the
launch
was.
If
I
remember,
I
think
it's
that
one
yeah,
we
gave
everyone
yubikeys
for
signing
their
containers
under
the
prize.
I
think
I
still
have
mine,
but
back
then
containers
in
production
were
really
new.
We
didn't
really
know
what
all
the
issues
were.
A
A
I
think
there
was
a
bunch
of
design
mistakes,
we'll
talk
about
some
of
them
in
a
minute,
but
now
the
security
needs
are
really
really
important
and
getting
this
absolutely
right
is
important,
and
you
know
we've
had
time
to
reflect
on
what
the
best
practices
are
and
where
we're
going
with
containers
and
where
we
want
to
be
in
the
next
five
years,
a
bunch
of
issues
to
fix.
Are
they
the
first?
One
really
is
that
notary
was
added
as
a
side
car.
A
You
know
not
not
not
sidecar
and
they
can
in
the
exactly
container
sense,
but
it
runs
on
the
side
of
a
registry.
It
basically
has
its
own
database,
its
own
server,
and
that
was
quite
good
for
just
shipping
it
without
modifying
stuff
and
without
changing
stuff,
but
it's
actually
we've
really
developed
since
then,
a
kind
of
whole
registry
native
ecosystem,
I
mean
steve,
you
can
talk
more
about
that.
You've
been
involved
a
lot
of
that
recently
as
well.
Haven't
you.
B
Yeah
I
mean
it's
just
the
the
there
was
so
much
value
in
the
work
that
was
done
in
registries
and
the
abstraction
models
that,
when
we
started
looking
at
other
artifacts
that
we
needed
to
manage
in
deployment
such
as
helm-
and
you
know
we-
we
started
this
project
as
to
how
to
kind
of
stuff
it.
In
that
the
more
we
were
doing
that
you
know,
singularity
came
up,
say:
hey
we'd,
also
like
to
use
a
registry.
Cnab
was
starting.
There
were
some
questions
around
terraform
and
others
and
be
like.
B
Can
we
just
leverage
this
common
infrastructure
so
that
it
works
across
all
clouds,
because
we
can't
really
ask
the
helm
team
to
build
a
helm,
cli,
that's
unique
to
azure,
it
needs
to
work
across
all
registries,
so
we
went
down
this
oci
artifacts
approach
and
what
was
really
nice
about?
That
is
just
there's
so
many
things
that
we
can
now
leverage
that
you
know
global
production,
resilient
infrastructure,
including
putting
signatures
in-
and
you
know
where
we
talk
about
the
side
car
like
who
cares
about
the
implementation
details?
B
That's
kind
of
one
of
the
key
scenarios
we
see
is.
I
have
a
pub
something
that's
on
some
public
registry.
I
need
to
bring
it
in
my
private
registry,
so
I
have
control
over.
You
know
that
content,
whether
it
be
v-nets
or
making
sure
that
nothing
changes
on
me
if
the
signature
doesn't
move.
How
do
I
prove
that
this
thing
that
I've
got
in
my
private
registry
is
the
same
thing
it
wasn't
originally,
so
those
were
some
of
the
key
problems
that
we
just
had
to
overcome.
To
make
this
thing
usable.
A
And
it
is
so,
in
addition
to
that,
being
I
mean,
I
think,
in
a
way,
that's
been
our
number
one
requirement
the
way
through,
but
I
think
there's
a
whole
bunch
of
requirements
are
really
important
about
usability
and
usage.
We've
we
haven't
had
a
lot
of
people
using
notary
v1
we've
had
a
lot
of
complaints
about
usability
issues
with
it,
and
I
think
I
put
a
separate
thing
about
observability
and
understandability
and
debugging.
A
So
it's
kind
of
the
way
it
is
used
is
that
when
you
pull
a
container
it'll
pull
the
signatures
and
check
them
the
first
time
in
the
same
way
as
ssh
does-
and
it
says,
do
you
want
effectively
necessary
says:
do
you
want
to
use
this
host
key,
but
for
containers?
This
doesn't
work
very
well
because
a
lot
of
times
you're
pulling
a
container
it's.
Basically
it's
a
it's
an
ephemeral
server,
it's
always
first
use,
and
so.
B
And
that
is
something
that's
changed
a
lot
right
since,
when,
when
we
docker
first
started
this
project
years
ago,
we
didn't
really
think
about
the
serverless
kind
of
model
where
every
time
a
container
is
pulled.
It's
a
clean,
pristine
environment
that
the
clouds
kind
of
hand
you.
So
you
only
pay
for
what
you
use.
We
were
very
focused
on
you
know.
Vms
would
come
up
and
they'd
have
lots
of
cached
images,
and
that
was
okay.
A
Yeah
that
becomes
a
challenge,
and
people
are
actually
doing
this
for
security
reasons,
because
having
clean
starts
is
actually
a
good
way
of
keep
making
sure
that
everything's
refreshed
attackers
are
out
all
those
things.
So
there's
actually
good.
I
mean,
I
think
the
ephemeral
thing
is
actually
one
of
the
great
things
we've
actually
managed
to
do.
B
A
A
A
And
january
I
think
february,
even
though
I
think
we
had,
we
had
a
few
meetings
in
person
which
was
actually
really
great
lots
of
people
involved.
Nyu
with
the
people
who've
been
working
on
tough,
ibm,
red
hat,
vmware
loads,
more
jfrog,
all
sorts
of
other
people
have
been
involved.
So
it's
a
very
broad
harbor.
Yes,
yes,
multiple
parts
of
vmware,
so
it's
been
a
really
great
community
thing
coming
together
to
to
really
work
out
as
a
community.
A
What
everyone
wants
we
had
hoped
it
would
finish
sooner.
I
think
growing
consensus
is
actually
really
difficult,
especially
for
I
mean
things
like
signing
and
supply
chain.
Security
are
complicated
and
and
trying
to
get
everyone
aligned
with
understanding
what
each
other's
problems
are
and
what
the
kinds
of
solutions
that
we
want
to
explore
are
and
why
things
are
not
working
now
was
actually
really
complicated,
and
I
think
that
that
that
did
take
a
long
time.
A
A
A
B
B
It
meets
the
scenarios,
everybody
needs
and
then
we're
good
yeah
no
big
deal,
and
you
know
this
kind
of
highlights
that
what
we've
also
some
of
the
problems
we've
talked
about
is
you
know,
multiple,
the
multiple
signature
problem,
which
has
an
effect
in
a
number
of
different
ways,
so
we'd
like
to
try
to
have
fun,
because
some
things
can
get
way
too
serious.
So
we
have
you,
know
a
company.
B
We
just
you
know
fictitious
wabbit
networks
that
creates
this
net,
monitor
software,
it's
an
isv
kind
of
product
where
they're
building
it
in
their
environment
and
they
create
a
key
that
their
that's
great
key.
They
create
a
signature
with
their
key
and
they
sign
it.
They
could
then
post
that
content
on
docker
hub
as
a
curated
location,
but
acme
rockets
is
this
company
that
wants
to
consume
the
software.
They
don't
know
who
abbott
networks
are,
but
they
trust
docker.
So
docker
certifies
that
content.
So
great.
Okay,
I
will
now
use
that.
B
So
if
I
go
to
docker,
I
will
actually
see
two
signatures,
one
from
docker,
one
for
web
at
networks,
and
then
the
acme
rocket
team
can
bring
that
software
in
as
they
bring
the
net
monitor
software
into
their
environment.
They'll
only
bring
in
stuff,
they
trust
from
docker
hub.
So
that's
a
first
check
for
the
signature.
They'll
run
some
functional
tests
to
make
sure
it
works
in
their
environment
because
it's
not
just
the
first
time
they
wouldn't
be
able
to
get
updates.
B
And
then,
if
it
passed
those
tests,
they
will
add
a
third
signature
which
is
the
acme
rocket
signature.
So
now,
when
they
try
to
move
it
to
their
production
environment,
they
actually
don't
care
about
the
other
signatures.
They
only
will
put
things
in
production
that
has
an
acme
rockets.
You
know
validation
on
it,
so
you
can
kind
of
see
this
progression
of
flow
where
the
software
starts
all
the
way
over
and
you
know
way
upstream.
If
you
will
moves
into
another
environment,
the
signatures
move
with
it.
B
A
A
Is
there
were
too
many
keys
and
there
was
a
huge
proliferation
of
them,
and
so
we
need
to
arrange
keys
in
manageable,
but
not
too
extreme
way,
and
so
there's
all
sorts
of
of
issues
that
follow
on
from
from
these
scenarios
that
we're
going
through
we've
made
quite
a
lot
of
progress
with
storing
signatures
in
registries.
I
think
I
mean
we've.
We
one
of
the
aims
here
has
been
to
have
a
very
generic
signature
model.
We
don't
want
to
just
constrain
signatures
to
being
something
that's
specific
for
notary
v2.
A
It's
like
signatures
are
something
that
could
be
used
for
other
security
applications,
and
so
we
want
to
have
a
generic
solution,
but
there's
we
had
one
requirement.
That
really
is
one
of
the
things.
That's
actually
meant
that
we've
looked
at
actually
making
registry
changes.
So
there's
this
question
about
inline
versus
detached
signatures.
A
Really
the
way
to
understand
that
is
like
is
the
signature
part
of
an
object
and
therefore,
as
registry
is
a
content
addressable
part
of
its
content
hash,
because
either
the
signature
is,
you
know
actually
part
of
the
json
manifest
of
the
object
or
it's
referenced
from
the
json
manifesto
of
the
object.
And
so,
if
you
want
to
change
the
signature,
you
actually
have
to
change
the
object
and
then
in
the
registration
means
you
have
to
re-tag
the
object.
A
Now,
a
lot
of
people
don't
like
the
re-tagging
model
in
registries.
Lots
of
registries
are
now
implementing
models
where
tags
are
immutable
and
can't
actually
be
changed.
Once
they've
been
once
the
content
has
been
shipped
to
the
registry,
and
so
there's
there
was
a
big
we
don't
want,
we
don't
want
re-tagging.
We
want
to
be
able
to
add
another
signature
for
a
different
purpose.
A
A
A
Like
it's
actually
an
additional
signature,
it's
not
a
problem,
it's
still
genuine
abundance.
So
these
kinds
of
concerns
meant
that
we
we've
been
actually
proposing
a
major
change
in
registries
to
add
extra
apis
which,
where
you
can
basically
have
a
well
a
detached
signature
which
basically
just
means
a
signature.
That's
in
a
separate
document,
but
not
just
detected
as
in
a
separate
document,
but
as
in
there
isn't
a
pointer
to
it
from
the
registry
object.
A
B
B
I
mean
one
of
the
one
of
the
interesting
ones
about
that
is
you
know
people
have
these
diploma
manifests
whether
it
be
a
kub
yaml.
You
know
a
helm
chart
or
just
even
a
docker
file
that
references,
something
with
a
digest
and
a
a
tag
and
there's
you
know,
tag
locking
you
know
semantics
and
so
forth,
and
even
docker
lock
and
some
projects
that
are
out
there.
B
A
Yeah,
so
so
just
to
talk
about
we've
started
prototyping
a
couple
of
months
ago.
Now
these
are
not
in
any
way
meant
to
be.
Like
final
things
do
not
use,
and
but
these
are
kind
of
worked
examples
to
show
how
things,
how
things
work,
how
we're
putting
things
together,
how
we're
experimenting
with
things
and
the
kinds
of
things
so
that
people
can
see
them
and
but
they're
they're,
not
totally
they're,
not
just
totally
fake.
A
These
actually
implement
the
actual
work
behind
the
scenes
and
there's
actual
things
in
registries
and
their
signatures
and
registries
and
all
the
formats
and
so
on
are
documented
and
they're
all
in
the
and
the
pr's
that
these
correspond
to.
So
it's
it's
pretty
it's
it's!
It's
very
real
prototyping,
not
just
mock-up.
B
So
there
nothing
up
our
sleeves,
we'll
take
a
look.
We
have
nothing
locally
on
the
machine.
So
what
I
want
to
be
able
to
do
is
you
know,
build
and
sign.
So
what
I'm
going
to
do
is
just
make
things
a
little
bit
simpler
and
I'm
going
to
set
an
environment
variable
for
our
image.
We've
got
this
registry
nv2.azurewebsites.net
and
it's
hosting
the
changes
that
we've
been
proposing
to
the
distribution,
docker
distribution
for
the
distribution
spec.
B
B
That's
not
helpful,
oh
thank
you
minor
detail.
So
now
we
can
see
that
I've
got
my
image
and
that's
great
and
there's
that
image
now.
What
I
want
to
do
is
when
I
sign
it,
I
you
know
I
need
to
enable
the
notary,
workflow
so
I'll,
say
docker
notary
enabled
and
what
we're
doing
here
is
we're
enab.
You
know,
saving
this
configuration
and
now
you'll
see.
We've
got
this,
for
you
know
our
initial
prototype,
where
we
have
just
enabled
true
so
now,
when
I
want
to
sign
this
thing.
B
I'm
going
to
say
docker
notary
sign
and
I
give
it
the
key
for
the
azure
website's
net
key
and
the
cert
for
these
two
and
then
I'm
signing
this
image
so
we'll
generate
the
manifest
because
we're
signing
a
manifest
and
we
output
this
jwt
token.
That's
what's
the
the
oci
artifact.
That
is
the
signature
that
we
want
to
push
to
the
registry,
but
we
don't
want
users
to
have
to
think
about
pushing
and
pulling
signatures
independently.
B
We
want
to
build
it
into
the
experience,
so
I'm
going
to
log
into
the
registry
and
when
I
do
docker
push,
what
we'll
see
is
because
we're
using
this
docker
plug-in
right.
We
didn't
actually
change
the
docker
api
docker
cli,
yet
we're
using
a
docker
plug-in
that
we're
routing
this
to
so
now
we're
just
going
to
be
able
to
do
is
push
the
actual
image
and
we're
going
to
push
the
signature.
It's
just
an
additional
component
and
then
when
I
want
to
do
a
docker
pull,
but
let's
get
rid
of
our
images.
First.
B
I
do
a
docker
pull.
I
have
not
configured
anything
yet
so
it's
going
to
go.
Look
because
I
have
notary
enabled
says:
hey,
I
found
a
signature,
but
you
don't
have
any
of
those
keys
so
by
the
fact
that
I
don't
have
any
keys
local
that
match
it.
It
is
by
definition,
going
to
fail,
so
I
don't
have
anything
to
enable
this.
B
What
I
do
is
I
put
the
reference
to
the
key:
the
azure
web
notary,
v2,
azure,
website,
cert
and
save
that,
and
now,
because
I
have
that
key
locally.
It's
going
to
look
up,
signatures
found
a
signature
and
it's
a
valid
signature.
So
now
I
am
going
to
be
able
to
see.
I
have
now
found
our
image,
so
that's
the
prototype
right,
we're
talking
about
prototypes
from
end
and
we'll
continue
to
experiment
with
this
and
decide.
B
Yes,
so
what's
interesting
and
justin
was
trying
to
talk
about
this
before?
Is
you
know
everything
related
to?
How
do
I
want
to
store
things
in
registries?
It's
more
than
just
images
and
home
charts
and
cnabs
and
oppas
and
wasms
and
other
things
I
want
to
be
able
to
store
signatures
and
have
a
movement.
In
fact,
I
want
to
be
able
to
put
verification
objects
signatures
on
any
of
those
things.
B
Well,
I
also
need
to
know
well
who
signed
the
signature.
What
git
commit
was
associated
with
it.
What
was
the
when?
Was
it
added
and
so
forth,
so
we
and
once
we're
doing
this-
reverse
lookup
on
hey,
I'm
looking
for
this
thing,
what
artifacts
are
impacted
by
this
thing,
then
we
can
actually
start
to
create
more
of
this
generic
api
for
finding
these
things
and
that's
been
kind
of
a
lot
of
the
feedback
as
well.
It's
like,
if
you're,
going
to
make
a
change
to
our
registries.
B
A
B
Generic
apis
and
the
generic
tools
that
will
do
that.
So
we
are
definitely
making
sure
that
we're
doing
that,
you
know
canonical
balancing
of
the
boiling
the
ocean,
which
will
never
happen
and
versus
a
small
pond.
So
we're
trying
to
make.
We
feel
good
about
the
progress
thus
far,
and
these
prototypes
help
us
figure
out,
for
instance,.
B
A
A
Then
get
agreement
on
the
directions
from
these
prototypes
and
make
sure
there's
actually
other
process
other
than
what
we're
showing
that
are
going
on
as
well,
so
and
so,
and
then
we
have
to
check
everything
against
our
threat,
modeling
that
we
did
earlier
on
and
then
get
production
next
year,
which
would
be
great
for
fun.
So
if
you're
we
just
it's
just
let's
just
get
past
twenty-two.