►
From YouTube: CRI-O: Secure, Performant, and Boring... Peter Hunt, Urvashi Mohnani, Mrunal Patel & Sascha Grunert
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
CRI-O: Secure, Performant, and Boring as Ever! - Peter Hunt, Urvashi Mohnani, Mrunal Patel & Sascha Grunert, Red Hat
Anyone who has followed CRI-O, the OCI compliant implementation of the Kubernetes Container Runtime Interface (CRI), knows that it aims to be secure, performant, and over-all boring. Implemented as exactly the CRI implementation Kubernetes needs, and nothing more, allows it to be optimized, secured, and version-locked for Kubernetes. In this talk, Sascha Grunert, Mrunal Patel, Urvashi Mohnani, and Peter Hunt will give an overview of CRI-O, as well as discuss some recent improvements that highlight these three key aspects of CRI-O. The talk will cover the ease with which it transitioned between CRI versions, optimizations in container exec probes with conmon-rs, security improvements regarding SELinux relabelling for container volumes, and general security enhancements by running seccomp by default. People who join us, whether seasoned end-users or budding community members, should learn what CRI-O has to offer as the container manager that loves Kubernetes the most.
A
B
D
B
Is
a
lightweight
oci-based
container
engine
implementing
the
kubernetes
container
runtime
interface?
It
supports
all
oci
based
container
images,
including
older
docker,
formatted
ones,
and
it
also
supports
all
oci
container
runtimes
such
as
francie
carter,
geovisor
and
cran.
To
name
a
few
cry
is
built
on
top
of
various
building
blocks,
focusing
on
different
aspects
such
as
storage,
image
and
networking.
B
With
this
structure,
crow
is
able
to
pick
and
choose
features
from
the
building
blocks,
while
these
projects
can
evolve
at
their
own
rate.
This
ensures
that
we
maintain
stability
while
enhancing
cry
with
new
features.
At
the
same
time,
from
day,
one
cryo
was
built
with
only
kubernetes
in
mind.
Our
versions
walk
in
lockstep
with
those
of
kubernetes
and
cry
evolves,
as
kubernetes
does.
Cryo
also
focuses
on
making
workloads
running
in
production
as
secure
as
possible.
B
So,
let's
take
a
quick
look
at
some
of
the
updates
from
the
last
few
months.
Cryo
now
supports
the
v1
implementation
of
the
cri,
as
a
cubelet
has
finally
transitioned
to
it.
While
the
cubelet
was
transitioning
from
v1
alpha
2
to
v1
cryo
had
added
support
for
both
cri
implementations
to
ensure
version
fewer
objects
for
common
operations
by
only
creating
a
new
object
when
a
change
has
been
detected.
B
D
Thank
you.
Let's
talk
about
how
we
can
rethink
container
lifecycle
management
in
cryo,
so
for
many
releases,
con
mon
was
the
desired
oci
runtime
monitor,
which
is
able
to
interact
with
run
c
and
c
run
as
lower
level
container
runtime.
You
can
check
this
project
out
on
the
containers
kanban
on
github,
if
you
would
like
to,
but
in
general
this
little
helper
tool
helps
us
to
ensure
the
communication
between
cryo
and
oci
runtime.
So,
for
example,
it
takes
care
of
the
container
creation.
D
It
also
takes
care
of
the
cleanup
when
the
container
terminates
and
it
also
provides
endpoints
for
executing
processes
inside
of
containers
and
also
provides
endpoints
for
attaching
to
a
containers.
For
example,
kronman
is
capable
of
writing
the
container
log
files
for
kubernetes
and
which
have
to
be
written
into
a
special
format,
and
it
also
decided
for
this
use
cases
for
podman
as
well.
D
Conmon
is
generally
designed
to
have
the
lowest
possible
memory
footprint,
but
there
are
some
drawbacks
with
this,
so
it's
written
in
c
and
therefore
finding
new
maintainers
for
com1
is
really
hard
for
us
and
the
main
interface
to
interact
with
con
man
is
the
cli.
So
over
the
past
years
we
extended
the
cli
in
the
same
way
as
we
extended
features
to
con
mon,
which
kind
of
makes
the
usage
a
bit
more
complicated
than
we
initially
thought.
D
Adding
new
features
to
con
man
also
increases
technical
depth.
Therefore,
so
not
we
don't
only
have
issues
finding
new
maintainers
for
con
one,
but
we
also
increased
like
a
good
app,
so
we
are
kind
of
moving
into
a
sinkhole
here
and
it
besides
that
it
still
has
some
run
time
dependencies
like
glib,
and
this
makes
static
linking
hard.
So
we
provide
a
for
example
for
cryo.
We
provide
a
static,
binary
bundle
which
contains
all
runtime
dependencies
out
of
the
box,
and
con
m1
relies
on
g-lab.
D
So
we
have
a
little
bit
of
troubles
linking
statically
and
today
we're
happy
to
announce
that
a
possible
successor
of
conmon,
which
is
called
mono,
is
kanmon.
Is
contains
a
completely
new
software
architecture,
for
example,
we
can
utilize
the
programming
language,
rust
and
also
the
asynchronous
took
you
around
time
to
rethink
what
we
have
done
in
the
past.
Beside
that,
we
can
also
target
the
lowest
memory
usage
as
possible.
We
can
support
something
like
having
multiple
containers
like
parts
in
one
instance
of
kanban,
and
we
can
re-engineer
the
indoor
containers
process
execution
beside
that.
D
We
can
provide
an
extensible
api
and
the
golan
client
out
of
the
box
to
be
used
from
container
end
times
like
cryo
apartment,
but
how
to
provide
an
api
between
rust
and
golem.
Well,
we
decided
to
choose
captain
broto
for
that
and
captain
brother
is
an
insanely
fast
data,
interchange
format
and
it's
in
capability
capability-based
rpc
system.
So
it's
like
protocol
buffers,
but
a
little
bit
faster,
and
it
also
has
a
smaller
memory
footprint.
So
we
thought
it
would
be
a
great
use
case
for
captain
proto
to
try
this
out.
D
Captain
proto
is
faster
than
protobuf
and
also
has
a
smaller
memory
footprint
than
a
grpc
runtime.
So,
for
example,
if
we
want
to
create
a
container
here,
then
we
can
do
it
in
the
same
way
as
we
do
it
in
grpc.
So
we
define
a
create
container
request,
which
contains
the
request
and
as
well
as
the
response
and
the
content
create
container
request
itself
requires
all
the
necessary
data.
We
usually
have
been
passed
over
via
the
cli
to
command
before
so
we
need
the
container.
D
I
t
the
bundle
path
determine
and
we
have
to
decide
if
we
want
to
choose
a
terminal
or
not,
and
for
example,
we
can
also
specify
different
log
drivers
now,
and
this
is
new,
so
we
can
specify
multiple
log
drivers.
For
example,
we
create
a
new
structure,
lock
driver
which
has
that
type
and
the
path
and
for
now
we
only
support
the
container
runtime
in
the
face.
D
A
D
Arguments
to
start
conon
beside
that
it
allows
us
to
use
the
streaming
capabilities
of
captain
proto
rather
than
using
the
sockets
for
attach
right
now
we
still
rely
on
sockets
because
we
kind
of
want
to
mimic
the
old
behavior
of
kanban,
so
kanmon
is
unfortunately
not
ready
for
production
uses.
Yet,
but
right
now
we
are
working
on
the
integration
into
cryo
and
this
helps
us
because
we
are
already
having
a
feature
complete
implementation
right
now.
But
for
sure
this
has
to
be
battle
proof.
D
So
in
the
future
we
are
going
to
follow
up
to
make
it
ready
for
apartment
as
well
check
the
project
out
on
github
and
tell
us
what
you
think,
and
I
would
like
to
provide
you
a
little
demo
about
con
monae's
to
actually
use
gunman
is
you
would
have
duranik
modified
version
of
cryo,
which
already
implements
the
current
work
and
progress.
State
of
kanban
is
so
now.
D
Let's
do
that
we
start
cryo
and
it
starts
up
as
usual
as
expected,
and
then
we
can
run
via
cryctl
a
test
container
inside
of
a
test
sandbox,
and
if
we
do
that,
then
everything
looks
as
we
would
expect.
It
from
cryo,
so
we
run
a
new
container
and
got
the
container
id
and
if
we
run
rcglps
and
we
can
see
that
the
container
is
up
and
running,
but
now,
let's
double
check
that
by
looking
at
the
local
processes.
D
Which
then
gets
returned,
so
this
works
as
intended
right
now.
Our
overall
plan
is
to
integrate
con
mana
s
in
the
same
way
as
we
integrate
con,
monitor,
cryo
and
make
the
process
of
transitioning
from
kanban
to
commonwealth
as
easy
as
possible
by
making
everything
work
out
of
the
box.
Thanks
for
listening
to
this
demo,.
A
Thanks
sasha
now
we'll
go
over
some
optimizations
that
cryo
recently
made
to
its
handling
of
selinux
volume
relabeling
before
we
do
so
we'll
quickly
review
how
east
linux
works
in
kubernetes,
pods
or
containers
can
be
assigned
an
selinux
option
structure
in
the
structure.
A
user
can
specify
every
field
needed
to
construct
an
selinux
label,
the
user
role,
type
and
level.
There's
a
summary
of
what
each
of
these
fields
are
here.
A
The
fields
we're
most
concerned
with
is
level
as
the
other
fields,
don't
typically
change,
unless
a
user
specifies
otherwise
to
access
a
file
on
the
node.
The
level
field
is
a
way
of
subdividing
content
among
like
types
in
this
case.
It's
how
different
containers
cannot
access
files.
They
themselves
are
not
given
explicit
permissions
to
the
sc.
Linux
option
is
passed
from
the
api
to
the
cubelet
and
finally
to
cryo.
Cryo
then
takes
every
unfilled
field
and
populates
it
with
its
storage
library.
A
The
full
label
will
be
generated
and
the
mount
label
will
be
used
for
the
root
fs
and
volume.
While
the
process
label
will
be
used
for
the
container
process
itself
note,
the
mount
label
will
only
be
used
if
the
volume
plug-in
opts
into
it
and
hostpath
as
a
volume
plug-in
doesn't
opt
into
this.
So
you
don't
get
automatic
re-labeling.
A
A
The
security
profile
operator
or
sbo
is
useful
for
distributing
sc
linux
profiles
that
can
help
avoid
setting
your
containers
to
spct
if
they're
blocked
by
sc
linux.
At
this
point,
the
scada
state
of
kubernetes
and
iclinic
seems
great.
We
have
a
way
for
containers
to
access
content
they're
allowed
to,
as
well
as
a
way
of
disallowing
containers
from
accessing
content.
They
don't
own.
This
prevents
the
possibility
of
container
processes
breaking
out
of
the
root
from
affecting
anything
on
the
host
they're
not
allowed
to
touch.
A
However,
there's
also
a
problem
note
how,
before
the
user
or
container
storage
library
choose
a
new
level
for
each
pod
on
startup,
as
each
new
container
gets
created,
volumes
that
it
has
access
to
may
need
to
be
relabeled,
so
the
container
has
access
to
them,
since
this
label
is
owned
by
cryo,
as
it
sometimes
generates
it
and
is
responsible
for
creating
the
container's
root
ifes
and
mounting
the
host
volumes
into
it.
The
volume
plugin
instructs
crowd
to
relabel
the
volume
each
container
creation.
A
The
number
of
files
in
a
volume
is
not
restricted
in
any
way,
and
thus
relabeling
can
take
an
arbitrary
amount
of
time.
When
the
cubelet
asks
cryo
to
create
a
container,
it
must
cap
the
amount
of
time
cryo
has
and
cancel
the
request
after
that
time.
Otherwise
the
request
could
have
gone
to
the
void
and
the
container
creation
could
never
happen
when
cryo
can't
relabel
the
volume
in
time
and
the
cubit
times
out
it
causes
to
do
the
two
process
to
bicker
about
it.
A
A
Our
goals
to
fix
this
problem
are
as
follows:
a
container
must
still
be
able
to
have
a
process
label
that
can
access
the
volume
or
else.
What's
the
point
of
mounting
that
volume,
a
volume
must
be
labeled,
so
the
authorized
containers
can
access
it,
but
non-authorized
containers
cannot,
and
we
want
to
relabel
as
few
times
as
possible
to
ensure
container
creations
and
restarts
don't
time
out
if
it
can
be
avoided
satisfy
these
conditions.
We
came
up
with
two
solutions.
A
The
first
solution
is
to
conditionally
skip
the
relabel.
If
the
top
level
directory
is
almost
correct,
assuming
the
container
was
specifically
allowed
to
do
so.
We
use
crow's
allowed
annotations
functionality
to
ensure
an
admin
wanted
this
behavior
to
be
enabled
on
the
node
and
that
the
pod
author
actually
wanted
to
opt
into
it.
Some
advantages
to
the
solution
are
the
container
still
gets
labeled
properly
confining
it
in
the
case
it
isn't
totally
trusted.
It
can
be
enabled
on
a
per
pod
granularity,
so
only
containers
that
need
it
get
this
functionality.
A
A
A
The
label
does
have
to
happen
once
if
the
volume
isn't
relabeled
ahead
of
time.
The
first
container
in
the
pot
is
liable
to
time
out.
However,
subsequent
containers
and
restarts
will
succeed.
If
a
file
in
the
volume
was
relabeled
outside
of
this
process,
then
the
container
will
be
denied
permission.
A
A
Our
second
solution
is
to
skip
the
re-label.
If
the
container
is
sufficiently
privileged.
Remember
the
special
sc
linux
type
super
privileged
container.
We
leverage
the
fact
that
the
container
is
essentially
unconfined
to
avoid
re-labels
completely.
The
solution
is
faster
than
the
default
or
the
first,
as
no
re-labeling
is
needed
ever
simpler
because
it
doesn't
require
configuring,
cryo
or
adding
special
annotations
to
the
pod
and
portable.
Any
volume
can
be
mounted
into
any
privileged
pod
and
never
incur
the
cost
of
relabeling.
A
A
In
other
words,
the
crowd
team
has
presented
users
with
three
options
for
their
pods,
depending
on
how
secure
and
safe
they
want
their
pods
to
be
versus
how
quickly
they'd,
like
the
relabeling
done
the
first
option.
The
default
is
the
least
performant,
as
it
relabels
every
time,
but
is
the
most
secure,
as
the
volume
is
correctly
labeled
each
time
and
the
pot
is
fully
confined.
A
The
second
option.
Conditionally
skipping
is
slightly
more
performant,
as
it
is
more
performant,
as
it
skips
the
relabeling
sometimes,
but
is
slightly
dangerous
to
the
pod.
As
the
contents
may
change,
label
underneath
cryo
and
the
pod
will
be
denied
permission
if
they
try
to
access
it.
It
is,
however,
similarly
secure
as
the
first
option,
because
the
pod
is
confined
to
files
that
it
wants
to
access
the
final
option
skipping
if
privileged
is
the
most
performant,
because
the
label
never
happens.
However,
such
a
powerful
pod
must
also
be
trusted.
A
In
the
future,
with
cap1710,
we
hope
to
avoid
this
label
entirety.
The
cap
is
basically
the
way
the
cap
works.
Is
it
the
cubelet
instead
of
passing
down
the
relabel
option
to
cryo
upon
mounting
the
volume
will
pass
a
mount
flag
to
the
mount
command
that
will
do
the
relabeling
for
it.
So
none
of
this
overhead
will
be
needed
because
it'll
be
relabeled
upon
mount.
E
E
Secomp
allows
you
to
restrict
the
syscalls
that
your
container
processors
can
make
effectively
reducing
the
surface
area
of
attack
on
the
underlying
column,
container
runtimes.
Allow
you
to
install
and
pick
secomprofiles
that
specify
exactly
which
syscalls
are
allowed
container
runtimes
ship
with
pretty
good
default
profiles
that
allow
only
a
subset
of
syscalls
that
are
considered
safe.
E
I
get
involved
in
various
container
cvs,
and
quite
a
few
of
those
cvs
could
have
been
mitigated
if
seccomp
was
enabled,
unfortunately,
secomp
isn't
enabled
by
default
in
kubernetes.
Yet
by
default,
all
pods
are
unconfined
unless
they
are
configured
to
pick
the
runtime
default
or
a
particular
profile.
E
E
B
In
our
previous
cube
contacts,
we
introduced
the
concept
of
runtime
classes
that
can
be
set
in
the
cry
config
allowing
users
to
use
different
container
runtimes,
such
as
run
cc,
run
jivas,
etc
for
different
workloads
in
the
same
cluster,
we're
extending
runtime
classes
to
allow
users
to
use
multiple
storage
drivers.
Now
a
major
use
case
of
this
is
giving
vm
based
containers
the
ability
to
use
a
different
storage
driver,
such
as
device
mapper
instead
of
the
standard
overlay
fs.
Making
the
performance
of
vm
based
containers
require
much
better.
B
The
next
one
is
cosine
and
six
star
support
which
allows
signing
a
container
image
and
storing
that
signature
in
the
registry
that
can
be
used
for
image.
Verification
later
on,
a
lot
of
work
is
being
done
in
the
containers
image
backing
library
for
this,
and
once
that
is
ready,
we
will
be
adding
support
for
it
to
podman
and
cryo.
B
Additionally,
there
is
work
being
done
in
upstream
kubernetes
to
support
forensic
container
checkpointing.
This
will
allow
users
to
take
a
snapshot
of
a
running
container
that
can
then
be
transferred
around
to
another
node,
and
the
original
container
will
not
know
that
a
snapshot
of
it
has
been
taken.
B
The
restore
portion
of
this
will
be
implemented
in
the
container
engines,
so
once
cr
once
the
cri
api
has
been
extended
for
checkpointing
cryo
will
add
in
the
resource
support
as
well,
and,
finally,
crowd
can
now
export
open,
telemetry
trace
data
when
needed.
It
is
currently
an
experimental
phase,
as
you
wait
for
cubelet
to
get
support
for
it
as
well.
B
This
will
be
happening
in
version
1.25,
so
yeah
a
lot
of
exciting
stuff
coming
up
for
cryo,
and
these
are
all
the
updates
that
we
have
for
you
on
how
crying
continues
to
remain
secure,
performant
and
boring.
As
ever.
We
can
take
any
questions
now
or
you
can
also
find
us
in
the
cryo
channel
on
the
kubernetes
slack.
If
you
have
any
questions
later,
thank
you
for
attending
our
talk.