►
From YouTube: Container Checkpoint/Restore at Scale for Fast Pod Startup Time - Ritesh Naik, MathWorks
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Container Checkpoint/Restore at Scale for Fast Pod Startup Time - Ritesh Naik, MathWorks
Are you interested in a faster startup time for your container-based application? Are you also interested in pre-warming your container-based application for faster cold start performance? You can have your cake and eat it too! In this talk, Ritesh Naik will talk about how MathWorks cloud infrastructure was able to achieve *both* of these seemingly contradictory goals, fast startup time and pre-warming, using Container Checkpoint/Restore. With several years of experiencing using this technique in production, Ritesh will share design considerations, lessons learned and best practices for supporting container checkpoint/restore non natively in Kubernetes. He will also share a vision for what native support for CRIU in Kubernetes can look like. If you are interested in improving system scalability and utilization, this session is for you!
A
A
A
A
A
A
A
A
A
A
A
Now,
let's
do
the
same
exercise
again
and
stack
it
against
the
gold,
so
it
does
address
the
core
start
pane
setting
up
of
a
container
environment
and
initializing.
The
application
is
done
ahead
of
time,
but
it
still
suffers
from
handling
this
sudden
burst
in
the
workload
and
since
we
are
maintaining
these
pre-warmed
pods,
there
would
be
lot
of
waste
and
resources.
If
these
spots
are
sitting
idle
and
traffic
is
slow.
A
A
A
A
Basically,
gopos
is
simulating
an
app
doing
real
startup
and
pre-warming
that
takes
about
30
seconds
before
it
starts
listening
over
port
4000.,
and
you
can
see
how
a
checkpoint
restore
this
30.
Second
initialization
has
to
happen
just
once
during
the
checkpoint,
and
every
register
of
the
container
would
skip
the
initialization
and
be
ready
in
couple
of
seconds.
A
A
A
A
A
A
As
you
see,
it's
waiting
for
the
application
to
finish
initialization
and
it
would
take
around
30
seconds
to
initialize
while
we
are
waiting,
let's
start
pulling
port
4000.
This
should
get
the
response
back
once
they
go
up,
finishes
up
initialization
and
starts
this
thing
on
port
4000,
so
container
is
ready
if
we
get
the
string
back
as
the
response,
since
the
container
is
ready,
let's
check
point
using
run
c
checkpoint.
A
So
if
you
look
at
container
lifecycle
through
the
straight
transition
lens
with
checkpoint,
restore,
we
were
able
to
skip
the
initialization
and
jump
to
ready.
So
to
summarize,
checkpoint
restore
is
the
answer
to
fast
container
startup
type.
Wonderful,
isn't
it?
Let's
leverage
this
in
our
world
of
kubernetes,
unfortunately,
as
of
today,
there
is
no
need
for
support
of
checkpoint,
restoring
kubernetes.
A
A
A
Second
option
is
to
take
a
non-native
approach
without
touching
the
kubernetes
code
base,
and
in
this
talk,
I'll
walk
you
through
our
solution
to
non-native
approach,
so
checkpoint
restore
basically
has
different
use
cases.
It
could
be
used
to
address
cold
start
that
we
talked
about
or
migrate
a
container
from
one
host
to
the
other,
maybe
during
maintenance
among
many
other
use
cases,
given
the
different
use
cases
and
complexity
of
checkpoint
restore
in
kubernetes.
A
A
Restores
basic
functionality
is
to
restore
a
container
from
the
checkpoint
checkpoint,
accepts
container
image
and
configurations
required
to
run
a
container
waits
for
the
container
to
start
and
complete
its
initialization
and
checkpoints.
The
container
restore
uses
this
checkpoint
image
and
restores
the
container
to
the
state
at
which
the
container
was
checkpointed.
A
We
needed
a
way
to
introduce
container
runtime
in
kubernetes
that
supports
checkpoint,
restore
functionality
and
not
managed
by
kubernetes.
Hence
we
went
with
the
design
of
container
runtime
and
container
runtime,
as
the
name
suggests.
It's
running
a
container
runtime
inside
the
container
of
a
pod
spawned
by
container
runtime
that
is
managed
by
kubernetes
parent
container.
A
Let's
take
another
look
of
the
state
from
pre-checkpoint
to
post,
restore
in
a
pre-checkpoint
state
pod
running
checkpoint
service
is
spawned.
That
runs
the
inner
container
to
be
chuck
pointed
in
a
checkpoint
state.
Checkpoint
image
is
made
available
in
a
post
restore
state.
There
is
a
pod
running
restore
service
that
has
a
restored
container
running
inside
the
outer
container,
using
the
checkpoint
image.
A
Now,
let's
look
from
the
sequence
of
events.
Anchor
in
checkpoint
sequence,
checkpoint
service
is
deployed.
Checkpoint
service
starts
the
application
container
running
as
a
child
container
and
waits
for
application
to
become
ready
once
it
is
ready,
checkpoint.
The
container
and
save
the
checkpoint
in
the
location
that
could
be
accessed
by
the
restore
service
now
moving
on
to
the
restore
sequence,
restore
service
is
deployed.
A
A
A
A
A
This
is
the
config
map
where
properties
are
passed
through.
It
basically
has
a
probe
to
check
the
readiness
of
the
container
running
inside
the
parent
container.
Now,
let's
deploy
all
the
resources
on
the
right,
you
should
see.
There
are
three
parts.
First,
one
is
checkpoint
pod
that
is
starting
an
application,
basically
running
the
container,
with
the
go
web
app
and
waiting
for
it
to
initialize.
A
A
As
you
see,
all
the
ports
are
up
for
the
restore
port,
but
the
vanilla
pod
is
still
coming
up
and
it
is
initializing.
So
you
can
see
what
how
much
of
this?
How
much
of
a
difference
this
would
mean
make
if
you
are
scaling,
two
thousand
parts
or
ten
thousand
ports
and
every
pod
is
trying
to
initialize.
Whereas
with
restore
you
have
you
can
skip
that
part
and
ports
could
come
up
ready
in
couple
of
seconds
or
in
milliseconds.
In
fact,.
A
A
A
A
A
With
this,
we
wrap
up
the
non-native
solution
for
checkpoint
restore
in
kubernetes,
and
this
zero
to
checkpoint,
restoring
kubernetes
should
give
a
complete
picture
of
what
it
looks
like
for
native
support.
I
would
encourage
you
all
to
attend.
Adrian
rivers
talk
tomorrow
to
learn
more
on
the
vision
of
adding
checkpoint
rush
to
support
in
kubernetes
natively.
A
A
A
A
Let's
go
over
some
of
the
lessons
learned
using
checkpoint.
Restore
one
thing
to
note
is
that
checkpoint
restore
behavior
is
sensitive
and
could
have
unexpected
results.
If
there
are
additional
processes,
that's
added
or
removed
from
the
container
or
if
there
are
new
network
connections
at
the
time
of
checkpoint,
it
could
specially
become
challenging
if
it
is
an
application
not
owned
by
the
team
that
consumes
it
and
has
to
control
over
what
goes
into
and
out
of
the
containerized
application.
A
A
For
example,
there
was
a
change
in
the
way
checkpoint
location
could
be
specified
during
restore
and
docker,
and
though
there
was
no
change
in
the
lower
container
runtime
like
run
c,
it
broke
the
docker
workflow
since
we
were
so
tied
to
using
docker
runtime.
It
took
us
some
time
to
make
a
transition
to
using
different
runtime
or
different
version
of
the
docker.
A
A
Some
of
these
practices
listed
down
here
should
hold
good
for
any
cloud
native
application
and
holds
good
with
checkpoint,
restore
2.,
highly
recommend
enabling
locks
and
metrics
on
checkpoint
restore
these
locks
and
metrics
are
really
helpful
and,
at
the
same
time,
maintainers
are
well
accessible
to
discuss.
If
you
run
into
any
issues
I'll
make
this
logs
and
metrics
as
easily
as
accessible
as
possible.