►
From YouTube: Kubernetes SIG Node Intro and Deep Dive - Sergey Kanzhelev & Dawn Chen, Elana Hashman & Derek Carr
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Kubernetes SIG Node Intro and Deep Dive - Sergey Kanzhelev & Dawn Chen, Google; Elana Hashman & Derek Carr, Red Hat
These are exciting times for Kubernetes SIG Node. Come to our maintainers track session to learn about the just released version 1.24 of Kubernetes, full of exciting improvements and get a glance into the SIG Node roadmap. SIG Node owns components that control interactions between pods and host resources, including the Kubelet, Container Runtime Interface (CRI), and Node API. SIG Node is responsible for the Pod’s lifecycle from allocation to teardown, to liveness checks and shared resource management. We work with the various container runtimes, kernels, networking, storage, and more; anything a pod touches is SIG Node’s responsibility! We will discuss how Container Runtime logic was streamlined and simplified with dockershim removal, progress on cgroup v2 support, as well as many other improvements. Join this session to learn more about our SIG, and how you might get involved to make Node even better!
A
Hello,
everyone
welcome
to
2028
use
the
cubicle
europe.
Here
is
the
signal
the
intro
and
the
deep
dive
talk.
My
name
is
dan
chen.
I
am
a
software
engineer
from
google
and
I
currently
work
on
the
gk
and
source.
I'm
the
one
of
the
founding
engineer
for
kubernetes
and
the
initiate
signaled
back
in
2016.,
derek.
B
Yeah
great
to
be
here
with
everybody
at
kubecon
again,
my
name
is
derek
carr.
I
am
a
engineer
at
red
hat
working
on
our
openshift
product
and
have
been
with
dawn
and
the
sigs
since
our
early
days
as
well-
and
I'm
excited
to
be
here
with
some
of
our
newer
contributors,
alana
and
sergey
solana.
You
want
to
introduce
yourself.
C
A
Next,
so
before
we
get
into
the
today's
agenda,
I
want
to
briefly
mention
the
previous
signal.
The
update
we
made
either
cube
account
last
year.
You
can
access
related,
slides
record
by
click
all
those
links
here.
Next,
here's
the
today's
agenda.
We're
going
to
first
introduce
the
signals
responsibility.
A
Then
we
are
going
to
talk
about
the
current
activity
since
last
updates
the
roadmaps
from
1
1.23
to
the
to
the
125
and
the
br.
Then
some
interesting
projects
and
efforts
are
currently
driving
by
the
signal,
for
example,
one
that
the
24
docker
share,
removal,
sequel
provision
2
and
the
cia
projects
continue
from
the
last
update.
A
Then
we
want
to
share
the
node
contributor
later
with
the
community.
We
discussed
this
within
the
signal
community
for
a
while.
Now
we
have
feminized
yet.
Finally,
we
are
going
to
talk
about
how
to
get
involved
and
how
to
get
help
from
the
community.
Next,
what
is
the
signal,
and
it
is
responsibility?
A
Let's
briefly
talk
about
the
node
responsibility
in
kubernetes.
Kubernetes
is
a
cluster
ultraspecial
solution
for
container
like
containerized
applications
and
services.
Those
containers
included
of
the
kubernetes
controllers
are
running
on
the
nodes.
On
each
node,
there
is
an
aging
co-current
kubernetes
registered
node
to
kubernetes
master
kubernetes
together
with
container
runtime,
manages
part
and
containers
lifecycle
of
the
node
setup,
run
tier
down
and
clean
up.
A
Kubernetes
also
does
the
node-level
resource
management
such
as
ensure
the
application,
get
the
request,
the
resources
detect
the
node-level
resource
establishing
issues
and
take
the
text
exit
to
prevent
out
of
resource
situation.
Kubernetes
also
sends
the
status
back
to
the
control
plan
to
make
the
follow
decision
correction.
A
Next,
please,
in
summary,
signaled
on
all
control
are
controllers
running
on
the
nodes
which
ensure
the
node
itself
and
application
running
happily.
Signaled
is
very
large
and
owns
many
projects.
You
can
click
the
links
and
here
to
find
more,
I
this
time
I
want
to
specially
call
out
the
new
sub
project.
We
start
linking
special
resource
operator
which
helps
the
user,
manage
the
deployment
of
the
kernel
modules
and
the
drivers
and
come
to
join
our
signal.
The
weakness
meeting
to
know
all
those
projects
in
details.
C
So
I
grouped
this
section
together
because
there's
sort
of
this
overall
goal
between
graduations
and
deprecations
of
cleaning
up
tech,
debt,
reducing
maintenance
service.
So
when
a
feature
graduates
in
kubernetes,
it
was
previously
behind
a
feature
flag
typically
defaulted
to
on
and
when
we
graduate
it,
we
get
rid
of
that
feature
flag.
It's
no
longer
conditionally
on
it's
on
for
everyone
by
default.
C
C
On
the
graduation
side,
we
are
also
graduating
c
group
v2
or
the
version
two
of
kernel
control
groups
to
feature
parity
in
the
122
release
and
then
a
couple
more
minor,
deprecations
and
graduations
we've
removed
support
for
dynamic,
cubelet
config.
That
was
previously
deprecated
and
ultimately
removed
in
124
and
we've
also
graduated
pod
overhead,
which
helps
keep
track
of
additional
resources
at
the
pod
level,
so
no
more
feature
paid
for
that
for
beta
graduations.
C
This
is
when
a
feature
may
have
been
added
at
an
alpha
level,
and
now
we
default
that
feature
gate
to
on.
So
in
the
past,
a
couple
of
releases
123
and
124-
we've
graduated
a
number
of
features
to
beta,
including
ephemeral
containers
which
had
quite
a
long
time
between
alpha
and
beta.
So
we're
very
excited
about
that,
but
we
also
graduated
the
cubelet
cri
support.
We
now
support
a
v1
api,
which
is
very
exciting.
C
We
have
graduated
cubelet
credential
provider
to
beta
and
we've
added
support
for,
and
then
graduated
grpc
probes
to
beta
and
so
on.
So
if
you
want
to
look
up
any
of
these
particular
features,
you
can
do
so
by
enhancement
number,
which
I've
included
on
the
slide
and
in
the
copy
of
the
slides
that
we're
sharing
will
also
include
some
links.
C
Finally,
I'll
talk
about
the
alpha
features
that
we've
added
to
the
sig
over
the
past
couple
of
releases.
These
are
net
new
features,
so
they're
introduced
behind
a
feature
gate,
but
those
feature
gates
are
disabled
by
default
and
you
could
turn
them
on
and
test
them
out.
We
have
a
couple
of
these
alpha
features
that
we
have
added.
The
first
is
c
advisor
lists
and
cri
full
hot
container
and
pod
stats,
allowing
the
cri
the
container
runtime
to
provide
statistics.
C
C
A
Thanks
anella
before
getting
into
docker
shim
removal,
I
want
to
briefly
talk
about
kubernetes
container
runtime
interface
ci.
It
is
a
grpc
interface
which
defines
how
kubernetes
the
agent
running
under
every
node
interacts
with
a
wide
variety
of
the
container
render
we
published
the
first
version
of
the
cri
back
in
may
2016
and
the
first
implementation
of
the
cia
was
introduced
to
kubernetes
1.5
in
2017..
A
Next,
please,
as
the
first
implementation
to
support
the
cri
docker
shim
is
the
built-in
module
in
kubernetes.
Why
did
we
choose
to
do
this
then?
So,
then
we
have
to
deprecate
and
remove
it.
Today
in
2016,
docker
was
the
only
production
id
container
rental
we
needed
to
validate
the
interface
we
created
quickly
to
ensure
the
incremental
deployment
it
take.
A
It
took
us
a
little
bit
more
than
one
year
to
have
the
first
version
of
the
production
right,
the
interface
and
the
implementation,
then
why
we
bundle
it
with
kubernetes
as
the
single
battery
back
then
kubernetes
iterate
very
dramatically,
cri
was
changed.
All
the
time
bundle
found
only
it
was
kubernetes
so
that
we
can
tolerate
breaking
api
changes
for
the
faster
iterations.
A
This
also
allows
us
we
can
switch
to
using
ci
as
the
default
for
the
in
pro
in
process
docker
integration,
while
the
api
is
still
in
alpha.
Hence
we
created
of
the
docker
ship,
and
that
is
that
was
the
only
choice
for
kubernetes
users.
Until
two
years
later,
we
introduced
g8
the
second
container
runtime
container
d.
Then
cryo
follow
next.
Please,
since
we
published
the
first
version
of
the
cii,
there
were
many
efforts
to
build
different
container
runtimes
to
serve
different
purposes.
Different
work
notes,
different
business
reasons
and
so
on.
A
Signaled
supported
all
of
them,
but
worked
very
closely
with
continuity,
cryo
and
the
frac
friday,
in
addition
to
docker
shim.
To
make
sure
the
interface
we
defined
covers
the
majority
use
cases
we
also
introduced
ci
test
suite
and
the
cloud
cattle
tool
sets
to
help
the
develop
development
and
the
usability.
A
Next,
since
we
published
the
first
version
of
the
cloud.
Oh
sorry,
since
we
published
the
first
version
of
the
cri
in
2016
two
years
later,
kubernetes
container
d
went
to
ga
and
graduated
from
the
cf
incubation
later
cryos
production
ready
and
graduate
both
container
runtimes
support.
The
cri
and
oci
compatibility
since
day,
one
both
the
container
run
times
or
future
parity
with
the
docker
shield
before
ga
and
over
the
time
we
introduced
more
features
to
them
respectively.
A
Both
container
runtimes
met
the
test
coverage
requirements
defined
by
signaled
before
the
production
writing
next.
Why
do
we
want
to
remove
docker
share?
Why
not
maintain
an
inbox
solution
for
the
users
from
the
previous
slides?
One
can
see
that
since
the
beginning
of
its
knife,
we
treat
it,
we
treated
it
as
the
temporary
solution.
We
fixed
many
integration
issues
using
it
to
support
existing
users
but
decided
not
to
introduce
new
features,
especially
after
we
had
the
two
alternatives
three
years
ago.
A
B
Thanks
tom,
so
as
discussed
earlier,
we've
made
a
lot
of
progress
in
the
community
around
secret
speed,
v2
enablement.
As
we
started
this
project,
we
wanted
to
make
clear
on
what
our
initial
goals
were
with
respect
to
secret
v2,
and
that
was
largely
to
get
parity
with
existing
feature,
support
in
c
groups
v1.
So
all
of
the
resource
controllers
that
kubernetes
leverages
to
restrict
the
amount
of
cpu
or
memory
or
pids
or
huge
pages
that
a
given
pod
can
consume.
They
are
restricted
today,
primarily
by
secret
v1
controllers.
B
B
Today
we
are
not
deprecating
secret
b1
support
by
adding
v2,
but
we
are
making
a
statement
that
a
new
resource
controllers
are
intended
to
be
only
added
for
for
v2
only
and
we'll
talk
about
some
of
those,
so
one
of
the
major
activities
to
enable
v2
support.
Besides,
the
code
is
just
to
make
sure
that
you
have
viable
test
coverage.
B
A
number
of
distributions
over
the
last
few
years
have
started
to
change
their
default
boot
time
configuration
to
be
v2
enabled
so
that
includes
fedora,
ubuntu,
cos
and
others,
and
now
that
that's
happened,
we
get
more
pressure,
obviously
in
the
kubernetes
community,
to
support
those
hosts.
One
example
of
new
features
that
we
are
excited
that
you
can
leverage
in
a
v2
environment
that
is
in
an
alpha
state.
Right
now
is
memory
quality
of
service.
B
B
D
B
Kubernetes
infrastructure
provider,
it
does
have
some
meaning
right.
You
have
to
know
if
your
operating
system
is
configured
for
v2,
you
have
to
make
sure
that
your
container
runtime
supports
av2
environment
and
typically
that
does
mean
now
you'd
have
to
change
how
you
deploy
a
secret
manager
in
both
cable
and
runtime.
To
be
systemd,
aware
in
practice,
though,
while
it's
true
that
most
workloads
shouldn't
be
impacted,
it's
hard
for
us
to
immediately
know
these
things
without
testing
and
feedback.
B
B
So,
while
projects
like
c
advisor
have
added
v2
support,
your
own
unique
environment
might
have
some
unique
agents
running
on
your
environment
that
may
not
yet
be
v2
compatible.
So
just
be
aware
that
the
intersection
of
your
total
set
of
solutions
deployed
to
a
node
might
need
to
account
for
v2
in
the
future.
B
B
B
So
if
you're,
using
a
particular
language
runtime
in
golang,
there's
native
v1
support
to
determine
how
to
properly
configure
gomax
prox,
but
right
now,
my
understanding
is,
I
think,
on
golang
you
actually
have
to
manually
set
that
on
a
v2
host,
I'm
sure
over
time,
as
golem
evolves.
You
won't
have
to
care
about
anymore,
but
sometimes
these
quirks
exist.
Similarly,.
B
B
Right
now,
we
are
working
towards
landing
that
in
an
alpha
feature
state,
but
we
would
love
help
to
be
able
to
ensure
that,
as
that
feature
lands
that
it
is
v2
tolerant
right
now,
there's
gaps
that
we
need
to
close,
in
particular
around
auto
scaling,
a
lot
of
things
between
v1
and
v2.
The
metrics
do
change,
and
so,
if
you
have
exotic
auto
scalers,
we
really
just
need
more
testing
against
those
metric
providers
that
are
feeding
your
autoscaling.
D
B
That
feedback
brought
back
to
the
community
so
like
a
general
call
to
action
if
you're
watching
this
is
if
your
domain,
your
industry,
your
language,
runtime
or
your
vendor
has
any
known
gaps
that
they
want
to
share
with
us.
The
only
way
we
can
get
them
mitigated
is
to
to
share
that
feedback.
So
please
do
look
to
join
us
going
forward,
though,
with
secret32.
There
are
interesting
features
that
we
look
to
explore.
B
That
are,
if
you
are
interest
to
you
as
a
potential
user
or
contributor,
we'd
love
to
learn
more
about
so
things
like
pressure
stall,
information,
metrics
to
drive
more
efficient
eviction
or
node.
Reliability
is
an
exciting
feature
for
us,
as
well
as
better
leveraging
user
space
zoom
killers
like
md.
So
hopefully
we
see
that,
as
we
evolve
to
v2,
we
can
get
better
more
reliable
nodes
for
the
broader
community.
B
D
D
D
D
So
there
is
this
connection
and
then
kublet
also
talks
to
continue
runtimes
through
workers,
and
it's
like
a
schedule.
Sport
and
kills
them
when
needed,
plus
it
monitors
port
status,
periodically
querying
container
on
time,
information
and
this
information
about
pods
being
generated
as
events
and
plague
and
this
event's
been
fed
back
into
kublad.
So
it
approaches
all
that
from
all
the
signals
from
all
the
different
components
to
build
up
a
final
picture.
D
What
is
currently
happening
with
spots
and
containers
in
the
spots
in
122,
let
me
say,
site
in
122,
we
fixed
the
bug
that
bug
was
related
to
reporting
the
terminated
status
of
certain
containers.
There
was
some
race
conditions
and
to
eliminate
these
conditions.
We
put
more
logic
together
as
a
single
source
of
truth
of
port
termination
status.
D
As
I
said
last
time,
there
were
many
race
conditions
and
the
other
bugs
we
discovered
as
a
result
of
this
refactoring.
It
was
a
good
refactoring,
it
fixed
a
lot
of
bugs
but
and
then
regression
was
caught
and
we
shipped
quite
a
stable
product.
Unfortunately,
we
found
another
problem
in
cobalt
in
122.,
so
this
was
represented
as
out
of
cpu
message.
When
you
try
to
schedule
many
ports
simultaneously
on
quite
busy
notes.
D
D
D
While
api
server
doesn't
know
about
it,
and
then
api
server
will
push
some
nodes.
Some
ports
into
the
node
and
node
will
not
have
time.
We
will
not
have
cpu,
because
static
port
just
got
scheduled
and
api
was
just
wasn't
aware
of
that
there
were.
Other
cases
are
caused
out
of
cpu
before
so
we
investigated,
we
thought
that
it
may
be
some
scheduler
problem.
D
D
Oh,
I
didn't
explain
what
the
root
cause
so
root
cause
was
that
when
we
fixed
the
pod
termination
detection-
and
we
put
it
in
a
single
place-
we
made
kublet
know
about
pod
being
terminated
earlier
than
before,
and
since
kublet
knows
about
it,
it
will
sync
the
status
with
api
server
and
api
server
will
know
that
paul
is
terminated.
So
it's
time
to
schedule
newport
in
place
of
this
one,
what
the
fix
was
to
delay
a
report
in
the
status
to
api
server.
D
I
highlighted
this
like
key
codes
that
participate
in
that
fix,
but
they
were
pr
was
much
bigger
than
that.
So
now
we
will
delay
reporting,
terminated
status
of
report
to
api
server
before
all
the
resources
are
cleaned
up
and
all
the
resources.
I
mean
not
all
all
resources,
it's
only
cpu
and
memory.
In
this
case,
we
made
some
observations
while
we've
been
fixing
this
bugs
beyond
just
the
distributed
systems,
a
heart
and
need
to
find
balance
between
speed
and
consistency.
D
We
also
highlighted
again
that
terminated
port
is
not
equal
deleted
port.
A
terminated
port
can
report
its
termination,
but
it
still
will
clean
up
some
resources,
and
one
resource
in
particular
that
will
be
clean,
will
be
turned
down
during
the
termination
period
is
volumes
and
volumes
may
take
a
lot
of
time
to
to
be
turned
down
and
cool
blood
account
for
that.
So
a
new
port
can
be
scheduled
that
want
to
use
the
same
volume
and
it
will
wait.
D
Till
volume
will
be
completely
cleaned
up
and
then
we
will
pick
it
and
pick
up
a
new
volume,
so
this
behavior
is
different
from
what
we
made
for
cpu
and
memory
and
in
the
future
we
may
consider
improving
this
for
cpu
and
memory
as
well.
So
scheduling
will
happen
faster
and
kublat
will
just
wait
a
little
bit
for
cpu
memory
to
finally
freed
up,
but
it
may
be
a
future
improvement.
D
It's
all
highlighted
again
that
we
need
to
have
good
ci
for
kubernetes
and
make
sure
that
we
cover
all
the
bases
when
we
test
it.
So
we
added
end
to
end
test
for
this
regression
that
will
schedule
many
ports
and
we'll
make
sure
that
this
signal
wouldn't
be
a
terminated
signal.
Wouldn't
be
synced
up
with
apis
or
too
early
in
general
ci
sub
project
meet
every
week
and
what
we
do
is
we
look
at
all
the
test
grid.
We
analyze
what's
happening,
what's
going
badly,
what
is
going
good?
D
We
fix
issues
and
adjust
tests
and
we
also
create
a
new
test
coverage.
So,
if
you're
interested
to
join
signature-
and
you
don't
know
where
to
start,
the
cisa
project
may
be
a
very
good
place
to
start,
we
you'll
learn
kubernetes
through
facing
kubernetes,
and
it
will
give
you
a
lot
of
inside
knowledge
like
how
to
test
and
what
kind
of
situation
kubelet
may
get
into.
D
We
recently
published
achievements,
blog
post,
I
linked
it
here.
You
can
go
and
check
it
out.
We
have
18
active
contributors
that
we
highlighted.
We
really
appreciate
all
of
your
work.
We
want
to
highlight
it
more
and
more,
and
we
don't
have
time
in
this
talk
to
name
names
but
yeah
hi
everybody.
Thank
you
for
your
contribution.
D
I
also
posted
some
statistics
here.
It's
not,
I
mean
it's
from
some
time
ago
and
it's
kind
of
when
you
symmetric
it
just,
but
what
it
shows
it
shows
numbers
like.
We
do
a
lot
of
work.
We
do
reviews
we
do
bug
fixes
so,
if
you're
interested
join
us
and
help
us
in
this
hard
task,
we
also
like
this
is
how
we
track
work
in
the
cisa
project.
We
have
a
special
project
for
test
and
we
do
receive
early
reliability
signal
by
doing
bug.
Cache
with
that.
B
Yeah
thanks
so
guys,
so
one
of
the
challenges
in
a
open
source
project
at
the
scale
of
kubernetes
and
cigna
itself
is
just
the
breadth
of
things
that
we
need
to
keep
in
mind
as
maintainers,
that
we
don't
break
or
regress
the
past,
and
we
can
continue
to
build
responsibly
for
the
future.
So
signoid
is
the
third
largest
sig
by
absolute
workload.
B
Lots
of
pr's
get
opened,
lots
of
prs
need
to
get
merged
closed
and
triage,
and
contributions
can
come
from
a
variety
of
member
companies
who
work
in
the
community
sometimes
enduring,
and
sometimes
just
in
particular,
feature
function
areas.
What
this
means
is
in
general
for
us
as
a
project
to
be
healthy.
B
B
C
C
Yeah,
let's
do
it
so
we
often
talk
about
our
contribution
priorities
and
this
sort
of
draws
back
to
a
few
of
the
things
I
said
about
road
map.
We
try
to
prioritize
stability.
First,
we
want
to
ensure
that
text
tests
are
fixed.
We
want
to
ensure
that
bugs
are
fixed
and
open.
Triage
issues
are
fixed
and
we
want
to
do
that
before.
We
start
working
on
new
features,
because
otherwise
we
might
get
overwhelmed
in
the
number
of
bugs
and
test
breakage
that
we're
working
on.
C
We
really
it's
important
to
us
to
ensure
that
our
test
infrastructure,
monitoring
and
health
is
good,
and
you
know
our
tests
are
green.
We
have
good
signal
to
work
with,
so
this
is
sort
of
how
we
prioritize
contributions
and
if
you
really
want
your
new
contributions
to
get
looked
at,
you
are
more
likely
to
get
a
positive
response
if
you
are
fixing
something
that's
broken
initially
versus
like.
C
Other
areas
that
you
might
be
interested
in
contributing-
and
I
often
try
to
call
this
out
especially
to
folks
who
might
be
using
the
cubelet-
have
opinions
on
sig
node,
but
aren't
necessarily
going
to
write
code.
You
can
still
contribute.
You
can
still
help
us
out
either
by
doing
performance,
testing
or
writing
documentation
or
giving
us
feedback
on
blogging
and
metrics
and
how
we
can
improve
that
or
just
helping
us
triage
and
keep
on
top
of
pr's
and
issues.
C
C
We
talk
about
major
features
that
are
being
worked
on
enhancements
and
what's
going
into
a
given
release
and
milestone
high
priority
bugs
and
more,
if
that
seems
kind
of
intimidating
to
join
when
there
are
40
plus
people
in
a
meeting,
you
could
try
coming
to
our
ci
and
triage
sessions
where
we
have
a
smaller,
more
hands-on
group
and
we
work
directly
on
specific
bug,
specific
issues.
That
kind
of
thing,
as
I
said
previously,
you
can
also
participate
in
code
reviews
issues
documentation.
C
This
is
all
awesome,
a
big
shout
out
to
all
of
the
folks
who
have
been
helping
out
with
the
docker
shim
removal
documentation
this
release.
That
was
an
enormous
effort
and
a
great
help
to
sig
node,
and
it's
also
really
great.
If
you
adopt
features,
turn
on
alpha
feature
gates
and
let
us
know
how
they
work.
C
So
where
can
you
find
us
this
slide
links
to
our
regular
meeting,
which
is
currently
scheduled
on
tuesdays
at
10
a.m?
Pacific
time
our
ci
triage
meeting
is
every
week
on
wednesday
at
the
same
time,
10
a.m.
Pacific.
We
have
a
slack
channel
pound,
sig
node
on
the
kubernetes
slack.
We
also
have
a
mailing
list:
the
kubernetes
sig
node
mailing
list
on
google
groups
and
our
current
chairs
are
don
and
derek
thanks.
So
much
for
attending
this
talk.