►
From YouTube: Simplifying Windows runtime and deployment in Kubernetes - Muzz Imam, Microsoft & Michael Michael
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Simplifying Windows runtime and deployment in Kubernetes - Muzz Imam, Microsoft & Michael Michael, VMware
The leaders of SIG-Windows will provide an update on the efforts to bring Windows to Kubernetes. This session will concentrate on presenting new features and capabilities as well as focus on advanced capabilities like Kubeadm support, ContainerD integration, and Cluster API for Windows. At the end, we will open the floor for Q&A with customers and members of the SIG-Windows community. Some familiarity with Windows on Kubernetes is required for the deep dive part since we will have an in-depth discussion on key features that are in the pipeline for Windows, explain their implementation and have a discussion on trade-offs with the community.
https://sched.co/ZeyK
A
Hello,
everyone
and
welcome
to
simplifying
windows
runtime
deployment
in
kubernetes.
My
name
is
maz
imam.
I
have
I
work
at
microsoft
as
a
senior
program
manager
on
the
windows
containers
team.
I
have
been
a
sig
windows
contributor
for
some
time
and
with
me
is
michael
michael:
do
you
want
to
go
ahead
and
introduce
yourself.
B
B
B
Let's
take
a
look
at
first,
our
history
and
how
windows
started.
Our
first
release
was
1.5.
It
was
our
alpha
release
where,
essentially,
all
we
had
was
a
cubelet
and
the
q
proxy
running
on
on
windows
and
kind
of
showed
everybody
the
art
of
the
possible.
There
were
a
lot
of
limitations
here,
but
it
kind
of
encouraged
the
community
to
come
in
and
contribute
on
windows
in
kubernetes
with
1.9.
B
We
had
our
beta
release
where
we
had
significant
advancements,
including
some
initial
cni
support,
1.14
that
shipped
a
little
bit
over
a
year
and
four
months
ago
we
had
our
first
stable
release.
That
means
that
we
have
supporting
windows,
containers
on
windows,
server,
2019
nodes
for
production,
workloads
on
kubernetes,
amazing
stuff,
like
the
community
kind
of
banded
together,
and
got
that
release
out.
B
Since
then,
we've
had
four
releases
of
kubernetes,
starting
with
115
116
and
117,
where,
as
a
community
who
made
advancements
in
cuba,
dm
csi
run
as
username
volume
support
and
also
added
support
for
additional
operating
systems
like
windows,
server
1903
with
118.
That
was
the
release
that
came
out
about
four
months
ago.
We
graduated
to
stable
support
gmsa,
which
stands
for
group,
managed
service
accounts
and
run
his
username
to
enable
our
users
to
be
able
to
take
advantage
of
active
directory
identities
for
windows
workloads
in
kubernetes.
B
B
118
was
pretty
much
our
biggest
feature
rich
release
since
141.14,
when
we
actually
went
ga
now
with
119
we're
adding
csi
proxy
work
to
beta
container
d
is
going
to
beta,
so
you
can
see
we're
having
this
gradual
improvement
and
gradual
moving
of
our
support
to
an
offering
that
gets
better
and
better,
with
every
release,
we're
adding
support
for
dsr
and
endpoint
slices,
so
lots
of
networking
improvements
and
additional
stability
and
performance
improvements.
So
when
you
kind
of
think
about
this,
our
community
is
vibrant.
B
We're
executing
a
ton
of
changes
across
the
board
from
storage
to
networking
to
compute
and
we're
advancing
the
vision
of
windows
containers
in
kubernetes
with
every
release
in
the
next
slide.
We're
going
to
have
maz.
Take
you
through
a
deep
dive
of
container
d
and
some
of
the
work
we're
doing
there.
A
A
The
other
thing
that
that
you
know
is
a
big
benefit
of
container
d
was
from
a
networking
perspective.
We
were
able
to
enable
new,
cni
plugins,
and
the
reason
we
were
able
to
do
that
is
container
d
is
able
to
leverage
fully
the
cni,
the
the
guidance
and
it
adheres
to
it,
which
includes
creating
a
new
network
name
space
before
invoking
any
plugin.
For
example-
or
you
know
you
know
the
runtime
must
add,
you
know,
you
know,
must
not
add
twice
before
deleting
it.
A
A
Some
of
the
new
features
that
we
already
are
supporting
today
and
we
were
able
to
because
of
the
container
d,
is
especially
I'm
going
to
go
into
the
v2
schema
in
a
bit.
But
basically
the
features
that
are
today
available
in
container
d
and
not
in
docker
are
single
file.
Mapping,
for
example,
graceful
shutdown
of
pods.
We
talked
about
the
improved
workflow
of
cni
and
the
true
name
spaces,
so
the
other
benefit
and
a
lot
of
benefit
container
d
is
getting
is
because
it
it's
based
on
the
hcs
v2
schema.
A
A
It
accepts
the
json
files,
so
initially
we
came
up
with
the
schema
and,
as
we
got
feedback,
especially
from
kubernetes,
we
had
we
improved
that
schema
and
that
became
v2
schema,
in
particular
the
support
of
of
pods
and
the
hyper-v
that
that
was
very
particular
feature.
That's
added
to
v2
schema,
but
it
has
a
it
has
a
bunch
of
benefits
and
container
d
is
based
on
v2
schema.
That's
why
we
are
able
to
to
provide
these
new
features
now.
A
A
The
reason
it's
important
to
understand
the
runtime
architecture
and
specifically
container
d
architecture,
because
you
know
the
community
is
moving.
That
way,
as
michael
just
said.
Is
that
number
one?
If
you
are
a
kubernetes
developer
or
a
user
using
windows
containers
on
kubernetes,
you
probably
want
to
know
the
whole
stack,
especially
the
runtime,
the
different
pieces
and
components
how
they
fit
together,
how
they
flow.
A
So
if
you
found
you
know,
if
you
are
having
an
issue,
you
know
which
component
you
know
where
it's
coming
from
number
two
is
when
you
know
where
it's
coming
from,
you
can
go
and
get
help
against
it
and
we're
gonna
go
into
into
that
and
how
you
can
get
support
if
you're
running
into
issues.
So
let's
look
at
the
docker
runtime.
First,
the
majority
of
the
windows
containers
on
kubernetes
implementations
today
are
using
docker
runtime,
so
in
the
docker
runtime.
A
If
you
look
from
the
left
side,
you
have
cubelet
within
the
kubernetes
and
then
you
have
docker
shim
or
integrated
into
the
cubelet.
This
docker
shim
is
basically
a
shim
layer
for
docker
that
helps
cube.
Let
talk
to
docker
and
it's
it's
basically
a
cry
interface
for
docker,
but
it
is
within
the
cubelet.
A
Now
the
the
way
the
communication
works
is
the
cubelet
is
going
to
call
through
docker
shim
docker
runtime,
that's
in
the
middle,
the
middle
box
and
then
the
docker
runtime
is
going
to
call
the
hcs
and
the
hcs
is
going
to
then
create
containers.
The
hts
is
host
compute
service.
We
just
talked
about
the
one
thing
to
note
here
is:
although
the
v2
schema,
we
developed
very
closely
with
the
docker
mobi
project
and
container
d,
both
of
them
docker
used
v2
and
experimental,
but
they
are
still
using
v1
schema.
A
A
Now,
if
we
move
to
the
container
d,
how
that's
different
so
right
off
the
bat,
if
you
look
at
the
left
side,
the
cubelet
no
longer
has
that
container.
Runtime
shim,
so
we
decoupled
it
and
the
cry
is
now
part
of
the
container
d,
the
container
runtime
itself.
So
that
helps
you
know
with
with
you
know
faster
development,
and
you
know
all
the
benefits
of
decoupling.
A
So
now
the
cubelet
is
going
to
call
container
d
using
the
cry
api
cryo
api
is
developed
for
mainly
for
kubernetes,
and
it's
it's
enabled
by
default.
It's
optional
to
use
cry
api,
but
it's
enabled
by
default
so
and
we're
gonna
go
into
more
detail
on
how
this
works,
but
basically
it
goes
to
cry
and
then
the
cry
is
gonna
use,
container
d
to
call
hcs
chimp.
A
So
that's
another
improvement
in
continuity
overall
flow,
so
hcs
shim
is
basically
an
open
source
repo
published
by
microsoft,
which
is
a
shim
around
hcs
and
and
it
has
a
lot
of
runtime
features.
Basically,
it's
a
golang
interface
for
hds
and
has
a
lot
of
helper
functions
for
hns
as
well.
So,
and
it
is,
you
know
it
can
be
used
to
do
a
lot
of
runtime
functions.
So
another
thing
in
container
d
is:
we
are
able
to
fully
leverage
the
the
features
of
hcs
shim.
A
So
that's
another
decoupling
right
there
and
then
hcsgm
is
going
to
call
to
hcs.
Now
hcs
is
using
v2
schema
and
then
you
know
html
is
going
to
create
the
containers.
So
that's
the
overall
flow
and
the
difference
between
the
runtime.
Now,
let's
go
deeper
into
the
container
d.
So
if
I
go
to
the
next
slide,
if
so,
let's
just
start
with
the
process-
isolated
containers,
the
you
know,
which
is
supported
today,
the
in
windows
containers.
A
Sorry,
the
only
container
that's
supported
in
kubernetes
today
is
the
process.
Isolated
windows
containers
the.
So
if
you
look
at
this,
this
diagram
think
of
it
as
as
a
vm
or
a
physical
host,
basically
running
windows,
server
on
the
bottom.
You
have
the
windows
kernel
and
you
have
this
red
boundary,
which
we
call
the
virtualization
isolation
boundary.
A
It's
also
the
security
boundary
I'm
going
to
talk
in
a
bit
about
you
know
the
security
aspect
of
it
as
well,
but
let's
look
at
the
big
picture
right,
so
the
gray
dotted
lines.
If
you
can
see
the
gray
box,
there
are
two
of
them
on
the
left
side
is
your
route
or
the
host
within
microsoft
we
call
it
root
silo
on
the
right
side.
You
have
the
the
container.
It's
a
fancy
way
of
saying
is
just
a
process
isolated
container.
A
So
those
are
the
two
boxes
right
here
now
when
you
know
so
the
how
the
flow
works
is,
as
you
see,
on
the
top
cubelet,
usually
or
cry
ctl,
if
you're
a
developer,
and
you
want
to
debug
kubernetes
without
installing
all
the
components.
That's
when
you
use
chrystl
both
of
them
are
gonna
call
the
cry
api
in
container
d,
we
just
talked
about
it.
Now.
The
major
components
within
container
d
are
container
d
api,
which
is
very
powerful,
but
for
kubernetes
we
use
prior
api.
A
Now
the
components
that
container
d
relies
on
are
snapshot
the
networking
and
tasks.
Now,
if
you
look
from
container
d,
when
we
are
gonna
talk
to
the
operating
system
right
to
the
windows,
there
are
three
distinct
paths
going
down
on
the
very
left
side
you
have
snapshots
talking
to
windows
graph
driver.
A
Basically,
snapshots
is
used
to
create
images
to
to
unpack
a
different
layer
of
images
or
pull
images
so
think
of
this
windows
graph
driver
as
the
windows
layer
management,
so
that
piece
and
it's
not
shot
directly
talks
to
it.
Now,
in
the
middle,
you
have
networking
which
create
which
provides
api
to
talk
to
cni
plug-in,
and
we
just
talked
about
them.
You
know
the
newer
plugins
and
the
improved
plugins.
We
were
able
to
leverage
in
continuity
that
in
turns
talk
to
host
network
services.
A
This
middle
path
basically
makes
sure
that
hns,
you
know,
creates
all
the
plumbing
for
networking
behind
the
scenes.
For
you
for
your
container
and
on
the
very
right
side,
you
have.
The
hts
champ
also
called
the
container
dsm
where
we
just
talked
about
hcshim,
so
the
cry
api
is
going
to
go
through
the
con,
the
hcs
shim
to
call
into
hcs.
A
Now,
once
it
gets
to
hcs
hcs
in
process,
isolated
containers
is
the
one.
That's
gonna,
create
this
silo
or
the
container
think
of
the
container
in
windows
as
an
advanced
job
object.
Basically,
so
it
creates
that
you
know
that
gray
box
on
the
right,
the
gray,
dotted
lines
and
once
it's
up
the
the
service,
the
main
interface
between
your
application,
running
on
the
container
and
to
the
operating
system
is
see
exact
service.
A
So
that's
the
service
that
talks
to
hcs
and
if
your
container
application
needs
to
talk
to
windows,
kernel
or
leverage,
some
of
the
you
know
some
of
the
functions
in
the
kernel
see
exact
services
also
calls
directly
to
the
libraries
in
kernel.
So
that's
the
flow
for
the
process,
isolated
containers.
Now
the
important
thing
to
note
here
is,
if
you
are
running
multiple
containers
right
process,
isolated
containers,
the
left
box
is
going
to
stay
the
same.
A
The
right
box
is
going
to
multiply
right
as
many
as
you
want,
but
as
you
can
see,
the
security
boundary
the
virtualization
isolation
boundary,
it
stays
the
same.
So
this
the
security
boundary
is
the
the
red
dotted
line
and
that's
important
to
keep
in
mind.
Now,
let's
take
a
look
at
hyper-v,
isolated
containers
and
the
benefit
it
provides
in
terms
of
security.
A
So
if
we
go
on
to
to
this
architecture,
so
now
the
the
previous
slide,
what
we
were
looking
at
the
boxes,
think
of
it.
That's
on
the
left
side
now
and
we're
kind
of
zoomed
out,
so
the
flow
basically
stays
the
same.
There
is
a
slight
difference
in
the
workflow
and
I'll
talk
about
it
in
a
bit,
but
basically
here
you
have
the
hypervisor
on
the
bottom
and
the
root
has
the
kernel
and
then
the
on
the
right
side.
This
this
red
dotted
line
we're
calling
w
cow
uvm.
A
It's
fancy
way
of
saying
a
lightweight
virtual
machine.
It's
a
utility
vm,
so
in
hyper-v
container
behind
the
scenes
it
creates
this
light
very
lightweight
utility
virtual,
the
vm,
and
this
vm
is
where
the
container
is
hosted
and
that's
why
it
provides
the
security
boundary.
As
you
can
see.
Now,
if
you
have
multiple
hyper-v
containers,
each
one
is
within
that
security
boundary.
A
When
you
ask
for
a
new
container
on
the
right
side
that
whole
uvm
is
gonna
to
get
started,
and
then
the
service
that's
going
to
talk
to
is
gcs
in
the
previous
workflow,
we
used
to
have
the
gcs
gcs
talking
to
hcs,
just
like
in
process,
isolated
containers,
but
now
container
d
directly
talks
to
gcs
and
then
gcs
from
that
point
onward
within
the
uvm.
You
have
the
root
just
like
we
saw
before,
and
you
have
the
container
right
and
its
exact
same
flow.
A
The
c
exec
services
is
the
main
service
within
the
hyper-v
container,
and
it
talks
to
gcs
and
gcs
is
basically
managing
the
flow.
So
that's
the
architecture
of
process,
related
containers
and
hyper
vr
containers.
Now.
One
thing
I
wanted
to
talk
about
besides
the
security
boundary
is,
if
you
see
on
the
right
side,
I
have
a
legend
which
says,
ownership
and
support
and
the
reason
I
brought
that
up
is-
and
I
color
code
it
is-
is
often
we
get
questions
on
like
where
you
know.
A
If
we
are
having
issues
here,
where
should
we
go
so
the
answer
is:
if
you're
looking
at
the
top
cubelet
or
cry
ctl,
they
both
belong
to
the
kubernetes
community.
Most
of
you
are
already
contributor.
If
you're,
not,
we
encourage
you
to
become
contributors,
so
you
can
go
to
the
kubernetes
community
for
support
for
cubelet
and
cryctl.
A
Now,
if
you
go
to
the
next
layer,
that's
the
green
boxes,
right
the
container
d,
the
cni
plug-in
and
container
hds
shim.
All
three
are
open
source
as
well.
They
have
their
own
repo
and
you
can
contribute
you
can
file
issues.
You
can
do
prs
against
the
repo
and
we
encourage
you
to
be
involved
in
those
as
well.
A
A
It's
basically
microsoft,
slash
windows,
containers
and
if
you
go
there,
you
can
file
issues
if
you're
having
any
issues
from
the
one
of
these
components.
You
can
file
issues
and
the
windows
team,
triages
them
and
gets
back
to
you
and
fixes
them
and
if
needed,
they
can
even
backboard
it
to
the
earlier
versions.
So
so
that's
a
little
bit
of
support,
model
and
architecture
and
if
you
have
any
question,
I'm
happy
to
answer
it
in
the
q,
a
section.
B
Thank
you
mars.
By
the
way
this
has
been
super
exciting.
Thank
you
for,
for
the
drill
down
into
container
d.
I
think
that
it's
super
beneficial
for
everybody
to
understand
all
the
different
moving
pieces,
what's
supported
by
whom
and
kind
of,
also
see
the
tremendous
effort
that
has
happened
out
of
our
community,
with
contributions
from
docker
and
microsoft
and
other
contributors
to
actually
make
this
a
reality.
It's
a
huge
effort.
B
B
Here
when
we
started
with
1.14
our
first
ga
support
of
containers
running
windows
in
kubernetes,
we
only
supported
one
operating
system,
and
that
was
windows,
server,
2019
and
what
ltsc
stands
for
in
the
legend
is
the
long-term
servicing
channel
that
microsoft
is
providing
that's
the
operating
system,
that's
going
to
be
supported
for
a
number
of
years,
and
the
url
at
the
bottom
indicates.
The
support
stands
for
microsoft,
including
extended
support
options
that
you
can
purchase
so
earlier
when
maz
was
talking
about
the
blue
components,
actually
the
gold
components
or
brown
being
microsoft.
B
B
The
second
type
of
support
that
we
have
is
the
sac
or
sag,
and
it
stands
for
semi-annual
servicing
channel
and
our
goal
from
the
kubernetes
community
has
been
to
support
one
long-term
servicing
channel
and
two
additional
sec
channels
from
microsoft,
most
likely
the
two
most
recent
ones.
So
we
started
the
windows
server
version
1903,
which
is
no
longer
supported
now
by
the
kubernetes
community
and
the
with
version
119
we're
supporting
three
operating
systems,
the
long
term
windows,
server,
2019,
the
sac
1909
and
the
sac
2004.
B
A
Sure
so
yeah
it's
really
exciting
and
I
love
your
excitement
about.
You
know
the
container
d
and
the
improvement
in
the
overall.
You
know
kubernetes
for
windows,
so
in
container
d
support
you
know
we
started
with
alpha
support
in
version
118,
and
the
container
d
version
was
one
four.
That's
when
we
started
supporting
you
know
in
alpha
stage.
A
Of
course,
we
tested
it
on
2019,
the
ltsc
you
just
talked
about,
and
recently
we're
super
excited
that
you
know,
as
you
you
said,
container
d
graduated
to
beta.
The
thing
to
call
out
is
it's
without
hyper-v.
For
now
and
it's
in
119.
again,
the
container
d
version
stays
the
same.
It's
one
four,
the
last
row
is
actually
is
our
aspirations.
That's
why
I
put
tentative
there.
A
So
we
want
to
go
ga
with
container
d
support
for
windows,
containers
on
kubernetes,
with
hyper-v
and
gmsa,
and
we
are
planning
to
do
it
at
120,
the
kubernetes
version.
The
container
d
version
is
probably
going
to
be
1.5
and
the
windows
release.
As
just
michael,
you
said,
the
we
are
going
to
test
it
and
support
it
for
the
latest
ltsc,
which
is
2019
and
that
the
the
latest
two
sag
for
now
are
1909
and
2004..
B
Yeah
absolutely
thank
you
mars.
So
let's
take
a
look
at
some
of
our
networking
enhancements
here.
So,
in
addition
to
compute
and
storage,
we
also
have
networking
work,
that's
going
on,
and
I
want
to
call
out
that
some
of
our
storage
enhancements
around
csi
we're
talking
about
those
in
our
intro
to
windows,
containers
session
that
we
have
so
from
a
networking
standpoint.
We've
seen
that
tremendous
effort
around
the
kubernetes
community
to
support
dual
stack
ipv6.
B
So
it's
not
ipv6
only
support
it's
dual
stack
and
we
now
support
it
on
windows
as
well,
so
you're
going
to
see
that
as
kubernetes
is
getting
more
and
more
features
out
our
goal.
Let's
seek
windows
communities
to
make
sure
that
we
provide
adequate
support
for
those
features
for
for
the
windows
containers
community,
some
other
improvements
to
kubernetes
services
and
point
slices.
Endpoint
slices
are
amazing.
They
basically
with
an
endpoint
slice.
Is
it
allows
you
to
reference?
B
So
when
you
enable
endpoint
slices
and
with
the
support
that
we
have
now
in
windows,
you're
essentially
providing
a
performance
improvement,
so
we
can
support
services
with
a
large
number
of
endpoints
efficiently.
This
is
great.
You
know
we
can
have
a
large
number
of
endpoints
and,
more
importantly,
with
the
next
set
of
support
that
we
have,
which
is
dsr.
B
Now
we
support
you
to
scale
to
a
large
number
of
services
efficiently,
so
two
things
scale
to
a
large
number
of
services
and
scale
to
a
large
number
of
endpoints
per
service,
so
great
we're
at
the
support
for
session
affinity.
So
a
lot
of
customers
and
users
call
sticky
sessions.
We
have
dynamic,
I
pre
preservation.
B
B
We
want
to
continue
our
investments
in
deployment
and
cycle
management,
so
you're
going
to
see
cuba,
dm
cluster
api
investments
and
then
storage
and
backup
and
recovery.
We
want
to
promote
the
csi
work
to
stable,
along
with
the
container
d
work
that
we're
doing.
We
want
to
add
more
storage
providers.
We
want
to
light
up
valero
support,
csi
snapshots,
so
you
can
backup
and
recover
your
windows
containers
on
kubernetes,
as
well
as
deprecate
some
of
the
in
three
storage.
B
If
you're
gonna
reach
us
as
a
windows
community,
we're
on
slack
or
a
mailing
list,
we
have
our
documentation,
our
github
repo.
We
have
a
youtube,
playlist
and
community
meetings
that
happen
every
week
on
the
left.
We
have
the
co-chairs,
as
well
as
the
technical
leads
for
the
project,
feel
free
to
reach
any
of
us,
either
on
slack
or
github
muzz,
and
I
would
like
to
thank
you
for
attending
our
session
and
we're
going
to
open
this
up
for
q.
A
right
now.
C
B
B
If
we
run
out
of
time,
we
can
continue
the
conversation
on
slack.
So
if
you
go
to
the
cncf
slack
under
2-cubicon,
maintainer
there's
a
new
thread
that
was
created
a
few
minutes
ago
for
the
for
this
specific
presentation.
So
you
can
also
ask
questions
there,
offline
or
throughout
the
day
we'll
try
to
monitor
it.
You
can
call
out
either
me
or
maz
that
way:
you'll
get
the
notification
as
well.
If
you
like.
B
We're
also
encouraging
folks
to
attend
the
other
sequin
dos
presentation.
That's
that's
on
the
schedule!
So
it's
if
you're,
if
you
want
to
learn
more
about
additional
areas
on
sick
windows
like
our
csi
support
and
some
of
the
other
work,
we're
doing
around
improvements
around
networking
and
storage
or
highly
encouraged.
That
presentation
is
tomorrow
I'm
trying
to
find
the
time
it's
tomorrow
at
10
a.m.
Central
so
at
six
hours
for
for
european
time
zones
against.
B
Given
that
there
are
no
questions,
the
team
here
is
mentioning
to
us
that
they
might
end
the
webcast
please
if
anybody
have
any
questions,
maz-
and
I
were
more
than
happy
to
answer
it.
Thank
you
all
for
spending
time
with
us
to
learn
about
windows,
but
if
you
think
something
off
later
on
just
find
us
on
slack
also,
as
we've
shown
on
the
previous
slide,
I'm
going
to
go
to
the
slide
again.
B
C
A
B
Absolutely,
and
even
yesterday,
james
one
of
our
contributors
from
microsoft
just
submitted
a
proposal
for
introducing
cluster
api
work
for
for
windows.
So
anyone
that's
interested
in
programmatic
api,
declarative
way
of
provisioning
clusters
and
getting
up
and
running
with
some
impact
on
windows
as
a
great
area
to
come
in
and
contribute
and
learn
on
a
hot
area
of
kubernetes,
which
is
clustered
api.