►
From YouTube: Windows Containers in Kubernetes and a... Mark Rossetti, Kalya Subramanian, David Schott & Jay Vyas
Description
Don’t miss out! Join us at our upcoming event: KubeCon + CloudNativeCon North America 2021 in Los Angeles, CA from October 12-15. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Windows Containers in Kubernetes and a Deep Dive into Windows Networking - Mark Rossetti, Kalya Subramanian & David Schott, Microsoft; Jay Vyas, VMware
Members of the SIG-Windows community will provide an update on the efforts to brining Windows workloads to Kubernetes. We'll start off by going over recent improvements. Next we'll talk about planned future improvements and demo privileged contianers for Windows. We'll then start deep diving into some networking topics like network load balancing with DSR, discussing network policy, and providing updates on the states of various CNI solutions for Windows. Lastly we'll finish up with some Questions and Answers with the speakers.
A
A
B
Hi
everyone,
I'm
kalia
subramanian
and
I'm
a
software
engineer
at
microsoft
and
I'm
currently
a
member
of
the
azure
container
upstream
team
with
mark,
but
I
was
formerly
on
windows
container
networking
and
my
past
windows
project
have
included
windows,
overlay,
networking,
flannel
and
cube
admin.
You
can
find
me
on
github
at
k,
superman.
C
D
A
A
A
This
is
a
great
starting
point
for
anybody
who's
interested
in
getting
windows
nodes
added
to
their
clusters
for
the
first
time,
I'm
going
to
go
pretty
quickly
here
so
that
we
can
get
into
our
network
deep
dive,
which
is
going
to
be
kind
of
the
meat
of
this
talk,
as
we've
got
a
lot
of
requests
for
david.
Do
you
want
to
take
it
over.
C
Yeah
sure,
thanks
mike
so
many
are
familiar
with
container
networking
on
linux
and
all
the
kind
of
iptables
magic
that
happens
under
the
hood
was
covered
in
past
presentations.
However,
windows
still
remains
a
mystery,
so
let's
take
a
look
at
some
of
the
windows.
Networking
constructs
that
are
relevant
to
kubernetes
networking.
C
C
So
the
first
step
of
transforming
a
typical
windows
server
box
into
a
kubernetes
node,
is
to
create
a
virtual
switch.
This
provides
l2
switching
capabilities
and
also
enables
l3
based
network
communication
for
virtual
devices
that
are
attached
to
it.
A
pre-configured
virtual
switch
ready
for
container
networking
is
automatically
generated
by
creating
a
hns
network.
C
C
C
So
the
final
missing
piece
here
is
hms
policies.
These
rely
on
vfp
and
vfp
stands
for
virtual
filtering
platform.
This
is
essentially
a
virtual
switch
extension
that
allows
us
to
define
rules
to
process
incoming
and
outgoing
network
traffic
and
apply
actions
to
them.
The
hns
policies
resource
is
used
to
program,
vfp
and
plumb,
rules
that
can
apply
load,
balancing
network
address
translation
and
much
much
more
to
network
traffic.
C
D
Yeah,
so
cni
ecosystem
is
rapidly
evolving
for
windows,
and
a
lot
of
a
lot
of
folks
are
really
stepping
up
and
increasing
their
windows.
Support.
We've
got
news
on
the
andrea
front,
where
we
support
both
container
d
and
docker
at
this
point,
and
so
we're
really
happy
about
that
shout
out
to
ruequau
for
a
lot
of
that
work
and
perry
and,
of
course,
calico
as
well
now
is
on
aks.
D
You
can
preview
calico
with
network
policy,
support
for
windows,
nodes
and,
and
actually
it's
it's
for
in.
You
can
also
do
this
on
c2
as
well.
If
you
manually,
install
calco
and
there's
several
other
cri
players,
win
cni
with
windbridge
overlay
and
then
there's
azure
cni
as
well,
and
you
can
overlay
different
policy
providers
on
that,
and
so,
as
far
as
so
network
policy
goes
you
for,
for
for
andrea,
you
can
for
andrea.
D
You
have
network
policy
on
windows,
for
both
containing
docker
and
for
calico
it's
supporting
docker
network
policies
and
below
you
can
see
a
screenshot.
We
now
have
updated
the
upstream
kubernetes
network
policy
tests.
We
have
a
pr
in
flight
and
we
have
a
demo
of
this
on
the
recent
tgik
episode,
if
you're
interested
in
watching
where
we
have
the
we
have
truth
tables
that
actually
validate
the
network.
A
C
All
right,
so,
let's
next
take
a
closer
look
at
the
topic
of
cni
plugins
on
windows,
ci,
plugins
and
windows
are
invoked
the
same
way.
They
are
on
linux,
with
the
container
runtime
calling
a
cni,
plugin
and
essentially
just
reads
a
cni
configuration
file
and
any
additional
cni
args,
such
as
what
commands
to
execute.
C
C
So
the
original
docker
shim
workflow
in
windows
has
a
drawback
in
the
sense
that
there
are
repeated
consecutive
cni
calls
that
need
to
be
made
for
each
workload
container
in
the
pod.
One
possible
flaw
is
that
in
step
nine
displayed
here
there
is
dns
information
that
needs
to
be
set
in
registry
keys.
In
theory,
this
can
even
become
a
problem
because
this
information
gets
populated
when
the
container
is
already
running.
C
There
is
no
guaranteed
dns
readiness
on
plot
startup,
if
step
9
were
to
take
longer
than
expected
to
finalize
so
could
be
some
race
conditions
with
this
workflow
next
slide.
So
for
completeness,
here's
also
showcasing
how
there
are
multiple
consecutive
cni
calls
for
pod
teardown
on
the
document.
Workflow.
C
And
yeah
this
previous
problem
was
overcome
by
introducing
a
new
hns
entity
called
the
namespace,
so
the
namespace
contains
a
reference
to
the
network
compartment
unique
to
a
pod,
the
hns,
endpoint
and
containers
that
will
be
placed
there.
This
enables
us
to
update
the
workflow
so
that
there
is
only
one
cni
add
call
per
pod
creation,
some
sample
plugins
that
are
enlightened
to
use.
This
workflow
can
be
found
at
ak
dot.
Ms
slash
win
cni,
so
next
up
is
coulia.
She
will
be
talking
about
q,
proxy
and
load
balancing
improvements.
B
B
The
packet
is
routed
to
the
host
container
port
for
load,
balancing
resolution,
and
then
the
packet
leaves
the
host
with
the
host
ip
as
the
source
ip
in
dsr,
the
backend
pod
ip
is
selected
in
the
originating
container
port,
so
the
packet
leaves
the
host
with
the
originating
pod
ip
as
the
source.
Ip
dsr
ensures
lower
latency
and
greater
scalability,
because
we
don't
have
to
make
that
extra
hop
from
the
originating
container
port
to
the
host
container
port,
and
this
mode
is
supported
in
kubernetes
120
and
with
windows,
server,
2019
or
higher.
B
So
to
enable
and
set
this
feature,
you
have
to
set
the
following
flags
in
cube
proxy.
You
have
to
set
the
win.
Dsr
equals
true
feature
gate
since
dsr
is
an
alpha
feature,
and
then
you
also
have
to
enable
dsr
equals
true,
because
non-dsr
is
still
the
default
mode.
When
q,
proxy
programs,
the
h
and
s
policies.
B
So
these
diagrams
explain
the
flow
of
the
packet
for
non-dsr
and
dsr
for
l2
bridge.
The
flow
is
a
little
different
in
overlay,
but
the
underlying
concept
remains
the
same
and
non-dsr.
The
packet
leaves
the
container
port
and
it's
encapped
so
that
it's
destined
to
the
host
v-neck
and
then,
when
it's
in
the
host
phoenix
bfp
is
responsible
for
selecting
the
back
end
ip
and
then
the
packet
is
hairpinned
and
sent
out
of
the
host
with
dsr
when
the
packet
leaves
the
container
port.
B
It
reduces
the
network
latency
and
has
a
better
data
path
performance
again,
because
we
don't
have
that
extra
hop
to
the
host
v-neck
and
then
this
greatly
simplifies
the
network
traffic
flow.
So
if
you
were
to
actually
look
at
the
flow
of
the
packet,
I
think
it's
a
little
more
intuitive
with
what
we
would
expect
when
you're
in
dsr
mode.
B
B
We
also
have
a
new
feature
session
affinity.
Betsy
worked
at
this
worked
on
this.
The
windows
container,
networking
team
at
microsoft,
and
so
this
feature
ensures
that
connections
from
a
particular
client
are
past
the
same
pod
each
time
this
is
supported
in
kubernetes,
119
and
windows,
server,
vnex
insider
preview,
build
19551
or
higher,
and
to
enable
this
just
set
service
accession
affinity
to
client
ip.
B
So
this
parallels
the
linux
work
that
has
been
done
in
this
space
as
well,
and
this
is
supported
in
kubernetes
120
in
windows,
server
version,
2004
and
higher.
It
is
compatible
with
l2
bridge,
but
it's
not
supported
for
overlay
in
vxlan.
So
keep
that
in
mind
and
then
see
the
windows.
Ipv4
ipv6
dual
stack
documentation
for
more
info
on
how
to
set
that
up.
B
So,
with
all
those
networking
scenarios,
sometimes
it
might
be
necessary
to
troubleshoot.
Fortunately,
we
do
have
a
set
of
tools
in
github,
so
we
have
a
validation
script.
So
this
is
a
powershell
script.
That
kind
of
looks
and
inspects
your
cluster
by
running
a
test
scenario
with
windows
web
server,
and
it
ensures
that
all
of
the
h
and
s
policies
are
being
configured
properly.
B
And
then
you
can
also
analyze
the
h
and
s
networking
state
more
easily
using
the
collect
logs
ps1
script.
So
this
way
you
don't
have
to
manually
check
it
in
powershell
yourself.
You
can
just
get
the
state
of
hns
and
then
have
that
outputted
to
a
text
file,
and
we
also
have
a
great
kubernetes
network,
troubleshooting
guide
that
david
helped
put
together.
D
A
That
is
a
good
point.
Thank
you.
Jay.
All
right
now,
we'll
talk
a
little
bit
about
kind
of
the
future
plans
and
what's
next
for
windows,
workloads
and
kubernetes,
the
big
kind
of
feature
enhancement
that
we've
been
working
towards
are
host
process
containers
host
process
containers
are
roughly
equivalent
to
privileged
containers
on
linux,
but
due
to
kind
of
the
nature
of
how
windows
and
windows
containers
work.
A
I
think
this
is
really
gonna
enable
a
lot
of
different
scenarios
for
windows
right
now,
we're
targeting
an
alpha
release
in
v
122,
which
should
be
coming
out
in
the
next
couple
of
months
and
there's
a
big
enhancement
proposal
there.
Please
read
it
comment,
share
your
thoughts
and
anything
else.
Just
one
note
that
this
will
only
be
supported
with
container
d,
as
that
is
kind
of
the
future
of
the
sierra
of
sierra
state
and
everything
moving
forward.
D
Yeah,
so
windows
is
a
lot
of
support
here,
for
folks
want
to
get
involved
with
windows.
Our
our
tooling
is
getting
better.
It's
not
it's!
It's
it's!
It's
obviously
not
in
the
same
state
as
the
linux
stuff,
but
it's
getting
better
every
day
and
we've
got
people
contributing
and
getting
involved
to
make
that
better.
There's
a
sig
windows
community
page:
you
can
jump
in
on
there.
D
D
We
transition
into
just
like
hacking
around
on
stuff
and
people
talking
about
the
problems
they're
having
and
going
through
random
issues
and
trying
to
build
stuff,
and
so
it's
a
really
cool
way
to
just
get
in
from
that
kind
of
visceral
ground
level.
You
don't
need
any
training
or
anything,
and
so
just
come
hang
out
with
us.
If
you
don't,
if
you
don't
know
anything
about
windows
or
kubernetes,
that's
fine.
D
D
He
done
so
much
stuff
with
images,
testing
making
sure
we've
got
the
right
tooling
in
place
so
that
our
e2es
work,
properly
and
and
and
so
many
other
things
he's
worked-
worked
on
the
low
level
problems
that
are
very
hard
to
figure
out
and
stuff
perry
as
well.
A
lot
of
deep
deep
technology
he's
built
out
with
rui
and
other
people
around
the
container
d
side
around
the
cni
side
has
been
working
with
both
actually
the
calico,
as
well
as
the
android
folks.
D
On
that
microsoft,
container,
networking
team
there's
so
many
people
there.
I
don't
even
know
all
their
names,
maybe
kayla
does,
but
they
they
have
been
working
with
perry.
They've
been
working
with
other
all
of
us
to
answer
questions
and
meeting
with
us
and
stuff
there's
a
very,
very
vibrant
group
of
people.
That'll
just
talk
to
you
anytime.
If
you
just
come
on
slack
and
start
hanging
out
so
yeah.
A
And
I
wanted
to
reiterate
what
jay
had
said
for
that
sig
windows
pairing
meeting:
it's
come
if
you
are
familiar
with
windows,
but
not
familiar
with
kubernetes
and
want
to
learn
more
with
that,
come
if
you
are
familiar
with
kubernetes
and
linux,
but
not
with
windows
and
want
to
see
how
things
work
on
windows
come
to
just
hang
out
and
see.
What's
going
on,
really
encourage
everybody
to
join
that
as
well.
D
D
Is
this
me,
okay,
yeah,
my
mark
is
the
co-chair
me
and
james
are
the
tech
leads,
we've
got
a
youtube
playlist.
I
also
have
yeah.
We
have
a
youtube.
Playlist
we've
got
the
sig
windows
slack
channel.
That's
really
the
ingress
point
for
figuring
out
what
to
do.
Come.
Join
us
in
syd
windows,
upstream
slack.
There
is
a
google
group
also
for
more
long-form
formal
questions,
but
yeah
file,
an
issue
get
on
slack,
come
hang
out
with
us.
That's
how
we
do
things
very,
very
informal
and
and
very
active
community.
A
Thank
you,
jay.
Thank
you,
everybody
for
talking.
We
do
have
a
couple
of
appendix
slides
we
wanted
to
include
in
this
talk
for
future
reference.
If
anybody
is
interested,
please
download
the
slides,
I
believe,
there's
a
link
to
the
slides
on
the
skedge.com
page,
there's
just
more
information
on
how
kind
of
all
of
the
different
networking
scenarios
work.
What's
all
supported,
what
versions
of
windows
they're
supported
on
and
all
that
good
info
here
kind
of
just
page
through
these
really
quickly,
and
that
concludes
our
talk.
Thank
you.