►
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Intro + Deep Dive: Windows SIG - Michael Michael, VMware & Patrick Lang, Microsoft
The chairs for SIG-Windows will provide an update on the efforts to bring Windows to Kubernetes. This will concentrate on presenting new features that are being delivered and the General Availability of Windows Containers on Kubernetes. We will also have a detailed discussion on our future roadmap, key functionality that we want to enable, and open the floor for Q&A with customers and members of the SIG-Windows community. This is both an introduction to Windows on Kubernetes and a deep dive into the efforts of the team. Some familiarity with Windows on Kubernetes is required for the deep dive part since we will have an in-depth discussion on key features that are in the pipeline for Windows, explain their implementation and have a discussion on tradeoffs with the community.
https://sched.co/MPlM
A
All
right,
hello,
everybody
and
welcome
to
the
Sig
Windows
introduction,
as
well
as
deep
dive.
I
know.
We
don't
have
a
lot
of
you
guys.
Today.
We
actually
have
a
conflicting
Windows
talk
right
next
door
on
Windows
storage.
So
for
any
of
you
that
also
want
to
watch
that
one
I'm
sure
they'll
have
them
recorded,
so
you'll
be
able
to
check
it
offline
and
vice-versa.
Those
guys
will
see
ours.
My
name
is
Michael
I'm,
director
of
product
management
at
VMware
and
I'm,
one
of
the
two
chairs
for
sick
windows.
A
Get
started
we
want
to
keep
this
interactive.
This
is
one
of
the
sig
windows
talks.
So
if
you
guys
have
any
questions
or
anything
that
you
guys
want
to
ask
raise
your
hand
if
we
addressed
in
a
future
slide,
will
ask
you
to
wait
otherwise,
we'll
answer
it
right
away
all
right.
So
I
want
to
talk
to
you
a
little
bit
about
how
we
got
here.
So
when
we
started
sick
windows
initially
was
more
of
a
proof-of-concept.
A
We
wanted
to
make
sure
that
the
things
that
we
wanted
to
accomplish
were
possible
within
the
kubernetes
ecosystem,
and
then
we
started
to
put
the
right
pieces
in
place
so
that
it
can
get
us
to
GA.
So
our
first
release
was
with
kubernetes
1.5
and
that
release
really
centered
around
showing
the
art
of
the
possible
illustrate
him
to
everybody
that
you
can
have
a
cubelet.
It
can
run
on
Windows
the
same
thing
with
the
cube
proxy
and
we
can
schedule
containers
on
the
cubelet
and
on
the
node
effectively
managing
them
using
kubernetes.
A
There
were
a
lot
of
hacks
into
the
release
and
we
admit
that,
but
it
kind
of
rallied
the
community
behind
our
effort
and
allowed
us
to
make
more
contributions
from
many
companies
to
get
us
to
the
next
stage,
which
was
with
version
1.9
in
December
of
2017.
That's
when
we
announced
beta
and
we
started
gathering
a
lot
of
the
momentum.
A
new
version
of
the
operating
system
was
out.
A
We
had
advancements
at
the
networking
layer
that
allowed
us
to
start
building
sea
and
ice
on
top
of
our
solution,
and
then
we
started
kind
of
streamlining
all
the
things
that
we
needed
to
do
to
get
to
GA
with
kubernetes
1:14.
In
March
of
this
year
we
had
our
stable
release
that
allows
you
to
run
production
level
workloads
on
kubernetes.
That
means
we'll
support
you
as
a
community
if
you
want
to
put
your
business
applications
on
kubernetes
now
that
doesn't
mean
that
were
completely
done.
A
So
how
do
you
get
here?
So
kubernetes
114?
If
you
guys,
are
familiar?
That's
the
cocoon
Eddie's
logo
that
I,
don't
the
seek
beard
decided
that
was
going
to
be
the
logo
for
this
release.
Patrick
myself
and
our
entire
cig
windows
team
really
worked
day
and
night,
the
last
couple
of
months
to
get
us
to
GA.
A
That
means
you
got
Windows
Server,
2000
19
support
who
had
significant
advancements
in
code
quality
and
scalability
of
Windows
containers
in
kubernetes,
and
then
we
did
a
complete
end
to
invalidation
of
user
scenarios
if
you're
familiar
with
kubernetes
there's
a
process
called
the
kubernetes
enhancement
proposal
or
the
caps.
So
we
had
to
write
a
very
detailed
cap
that
outlined,
would
you
will
be
able
to
accomplish
what
won't
work
and
what
may
never
work
with
114?
A
So
now
you
can
assign
to
your
pod
or
do
your
container
GMS
a
that's
managed
by
your
ad
administrator
and
assigned
to
the
node
where
your
container
is
running
on
and
then
any
outgoing
course
on
the
network
from
the
container
will
carry
that
identity.
That's
very
useful
in
Windows
workloads,
think
of
the
ability
to
preach
of
a
file
server
that
are
called
for
the
identity
or
reach
a
database
that
will
only
authenticate
a
specific
account
so
as
you're
running
distributed
applications
across
your
windows
ecosystem.
That's
a
necessary
component
to
make
that
possible.
A
We
will
continue
advancing
their
work
to
your
stable,
probably
towards
the
end
of
this
year,
but
the
initial
feedback
is
really
great.
People
are
using
it.
It
worked
very
well
and
last
but
not
least,
you
know
extensive
automation,
extensive
documentation,
so
we've
spent
a
lot
of
time
to
capture
end
user
scenarios
getting
started
scenarios
introducing
a
node
into
kubernetes
scenarios.
B
Yes,
so
the
question
was
what
about
Windows
Server
2016
support,
so
one
of
the
things
that
changed
a
lot
between
2016
and
2019
was
actually
the
way
that
the
networking
stuff
was
all
deployed,
and
that
enabled
us
to
actually
be
able
to
do
multiple
containers
per
pod.
But
in
order
to
do
that,
we
had
to
make
some
trade-offs
and
remove
some
of
the
code
that
was
needed
for
2016,
and
so
it
was
going
to
be.
B
C
B
Yeah,
so
the
windows
team
doesn't
back
port
in
new
features
like
that
back
to
previous
releases
and
so
I
don't
really
see
2016
being
feasible.
That
was,
you
know,
one
of
our
goals
and
the
development
process
was
to
get
it
landed
with
the
next
long-term
release,
which
was
the
2019,
and
so
that's
just
you
know
what
the
trade-off
that
we
ended
up
with
there.
But
that
is
a
release.
A
B
But
I'm
not
aware
of
any
ski
restriction.
In
fact,
we
might
not
be
able
to
get
Windows
10
working
for
non
production
stuff,
but
the
container
role
is
there
and
in
both
the
standard
and
the
enterprise
and
the
data
center
SKUs.
And
so
like
that's
what
you
need
to
enable
on
the
windows
side
as
a
prerequisite,
then
you
go
and
install
the
components
that
run
on
the
node,
like
the
docker,
Enterprise
Edition
basic,
the
cubelet
cube
proxy,
and
things
like
that.
I.
A
B
Okay,
so
one
of
the
big
changes
that
you
know
that
I
mentioned
a
bit
earlier
was
that
would
change
the
networking
model
in
2019
and
then
over
the
course
of
kubernetes
13
and
14.
We
were
able
to
spend
a
lot
of
time
basically
doing
some
end-to-end
testing
and
tuning
the
different
container.
Networking
infrastructure
implement
CNI
plugins
over
the
course
of
that,
and
so
Windows
itself
has
kind
of
three
different
ways
that
it
can
route
traffic
but
and
that
enables
different
types
of
network
plugins
to
be
built.
B
So
if
you've,
if
you
want
to
do
an
overlay
Network
where
you
know
everything's
virtualized
encapsulated
with
VX
LAN,
that's
something
that
was
enabled
in
and
as
part
of
the
cumulative
updates.
A
couple
months
ago
there
were
a
few
things
that
weren't
quite
right
when
it
when
Windows
Server,
2000
nine
teen,
VX
LAN,
would
work.
B
But
now
that's
done,
and
so
you
don't
need
to
install
that
KB.
Specifically,
you
just
need
to
install
anyone.
That's
that
that
one
or
newer,
and
so
as
long
as
you're
patching
the
server
you're
gonna,
have
what
you
need
to
be
able
to
enable
VX
LAN
and
then
for
each
of
these
networking
modes.
Microsoft
has
provided
an
open-source
kind
of
like
a
reference
implementation,
and
this
is
designed
to
make
it
a
lot
easier
for
you
to
actually
hook
your
networking
up.
So
we've
got
a
plug-in
called
win
overlay
that
works
with
the
via
works.
B
You
know
with
overlay
networking,
but
basically
what
these
do
is
program
the
network
interfaces
so
where
people
may
have
gone
and
created
a
virtual
network,
adapter
and
hadn't
applied
IP
tables
on
Linux.
Instead,
you
can
call
this
pre-made
C&I
plugin.
It
calls
the
right
windows.
Api
is
to
configure
those
adapters.
The
way
you
want
and
then
you
can
go
from
there
and
so
things
like
flannel
that
are
but
can
actually
build
on
top
of
this.
B
So
if
you
want
to
do
like
a
custom,
IPAM
implementation,
which
is
like
the
court,
the
case
of
what
was
done
for
a
Google
cloud
engine
to
bring
those
tests
online,
Peter,
horn
yak
and
you
Joo
Hong,
basically
built
on
top
of
the
next
one,
which
is
the
wind
wind
bridge
plug-in
and
but
they
basically
just
use
that
that
delegate
as
is-
and
so
we
expect
that
to
make
it
easier
for
other
plugins
to
be
developed.
On
top
of
that.
B
So
the
next
to
networking
modes
are
very
similar
but
a
little
but
there's
one
key
difference
between
them.
So
an
l2
bridge
is
going
to
give
an
network
interface
to
that
pod.
And
then
it's
going
to
get.
You
know
a
regular
pot
IP
and
then
we
program
all
the
routing
rules.
You
through
cube
proxy
there
and
that's
basically
the
same
way
that
like
cube
networks
or
if
you're,
using
flannel
in
the
less
common
host
gateway
mode,
that's
the
way
that's
going
to
work
as
well.
B
So
whenever
things
services
are
started
and
it
shut
down,
all
these
rules
need
to
be
reprogrammed
by
a
cube
proxy
plus
that,
but
but
it
works
pretty
well.
However,
once
you're
at
a
larger
scale,
then
that's
where
the
VX
land
can
work
a
little
bit
better,
because
some
of
that
data
plane
stuff
doesn't
need
to
you
don't
need
to
redo
all
the
routing
rules,
just
because
a
pod
was
added
or
removed,
and
so
because
all
it's
simpler
and
actually.
A
On
top
of
that,
because
of
all
the
reading
of
the
routing
rules,
we've
had
some
performance
issues
in
the
past
that
we've
addressed
so
yeah.
We
got
feedback
from
the
community
with
the
team
of
Microsoft
and
the
flannel
larger
team
did
go
and
work
on
that,
and
now
it's
a
lot
more
efficient
in
terms
of
how
its
configuring
that
yeah.
B
And
then
the
similar
mode
to
l2
bridge
is
one
called
l2
tunnel,
and
so,
if
you've
got
you
know
like
a
top-of-rack
switch
or
some
sort
of
network
appliance
running,
you
could
actually
program
that
to
route
all
all
traffic
for
you.
So
instead
of
having
to
handle
all
the
routing
rules
on
the
node,
you
could
basically
delegate
that
to
something
else,
and
so
100%
of
your
packets
will
be
forwarded
to
that
and
then
distributed
from
there.
But
if
you
were
going
to.
C
B
A
different
type
of
implementation,
like
I've,
seen
this
done
and
done
integer
as
well
for
some
of
their
appliances.
They
will
use
the
tunnel
mode
rather
than
a
bridge
mode,
so
that
way,
they're
not
making
those
routing
decisions
on
the
node
it's
being
done
after
it
leaves
box
and
then
the
last
one
is
for
the
last
I,
don't
know
if,
since
probably
2012
I
think
maybe
earlier,
Microsoft
has
had
a
what's
called
a
virtual
switch
extension
that
actually
started
in
hyper-v.
But
what
lets
you
do
is
just
completely
replace
the
network
stack
and.
B
So
the
open
virtual
switch
from
the
OpenStack
project
can
actually
be
used
instead.
So
if
you
want
networking
infrastructure
that
is
consistent
and
managed
using
open
virtual
networking
using
like
a
nouvion
controller
from
OpenStack,
you
can
actually
use
that
as
well
to
give
your
to
give
all
of
your
pods
connections.
Using
that
open
virtual
switch
and
open
virtual
networking
combination
and
I.
Think
that's
sort
of
similar,
like
dude.
A
A
So,
if
that's
something
that's
important
to
you,
guys
on
the
open
source
side
that
supports
ya,
if
you're
looking
for
the
most
stable
saying
I,
probably
would
have
heard
so
far
is
the
flannel
host
gateway
is
the
one
that
most
folks
use,
but
it's
more
of
a
basic
network
implementation
as
you're
advancing
and
you're,
putting
a
little
bit
more
security
and
harden
in
your
environment
and
seeing
eyes
like
the
nsx,
T
and
calico
and
others
that
are
coming
down
the
line
or
are
gonna
kind
of
fill
that
gap.
Yeah.
B
And
in
calico
actually
rather
to
gara
announce
that
they're
commercially,
supporting
calico
on
Windows
for
just
for
the
policies
right
now,
they're
probably
going
to
do
more
later.
But
that
is
something
that's
available,
both
in
the
open
source
project
and
commercially,
and
so
we
expect
more
and
more
stuff
to
come
in
this
ecosystem.
As
people
are
deploying
more
things
in
enterprises.
C
B
So
the
answer
on
that
one
is
that
probably
not
immediately
you
know
like
in
order
to
do
that,
we
would
really
need
to
get
the
hyper
V
isolation
layer
in
place
first,
because
that's
needed
to
be
able
to
create
and
manage
of
the
virtual
functions,
so
the
hyper
V
support
and
kubernetes
is
still
pretty
early,
but
once
we
get
that
developed
a
little
bit
further
along.
That
would
be
a
good
time
for
for
us
to
look
at
that.
B
Oh
yeah,
that's
a
good
piece
of
feedback
because
it
works
for
V
ends
today,
and
so
we
should
be
able
to
make
it
work
for
pods
once
we
won
once
we
get
a
few
other
prerequisites
met
and
I've
actually
got
slide
later
talking
about
how
we're
changing
some
of
the
runtime
I.
Don't
talk
to
SR
iove
specifically,
but
there's
a
few
things.
We
need
to
do
to
get
more
of
the
features
of
Windows
working.
D
B
B
Okay,
so
when
you're
going
to
just
start
deploying
things
in
a
mixed
cluster,
there's
a
problem
there's
a
couple
things
that
you
need
to
look
at
so
one
day.
If
you
only
wanted
to
run
Windows
apps,
then
what
you
could
do
is
basically
just
run
your
control
plane
on
Linux
and
then
add
a
bunch
of
Windows
nodes,
and
you
could
do
that.
B
B
That's
running
some
Linux
workloads
and
add
Windows
nodes
to
it,
and
if
that's
not
what
you
want
to
do,
let's
talk
about
that
more
detail
at
the
end,
but
in
that
mode
the
easiest
way
to
avoid
breaking
stuff
is
to
go
ahead
and
add
a
taint
to
the
windows
nodes.
And
then,
when
you
create
a
deployment,
you
just
put
a
corresponding
Toleration
on
there
and
so
that'll.
A
The
current
expectation
on
that
is
that
most
customers
have
a
tremendous
number
of
investments
into
existing
llaman
files
and
pot
specs
for
their
existing
apps.
We
didn't
want
to
impact
the
Linux
ecosystem
where
they
had
to
go
and
change
every
of
them
and
put
note
selectors
in
it.
So
the
easiest
way
for
us
was
to,
as
Windows
was
just
going
into
GA
right
now.
A
Allow
you
guys
to
co-create.
The
tents
are
the
Toleration
and
since
you're
writing
new
pod
specs
for
Windows
workloads,
and
it's
a
simple
addition
to
add,
and
it
enables
you
to
kind
of
isolate
your
Linux
workloads
from
your
Windows
workloads.
In
fact,
some
of
the
distributions
that
we
have
that
are
the
turnkey
solutions,
automatically
add
the
taint
for
you
and.
B
The
other
thing
related
to
that
is
in
kubernetes
1.15.
There's
a
proposal
to
add
scheduling,
constraints
to
what
are
known
as
runtime
classes
and
so
runtime
class
is
a
new
field
that
was
added
as
alpha
in
either
13
or
14.
But
what
it
lets
you
do
is
define
a
basically
a
I'm
gonna
step
backward
so
when
the
first
use
cases
that
they
had
intended
for
runtime
class
was,
if
you
want,
when
you're
doing
a
deployment.
B
If
you
want
to
have
a
group
of
settings
like
I,
want
to
use
cada
and
I
want
these
other
settings
applied
on
the
Linux
side
and
I've
got
other
deployments
that
I
just
want
to
run
using
using
docker
without
kata,
then
what
you
could
do
is
create
two
different
runtime
classes.
This
gives
admins
a
way
to
sort
of
codify
what
settings
they
want
to
pin
on
all
the
deployments
and
then
the
admin
or
developer
that's
to
playing
their
app
just
says.
B
B
So
the
other
thing
you
need
to
look
at
when
you're
doing
a
pod
spec
is
that
some
of
the
Linux
templates
have
like
really
low
memory
limits
of
like
a
hundred
Meg's
or
so
in
some
of
the
examples.
And
if
you
want
to
start
a
Windows
pod
you're
the
containers
going
to
need
at
least
three
hundred
Meg's,
and
that's
because,
with
all
the
windows
services
like
the
local
security
agent
and
things
that
are
needed
to
support
a
lot
of
the
identity,
api's
and
stuff
within
that
container,
need
to
be
duplicated
for
each
container.
B
They're,
not
shared
they're,
always
isolated,
and
so
like
this
just
concept
on
on
linux,
where
there's
a
kappahd.
Sorry,
a
security
option
that
says
run
as
non-root.
Isn't
there,
because
on
Windows
that
identity
is
never
shared
between
the
hosts
in
the
container.
It's
it's
coming
from
a
completely
separate
security
database.
B
The
other
thing
you
need
to
be
sure
it
deployments
like
we
already
had
the
question
about
the
Windows
version
support,
and
that
is
that
you
need
to
make
sure
that
the
that
the
container
version
matches.
So,
if
you
try
to
deploy
a
container
that
was
built
using
Windows,
Server
2016
on
2019
today,
it's
gonna
fail
and
likewise
they
just
released
a
Windows
I.
Think
it's
1903,
which
is
weird
because
it's
it's
not
1903,
it's
later
on
that,
but
but
but
that
release
that
just
came
out
is
going
to
have
corresponding
container
images
as
well.
B
Those
don't
work
on
2019,
and
so
the
best
practice
is
that,
whenever
you're
going
to
deploy
something
when
you're
building
it
in
your
docker
file,
make
sure
that
your
from
line
is
using
a
container
with
a
specific
OS
version
and
a
tag.
And
so
if
you
go
and
look
at
things
like
the
Microsoft,
asp.net
images,
you'll
see
things
like
four
point:
seven
point:
two
windows
server
core
2019,
something
like
that.
But
that
way
you
could
tell
what
the
OS
version
is
if
you've
used
latest
some
of
those
ones
are
designed
to
be
able
to.
B
A
And
a
couple
more
things:
1903
is
not
supported
in
kubernetes
just
yet
so
that
might
be
something
we'll
address
in
the
future
release,
but
with
1:14
it's
not
supported,
and
earlier
we
talked
about
taints.
So
a
very
good
practice.
Is
it
not
only
tended
saying
in
Windows,
but
you
can
actually
indicate
the
version
of
Windows
so
so,
for
example,
you
contained
on
Windows,
Server,
2000,
19,
+
or
1809.
It's
the
same
OS
and
that
way,
when
you're
building
your
pod
specs,
you
can
add
the
Toleration
for
the
exact
same
version.
A
B
A
B
And
so
the
patch
version
can
differ,
it's
a
major
version
that
cannot,
and
so
you
don't
so
you
can't
run
2016
on
2019.
You
can't
run
1903
on
29:19,
but
if
one
of
the
images
was
built
against
the
December
patch
level
and
another
one
was
built
and
then
your
machine
is
running,
the
latest
patches
for
May,
the
demo
I've
got
set
up,
is
actually
doing
that
and
that
and
that
and
that's
fine.
C
B
It
didn't
work
in
2016,
it
works
in
2019
and
and
so
they
they're
basically
trying
to
make
sure
that
those
patches
don't
introduce
differences
there.
The
other
thing
we're
doing
is
with
the
hyper-v
support,
where
it's
going
to
be
possible
for
a
newer
OS
like
1903,
to
be
able
to
run
an
older
one,
but
we've
right
now,
that's
still
not
quite
working
100%
kubernetes
and
that's
one
of
the
areas
that
I'm
gonna
be
working
on
over
the
next
couple
of
releases
to
get
that
better.
B
B
And
so
when
a
kubernetes
releases
cut,
they
fork
the
tests
and
they
keep
that
running.
And
so
you
know
if
I
were
to
summarize
what
does
it
take
to
GA
something
in
kubernetes?
It
means
getting
test
grid
green,
that
that
is
the
biggest
factor,
because
that's
the
only
way
that
you
can
prevent
other
other
people
from
breaking
you,
and
so
we
finally
got
to
that
cut
to
that
point
for
14
and
g8
and
then
now
you
know
there
were
no
bugs
introduced
in
14.1
14.2.
Those
all
work
great
on
from
a
Windows
standpoint.
A
Big
shout
out
to
the
community
a
lot
of
folks
from
cloud
base
and
Patrick.
You
know,
there's
been
a
tremendous
amount
of
time
getting
these
two
green,
that
involved
evaluating
the
entire
suite
of
tests
that
kubernetes
had
identifying
some
of
them.
They
are
not
applicable
to
Windows
because
of
features
that
they
don't
support
and
then
reducing
the
list
to
the
things
that
we
really
do.
Support.
Adding
new
tests
to
test
our
new
functionality,
making
sure
that
that
has
grid
becomes
green
and
stays
green
throughout
the
release.
So
that
was
a
lot
of
work.
Yeah.
B
But
you
know
I
think
that
within
a
couple
releases,
we'll
get
to
a
point
where
the
kubernetes
conformance
logo
could
also
be
applied
to
two
windows.
We're
just
working
out
some
of
the
details
of
how
that
testing
is
actually
going
going
to
happen.
But
that's
you
know.
One
of
the
things
that's
been
really
fun
about
working
in
this
community
is
that
you
know
by
working
together
we're
building
stuff,
that's
better.
B
In
the
end,
and
so
like
a
lot
of
the
things
that
we
did
to
help
make
Windows
and
Linux
work
well
together
are
also
helping
pave
the
way
and
we're
also
joining
forces
with
the
people
that
are
doing
things
like
getting
stuff
running
on
arm,
64
handling,
multi
architecture,
images
and
thing
and
stuff
like
that.
That's
needed
to
test
all
these
new
scenarios
that
people
want
to
do,
and
so
it's
it's
been
a
really
fun
community
effort
from
that
website.
A
Absolutely
and
on
the
conformance
testing,
there's
a
open-source
tool
called
sonobuoy
that
kind
of
validates
conformance
across
clusters
or
if
someone
has
to
wants
to
have
their
own
private
distribution.
So
Nobu
is
what
they
used
to
kind
of
validate
that
so
we're
working
on
going
down
the
path
Wars
and
we
can
validate
Windows
clusters
with
our
conformance
profile
and
the
work
that
Patrick
mentioned
earlier.
B
B
Well,
so
probably
the
best
answer,
so
the
question
was:
where
does
test
greater
on
that?
The
best
answer
would
be
look
at
to
talk
that
Aaron,
Creek
and
Berger
did
cubic
on
us
in
December
because
they
had
a
whole
talk
about
the
kubernetes
test
infrastructure,
but
there's
kind
of
two
main
components
that
are
that
our
open
source
and
they're
used.
B
Actually
the
test
grade
is
not
open
source,
but
but
the
format
it
reads
from
is
and
so
like
prowl
is
what
handles
merging
all
the
PRS
and
scheduling
all
the
test
passes,
that's
completely
open
and
then
for
each
one
of
these
test
passes
that
we
have
running-
or
you
know
like
before,
like
the
1.14
like
the
one
on
master
branch
like
all
the
different
iterations
there.
Those
are
all
scheduled
by
prowl
and
then
the
results
get
uploaded
to
Tet
to
a
Google,
Storage
bucket
and
then
test
grid
parses
them
from
there.
B
A
The
actual
test,
like
at
least
for
Windows,
who
are
running
them
all
over
the
place,
so
we
have
a
vSphere
suite
that's
running
on
this
fear
on
Prem.
We
have
test
Suites
running
on
a
KS,
so
so
we
have
multiple
locations
who
are
actually
running
the
test
and
we're
getting
result
across
the
board
of.
We
also
have
a
DCP
test
suite
as
well,
so
the
actual
compute
environment
is
different,
but
they
all
get
pulled
together
and
they
are
displayed
in
tests.
Great
yeah.
B
The
other
thing
you
can
do
is
like
what's
going
on
vSphere
is,
you
can
also
just
run
the
same
test
suite
and
then
upload
the
results
to
a
storage
bucket
and
test
griddle
federated
amount
for
you
and
that
so
that
might
be
the
easiest
way
to
do
it.
If
you've
already
got
your
own
test
lab,
you
know
my
test
lab
has
I,
don't
know
50
data
centers
so
but.
A
A
Well,
thank
you,
Craig
I
love
that
do
you
raise
your
hand?
So
so
that's
that's
great.
So
you
know,
if
you
guys
have
tried
that
whole
process
I
mean
there's
some
difficulties
right.
You
know
you
have
to
follow
our
dogs
line
by
line.
Don't
make
a
mistake,
you
have
to
make
sure
you
set
up
if
you're
using
flannel
set
up
the
right.
You
know
start
up
the
flying
out
the
edge
and
make
sure
you
configure
everything
properly
give
it
the
IP
addresses
and
the
ranges.
So
all
of
that
stuff
is
a
little
bit
complicated.
A
So
we're
gonna
do
a
lot
of
things
to
make
that
easier
for
you,
you're
gonna,
we're
gonna
talk
about
cube
ATM
later
on.
You
know
as
a
premier,
provisioning
and
node
addition
tool
in
kubernetes
and
how
we
are
working
on
that,
but
we're
also
working
with
a
lot
of
vendors
that
are
providing
turnkey
solutions
that
it's
simply
like
select
what
you
want.
I
want
a
Windows
cluster
I
want
this.
A
Many
nodes
I
want
this
type
of
compute
profile
on
it,
and
you
wait
a
few
minutes
and
you
have
a
cluster
I've,
been
running
and
ready
for
you
to
deploy
your
application.
So
the
entire
end-to-end
process
is
significantly
simplified
and
they
offer
a
lot
of
other
things
like
services
and
support.
And
you
know
you
get
you
get
a
throat
that
you
can
choke.
If
you
get
an
issue
and
most
of
these
vendors,
you
know,
Patrick
and
I
do
come
from
one
of
each
of
these
vendors.
They
also
do
work
in
our
community.
A
They
have
some
of
them,
have
developers
that
are
working
with
us
and
that's
how
they
get
stay
engaged.
They
kind
of
make
requests
for
their
features,
and
we
put
them
to
work
as
well.
So
we
have
a
very
good
two-way
communication
with
all
of
them
kind
of
put
them
in
alphabetical
order
here,
so
nobody
gets
upset.
So
we
starting
with
Amazon
eks
dr.
enterprise,
Google
gke,
huawei
cloud
Microsoft,
IKS,
Rancher,
Hadouken
shift
and
VMware
enterprise
PKS
via
more
people,
dog
Enterprise
PKS.
A
So
so,
if
you
guys
are
interested
in
trying
them
out
when
you
do
get
the
gap,
deck
there's
links
to
the
signup
pages
for
every
one
of
them.
Some
of
these
are
private
betas.
We
can't
guarantee
that
you
will
get
in
and
you'll
get
access,
but
at
least
you
can
do
is
try.
So
if
you're
interested
in
any
of
these
follow
the
links
and
go
sign
up
so
before
we
get
into
the
the
future
plans.
A
I
do
want
to
give
a
shout
out
to
some
of
the
companies
that
really
did
put
a
lot
of
development
and
are
in
the
engineering
behind
cig
Windows
who
had
Microsoft
docker
VM
were
pivotal
Google
and
cloud-based
solutions.
Each
and
every
one
of
those
companies
added
developers,
resources
the
run
test.
They
basically
helped
us
fix,
bugs,
write,
documentation
and
a
big
shout
out
to
all
of
those
guys
that
helped
us
get
to
this
point,
a
friend
as
well
Prentis
defunct
now,
but
yeah
we'll
give
them
we'll
give
them
the
shoutout
all
right.
B
So
for
those
of
you
that
don't
know
yeah,
this
is
through
the
container
runtime
interfaces
CRI,
and
it
was
basically
designed
to
make
it
easy
for
the
runtimes
to
be
pluggable
underneath
when
kubernetes
started
off,
they
were
you
had
a
code
base
called
docker
sham
and
it
was
calling
the
docker
API
underneath
today
there's
two
other
implementations.
One
is
contained
well
there's
more
than
two,
but
the
two
that
a
lot
of
people
are
migrating
to
our
container
D
and
which
is
an
open-source
project
and
then
cryo,
which
was
started
by
right
happened.
B
You
know
they're
both
public
publicly
public
open
projects
and
what
and
part
of
the
reason
for
this
is
that
the
docker
API
doesn't
know
what
a
pot
is,
and
so
there
were
a
lot
of
workarounds
I
had
to.
You
know
had
to
be
done.
The
kind
of
Cobble
things
together
with
you
know,
Matt,
where
the
qubit
would
you
know,
do
things
like
set
up
C
groups
on
Linux.
B
You
know
copy
some
namespaces
in
when
they're
starting
containers,
and
there
were
a
lot
of
things
that
were
just
kind
of
hacky
to
make
it
all
work
and
CRI
cleans
a
lot
of
that
up.
The
other
thing
is,
it
gives
us
a
place
to
abstract
away
the
OS,
some
of
the
OS
specifics
underneath
and
so
that
way
the
implementations
can
differ
a
bit
more,
and
that
mean,
and
that
and
so
means
that
those
are
those
are
pluggable.
B
So
you
end
up
with
on
the
next
you've
also
got,
you
know
like
the
Cotter
runtime
can
be
implemented
through
CRI
and
under
container
D.
You
know
if
you
want
KVM
based
isolation
on
windows,
real
we
do
process
or
hyper-v
based
isolation,
but
the
important
thing
about
container
D
in
Windows
is
that
the
team
that's
actually
been
developing.
All
of
the
like
backend
kernel
code
to
make
containers
work
on
Windows
is
also
the
team.
B
So
it's
got
a
nice
common,
shared
layer
underneath
there,
but
when
you've
got
a
node
where
you're
not
going
to
be
doing,
builds
and
stuff
like
that,
and
all
you
want
is
secure
container
runtime
then
cry
container
D
fits
that
bill
very
well,
and
so
that's
what
we're
currently
working
on.
But
in
order
to
get
that
done,
we've
got
a
lot
of
hurdles
that
we
need
to
overcome.
B
B
We
could
start
actually
doing
some
of
the
new
development
around
on
the
windows
side
and
that's
where
the,
where
I'm
going
to
resume
development
on
on
hyper-v
because
container
D.
Let's
gives
fine-grained
control
over
the
stuff,
that's
needed
in
hyper-v,
and
so
when
we
want
to
do
things
like
start
building
backwards
version
compatibility.
B
If
you,
if
you
use
a
hypervisor,
you
can
actually
restrict
it
down
and
set
different
boundaries.
So
that
way
it
would
be
feasible
to
create
a
pod
where
it's
like
it
owns
these
cores.
Nothing
else
can
touch,
and-
and
so
you
know
when
you
want
to
do
defensive
and
defense-in-depth
will
be
able
to
start
designing
new
security
profiles
around
that
and
implementing
them
through,
like
a
runtime
class.
So
that's
kind
of
the
direction
we're
going
I'd
like
to
bet
that
the
base
layer
there
and
so
we've
got
this
outlined
in
a
cap.
B
A
Right
so
in
terms
of
115,
so
like
Patrick
mentioned,
CRI
container
D
is
going
to
be
over
a
multi
release.
It's
on
a
multi
release
plan,
but
there's
also
a
few
things
that
are
trying
to
target
for
115
to
make
your
experience
better
and
also
in
allow
us
to
better
integrate
with
partners
and
kind
of
bring
more
things
into
our
ecosystem.
A
So
so
CRI
container
D
by
the
way
is
done
within
with
collaboration
with
sick
node
and
other
members
of
the
community
and
Patrick
is,
and
his
team
are
really
working
with
all
of
them
to
make
that
a
reality.
So
cube.
Idiom
I
mentioned
that
earlier,
so
we're
going
to
try
to
get
initial
support
with
1.15.
That
will
allow
you
to
add
a
Windows
node
and
added
into
your
environment
and
control
of
difficulties.
A
There
there's
a
lot
of
assumptions
are
being
made
not
only
in
cubed
en
but
throughout
kubernetes,
that
there
are
components
that
are
built
and
run
and
execute
on
Linux,
and
there
are
difficulties
there
in
terms
of
identifying
them
and
resolving
them.
So
you
can
get
killed
every
DM
to
two
other
windows,
node
and
I.
Believe
the
woman
you
added
a
Windows
note
the
other
day
right
was
a
successful
or
it
works
so
logo,
mirin
and
a
few
other
folks
are
really
kind
of
driving
that
effort
from
us
laboum
is
from
C
cluster
life
cycles.
A
A
Identity,
so
I
spoke
about
group
managed
service
accounts
earlier
will
continue
advancing
that
we
did
make
some
API
changes
in
that
arena
and
making
fixes
and
stability
fixes,
but
we're
also
going
to
add
support
for
run
as
username,
so
that
basically
gives
you
the
ability
to
add
an
identity.
To
your
container,
of
course,
you'll
expect
more
CN
eyes
and
storage
plugins,
so
we're
working
both
in
the
open
source
world,
as
well
as
with
private
vendors
they're
having
a
CNI
solution
in
kubernetes,
so
they
can
put
those
two
windows
and
the
greater
effort.
A
A
One
of
the
things
who
kind
of
were
were
working
on
is
there
are
no
privileged
containers
in
Windows,
at
least
not
yet,
and
not
in
the
immediate
future,
and
part
of
that
effort
means
that
CSI
is
one
really
all
work
on
Windows
within
the
current
architecture,
so
we're
working
with
the
community
and
that's
the
talk,
that's
happening
next
doors
to
figure
out
what
should
we
do
about
storage?
What
are
our
options,
and
how
are
we
going
to
tackle
that?
A
So
if
any
of
you
guys
are
interested
in
coming
and
contributing
with
us,
you
know
just
like
the
greater
kubernetes
community,
there's
kind
of
three
stages
that
you
can
contribute.
You
can
start
as
an
end
user.
That
means
go
breeder
dogs.
Try
out
the
code,
give
us
feedback,
maybe
attend
community
meetings,
and
here
are
what
we're
doing
and
that
kind
of
the
first
layer
of
the
getting
started
in
kubernetes
by
the
way.
A
This
applies
to
all
six,
not
just
Windows,
then
become
a
contributor
go
find
maybe
a
bug
in
documentation
or
a
bug
in
the
code
and
and
get
it
fixed.
A
meter
PR
get
feedback
from
the
community
were
a
very
accepting
community.
We
kind
of
hop
new
folks
to
get
started
with
with
feedback
who
will
even
offer
some
pair
program.
If
anybody
wants
that,
but
then,
once
you
kind
of
fix
your
folks
bug
will
wreck
and
then
slowly
fix
the
second
and
the
third
and
the
fourth
bag,
or
pick
up
a
new
feature.
A
Then
you
really
become
a
contributor
to
our
community
and
then
from
then
on.
The
next
stage
is
to
kind
of
become
a
tech
lead.
You
know
maybe
identify
an
area
or
new
work
and
kind
of
take
ownership
of
that,
and
we've
had
quite
a
few
folks
step
up
in
in
the
recent
last
couple
of
releases
and
own
an
entire
area
by
themselves
that
kind
of
drive
that
to
resolution.
B
As
far
as
the
meetings
go,
you
know
if
you
join
the
sig
windows
mailing
list
group,
then
the
agenda
is
open.
So
if
there's
a
topic
you
want
to
present
on,
if
there's
just
your
even
questions
that
you
want
to
like
preload,
so
that
we're
we're
getting
ready
to
answer
and
feel
free
to
add
stuff
to
the
agenda,
because
we'd
love
to
hear
from
you
and
success
stories,
questions
failures,
look
I
mean
all
that
all
that's
good,
that's
yeah!
We
don't
want
to
be
just
us
talking
absolutely.
A
All
right,
so
we
do
have
weekly
meetings
like
clockwork,
12:30
Eastern,
all
right.
We
realize
that
that's
not
very
friendly
to
asia-pacific
or
European
time
zones.
Our
European
folks
seem
to
be
managing
okay,
but
obviously,
if
we
start
expanding,
then
it
seems
that
there's
a
need
for
additional
time
zones.
Patrick
and
I
are
open
to
that
and
we
can
add
a
second
meaning.
We
do
record
every
single
meeting,
I
think
we're
one
of
the
most
consistent
SIG's.
A
We
post
everything
on
our
YouTube
playlist,
so
you'll
find
meetings
going
back
like
18
months
and
we
do
have
a
project
board.
That's
linked
here.
That
kind
of
has
swim
lanes
for
every
single
release
and
what
we're
planning
to
do.
So,
if
you're,
if
you
want
to
see,
what's
gonna,
come
neck
in
Windows,
you
can
read
our
caps
come
and
look
at
our
project
board
and
you
see
what
we're
fixing
both
bugs
as
well
as
features
and
obviously
help
us
write
additional
documentation
and
user
stories.
A
If,
if
that's
your
forte
now,
how
can
you
find
us?
Patrick
mentioned
the
mailing
list.
So
if
you
wanna
email
us
we're
up,
there
were
on
slack
under
sick
Windows
Channel,
my
alias
is
M.
Patrick's
is
Patrick
Lang,
so
you
wanna
privately
messages.
You
can
do
that
as
well.
We
have
our
YouTube
playlist
on
the
right,
so
if
you're
interested
in
kind
of
viewing
all
our
videos,
it's
right
there
underneath
that
we
have
our
entire
documentation.
A
So
if
you
go
to
that,
URL
you'll
find
all
the
dogs
on
windows
from
getting
started
to
adding
you
note
to
configuring,
that's
our
zoom
link.
So
all
our
meetings
are
following
that.
Obviously,
if
you
sign
up
you'll
see
the
invite,
so
you
don't
really
need
to
memorize
that
and
then
we're
on
github
you'll.
Under
that
link.
We
have
our
getting
started
guide
that
has
additional
things
like
hey
I
want
to
become
a
contributor.
How
do
I
get
started
do
have
a
contributing
guide,
both
in
kubernetes
and
especially
a
specific
one
for
Windows.
A
A
B
Yeah
and
probably
the
slacker
channels,
probably
the
most
responsive,
because
we've
got
a
lot
of
people
that
are
involved
in
there
and
particularly
for
recent
issues.
It's
probably
already
been
discussed
there.
You're
like
we
had
one
going
back
and
forth
with
relating
to
a
network
problem.
The
way
of
you
know,
but
just
yesterday
we're
like
okay,
here's,
here's
a
patch,
that's
there,
and
so
people
have
been
testing
and
have
been
confirming
on
that
thread.
So
that's
that's
pretty
much.
The
easiest
actually.
A
The
other
thing
is
within
the
spirit
of
the
kubernetes
community
and
how
great
is
to
work
on
kubernetes.
Initially,
when
we
started,
we've
had
a
couple
of
public
cloud
provider,
vendors,
the
had
a
solution
working
with
Windows
and
when
others
came
in
and
they
were
asking
questions,
who've
seen
one
cloud
provider
hub
the
other,
which
was
amazing
to
see
that
they
will
I
mean
the
problems
and
the
automation
that
they
need
to
make
is
very
similar
and
they
were
helping
each
other,
even
though
they
have
competing
solutions.
And
that
was
great
to
see.
F
A
I'm
sorry
Craig
I
forgot
to
add
that
to
the
deck
that
we
do,
need
that
and
actually
one
of
the
pin
conversations
there
is:
what
scenarios
do
you
want
to
use
windows
for
so
any
of
you
are
thinking,
hey
do
I,
need
this
window
stuff
or
not
go
read
the
discussion
forum.
You
see
what
other
folks
are
using
this
forum.
C
B
A
General
practice
there
is
not
to
grab
the
Windows
Server
2008
OS
and
run
it
in
hyper-v
and
because
there
always
is
still
unsupported
right.
So
also
that's
the
the
general
problem,
but
what
we
are
recommending
and
seeing
is
a
lot
of
customers
are
either
taking
a
lift
and
shift
approach
where
they're
taking
that
workload,
maybe
it's
iis
base
or
asp.net
or
WCF
based
and
they're,
shifting
it
and
putting
it
inside
a
container
setting
up
the
same
policies
and
configuration
that's
needed.
A
Maybe
sending
up
a
group
managed
service
account
so
that
the
workload
has
an
identity
in
the
network
and
that's
how
their
container
izing
it.
So
it's
a
lift
and
shift
approach
or
modernization
if
someone
wants
to
react,
attacked
it
a
little
bit,
but
at
the
very
least
that
approach
works
very
well,
because
you
have
the
same
changes.
Dotnet
is
backwards
compatible
in
a
lot
of
cases
that
lift
and
shift
is
not
a
tremendous
amount
of
work
and
in
some
cases
you
might
be
able
to
do
this
in
a
couple
of
hours.
B
It's
like
the
application
I
have
here
was
based
on
asp.net
and
let
me
just
find
a
docker
file,
real
quick,
and
so
this
one
was
actually
built
using.
So
this
is
an
application
that
was
running
on
dotnet
four
point:
seven
point:
two
and
and
Microsoft
has
a
pre
really
a
prebuilt
image
for
asp.net,
and
then
this
is
actually
that
what
I
mean
by
the
durable
version
tag.
B
It's
the
webservice
rude
and
configure
the
app
pool
and
then
copy
the
binaries
in
there,
and
so
this
build
could
still
be
done
under
visual
studio.
I'm
not
going
to
go
through
the
whole
build
process
right
now,
but
you
can
basically
build
and
then
use
tools
in
there
to
actually
help
you
containerize
that
application
and
so
like.
B
Look
like
this,
but
basically
I've
got
the
container
image
that
one
happens
to
be
a
Windows
image,
but
the
couple
of
things
I
want
to
call
out
from
the
slides
were
making
sure
that
you've
got
the
node,
selector
and
then
you've
got.
You
know
the
resource
request.
Some
limits
set
high
enough
for
the
application
to
actually
run,
and
so
this
is
up
on
up
on
my
github
repo
and.
B
Let
me
see
if
I've
got
and
I
actually
presented
like
a
an
hour,
long
version
of
this,
where
we
went
step
by
step
so
that
you
could
try
it
out.
That
was
recorded
at
cube
con
back
in
december
in
seattle,
but
all
the
resources
from
that
are
all
available
at
this
at
this
repo
here.
So,
if
you're
looking
for
one
place
to
get
to
get
the
sample
apps
as
well
as
the
step
by
steps
for
that
check,
this
one
out
and
then
went
down
here,
this
is
the
lift
and
shift
this.
B
You
can
also,
since
it's
still
using
just
a
regular
Bernays
api's,
then
you
can
go
ahead
and
use
deployment
tools
like
like
Helmand
draft.
You
know,
there's
probably
others
that
work
as
well,
but
those
are
the
ones
I
use
the
most
to
actually,
you
know
build
us
a
stronger
template
around
the
application
that
you
want
deploy.
B
B
One
way
that
you
could
do
things
in
a
production
is
to
port
it
away
rather
than
using
a
developer
edition,
so
that
sort
of
you
know
drives
how
some
of
these
things
are
going
to
be
deployed,
and
then
it
makes
sure
that
you
know
the
things
I
specified
like
the
image
name
and
the
image
pull
secrets
from
that
values.
File
just
get
passed
into
these
templates,
and
so
it
makes
it
really
really
easy
for
you
to
do
repeatable
deployments
and
so
once
you've
actually
deploy.
This
I've
already
got
one
deployed
here.
B
And
it
helm
gives
you
you
know
one
nice
place
for
that
for
that
application
deployment,
where
you
can
see
all
the
resources
I
will
create
it
as
a
result,
both
of
those
are
28
days,
not
28
hours.
Yes,
I
deployed
this
yesterday,
but
we've
got
the
sequel
instance
and
the
and
the
web
service
running
there,
and
then
we
get
the
external
IP.
B
And
then
you
know,
we've
already
got
the
service
up
and
running,
and
so
I
didn't.
You
know,
change
any
other
code.
That's
there.
All
I
did
was
do
a
little
bit
of
tooling
to
package
that
up
as
a
container
and
then
I
was
using.
You
know
standard
kubernetes
tools.
You
know
like
like
helm
and
Cube
control
that
actually
check
that
stuff
out
and
if
I
want
to
go
actually
look
at
what's
inside
here
you
can
use
the
state
regular
commands,
like
you
know,
cube
control.
B
B
B
Okay,
and
so
this
is
running
inside
the
inside
the
container,
so
if
I
want
to
go,
you
know,
look
at
things
like
you
know
the
config
on
that's
been
there
or
I'm,
not
sure.
What's
in
there
that
you
can
do
that,
but
also,
if
I
want
to
go
to
look
at
look
at
the
logs,
you
can
do
all
that
stuff
interactively
it's
just
the
same
way
that
you
do.
You
know
cube
control
exact
into
a
Linux
container.
You
could
do
it
into
a
Windows
container
and
run
yourself
choice.
B
So
all
the
common
tools
would
work
and
in
the
case
of
you
know,
Cube
control
helm
most.
Those
have
Windows
builds
as
well
right
now,
I'm
actually
running
this
out
of
remote
Linux
node,
because
that's
my
easiest
way
to
deal
with
conference
Wi-Fi
that
way,
if
I'm,
in
the
middle
of
doing
something
and
it
dies
well,
T
MUX
keeps
it
alive
and
I
just
reconnect,
but
so
that's
got.
You
know
just
a
little
bit
a
little
bit
of
the
workflow
that's
available
there.
A
Alright,
we
might
end
a
little
bit
early
them
today.
Well,
thank
you
all
for
attending
Patrick
and
I
will
probably
stay
here
a
few
more
minutes.
If
anybody
wants
to
talk
to
us
in
private,
please,
please
do
so,
but
yeah
check
out
the
slides.
The
ways
to
get
involved
would
love
to
hear
your
feedback.
Would
love
to
you
guys
to
try
this
out
and
we're
gonna
continue
investing
in
this
area
we'll
make
this
better
and
better
with
every
release
of
communities.
Thank
you.