►
From YouTube: What's New With SIG Windows and Deep Dive... Mark Rossetti & Brandon Smith, Jay Vyas, Claudiu Belu
Description
Don’t miss out! Join us at our upcoming hybrid event: KubeCon + CloudNativeCon North America 2022 from October 24-28 in Detroit (and online!). Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
What's New With SIG Windows and Deep Dive Into Windows Container Users - Mark Rossetti & Brandon Smith, Microsoft; Jay Vyas, VMware; Claudiu Belu, Cloudbase Solutions
Running Kubernetes on Windows is increasingly a viable production strategy for complex applications in multitenant environments. In this presentation we'll highlight recent improvements - such as the pod.OS field and advancements in host-process containers for infrstractuure - that make it easier to manage production clusters/workloads, show people how to rapidly prototype the development of new Kubernetes features using the SIG-Windows developer tools project, and also do a deep-dive into how container users work on Windows.
A
A
B
C
A
A
First,
one
is
what
we're
just
calling
v2,
but
that
really
is
just
a
way
to
use
gmsa
identities
in
containers
without
requiring
that
the
host
be
domain
joined
through
the
use
of
a
container
credential
guard
plugin.
This
makes
it
much
easier
to
manage
your
windows
nodes,
because
you
don't
have
to
worry
about
joining
them
to
the
active
directory
domain
before
joining
them
to
your
kubernetes
cluster.
A
We
have
support
for
this
in
the
admission
web
hook.
That's
part
of
the
windows
gmsa
project
linked
here
and
support
was
added
in
version
0.3.0.
We
also
have
a
reference
plugin
implementation
available
at
on
github.
This
reference
plugin
is
using
an
azure
key
vault
to
store
the
credentials
for
doing
the
domain
off
on
behalf
or
in
place
of
the
having
the
node
be
part
of
the
active
directory
domain
and
then
there's
more
information
about
how
all
of
this
works
under
the
hood.
A
At
the
link
provided
below
the
other
big
update,
is
we
now
have
a
helm
chart
used,
that's
used
to
deploy
the
all
of
the
gmsa
work
that
you
can
find
in
the
same
windows.
Gmsa
repository.
Please
check
that
out
if
you're
interested
the
next
update
is
this
pod
os
field.
I
think
we've
mentioned
it
at
one
of
the
last
cube
contacts
now
this
enhancement
has
been
promoted
to
beta.
A
If
your
windows
containers
accidentally
land
on
linux
nodes,
previously,
you
used
to
see
some
pretty
I'd
say
like
convoluted
error
messages,
just
saying
that
your
image
failed
to
unpack
and
we
usually,
this
is
one
of
the
first
issues
that
people
who
are
first
experimenting
with
windows,
containers
and
kubernetes
cluster
run
into
it's
really,
not
obvious.
What's
going
on
there,
but
now
you
get
this
nice
error.
That
just
says
you
know
the
pod
failed,
because
you
meant
to
schedule
a
windows
container,
but
it
landed
on
a
note
of
a
different
operating
system.
A
There
are
a
lot
of
other
improvements
or
functionality
that
happens
at
api
validation
too,
if
you're
interested
in
that
check
out
the
the
kubernetes
website
for
more
information
and
full
docs.
On
that
last
update,
I
wanted
to
give
was
on
this
operational
readiness
specification
for
windows
nodes
or
what
were
kind
of
what
we're
calling
windows
conformance
today,
there's
no
good
way
to
validate
that.
Your
windows,
node
is
set
up
correctly
when
it's
joined
into
or
one
is
joined
into
a
kubernetes
cluster,
and
this
solves
that.
A
So
what
we've
done
here
is
we've
wrote
a
bunch
of
tests
that
help
validate
the
configuration
of
the
windows
nodes
and
bundled
them
up
in
a
sono
buoy
plugin
for
easy
to
make
it
easy
to
run
this.
That
tests
just
things
like
making
sure
your
network,
like
your
network
rules
and
everything,
are
set
up
correctly,
making
sure
you
defender
has
the
correct
settings
all
of
that
kind
of
node
level
configuration
work
that
isn't
tested
by
any
of
the
kubernetes
ede
tests.
There's
some
links
in
a
video
with
more
information
here,
too.
A
B
Yeah,
so
one
of
the
most
common
questions
we
get
from
the
community
is
how
user
accounts
work
in
windows
server
containers.
So
I'm
going
to
spend
some
time
today
to
provide
an
overview
of
how
all
this
works.
So
to
start,
we
need
to
first
get
a
grasp
on
how
windows
server
containers
are
built.
In
the
first
place.
Remember,
windows,
server,
containers,
sandbox
the
entire
user
mode
portion
of
the
os,
because
they
share
the
host
kernel.
B
They
don't
need
to
run
through
the
kernel
init
process
on
boot
after
configuring,
the
ob
name,
space
file
system
and
registry.
An
entirely
new
instance
of
the
session
management
subsystem
or
smss
is
created.
Smsf
manages
the
startup
processes
of
all
the
user
sessions
in
windows
performing
the
same
actions
within
the
container
as
it
would
on
the
host
next
slide.
B
So
why
do
I
mention
this?
Well,
when
talking
about
user
accounts
in
containers,
it's
important
to
understand
the
isolation
properties
of
containers,
they're
running
it,
unlike
virtual
machines
that
virtualize
at
the
hardware
level,
containers,
virtualize
access
to
various
namespaces
and
a
namespace,
provides
access
to
information
objects
or
resources
via
a
name.
The
security
of
a
windows
container
is
derived
from
the
degree
of
sharing
that
occurs
with
its
host.
B
The
security
boundary
of
a
container
is
constituted
by
the
isolation
mechanisms
that
separate
the
container
from
said
host
and,
most
importantly,
these
isolation
mechanisms
define
which
processes
in
the
container
can
interact
with
the
host.
So
looking
at
this
chart
here,
we
could
see
where,
on
the
isolation
spectrum,
the
different
container
types
lie.
While
the
design
of
windows
server
containers
differ
vastly
from
linux
containers,
they
offer
similar
degrees
of
security
and
isolation
guarantees.
B
So
you
can
see
here
on
the
left
there.
We
have
just
regular
processes.
You
know
on
your
on
your
system.
There
have
direct
access
to
the
host
and
directly
work
with
you
know
the
scheduler
and
the
os
and
have
access
to
all
the
information
that
are
only
limited
by
user
accounts,
whereas
on
the
other
side
we
have
hardware
based
virtualization,
which
totally
separates
a
you
know,
a
kernel
user
space
and
everything
else
from
the
original
host
next
slide.
B
So
windows,
server,
containers
by
default,
offer
two
user
accounts,
a
container
user
and
container
administrator
each
with
their
own
specific
purpose.
These
user
accounts
are
only
visible
to
the
container.
The
host
does
not
have
any
awareness
of
the
container
user
accounts
as
they're
managed
using
a
virtualized
sam
database
within
the
container
container.
Accounts
can
be
created
while
running
or
during
build
time
and
be
stored
within
the
sam
database
before
deployment,
so
because
each
account
exists
purely
within
the
container.
The
host
is
unaware
of
user
accounts
that
exist
within
each
container.
B
The
alternative
is
true
as
well.
Containers
have
no
visibility
into
the
user
accounts
available
on
the
host.
This
high
degree
of
isolation
is
one
of
the
benefits
of
being
able
to
run
an
entire
copy
of
the
user
mode
os
in
a
dedicated
space.
The
caveat
here
is
that
if
the
host
ever
shares
resources
with
one
or
more
containers,
it's
possible
to
hit
conflicts
between
user
and
group
s
ids
mounting
volumes
with
specific
container
user
access
rights
is
not
possible.
B
Under
this
model,
all
container
accounts
will
be
able
to
see
what
their
well-known
group
of
group
membership
allows
them
to
see.
So
here
we
kind
of
have
like
you
know
what
I
would
call
a
sphere
of
influence
where
the
container
administrator
doesn't
have
access
to
pretty
much
any
of
the
resources
on
the
host
unless
we
explicitly
allow
it
and
both.
B
But
the
the
interesting
thing
here
which
kind
of
refers
to
the
the
architecture
and
design
of
windows
versus
linux
is
both
container
administrator
and
and
administrator
are
members
of
the
same
administrators
group
because
they're
built
in
a
way
that
they
assume
isolation
from
one
another.
So
the
administrator
is
part
of
the
built-in
administrators
group
that
exists
on
the
host
is
the
same
sid
as
the
built-in
built-in
administrators
or
the
container
administrator.
But
it's
all
relevant
to
the
scope
that
they're,
in
which
they're
you
know,
have
access
to
the
next
slide.
B
So
let's
go
dive
a
little
bit
into
each
specific
account,
so
the
container
administrator
account
enables
you
to
use
the
container
for
administrative
purposes,
installing
services
and
software,
such
as
enabling
iis
within
a
container
making
configuration
changes
and
creating
new
accounts.
These
tasks
are
important
for
several.
It
scenarios
that
may
be
running
in
custom,
on-premise
deployment
environments
by
default
container
administrator
does
not
have
the
ability
to
interact
with
the
host.
Any
direct
interaction
must
be
first
granted
explicitly
by
an
account
of
higher
privilege
on
the
host.
B
Despite
its
name,
a
container
administrator
has
very
few
privileges
enabled
by
the
default,
as
we
can
see
here,
as
the
account
does
have
several
powerful
privilege
granted
to
it,
though.
So
this
emphasizes
the
need
to
one
limit
usage
of
the
the
account
to
only
trusted
trusted
workloads
and
two
ensure
trust
in
the
source.
The
container
image,
the
container
administrator
account,
can
install
services
and
perform
administrative
functions
within
the
container
that
persist
across
the
build
and
deployment
cycle.
B
Even
if
you
enforce
a
policy
in
production
to
run
workloads
under
low
privileged
accounts,
untrusted
container
images
can
use
the
container
administrator
account
to
install
administrative
services
that
can
still
run
with
high
privileges
on
booth
granted.
The
impact
of
these
services
will
be
limited
to
within
the
container
workload
unless
access
to
the
host
was
explicitly
given
and
the
next
one,
the
container
user
account
exists
for
all
of
the
scenarios
where
administrative
privileges
and
windows
are
not
needed,
for
example,
in
kubernetes.
B
If
the
user
is
container
administrator
and
the
host
volumes
are
permitted
to
be
mounted
into
the
pod,
then
the
user
could
mount
the
dot,
ssh
folder
and
add
a
public
key
as
container
user.
This
example
would
not
be
possible.
It
is
a
restricted
user
account
designed
purely
for
applications,
but
you
do
not
need
to
interact
with
the
host.
B
B
B
C
Hey
everybody
so
yeah
kpng
coupe
proxy
next
gen.
So
this
was
a
project
originally
originally
created
by
mikhail
cluseau,
and
we
over
at
sig
network
and
sig
windows,
have
sort
of
found
that
it's
solving
a
lot
of
potential
long-term,
well,
not
potential
a
lot
of
very
real
long-term
problems.
We
have
both
for
windows
and
for
the
broader
sig
network
community.
So
the
idea
my
beautiful
myro
board
diagram
over
here
is
the
you've
got
the
coup
proxy.
It
breaks
the
coupe
proxy
into
two
separate
pieces.
C
One
part:
that's
like
a
brain
that
talks
to
the
kubernetes
api
and
that's
the
coupe
to
store
thing
right
there
on
the
over
on
the
left
in
the
top,
and
then
it
can
also
sort
of
and
and
as
it
takes
in
information
from
the
kubernetes
api.
It
then
can
feed
that
over
grpc
to
a
backend
sync
which,
in
in
the
windows
scenario,
is
you
know
a
a
golang
program
that
that
writes
hns
load,
balancing
rules.
C
I
mean
that's
how
the
kernel
space
windows,
proxy
or
works
now,
or
a
user
space
proxy
that
does
like
netsh
rules
and
things
like
that
or
whatever
you
want
to
do,
and
so
the
kp
g
windows
work
is
now
underway
and
we
do
have
we've
got
it
about
80
or
about
70
percent
ported
over
to
run
in
kpng
in
this
new
architecture.
So
the
windows
kernel
space
work.
C
Once
we
get
kernel
space
windows
proxy
working
in
kpng,
we
believe
that
we
will
have
a
new
coupe
proxy
that
is
totally
modular,
that
we
can
use
for
a
lot
of
different
use
cases
and
even
entirely
replace
the
cube
proxy
entry,
which
is
the
long-term
goal
of
this
project.
Next
slide,
please
yeah!
So
if
you
wanted
to
test
it,
I
have
a
blog
post.
C
This
is
an
example
from
calico
right
for
calco,
ultimately
calls
this
hns
psm
sort
of
module,
and
that
creates
a
a
an
hns
network
like
sort
of
manually.
As
you
can
see
in
a
firewall
rule
and
then
after
that's
created,
then
then
the
coupe
proxy
can
get
to
work
and
start
using.
Those
network
constructs
to
write
its
rules,
but
so
one
of
the
things
we're
talking
about
doing
possibly
is
making
the
coup
proxy
for
windows.
C
Maybe
a
little
smarter
so
that
you
don't
need
to
do
any
initial
bootstrapping
logic,
and
maybe
it
can
just
write
those
rules
for
you
initially
before
startup,
so
that
it
can
just
kind
of
run
standalone
without
having
to
depend
on
any
cni
or
other
external
scripts
to
bootstrap
it.
So
that's
an
idea
we're
playing
with.
If
folks
have
ideas,
please
reach
out
to
us
on
that
and
yeah
next
slide.
Please,
oh
yep!
That's
that's!
I
think
you
mark
or
claudio.
D
A
D
D
Next,
we
are
going
to
look
into
host
process
containers
and
how
to
improve
its
usability.
First
of
all,
we
are
going
to
add
more
options
for
runners
users.
At
the
moment,
you
only
have
anti-authority
local
service,
anti-authority
system
and
anti-authority
network
service
as
available
users
that
you
can
use
for
host
process
containers,
add
more
fine-grained
scopes
for
new
potential
users
that
you
can
use.
D
D
Hinder
its
usage,
the
idea
is
to
potentially
have
a
union
of
host
os
and
container
file
systems
and
to
have
the
pilot
payload
of
the
container
in
a
special
folder,
but
only
visible
to
the
container
itself
ctl
log
logs
work
at
the
moment.
If
you
actually
want
to
inspect
the
logs
from
all
your
deployed
nodes,
you
would
actually
have
to
ssh
into
those
nodes
and
inspect
the
logs
manually.
D
This
is
going
to
improve
how
users
can
debug
potential
issues
whenever
a
pod
or
container
fails
to
start
on
windows
nodes.
In
that
case,
you
can
simply
get
the
necessary
logs
and
inspect
them
and
address
the
issues.
If
there
are
any
next,
we
are
going
to
look
into
performance
tests
for
windows
nodes.
D
The
idea
is
to
use
primitives
metrics
under
windows,
not
exporter,
to
export
a
couple
of
metrics,
which
will
be
useful
to
detect
those
issues.
Some
metrics
like
memory
usage,
no
storage,
open
file,
handles
or
processes,
and
so
on
and
finally,
one
of
the
things
that
we
are
going
to
look
into
is
file
acl
improvements.
D
Next,
we
are
going
to
give
our
special
thanks
to
our
community
members
and
to
new
contributors
for
david
schott
for
all
the
help
related
to
windows,
network
internals,
arvind
put
the
putiyaram
bill.
Sorry,
if
I
butchered
the
names
for
all
the
help
with
the
docs
bug,
triage
code
contributions
and
more
jamie
phillips
for
contributing
for
the
gmsa
webhook
and
including
the
web,
the
helm,
charm,
helm,
charts,
ravi,
gumi,
gudi
medla
for
quote
contribution
contributions
and
running
our
weekly
ci
signal,
triage
meetings.
D
If
you
want
to
learn
how
to
contribute
to
windows
and
see
windows
in
kubernetes,
please
visit
our
c
windows
committee
page.
Follow
the
contributor
guide
join
our
weekly
meetings,
which
is
every
tuesday
at
12,
30
eastern
time,
and
join
our
sig
windows,
pairing
sessions,
which
are
after
the
main
meeting.
D
D
Next,
we
have
here
the
community
resources.
We
can
contact
the
that
see,
windows
chair
or
the
technical
leads,
which
is
myself
jay
and
james.
You
can
also
join
us
on
slack
on
seek
windows.
You
can
group
our
google
group
and
you
can
read
the
documentation
for
deploying
windows
and
having
production
environments
for
windows.
D
You
can
contribute
to
kubernetes
itself
under
the
windows
label
and
you
can
also
check
out
the
kubernetes
community
seek
windows
repository.
You
can
also
check
the
youtube
playlist
in
which
we
have
all
our
sig
windows
meetings
and
you
can
join
our
weekly
zoom
meeting
at
the
following
url
next
slide.