►
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Deep Dive into Cloud Provider for Azure - Andy Zhang & Ernest Wong, Microsoft
In this session, we'll dig into the implementation of the Azure cloud provider, and current work to enhance the operations of Kubernetes. Updates for Kubernetes v1.18 and designs for the v1.19 versions will be discussed. Demos of the newest capabilities will make this. concrete. You will also learn how the maintainers set up their development environments so you can contribute easily too.
https://sched.co/Zew3
A
A
Cool
so
in
this
presentation,
we're
going
to
talk
about
our
mission,
we're
going
to
dig
into
the
implementation
of
the
cloud
provider
azure.
Some
of
our
recent
developments
for
kubernetes
118
and
119
release
how
we
are
validating
kubernetes
on
azure
and
we're
also
going
to
show
you
guys
a
small
demo
on
azure
file
based
nfs
service.
A
So
our
mission
really
is
just
to
make
azure
the
best
place
where
you
can
run
your
kubernetes
workload
over
the
past
half
year,
or
so.
We've
invested
a
lot
to
ensure
that
our
customers
have
the
smooth
experience
when
running
their
kubernetes
workload
on
azure
and
one
of
the
work
items
I
want
to
highlight
is
that
we've
taken
steps
to
improve
the
test
coverage
and
prevent
regression
introduced
to
our
end
users.
A
In
addition
to
that,
as
more
and
more
customers
are
on
boarding
their
windows
workload
to
kubernetes,
we
are
also
planning
to
add
pull
request
jobs
that
runs
end-to-end
validations
on
azure
windows
clusters.
So,
overall,
we
will
continue
contributing
to
kubernetes
transparently,
while
ensuring
that
our
existing
and
potential
customers
can
run
kubernetes
workload
smoothly
on
azure
infrastructure.
A
A
First
of
all,
we
want
to
have
a
smaller
codebase,
which
will,
which
is
going
to
be
much
easier
for
the
kubernetes
maintainers
and
the
developers
from
the
cloud
providers
doing
t
if
you
think
about
it.
If
there
are
100
cloud
providers
for
kubernetes,
it's
not
really
scalable
to
have
100
cloud
provider
implementation
living
within
the
kubernetes
repository.
A
So
what
kubernetes
did
is
that
it
creates
a
cloud
provider
interface
that
each
cloud
provider
must
implement.
So
this
way,
only
the
necessary
logic
is
embedded
within
the
kubernetes
code
base
and
all
the
external
cloud
dependent
logics
are
implemented
somewhere
else
and,
second
of
all,
with
separate
projects,
we
can
have
release
cycle
independence
to
kubernetes,
so
it's
beneficial
when
we
want
to
have.
We
want
to
release
a
critical
bug
fix
within
the
cloud
controller
manager,
and
we
can
release
it
quickly
within
a
reasonable
time
frame.
A
A
So
here's
a
very
simplified
architectural
diagram
of
how
kubernetes
works
with
cloud
provider
azure
on
the
top.
You
have
the
component
relationship
between
the
cloud
provider
and
the
cubelet
and
controller
manager
and
for
the
cubelet
you
can
think
of
it
as
a
node
agent
that
exists
in
each
of
your
master
and
agent
nodes.
A
A
Before
csi
cloud
provider
like
us
will
have
to
implement
individual
storage
plugins
within
kubernetes,
but
now,
instead
of
developing
individual
source
plugins
within
the
core
kubernetes
repository,
you
can
now
instead
implement
the
csi
drivers
specific
to
your
source
plugin.
So
now,
currently,
we
have
two
main
csr
drivers
that
we
maintain
actively.
They
are
the
azure
disk
csi
driver,
as
well
as
the
azure
file
csi
tracker.
A
Last
but
not
least,
there
are
there
is
the
external
credential
provider,
as
I
mentioned
earlier,
kubelet
is
responsible
for
fetching
tokens
to
pull
images
from
a
private
registry,
and
we
realize
that
different
cloud
provider
has
their
own
implementation
implementation
of
how
they
achieve
it.
So
now
that
we
have
an
external
credential
provider
interface,
a
cloud
provider
like
azure
can
now
implement
it
and
extract
the
logic
in
a
separate
project.
A
So
now,
with
this
external
credential
providers,
we
can
fetch
tokens
from
azure
active
directory
so
that
users
don't
have
to
provide
credentials
when
pulling
images
from
azure
container
registry.
So,
overall
the
goal
of
moving
all
these
things
out.
The
tree
is
to
make
the
development
process
more
friendly
and
avoid
redundant
logic
within
the
core.
Kubernetes
repository.
A
So
currently
the
extracted
logic
for
cloud
control
manager
and
cloud
node
manager-
they
all
live
within
the
kubernetes
6
orc,
more
specifically
they're
living
inside
the
cloud
provider
azure
projects.
Currently
the
status
is
in
beta
and
we're
looking
to
move
to
ga
soon
and
for
the
two
main
csi
drivers
for
azure
csi
drivers.
A
We
provide
a
couple
ways
to
provision
self-managed
clusters
on
top
of
azure
infrastructures.
First,
we
have
the
cluster
api
provided
azure,
which
is
basically
an
implementation
of
cluster
api
for
azure.
We
also
have
we
also
offer
aks
engine
where
you
can
prohibition,
self-managed
kubernetes
clusters
on
azure
with
arm
templates.
A
And
if
you
want
to
mount
any
secrets
from
azure
key
vault
to
your
workload,
you
can
try
out
secret
store,
csi
drivers
and,
last
but
not
least,
if
you
want
to
mount
a
blob
storage
as
a
file
system
to
your
pods,
you
can
try
out
fluffy
css
drivers,
and
these
are
just
some
of
the
projects
that
we
maintain.
And
if
you
want
to
check
out
more
kubernetes
related
projects,
you
can
always
check
out
the
kubernetes
organization
on
github,
as
well
as
the
azure
org.
B
Andy
yeah:
now
we
are
going
to
do
a
quick
go
through
of
all
the
major
bug,
fixes
and
features
in
1.18,
so
in
1.18,
in
idfi
and
disk
entry
drivers,
we
have
fixed
a
few
boxes
like
the
dangling
attack
issue,
which
would
cause
api
soldering
also
for
the
id5
driver.
We
use
the
new
mount
library
that
prevents
blocking
of
sensitive
bounce
options
and
we
have
also
introduced
used
force
detached
for
azure
disk.
B
I
will
introduce
this
feature
later
and
also
we
have
added
the
audio
disk
write
accelerator
support
for
the
specific
azure
vm
skills
on
the
network
side.
We
have
also
fixed
a
few
issues
like
a
concurrency
issue
in
load
balance
equation,
and
we
also
have
ideal
support
using
azure
network
resources,
independent
ad
tenant
and
subscription,
and
we
also
have
added
support
for
a
single
stack
ipv6
and
in
credential
provider
side.
We
have
also
added
a
support
for
private
as
a
cloud
so
nest
in
1.19
for
the
azure
desk
and
the
file
entry
drivers.
B
We
have
fixed
some
issues
in
the
id5
csi
translation
and
we
plan
to
go
better
for
the
edge
file
in
three
driver
migration.
Feature
calculator
in
1.20
and
we
also
have
added
a
shared
disk
support
on
load
block
and
the
ship.
The
shared
disk
could
make
user
to
attach
one
disk
to
multiple
engine
nodes,
and
we
have
also
added
a
text
support
for
added
scan
the
file
drivers
and
for
the
azure
disk
in
true
java
to
csi
java
migration.
B
This
feature
just
goes
better
in
1.19
and
in
the
network
side
we
have
also
has
fixed
some
issues
like
public
eye
connection,
after
designing
product
ip2
azure
vms
and
for
the
credential
provider
side.
We
have
also
fixed
it
back
in
poor
image,
which
could
which
could
cause
pre-image
error
using
azure
manager
as
a
manager,
identity,
okay.
B
B
We
have
worked
out
two
methods
to
mitigate
this
issue.
First,
from
community
side,
we
were
the
detached
disk
using
dangling
error,
so
this
could
be
capturing
attached
stage
on
another.
One.
Even
party
is
removed
in
certain
condition.
So
with
this
feature,
one
part
is
moved
to
node
two,
either
disk
driver
with
a
trigger
dangling
error
and
after
a
while
disk
would
be
detached
from
node
1
and
the
feature
is
available
in
a
few
kubernetes
versions
and
the
second
from
azure
cloud
provider
side.
B
We
have
already
switched
to
use
to
be
detached,
setting
to
detach
an
azure
disk
that
could
force
detach
a
disk
in
the
future
and
sometimes
detach
disk
may
fail
in
the
middle,
which
would
turn
vm
into
failed
stage
used
to
be
detached.
Seating
to
detach
a
disc
couldn't
make
vm
continue
detached
if
former
detached
operation
failed
set
would
also
solve
the
detect
issue.
So
these
are
the
two
methods
to
solve
the
this
disk
detach
issue.
A
All
right,
thanks
andy!
So,
as
I
mentioned
earlier,
we
have
invested
a
lot
in
the
testing
infrastructures
and
here
I'm
going
to
really
quickly
go
through
how
we're
testing
kubernetes
on
azure
infrastructures.
I
want
to
talk
about
the
test
infrastructures
and
we
actually
utilized
our
test
infra
from
the
main
kubernetes
repo,
and
if
you
guys
don't
know,
we
use
prowl
as
the
ci
for
running
and
validations
on
azure
and
pro
is
basically
a
community
ci
that
is
available
for
everyone
to
run.
A
Tests
on
and
the
general
testing
flow
is
something
like
this.
It's
very
simple.
First,
we
build
kubernetes
against
a
pr
branch.
If
the
test
is
running
against
the
pr
we'll
build
kubernetes
against
a
vr,
but
if
we're
building
against
a
release
branch,
then
we
will
check
out
the
release
branch
and
then
build
kubernetes
from
there.
So
we
build
kubernetes
and
then
we'll
have
push
all
the
images
and
artifacts
to
azure
storage
or
azure
image,
azure
container
registry.
A
With
that
upload,
it
will
provision
kubernetes
clusters
with
aks
engine
that
points
the
images
to
the
custom
built
kubernetes
just
that
we
built
in
from
step.
So
now
we
have
a
grenade's
clusters
running
with
custom
kubernetes
version.
Now
we
can
run
various
validations
against
the
clusters
and
that's
how
we
sort
of
build
the
clusters
and
run
validations
against
whatever
changes
are
attracted
and
the
test
results
for
pre-submit
jobs
and
periodic
jobs
are
available
publicly.
You
can
check
out
testgrid
slash
provider
azure.
A
If
you
guys
don't
know,
tesco
is
simply
a
dashboard
for
the
public
to
check
out
the
test
results
from
prowl
and
you
can
check
out
the
historic
test.
Failure
rate
and
also
you
can
very
easily
see
which
test
cases
are.
Flaky
is
a
very
beautiful
dashboard
and
here's
a
screenshot
of
some
of
the
pull
requests
that
we
opened
and,
as
you
can
see,
there
are
a
couple
of
azure
related
pr
jobs
that
we
run.
A
For
example,
here
we
are
running
conformance
tests
against
our
aks
engine
clusters
and
we
also
run
tests
that
are
related
to
azure
storage.
In
here
we
have.
We
are
running
azure,
disk
antenna,
validations
on
aks
engine
clusters
and
also
we
are
running
the
ntn
validations
on
vmss
clusters
and,
as
I
mentioned
earlier,
we're
starting
to
invest
a
lot
in
windows
testings.
A
So
what
that
means
is
that
we're
trying
to
migrate
the
entry
azure
disk
driver
to
csi
drivers
so
that
users
who
are
using
the
entry
drivers
can
seamlessly
transition
to
use
azure
disk
csr
drivers,
and
we
make
sure
that
all
the
prs
that
checks
into
these
kubernetes
repositories
are
passing
before
we
merge
them
to
ensure
that
we
have
a
consistent
test
signal
against
the
master
branch
and
with
that,
I'm
going
to
give
it
back
to
andy.
For
the
demo.
B
So
could
you
see
my
screen
now
honest
yeah?
Okay,
that's
great
so
in
the
first
volume
on
95.
Yes,
now
on
audio
file
is
now
in
product
preview,
and
here
the
page
to
register
azure
storage
nfs
preview
program.
This
page
actually
provides
two
nfa
services.
One
is
at
your
firebase.
It's
based
on
4.1
and
another
is
another,
is
block
study
based
there
for
different
workloads,
and
we
have
already
integrated
the
ideal
file
based
in
the
first
service
with
other
files
inside
driver.
B
So
so,
if
you
go
to
the
id
file,
csi
driver
page,
oh
okay,
let's
skip
anyway.
If
you
go
to
the
audio
file
csi
java
page
and
go
to
the
nfs
session,
and
you
can
see,
we
have
already
supported
the
nfs
and
now
it's
in
our
upper
stage
and
only
east
us
region
is
available
now
and
to
use
this
feature.
User
first
need
to
register.
I
don't
install
nfs
program
so
yeah
this
page
and
another
one
got
confirmation
from
the
azure
starter
team.
We
need
to
register.
B
B
So,
to
save
time,
I've
already
created-
and
I
just
started
account
so
here
to
start
your
accounts
name-
is
cube,
account
demo
and,
let's
click
into
it
yet
and
there
in
the
fireworks
and
the
virtualness
works.
I
already
select
one
subnet
of
the
of
of
the
aks
cluster,
so
yeah,
let
me
show
you
so
here
is
the
aks
cluster.
B
Trip
control
to
apply
the
starter
class,
and
then
we
can
use
this
in
nfs
driver,
so
I
will
show
you
example
to
create
a
wordpress
application
based
on
the
nfs
driver.
The
step
is
quite
simple:
we
didn't
really
need
to.
First,
we
only
need
to
add
the
repo
and
then
use
the
help
install
to
install
the
wordpress
application.
We
just
specify
the
starter
class
and
specify
the
persistent
volume
size.
B
B
So
now
it's
in
waiting
stage-
and
here
is
the
pvc
you
can
see.
Actually
I've
already
created
a
one
two
two
demo,
because
it
could
save
time
and
there
are.
You
could
see
that
two
two
pvc
is
created
one
years
using
default,
starter
class
and
actuates
using
iodized
another
years
using
nfs
file
driver
and
there
let's
go
to
the
start,
account.
B
B
Okay,
so
let's
go
back
to
the
slides,
so
here's
our
future
plan,
we
are
still
working
on
the
car
provider
moving
out
of
tree.
It's
now
in
better
and
we
are,
we
are
working
on
to
make
the
j
in
the
future
and
also
we
are
working
on
aks
integration
with
out
of
three
car
provider
and
csi
drivers,
and
also
trying
to
work
out
first
version
of
external
credential
provider
for
azure.
B
Besides
that,
we
are
also
working
on
azure
cloud
provider.
New
features,
for
example,
support
sharing
the
same
ip
by
multiple
services.
Azure
stack
support
for
the
csi
java
part.
We
will
support
broad-based,
nfs
service
too,
and
also
we
will
support
that
file.
Snapshots,
detach
disk,
using
dangling
arrow
support
on
csi
java
and
also
for
windows,
csi
support.
We
will
move
to
beta
in
the
future.
A
Thanks
andy
and
yeah
contributions
are
definitely
encouraged
and
here's
how
you
can
get
involved.
You
can
join
our
azure
cloud
provider,
community
meetings,
every
other
mondays
at
6,
00
pm,
pacific
time
and
here's
the
link
to
our
communities
call,
and
you
can
also
check
out
the
second
link
for
our
meeting
notes
and
for
all
the
code
owners.
A
You
can
check
out
this
github
link
to
check
out
who
owns
which
part
of
the
cloud
provider
azure
code
and
obviously
you
can
always
reach
out
to
us
and
ask
any
questions
on
the
kubernetes
slack
at
the
provider.
Azure
channel
and
last
but
not
least,
if
you
want
to
you,
know,
create
your
first
full
request
to
contribute
to
kubernetes.