►
From YouTube: Managing a Managed Kubernetes Platform - Annegies van 't Zand & Wiender Sarup, Nationale Nederlanden
Description
Don’t miss out! Join us at our upcoming events: EnvoyCon Virtual on October 15 and KubeCon + CloudNativeCon North America 2020 Virtual from November 17-20. Learn more at https://kubecon.io. The conferences feature presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
Managing a Managed Kubernetes Platform - Annegies van 't Zand & Wiender Sarup, Nationale Nederlanden
Over the past year Nationale Nederlanden (NN) has built a managed Kubernetes platform for their DevOps teams. Now, a year later, we are managing over twenty Kubernetes clusters that are running multiple production workloads. Although Kubernetes is the main component, it turned out that lifecycle management and reliable delivery of changes to in-use platforms proved to be the most challenging part. All-in-all there are fifteen components that provide all the platform features and services, so we had to get creative to make sure that we kept up to date with all of them. In this talk we will share our solutions for setting up a platform delivery pipeline and the lifecycle management of our platform components. Fan-in/fan-out cluster deployments, platform integration tests, version dashboards and feature flagging are some of the subjects that we will address.
https://sched.co/ZepK
A
A
Drivers
and
introduction
how
arie
and
god
is,
and
so
bota
logical
and
a
yarn
of
the
container
patrick
ie
again
the
team
that
started
in
january
2019,
you
can
16
as
x.
Members
were
part
of
the
en
and
cloud
and
hosting
team
en
central
high
tea,
department'
and
even
the
cloud
discolored
lies
were
part
of
the
cloud
factory
consisting
of
three
teams
about.
A
Nn
nn
group
is
a
financial
service
company
and
operating
in
about
ethics,
country
smokey,
european
countries
and
cheap
and
our
head
off
again
about
110
million
customers
and
15,000
employees,
a
head
of
is
it
located
in
the
hague,
the
netherlands,
and
here
you
can
see
all
production
and
service
who
f
an
excuse.
I
mostly
young
insurance
in
vest
with
management
for
time,
service
and
banking,
so
disney's.
Yet
another
bang
creation
and
it
technically
were
very
strict
us
in
certain
aspects
likes
kurti
and
compplan,
see
so
first
out,
dachser
builds
how
to
manage
hey
container
platform.
A
What
what
is
a
platform?
So
one
might
think
that
one.
You
choose
an
a
label.
Google
ads
manage
cabin
service.
There
is
not
much
did.
I
know
you
het,
usa,
cloud
provider
and
jos,
we,
the
few
clicks
of
we,
the
view
coomans.
You
can
spin
on
the
cabinet
that
would
have
service
and
always
in
background
and
age
and
grab
your
cabinet
is
quraysh
property
and
you
can
start
the
fold
and
when
comes
a
workloads.
What
is
that
your
platform,
and
so.
A
William
guy
carr
notes,
fleece
concubines
warming.
Texting
service
is
not
that
hard.
The
challenge
weebly
loses
is
license
santing
health
club
built
in
what
I
say
is?
U
don't
hear
what
come
polis?
You
want
to
install
harry,
oh
monsieur
the
creator
club
like
who
doesn't
connect
to
access
10
of
structure
and
higher
flower,
so
shirky.
A
That's
how
her
already
teams
to
think
about
why
you're
on
google
ads
specially
in
an
enterprise
search
parents
has
think
of
all
the
things
and
much
more
item
ivf
en
application,
je
mond
think
about
having
a
load
balancing
you
want
to
has
monitoring',
because
how
effect
is
then
matrixx,
of
course,
and
if
you
see
in
dress
a
lacquer
with
seats,
someone
between
you
when
it
got
a
and
how
long
and
stuff
like
role
based
access
control'
is
the
pardew
platform
ar
you
go
to
apply
a
path.
Security
policy
is
we?
A
A
However,
the
solutions
porches
very
difficult
to
integrate,
with
our
landscape
anyway,
weir
to
use
the
cabin
the
service
of
an
abs,
that's
called
eps,
the
central
sqq
binet
die
service
and
so
again
die
the
experiment
sms
service.
So
this
was
the
right
choice:
grass
and
yes,
using
the
tss
a
base.
How
jack
there
nothing
and
found.
A
A
heart
container
platform
die
man
about
someone
production
rebuilt
again
when
someone
brush
and
monasteries
and
get
together
all
customers,
runabout
eat,
100,
plus
upload
your
production,
so
its
different
and
yes,
cosmos,
raged
a
grader
and
who
manage
all
this
for
them
and
very
important
part
of
a
platform.
I
wanted
to
provide
our
customers
and
very
easy
way.
It
on
board
al
platform,
what
an
honor
novus
users
or
at
once
users.
A
Yes,
such
a
mania
components,
my
etfs
sable
de
livery
lawyer
a
platform
to
make
sure
that
bridge
photos.
Work
study
is
getting
old
in
the
interruptions.
Many
composer
depend
on
each
other
in
or
to
work
properly
en.
So
you
can
imagine
that
is
that
your
case
one
must
not
the
proprietari
for
easy
to
customer
and
detection
brinks.
As
to
the
subject.
A
One
you
deliver
and
manage
the
service
zo
vriest.
Subject:
je
email
de
charlie,
je
suis
het
en
als
als
de
solutions
met
en
zo.
Some
of
these
things
will
this
costs
will
be
recognized,
wil
bij
for
some
of
you
and
het
ook
new
king
of
je,
something
you
could
use
in
your
own?
U,
warming!
So
I
wonder
how
the
lifecycle
management,
so
lifecycle
management
is
something
how
you
already
do
so
to
team
do
put
a
counter,
and
especially
in
the
culinary
world
northlease
system
do
put.
A
When
I
started
with
a
platform
for
developing
and
fewer
developing
a
platform
who
I
have,
this
grading
still
composer,
that
were
necessary
in
her
platform
for
customers
and
you
do
jurgen
operation
how
and
it
on
how
it
and
family
and
it
awd
production
where
the
glass
there
with
all
those
component
and
integrations
preserve
of
course
week
or
so.
You
have
the
lcm
so
edgy
you
here
all
the
project
ef
and
in
salt
and
all
platform.
A
Yes,
providers
become
latest
criminali
surgeons,
and
if
so,
this
is
about
ten
rio
is
f
check
out,
look
out
for
en
dan.
We
were
that
how
you
manage
as
a
driver
notes
for
work
or
notes,
for
I
guess
jas
en
for
a
day
with
bag
provider.
Room
service
for
management
group
spat
grammy
started
was
a
two
bill,
so
we
create
a
room
solution
en
so
far
that
we'll
be
there
to
jack
wat
de
lilies
purchase
arboretum
and
the
versions
are
also
for
the
fun
deployments.
A
Credits,
the
cuban
eddy's
platform,
lifecycle
management
tool,
michel,
so
many
which
opens
worst
and
so
had
to
chair.
We,
the
porridge
there
input,
and
this
provides
the
image
search,
your
imf,
the
and
al
the
chargers
wf
and
see
how
the
tooling
and
busy
show
like
also
shows
like
the
horror
versions.
A
A
B
B
Out
that
to
is
wel
became
a
version
on
europe,
then
tick
and
accelerating
toilet
the
freezing
point
more
than
40%
die
for
your
changes
and
building
action
that
put
and
adjust
0.
Can
you
do
so
the
mee
vector
or
the
skill
to
do
it?
Your
spa
2
minutes
on
wide,
if
you
can
a
medium
things,
symbol
platform
is
also
where
the
process
in
the
self
bed
again
reports.
Football
because
of
this
will
put
look
out
for
info
piece
of
greenhouse.
You
can
jazz
and
my
exactly
killer
ruby's
pipeline.
B
So
this
this
can
you'll
do
a
new
scotts
bit.
What's
7
times
you
each
it
cost
or
you
you
inside
team,
the
impact
what
weekend
also
a
youtube
for
me,
a
color
unemployed
for
gossamers
so
became
a
mixer.
Yes,
no
chance
de
also
the
campsites
production
number
salt,
and
that
lesson.
When
is
that
you
know
what
a
pipeline
to
take
two
long
deliberation
during
the
heart.
Libie
comes
irish
clements.
B
B
C
B
Super
picture
preferably
shows
and
for
in
facebook.
This
release
is
to
customers
die
stones
instead
of
all
at
once
done
this
way,
we
can
check
the
action
of
customers
and
skill
of
pack
and
thakurji.
Is
it
on
cyprus
or
so
dare
belt
a
stones,
and
you
staging
the
production
and
production
and
just
to
clarify
for
is
toe?
Is
there
open,
semesters
it
and
and
those
measurements
security
measures
that
reaction
time
and
charlie's
minutes
bob
so
week
in
our
customers
to
blisters
a
walk?
So
I
experience
with
products
further
per
process
and
sector
production
cluster.
B
This
is
a
youtube
already
for
plush
workloads
so,
and
we
study
lies
the
start
with
pain
from
left
to
right,
something
like
art
in
the
staging
garment,
and
this
will
bring
so
there
is
lust.
There
is
about
production,
crashed
on
the
noise,
your
web
shop
applications
and
perform
car
rental
and
tess.
If
we
have
a
moment
in
jaipur
now
per
off
the
coast
and
sisters,
I
already
see
the
test
strip,
don't
really
see
the
kitkats
bit
or
the
release.
B
B
That's
how
you
have
to
be
very
basically
a
free,
always
in
the
pipeline,
to
create
a
pleasure
roll
porridge
as
many
that's
right
and
die.
Starcraft
labels
machines,
tandy
the
power
worker
notes
of
formal
blue
green.
I
want
the
width
4
it
and
then
push
got
the
platform
with
link
spaces
with
access
control
next
comes
away
and
from
the
aeronaut
I
just
two
faces
of.
B
Do
you
have
a
bleeding
beast
under
angie's
van
the
chair,
it's
the
atmosphere
at
work,
fury
upstairs
was
in
a
shirt
and
Snellius,
leggings,
etc
and
bomb
that
fell
did
her
job
to
appease
the
orchestra
wini.
After
all,
not
the
time
to
update
pepp,
buggy
persians
and
test
is
back.
They
are
over
moovly.
B
So
am
I
this
with
you?
This
is.
There
seems
the
new
the
pipeline.
The
point
two
in
that
climbing
team
production
warmer
stop
is
raging
new
addition
to
show
them
wheels,
gifts,
christmas
and
time
to
check
the
workloads
it
sees
er
now
manifest
them.
We
work
what
a
in
a
case
I
now
on
all
the
customer's
hjoed
for
the
dual
action
with
the
moraira
pillow
busy
brother
long
white,
also
portuguese.
I
well
so
smart,
it
and
sommelier
list
it
who
not
to
be
able
to
the
fresh
ecru
christus.
Did
you
join
so?
B
Do
you
want
a
wide
and
then
production,
western
and
continuous,
with
the
percy
christus,
madam,
to
change
a
real
or
is
it
a
matter
but
as
for
feature,
and
you
really
have
to
send
your
e-mail
desserts?
Those
critiques
of
the
bur
look
strange.
You
test
this
in
the
week
to
know
is
possible.
She
treaty
and
further
of
a
platform
is
a
60
product
and
of
lydia's
continuous
wishes
to
brighten
the
back
up
at
fuse
the
tim
production
process.
B
Reading
with
that
fold,
script
and
the
suitcase
a
product
girl
and
me
now
it
it
and
call
it
verse
him
all
that
voltage
sampling
and
is
that
from
in
jane's
in
the
product
am
still
too
far.
They
are
so
these
I
lessons
configuration.
Is
it
packers
also
producteur?
They
are
standing,
and
that
is
about
version
not
it
in
release,
I'm
sorry
so
that
they,
your
team,
for
they
hear
on
the
new
feature
of
the
classroom
and
dealer,
show
become
out
of
impact
janitors
and
who,
should
s,
add
the
video
to
products
product?
B
Who
is
able
to
now
plan
skill
if
we
keep
in
the
customer
insight
with
the
page,
especially
if
we
have
a
lot.
This
is
my
lie:
trick
we
influenced
and
then
about
test
that
also
tied
next,
it
out
so
testing
supportersbus
tips
and
tricks
to
a
landing
in
a
plus
in
wordpress,
in
joe's,
so
peace
to
kiss
and
control.
B
It's
the
core
for
components
for
self-service
interface,
who
can
do
it
them
single
tower
and
the
coast
in
a
cooler
to
order
vinckx
customer
can
create
and
no
space.
Is
it
pop
art,
rouge
and
0
bulls,
jurgen,
strict
dossier
and
about
dysfunctional
teaser?
I
censorship
test
it
13
complete.
I
was
not
a
container
platform.
Was
he
a
musician
open
source
project,
the
heart,
the
pinpoint
in
whom
black
testing
barry
cookies
chips
critique,
2
bottles?
B
B
And
the
printing
plan
are
your:
just:
the
pooh
bear
the
project
and
small
container
the
chat
you
can
a
pointer
for
my
part
of
the
west
and
see
really
kian.
This
is
job
drives,
say
they
offer
tests
with
cools.
Not
exactly
is
man
in
need?
Is
it
searches,
hushed
breaking
santa's
and
despars,
a
nice
shirts,
kon,
6
or
notebook
component
in
it's
compustunt
online?
Well
de
so
sweet
still
floated
ns
before.
C
B
Controller
minutes,
jungle,
drifts,
relation
or
the
test
so
for
sample.
So
in
c
come
a
manifest,
a
new
book,
and
so
you
can
say
I
finger
him
the
size
of
the
new
group,
the
tike
they
peck
is
an
economy,
and
there
is
sir
thomas
and
share.
This
means
it's
in
sherry's.
Even
if
you
once
senior
post
spring,
the
ditch
so
cute
couple
suffering
is
on
its
way
be
direct.
You
need
to
know
blue
green
from
the
skin
group
hairdresser,
flash
commission
des
kitty
bruce
and
today,
so
you
need
smooth
politically
looks
like
your
minutes.
B
When
it's
want
olivieri
you
get
to
death
as
we
test
skilling
it
to
say
your
time
changing
to
a
strict
toolchain.
So
I
have
you
can
do.
Welcome
to
mali.
Test
is
togethere
it
do.
I
want
you
opposite
version
is
and
die.
Coole
became
a
parallel
book.
Is
the
dashboard
your
still
using
business
so
will
er
13
the
container
platform
with
the
integration
kinks
pints
in
smoor
short
test,
engineer,
estimate
this
and
chains.
I
will
base
fries
test.
B
I
have
for
you
the
great
and
former
staff
to
see
before,
but
we
still
function
and
she
so
still
and
is
for
the
is
also
the
and
keypoint
forth
and
lucas
margins.
As
I
the
tutorial,
would
the
competition
be
someone
fine
in
the
garden
and
do
sparkling
because
of
that
from
catholic
launch,
successfully
saved'
happy
customers
joske
may
want
tee
eff
shoot
away,
am
lucas
and
the
best
you.
A
Watched
less,
if
not
supplemented,
disk
percents
you
you
on
your
founder
challenges
and
solutions,
interest
in
helpful
such
kind
of
what
nothing
for
as
it
is
your.
I
started
working
on
a
free,
our
container
platform
on
edge
there
using
them
and
manage
version,
my
service.js
and
circular
service
and
we
crush
the
super,
my
travel
cheese
market
to
easily
migrate.
There
were
closed
from
one
cloud
prior
to
another.
You
want.
A
We
put
this,
whose
comes
the
last
slight
zombie
region
of
to
share
with
you,
we
heft,
reboot
and
and
so
comfy
box
and
view
it
seems
even
more
information
on
a
container
platform
and
especially
so
much
importance,
chat,
website
and
economic
side
act
as
if
you
have
any
questions,
entered
and
minor,
pewds
no
box
and
will
answer.
Then
you.
A
A
Can
you
live
hi
hi
guys
so
who
can
of
technology
shoes
who
to
switch
to
one
laptop?
So
sorry,
but
that
will
quickly
start
with
entering
your
questions,
the
fairphone
van
niels
van
brecht.
It
will
be
I
center
you,
or
will
you
be
using
cloud
and
eat
or
composer
of
widow
thrasher?
Is
there
a
clearly
defined
order,
zoom
your
who
turnout,
how
this
a
their
way
color,
and
this
became
bad?
A
How
to
turn
out
who
use
olaf
cloud
9
service,
especially
made
of
glass
and
my
sweet
shoe
escher
reaction
de
so
agree
on
john
paul
ivf,
usm
printers
van
e
tobias.
You
really
want
similar
of
d.
No,
I
want
a
tallinn,
adger
dat,
sam
production
of
secrets,
zo
de
van
reborn,
nuclear
border,
bad.
Yes,
yes,
yes,
turn
it
off
joyce.
What
fitz
in
your
solution,
what
you
we
use!
A
love,
no
die
service
en
safari
to
work
out
and
expression.
B
Jako
polo
fashions
outside
the
blue
green
note.
I
play
man
such
a.
What
exactly
was?
What
is
what
I
should
do,
former
blue
green.
He
does
nothing
turgot
the
occasions
do
what
them
that
tent
are
so
refute,
blue
skinny
pants
am.
I
want
nose,
chin
and
baby
jungle,
rubber,
great
script,
chichester,
pythons
script
and
the
skills
have
a
new
fear
of
need.
B
The
new
notes,
aries
and
then
immediately
began
notes
week,
formal,
the
green,
when
your
nose
is
so
and
and
is
with
all
the
pot
together
now
need
skilled
at
the
hole,
scouring
group
and
it's
ridiculous
lucas
more
channel
on
the
fiction
budget
right
on
that
got
into
a
fight
with
not
even
unless
you
thump
them
as
a
who
tensioner
testing
I,
also
work
for,
according
to
test
urine
a
cry
and
expression
lodge
and
strategy.
Yes,.
A
A
Yes,
en
zo
greg,
you
had
a
fight,
tiense
interests
and
as
if
you
just
to
help
them
out
in
taking
net
step
care,
would
platform
wii,
be
some
kind
or
constant
teams
on
you.
So
they
are
expected
to
helps,
shows
the
van
teams
to
get
up
and
running,
and
both
I
had
or
cost
some
seniors
al
team
itself,
the
knowledge
and
to
put
the
start
to
work
on
en
combination's.
B
Oh
yes
and
expressive
to
ondrej
all
you
tuck
already
such
a
must
monitoring
for
customers
to
access
to
wit
and
how
you
already
such
a
managing
it
would
you
have
for
monitoring'
under
the
matrixx,
so
__
],
cpu
and
staff
who
provider.
I
him
slightly
installation
to
go
another
manager
and
hazards
for
visualization
and
intel
a
little
compost
beer
or
with
it's
only
wolves
customers
to
send
orders
to
their
own
just
stopped
gps
and
that
guy
on
in
sr
so
fat
and
really
on
duty
and
even
a
museum
falcon
about
the
van
drivers
who
monitor.
A
B
B
Yes,
so
extremely
beginning
of
the
front
platform:
wii?
U
basic!
U
up
to
6
jazz
ren
stitches
application!
Do
you
understand
because
again
and
again
not
the
database
use
the
cloud
service
would
win
in
those
laws.
User
abs,
then
me
db,
vox,
have
their
end
lead
to
start
heerings
anthisnes
wanting
to
always
host
jet
it
with
single
men,
rick
well
again,
foxy
leggings
style
by
or
also
be
shipping
always
can.
B
A
B
B
Needed
a
chest
and
the
needed
so
since
it
is
so
a
work
to
problem
yes
with
the
manager
simson
us
is,
the
sea
was
with
the
just
there
needed
for
me
or
blogging
and
great
for
take
a
burning.
The
double
the
mount
of
nose
in
the
in
a
in
the
cluster
and
to
all
the
post
get
moved
to
nu
notes,
and
then
I
green
and
the
battled
on
upscaling
again
in.
A
B
Yes,
so,
but
I
do
the
nice
je
test
application
de
deuce
that
the
old
features
are
up
to
your
skin
bowie
visible
to
me
and
a
home
upgrade
or
the
de
dedication
and
miela
is
a
formal
performance
test
that
is
between
check,
is
the
performing
wright
and
between
air
lot
of
and
when
my
test
seem
critical
security
burton
mack
rides.
It
kids
through
a
word
punching
bag
sample
bc.
Fv
slugging
youtube
a.
A
C
B
C
A
B
Seems
a
handle
costs
a
lesson
hour:
yes,
beets.
Do
it
picas
a
big
barda
grecian
the
die
service,
so,
who
is
he
and
video
becomes
war?
Was
focus
to
this
does
springs,
have
a
means,
bits
tips
and
tools
for
security
in
staff
and
sometimes
workload
so
labeled
rune
support
cluster.
Just
for
a
heart
networks.
Operation
for
iphone,
both
again
in
your
laws,
be
able
to
be
flexible,
became
to
sir
singer
container
platform
peppers
who
new
again
french,
some
of
the
be
you
wouldn't
accept
one
large
platform,
something
like
this
variable
in
the
weather.
B
A
B
C
A
A
Juliette
lat,
if,
yes,
how
do
you
think
in
the
beginning,
wie
quickly
find
out
that
dokumentation
was
so
important,
jos
en
providing
all
necessary
information
for
them?
To
start
a
de
demo,
anais
explained
box
can
or
links
rx
baseboards
to
very,
very
detailed
right
step
by
step,
nfs,
most
of
them
or
customize.
The
food
was
really
helpful.
Having
that
dokumentation
now
you
then
there
to
and
erections
a
the
box
can
or
links
and
p
to
their
dation
brothers.
Therefore,
very.
B
Friborg
I
got
in
feeders
repositories
and
e-commerce
its
time.
What
are
there
on
the
that
I
just
move
2
monitor.
We
have
stain
already
direct
shipping
care
super
product
with
programs,
monoï
diffuser
de
kit,
je,
if
you
je
masseurs
de
production,
cold
and
and
feature
changes,
got
into
a
meticulous
industry
and
has
your
press
offers
merge
via
google
glass
love
so
yet
another
taste.
B
B
B
A
So,
family,
the
partners
new
bad,
the
like
also
mini
case
freelance,
well,
how
you
partner
of
the
eye
care
committee,
doing
and
room
en
kom
it's
right
to
open
source
project
of
user
project.
I
think
it's
there,
but
one
you
zit
santing,
karnak
temple
the
ice
judgment
funny
thing
je
see
and
then
you
greate
pier
en
die
after
eight
from
to
merge,
het
and
religion.
Teacher
links.