►
Description
Using Kubernetes to Change Legacy Systems and Processes in the Public Sector [B] - Audun Fauchald Strand, Norwegian Welfare Administration
Kubernetes is the implementation of the modern software development process. Continuous Release and “you built it, you run it”. For the last few years I have been working on introducing kubernetes into an organization with continuous release, microservices and “you build it, you run it”, as presented at Kubecon in Berlin 2017.
Now I work for the public sector in Norway, where the systems are old, and the processes are older. I will present the experiences from working on changing these legacy organisations, using containers and kubernetes as the main tool. I will cover:
- migrating old legacy apps to kubernetes, is it possible
- manual testing done easy with containers
- monitoring for everyone
- making a PAAS that everyone can use
- stable and robust deployment, but not just 4 times a year
- how to leverage all the hardware that is owned by the public sector
About Audun Fauchald Strand
Team Lead - Platform and automation, NAV - Norwegian Welfare Administration: k8s, ddd, jvm, Kafka, distributed systems, testing, Tottenham. Almost called "Large viking shaped Norwegian" in LWN
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
Hello,
when
I
left
Norway
on
Monday
morning,
I
had
to
walk
through
snow
to
get
to
the
bus
to
get
to
the
airport,
and
then,
when
I
arrived
in
Austin
about
18
hours
later,
it
was
what
we
know,
which
was
call
a
really
beautiful
summer's
day
and
I
thought
I'd,
packed
all
the
wrong
clothes
and
I
thought
I
need
to
adapt
to
this
turns
out.
I
didn't
have
to
adapt
at
all
Austin
adapted
to
me,
because
yesterday
it
snowed
in
Austin,
probably
my
fault
I,
don't
know.
A
We
were
stuck
on
promising
around
data
centers.
We
had
waited
in
the
test
environments
where
the
access
control,
which
was,
to
course,
so
it
was
problem
for
developers.
We
had
networks,
owns
almost
hindering
any
kind
of
development
pace.
We
needed
some
kind
of
overview,
although
dependencies
that
we
could
trust.
We
had
really
bad
monitoring
and
observability
in
our
systems.
It
was
difficult
and
cumbersome
to
create
new
applications
and
we
had
loads
of
nightly
batch
jobs.
A
But
first
a
bit
about
me:
I
used
to
be
a
Java
developer
for
many
many
years
and
I
think
the
fact
that
I'm,
a
devil
going
into
the
infrastructure
space,
makes
a
difference.
I've
heard
a
lot
about
all
the
loads
of
the
keynotes
I
talk
about
how
we
want
to
start
solving
the
developer's
problem
and
an
increase
the
development
pace
and
they
make
better
the
deployment
pipelines.
A
All
of
those
things
have
been
kind
of
my
main
goals
for
starting
to
work
with
infrastructure.
It's
not
about
it's
about
increasing
this
pace
and
the
development
speed.
So
when
I
started
to
when
I've
been
a
developer
for
like
10
years
and
had
all
this
bitterness
built
up
inside
me,
I'm
thought
I
could
use
all
those
experiences.
You
know
that
bitterness
and
try
to
solve
the
problems
for
all
the
other
developers,
so
they
didn't
have
to
have
all
the
problems
I've
had
so
I
started
to
work
with
infrastructure
and
platforms.
A
I
started
with
Kafka
operated
Kafka
a
few
years
from
my
former
company
and
then
I
started
the
migration
to
given
at
this
I
presented
something
on
Cuba
in
Berlin
this
spring
then
I
worked
for
a
company
called
him.
Please
one
of
the
largest
websites
in
our
rates,
a
classified
marketplace,
and
what
I
talked
about,
then,
was
how
we
can
maintain
continuous
delivery
walls
migrating
to
kubernetes,
and
that's
been
a
great
success
right
now.
A
A
Basically,
it's
got
its
offices
all
around
Norway
and
people,
the
citizens
of
Norway,
when
they
have
problems
when
I
lost
a
job
or
when
they're,
sick
or
when
they
have
kids.
They
can
come
into
this
office,
these
offices
and
talk
to
the
different
people
we
were
after
and
get
help.
Basically,
we
have
50
different
benefits.
We
pay
out
about
1/3
of
the
national
budget
of
Norway,
and
one
of
the
richest
countries
in
the
world
is
paid
that
paid
up
by
this
agency.
A
A
The
when
I
got
to
know
I
thought
this
is
really
old
and
strange
and
difficult,
and
then
I
realized.
There's
a
history
here
now
I've
got
its
first
computer
system
in
1967.
The
database
of
that
system
still
runs
it's
a
large
10
gigabyte
file.
We
built
in
new
service
on
top
of
that,
but
we
still
use
the
database
it's
in
read-only
mode.
Now
it
contains
the
salaries
of
people
living
in
Norway,
from
like
67
and
up
till
2010
ethic,
and
the
big
file
is
a
bit
difficult
to
handle.
A
So
there's
this
old
lady
she
comes
in
on
Tuesdays
and
Wednesdays.
She
was
a
part
of
the
team
that
built
that
system
and
she
helps
the
team.
That's
now
trying
to
be
like
architect
into
this
large
10
gigabyte
file
in
an
strange
language.
They
don't
understand
and
try
to
understand
and
fix,
find
all
the
data
that
I
need
to
get
that
data
to
use
it
for
calculating
pensions
for
no
reason
Norwegians.
That
was
the
first
system
and
of
course,
we
have
mainframe
systems.
Those
are
still
in
use.
A
We
have
this
really
strange,
Oracle
Java
thing,
which
was
created
around
2000,
where
the
Java
objects
are
instances
of
those
objects
are
in
the
database.
It's
not
like
an
app
server
so
for
all
the
developers
have
to
have
their
own
Oracle
database
to
do
anything
at
all.
That's
stealing
still
running
loads
of
IBM
WebSphere,
je
applications,
most
of
those
are
still
running
we're
an
IBM
MQ
message:
queue
still
running
last
five:
twenty
ten
years
they've
been
migrating
to
more
modern
stacks
like
J,
bus
and
stuff.
A
Most
of
our
applications
now
are
in
jboss,
but
we
realized
well,
they
have
all
this
other
systems.
We
what's
the
harm
of
introducing
a
few
more
so
we
started
to
work
on
doctor
and
kubernetes
oops
forgot
about
images.
That's
actually
probably
this
Norwegian
for
production.
This
was
the
production
system
at
some
point
in
Mouse
history,
and
now
we
want
to
be
somewhere
around
here
so
novice
at
a
private
cloud
for
a
few
years,
based
on
VMware,
it's
kind
of
self-service
where
the
developers
go
into
different
web
apps
and
provision
the
stuff
they
need.
A
They
get
VM
VMs
for
JBoss
areum's
for
IBM,
Webster
and
all
databases
and
there's
a
really
strict
handle
between
developers
and
Ops
guys.
So
the
Ops
guys
have
no.
The
developers
have
no
access
at
all
to
production
and
if
there's
an
error
in
production,
absque
eyes
find
it
and
then
they
might
call
the
developers
developers
later,
at
least
until
last
day.
There
was
a
three-month
release.
I
could
before
we
test
period
afterwards,
and
there
was
set
of
testers
and
release
managers.
A
A
So,
as
you
can
see,
the
challenge
I
wanted
was
bigger
than
the
one
I
had
before.
I
just
had
to
maintain
continuous
delivery.
Whilst
migrating
is
given,
it
is
now
we
want
to
do
use,
kubernetes
and
use
everything
to
kind
of
change.
This
organization
and
change
all
this
technical
difficulties,
and
the
reason
I
want
to
this
is
this
guy,
my
boss's
boss's
boss's
boss,
his
name
mr.
Vernon
histor
is
the
CIO
or
now
and
he's
a
former
developer.
So
he
understands
things
and
he
wants
us
to
deliver
twice
the
speed
at
half
the
price
and.
A
So
what
we
created
to
achieve
this
twice
twice
to
speed,
half
the
price,
there's
lots
of
things
you
need
to
do
and
I
don't
know
half
of
those
things,
but
I
know
how
to
make
platforms.
So
I
wanted
to
use
my
experience
to
build
a
cuban
artists
based
platform
to
try
to
change
as
much
as
possible,
changing
the
culture,
increasing
the
development
speed
and
try
to
improve
the
application
architecture.
At
the
same
time,
I.
A
Was
at
the
talk
with
people
from
a
German
bank
yesterday
and
I,
you
talked
about
how
you
you
had
the
kind
of
the
same
challenge,
but
you
I
think
you
had
the
harder
time
constraints,
so
you
needed
to
do
this
without
improving
the
application
architecture
as
much
and
without
doing
a
lot
of
stuff.
We
don't
have
those
time
constraints
away.
We
want
to
use
this
our
opportunity
for
change
to
actually
change
more
than
just
moving
to
the
cloud.
A
So
a
tiny
repetition
we're
stuck
on
premise.
We
had
two
minutes
test
environments.
Access
control
was
a
problem
for
developers.
We
had
Network
zones
and
firewalls
that
stopped
almost
any
kind
of
development
pace.
We
needed
overview
of
our
dependencies
that
we
could
trust.
We
had
bad
monitoring,
it
was
difficult
to
create
any
applications
and
we
had
loads
on
nightly
bad
jobs.
A
So
the
first
part
we're
stuck
in
our
own
data
centers
and
because
we're
a
government
agency,
we
are
load
to
sensitive
data,
the
things
similar
to
the
witness
protection
programs.
The
addresses
of
those
people
are
in
our
database,
so
we
need
to
kind
of
protect
the
data
a
lot,
so
we
can't
just
move
them
outside
of
Norway
without
having
proper
control
and
there's
no
public
cloud
data
centers
in
all
right.
A
So
we
kind
of
wish
the
compromised,
basically
but
then
again,
there's
loads
of
stuff
happening
in
the
open
source
or
in
the
public
cloud
work
which
we
want
to
leverage.
We
want
to
be
able
to
use
all
those
serving
services
or
product
as
a
service
platform,
so
service
products
that
Amazon
and
Google
and
everybody
offers
both,
because
it
will
make
us
quicker
and
it
can
make
everything
cheaper
and
I
think
we
think
we
can
move
at
some
point.
A
If
we
want
to
migrate
everything
outside.
It's
going
to
take
a
long
rewrite
that
we
can't
migrate
our
mainframe
systems,
for
instance,
so
the
solution
to
this
at
least
the
start
of
the
solution.
This
is
kubernetes
because,
when
I
speak
to
my
developers
or
the
development
developers
into
the
in
teams,
but
what
they
need
to
do
to
be
able
to
make
an
application
run
in
a
public
cloud
and
what
they
have
to
do
to
make
the
applications
run
in
kubernetes,
at
least
the
first
10
to
15
steps
are
exactly
the
same.
A
A
Also,
we
can
mirror
or
replicate
most
of
the
cool
things
we
want
in
the
public
cloud
offerings
in
kubernetes.
We
can
have
function
as
a
service
and
hosted
databases
with
the
less
operational
cost
for
developers,
and
we
can
also
leverage
all
the
different
cloud
native
technologies
that
exist
around
there
for
monitoring
and
service
meshes
and
security.
I
forget
all
on
the
animation
side
and,
of
course
we
can
reduce
cost.
We
have
really
large
I
think
now,
which
isn't
really
Norway's
in
the
biggest
country.
A
So
our
we
are
kind
of
a
traffic
limit
will
never
have
more
than
at
least
we
can.
We
can
estimate
our
user
user
growth
quite
good.
We
have
approximately
5
million
users
now
and
we'll
have
approximately
4
million
users
for
quite
a
few
years
unless
there's
some
kind
of
war
situation
where
Norway
takes
other
countries
but
I,
don't
think
that
will
happen.
A
So
instead,
we
don't
we
don't
it's
more
important
for
us
be
able
to
run
our
applications
more
efficiently
than
hand,
like
large
large
scale
increases
so
to
conclude.
Moving
to
communities
at
least
the
start
is
the
same
as
its
a
good
stepping
stone
to
move
to
a
public
cloud,
and
we
get
notes
of
all
the
early
on
in
the
process.
A
So
the
next
problem
had
we
had
to
minitest
environments,
more
than
20,
distinct,
test
environments,
loads
of
them
in
production
data,
which
means
some
of
the
data
of
all
the
people
in
the
witness
protection
programs
that
we
really
really
really
don't
want.
Anyone
to
see
and
all
those
environments
are
difficult
to
coordinate,
I
think
it's
a
move,
but
I'm
not
entirely
sure
kid
be
just
a
really
long.
A
Jeff
give
Jeff
yes,
I've
had
all
the
SAS
environments
and
coordinating
those
it's
difficult,
there's
three
or
four
people
whose
job
it
is
just
to
make
sure
that
you
have
control
over
which
versions
of
it
services
runs
in
which
test
environments,
and
we
have
some
of
the
legacy
systems
can't.
We
can't
have
one
copy
of
all
the
legacy
systems
in
each
test
environment.
So
we
have
to
kind
of
have
some
system
to
handle
all
that
and
my
I
think
the
reason
we
have
all
these
test
environments.
A
It's
because
we
have
this
really
long
release
cycles,
because
when
we
have
three
or
two
foreign,
the
developers
each
going
to
release
something
at
the
same
time
four
times
every
year,
they
need
different
and
they
it's
difficult
for
them
to
integrate
before
that.
So
they
need
separate
environments
and
then
some
kind
of
test
period
afterwards.
A
So
what
did
we
do
to
solve
this?
Basically,
we
created
two
Cuban
artists
clusters
in
our
solution,
production
and
not
production,
and
in
not
production.
We
made
it
possible
to
create
all
this
different
two
test
environments
using
namespaces,
but
every
time
someone
asked
for
a
new
test
environment.
We
said
no
at
least
twice
and
then
so
they
really
have
to
have
good
reasons
to
be
able
to
do
this.
A
But
of
course
you
can't
just
take
away
all
those
opportunities
at
once.
You
have.
We
want
people
to
actually
use
things
to
build
as
well,
so
they
have
to
make
it
easy
when
they've
described
us
that
I
really
need
this,
so
the
automatic
provisioning
of
these
namespaces
afterwards,
the
diagram
is
supposed
to
show
how
we
there's
many
instances
of
my
notifications
running
in
not
production,
but
in
production
there's
only
one
of
each
and
then
we're.
A
Well,
this
makes
it
possible
to
create
in
test
environments
as
the
increases
as
we
need.
We
can
have
somebody's
isolation.
We
can
either
limit
on
the
namespaces,
but
it's
still
easier
to
reduce
afterwards
and
we
have
some
we
send
our
bit
so
that
people
think
why
they
need
all
those
test.
Environments.
A
A
The
access
control
in
our
old
system
was
that,
basically,
it
was
either
you
had
access
to
production
or
had
no
work,
no
access
to
production
and
the
operations
people
have
access
and
the
developers
didn't
and
this
kind
of
handover
process
where,
if
you
wanted
to
do
anything
in
production,
you
had
to
get
you
had
to
hand
over
your
application
to
the
operations,
people
that
didn't
really
work
and
it
created
it
introduced.
It
took
away
a
lot
of
the
trust
between
the
different
departments,
so.
A
What
we
then
wanted
to
do,
what
we
basically
want
to
do
was
to
reduce
the
distance
between
the
people
who
caused
the
pain
and
the
people
who
had
the
pain,
because
now
the
distance
was
very
much.
The
developers
made
errors,
but
they
didn't
will
they
feel
the
pain
of
those
errors
that
was
done
by
the
operations
people.
So
we
wanted
to
let
the
developers
feel
all
the
error
state,
all
the
pain
they've
created
themselves,
but
then
we
had
to
make
it
possible
for
them
to
get
access
to
production.
A
We
couldn't
give
them
access
to
everything,
because
we
have
sensitive
data
and
we
need
some
kind
of
control.
So
we
implemented
the
open
ID
connect
system
where
we
have
an
azure
ad
active
directory,
which
is
have
all
the
IDS
of
all
our
developers.
And
then
we
just
use
the
given
artists
namespaces
to
say
that
the
development
teams
are
not
access
to
stuff
in
their
own.
A
So
network
sounds
we
had
loads
of.
We
had
three
or
four
different
network
sounds,
and
we
had
a
far
role
between
them,
which
wasn't
it
was
handled
by
the
operated
on
manually,
which
made
it
when
you
have
that
for
10-15
years.
There
really
is
no
control
anymore.
You
just
have
loads
of
holes
in
the
firewall
and
don't
really
know
where
they
are
and
it's
difficult
and
slow
to
create
new
to
open
for
applications.
A
So
basically,
what
we
want
to
do
is
to
automate
all
this
stuff,
and
we
want
to
do
this
by
using
network
policies.
I
will
do
this
by
using
Network
process
in
communities.
So
then,
when
we
have
namespaces
for
each
of
the
different
teams,
we
can
basically
in
a
declarative
and
fully
automated
way,
just
recreate
all
the
actual
rules
from
the
firewalls
and
have
the
necessary
necessary
isolation
between
the
different
teams
applications,
and
we
also
look
at
spiffy
and
siren
all
new
cool
stuff
I
learned
about
last
few
days
in
hopes
to
increase
this
even
further.
A
Although
we
had
the
oldest
architects
and
architects
like
to
draw
likes
to
draw
diagrams
of
boxes
and
arrows
and
how
everything
fits
together,
the
problem
is,
of
course,
as
is
difficult
to
do
that
in
a
correct
manner.
Most
of
the
time
they
do
it,
either
development
time
or
deployment
time
they
talk
to
developers
or
they
try
to.
They
try
to
do
some
static
code
analysis
to
see
who
talks
to
who
and
that's
never
correct.
A
So
the
next
problem
had
was
monitoring.
The
monitoring
solutions
had
were
mostly
made
for
the
operations,
people
and
the
bosses,
because
those
had
once
or
scared
or
those
are
the
ones
whose
header
on
the
block
when
something
goes
down,
and
that
means
you
had
a
separate
team
trying
to
create
monitoring
solutions
and
they
didn't
really
talk
to
the
developers
at
all.
A
So
here
we
basically
did.
Quite
simply,
we
introduced
promises
as
a
metrics
database
on
a
matrix
collection
system
and
using
the
cuban
Etta's
method.
When
you
call
data
using
Prometheus
on
Cuban
notice,
you
get
all
the
metadata
from
the
deployments
on
the
time
series
and
that
specific
that
that
data
makes
it
possible
to
create
like
generic
dashboards,
like
the
one
you
can
see
here,
which
is
actually
shows
data
for
any
application
running
in
your
cluster.
But
you
can
drill
down
and
see
only
data
for
one.
A
So
we
created
this
graph
on
our
dashboard,
which
has
shows
the
generic
information
with
a
drop-down
with
with
applications
running
with
all
the
applications
running.
So
as
soon
as
you
deploy
an
application
to
our
cluster,
you
can
go
to
the
graph
and
on
dashboard
and
just
pick
your
application
and
in
your
environment
and
in
your
cluster
from
Reforma
and
see
all
the
data
and
we
use
the
default
exports
for
Prometheus.
A
So
we
can
have
JVM
stats
and
node
stats
and
stuff
like
that,
and
CPU
and
memory
data
comes
from
hipster
and
we
can
even
using
this
data
quite
easily
quite
some
kind
of
building
solution
where
we
measured
resource
utilization
for
each
of
the
different
systems,
because
they're
all
reported
in
exactly
the
same
way.
So
we
can
create
building
dashboards,
saying
basically
introducing
something
we
haven't
had
before,
that
the
development
team
have
to
take
into
consideration
the
cost
of
what
I'm
and
creating.
A
So
and
the
next
problem
it
was
difficult
to
create
new
apps
I
talked
a
bit
about
the
old
kind
of
private
cloud
they
had
where
all
the
provisioning
was
done,
manually
endeavor
applications
which
were
homemade.
So
we
had
to
go
to
some
systems
to
create
the
VMS
and
create
applications
and
create
databases
and
other
systems
to
create
configurations
for
different
applications
and
track
to
deploy.
You
had
to
create
a
ticket
in
JIRA
and
the
ops
people
did
the
deployment,
so
we
created
something.
We
created
a
separate
application,
err
called
a
nice
deployment
daemon.
A
It
is
a
go
plication
running
inside
the
cuban
artists
custer,
and
when
you
want
to
deploy
an
application,
you
do
an
HTTP
POST
at
the
endpoint.
With
some
the
file
you
can
see
on
the
side.
There
is
all
the
different
things
you
can
configure,
but
there's
slow
to
sensible
default
and
we
created
specifically
for
now
so
normally
there's
like
ten
to
fifteen
five
to
ten
lines
of
UML
for
an
application,
I'm
just
saying
what
in
what
you
need
and
then
then
this
nice
deployment,
even
fetches
this
file
and
the
dr.
A
container
and
deploys
it
and
the
fact
that
we
use
go
us
and
have
created
each
application.
Ourselves
makes
it
possible
for
us
to
tailor
a
specifically
to
our
needs
for
deploying
an
application.
So
we
can
use
this
to
integrate
with
some
of
the
older
systems
like
the
system
here
for
application
configuration
has
an
API
as
well
as
a
web
GUI,
so
we
can
reuse
all
the
application
configuration
that
makes
it.
A
That
makes
it
much
easier
to
migrate
applications
to
our
new
system
and
then,
when
we're
done-
and
we
have
a
majority
of
this
application
inside
our
cluster,
we
can
try
to
reduce
some
of
those
need
for
those
configurations,
and
this
also
integrates
with
all
the
different
other
system.
Shell.
So
you
can
say
this
says
this
turns
on
promises.
If
the
application
exposes
Prometheus
metrics-
and
it
happens,
everything
and
there's
been
loads
of
discussions.
A
We
have
this
file
together
with
the
application
files
and
doctor
file
and
one
of
the
things
I
like
about
this
is
then
it's
not
my
responsibility
when
it's
inside
the
developers
to
get
repository
to
the
developers,
responsibilities
and
it's
them
who
owns
the
application
and
all
the
configuration
needed
and
we
get
the
same
advantages
with
git
commit
log
and
everything
for
all
the
changes.
So
the
nice
deployment
takes
this
information
and
transforms
that
into
cuba
net
this
resources,
so
bad
jobs
because
of
our
old
architecture.
A
A
And
then
you
come
to
that
to
your
day
because
that's
when
the
users
use
our
systems,
so
instead
we
try
to
introduce
batch
jobs
as
a
concept
in
Nice,
where
we
have
a
separate
file,
nice
job
iam
on
file,
and
we
have
the
battery
of
the
separate
containers.
And
again.
This
is
the
same
kind
of
thought
processes
us
behind
the
nice
EMF
every
have
a
sensible
default
and
try
to
reduce
the
amount
of
llamo
and
the
developer
has
to
write.
A
This
is
kind
of
the
architecture.
I
think
I'd
talk
most
what
we
try
to
layer
it
so
that
it's
possible
to
run
on
many
different
platforms.
We
have
the
applications
on
top
and
then
I
what
we
call
the
platform
applications
which
is
stuff
like
rook,
a
nice
toe
and
reformer
and
Prometheus,
and
a
few
operators,
mostly
written
in,
go
running
inside
the
trusted
doing
stuff
and
the
fact
that
we
base
all
this
on
the
communities
api's
makes
it
possible.
A
A
And,
as
I
know,
this
is
my
second
second
go
at
migrating.
A
large
system
to
inter
communities
and
I
realized
that
creating
the
platform
it's
so
much
easier
than
migrating
the
applications,
probably
because
we
want
to
increase
the
quality.
Whilst
we
do
this,
but
just
writing
an
app
and
platform
would
take
us
a
lot
less.
If
you
didn't
have
to
tailor
it
to
the
fact
that
you
had
to
migrate
applications,
so
the
first
one
we
want
to
build.
That's
going
to
be
the
next
step.
A
When
you
migrate
with
most
of
stuff,
we
can
try
to
create
the
perfect
application.
No,
the
perfect
platform,
if
you're
still
there,
of
course,
and
the
fact
that
we
can
reuse
parts
of
the
private
cloud
and
makes
it
much
easier
to
migrate
for
storage.
We
just
started
use
route
IO
and
that
works
really
well,
it's
I
heard
the
rumors.
That's
going
to
be
a
CN
CF
hosted
project
really
soon,
and
it's
I
always
thought
setting
up
like
distributed
storage,
which
was
difficult
and
scary,
but
using
route
makes
this
incredibly
easy.
A
A
We
open
sourced
everything,
not
necessarily
because
we
think
anyone
can
reuse
what
we
do,
but
because
we're
a
tax
funded
government
agency,
we
think
what
we
do
should
be
open
and
public,
because
it's
basically
the
our
interpretation
of
some
laws
and
having
the
open
source
model
also
works
really
well
internally.
I
normally
give
cake
to
anyone
outside
my
team
who
creates
pull
requests
in
tonight's
and
cake
works
really
well
to
get
people
to
do
stuff
for
you.
A
Success
without
for
many
years
at
Wyndham
manage
the
whole
organization
to
change.
Do
weeks
of
work
just
because
I
promised
them
cake
and
any
team
that
migrates
an
application
to
nice
and
gets
proper
traffic
in
production
gets
cake
and
it
works
because
it
creates
enthusiasm
and
it
creates
visibility.
A
A
Doing
this
I've
never
asked
anyone
to
migrate
to
applications
they
just
they
come
knocking
on
the
door
straightaway
when
it's
when
we
start
to
talk
about
what
we
have.
And
lastly,
if
you
want
to
be
liability,
run
it
culture
where
the
developers
do
the
most
of
their
operations
themselves.
Kubernetes
gives
you
loads
of
good
tools,
flattened
and
also
reduces
the
amount
of
work
you
have
to
do.
A
Yeah
well,
I
think
it's
difficult
to
describe
culture
but
I.
Think
lack
of
trust
was
a
really
good
word
when
we
started
this
journey.
The
first
kind
of
not
explosion,
but
the
first
kind
of
thing
that
happened
was
when
developers
wanted
access
to
production
because
the
Ops
guys
didn't
want
to
do
that.
But
that's
what
we
wanted
to
achieve
and
that's
kind
of
the
first
thing
where
we
saw
that
we
had
to
change
the
culture
and
we
managed
to
kind
of
do.
A
We
managed
to
create
some
kind
of
system
where
we
have
them
up
skies,
because
a
lot
of
our
developers
haven't
really
got
too
much
experience
with
running
stuff
in
production
either.
So
we
need
to
train
them
as
well
and
we
try
to
help
make
the
ops
guys
help
it
out
so
and
also
we
need
to
start
small.
You
can't
just
put
the
most
important
thing
in
production
right
away.
You
have
to
build
trust
on
people
have
to
see
that
this
will
work.
A
A
All
the
men
from
developers
keeps
telling
me
that
we
had
this
50
years
ago.
They
say
hey
and
of
course
some
of
it
is
true.
They
have
like
this
shared
the
pool
of
resources
and
how
they
use
it
is
very
similar
and
they
even
had
all
the
building
stuff
made.
When
I
when
I
started
to
say,
oh,
we
can
make
pilling
with
this.
They
just
start
laughing
because
they
had
this.
It
was
kind
of
built
into
the
system
from
the
start,
so
it
obviously
concept
are
similar,
but
this
is
on
data
center
level.
A
No,
but
we
haven't
not
too
long.
This
is
the
last
slide
I
made
basically
because
I'm
coming
here,
we
have
a
few
apps
using
it,
but
the
main
guy
who
did
the
work
says
it's
at
least
it's
easy.
Let's
see
if
it
works
as
well,
but
yeah
Seth
works
and
I
remember
from
Berlin
route.
People
say
well,
Luke
is
just
for
setting
up
Seth
it's
not
on
the
data
path.
Then
it's
just
normal
stuff,
yeah,
sorry
doctor
doctor,
yeah,
no
I
kind
of
want
to,
but
I
never
had.
Anyone
tell
me
they've
done
that
success.
A
A
Loads
of
pushback,
but
I
think
we're
not
done
here.
We're
kind
of
we
know
where
we're
going
to
enemy.
We
done
some
of
it,
but
we
know.
Oh
sorry,
she
said,
stop
I'll
just
finish.
We
know
this
is
going
to
take
years,
because
some
of
the
systems
has
to
be
rewritten
from
scratch.
There's
no
and
that's
some
of
our
core
systems,
so
a
lot
of
the
old
processes
will
be
there
for
many
years.