►
From YouTube: Kubernetes at DigitalOcean: Building a Platform for the Future [B] - Joonas Bergius, DigitalOcean
Description
Kubernetes at DigitalOcean: Building a Platform for the Future [B] - Joonas Bergius, DigitalOcean
Like in much of our industry today, DigitalOcean has been on a journey to evolve its monolithic architecture of the early days towards one made up of cloud-native micro services. This talk will provide an overview our journey from where we started to what our motivations were, share what we built and provide you with lessons we learned along the way.
You can expect to take away examples and ideas of how you can make Kubernetes your own as the platform for the future reinforced by concrete examples of exactly how and what we have done at DigitalOcean.
About Joonas Bergius
Joonas Bergius is an Engineering Manager at DigitalOcean, where he has been working with Kubernetes since late 2015 to build a runtime platform for the engineering teams at DigitalOcean.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
And
that
should
be
enough
to
indicate
that
this
is
actually
a
real
problem
or
not
a
problem,
but
it's
a
it's
a
meaty
enough
thing
that
we
need
to
hire
people
who
specialize
in
it
and
so
there's
a
couple
of
interesting
things
in
the
industry
right
now
that
are
sort
of
making
this
even
more
difficult.
So
one
thing:
that's
happening
industries
we're
seeing
companies
increasingly
break
down
the
monolith.
What
we
are,
what
we
affectionately
call
monoliths
down
is
a
set
of
micro
services.
A
That's
an
ongoing
trend
that
companies,
large
and
small,
are
starting
to
how
they're
starting
to
build
their
software,
and
the
other
thing
that's
happening
is
that
the
traditional,
more
traditional
operations
role
is
actually
kind
of
fading
away
and
it's
being
replaced.
What
what
we
all
probably
know
is
DevOps
and
as
far
as
I'm
concerned,
that's
just
basically
the
same
role,
but
getting
a
little
bit
fancier
tools,
but
but
the
thing
it
is
is
that
many
companies
have
come
to
the
conclusion
that
they
have
the
more
traditional
operations
role.
A
The
you
know,
I
see
what
what's
considered
is
IP
is
actually
just
a
cost
center,
and
they
they
don't
want
to.
They
don't
have
that
role
at
their
company.
Smaller
companies
are
definitely
this
way
you
talk
to
any
startups.
They
typically
don't
have
these
dedicated
operations
folks,
and
what
that
means
is
that
the
developers,
the
folks
who
are
shipping
and
making
the
products
they're
now
getting
this
additional
set
of
responsibilities
added
on
to
their
to
their
job
description.
A
That's
pretty
good.
My
job
here
is
done
so
just
for
those
who
don't
know
what
we
do,
we're
a
cloud
provider
similar
to
some
of
the
other
folks.
You
might
have
met
here
at
the
conference.
Hopefully
the
server
is
kind
of
a
giveaway
for
that,
and
so
let's
begin
our
story
or
the
journey
so
I
joined
the
company
in
early
2013.
A
So
when
I
joined
the
company,
our
sort
of
approach
to
operations
was
what
his
operations
even
mean
like?
What
does
what
do
I
do
so?
We
had
some
operations
folks,
but
they
were
not
dedicated
to
supporting
the
application
in
the
software
that
we're
deploying
they're
actually
focused
on
other
things
that
we
were
operating
and
so
sort
of
over
the
course
of
that
year.
A
We
we
came
to
the
conclusion
that
we
should
actually
get
some
folks
who
actually
really
know
and
care
and
understand
this
space
well,
and
so
what
we
we
ended
up
doing
is
actually
hired
a
bunch
of
folks
who
were
very
well-versed
in
the
industry,
more
advanced
tooling,
that
is
known
as
configuration
management
today,
and
so
you
know,
these
are
your
DevOps
folks
and
they
became
our
sort
of
embedded
operations.
Folks
of
every
software
engineering
team
would
have
it
he's
one
of
these
folks
embedded
in
and
working
with
it.
A
Now
we
pretty
quickly
realized
that,
as
we
were
starting
to
scale
our
engineering
organization,
that
that
wasn't
necessary
the
most
effective
model,
because
we
could
hire
these
engineers
a
lot
more
quickly
than
we
could
hire
operations.
Folks.
So
early
2015,
we
we
sort
of
split
the
model
on
its
head
a
little
bit
and
actually
the
centralized
operations
organization
became
a
group
of
advisors.
A
Organization
anymore.
We
we
have
a
group
or
a
number
of
software
engineering
teams
who
are
fully
responsible,
end-to-end
responsible
for
software,
so
they
do
all
the
operational
and
running
and
marring
deployments,
and
all
all
that
good
stuff
and
so
the
other
term.
The
other
trend
that
we've
sort
of
been
observing
in
the
industry
for
the
past
and
as
long
as
I
can
remember,
but
especially
now
more
recently
and-
and
this
is
how
we've
been
sort
of
tracking
that
is
when
I
started
at
the
company.
We
had
two
miles.
A
We
started
doing
introducing
these
new
features
that
we
were
building
or,
as
we
were,
building
new
features
and
changing
existing
features.
We
started
building
them
as
services,
and
so
that
was
sort
of
when
we
were
like
okay.
This
is
this
is
like
a
way
we
could.
We
could
actually
start
building
new
stuff
and
then
sort
of
towards
the
end
of
that
year
and
the
beginning
of
the
next
following
year.
A
What
we
started
doing
is
just
making
waiting
in
the
service,
so
we
were
no
longer
really
while
we
were
still
contributing
to
the
malas
codebases
we
were
actually
starting
to
just
everything
would
be
a
service
on
its
own
right.
We
do
right
any
new
features
into
the
individuality
code
basis
and
and
sort
of
a
little
bit
more
recently.
A
A
The
problem
with
all
this
is
that
you
combine
the
fact
that
we,
the
software
engineers,
were
responsible
for
the
whole
gamut
of
things
that
were
related
to
their
services,
with
the
fact
that
we're
shipping
more
and
more
services,
so
you
know
you
might
have
in
the
good
old
days
you
might
have
had
with
10
engineers
to
a
monolith
or
maybe
100
engineers
to
my
lab.
Now.
A
Imagine
you
flip
that
on
its
head,
where
one
engineer
is
responsible
for
three
to
five
to
ten
services,
the
amount
of
time
you
actually
spend
doing
all
that
automation
and
that
operationalization
of
your
services
just
grows
sort
of
not
exponentially
but
very
quickly.
The
fixed
overhead
becomes
a
really
big
pain
point,
and
so
you
know
when
we're
getting
into
it.
We
were
like.
A
Okay
sure
we
just
do
a
little
bit
of
automation
ahead
of
time
and
they'll
pay
off
for
ember
and
it's
going
to
keep
going,
but
in
reality
what
was
actually
happening
is
more
what
you
see
the
in
the
bottom
xkcd
cartoon,
and
so
you
know
we
came
to
the
conclusion
that
there
has
to
be
a
better
way.
So
myself
and
another
engineer
sat
down
towards
the
tail
end
of
2015
and
we
started
thinking
about
right.
What
can
we
do
here?
We?
A
Obviously
this
isn't
going
to
be
a
sustainable
way
to
keep
shipping
new
software,
as
we
were
growing,
more
and
more
services,
and
so
we
we
started
looking
around
and
figuring
out
what
what
the
rest
of
the
industry
was
doing,
and
so
we
we
sort
of
came
up
with
these
four
different
things
that
we
could
do
start,
starting
with
them
maintaining
status
quo,
which
means
that
we
weren't
just
going
to
do
anything
we
just
let
it
be.
You
know
who
cares?
We
could
just
hire
more
people
and
they'll
they'll
do
it.
A
A
We
wanted
to
set
aside
some
requirements
or
some
things
that
we
would
use
as
the
guiding
principles
for
for
making
this
decision,
and
it's
a
first
and
foremost
the
most
important
thing
that
we
realized
we
needed
was
a
a
project
or
set
of
tools
that
work
had
a
really
active
community
in
ecosystem
around
them,
because
we
we
didn't
want
to
be
the
only
ones
using
it.
We
didn't
want
to
be
the
only
ones
contributing
to
it.
We
wanted
to
make
sure
that
there
is
life
beyond
just
our
company
on
that
tool.
A
So
we
had
to
be
a
little
bit
careful
about
how
we
would
how
we
would
approach
that.
The
other
thing
we
wanted
to
do
is
we
wanted
to
treat
whatever
tooling
we
have.
We
ended
up.
Picking
is
just
another
service.
It
should
be
automatable
with,
with
the
node
same
way
that
we
would
build
automate
our
internal
software
or
interact
with
our
internal
software.
Instead
of
you
know,
slinging
configuration
files
around
or
something
like
that.
We
really
wanted
it
to
be
something
that
we
could
drive
with
an
API.
A
Even
though
you
could
argue
that
you
could
make
an
API
around
the
configuration
files,
but
you
know
I
digress.
First
class
container
support.
We
had
already
started
using
containers
for
our
build
pipelines
for
for
the
past
year
and
a
half,
and
we
had
really
good
success
with
that
and
and
it
just
felt
like
a
really
good
fit.
What
it
would
allow.
A
Our
software
teams
to
do
is
actually
sort
of
take
ownership
of
the
know
if
their
dependencies
and
package
that
into
the
container
and
then
we
would
just
give
them
a
tool
to
ship
the
container
with,
and
that
was
sort
of
like
the
contract
that
we
were
looking
for
and
then
finally
it
had
to
be
production
grade.
You
know
it
has.
A
We
shouldn't
be
the
only
company
or
hopefully
not,
that
largest
companies
running
it
in
production,
and
so
we
wanted
to
have
something
that
had
other
people
using
it
and,
and
there
were
stories
and
other
people
we
could
talk
to
about
their
learnings.
From
from
you
know,
the
early
stages
is
testing
to
going
to
production,
and
so
those
were
sort
of
our
rough
requirements.
So
we
looked
at
the
number
of
projects
we
looked
at
nomads,
but
it
was
had
just
come
out.
A
So
I
was
produced,
pretty
green,
we
looked
at
Apache,
Aurora
turned
out
to
be
pretty
tricky
to
get
installed
and
going,
and
so
ultimately
that
was
sort
of
where
we
kind
of
didn't
really
pursue
it.
We
looked
at
marathon,
which
we
just
pretty
promising.
It
showed
a
lot
of
promise,
so
that
was
one
we
we
definitely
considered
pretty
heavily
and
dr.
swarm
along
with
kubernetes.
The
dr.
A
swarm
is
I
believe
it
was
just
about
to
go
through
every
right
or
had
just
gone
through
every
right,
so
we
weren't
super
comfortable
with
it,
but
you
know,
obviously,
today
it
is
a
lot
more
mature
state,
but
we're
at
cue
console
I'm
sure
you
can
imagine
what
we
you
can
probably
figure
out.
What
we
went
with
and
so
kubernetes
was
our
choice.
A
However,
the
the
experience
that
we
had
had
from
using
other
configuration
management
tools
told
us
that
we
couldn't
just
take
the
tool
as
it
is
out
of
box
and
just
give
it
to
everyone
as
it
is.
So
what
we
started
thinking
about
is
like
what
is
it
really?
What
is
the
kind
of
user
experience?
What
is
the?
How?
What
is
it
that
we
want
to
give
our
end
users,
because
the
goal
of
this
team
that
that
was
put
together
for
doing
discovery
around
what
we
could
do
with
our
next
generation
deployment
and
operations?
A
The
goal
was
to
make
it
easier
and
simplified
so
that
we
didn't
have
to
deal
with
this
fixed
overhead
of
you
know
with
the
with
every
service,
and
so
we
came
up
with
this
very
small,
relatively
small,
set
of
guiding
principles
that
that
were,
we
were
using
as
the
way
to
come
up
with
the
solution,
and
so,
first
and
foremost,
the
user
experience
that
we
wanted
to
provide
had
to
be
simple.
It
had
to
be
easier
than
the
tool
itself
out
of
the
box
if
we
weren't
able
to
deliver
that.
That
would
be.
A
A
Let's
do
that
and
then
finally,
we
wanted
to
make
deployments
declarative,
meaning
that
every
time
somebody
would
change
their
application,
we
would
consider
that
a
new
version
of
the
application
and
deploy
deploy
a
new
version
of
the
application
we
didn't
want
to
keep
changing
and
mutating
things
in
place.
So
essentially,
what
we
wanted
to
get
to
is
a
place
where
the
where
the
application
deployments
were
immutable,
and
so
anytime
anything
needs
to
be
changed.
We
could
just
deploy
a
new
version,
and
that
should
be
easy.
A
So
what
we
came
up
with
is
this
tool
called
GOCC.
It
stands
for
dissolution
control
center,
but
we
pretty
much
dropped
using
that
name,
because
it's
too
long-
and
so
we
just
call
it
GOCC-
and
it's
just
a
happy
accident-
that
it
sounds
like
dock,
which
is
a
nautical
thing,
we're
kind
of
a
nautical
company
in
that
sense,
and
so
what
it
is.
It's
just
a
simple
tool
for
deploying
to
terrorize,
stateless
applications
and
so
for
CC.
A
So,
instead,
what
we
came
up
with
this
is
this
idea
of
the
single
manifest
file,
and
so
the
manifest
file
is
essentially,
it
takes
all
the
Cabrini's
files
to
get
it
put
together
and-
and
it
allows
the
user
to
describe
your
application,
a
way
that
tells
us
what
how
they
want
their
application
deployed.
So
it
does
Spacek
things
like
declaring
ports
declaring
scale
of
application,
declaring
how
it's
exposed
to
the
internet
or
outside
of
the
cluster
simple
things
like
that.
A
In
addition,
the
manifest
tells
us
how
we
can
use
how
we
can
take
your
application
integrated
with
pre-existing
systems
that
we
had
in
place.
So
we
have
centralized
logging
pipelines,
we
have
metrics
collection,
alerting
pipelines.
We
have
monitoring
things,
so
we
just
wanted
to
make
sure
that
we
were
playing
nicely
with
the
existing
infrastructure.
We
weren't
looking
to
replace
everything
that
we
already
had
in
place.
A
A
So
it
was
really
important
for
us
that
when
a
service
were
was
behaving
poorly,
we
could
pinpoint
who
is
actually
responsible
for
that
service
and
actually
be
able
to
page
that
right
person
or
when,
let's
say
operations,
went
to
look
at
an
application
that
was
behaving
perhaps
in
an
odd
way.
They
would
know
who's
responsible
for
it
and
it
wasn't
a
guessing
game
of
like
ask
somebody,
and
they
will
tell
you
to
talk
to
another
person.
Who'll
tell
her.
You
talk
to
another
person,
we
didn't.
A
We
wanted
to
short-circuit
all
that,
and
so
what
we
did
is
we
basing
this
idea
of
ownership
directly
into
the
tool
and
so
I'll
walk
you
through
a
little
example.
What
an
potential
application
could
look
like,
so
the
the
file
you
see
here
is
it's
JSON,
but
it's
everything
you
need
to
basically
describe
your
application
for
our
system
to
be
able
to
deploy
it,
and
so,
as
you
can
see,
it's
pretty
simple
and
there
isn't
a
whole
lot
of
information.
You
know
we
need
to
know
the
name
of
your
application.
A
We
want
to
know
the
scale
of
your
application,
what
containers
the
application
is
made
up
of
and
what
version
of
those
containers
and
then.
Finally,
as
I
mentioned,
we
have
the
idea
of
a
maintainer.
Now,
typically,
we
want
maintainer
to
be
teams
and
not
people,
just
because
it's
easier
to
page
teams,
that
is
to
page
individuals,
and
so
you
wouldn't
normally
have
a
individual
like
myself
on
there,
but
you
instead
of
like
a
piece
and
so
what
our
service
then
does.
A
Is
it
takes
that
manifest
file
that
we
just
saw
and
it
creates
a
deployment
for
it,
which
then
obviously
creates
a
replica
said
instead
of
pods.
So
it's
very
simple
not
no
magic
there,
and
you
could
definitely
do
this
with
keep
TTL
just
as
easily.
However,
where
I
start
to
get
a
little
bit
more
interesting
is,
when
you
start
doing
other
things
like
declaring
a
set
of
ports,
and
so
let's
say
you
wanted
to
make
your
application
available
at
8080.
A
What
that
would
then
do
is
when
you
push
this
file
this
manifest
to
the
docc
server.
What
it
would
do
is
actually
create
a
service
which
has
an
endpoint
that
points
back
to
the
pods
and
that's
pretty
simple,
but
it's
just
one
line
change
instead
of
you
having
to
introduce
yet
another
kubernetes
primitive
you
just
added
one
line
to
the
configuration
and
off
the
goes
and
figures
it
out
for
you
now.
Let's
say
you
wanted
to
go
and
actually
exposure
your
service
outside
of
the
cluster
that
it's
running
in
so
we
have.
A
You
can
route
HTP
to
it
and
then,
when
traffic
hits,
this
address,
send
it
back
to
my
container
running
on
port
8080
in
the
pod
basic
stuff,
and
what
they'll
do
is
they'll
actually
go
and
create
a
numerous
for
you,
which
points
to
the
service,
which
then
sends
the
traffic
to
your
pods,
simple
stuff.
But
again
it's
only
a
couple
lines
of
text
instead
of
having
to
define
yet
another
object
and
then
one
of
the
cool
features
that
was
actually
added
pretty
recently
to
this
service
of
ours.
A
Is
we've
actually
created
these
standard
libraries
that
we
use
within
the
company
that
will
pull
out
some
of
these
concepts
and
it'll
automatically,
for
example,
if
the
secret
gets
rolled
it
will
automatically
reload
the
service
underneath
without
you
having
to
do
anything.
So
we
use
vault
for
generating
and
storing
new
secrets
and
vault
has
this
idea
of
kale
TPL,
and
so
when
the
TTL
expires,
we
get
to
your
new
cert
and
then
we
placed
it
in
the
pod
and
you
won't
be
any
wiser
for
it.
A
But
euro
service
will
keep
running
no
problem,
and
so
you
might
be
wondering
you're
like
well.
What
does?
What
are
we
required
for?
The
application
is
using
this,
so
we
went
with
the
twelve
factor,
ish
approach,
so
it's
a
the
reason
being
is
very
easy
to
talk
about
it.
It's
very
easy
to
reference
twelve
factors.
A
A
A
So
what
we
founded
that
in
in
practice
about
90
to
95
percent
of
our
applications
actually
fit
this
bill
to
spine,
and-
and
that
means
that
the
vast
majority
of
the
services
we
ship
at
the
dissolution
can
be-
can
be
deployed
the
docc
quite
easily.
So
you
might
be
wondering
alright.
Why
would
you
do
this?
Well,
there's
a
couple
reasons
for
us,
and
this
is
actually
true
for
communities
as
well,
depending
on
how
you
use
it.
A
It
allows
for
separation
of
responsibilities,
meaning
that
the
people
who
are
responsible
for
applications
can
be
solely
focused
on
the
applications
and
none
of
the
surrounding
infrastructure
that
that
powers
that
application
and
then
finally
we
have
we
can
we
have
this
team
who's
solely
focused
on
providing
the
service
and
the
clusters
that
we
have
instead
of
worrying
about
the
application.
So
there's
no
handoffs
between
the
two
of
them.
It's
fully
self
service.
A
It
allows
us
to
so
one
of
the
reasons
why
we
wrote
the
OCC
early
on
was
we
weren't
quite
sure
if
kubernetes
was
going
to
be
ultimately
the
right
fit
for
this?
So
we
we
kind
of
do
they
resolve
this
obstruction
so
that
we
could
potentially
switch
it
out
later
to
be
something
else
if
we
wanted
to
now
in
reality
that
hasn't.
A
As
you
can
see,
the
JSON
file
that
you
have
to
define
is
not
a
lot
of
not
a
lot
of
text,
but
it
does
pretty
powerful
things
for
you,
and
so
I
want
to
kind
of
show
you
how
the
system
Oh
actually
forgot
that
I
have
a
slide
for
you
for
how
the
command
line
actually
looks.
So
it's
a
very
simple.
It
looks
kind
of
like
UGL,
it's
not
quite
as
sophisticated,
but
it
has
a
bunch
of
commands
that
you
would
normally
do
in
the
operating
application.
A
So
it
allows
you
to
initialize
your
application
inspected
with,
what's
running
and
delete
and
deploy
and
all
those
good
things
that
you've
come
accustomed
to
with
communities
and
if
I
wanted
to
actually
shift
the
application
that
I
just
showed
you
I
would
just
run
this
command.
It
would
be,
it
would
be
up
and
running
available
on
the
Internet.
So
the
way
the
system
works
is
at
the
top.
You,
as
the
happy
user,
have
a
JSON
manifest
that
you
point
give
to
our
OCC
CLI
and
then
that
talks
to
our
docc
server
and
the
docc
server.
A
As
you
notice,
the
only
thing
that
talks
to
the
cube
abs
are
read
it,
so
our
vast
majority
of
our
users
do
not
ever
interact
with
kubernetes.
They
don't
know
anything
about
kubernetes
other
than
by
asking
us
questions
when
when
they
are
like,
what's
actually
underneath
the
hood
and
then
the
do
to
CC
server
is
actually
responsible
for
coordinating
both
with
the
API
server,
as
well
as
the
external
services
that
we
we
plug
into
the
system
and
and
within
all
of
our
kubernetes
clusters.
We
actually
have
a
set
of
controllers,
including
an
interest
controller.
A
Actually
a
couple
of
ingress
controllers
and
then
some
third-party
resources
which
actually
make
all
this
happen.
So
it's
all
in
all
it's
very
simple,
but
but
it
just
creates
that
separation
and
gives
us
a
way
to
manage
and
really
curate
the
experience
that
are
in
users
hat
and
then
so.
The
OCC
server
is
just
a
very
fairly
simple
piece
of
software,
which
essentially
translates
between
the
concepts
that
we've
created
within
docc
in
the
manifest
and
and
the
kubernetes
itself.
So
you
know
it
takes
survey
create
service.
A
A
The
cluster
administrators
are
like,
what's
actually
deployed
by
us
versus,
what's
deployed
by
GOCC
and
then
finally,
it
does
the
coordination
between
external
services
like
volt
and
and
other
fun
things
like
that
to
make
all
this
work
smoothly,
and
so,
at
the
end
of
it
once
you've
deployed
something
you,
you
get
a
dashboard
that
looks
like
this
or
one
of
the
dashboards
that
you
get.
That
looks
like
this.
A
So
in
this
case,
I
took
a
screenshot
of
an
application
that
was
deployed
just
recently,
and
so
what
you
can
see
is
this
dashed
line
of
indicating
deploy,
and
then
you
can
start
seeing
the
metrics
that
are
the
the
container
metrics
that
are
emitted
from
that
application
very
simple
stuff.
But
everyone
gets
this
for
free
without
having
to
do
anything
for
their
application.
They
don't
have
to
go
and
configure
Prometheus
or
our
stats
during
these
other
things,
it's
just
they're
ready
to
go.
A
So
it's
really
nice,
because
when
when
we
have
people
who
might
have
to
look
at
other
applications,
they
didn't
trust
that
they
have
a
standardized
dashboard
across
the
board
available
for
them
and
no
matter
which
application
they
look
at.
So
obviously
the
application
team
software
teams
actually
do
build
other
dashboards
around.
In
addition
to
this,
but
this
is
given
for
free
without
anybody
having
to
think
about
anything.
So
it's
it's
readily
available.
A
The
other
thing
that
we
do
is
we
actually
take
the
log
that
they
expose
in
their
in
their
container
and
we
pipe
them
into
our
centralized
logging
system,
and
they
can
so
they
have
this
Cabana
dashboard
that
they
can
hop
on
using
the
command.
Do
CC
open,
logs
and
it'll
just
automatically
open
the
logs
for
the
application
in
question
in
the
cluster
that
they've
they've
pointed
it
to,
and
and
so
what'll
or
often
they
look
like.
Is
this?
So
it's
all
there
automatically
done
without
them.
A
Having
to
worry
about
anything,
have
no
configuration
zero
configuration
to
worry
about.
Basically,
so
you
might
be
wondering
right.
So
is
this
really
worth
doing?
It
seems
like
a
lot
of
work
and,
to
some
extent,
yeah.
It
is
a
little
bit
of
work,
but
what
we've
seen
is
is
in
sort
of
practice
now.
Is
that
every
new
application
that
we
deploy
added
lotion
unless
it's
a
saiful
application
is
deployed
with
do
CC
no
questions?
Nobody
has
been.
A
No
one
deploys
their
software
with
the
old
way
of
deploying
things
anymore,
just
because
it
does
it's
not
worth
it
for
them.
The
the
big
thing
that
for
us
that
we've
changed
with
this
for
us,
was
that
the
deployment
of
new
services
is
no
longer
bomb.
Like
you
know,
we
at
worst
case
we
had
situations
where
it
might
take
as
long
as
or
if
not
more
than
sorry,
but
deployment
and
orchestra
operationalizing
an
application
might
take
as
long
as
or
more
than
actually
writing
the
software
that
you're
operationalizing
in
the
worst
case.
A
So
it
could
be
days
to
actually
operationalize
what
these
software
that
and
that's
really
that's
a
really
terrible
overhead
to
be
paying
for
so
like
small
micro
services
and
finally
I
guess
next.
The
other
thing
we've
noticed
is
that
existing
applications-
people
are
much
deploying
their
software
much
more
frequently
because
they
don't
have
to
do
it,
make
any
big
changes
into
like
version
control
or
other
things,
get
pr's
up
and
and
approved,
and
things
like
that
they
could
change
the
variable
and
deploy
the
software
and
off
it
goes.
A
We
also
have
a
bunch
of
CI
pipelines
that
just
do
all
this
automatically
for
every
team,
so
they
don't
have
to
worry
about
it
too
much,
and
so
what
we're
seeing
is
about
25
deploys
a
day
which
is
which
is
obviously
not
a
gazillion
deploys,
but
it's
still
a
pretty
good
number
for
us
because
most
of
you
that
services
don't
need
to
change
very
often,
but
when
they
do,
it's
like
it's
very
easy
to
do
that.
We're
seeing
somewhere
around
seventy
five
applications
per
cluster
at
the
most
and
some
clusters
have
a
little
bit
less.
A
We
have
about
16
clusters,
all
together
across
all
of
our
availability
or
data
centers,
and
one
of
the
things
that
was
really
nice
is
migrating.
Applications,
one
from
one
location
to
another,
has
become
just
a
matter
of
changing
your
deployment
target
like
you,
don't
have
to
worry
about
provisioning
and
bootstrapping
and
all
the
active
stuff
of
of
new
nodes.
You
just
say
I
want
to
deploy
this
cluster
and
also
goes.
A
A
This
is
like
the
tool
that
I
enjoyed
the
most
using
it
at
the
company
and
so
they're
really
really
embracing
it
and
they're
really
being
getting
a
lot
happier
in
doing
their
job,
because
they
can
now
focus
on
writing
the
stuff
that
really
matters
and
so
I'll
leave.
You
with
this
quote
from
a
colleague
of
mine,
reduce
the
developer
decision
for
Keeks,
so
they
can
make
decisions
that
matter
and
I
added
the
Pitts
bit
to
the
business.
A
So
really
what
what
we've
done
is
we've
made
it
so
that
people
don't
have
to
think
about
operationalizing
their
software
anymore.
They
can
now
focus
on
actually
making
the
software
and
and
making
sure
that
it
actually
does
the
thing
that
we
built
it
for
rather
than
worrying
about
the
fact
they're
right
after
they've
written
it,
they
have
to
spend
another
day
or
two
actually
operationalizing
it
and
bootstrapping
servers
and
all
those
all
those
things.