►
From YouTube: Keynote: Kubernetes: This Job is Too Hard: Building New Tools, Patterns and Paradigms to Democratize
Description
Keynote: Kubernetes: This Job is Too Hard: Building New Tools, Patterns and Paradigms to Democratize Distributed System Development - Brendan Burns, Distinguished Engineer, Microsoft
The simple truth is that there are more reliable online systems that need to be built then there are people who know how to build them. Building a distributed system is bespoke, manual and hard.
Fortunately, with the development of containers and Kubernetes, a foundation has been created for a new type of development environment to make building systems dramatically easier and more modular. But containers and Kubernetes, while necessary, are not sufficient. In this talk I introduce Metaparticle, a new standard library for easy distributed systems development on Kubernetes.
Metaparticle uses familiar, standard programming languages to enable developers and architects to design, develop and deploy their application from a single, easy to use environment.
About Brendan Burns
Brendan Burns is a software engineer at Microsoft Azure and co-founder of the Kubernetes project. Before Kubernetes he worked on search infrastructure at Google. Before Google he was a professor at Union College in Schenectady, NY. He received his PhD in Computer Science from the University of Massachusetts Amherst and his BA in Computer Science and Studio Art from Williams College.
Join us for KubeCon + CloudNativeCon in Barcelona May 20 - 23, Shanghai June 24 - 26, and San Diego November 18 - 21! Learn more at https://kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy and all of the other CNCF-hosted projects.
A
A
Why
am
I
here
and
and
I
wanted
to
tell
you
that
the
reason
that
I'm
here
and
the
reason
that
this
talk
is
about
it
or
the
reason
that
all
of
the
things
that
we
did
to
lead
up
to
this
moment
happened
is
because
I'm
incredibly
excited
about
empowering
people
to
build
systems
that
they
might
not
otherwise
have
been
able
to
build.
I
think
there
was
a
lot
of
great
stuff
that
been
talked
about
about
how
some
of
these
tools
allow
you
to
debug
systems
that
you
might
not
otherwise
be
able
to
debug.
A
Today,
I
want
to
talk
about
how
we
can
enable
people
to
build
systems
that
they
might
not
otherwise
be
able
to
build.
And
two
years
ago,
at
the
initial
cube,
con
I
said
that
I
wanted
to
make
distributed
systems
a
cs101
exercise.
I
think
that's
still.
The
mission
and
I
think
that,
sadly,
maybe
we
haven't
made
as
much
progress
on
that
as
as
we
might
have
hoped
or,
as
I
might
have
hoped,
so.
A
I
want
to
take
step
back
today
to
talk
about
some
new
ideas
that
have
been
bubbling
around
in
my
head
for
a
while
about
how
we
can
work
towards
that
mission.
I
wanted
to
start
motivating
this
by
taking
a
look
at
the
process
of
development
in
a
cloud
cloud
native
world
right
now.
Right
there
is
your
code,
there
is
a
compiler
that
turns
it
into
a
binary
that
you
then
write
a
docker
file
for
and
the
docker
tool.
A
It
turns
it
into
a
dock
image
that
you
push
to
a
container
registry,
then
there's
kubernetes,
which
is
the
cluster
and
acute
control
and
a
yamo
config.
There's
a
lot
of
stuff
right
and
a
lot
of
tools
and
a
lot
of
different
languages.
Right
you've
got
at
least
three
files.
You've
got
at
least
three
languages,
and
you've
got
at
least
three
tools,
and
in
many
cases
you
have
many
more
right.
This
is
even
before
you
throw
in
a
CDI
CD
system
or
any
kind
of
higher-level
development
system.
A
It
is
no
wonder
that
it
is
too
hard
to
build
distributed
systems
today
the
set
of
tools
alone
and
by
the
way
there
are
three
of
these
and
each
one
of
them
has
like
five
different
choices
that
you
could
make
right.
So
the
the
set
of
decisions
you
have
to
make
before
you
even
start
thinking
about
your
application
is
extraordinarily
daunting
and
it's
worse
than
that.
It's
not
even
just
about
sort
of
cognitive
overload.
It's
actually
about
the
fact
that
we
have
information
scattered
across
all
of
these
different
systems.
A
Consider
like
the
port
that
your
server
listens
on.
Well,
it's
in
your
code!
Clearly,
because
you
have
to
listen
on
that
port
in
your
code,
it's
probably
in
your
docker
file,
so
that
you
say
that
I'm
exposing
this
port
in
the
image.
It's
definitely
in
your
Cuban
Edison
fig,
so
that
it
tells
the
service
what
port
to
talk
to
and
there's
no
linkage
that
draws
those
pieces
together.
A
So
I
have
inferred
that
you
go
and
do
something
like
change
port
80
to
a
port
88
and
remember
to
do
it
in
your
code
and
remember
to
do
it
in
your
docker
file
and
forget
to
do
it
in
your
kubernetes
config
right,
you're
gonna
push
that
config
out
service
isn't
going
to
work.
Hopefully
you
catch
it
early,
but
the
very
fact
that
you
have
to
cut
and
paste
that
value
across
all
of
these
different
systems
points
to
the
fact
that
the
job
is
too
complex.
A
I'm
gonna
highlight
that
with
an
arrow
oops
all
right,
so
this
leads
to
one
of
the
first
principles.
Cuz
I've
been
thinking
a
lot
about.
Why
is
this
hard?
Why
is
this?
What
can
we
do
to
make
this
easier?
And,
and
what
can
we
do
to
make
this
better
and
it
led
to
the
the
first
principle
that
I
that
I
had
that
wanted
to
guide
the
system
that
we're
gonna
build,
and
that's
I
really
think
we'd
have
to
have
everything
in
one
place.
A
We
have
to
have
one
way
of
specifying
the
port's
that
we
listen
on
one
way
of
specifying
the
topology
of
our
application.
One
way
of
talking
about
where
we
want
images
to
reside.
We
have
all
of
these
different
pieces
that
are
scattered
across
all
of
these
different
files.
We
really
need
to
have
a
single
source
of
truth,
a
single
place
where
we
go
to
to
understand
our
application.
You
might
be
deploying
a
replicated
system
with
sharding
and
things
like
that.
A
It's
not
like
the
code
could
be
used
in
a
different
environment
and
yet
you've
represented
the
deployment
of
that
code
in
a
llamó
file.
While
the
code
itself
is
in
some
programming
language,
it's
just
not
a
recipe
for
success,
but
at
the
same
time,
we
other
approaches
to
this
across
the
history
of
cloud
platform.
A
Is
a
service
and
other
things
that
try
and
kind
of
give
you
this
everything
in
one
place,
opportunity,
and
but
we've
also
seen
that
those
are
ultimately
restrictive
in
ways
that
that
lead
people
back
towards
more
raw
infrastructure,
and
so
what
that
leads
to
is
the
second
principle,
which
is
that
we
need
to
build
libraries,
not
platforms.
I
need
to
be
able
to
opt
in
to
the
pieces
that
are
useful
to
me.
A
Leave
behind
the
pieces
that
are
not
and
have
escape
hatches
in
the
right
places
so
that,
when
I
hit
the
walls
of
the
environment,
I
have
places
to
go.
So
if
principle,
one
is
that
we
need
everything
in
one
place,
principle
two
is
every
once
in
a
while:
I
need
to
go
someplace
else,
and
then
the
third
part
of
this
is,
as
we've
seen
this
incredible
community
to
develop.
As
we've
seen,
all
of
these
people
come
together
with
all
of
these
great
ideas
to
build
things
that
we
never
even
thought
of.
A
We
need
to
encourage,
sharing
and
reuse
across
the
history
of
compute.
The
way
that
we
make
things
easier
is
by
building
a
single
implementation
that
everybody
shares
right.
It
used
to
be
long
ago
that
people
build
their
own
gooeys
people
built
their
own
button,
implementations
people,
you
know
splatted
pixels
directly
onto
the
screen.
No
one
does
this
anymore.
We
all
use
windowing
toolkits,
because
that's
expected
library,
it's
a
shared
library.
There
are
people
who
focus
on
that,
but
most
of
us
simply
consume
that
we
need
to
think
more
as
we
build
distributed
systems.
A
A
Well,
I!
Think
it's
all
leading
and,
as
I
see,
different
places
are
around
in
the
container
space
and
elsewhere.
I
think
it's
all
leading
to
what
I've
been
calling
lately
language
idiomatic
cloud
I
think
that
there
is
no
reason
that
the
one
place
where
you
need
to
build
everything
shouldn't
be
a
programming
language
right.
A
We
know
programming
languages.
Well,
anybody
who's
in
in
the
config
space
and
the
llamó
space
understands
that
configuration
ultimately
becomes
a
turing-complete
programming
language,
it's
just
a
very
poorly
designed
one
right.
So
why
not
start
with
a
language
that
has
been
designed
with
these
these
thoughts
in
mind?
Why
not
start
with
languages
where
there's
already
tooling
and
editors
there's
already
unit
tests?
How
many
people
at
their
unit
tests
their
configurations
I'm
guaranteeing
you?
A
The
number
is
a
lot
lower
than
the
number
of
people
who
unit
test
their
code
and
yet
shouldn't
we
be
unit
testing
them
both
and
that's
only
possible
I
mean
it's
not
only
possible,
but
it's
significantly
easier.
If
you're
already
in
a
programming
language
that
has
unittest
libraries
that
has
unit
test
frameworks.
All
of
these
benefits
that
we
get
from
the
programming
language
should
apply
to
the
configuration
and
deployment
of
our
systems
as
well.
Additionally,
they're
more
approachable
right.
A
There
are
a
large
group
of
people
out
there
who
are
lucky,
if
they've
sort
of
managed
to
a
managed
master,
single
programming,
language,
asking
someone
to
learn
a
programming
language
and
then
a
docker
file
language
and
then
a
yeah
mole
configuration
language
as
well
as
all
of
the
tooling
that
goes
with
them
is
simply
reducing
the
set
of
people
to
whom
this
technology
is
approachable.
We
need
to
expand
the
set
of
people
who
can
do
this
not
put
up
barriers
to
entry?
A
A
Keywords
like
synchronized
in
Java,
we're
gonna,
see
cloud
and
cloud
native
abstractions
move
from
the
assembly
language
of
kubernetes
to
being
language
features
of
future
programming.
Languages,
I'd
like
to
point
out
that
I
actually
don't
think
that
this
is
a
new
idea.
Someone
wants
said
the
future
is
already
here:
it's
just
not
evenly
distributed
right.
It's
a
quote
that
a
lot
of
people
have
heard
and
a
lot
something
that
I
think
about
a
lot.
A
If
we
take
a
look
at
things
like
Hadoop
Hadoop
said
hey,
if
you
implement
this
java
class,
the
set
of
Java
classes
I
will
give
you
a
distributed
system,
it's
targeted
to
a
very
specific
use
case
MapReduce,
but
it's
a
clear
example
of
how
you
can
bring
a
language
idiomatic
cloud
to
a
distributed
system.
Environment.
A
More
recently,
there's
been
work
on
this
thing
called
PI,
R
n
that
some
people
may
have
looked
at
and
they
basically
took
the
Python
map
function
and
they
wrote
a
back-end
that
uses
functions
as
a
service
to
do
parallel
to
paralyze
your
map.
Suddenly
everybody
who
can
write
a
Python
map
function
has
access
to
a
supercomputer
too,
as
access
to
the
cloud
without
them
even
knowing
about
it
right,
this
democratizes
distributed
systems
in
a
way.
A
That
is,
unlike
anything,
that
we
have
seen
up
to
this
point,
but
these
are
targeted
to
very
specific
use
cases
and
I.
Think
what
we
need
to
do
now
is
to
actually
say
how
do
we
take
these
example
proofs
and
build
the
general
system
and
by
the
way
this
was
always
the
plan.
This
was
something
that
has
been
in
the
back
of
my
head
for
a
long
time
in
containers.
We
have
an
analogue
for
objects.
A
It's
how
it
may
even
be
completely
wrong,
but
I
am
highly
confident
that
something
like
this
will
be
the
way
that
we
actually
build
bring
cloud
native
computing
to
the
world,
to
cs101,
to
hobbyist
programmers
to
my
kids
in
middle
school.
This
is
going
to
be
the
language.
Idiomatic
cloud
is
going
to
be
the
way
that
we
bridge
that
gap
all
right,
give
you
an
example
of
how
this
works.
I
want
to
take
a
look
at
a
very
simple
java
program.
This
is
a
very
simple
web
server
in
Java.
A
A
A
When
you
run
this
program,
you
get
a
container
image
pushed
to
a
registry
right
by
adding
these
annotations
I've
actually
created
an
executable
that
knows
how
to
package
itself
as
a
container
knows
how
to
push
itself
up
to
the
registry
and
by
the
way
this
isn't
just
in
Java.
You
can
do
the
same
thing
in
JavaScript
right.
The
same
idea,
I'm
going
to
I,
don't
have
annotations
in
JavaScript,
but
we
have
this
pattern
of
function,
pointers,
options,
objects,
I
can
pass
the
same
information
that
I
passed
in
that
annotation
as
an
options
object
to
JavaScript.
A
It
will
package
itself
as
a
container
push
itself
up
to
a
registry.
It's
very
easy
to
see
that
not
only
is
information
like
the
port
number
preserved
through
the
code
to
the
docker
file,
ultimately
to
the
kubernetes
yamo,
but
also
I,
didn't
have
to
teach
anybody
anything
if
they
knew
JavaScript
they
can
containerize
their
application
and
I.
Think
that
the
thing
that
that
allows
us
to
do
this
in
all
of
these
languages
and
to
be
idiomatic
within
all
of
these
languages
is
viewing
kerbin
entities
of
containers
effectively
as
a
compiler
target
right.
A
And
so,
if
you
look
at
the
details
of
meta
particle,
this
is
what
it
looks
like
under
the
hood
there's
code
bindings
for
each
language,
there's
an
intermediate
representation,
there's
a
compiler
that
knows
how
to
take
that
intermediate
representation
and
compile
it
into
containers
and
kubernetes
objects,
and
so
obviously
packaging.
Something
up
as
a
container
is
the
extent
of
what
you
wanted
to
do.
We
actually
want
to
be
able
to
deploy
it
to
the
cloud
as
well.
A
So
if
we
take
a
look
here,
I'm
adding
one
additional
annotation
to
my
java
program
that
I
had
before
and
that
I
have
a
runtime
with
replicas
equals
four
right-
and
this
is
said
actually
not
only
do
I
want
you
to
package
it
as
a
container
and
push
it
up
to
a
registry,
but
I
want
you
to
deploy
to
kubernetes
with
four
replicas
and
a
load
balancer
in
front.
So
let's
see
a
demo
of
this
in
operation.
Hopefully,.
A
All
right
so
there's
my
code,
you
can
see
here
the
runtime
annotation.
You
can
see
the
package
annotation.
That's
there.
You
can
see
that
simple
Java
server
quit
there
now
what's
happening
in
the
background,
is
I'm
actually
running
my
application
and
it's
building
itself
to
pick
up
any
changes
to
the
code
that
I
might
have
had
in
just
a
second.
You
can
see
that
it's
building
up
a
jar
in
just
a
second
you're
gonna
see
its
packaging
itself
as
a
container.
A
It's
pushing
itself
to
the
registry
and
in
just
another
second,
it's
going
to
have
deployed
at
the
elf
to
kubernetes
and
it's
detected
the
log
so
something
else
that
it
does
to
give
you
the
illusion
that
it's
running
on
your
machine,
it's
actually
attached
to
the
logs
that
are
coming
up.
All
of
these
containers
and
now
I
can
see.
Actually,
if
I
do
cube,
ctrl,
hey
I've
got
some
services.
I've
got
that
web
service
that's
up
and
running.
A
A
And
there's
my
code
running
on
that
code
that
I
had
in
that
java
application
running
replicated
four
times
up
in
the
kubernetes
cluster.
Just
by
running
that
job
executable
nice
of
you,
it
gets
cooler,
don't
worry,
it
gets
cooler
in
the
four
minutes
that
we
have
left
and
you
can
see
there
that
it's
actually
hitting
multiple
replicas
all
right.
So,
let's
exit
out
of
that
and
go
back.
A
This
is
always
the
part
right
hope
that
doesn't
go
back
to
the
beginning
of
the
technique.
Yes
mastering
PowerPoint
all
right
so,
but
if
you
can
replicate
things,
the
obvious
next
question
is
well
what
about
synchronization
if
you
can
replicate
lots
of
replicas
and
they
run
on
multi
machines.
One
of
the
hard
problems
that
everybody
immediately
runs
into
is
how
do
I
synchronize
things?
Well,
it
turns
out
that
this
needs
to
be
part
of
the
standard
library
as
well.
So
here
it
is
in
JavaScript.
A
I
can
pull
in
the
meta
particle
sync
library
I
can
create
a
lock
I,
give
it
a
function,
call
back
when
the
callback
is
called
I
have
the
lock
and
then
I
can
release
the
lock.
If
I
want
to
do
it
right,
familiar
to
people
who
do
JavaScript
familiar
to
the
JavaScript
idiom.
Of
course,
if
you're
going
to
do
it
in
Java,
it
looks
a
little
different.
A
I'm
gonna
create
a
lock
class
I'm
going
to
be
able
to
lock
it
I'm
gonna,
say:
I
have
the
lock
I'm
gonna
unlock
it
by
the
way
in
Java,
there's,
actually
an
abstract,
lock
class
that
this
lock
influenced
so
I
can
actually
pass
it
anywhere.
That
expects
a
lock
I
can
pass
this
this
class
around,
but
it's
a
distributed
block.
How
is
this
built?
Well
again,
we
want
to
build
it
in
such
a
way
that
it's
language
independent,
so
we're
going
to
use
a
sidecar.
A
A
If
I
have
locks
well,
it
actually
is
quite
easy
to
then
do
election
and
to
do
master
election,
and
so
here
is
the
JavaScript
code
to
do
master
election
pull
in
the
library
again
create
elect
an
election
object.
Give
it
a
name.
Obviously
The
Shard
that
I
want
to
do.
Election
on
first
function
is
called
when
you
become
the
master.
Second
function
is
called
when
you
lose
the
master
now
I
think
it's
clear
to
understand
that
this
is
way
more
approachable
way,
more
likely
something
that
you
can
teach
to
someone
than
giving
them.
A
You
know
the
SCD
manual
and
telling
them
to
figure
out
how
to
do
two
distributed
locks
right
and
so
I
think
again,
what
we're
trying
to
do
is
bridge
the
gap
between
cloud
native
ideas
and
a
language
level.
Primitive
that
understand
and
finally,
I
think
this
is
all
part
of
the
development
of
patterns
for
distributed
systems.
I
think
we're
to
this
point
that
this
the
construction
of
distributed
systems
has
been
artisanal
at
best.
A
There's
a
lot
of
leather
and
copper
and
and
hand
weaving
that's
going
on
as
we
build
distributed
systems
and
I
think
at
some
level
as
craftspeople
we
like
that,
because
it
makes
us
feel
important,
we
need
to
be
in
the
business
of
mass
production.
It's
the
only
way
that
we
will
scale
to
the
number
of
systems
that
need
to
be
built,
and
it's
the
only
way,
honestly
that
we
will
build
reliable
systems.
So
to
give
you
an
idea
about
how
this
looks.
A
Consider
sharding
sharding
is
the
process
of
having
a
request
that
comes
into
a
service
and
then
all
requests
for
a
particular
shard
go
to
a
particular
server.
Different
requests
come
in.
They
go
to
a
different
server,
so
hashing
on
the
user
name,
for
example,
with
meta
particle.
You
can
again
do
this.
The
same
annotations
in
this
case
instead
of
saying
replicas,
equals
4,
I,
say
shards
equals
4
and
I
add
something
else
which
is
the
shard
expression.
It's
a
regular
expression
that
applies
to
the
HTTP
path.
That
tells
me
what
chunk
of
that
request.
A
I
want
to
use
as
my
shard
key.
So
in
this
particular
case,
I'm
charting
on
the
user
name,
every
single
request
for
the
same
user
will
get
sharded
to
the
end,
to
the
same
server
and
again,
because
I
put
it
into
the
language
when
I
compile
and
run
this
application,
and
sadly
I
don't
have
enough
time
for
the
demo.
But
when
I
compile
and
run
this
application,
it
deploys
itself
as
a
sharded
system.
It
deploys
a
load
bouncer,
it
deploys
a
sharding
front-end.
A
It
deploys
the
shards
themselves
as
a
stateful
set
and
it
links
everything
up
with
load,
balancers
and
reusable
container
components
that
are
part
of
this
standard
library.
Let's
skip
through
the
demo
and
riches
of
time
and
I
want
to
say
this
is
really
just
the
beginning.
It's
an
open
source
project,
that's
available
out
on
github
at
a
particle
I/o
or
HTTP
meta
particle,
Daioh,
Twitter
anywhere
else.
Come
talk
to
me
after
the
conference
I'm
incredibly
excited
about
how
together
we
can
build
the
standard
library
that
truly
democratizes
distributed
systems
on
top
of
kubernetes.