►
From YouTube: Effective Development with Kubernetes: Techniques, Tools & Telepresence - Philip Lombardi, Datawire
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Effective Development with Kubernetes: Techniques, Tools and Telepresence - Philip Lombardi, Datawire.io
Every software development cycle is rife with inefficiency. Seasoned devs know the pain of getting access to essential remote systems, waiting for tests to run (and then fail), or debugging with only log files. This talk teaches you how to best leverage Kubernetes, remote infrastructure and related tooling to create a dev cycle that maximizes velocity and minimizes developer friction and frustration. Using tools such as Kubernetes, Docker and Telepresence, I will walk attendees through several advanced techniques that can be used to produce an effective developer experience and optimized dev loop. The goal of this is to eliminate many sources of frustrating inefficiency and reduce cycle time between releases. I will demonstrate how to incrementally adopt some of these techniques and how to approach introducing new and unfamiliar technology and techniques to skeptical dev teams.
To learn more: https://sched.co/GrRL
A
All
right,
let's
get
this
going,
so
welcome
everybody
to
effective
kubernetes
development
or
had
a
turbocharged
or
development
loop.
My
name
is
Philip
Lombardi
or
you
can
tweet
me
at
the
big
lebowski
on
Twitter
I'd
love
to
get
some
feedback
from
people.
I'd
love
to
hear
what
your
thoughts
on
the
slides
are.
If
you
want
to
shoot
pictures
out
all
during
this
thing,
that'd
be
great.
A
So
today
this
talk
I'm
going
to
share
a
cloud
native
workflow
that
we
have
been
building
for
three
years
at
data
wire
I/o.
It
is
a
three
year
endeavor.
This
talk
will
be
kind
of
a
journey
talk,
so
I
will
talk
about.
You
know,
steps
forward
the
steps
backwards,
the
steps
sideways
we
have
taken
and
our
twisting
twisting
journey
towards
a
brighter
future.
Just
a
little
bit
about
data
wire
I/o.
We
are
a
dev,
tooling
company.
A
We
build
dev
tools,
we
don't
really
have
services
so
much
that
public
consume,
but
we
have
products
that
people
use
and
we
have
services
that
we
use
internally.
I
am
an
internal
dev,
tooling
and
infrastructure
engineer
at
data
wire,
so
I
basically
help
the
developers
work
and
be
more
productive,
occasionally
I
hack
on
product
who
I
first
started
three
years
ago.
I
definitely
worked
on
the
product
way.
A
More
often
these
days,
I'm
mostly
concerned
with
how
to
make
my
team
much
more
efficient
when
working
on
cloud
native
software
I
am
definitely
an
impatient
person,
possibly
my
Italian
heritage,
possibly
the
fact
I'm
from
Boston
just
an
impatient
person.
If
you
were
thinking
myers-briggs
I'm.
Definitely
a
J
I
mean
cloud
native
software
makes
me
frustrated
at
times.
Right
now,
we
are
in
the
infancy
of
cloud
native
development.
The
tools
are
not
there
yet
so
there's
a
lot
of
room
for
improvement
and
I'm
really
excited
to
be
able
to
help
out
and
make
things
better.
A
This
is
kind
of
our
typical
development
loop.
Everyone
in
this
room
is
probably
familiar
with
this
I'm,
going
to
call
this
the
internal
or
inner
dev
loop.
It
is
the
thing
you
start
doing
the
moment.
You
get
a
product
feature
request
or
you
get
a
bug
to
work
on.
You
start
writing
code.
You
build
your
code,
you
deploy
your
code
and
then
do
some
kind
of
poking
at
the
code.
A
You
just
wrote
to
see
if
it
even
works
or
does
what
you
expect
it
to
do,
and
you
keep
repeating
this
over
and
over
and
over
now
I
know
when
I
say
dev
tests
I
do
not
mean
formal
tests.
I
am
NOT
talking
about
unit
tests.
I
am
NOT
talking
about
acceptance.
Tests,
I
am
talking
about
the
tests
you
do
with
curl
or
your
web
browser
to
even
make
sure
the
thing
you
thought
you
wrote
even
does
remotely
what
you
thought
it
was
supposed
to
do,
and
so
for
many
people
this
takes
different
forms
code.
A
Some
of
us
right
go
some
of
us
right,
Python,
some
of
us
right,
Kotlin,
some
of
us
being
working
Java
or
whatever.
We
all
have
different
forms
of
building
binaries
docker
images
can
just
be
packaging
up
static
assets.
Deployment
is
also
that
thing
we
do
where
we
have
to
get
the
code
running
somewhere.
It
can
be
a
form
of
coop
cuddle
apply.
It
may
be
something
else.
A
Data
migrations
are
part
of
deploy,
and
then
there
is
the
mythical
stuff
that
we
don't
really
know
what
that
is,
but
our
company
says
we
have
to
do
it
because
the
part
of
the
deployment
phase
and
then
dev
tests
that
takes
many
forms,
curl
may
be
something
formal
may
be
something
not
formal.
It's
also
stuff.
A
A
We
need
to
ship
fast
and
ship,
often,
as
the
matter
really
is,
and
so
our
dev
loop
was
really
primitive
at
the
time
it
was
basically
get
code
off
of
laptop,
get
it
on
the
internet
somewhere
poke
at
it
and
play
with
it
and
see
what
we
had
to
add,
fix
and
stuff.
It
wasn't
a
particularly
advanced
set
of
tools.
It
was
some
shell
scripts.
Some
you
know,
pythons
are
mostly
documented
somewhere.
A
Wasn't,
oh,
you
know
complicated,
and
the
way
we
were
doing
at
the
time
was
building
and
packaging
binaries
and
then
building
virtual
machine
images.
So
we
weren't
even
thinking
about
docker.
At
this
time,
docker
was
still
not
quite
the
thing
we
were
going
like
it's
the
future,
so
we
went
with
something
we
knew,
which
was
building
virtual
machine
images
and
then
deploying
virtual
machine
images
deployment.
A
So
we
immediately
hit
problems
with
this.
Basically,
after
a
few
months
of
working
with
it,
you
became
quite
aware
that
VM
images
take
forever
to
create.
They
are
a
10-minute
endeavor
at
best
they
may
take
even
longer.
If
you
have
complicated
things,
you
have
to
do
on
the
VM
to
get
them
up
and
started
generate
a
snapshot,
tear
them
down
if
the
build
fails,
for
whatever
reason
you
have
to
start
over
again,
it
could
be
ten
minutes
times
number
of
failures.
You
have
plus
the
time
to
trace
whatever
the
failure
was
further.
A
The
failures
don't
often
occur
in
parallel.
They
occur
sequentially.
So
as
you're
debugging,
the
thing
you
realize
well
I'm,
actually
solving
whatever
waiting
again
solving
another
error
waiting.
Think
that's
the
time.
It's
all
done.
Nope
third
error,
so
that
wasn't
great
people
developers
especially
really
hated
working
with
Packer,
which
is
a
hashey
corp
tool
as
part
of
their
developer
loop
now,
I'm,
not
trashing
on
Packer
as
bad
tool.
It's
a
great
tool
for
operations,
and
if
you
have
to
build
virtual
machines,
it's
possibly
one
of
the
only
tools
that
does
it
for
you.
A
It
does
a
really
good
job,
but
developers
don't
want
to
think
about
it.
They
don't
want
to
have
to
think
about
building
a
virtual
machine
image,
so
they
got
really
frustrated
when
they
had
to
start
using
this
tool
and
it
didn't
work
for
them
or
they
had
to
debug
it.
Finally,
the
shell
and
terraform
and
Anza
Bowl
combination.
Well,
it
was
built
really
hastily.
It
was
janky,
wasn't
perfect
and
caused
lots
of
frustrating
problems
for
people.
So
what
did
we
do?
Well?
A
My
first
thing
was:
ignore
all
the
speed,
complaints
naively
and
go
and
slap
a
fresh
coat
of
paint
on
all
this
by
hiding
the
complicated
bits
behind
make
files
in
Gradle,
scripts
and
I
was
basically
of
the
opinion
great.
We
are
gonna
everything's
awesome
mission
accomplished
we're
gonna
move
on
get
on
with
our
lives
and
the
reaction
from
my
CTO
and
my
co-workers
was.
This
is
a
disaster
and
they're
screaming
and
yelling
not
really
screaming
like
screaming
at
me,
but
they're
screaming
at
their
computer
screens
going?
Why
is
this
thing
still
not
working?
A
Why
is
it
still
a
pain
and
I'm
going
everything's?
Fine?
We
hit
all
the
complexity.
I
know
how
the
complexity
works,
but
you
don't
it's
fine.
So
my
lesson
first
lesson
here
was
don't
ignore
my
developers.
I
need
to
listen
to
understand
their
problems
much
much
better
before
we
start
actually
solution
earring,
but
this
creates
two
new
problems.
A
One
our
devs
do
not
always
know
what
their
problems
are,
the
first
place
and
then,
even
when
we
listen
to
our
developers
and
we
even
sit
down
and
agree
with
them
about
what
needs
to
be
solved,
we
fixed
the
wrong
problem.
So
we
got
all
the
greatest
minds
of
the
company
together
and
decided
deployment
was
actually.
The
first
problem
we
were
going
to
tackle
tends
to
be
the
case
that
people
you
know
engineers,
especially
we
look
at
a
problem
ago.
The
janky
thing,
the
thing
that's
got
crufty
areas
or
dark
areas.
A
That's
the
thing
we
should
fix.
First,
that's
the
thing
we
want
to
fix:
it's
not
sexy.
We
want
to
work
at
something,
that's
sexy,
so
we
went
off
and
started
worked
on
deploy
and
we
automated
the
workflow
build
fancy
custom
tools
to
get
around
that.
It's
a
fancy.
Custom
tool
worked,
we
built
this
thing
called
swap
top
I.
Did
all
this
fancy
orchestration
stuff
to
Bluegreen
deployments
for
folks
it
solved
some
pain,
but
it
also
created
a
lots
of
pain
and
new
pain.
A
It
still
doesn't
address
the
real
pain
that
developers
were
actually
having
at
the
company.
So
we
as
smart
people
as
engineers,
sat
down
and
went
well.
We
can
fix
this,
and
so
we
doubled
down
double
down
on
deployment
as
the
problem
through
more
technology
at
its
spinnaker
Jenkins
terraform
ansible
ec2,
another
cloud,
Google
Open,
VPN
I
can
name
probably
20
technologies
that
were
involved
in
what
we
were
going
to
call
the
data
wire
back-end
developer
platform
and
at
the
end
of
the
day
it
still
wasn't
enough.
A
The
expected
outcome
was
this:
everyone's
gonna
be
dancing
we're
all
gonna
be
going.
This
is
great.
We're
shipping
code,
shipping
faster
gonna,
get
the
product
out
there,
and
this
is
actually
my
reaction
for
my
coworker,
who
has
long
hair
going.
What
have
you
created?
You've
created
this
horrifying
disaster,
so
less
than
2.0.
Thankfully,
my
bosses
are
really
patient
people
who
are
interested
in
getting
to
an
actually
good
outcome
versus
just
you
know,
continuing
to
struggle.
A
So
we
learned
we
need
to
do
a
better
job,
identifying
the
real
pain,
the
actual
problems
that
people
are
having
don't
get
to
technology
crazy,
I
know
this
is
a
really
kind
of
an
ironic
statement,
as
we
sit
here
at
this
conference.
Going
in
both
the
vendor
is
looking
at
all
these
fancy
tools
going,
hey
I
should
use
this
I
should
throw
this
in
my
problem.
A
Try
to
actually
you
know,
dial
that
in
don't
don't
just
start
throwing
technology
at
your
problem.
It's
probably
not
going
to
solve
it
so
solution.
3.0
is
this
crap
at
all
and
restart
from
scratch,
and
we
were
very
fortunate
at
the
time
that
we,
the
business,
was
changing
a
little
bit,
so
we
were
given
some
time
to
basically
sit
down
and
reevaluate
where
things
were
slow
and
change
them,
make
them
better.
A
Do
the
actual
right
thing,
the
thing
we
forgot
to
do
in
the
first
place
so
well,
we
finally
came
to
conclusion:
was
the
inability
to
build
quickly
was
killing
us
more
than
deployment?
We
had
all
kind
of
come
from
a
world
where
writing
software
was
basically
a
matter
of
Rights
and
code
click,
a
button
or
kill
a
process
and
restart
that.
So
you
know
the
server
process
come
up.
New
code
is
running.
A
We
weren't
in
that
world
we'd
started
in
the
world
with
virtual
machine
images
that
had
to
be
baked
and
then
run,
and
then
we
got
to
a
world
where
we
were
starting
to
use
docker
a
little
bit
we
weren't
at
in
that
world
we
want
to
be
in
so
we
could
get
sort
of
closer.
We
switched
to
docker
that
cut
down
our
virtual
machine
problem,
so
we
no
longer
had
the
ten
minutes
of
you
know,
waiting
for
the
virtual
machine
to
build.
We
just
had
to
get
docker
images
built
that
shaved
off.
A
You
know
30
minutes
from
the
entire
process.
We
left
the
deploy
thing
kind
of
unanswered
for
a
while,
so
in
2017
early
2017,
maybe
you
could
really
say
late
2016
to
the
answer
to
who
was
gonna
win.
The
container
orchestration
you
know
and
problem
was
just
not
answered
at
all.
There
was
swarm
kubernetes
miso
snow
mad.
It
was
a
open
game,
so
we
kind
of
just
held
off.
A
We
didn't
really
need
to
think
about
at
the
time,
because
we
were
rebooting
the
business
rebooting,
the
engineering
org
mostly
code
and
to
built
worked
well
enough
for
us,
but
at
some
point
kubernetes
did
win.
It
is
now
really
hard
to
dispute
that
it
is.
The
onion
is
the
best
platform
for
running
containers
at
the
moment
and
everyone's
gotten
their
attention
behind
it.
So
we
adopted
kubernetes
and
the
gret.
The
good
news
here
was
scrapping
all
the
stuff
that
we've
done
before.
Moving
to
docker
moving
kubernetes
the
rethink,
it
was
a
huge
success.
A
Devs
were
happy
velocity
increased
if
I
could
have
a
little
chart
with
a
little
bar
graph
going
or
the
little
arrow
going
all
the
way
up
its
through
the
roof
compared
to
where
we
were,
but
developers
as
they
are
always
come
back
to
you
and
say
I
want
more.
I
want
to
be
faster,
and
this
is
not
fast
enough.
We
got
to
the
point
where
we
were
in
around
two
minutes
for
docker
images
to
build,
which
is
a
huge
improvement
from
where
we
were
in
types
of
ten
minutes.
A
Thirty
minutes
for
a
VM
images,
but
what
we
can
can
we
do
and
at
first
I
was
like
this
is
impossible.
You
can't
be
complaining
that
this
thing
is
still
slow.
It's
pretty
fast.
Two
minutes,
even
you
know,
is
just
way
better.
They're,
like
both
problem,
is
we're
going
through
this
loop
faster,
faster
and
faster,
so
were
actually
able
to
cycle
over
and
over
and
over
those
two
minutes
really
add
up,
and
they
were
totally
right.
Compile
the
code
30
seconds,
assemble
binary.
Another
30
seconds
build
your
container
image.
A
Another
30
seconds
push
the
container
image
over
30
seconds.
You
quickly
end
up
at
120
seconds,
which
is
a
totally
not
scientific
number
that
I
came
up
and
pulled
out
and
that's
where
you're
and
then
you
realize
it's
120
seconds.
It's
optimistic.
How
many
of
you
are
in
this
room
are
Java
developers
yeah?
A
You
have
probably
tried
building
Java
apps
in
the
container
and
realized
every
time
you
do
it
because
you
can't
cash
that
maven
repo
you're
pulling
all
those
dependencies
down
again
Python
developers.
How
many
times
have
you
had
to
build?
A
native
extension
had
to
install
through
apt
GCC.
Just
so
you
can
rebuild
Lib
PQ,
so
you
can
have
a
Postgres
library
all
right,
so
you
really
realize
quickly.
These
just
add
up
this
time.
Is
it
leaks
out
around
you?
Also
slow
networks?
Slow
networks
will
kill
you
all.
A
Those
dependency
you
have
to
have
dependencies
have
to
be
pulled
down,
rebuilt
every
time,
same
thing,
that
the
images
have
to
be
pushed
and
then
be
deployed
and
pulled
from
somewhere
every
time.
So
it's
actually
a
hundred
20
seconds
times
number
of
rebuilds
you're
doing,
and
it
adds
up
super
quick.
You
can
in
a
day,
if
you
have
fast
enough
of
luke
burn
an
hour
an
hour
a
day
and
a
week,
5
hours
gone
right
away.
So
we
start
looking
at
ways
we
can
optimize
our
doctor
files.
A
That's
the
simplest
solution,
everyone
you
know
when
we
started
with
docker
everyone
kind
of
does
the
naive
thing:
they
throw
a
bunch
of
instructions
in
a
docker
file
and
they
realize
there's
no
optimisation
around
the
caching
they've
come
up
with
because
they're
rebuilding
all
the
layers
every
time.
So
we
did
the
first
thing,
which
is
logically
reorder
the
instructions
to
get
the
best.
Caching
great,
that
save
some
seconds
next
step.
Multistage
builds
that
allowed
us
to
start
using.
A
You
know
a
base,
JVM
layer
that
we
didn't
have
to
rebuild
every
time
or
base
python
layer
that
had
stuff
built
in
great
small
base.
Images
should
be
self-explanatory.
We
don't
want
to
see
multi-gigabyte
images
floating
around
going
back
and
forth
between
registries
lots
of
time.
You
have
to
push
and
pull
so
we'd
like
to
get
small
base
images.
So
we
worked
on
shrinking
things
getting
to
alpine
using
from
scratch,
where
possible,
using
a
minimal
little
bun
to
we're,
also
possible.
You
know
we
carved
from
basically
having
one
put.
A
You
know
one
gig,
plus
images
down
200
megabyte
images
great
that
solves
the
entire
problem
of
well.
The
pushes
and
poles
are
really
slow,
cuz
moving
a
gig
over
the
Internet,
even
for
a
docker
image
repeatedly
is
a
pain,
but
it
wasn't
enough.
It
only
shaves
some
seconds
and
it's
just
not
enough.
Even
the
fastest
build
still
get
stuck
in
that
sync
of
time
known
as
registry
push,
redeploy
and
registry
pull
it's
the
chunk
of
time
deploy
stuff.
A
We
were
like,
ok,
what
we
haven't
have
you
can
still
try
one
last
thing
we
came
up
with
a
new
plan
which
was
a
background
daemon,
which
would
continually
run
on
developers,
computers
and
it
would
rebuild
images
as
they
coded,
and
this
worked
pretty
well
in
the
form
of
like
you've,
gotten
docker
image
instantaneously.
So
you
didn't
have
to
even
work
on
the
rebuild
part,
but
it
generated
monstrous
sized
images
even
worse
than
what
we
were
getting
before
when
we
tried
to.
A
Basically,
you
know,
shrink
things,
so
we
were
getting
gigabytes
to
gigabyte
three
gigabyte,
five
gigabyte
images,
really
not
good
I,
had
a
co-worker
finally
go
well.
What?
If
we
just
didn't,
build
and
push
images
at
all?
What,
if
you
know
we
could
get
it
basically
at
kubernetes
native
development
experience
where
you
could
code
in
a
way
that
felt
like
you
were
just
running
on
the
cluster
and
the
cluster
was
aware
of
your
code
as
you
worked
on
it.
A
That's
a
really
good
idea:
kind
of
brilliant
gets
us
back
to
the
world
where
we
just
click
a
button
or
launch
a
process,
and
we
see
our
code
running.
We
can
change
it,
reload
it
whatever.
So
what?
If
your
dev
machine
ran
your
code?
It
just
worked
the
same
as
you
ran
on
kubernetes.
We
tried
a
couple
approaches
with
this.
One
of
the
approaches
we
tried
was
the
idea
of
we'll
just
run
the
cubelet
on
your
laptop
and
then
we'll
join
the
laptop
into
your
cluster.
A
That
turns
out
to
be
a
really
not
a
great
idea.
One
CNI
is
a
pain
to
get
working
across
different
networks
to
the
problem
of
different
developers
have
different
setups
for
their
machines,
so
I
have
developers
who
use
Mac,
OS
I
have
developers
that
use
Linux
and
they
might
use
fedora,
they
might
use
Debian,
they
might
use
Ark.
It's
really
kind
of
a
mess
to
even
try
and
think
about
doing
that.
So
we
came
up
with
this
thing
called
telepresence
and
it's
a
CN
CF
project.
A
So
suddenly
the
dev
test
loop
no
longer
even
really
has
a
deploy
phase.
It
doesn't
even
really
have
a
build
phase
where
you
have
to
build
docker
images.
It
just
has
this
part
where
you
have
to
recompile
your
code.
Restart
your
process,
and
that
is
an
awesome
experience.
Just
give
a
little
idea
of
what
this
looks
like
say,
your
foo
app
like
well,
the
square
is
your
local
machine.
Obviously
the
Camaro's
cluster
is
the
r
septagon
over.
Here
you
have
a
proxy
agent
on
it.
A
It's
talking
to
an
app
on
your
side
and
then
you
have
the
telepresence
pod,
and
that
is
talking
to
whatever,
whatever
you
want
on
the
kubernetes
cluster.
So
if
you
have
traffic
coming
in
from
a
load
balancer
or
a
node
port
into
your
thing,
hits
you
know
a
service
represented
by
qu
proxy
that'll,
get
routed
into
the
telepath
of
telepresence,
the
telepresence
pod,
and
that
will
be
sent
over
to
your
local
machine.
A
We
also
do
DNS
hijacking,
which
sounds
really
scary,
but
it's
actually
not
so
that
when
you
actually
are
on
your
local
machine,
you
can
talk
to
the
kubernetes
services
as
if
they
were
right
available
to
you.
So
I
can
talk
to
you.
No
bar
app
dot
service,
cluster,
local
and
I
will
get
a
result.
I
do
not
have
to
change
anything.
I,
don't
have
to
rethink
how
I'm
going
to
connect
to
it.
A
I
just
natively
available
to
me,
so
it
actually
killed
two
birds
with
one
stone
we
no
longer
had
to
deploy
our
code
because,
basically
you
just
your
deployment
is
now
you
know,
restart
the
process
and
your
building
is
just
restarting
the
process
you
have
to
rebuild
your
binary
or
your
executable
or
whatever
you're
doing
and
just
drastically
simplifies
the
entire
problem.
There's
also
an
awesome
secondary
benefit.
Here
your
code
is
on
your
machine.
Think
about
that.
Again.
Your
code
is
on
your
machine.
You
can
do
all
the
things
you
do
with
code.
A
That
is
actually
on
your
machine,
which
means,
for
example,
a
debugger
is
super
easy
to
setup
from
your
local
machine.
You
can
set
up
breakpoints,
you
can
have
traffic
coming
in
through
the
production
crew
hairs
cluster.
Have
it
routed
to
your
pod.
Your
telepresence
pod
end
up
on
your
machine
where
you
can
then
set
breakpoints
and
see
the
entire
stack
of
the
request.
Awesome.
We
can
get
out
of
this
world
where
a
printf
debugging
it
horrifies
me
that
we
have
spent
decades
in
this
industry.
A
Building
awesome
debug
tools,
we
still
revert
resort
to
basically
printf
debugging
and
grapping
through
logs,
it's
brutal
so
back
to
our
expected
outcome.
Everyone's
gonna
be
dancing,
everything's
gonna
be
great
and
it
actually
was.
Finally,
we
got
to
this
actual
welcome
everyone's
dancing.
This
is
awesome.
I
can
write
code.
I
can
basically
hit
a
button
or
kill
a
process.
We
start
a
process
and
we
are
going
to
be
able
to
reload
my
code
and
use
it
on
the
cluster
right
mission
accomplished.
Never
no,
not
quite
devs,
are
never
satisfied.
A
You
give
them,
you
give
them
an
inch
they'll.
Take
a
mile
and
they'll
ask
for
the
last
patches
for
a
mile,
I'll
ask
for
a
leap,
a
giant
leap,
build
speed
is
great,
but
getting
started
and
bootstrapping
for
them
is
also
a
real
problem,
so
they
identified
the
build
we
solved
built.
We
made
it
so
they
didn't
have
to
build
and
they'd
enough
deploy
because
it
just
runs
on
their
machine
and
then
they
were
like
well
I
have
all
these
upstream
dependencies
I
want
to
use
and
I
want
to
consume
these
things.
A
I
need
to
set
up
an
environment,
that's
safe
for
me
to
work
on
these
things.
A
new
problem
deploy
bootstrapping.
It's
that
part
where
you
go
I
can't
just
let
you
use
Prada
love
to
have
their
own
place
where
your
own
dev
services
are
running.
So
the
attempt
number
one
we
came
up
with
was
you?
Do
the
obvious
use
mini
code?
Sorry
use
mini,
cube
or
run
a
local
on
your
local
machine.
You
could
just
take
the
manifest
that
we
already
have
for
all
the
kubernetes
stuff.
A
We've
been
building
and
apply
them,
and
it
should
just
all
work
out.
Everything
should
be
great
if
you
do
it
that
way.
Right,
that's
common
wisdom,
but
oh,
come
mini
cube,
is
fragile
and
tends
to
break
in
mysterious
ways
and
I.
Don't
I
don't
mean
to
be
dumping
on
the
mini
cube
authors
at
all.
Right
now,
but
I
can
name
three
failure
modes,
at
least
right
off
the
top.
My
head,
a
la
round
via
networking
clashing
with
the
host
networking
and
getting
a
go
panic
that
just
tells
you
nothing
what
to
do.
A
You
have
Mac
OS
VM
issues,
and
then
there
is
the
my
virtualization
isn't
set
up
on
my
developers.
Machine
and
mini
cube
doesn't
tell
me
anything
useful
about
that,
and
so
they
come
over
to
me
and
look
at
me
and
go
what's
wrong
and
I
go
I,
don't
know
many
cube
is
just
failing
to
start,
not
great.
If
you're
like
us.
Also,
your
deployment
logic
is
probably
not
perfect.
You
know
in
a
perfect
world
everything
kubernetes
works
on
a
basically
a
loop
retry.
A
Until
every
is
up
in
a
way
people
had
to
build
software,
they
tend
to
assume
they
can
have
control
over
dependencies,
especially
if
you're
porting
things
from
perhaps
a
pre
kubernetes
world.
So
there
is
a
chain
of
events
that
has
to
occur
that
often
gets
captured
either
in
documentation.
Other
tools
that
have
to
run-
or
it
just
makes
assumptions
about
a
CI
environment
with
well
configured
tools.
The
stuff
that
was
running
on
CI
was
different
from
the
stuff
that
was
on
developers
machines.
So
they
didn't
have
the
same.
A
You
know
they
didn't
have
the
same
deployment
pipeline
that
our
CI
stuff
would
have
which
they
all
this
really
ends
up
happening.
As
your
developers
look
at
you,
they
go
okay.
Well,
this
isn't
working
and
I
need
to
get
my
work.
Job
done
help
me
and
you
kind
of
look
at
them
ago.
I,
don't
know
why
it's
broken
I
don't
know
do
because
mini
cube,
isn't
working.
So
you
kind
of
go,
that's
bad.
A
So
my
entire
conclusion
about
mini
cube
is
it's
for
unicorns.
If
you
have
an
application,
which
is
stateless,
small
doesn't
have
many
upstream
dependencies
in
a
cube
is
great
for
you,
I,
don't
know
how
many
of
you
are
in
the
room
who
have
these
kind
of
unicorn
applications,
but
I
assume
it's
a
very
small
amount
of
us
for
the
rest
of
us.
We
actually
need
remote,
shared
environments.
If
you
already
have
any
cloud
services
or
any
back-end
services
somewhere,
you
likely
already
have
a
deployment
pipeline
to
get
that
stuff
running
in
production.
A
That's
great
because
it
actually
means
you
can
reuse
that,
and
once
you
can
do,
that,
your
developers,
the
ability
to
get
access
to
criminales
clusters
on
demand.
You
can
just
take
those
two
things
together
and
glue
them
together,
and
then
you
could
have
developers
using
their
own
kumail's
clusters
with
the
same
deployment.
Tooling
that
you
have
so
you
can
just
leverage
what
you
already
have
built.
I'm
sure
anyone
in
this
room
is
deploying
stuff,
you've
already
built
stuff
to
do
this,
so
our
attempt
number
two
was
actually
along
this
vein.
A
We
came
with
this
tool
called
kübra,
not
it's
a
tool
that
allows
you
to
get
a
remote
kubernetes
cluster
in
under
five
seconds.
It
does
it
by
actually
running
the
crew
nice
clusters
already,
so
there's
no
actual
spin
up
time.
We're
not
waiting
for
you
new
compute
instance.
That's
been
up
we're
not
waiting
for
nose
to
join
the
masters.
They're
already
go
they're
up
and
running,
and
developers
basically
set
exclusive
locks
on
clusters.
They
say:
I
want
a
cluster
for
X
number
of
minutes,
with
a
name
and
a
hands
them
back.
A
The
coop
config
with
that
thing,
and
then
they
can
release
that
cube
config
at
the
very
end
or
will
just
automatically
expire
out
of
existence.
If
they
don't
for
to
do
it,
it's
pluggable
to
support
a
range
of
cluster,
configures
and
backends.
This
is
actually
really
important
for
us
because
we're
tool
developers
we
build
products
about
services.
We
needed
a
way
for
people
to
get
a
range
of
environments
to
work
on.
So,
if
you
can
think
about
where
you
know,
our
stuff
runs
ambassador,
which
is
our
API
gate.
A
Our
API
gateway
runs
on
Azure,
Amazon,
Google,
bare-metal.
Anything.
You
can
think
of
it
can
be
run
on
a
system
that
has
any
number
of
CNI
or
CRI
configurations.
It
can
be
run
on
different
operating
systems
at
different
versions
of
kubernetes.
You
realize
your
test
matrix
suddenly
becomes
massive
or
you
reproduce
your
matrix
for
a
bug
becomes
massive.
You
can't
have
developers
waiting
for
minutes
to
spin
up
kubernetes
clusters.
Do
that,
so
we
were
like.
A
We
need
to
solve
that
problem,
so
we
came
up
with
this
kuba,
not
system
which
allows
them
to
just
grab
these
clusters
that
are
already
running
good
news,
for
this
is
this
is
gonna
go
open-source
in
2019,
it's
currently
not
even
available
really
to
the
public,
but
the
entire
thing
will
be
available.
I'll
probably
be
announcing
on
Twitter.
A
If
you
follow
me,
you'll
see
an
announcement
sometime
in
2019
about
that
super
excited,
because
it's
actually
well
the
first
things
I'll
be
releasing
myself
to
the
public,
so
Cooper
not
claimed
is
really
where
developers
start
now
at
data
wire.
They
start
out
with
you
Cooper
that
claim,
and
then
they
kind
of
start
writing
code
or
they
do
a
telepresence
start
and
then
write
code.
The
order
of
these
doesn't
matter
so
much,
but
the
idea
is
they
get
their
own
machine.
Their
code
is
on
their
local
machine.
A
You
know
they
telepresence
over
to
that
that
Cooper
not
system
can
already
have
all
the
backends
upstream
services.
They're
gonna
need
running
for
them,
and
so
their
loop
really
becomes
basically
code.
More
code,
dev
test
code
worked
at
more
code
dev
tests.
They
don't
have
to
keep
doing
Cooper
not
claimed
they
don't
have
to
keep
doing
telepresence
over
and
over
and
over.
They
just
have
to
write
code
check.
Their
code
write
code
check
their
code
now
def
testing,
it's
ugly,
it's
painful.
It's
fairly
difficult.
Without
shared
remote
infrastructure.
A
A
They
can
configure
that
production,
API
gateway
to
have
routing
rules
that
send
their
connections
over
their
development
clusters,
and
so
they
can
leverage
both
a
situation
where
they
have
to
talk
to
the
production
services
or
their
development
services
and
back
to
production
services,
as
well
as
their
stuff,
that's
running
on
their
laptop
using
telepresence,
and
it
really
helps
to
just
have
an
api
gateway
that
needs
to
be
self-service.
Once
you
have
this,
your
developers
open
up
it
just
opens
up
a
whole
new
world
to
how
they're
gonna
basically
build
software.
A
They
don't
have
to
find
someone
and
ask
them
to
add
new
routing
rules
for
them.
They
can
just
do
it
themselves,
so
we
have
code.
We
have
build
and
deploy,
build
and
deploy
with
really
claimed.
The
cluster
start.
Telepresence
write
more
code,
we
have
customizable
dev
routing,
and
then
we
have
dev
tests.
A
If
you
kind
of
think
about
this
to
describe
that
topology
a
little
bit
more,
it's
Internet
traffic
goes
to
the
load
balancer
that
load
balancer
delivers
traffic
right
to
ambassador
ambassador
talks
to
if
it's
a
production
stuff
goes
to
through
service
bar
service,
but
there's
that
link
that
link
allows
you
to
send
traffic
from
the
production
service
over
to
their
development
service
and
they
can
talk
to
their
developer
food
service,
their
bar
service.
If
they
need
to
talk
back
to
the
stuff
that's
running
in
prod,
they
can
do
that.
A
They
might
also
go
to
another
cluster
if
they
need
to,
you
can
think
of
kind
of
an
endless
number
of
topologies
you
can
come
up
with
here.
Also
I
will
say
that
the
dev
Kubb
cluster,
it
does
not
have
to
be.
If
you
don't
want
to
doesn't
have
to
be
a
physical
cluster,
it
could
be
a
namespace.
You
could
have
multiple
ambassadors
running
inside
of
your
system.
A
A
So
that's
where
we
are
today.
There
is
still
plenty
of
work
for
us
to
continue
to
do.
Devs
are
way
more
productive
than
when
we
started
there's
basically
minimal
things,
there's
minimal
waiting
for
them
left
in
this.
In
this
workflow.
Most
of
the
issues
we
have
now
are
around
UX
and
making
these
things
more
pleasant,
easier
to
debug
when
they
fail
making
it
easier,
so
they
have
less.
You
know
commands
to
have
to
think
about,
but
we're
gonna
keep
evolving.
This.
The
company
has
found
its
room,
basically
increased
productive
to
eat
massively
right
now.
A
It
works
really
well
for
eight
developers,
but
we're
gonna
be
scaling
up.
You
know
20
developers,
hopefully
by
the
end
of
next
year,
if
not
more,
you
want
to
have
be
a
little
the
ability
to
scale
this
as
the
engineering
org
grows,
and
we
believe
this
will
actually
work
and
we'll
be
really
productive
for
the
entire
company.
A
So
really
my
final
takeaways
here
Rome
was
not
built
in
a
day.
It's
a
famous
saying:
you
cannot
start
tackling
this
problem
by
basically
going
I'm
gonna,
throw
all
the
same
technology
at
it.
That
Phil
did
and
build
out
what
he
has
right
tomorrow.
Starting
tomorrow,
like
it
just
won't,
be
possible,
actually
talk
to
your
developers,
figure
out
where
your
pain
points
are.
Maybe
it
is
deploy.
That
is
actually
your
pain
point.
A
A
We
all
just
work
on
basically
fixing
the
problems
as
we
see
them,
we're
kind
of,
like
you,
know,
digital
plumbers
in
some
ways-
and
you
you
know,
I
want
to
say
this
right
now,
I
think
a
lot
people
will
kind
of
not
keep
my
advice
from
remote
clusters
and
will
continue
to
use
mini
cube
or
try
run
cube.
Edm
you're
gonna
have
to
do
it
at
some
point.
At
some
point,
you're
not
going
to
be
able
to
run
all
your
services
on
your
local
machine
or
you're
gonna
have
developers
basically
coming
to
you
going.
A
How
do
I
get
more?
You
know
more
capacity.
Do
I
can
I
run.
You
know,
can
I
get
a
laptop
with
64
gigabytes
of
memory?
Maybe
you
might
it's
cheap
these
days?
I
guess,
but
at
some
point
you're
also
gonna
want
to
have
like
data
storage
requirements.
You
may
have
data
storage
requirements
around
GDP.
Are
you
may
have
data
slaves
requirements,
just
huge
databases?
You
can't
actually
give
your
developers
for
whatever
reason
consider
using
remote
clusters
from
day
one
and
investing
in
that
anyways.
Thank
you.
A
Everyone
for
coming
I
will
be
at
the
data
wire
IO
booth
downstairs.
Please
come
and
chat
with
me.
I
am
super
friendly,
I
super
love
to
talk
to
people
check
out
ambassador
and
televisions
they're
both
open
source
projects.
Telepresence
is
a
CN
CF
project
as
well
and
tweet
me
at
the
big
lebowski.
I
would
love
to
get
some
feedback
on
this
talk.
I
think
we
have
time
for
Q&A
I,
think
this
o'clock
dinner
just
under
30.
B
A
Terminate
it
so
there
these
clusters
usually
running
Amazon
or
Google
as
an
auto
scaling
group
and
oh
sorry,
the
question
was
so
is
that
he
asked
the
once
a
developer
using
Cooper,
not
basically
just
cards
a
cluster.
What
happens?
How
do
you
reset
it
back
to
the
base
way
it
was,
and
the
answer
is:
it's
actually
running
an
auto
scaling
group,
so
you
just
put
a
bullet
in
its
head
and
the
auto
scaling
groups
in
Amazon
will
bring
a
brand
new
one
up
to
replace
it.
A
There's
a
I
mean
a
follow-up
quest
with
that.
One
would
also
be
how
do
you
keep
these
things
cheap,
so
you're
probably
going?
What's
that's
a
lot
of
money
to
spend
on
persistently
running
commands
clusters
that
no
one
is
using,
you
can
do
with
spot
instances
and
preemptable
VMs
that'll
save
a
lot
of
money,
but
also
just
kind
of
balance.
The
equation
as
developer
time,
verse
money
spent
on
infrastructure
anyone
else
over
there.
A
Yeah
so,
unfortunately,
I
don't
have
a
lot
of
students
from
gdpr
or
doing
this
kind
of
compliance.
Stuff.
I
could
totally
see
that
being
a
problem.
I
don't
have
a
great
solution
on
their
hand.
If
you
know,
if
you
come
on,
if
you
want
to
come
by
the
data
wire,
IO
booth
and
talk
about
that,
I'm
sure
we
can
think
of
something,
maybe
where
this
would
fit
in
I
could
see
a
situation
where
you
might
run
a
not
quite
production
cluster.
A
A
A
A
There
is
not
so
I'm
so
sorry,
you
know.
If
you
come
by
our
booth
talk
to
us,
we
may
be
able
to
we'd
love
to
hear
about
it.
Okay,
you're,
not
the
first
person
to
ask
for
Windows
support,
so
I
think
it's
mostly
a
matter
of.
We
need
to
understand
how
many
people
actually
want
Windows
support,
and
is
it
like,
you
know,
do
you
like
I,
see
windows?
Is
it
Windows
7
or
is
it
Windows
10
with
wsl
under
the
hood
yeah
come
by
the
data?