►
From YouTube: From Villains to Heroes: How an Improved DX Has Made Our Devs... Patricia Gaughen & Ana-Maria Cālin
Description
Don’t miss out! Join us at our next event: KubeCon + CloudNativeCon Europe 2022 in Valencia, Spain from May 17-20. Learn more at https://kubecon.io The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects.
From Villains to Heroes: How an Improved DX Has Made Our Devs Happy-ish- Patricia Gaughen & Ana-Maria Cālin, InfluxData
Developer Experience or DX is one of those terms that has become a dirty buzzword in recent years. Dirty because we all try to provide a good DX but we rarely achieve it. And as system engineers, we know that devs can be a tough crowd to please at times. In late 2019, InfluxData launched their Cloud 2.0 time-series database - a SAAS offering running on top of Kubernetes that is continuously delivered, with multiple commits landing in production daily. Since then, they’ve added several regions and Cloud providers, in addition to deploying many more services. Ana and Pat’s team support a wide variety of devs - from UI to low-level storage devs with different levels of Kubernetes knowledge. This talk will describe their journey going from villains to heroes - changing the way their devs test application code deployed with Kubernetes. They’ll share how transitioning from using local `kind` and Docker Engine for testing software changes, to using a dedicated remote Kubernetes cluster with garden.io and telepresence.io made their devs happy. They’ll also provide technical details of the setup and how their solution addresses the 4 different pillars of a good DX: “Function”, “Stability”, “Ease of use” and “Clarity”.
A
Myself
and
pat
are
here
to
tell
you
about
how
our
team
have
gone
from
villains
to
heroes
and
how
we
managed
to
improve
our
developers
experience
and
therefore
make
our
devs
happyish.
So
who
am
I?
My
name?
Is
anna
colleen
and
I'm
a
software
engineer
at
influx
data
and
I've
been
working
there,
working
on
maintaining
our
platform
built
on
top
of
kubernetes
and
cloud
and
also
making
some
of
our
other
software
engineers
happy
ish.
B
B
So
what
we're
going
to
tell
you
about
today
we're
going
to
tell
you
a
little
about
our
story,
we're
going
to
tell
you
about
the
company,
we're
going
to
tell
you
briefly
about
what
we
do
and
it's
going
to
describe
our
users
we're
going
to
talk
to
you
about
what
we've
tried
to
do
from
a
development
environment
perspective
and
then,
where
we've
ended
up
so
next
one
so
first
before
we
do
that
yeah!
So
first
influx
data.
We
are
a
company
that
we're
the
company
behind
the
time
series
database
called
influx
db.
B
B
I
think
it's
17,
but
I'm
going
to
say
approximately
15
environments
running
across
the
clouds
in
the
regions
and
they're
all
each
of
those
are
running
on
top
of
a
kubernetes
cluster
and
so
all
told
within
each
of
those
kubernetes
clusters
which,
when,
within
each
of
those
environments,
we're
running
30
plus
services,
all
these
services
together
make
the
database
and
how
you
interact
with
that
data
coming
in
and
going
out
so
very
quickly.
So
what
our
deployments
team
does
is.
B
We
are
in
charge
of
the
whole
development
and
delivery
pipeline,
so
our
team
runs
our
whole
ci
cd
pipeline
and
we
continuously
deliver
so
so
from
the
so
an
engineer
writes
the
code
commits
it
and
and
it
enters
ci
once
it
completes
that
if
it's
successful
it
goes
into
our
deployment
pipeline
and
it's
across
all
three
of
our
staging
clusters.
If
that's
successful,
it's
automatically
promoted
to
internal
production
and
then
finally,
it's
then
promoted
to
external
production.
B
Where
then,
it's
available
to
all
of
our
customers
to
make
that
happen,
we
needed
to
provide
an
environment
for
our
development
team
to,
as
the
writing
code,
have
a
development
as
much
as
they
can
have
a
production-like
experience,
and
so
I
wanted
to
quickly
before
I
handed
over
anna.
I
wanted
to
mention
who
our
team's
customers
are.
Our
team's
customers
primarily
are
the
development
teams,
so
I'm
going
to
let
anna
take
it
from
here.
A
So
now
that
we
do,
you
do
have
an
idea
of
what
our
team
does.
We
have
pictured
here,
two
personas
to
represent
our
different
engineers
that
we
serve,
so
they
are
somewhat
similar
with
a
few
exceptions.
So
the
front-end
engineer
that
sometimes
is
called
the
ui
developer,
expects
an
instant
feedback.
Loop,
expects
a
realistic,
prod-like
environment
and
doesn't
need
to
be
concerned
with
the
kubernetes
details
and
it
has
a
main
focus
on
the
ui.
On
the
other
hand,
we
have
the
back-end
engineer
who
expects
a
fast-ish
feedback
loop.
A
A
They
also
need
some
service
transparency
because
they
usually
work
on
more
than
one
service,
so
they
need
some
sort
of
monitoring
for
that
and
they
tend
to
deal
with
the
widespread
net
of
issues
and
we
do
have
in
both
of
our
front
end
and
back-end
teams,
engineers
that
are
very
good
with
kubernetes,
but
they
most
of
them
don't
like
to
touch
it
okay.
So
what
is
the
solution
that
we
came
up
with
in
2020?
A
You
would
have
kind
plus
docker,
then
you'd
have
the
app
deploying
its
own
name,
space
on
a
kind,
node
and
plus
other
namespaces
used
for
for
the
maintenance
of
the
whole
mini
the
whole
kind
cluster,
and
this
was
somewhat
functional,
but
we
had
a
few
issues
with
it.
But
before
we
go
into
those
issues,
would
you
like
to
play
a
game
with
me.
A
B
B
A
B
Let's
talk
about
so
true,
I
should
know
this
so
challenges
so
give
it
a
click
and
some
of
the
challenges
we
had
with
kind.
Well,
of
course
it
jumps
through
that's
fine,
we'll
keep
going
so
you
know
people,
people
didn't,
have
necessarily
resources
con
system,
resources
to
get
it
dedicate
to
kind
and
so
like
they
try
to
use
it.
The
machine
starts
sounding
like
an
airplane
and
they
can't
they
can't
work
on
code
while
they're
running
kind.
It
sounds
like
it's
dying.
B
Sometimes
people
install
software
from
different
places
and
then
we're
also
supporting
several
different
operating
systems,
operating
system
versions
or
distro
preferences.
All
these
sorts
of
things
give
me
another
click,
give
me
another
click,
and
then
you
know
people
don't
know
if
it's
them
or
it's
kind,
it
felt
very
brittle,
very
fragile
they'd
go
to
use
it
one
day.
It
would
work
that
day,
they'd
use
it
the
next
day
it
wouldn't
it
felt
hard,
it
felt
really
painful,
but
so
I
think
that
we
all
at
the
time,
were
kind
of
left
at
the
question
of.
B
Why
is
this
so
hard
like
it?
Just
it
just
felt
hard,
so
yeah
just
going
back,
so
our
developers
didn't
change,
but
we
realized
we
needed
to
change
something
like
what
we'd
offered
just
didn't
fit
the
demands
and
what
they
needed
from
us.
Still
the
same.
Nothing
changed
here,
so
these
are
still
our
developers
still
wanting
something
better.
B
So
what
we
came
up
with
is
remotel,
and
that
comes
from
the
combination
of
remote
and
local
and
anna's
going
to
tell
you
a
little
bit
more
why
it's
not
just
fully
remote
so
as
much
of
as
much
as
we
can
we're
pushing
it,
we're
pushing
things
off
to
a
remote
cluster,
getting
things
off
of
people's
systems
so
I'll
I'll.
Let
you
take
it
from
here.
Anna.
A
Okay,
so
we
came
up
with
a
dedicated
gk
cluster
in
which,
by
running
a
couple
of
cli
commands,
a
developer
can
spin
up
their
own
namespace,
containing
a
slimmed
down
version
of
our
application
and
all
of
its
component.
So
what
that
meant
is
that
none
of
these
developers
had
to
run
a
docker
for
desktop
and
any
more
kind.
A
A
So
after
deploying
the
app
in
your
dedicated
namespace
using
make
deploy
remote,
what
you
need
to
do
is
simply
run
this
one
command
here,
which
is
make
remote
dev,
and
then
you
pass
the
name
of
your
service,
any
back-end
service
as
a
variable
to
to
app
and
what
what
does
that
does
is
garden?
Would
sync
the
source
code
from
the
backend
repository
to
the
remote
deployment
or
stateful
set
that
matches
your?
A
Let's
say
you
are
the
person
whose
namespace
is
the
one
namespace
and
that's
triggering
a
build
of
a
new
binary
and
restarting
the
binary
on
any
change.
So
what
that
means
is
that
now
you
can
continue
interacting
with
your
repository,
that's
on
your
local
machine
and
every
time
new
code
is
built,
a
pod
will
restart
and
that
will
contain
all
of
the
changes
that
you
made.
B
So,
anna
before
you
click
before
you
click,
I
wanted
to
add
one
thing,
so
this
is
very
similar
similar
to
what
we're
doing
in
ci.
The
only
difference
is
we
don't
have
that
garden
system
name
space,
because
we're
we're
spinning
at
the
clusters
fresh
each
time
versus
a
developer,
making
changes
and
we're
wanting
to
update
their
service
within
the
environment.
So,
okay,
now
you
click.
A
B
A
A
Yeah
yeah,
the
other
thing
I
want
to
say,
is
yeah
for
front
end
developer
on
your
local
machine.
You
have
a
telepresence,
reverse
proxy
and
a
webpack
dev
server,
in
addition
to
all
of
the
other
things
that
you
need
as
a
back-end
developer
and
the
command
to
run.
This
would
be
very
similar
to
the
other
command,
except
that
the
viewpoint
is
a
front-end
service
app
rather
than
backhand
cool.
A
Well,
first
of
all,
the
first
pillar
for
a
good
developer.
Experience
is
function
and
really
it's
what
it
says
on
the
team,
what
we
promised
we
delivered.
We
asked
for
feedback
early
and
often
during
the
development
process,
and
there
are
things
that
we
kept
and
there
are
things
that
changed
completely.
A
The
next
developer,
experience
pillar
that
you
have
to
keep
in
mind
when
providing
a
developer
experience
is
clarity
and
what
that
meant
for
us
is
that
it
took
the
guessing
away
from
whenever
a
developer
would
come
to
us
to
say
hey.
This
is
not
working,
whether
it
was
a
user
error
or
whether
it
was
an
actual
concern,
so
by
virtue
of
our
engineers
having
their
kind
set
up
locally.
A
What
that
meant
for
me
and
my
teammates
was
that
the
troubleshooting
with
someone
could
take
up
to
a
day
because
you'd
go
through
things
like
okay,
so
is
it
working
for
anyone
else?
Can
I
recreate
the
issue
locally
on
my
machine?
If
not,
can
I
go
with
the
person
that
has
an
issue?
Can
I
go
to
every
single
one
of
their
steps?
There
are
so
many
different
components
in
play
that
it
just
wasn't
a
great
experience.
A
It
did
not
give
us
enough
transparency
to
be
able
to
troubleshoot
properly,
whereas
now,
if
any
engineer
comes
to
us
to
say,
something's,
not
working,
all
we
have
to
ask
is
okay.
What's
your
name
space
and
myself
or
my
colleagues
can
just
log
into
the
gke
cluster
into
that
namespace
and
see
what
the
issue
could
be.
B
Yeah,
yes,
I
mean,
I
know
things
though
yeah.
Oh
yes,
you
do
you
do
so.
I
think
this
is
like
to
me.
This
feels
like
kind
of
a
journey
when
it
comes
to
software
in
the
cloud,
but
we
so
like
through
the
development
process
like
as
we
were
working
on
this
like
as
we
like,
moved
from
kind
to
remokal
for
like
ci.
We
were
running
that,
like
we
were
running
the
new
way
in
parallel
to
the
old
way,
and
so
like
the
whole
time,
we
were
getting
feedback
like.
B
Are
we
getting
good
results?
Are
things
looking
good
launching
hundreds
of
instances
of
it
making
sure
launching
hundreds
of
namespaces?
Sorry
not
instances
of
to
to
make
sure
that,
like
kind
of
like
working
out
the
kinks,
seeing
if
we
can
make
it
work
better
see
if
we
can
work
faster?
We
also
are
monitoring
it.
So
we
have
we're
keeping
track
of
like
long-running
name
spaces,
seeing
what's
going
on
memory,
cpu,
utilization
and
and
and
all
of
that
so.
A
So
as
we
started
with
this
new
solution,
we
did
have
some
challenges.
Mostly
all
of
the
software
that
used
specifically
garden,
io
and
telepresence
did
what
we
wanted,
but
there
were
some
features
missing.
So
to
give
an
example
with
garden,
so
garden
doesn't
support
out
of
the
box
jsonnet,
but
we
would
dynamically
generate
yaml
using
our
jsonnet
code
and
then
garden
would
manage
that
yaml
code.
So
we
managed
to
find
the
solution
around
that.
A
A
We
also
had
you
know
as
any
experience,
there
was
a
health
crime
and
we
didn't
have
anyone
who
had
specific
expertise
with
telepresence
in
our
team.
So
we
all
we
had
to
learn
as
we
went
and
then
there
are
some
early
robustness
issues
with
telegram
with
telepresence
telegraph
is
one
of
our
little
components,
part
of
the
product.
So
if
someone
tried
a
new
version
of
telepresence
in
this,
the
the
whole
in
the
using
the
remote
cluster
that
would
bring
down
the
whole
cluster
for
everyone.
B
Okay,
can
I
take
it
over?
Can
I
take
it
over
from
here?
Okay,
cool
all
right,
and
so
I
think
I
think,
the
next
one
we
didn't
give
ourselves
a
check
mark
on
because,
like
it's
not
fully
gone,
we
don't
we
haven't.
We
have
not
completely
gotten
rid
of
like
we
still
have
a
local
development
environment
like
a
local
developer
system,
surface
area,
we've
greatly
reduced
it
by
moving
most
things
for
to
the
remote
cluster.
B
We
still
have
telepresence
running
locally,
and
so
you
know
we
had
to
do
some
stuff
in
the
code
to
make
sure
that
the
agent
and
the
telepresence
were
always
on
the
same
version,
but
we
we
got
through
that,
so
so
I
just
wanted
to.
When
we
launched
remotel,
it
was
actually
a
different
experience
than
with
kind
when
we
lost
kind.
We
like
had
to
kind
of
convince
users
that
they
had
to
use
it.
There
was
none
of
that
here.
You
know
what
I'm
talking
about
anna
they
got
on
it
and
so
right
away.
B
We
were
getting
a
lot
of
like
okay,
I
don't
you
decide
if
it's
a
lot,
I
felt
like
it
was
like
it
was
really.
I
was
really
proud
of
the
people
I
was
working
with
when
we
got
all
this
feedback.
I
mean
just
like
really
good
positive
feedback.
Did
it?
Did
it
stay
that
way
it
was
probably
best.
We
did
name
the
presentation,
happy
ish,
because
this
was
a
good
day
and
and
not
all
the
rest
of
them
are.
But
it's
been
it's
I
would
say
overall,
the
feedback's
been
really
good.
B
B
B
Utilization
turns
out,
people
did
were
using
it,
so
we
need
to
do
better
communicating
we're
working
on
being
better
at
communicating,
especially
about
things
we
don't
think
anyone
cares
about,
but
they
do
and
also
messaging
in
the
right
way.
So
they
know
like
why
they
might
maybe
care
and
then
we're
also
always
continuing
to
work
on
improving
this
stability.
I
think
we've
come
a
long
way,
but
it's
something
we
we
watch
we're
measuring
we're
keeping
track
of
we're.
B
So,
specifically,
we're
a
database
company
so
we're
trying
it
we're
currently
adding
a
feature
that
lets
them
pre-load
data
and
having
kind
of
a
data
catalog,
so
they
can
load
date,
different
types
of
data
based
on
what
they're
trying
to
get
done
so
next.
Finally,
anna
mentioned
that
it
was
something
she
was
doing
with
her
colleagues,
opie
amy
and
wojciech
are
two
of
the
like
developers.
That
did
a
lot
of
the
heavy
lifting
on
this
project,
and
I
can't
say
I
mean
the
whole
team.