►
Description
Mike Karampalas demonstrates the AB test of changing the auto-renew toggle to a cancel button for Gitlab.com subscriptions.
A
Folks,
in
the
interest
of
demoing,
some
of
the
changes
that
we've
made
recently
to
the
auto
renew
flow
for
our
lab
comm
customers
I
wanted
to
record
this
demo
so
that
folks
could
see
what
that
behavior
is
interaction
and
have
a
better
understanding
of
what
change
that
we
actually
made.
So
what
I'm
going
to
do
is
show
you
this
experience
on
staging
and
then
I
can
talk
to
what
it
is
previously
like
in
production.
A
Now
the
hypothesis
behind
this
change
was
that,
since
there
is
no
way
to
manually
renew
a.com
subscription
that
when
folks
were
toggling
auto
renew
off,
they
didn't
realize
that
they
were
essentially
canceling
their
subscription,
because
there
is
no
way
to
renew
there.
Isn't
a
button
like
this
for
dot-com
subscriptions
Auto
renew
is
the
only
way
currently
that
it
is
renewed,
and
you
know
people
may
just
see
this
Auto
renew,
toggle
and
think
well,
no
I,
don't
want
it
to
auto
renew
I
want
to.
You
know,
think
about
it
ahead
of
time.
A
You
know,
example:
I
use
is
like
I,
don't
do
auto
payments
on
my
bills,
I'd
like
to
see
them
and
process
them
and
pay
them,
and
I
suspect
that
there
is
a
set
of
customers
that
are
the
same
way
when
it
comes
to
get
labs
descriptions.
So
the
change
we
made
is
highlighted
here.
We
have
this
cancel
subscription
button.
So
if
user
sees
this
instead
of
just
toggling
auto
renew
to
off,
they
actually
have
to
click
cancel
subscription.
A
Let
me
present
them
with
this
modal,
where
we
can
get
a
little
bit
of
additional
information
about
why
they
chose
to
cancel
so
some
high-level
checkboxes,
as
well
as
the
ability
to
add
a
preform
comment.
This
is
optional.
We
didn't
want
to
prevent
folks
from
from
canceling,
but
this
additional
information
and
feedback
will
help
us
in
the
future
to
try
to
figure
out
where
else
we
can
and
generate
some
additional
hypotheses
to
test.
So
if
I
go
through
and
processes,
canceled
subscription
I'll
get
a
confirmation
message
at
the
top.
A
Tells
me
what
I've
done
and
how
long
my
subscription
will
remain
active
until
because
it's
a
yearly
subscription,
so
it's
still
good
until
they
would
have
needed
to
renew,
and
then
they
see
the
subscription
widget
here.
Actually
now
has
this
resumed
subscription
button
there
and
that's
to
get
you
back
basically
to
do
the
same
thing
as
toggling
auto
renew
back
on
wood.
So
we
didn't
actually
change
anything
in
the
backend.
We
are
still
using
the
same
same
approaches.
A
The
auto
renew
toggle
was
but
we've
just
changed
the
front-end,
the
information
we
display
and
the
calls
to
action
so
that
it's
more
clear
for
customers
what
is
happening
when
they
actually
do
this,
and
so
you
can
see
that
I've
successfully
re-enabled
it,
and
it
tells
me
now
that
my
subscription
will
automatically
renew
on
on
the
renewal
date,
and
so
that
is
the
the
change
and
we're
seeing
very
good
results.
So
far,
it's
early.
We
got
a
lot
of
continued
to
run
to
make
sure
it's.
A
You
know
significant
and
not
just
an
anomaly,
but
we
think
that
this
is
a
better
way
to
explain
to
customers.
What's
going
on
it's
more
clear
and
it
allows
us
to
move
closer
to
our
goal
and
our
vision
of
gitlab
will
renew
unless
you
cancel
and
so
just
again
to
go
over
what
it
the
control
experience.
Is
it's
this
Auto
renew
toggle
that
you
can
kind
of
go
off
and
on,
but
there
is
an
over
new
button,
uncom
cells,
so
that's
it
I
hope
that
you,
you
know
we're
able
to
understand
that
easily.