►
From YouTube: UX Showcase: Simple changes, significant results
Description
Kevin Comoli shares some of the experiments the Growth team has worked on recently.
A
Can
everybody
see
my
screen?
Okay,
yeah,
all
right,
so
hi,
everyone,
I'm
kevin
product
designer
in
growth
and
today
I'm
going
to
showcase
some
simple
changes
that
we
worked
on
with
the
conversion
team
that,
hopefully
are
going
to
result
to
significant
sorry,
hopefully
are
going
to
lead
to
significant
results.
So,
as
jackie
highlighted,
this
presentation
has
got
a
bunch
of
different
small
things
and
I
try
to
keep
it
bite-sized,
so
we
can
stop
in
the
middle
and
discuss
it.
A
So
if
you
see
me
stop
and
asking
for
question,
this
is
part
of
the
presentation.
This
is
not
the
end
of
the
presentation,
so
one
of
the
first
experiments
that
we
run
was
a
couple
of
copy
changes
and
one
on
the
user
onboarding.
So
in
the
last
showcase
I
briefly
touched
on
this,
but
when
we
experimented
on
the
new
user
onboarding,
we
prompted
users
with
this
screen,
d4
and
3df.
A
We
were
basically
asking
them.
How
would
they
describe
themselves
in
terms
of
like
professional
using
devops
like
where
they
novice
or
experienced,
and
what
we
noticed
is
that
a
majority
of
people
were
choosing
novus,
but
at
this
time
we
were
reluctant
on
concluding
that
the
majority
of
software
developers
that
were
signing
for
gitlab
sas
were
nervous,
mainly
because
of
this
cta
that
I
highlighted,
because
we
thought
that
show
me
everything
at
this
stage
of
the
flow
could
really
bias
you
to
pick
this
option.
A
A
So
what
we
did
was
just
to
remove
this
show
me
everything
and
swap
it
with
show
me
the
basics
for
novice
and
for
more
experienced
users.
We
just
added,
show
me
advanced
feature,
and
at
this
time
we
had
good
hope
that
the
distribution
would
change,
because
we
thought
that
this
was
much
clearer,
but
this
is
actually
what
happened.
Nothing
changed
not
that
much.
We
had
76
of
people
declaring
themselves
as
novice.
A
How
can
we
infuse
this
in
our
daily
work
and
not
only
at
the
beginning
of
the
flow,
not
only
in
sign
up,
not
only
in
onboarding
but
through
the
app
and
another
question
that
was
raised
was?
Would
it
be
the
same
for
self-managed
like
do
software
developer,
installing
gitlab
and
taking
care
of
everything?
Would
there
be
more
novice
or
experience,
and
at
the
moment
we
don't
know
this,
so
we
thought
this
would
be
an
interesting
follow-up.
A
Another
change
that
we
did
was
in
the
trial
sign-up
flow,
so
we
had
this
hunch
looking
at
the
data.
Basically,
we
saw
that
25
of
sasket
trial
came
from
within
the
trial
pages
and
from
one
specific
place
now
I'm
going
to
show
you.
This
is
one
of
the
trial
page
I'm
going
to
stop
real
quick.
Can
anyone
identify
something
that
could
be
wrong.
B
C
A
A
All
of
the
fields
here
are
basically
used
for
marketing
and
sales
to
qualify.
The
leads
on
the
trial,
but
really
what
is
happening
is
if
you
look
at,
I
mean
this
is
our
hypothesis.
We
might
be
wrong
as
well,
but
if
you
look
at
the
headline
and
the
copy
is,
we
are
saying:
free
gold,
trial
and
free
trial,
and
every
steps
before
in
the
marketing
website
clearly
emphasize
on
starting
a
free
trial.
A
Now
I'm
a
free
user.
I
also
have
this
option
in
the
bottom
of
the
page,
which
is
saying
ski
trial
and
continue
with
a
free
account,
which
is
a
bit
odd
right,
because
a
trial
is
supposed
to
be
a
free
account
right,
but
I
think
this
copy
was
here
it's
because
internally,
the
way
we
make
a
distinction
between
paid
trial
users
and
free
users
is
this
way
we
basically
say
a
free
account.
User
is
not
a
trial
user,
but
both
are
using
a
free
version
of
gitlab.
A
It's
just
that
one
has
more
feature
and
is
limited.
The
other
one
has
less
features
so
seeing
this,
our
hypothesis
was
like
simplifying
our
copy
is
likely
to
qualify
the
whole
process
of
completing
the
trial
film
and,
as
a
result
of
that,
we
might
see
an
increase
of
number
in
trial
started
so
again,
simple
change.
A
We
just
removed
everything
that
was
stating
free
account
next
to
skip
trial.
Basically,
this
is
an
active
experiment,
so
we
might
be
wrong.
We
don't
know
yet,
but
we
thought
that
this
was
awfully
confusing,
so
to
verify.
Basically,
to
measure
this,
we
want
to
observe
a
decrease
in
the
25
percent
that
I
showed
you
earlier
and
observe
an
increase
in
trial
started.
C
Yeah
I'll
ask
a
question:
those
who've
worked
with
me
for
a
long
time
know
that
ui
copy
is
it's
like
my
personal
passion
and
love,
because
I
think
it's
so
deeply
important
and
I
think
it's
something
we
forget
about
a
lot.
So
I
see
you
super
focused
on
ui
copy,
which
makes
me
very
very
happy
and
I'm
wondering
when
it
asks
us
the
right
way.
How
can
we
take
the
things
that
you're
learning
the
level
of
importance
that
you
are
giving
to
ui
copy
and
help
spread
that
knowledge
to
other
designers?
C
A
I
have
just
one
hint
for
now.
I
think
what
would
be
good
is
to
have
some
really
like
concrete
proof
that
you
know
changing.
This
copy
is
leading
to
that
which
we
might
actually
have,
because
we
run
a
similar
copy
experiment
for
users
that
were
already
having
a
git
lab
account
and
starting
a
trial,
and
we
also
changed
this
piece
of
copy
and
we
isolated
the
change
and
it
was
really
interesting.
That's
also
why
we
run
this
experiment
is
because
we
run
it
elsewhere
and
we
saw
really
good
results.
C
A
A
D
F
A
A
So,
to
verify
that
same
thing,
we're
currently
running
an
experiment
where
we
are
heading
to
button,
so
single
sign-on
button,
google
and
github.
We
choose
google
and
github,
because
what
we
saw
in
the
data
is
that
most
of
our
current
users,
so
not
users
that
are
registering
for
gitlab,
but
users
that
have
a
gitlab
account.
They
have
a
tendency
to
link
google
and
github
to
their
gitlab
account
and
massively
compared
to
the
other
options
that
we
have,
which
are
bitbucket,
twitter,
gyra
and
maybe
another
one
that
I'm
forgetting.
A
So
this
is
why
we're
starting
with
these
two
and
then
we'll
measure,
success
on
the
number
of
trials
started
in
this
case,
because
we
think
that
simplifying
this
registration
process
is
likely
to
lead
users
to
be
more
inclined
to
complete
it
and
we'll
obviously
observe
and
monitor,
what's
happening
with
those
two
buttons
like.
Are
they
clicked?
Are
they
really
used
or
not
again,
I'm
gonna
pause
quickly.
If
I
have
time,
do
you
have
any
questions
on
this.
A
All
right
and
the
last
one
okay,
so
here
I
should
have
images
sorry
about
that,
but
it
was
about
adding
reassurance.
So
if
you
check
other
sas
products
that,
for
some
reason,
don't
appear
here-
and
I
don't
know
why
sorry
about
that-
but
basically
this
page
is
also
missing.
Some
reassurances.
Like
what
features
are
you
going
to
try?
What
is
the
value
of
starting
a
trial,
and
our
hypothesis
is
that
most
likely
reassurance
could
play
a
crucial
role
to
friendly
nudge,
both
users
and
buyers,
persona
to
continue
the
trial
registration
process.
A
It's
actually
in
development,
so
to
see
basically
again,
are
we
observing
any
increase
in
trials
due
to
this
bits
of
content
added
and
what
we
will
also
monitor
as
the
differences
that
we
have
in
base
and
variants
in
people
that
go
to
the
first
step
of
the
trial
and
then
go
to
the
second
step
of
the
trial.
Are
there
any
drops
like?
Are
there
more
drops
for
people
that
didn't
get
exposed
to
their
reassurance
or
not
and
obviously
keep
monitoring
those
key
trial
rates?
F
After
the
free
trial
expires
after
30
days,
do
we
just
cancel
the
trial,
or
do
we
email
and
say
something
like
do
you
need
more
time
to
evaluate
it?
I
only
ask
because
base
camp
seems
to
do
that.
If
you
do
a
free
trial
on
base
camp,
they
email
you
and
say
like
do
you
need
more
time,
because
we
can
give
you
another
30
days.
A
So
we
do
have
some
emails
that
are
being
sent.
So
if
I'm
not
mistaken,
we
have
four
emails
so
one
a
week,
but
they're
meant
to
pitch
the
feature
that
you
can
use
during
your
trial.
I
actually
don't
know
if
we
have
this,
I
should
check
it
out.
That's
a
good
question,
because
we
are
planning
to
do
the
same
thing
in
app,
so
basically
what
you
said
just
reminding
users
that
their
trial
is
about
to
end
so
yeah.
This
is
something
I
should
look
into.
Thank
you
for
raising
this.
C
A
They're
considered
as
part
of
their
workflow,
but
I
think
marketing
can
work
more
on
this
and
now
what's
gonna
happen
in
with
conversion,
we're
more
gonna
work
down
the
funnel.
So
what
I
showed
here
was
like
up
in
the
funnel,
which
was
also
like
kind
of
overlapping
with
my
marketing
does,
but
since
we
cleared
up
like
one
year
and
a
half
ago,
the
trial
flow
and
that
we
have
more
and
more
people
starting
a
trial.
A
A
But
then
we
also
have
to
consider
that
the
person
that's
starting
a
trial
might
not
be
the
person
using
the
app
the
person
tagging.
The
trial
might
be
a
buyer
persona
and
doing
this
for
his
organization
and
the
people
that
are
exposed
to
the
feature,
and
that
actually
see
the
benefits
of
the
trial
might
be
different,
and
our
hypothesis
and
conversion
is
that
we
should
target
the
people
that
are
using
the
app
so
that
they
can
afterwards
kind
of
bring
back
the
knowledge
to
buyers
persona.
A
C
Okay,
because,
as
you're
talking,
I
think
there's
some
really
interesting
problems
here,
so
as
the
buyer
who's
not
involved
in
usage.
If
those
emails
told
me
the
impact
to
my
team
on
the
trials
like
this,
many
people
have
been
using
it.
This
is
how
much
they've
been
using
it
possibly
like
this
is,
who
has
been
using.
It
could
be
really
really
powerful.
So
I
get
it
there's
a
lot
to
do.
You
gotta
prioritize
what
you're
doing,
but
I
wouldn't
let
that
fall
off.
My
radar
sure.
G
Yeah,
just
on
that
I'll
say
that
I
signed
up
for
a
gold
trial
back
in
august
and
received
a
couple
of
emails
with
links
to
youtube
videos
that
were
a
little
outdated,
and
I
shared
that
with
taylor,
my
product
manager,
there
was
one
I
got
a
couple
of
emails.
That
was,
you
know
telling
me
how
I
can
optimize
my
free
trial
and
the
things
that
I
get
with
gold
that
I
wouldn't
get
with
a
free
trial
and
some
of
those
things
I
know
with
regards
to
application
security
anyways
were
outdated.
G
E
This
is
a
broader
question,
but
do
we
have
documentation
on
like
the
patterns
for
design,
templates
or
email,
templates
and
stuff?
I
ran
into
a
case
where
a
gitlab
sends
users
a
file
over
email,
and
I
was
like.
I
have
no
idea
how
this
email
looks
or
like
what
they
get.
What
type
of
messaging
goes
in
the
body.
B
Yeah,
I
don't
think
we
do
for,
but
in
product
emails
that
we
send,
we
should
have
it,
but
we
don't
marketing
does
have
some
of
this
templates
and
some
guidelines
for
emails
that
they
send.
So
I
would
say
we
have
to
look
at
what
we
already
do
and
try
to
be
consistent
as
much
as
possible,
but
I
think
it
would
be.
You
know
it
should
be
in
the
backlog
somewhere.
We
should
have
it
on
the
to-do
list
that
we
that
we
should
add
this
to
pajamas
or
somewhere
appropriate.
C
I
agree
we're
working
on
the
fy
22
ux
direction
right
now,
and
this
came
up
as
an
idea
for
something
that
we
should
focus
on
long
story
short.
I
think
it
was
too
tactical
for
our
our
annual
direction,
but
that
doesn't
mean
we
shouldn't
do
it
and
then
just
as
a
reminder
in
our
design
system
site,
there's
a
section
for
even
branding
content.
So
if
this
doesn't
fit
necessarily
under
the
the
product
design
content
but
is
more
closely
aligned
with
branding
like
we
have
a
spot.
F
F
A
B
D
So
if
it's
pedantic
just
ignore
me,
I
love
putting
predictions
in
hypotheses
because
that's
kind
of
how
we
learn
it's
okay
to
be
wrong
and
the
prediction
primes
us
for
learning
to
be
redundant,
and
so
I'm
wondering
do
you
put
predictions
in
your
hypotheses
and
then
also
do
you
have
like
a
go-to
template
that
you've
found
really
resonates
with
the
team
yeah?
Those
are
my
basic
questions.
A
A
Yeah,
so
we
do
yeah.
Obviously
we
do
try
to
have
this,
because
otherwise,
basically
we're
just
kind
of
randomly
moving
metrics
right,
but
then
kind
of
observing
what's
happening
is,
to
be
honest,
always
a
bit
of
a
struggle
for
us,
mainly
because
of
our
tech
stack
on
how
we're
conducting
experiments,
because
right
now
we're
using
kind
of
a
custom
implementation
using
gitlab
future
flags.
A
And
for
your
other
question
on
the
template,
we
do
have
a
template
template
issue
that
we're
using
that
is
composed
of
a
hypothesis,
a
business
problem,
a
design
section,
and
then
we
just
show
how
do
we
split
up
everything
I'll
link
it
at
the
dock
and
yeah?
And
then
we
try
to
say
how
do
we
want
to
measure
success
on
this.