►
From YouTube: Characterizing and detecting incivility in open source code review discussions | Isabella Ferreira
Description
Characterizing and Detecting Incivility in Open Source Code Review Discussions
Code review is an important quality assurance activity for open source software development. Yet, code review discussions among developers and maintainers can be heated and sometimes involve personal attacks and unnecessary disrespectful comments, demonstrating, therefore, incivility. Although incivility in public discussions has received increasing attention from researchers in different domains, the understanding of this phenomenon is still very limited in the context of software development and, more specifically, code review. To address this gap, this proposed talk will present the results of a qualitative analysis conducted on 1,545 emails from the Linux Kernel Mailing List (LKML) that were associated with rejected changes. From this analysis, we identified the features of discussion of civil and uncivil communication as well as the causes and consequences of uncivil communication. Based on our results and with the goal to create healthier and more attractive open source communities, we will also discuss in this talk (i) approaches that could be used to address incivility before and after it happens, (ii) pitfalls to avoid when trying to automatically detect incivility, and (iii) heuristics for detecting incivility in code review discussions.
A
A
The
thing
is
that
currently,
there
is
no
knowledge
about
how
incivility
manifests
in
code
review
discussions,
as
well
as
the
costs
and
consequences
of
incivility.
So
currently,
there
is
no
clear
way
to
effectively
address
the
problem
of
incivility
and
to
fill
this
gap.
We
want
to
establish
more
knowledge
about
instability
by
characterizing
it
and
also
understanding
its
causes
and
consequences.
A
So,
to
create
healthier
environments,
we
first
need
to
identify
the
features
of
discussion
found
in
code
review
discussions
and
based
on
the
features
of
discussion
that
convey
a
disrespectful
tone.
Then
we
can
assess
the
frequency
of
instability
in
code
review
discussions
as
well
the
causes
and
consequences
of
uncivil
communication
and
by
understanding
all
of
these
we
can
then
provide
means
to
create
healthier
work
environments
in
open
source
communities.
A
A
Then
we
developed
an
approach
to
identify
rejected
chains
and
we
found
that
around
48
of
the
email
threads
they
were
rejected
and
then
we
filtered
the
email
threads
that
contained
only
one
email.
That
means
no
patch
discussions
or
threads
that
were
actually
accepted
after
doing
a
mental
analysis
and
after
that,
we
statistically
sample
the
email
threads
to
do
a
qualitative
analysis
resulting
in
262
email,
threads
composed
of
1545
code
review
emails.
A
A
A
A
So
concerning
name
calling
naming
calling
appears
in
sentences
that
include
mean
or
offensive
words
directed
at
the
person
or
a
group,
a
group
of
people,
for
example.
If
you
want
to
provide
more
accurate
documentation,
then
you
better
come
up
with
something
which
is
helpful
instead
of
a
completely
useless
blurp
like
the
below
and
finally
patience,
so
participants
might
demonstrate
in
patients
when
they
express
a
feeling
that
is
taking
too
long
to
solve
a
problem,
to
understand
a
solution
or
to
answer
a
question
or
when
someone
needs
to
repeat
the
same
information
over
and
over
again.
A
An
email
is
technical.
If
it
contains
only
straightforward
forward
technical
discussions
and
do
not
contain
any
feature
of
discussion
and
technical
emails,
they
represent.
89
of
the
analyzed
emails
then
uncivil
emails
are
those
that
express
at
least
one
uncivil
feature
of
discussion
and
seven
percent
of
the
code
review
emails
there
on
civil,
and
then
we
have
the
civil
emails,
which
are
those
that
express
positive,
neutral
or
negative
features
of
discussion,
and
none
of
those
features.
They
are
uncivil.
A
A
A
So,
concerning
uncivil
email
sent
by
developers,
the
most
frequent
cause
was
first,
the
maintainer's
feedback
and
the
most
frequent
consequences
are
that
the
maintainer
escalated
in
civil
conversation.
That
means
the
conversation
got
hated.
The
maintainer
discontinued
further
conversation,
so
the
maintainer
just
stopped
talking
and
the
maintainer
kept
discussing
things
in
a
civil
way.
A
The
second
most
frequent
cause
was
the
violation
of
community
conventions,
so
community
conventions
are
related
to
the
workflow
imposed
by
the
community,
by
not
understanding
why
they
need
to
do
a
tedious,
workflow
or
out
of
a
surprise,
a
workflow
that
the
develop
developer
was
not
informed
of,
and
when
that
happened,
the
maintainer
most
frequently
discontinued.
The
conversation
also
developers
on
civil
comments
were
sometimes
triggered
by
being
misinterpreted
by
the
maintainer
or
being
unable
to
follow
the
maintainer
restrictions,
and
when
that
happened,
the
maintainer
most
frequently
escalated.
A
The
uncivil
conversation
in
some
times
developers
were
uncivil
due
to
the
maintainers
on
civil
behavior
and
when
that
happened,
the
maintainer
often
reinforced
their
standpoint
or
escalated.
The
uncivil
conversation
and
finally
developers
were
also
on
civil
due
to
inappropriate
solution
proposed
by
the
maintainer,
and
when
that
happened,
the
maintainer
often
provided
a
technical
explanation
or
escalated.
The
uncivil
conversation
now
considering
uncivil
emails
sent
by
maintainers.
A
A
Maintainers
were
also
uncivil
because
developers
were
violating
the
community
conventions
and
when
that
happened,
developers
often
discontinued
further
conversation
or
accepted.
The
mountaineers
criticism
also
maintainers
and
civil
comments
were
sometimes
triggered
by
being
misinterpreted
by
developers
and
the
most
frequent
consequences
were
the
acceptance
of
the
montana's
criticism
and
also
escalating
the
uncivil
conversation.
A
So
now
we
know
the
cause
and
consequence
of
uncivil
communication,
so
we
can
provide
means
to
create
healthier
working
environments
for
open
source
communities.
We
suggest
two
approaches
for
that.
First,
is
we
call
it
proactive
approaches
which
are
approaches
that
are
done
to
address
the
causes
of
instability
before
it
happens,
and
then
we
have
the
reactive
approaches
which
are
approaches
that
can
be
done
to
address
incivility
after
it
happens.
A
Also,
maintainers
should
always
include
why
the
patch
was
rejected
to
give
a
feedback
to
the
developer
and
also
open
source
communities
could
gamify
the
review
process
so
that
developers
that
follow
the
community
conventions
they
gained
more
reputation
and
status
when
the
cause
of
incivility
was
communication
issues,
so
open
source
communities
could
develop.
A
code
of
conduct
focused
on
the
code
review
process.
A
When
the
cause
of
instability
was
inappropriate
solution,
then
developers
should
always
include
a
technical
explanation
of
their
solution,
including
the
downsides
of
the
solution.
The
motivation
of
the
proposed
the
proposed
patch,
as
well
as
the
limitations
when
the
cause
of
incivility
was
poor
code
quality.
A
Then
open
source
communities
could
include
the
training
for
newcomers
and
developers
to
ensure
that
everyone
is
aware
about
the
community's
expectations
in
terms
of
code
quality.
Also
open
source
commits
could
adopt
existing
analysis
code
analysis
tools
and
integrate
them
into
the
developer's
workflow.
A
A
So
we
observe
that
incivility
actually
cannot
be
reliably
captured
by
just
analyzing
the
sentiment
of
a
text,
and
this
is
because
incivility
has
many
dimensions
not
captured
by
those
two,
such
as
the
context
of
the
conversation
in
our
case
is
the
previous
email
using
the
email
thread,
also
the
familiarity
among
people
and
the
granularity
of
analysis.
If
it's
in
the
thread
in
the
email
or
in
the
sentence
level
of
analysis,
also
some
features
of
discussion,
they
might
be
hard
to
be
detected
by
those
tools
such
as
irony,
mocking
and
threat.
A
So,
concerning
building
instability,
specific
detectors,
we
suggest
that
researchers
or
open
source
communities
could
develop
heuristics
to
identify
the
discussions
features
found
in
our
study
also
to
accurately
detect
incivility
these
heuristics.
They
need
to
consider
the
context
of
the
conversation,
the
familiarity
among
people
and
the
granularity
of
analysis,
also
in
order
to
improve
the
performance
of
classifiers,
an
incivility-specific
lexicon
for
each
discussion,
feature
could
be
built
and
that
could
be
done
based
on
our
data
set,
that
is
publicly
available
and
could
be
extended
to
train
machine
learning
models
to
detect
incivility.