►
Description
Presenter: Kenny Joseph
Abstract/Description: Twitter users signal social identity in their profile descriptions, or bios, in a number of important but complex ways that are not well-captured by existing characterizations of how identity is expressed in language. Better ways of defining and measuring these expressions may therefore be useful both in understanding how social identity is expressed in text, and how the self is presented on Twitter. My presentation will discuss work that extends and makes three contributions in this stead. First, I will introduce the concept of a personal identifier, which is more representative of the ways in which identity is signaled in Twitter bios than the strict social psychological definition of identity. Second, I introduce a simple method to extract all personal identifiers expressed in a given bio. Finally, I will discuss a series of validation analyses that explore the strengths and limitations of our proposed method. This work is joint with former UB student Arjunil Pathak (now at Amazon), and current UB student Navid Madani.
Bio: Kenneth Joseph is an assistant professor in Computer Science and Engineering at the University at Buffalo. Prior to joining the University at Buffalo, he was a postdoc at The Network Science Institute at Northeastern University, and graduated from the Societal Computing program at Carnegie Mellon University. He identifies as a computational social scientist, and is primarily interested in developing ways to measure and model how social inequality arises and is maintained, with a particular focus on the United States.
A
It
is
my
pleasure
to
have
Professor
K
Joseph
with
us
to
speak
about
his
work
on
multiple
social
identities
from
Twitter
bios
Dr
Joseph
is
an
assistant
professor
in
computer
science
and
engineering
at
the
University
of
Buffalo
prior
to
joining
the
University
at
Buffalo.
He
was
a
postdoc
at
the
network,
Sciences
too
at
Northeastern
University
and
graduated
from
the
societal
Computing
program
at
the
hanuki
Military
University.
A
He
identifies
as
computational
social
scientist
and
is
primarily
interested
in
developing
ways
to
measure
and
model
how
social
inequality
arises
and
is
maintained,
with
a
particular
focus
on
the
United
States.
So
then
I
turn
it
over
to
Dr.
Joseph
I
also
request
everyone
else
to
keep
your
microphones
muted
and
if
you
have
any
questions
during
the
talk,
please
use
the
chat
feature
and,
of
course
we
will
have
the
discussion
phase
after
Dr
Joseph's
presentation.
C
B
People
see
the
main
screen
all
right:
cool,
hi,
everyone,
I'm,
Kenny,
I'm
gonna,
talk
about
a
paper
that
he
wrote
for
cscw
last
year
and
title
the
method
to
analyze
multiple
social
identities
in
Twitter
bios.
So
before
I
get
started.
A
quick
nod
to
arjunopathic,
who
is
a
former
Master's
student
here
at
UB
who's
now
at
Amazon,
was
the
lead
author
and
Naveed
Naveed.
Hopefully
this
is
an
okay
picture
of
you,
if
not
I,
think
I
gotta
get
another
one,
although
I
like
it.
B
B
So
I
want
to
be
clear
on
what
I
mean
by
identity,
which
is
in
sort
of
the
traditional
social
psych
sense
of
the
words
and
phrases
that
we
use
to
label
ourselves
and
others,
and
these
identities
are
important
because
they
tell
a
very
rich
story
about
who
we
are
right.
So
currently
I
am
holding
the
Identity
or
you
could
assign
me
the
identity,
right
presenter
and
you
all
are
sort
of
like
implicitly
assigned
the
identity
audience.
Member
and
one
story
that
that
tells
about
you
and
me
is:
it
explains
our
Behavior
right.
B
The
reason
that
I'm,
just
like
sitting
here,
talking,
admittedly
to
a
screen-
and
you
are
maybe
checking
email
but
also
listening-
is
because
of
the
identities
we
hold
here
so
identities
can
help
are
important
because
they
can
help
explain
our
behavior
and
they.
What
it's
with
can
do
that
because
they
have
really
strong
culturally
shared
meanings,
often
label
the
stereotypes.
I'm
not
going
to
talk
so
much
about
what
identities
mean
today
as
much
as
I
am
trying
to
think
about
like
what
are
these
words
and
phrases?
B
What
aren't
they
and
what
can
we
say
about
that?
Okay,
so
the
other
thing
I
want
to
do
is
is
kind
of
caveat.
What
I
just
said:
it's
not
necessarily
that
the
identities
we
hold
explainer
Behavior,
it's
the
identities
we
choose
to
present
in
a
particular
social
context
that
are
telling
the
story
about
us
right.
B
So.
This
is
really
important
in
the
context
of
social
media
right
because
who
we
present
on
social
media,
of
course,
is
not
necessarily
the
self
that
we
are
offline,
and
so
it's
it's
necessary
to
kind
of
caveat
everything
or
say
with
with
that
observation.
So
I
want
to
state
that
up
front
okay,
so
identity
is
important
because
it
can
help
to,
for
example,
explain
behavior,
but
there's
also
a
ton.
B
We
don't
really
understand
about
identity
right,
so
we
can
ask
questions
like
why
do
we
present
certain
identities
in
certain
contexts
or
who
gets
to
be
what
identities
right
and
what
structural
constraints
exist?
To
prevent
a
certain
person
from
being
able
to
to
take
on
a
particular
identity,
okay
and-
and
so
my
interest
in
in
social
media
and
studying
identity-
is
that
it's
a
very
rich
source
where
on
some
social
media,
not
all,
but
some
social
media
people
present
their
identities
to
us
in
a
relatively
structured
way.
B
Okay,
so
there's
two
key
Parts
here:
one
is
that
we
get
to
see
something
about
their
identities.
The
second
key
part,
which
is
really
the
talk
today,
is
that
it's
only
relatively
structured.
So
actually
we
have
to
do
quite
a
bit
of
work
conceptually
and
methodologically
to
pull
out
meaningful
signifiers
of
social
identity
right.
But
if
we
can
do
that,
I
think
we
can.
B
Then
we
get
to
have
sort
of
this
dual
win
in
some
ways
where
we
get
to
use
social
identity,
both
to
better
understand
from
a
social
science
perspective,
what
identity
is
and
also
we
can
use
those
identities
to
understand
user
Behavior
right
so
like
a
quick
example.
Here
are
two
tweets,
one
of
which
I
think
you
could
align
with
my
identity
that
I
present
there
as
a
father
and
the
other.
B
That
makes
a
lot
of
sense
in
my
identity
as
as
a
professor
right
and
so
the
identities
that
I
had
there
could
have
helped
to
predict
this
Behavior.
This
tweet
that
I
sent,
but
also
you
could
imagine
that
we
could
use
the
tweets
that
people
send
to
better
understand
identities
and
so
there's
kind
of
like
both.
B
We
can
use
social
media
data
to
to
do
both
of
these
things,
which
is
really
exciting.
Okay.
So
the
challenge
as
I
alluded
to
is
that
actually
determining
which
identities
people
are
presenting
on
social
media
is
is
really
hard.
Okay
and
there's
a
couple
different
reasons
why
it's
hard
I'm
going
to
sort
of
flag,
two
one
is
well
what
what
counts
as
as
an
identity
right.
So
here's
aoc's,
Twitter,
bio,
this
first
first
part
in
blue
right.
B
She
says:
she's,
a
U.S
representative
in
New,
York
District,
the
14th
District
of
New,
York
parentheses,
Bronx
and
queens
okay.
So
from
there
what
do
I,
what
exactly
am
I
pulling
out
as
like
a
single
identity
versus
like
different
identities
that
she's
trying
to
present
right.
So
where
is
like
the?
What
cons
is
like
one
identity?
Okay,
so
that's
one
challenge
the
Second
Challenge
is:
what
do
we
do
with
information
where
there's
clearly
a
signal
to
an
identity?
B
But
you
can't
identify
you
can't
label
the
term
as
the
sort
of
canonical
version
canonical
definition
of
identity,
which
is
a
label
that
someone
is
applying
to
themselves
or
someone
else
right.
So
when
AOC
says
she's,
100
people
funded
with
no
lobbyist,
money,
she's
signaling
an
identity.
This
is
identifying
information,
but
it
isn't
it
isn't
in
and
of
itself
what
we
want
to
call
a
social
identity
in
social
science.
B
Okay,
so
this
paper
is
all
about
trying
to
address
these
challenges.
What
we
do
is
first,
we
we
Define
this
concept
of
a
personal
identifier
right.
The
high
level
here
is
that
we're
going
to
give
up
I'm,
trying
to
only
pick
out
identities
from
bios
and
take
a
slightly
more
General
concept
of
personal
identifiers,
which
is
roughly
like
things
that
signify
an
identity.
B
Okay.
So
once
we
have
this
concept
or
in
reality
actually
as
we're
developing
this
concept,
we're
also
developing
a
method
to
extract
these
personal
identifiers
from
Twitter,
bios,
okay
and
then
the
final
thing
I'm
going
to
do
is
I'm
going
to
show
you
in
a
variety
of
ways
and
even
more
in
the
paper
that
this
this
concept
is
reliable
or
invalid
and
that
the
method
we
have
to
extract.
It
is
like
good
enough
to
do
the
job.
B
C
Yeah
and
your
what
you're
talking
about
here
are
we
and
you
may
get
into
this
but
you're
talking
about
the
concept
of
more
of
a
personal
identifier
versus
when
we
think
of
like
social
identity
I
mean?
Are
we
going
to
be
getting
into
the
social
identity
aspect,
or
is
this
more
of
the
kind
of
labeling
of
the
identifier.
B
Okay,
so
that's
a
good
question:
I
would
say:
personal
identifiers
are
signals
of
social
identity.
Okay,
I
think
that
that
will
address
I.
Think
yes,
maybe
in
a
couple
slides,
I'll
address
your
concern,
but
like
a
social
identity
is
like
the
label
and
personal
identifiers
are
phrases.
People
use
that
signify
one
of
those
labels,
so
I
think
we
will
be
talking
about
social
identity.
Billing.
B
The
first
thing
that
I'm
going
to
talk
about
is
like
what
a
personal
identifier
is
and
again
and
sort
of
how
we
got
there
so
again,
like
the
general
idea
here
is
that
it's
really
hard
to
pick
out
these
ways
in
which
people
are
labeling
themselves
that
fit
the
canonical
definition
of
a
social
identity,
okay
and
so
we're
gonna
back
out
and
say:
okay.
Well,
here
are
some
words
and
phrases
that
seem
to
Signal
personal
identif,
personal
identity,
and
we
think
those
are
useful
too.
Okay.
B
In
his
work,
he
identified
a
bunch
of
delimiters,
and
essentially
we
were,
we
ended
up
doing
the
same
thing
right,
so
we
constructed
so
once
we
identified
this
this
notion
that
we
should
split
up
the
profile
into
a
bunch
of
different
segments.
Then
we
we
began
this
iterative
process,
which
kind
of
helped
us
to
develop
our
method
and
to
develop
this
concept
of
a
personal
identifier,
okay.
B
So
the
first
thing
is
like:
let's
construct
a
regular
expression
that,
like
cuts
on,
let's
say
periods
and
commas,
and
those
like
long
slash
things
that
I
don't
actually
know
what
they're
called
right
so
split
out
bios
into
using
those
different
delimiters
and
then
on
those
subsets
of
BIOS
do
qualitative
analysis
to
say
well
what
are
these
things
that
we're
extracting?
What
do
they
look
like?
What
can
we
do
with
them?
B
Okay,
and
once
we
came
to
both
a
definition
of
of
that
like
what
was
interesting
at
the
start,
what
was
not
and
how
to
split
the
BIOS.
We
did
some
sort
of
validation,
so
usually
that
validation
amounted
to
some
measure
of
iterated
reliability,
and
we
just
iterated
this
process
over
and
over
and
over
improving
the
regular
expression,
improving
this
concept
of
what
we
wanted
to
extract
from
bios
and
validating
until
we
had
convergence
in
the
validation
metrics.
B
Okay,
so
this
process
gave
us
both
like
how
we
were
going
to
split
up
the
profile
bios,
which
was
a
huge
part
of
our
method,
and
it
also
gave
us
this
concept
of
a
personal
identifier
which
basically
is
like
what
are
the
things
that
are
valuable
to
extract
from
bios,
okay
and
then
after
we
did
all
this
again,
we
sort
of
I
just
want
to
emphasize
is
that,
like
our
validation
was
not
like
within
this
iteration,
it
was
once
all
of
this
was
done.
Then
we
began
our
evaluation
of
our
method.
B
Okay,
so
this
is
kind
of
how
we
did
this.
How
we
defined
a
personal
identifier
here
is
like
the
formal
definition,
which
is
by
and
large,
let's
pull
out
things
that
signify
social
identity
and
importantly
also
for
us
was
I.
Don't
need
any
additional
context
and
I'm
pretty
sure
that
the
identity
that
it's
signaling
to
me
is
the
same
identity
that
you
meant
to
show.
Are
you
meant
to
display?
Okay
and
so
actually,
I?
B
So
so
it's
clear
that
this
person
is
signaling
religion,
but
it's
not
clear
like
which
specific
religion-
okay,
so
this
was
actually
an
education.
I
think
this
was
a
one
that
we
went
back
and
forth
on,
but
ultimately
this
fell
into
the
I
need
more
context.
Okay
and
the
final
one
is
formerly-
and
of
course,
right
formally
means
something,
so
they
they
did
something
before,
but
but
formerly
what
right
so
formerly
on
its
own
is
not
just
like
a
way
we
identify,
or
it
doesn't
even
signal
anything
we
might
identify
with
okay.
B
So
those
are
like
not
personal
identifiers,
and
then
here
are
five
ways
that
we
saw
personally
saw
social
identity
being
expressed
that
we
did
label
as
personally
identify.
So
of
course
like
if
people
had
identities
in
their
profile-
and
they
often
did
we
extract
we,
we
want
to
count
those
as
a
way
of
personally
identifying
as
identifying
also
preferences.
B
So
a
lot
of
people
said
things
like
I
love,
basketball
to
us
that
signaled
sort
of
directly
the
identity,
basketball
lover
that
a
lot
of
other
people
had
in
their
profiles,
personal
descriptors,
so
people
said
I
am
old.
We
thought
that
it
was
reasonable
that
that's
a
an
identifier
or
a
signal
of
the
Identity
or
person
people
often
had
like
affiliations
to
political
or
social
groups
like
hashtag
bernie2020,
which
signifies
the
identity,
Bernie
supporter
and
actions
not
like
hugging,
but
like
hooping.
D
E
E
B
Yeah,
so
that
that's
that's
another,
really
good
education,
so
who
needs
the
context?
And
of
course
the
the
underlying
piece
there
is
like
who
the
annotator
is
is
something
that
I
I,
don't
think,
frankly,
figured
out
like
I
think
you'll
have
to
like
again
our
quotable
algorithm,
which
I'm
going
to
show
you
this
in
the
next
slide,
which
is
extremely
simpler,
simple,
like
it
has
a
perspective
right
so
like
the
algorithm
itself
has
like
a
cultural
perspective.
B
You
know
and
I
think
that
is
hard
to
say
with
such
a
basic
model
like
if
we
trained
it
on
annotations,
you
could
say
well,
like
all
the
annotators
had
the
US
cultural
context,
so,
like
our
models
should
do
and
that's
how
we're
defining
context
here,
I
think
it's
like
pretty
vague
and
open
for
like
future
work.
I,
don't
know
if
that
answers
your
question.
Michael
yeah
I
was
just
curious.
E
B
Yeah
and
so
I
think
it
was
important
rights,
and
you
know
I
raised
in
U.S
cultural
context.
I
also
went
to
Michigan
for
undergrad,
so
like
go,
Blue
was
like
a
very
silly,
an
example
of
like
I
think
it
was
nabid.
Even
it
was
like
this
is
someone's
talking
about
a
color
and
I'm
like
no
actually
for
better
or
worse.
B
That's
like
a
very
clear
signifier
of
like
this
Niche,
and
so
actually
that's
why
we
wrote
about
it
in
the
paper,
because
I
don't
think
we
have
an
answer
to
how
to
do
that.
But
it's
clear
that
it's
a
problem
that
can
only
be
solved
by
defining
the
cultural
context
in
which
you're
going
to
claim
context.
Yeah
yeah.
D
B
B
I
think
it's
fair
to
say
like
okay,
well
like
why,
and
certainly
we
got
this
in
some
of
our
reviews-
Michael
I,
don't
know
if
you
saw
this
near
too,
like
well,
okay,
why
are
you
doing
this
so,
like
part
of
our
evaluation
is
like
I'm
gonna?
Try
to
try
to
convince
you
that
there's
a
reason
to
do
this.
B
Okay,
cool,
so
thank
you
for
those
questions.
Those
are
great
all
right,
so
here
now
we've
defined
like
what
we
want
to
pull
out.
How
do
we
actually
do
that
and
again?
This
is
the
method.
I
will
emphasize
that
the
key
here
is
that
it's
very,
very
simple,
so
I
think
there's
a
lot
of
room
to
grow
here
and
you
could
do
some
interesting
like
distance
supervision
or
like
any
number
of
things,
but
for
us
here,
because
we
were
sort
of
introducing
a
new
concept
as
well.
B
We
split
on
the
delimiters
that
we've
ident
we
identified
in
the
prior
step,
and
then
we
do
some
like
basic
cleaning
so
like
remove
the
phrase
like
I,
am,
etc,
etc,
and
then
we
filter
out
some
like
obvious
non-cases
So,
like
things
with
excessively
long
phrases,
we
noticed
were
almost
always
not
anything,
meaningful
and
so
on
and
so
forth.
Okay,
so
it's
basically
a
glorified
regular
expression
with
some
cleaning
and
filtering.
B
So
once
we
have
this
tool,
I
just
want
to
show
so
now
that
we
have
this
tool
right.
I
just
want
to
show
some
output
to
show
like
the
the
the
flaws
in
the
tool
right
so
but
also
sort
of
the
utility
right.
So
here's,
my
bio,
here's,
the
bio
of
Biden
and
it's
pulling
out
like
things
that
we
might
want
it
to
pull
out
like
him,
studying
people
with
computers.
Those
seem
reasonable.
B
Vice
versa,
probably
something
we
would
not
want
for
Biden
husband
too,
is
separated
from
like
who
he
is
the
husband
to.
Maybe
we
want
that?
Maybe
we
don't
right.
Tweets
may
be
archives.
Clearly,
we
don't
but
46,
president
of
the
United
States,
like
that's,
probably
the
thing,
the
main
thing
that
we
would
want
to
pull
up.
Okay,
so
this
thing
seems
like
well:
maybe
it
kind
of
works
it
pulls
out.
Some
useful
stuff
pulls
out
some
stuff.
It's
not
useful!
So,
like
all
right.
B
The
first
is
manual
annotation
and
we
use
manual
annotation
for
integrator
reliability
to
evaluate
the
validity
of
a
concept
and
then
accuracy
of
this
method
right.
So
we
pull
out
a
bunch
of
things,
pull
out
a
bunch
of
like
phrases
within
bios
and
code
them
manually
for
whether
or
not
they're
personal
identifiers,
okay.
The
second
thing
we're
going
to
do
is
compare
compare
how
people
with
different
demographics
and
different
levels
of
status
as
marked
by
Twitter,
how
their
personal
identifiers
vary
and
we're.
B
Looking
here,
both
at
convergent
validity
like
do
identifiers,
we
should
expect
to
match
certain
demographics
actually
match
those
demographics
and
also
discriminate
validity.
So
again,
like
discriminant
validity,
addresses
this
question
here
of
like
well.
Does
it
tell
us
something
above
and
beyond
demographics?
That's
useful!
That's
also
sort
of
the
context
in
which
we
evaluate
utility,
so
I'll
show
a
quick
like
clustering
example
and
talk
about
why
we
think
these
things
are
useful.
B
B
B
We
we
labeled
a
bunch
of
different
sub
samples
of
this
data
and,
like
the
agreement
levels
were
somewhere
between
in
terms
of
criminals
Alpha
between
like
0.6
and
0.8,
so
the
takeaway
here
is
like
we
eventually
got
to
a
concept
of
personal
identifier,
that's
pretty
robust
that
we
think
can
be
applied
and
used
by
other
Scholars,
with
a
reasonable
definition
of
what
it
means.
Okay,
the
second
piece
is
is
accuracy
right,
so
if
so,
we
took
a
couple
hundred
phrases
that
have
been
extracted
from
bios.
B
We
said
well
what
percentage
of
them
in
these
three
different
data
sets
are
personal
identifiers,
okay,
and
so
the
the
main
takeaway
here
is
like
the
majority
of
them.
Are
the
vast
majority
for
the
panel
data
sets
not
necessarily
the
vast
majority
for
the
random
sample.
Okay,
so
like
in
this
data
set
this
panel
data
set
where
we
are
very
or
pretty
certain
that
we're
looking
at
people,
this
thing
performed
pretty
well
on
the
random
subset.
It
performed
less
well.
I
think
there
are
a
couple
reasons
for
that.
B
One
is
that
the
random
profiles
do
I,
think
look
a
little
different,
like
I
said
than
the
panel
profiles,
and
so
I
think
some
tweaks
would
be
needed
or
something
really
fancy.
The
other
thing,
though,
is
I,
think
basically
a
study
designed
law,
so
we
extracted
from
the
random
sample
both
English
and
Spanish
profiles
and
then
restricted
our
annotation
to
not
label
anything
in
a
foreign
language
as
a
personal
identifier,
unless
we
directly
knew
the
translation,
and
so
that's
the
reason
why
this
Green
Dot,
which
is
observations,
will
be
labeled.
B
D
B
Again,
it's
a
simple
method:
I'm
not
trying
to
defend
or
claim
that
like
there's,
not
ways
to
improve
it,
but
like
I,
also
do
think
that
you
know
I
think
this
thing
is
useful
on
most
Twitter
data
sets
kind
of
kind
of
as
this
okay,
so
switching
over
to
questions
about
convergent
validity.
So,
basically
again,
this
is,
does
this
thing
kind
of
do
what
we
would
expect
when
we
compare
it
to
demographics
or
to
other
things,
I?
Think
by
and
large.
B
B
B
I
can
say
that's
the
log
odds
of
followers
to
friends,
so
people
who
have
higher
values
here
have
more
followers
than
they
do
friends
on
Twitter,
and
so
these
are
the
top
10
personal
identifiers
for
people
with
higher.
What
we're
calling
status
here-
and
these
are
the
the
bottom
tips.
Okay
and
I
think
like
just
looking
up
here
like
again
I,
think
it
sort
of
checks
out
face
solidity,
wise
that
we
would
we're
seeing
what
we
would
expect
here
now.
B
B
Okay,
the
same
thing
in
white
versus
Asian,
here
Asian
means
only
one
thing
on
the
demographic
data
that
we
have,
but
of
course,
Asian.
This
sort
of
broad
Asian
identity
means
a
lot
of
things
to
a
lot
of
people,
and
we
can
see
again
here
like
what
specifically
it
means
to
be
Asian
to
the
to
certain
individuals,
okay
and
I'm,
going
to
show
in
a
couple
slides
that
I
think.
Actually.
B
Not
only
does
this
help
us
understand
who
we're
looking
at
better
but
I
think
it
can
help
predict
Behavior
better
as
well.
Okay,
so
the
final
thing
we
did
was
we
looked
at
how
these
personal
identifiers
clustered,
so
the
high
level
is
like
two
identifiers
cluster
to
the
extent
that
they
tend
to
show
up
in
the
same
profiles.
Okay,
so
here's
like
a
list
of
all
the
interesting
clusters
we
found-
and
this
is
like
a
pretty
diagram
of
it.
I
don't
know
I
just
think.
B
It's
super
pretty
not
sure
it's
meaningful,
but
I
want
to
I
want
to
just
drill
down
to
to
these
three
because
I
think
if
you
read
through
these
so
each
each
chunk
here.
These
are
three
separate
clusters
and
I.
Think,
if
you
look
at
these-
and
this
is
going
to
be
a
very
informal
statement-
I'm
happy
to
discuss
it
like-
if
you
look
at
these
to
me,
these
kind
of
like
make
sense
together
these
identities,
just
like
feel
like
they
go
together.
E
B
Oh
okay,
so
last
couple
slides
like
where
are
we
headed
now,
so
one
of
the
things
that
our
method
doesn't
do
either
theoretically
or
methodologically?
Is
they
say
like
well?
When
do
two
personal
identifiers
mean
the
same
thing
right
and
so
I
think?
That's
actually
really
a
really
hard
conceptual
question
that
I
I
don't
have
an
answer
to
like
is
a
father
of
two,
the
same
thing
as
a
father
of
two
boys.
Well
like
Maybe.
B
B
If
you
look
at
trying
to
extract
a
very
similar,
similar
measure
of
similarity
in
a
similarly
constructed
embedding
space
using
the
profile
data,
you
get
like
other
phrases
that
we
think
better
align
with
Mom
and
so
starting
to
think
about,
like
this
concept
of
like
similarity
between
people
as
being
distinct
from
semantic
similarity,
which
is
what
word
vectors
try
to
encapsulate.
Is
it's
really
interesting
for
us?
B
So
that's
like
more
in
the
social
psych
side
on
the
Practical
side
right
so
here
is
just
like
the
personal
identifiers,
so
we
put
like
personal
identifiers
into
aggression
to
predict
fake
new
sharing
very
quickly
right.
This
is
is
a
pretty
informal
analysis
and
we
can
see
how
like
we
can
start
to
right.
So
the
the
canonical
knowledge,
as
far
as
I'm
aware,
is
like
it's
older,
right-leaning
adults
that
share
misinformation,
but,
like
I,
think
we
can
drill
down
much
further
when
we
think
about
them.
B
D
A
B
E
E
E
Yes
yeah:
this
is
awesome.
Work
I
had
a
couple
other
questions,
so
I
guess
like
looking
at
the
some
of
the
Divergent
validity
things
you
were
showing
was
really
interesting
and
also
in
the
Clusters
I.
Guess
it's
sort
of
like
the
cluster
sort
of
made
sense,
like
you
said,
but
sometimes
they
weren't
strictly
sort
of
categorized
by
how
usually
social
scientists
sort
of
divide
up
these
identities
like
these,
like
I,
don't
know
what
you
call
them
Dimensions,
or
what
categories?
What
have
you
like,
gender
or
race
or
religion?
E
B
Yeah
I
I
mean
for
me
dating
back
to
to
my
time
in
Dr
Perry's
lab,
like
that's,
been
the
real
question
of
I,
think
there
are
other
sort
of
more
it's
the
most
interesting
theoretical
question
like
what
is
it?
Those
like
I
said
those
make
sense
together.
If
I
asked
you,
if
we
did
some
sort
of
like
intrusion
task
right
like
you
would
be
able
to
figure
it
out
like
exactly
why
you
would
be
able
to
do
that.
B
I
I,
don't
I,
don't
have
a
good
sense.
My
my
view
would
be
that
they
are
similar
on
some
dimension
of
perception
about
people.
But
what
so?
That's
that's
a
hypothesis
right,
of
course,
maybe
of
course
I
don't
know
but
like
how
we
characterize
that
Dimension
we're
turning
now
to
a
data
driven
way
of
characterizing
that
Dimension,
which
is
really
exciting,
I,
just
don't
know
how
to
do
it.
Theoretically,
yet,
okay.
E
E
There
were
lots
of
personal
identifying
and
for
not
like
personal
identifiers
sort
of
things
like
people
were
presenting
their
identities
in
lots
of
places
on
their
profiles
outside
of
the
little
the
text,
bio
box,
I,
I
kind
of
have
the
feeling
that
on
Twitter
this
is
maybe
a
little
less
so
so
on.
On
Tumblr,
like
usernames,
often
had
a
lot
of
information
on,
like
you
know,
yeah
Star
Wars
fan
lover
whatever
and
then
also
also
photos.
I.
D
E
B
A
Thank
you
if
I
may
ask
a
question-
and
this
is
very,
very
interesting
work
and
we
can
see
so
many
implications
to
it
in
marketing
business,
political
communication,
ad
campaigns
and
many
much
more
right,
so
I'm
wondering
I
have
two
thoughts.
One
is
have
you
thought
of
how
a
Time
plays
a
role
in
the
evolution
of
these
social
identities?
That
is
the
first
question,
and
is
this
possible
to
model
that
evolution
of
social
identities
at
a
macro
scale?
A
The
other
question
that
I
have
is,
let's
say:
if
someone
takes
the
findings
from
your
research
and
starts
to
build
a
predictive
or
machine
learning
model
for
folks,
let's
say
you
know
you
have
their
declarations
on
their
bio
and
then
their
inferences
based
on
their
declarations,
and
of
course
much
of
this
is
done
based
on
what
is
represented
in
these
large
data
sets.
Sometimes
these
could
feed
into
biases
of
those
algorithms
and
May,
and
those
algorithms
May
perpetuate
those
biases
or
amplify
certain
biases.
B
Yeah,
that's
both
awesome
questions
in
terms
of
the
temporal
Evolution,
there's
been
a
little
bit
so
like
as
we
were,
and
Michael
as
we
and
Michael
were
developing
some
of
these
ideas.
There's
another
person
Jason
Jones
at
Stony
Brook,
who
was
thinking
a
little
bit
about
that
this
I
think
he's
had.
He
has
some
work
that
I
can
share
with
the
group
on
how
certain
specific
identities
have
evolved
over
time
and
I.
B
What
it's
hard
to
think
about
like
what
the
algorithm
is
that
would,
okay,
so
maybe
I'll
take
your
question
in
a
particular
way,
just
like.
Let's
come
up
with
a
marketing
context
where,
like
people
are
going
to
use
these
labels
to
only
target
their
marketing
towards
a
specific
kind
of
person,
we
have
a
pretty
lengthy
section
in
the
paper
about
this
and
we
went
back
and
forth
on
it
for
a
while
I
think.
My
justification
to
myself
whether
or
not
this
is
like
a
valid
justification.
B
A
Certainly,
thank
you
for
the
thoughts
I
was
curious
when
I
saw
a
mention
of
word,
she
or
her
in
the
bio
leans
more
towards
Democrats
and
conservatives,
so
those
type
of
labelings
could
that
feed
into
unfounded
biases
and
perhaps
a
predictive
modeling
type
algorithm,
could
feed
into
that.
Could
feed
from
that
bias
and
perpetuate
it
even
more.
So
that's
the
the
direction
I'm
coming
from.
But,
of
course
this
is,
you
know
we're
not
predicting
that
it's
just
a
matter
of
what
the
data
is
showing,
but.
B
So,
but
that's
that's
a
really
good
point
of
like
you
have
to
be
really
careful
about,
like
yeah
and,
admittedly
right,
like
people
can
take
our
methods
and
do
what
they
want
with
them,
and
so
so
it's
a
concern.
I
think
there.
It's
so
important,
of
course,
to
remember
like
that,
we're
looking
at
how
people
are
presenting
themselves
right
once
you
once
you
say,
like
okay,
Democrats
are
more
willing
to
present
their
pronouns
on
Twitter.
B
Then
it
makes
a
lot
of
sense,
but
like
you're
saying
like
you're
saying,
if
you
just
take
that
and
like
oh,
we
can
do
gender
gender
inference
using
this
tool.
Then
that's
like
going
to
be
problematic
in
a
number
of
ways.
So
yeah,
that's
a
really
good
point
that
I
I
admitted
the
had
not
thought
about
until
you
said
it
so
I'll
have
to
take
that
back
and
tune
it.
A
C
B
Yeah,
the
only
thing
so
the
the
closest
we've
come
to
that
Billy.
The
short
answer
is
no.
We
have
it.
The
closest
that
we've
come
is
we're
doing
some
work,
trying
to
look
at
supporters
of
Bernie
Sanders
Jay.
This
is
a
very
different
realm,
but
like
supporters
of
Bernie
Sanders
during
the
2020
DNC,
and
we
used
this
approach
in
combination
with
with
some
other
approaches
to
try
to
identify
a
group
of
Bernie
supporters
and
all
I
can
say.
B
B
They
all
or
the
vast
majority
of
them
seem
to
be
Bernie
supporters.
So
it's
not
like
people
seem
to
be
using
these
things
to
like
co-opt
support
but
like
as
true
signals
of
who
they
think
they
are
and
I
think
like
in
combination
with
the
social
network,
with
in
combination
with
The,
Social,
Network
I
think
these
can
be
useful
signals
for
you
know:
group
identity,
yeah,
hello,
that's.
C
A
And
if
not,
that's
totally
fine,
too,
because
we
do
have
a
couple
of
announcements
so
again.
Thank
you
so
much
Kenny
for
doing
this
presentation.
As
you
can
see,
it's
not
usual
that
we
have
so
many
questions
discussions.
You
know
throughout
the
talk,
as
well
as
after
the
talk.
So
thank
you
again
for
doing
the
presentation
now.
I
also
would
like
to
invite
everyone
here
to
visit
our
SBP
prints
website.
We
have
the
conference
annual
conference,
social
Computing,
behavioral,
cultural
modeling
and
prediction.
A
That's
going
to
be
held
from
September
20th
to
23rd
this
year
and
the
call
for
papers
is
out.
So
if
you
have
research,
you
would
like
to
submit
go
ahead.
Please
check
out
our
call
for
papers.
We
also
have
doctoral
Consortium,
which
is
actually
led
by
Professor
Joseph.
So
if
you
have
doctor
students,
you
are
a
doctoral
student
or
a
graduate
student
exploring
research,
please
check
the
call
for
papers
or
call
for
submissions
for
the
doctrine
Consortium
on
the
website.
A
A
D
A
As
always,
we
give
Awards
to
the
best
submission
in
the
challenge
problem
and
the
best
submissions
also
get
invited
for
a
special
issue
at
the
c-mart
journal
that
is
led
by
Dr
Kathleen
okay.
So
there
are
a
lot
of
other
information
on
the
website.
Please
go
check
out
the
website
and
I
encourage
everyone
to
submit
your
work
or
your
students
will
encourage
them
to
submit
their
work
and
again.
Thank
you.
Everyone
for
joining
us
this
morning
for
the
cyber
security
working
group.