►
From YouTube: The Ideology of Content on YouTube
Description
Sept. 2nd, 2021
Presenter: Megan Brown, James Bisbee,and Angela Lia
Bios:
Megan is a research engineer and research scientist at NYU's Center for Social Media and Politics. Megan researches disinformation, content moderation, and quantitative methods for measuring online behavior on Twitter, YouTube, and Reddit.
James is a postdoctoral fellow at NYU’s Center for Social Media and Politics. Previously, James was a postdoctoral fellow at Princeton’s Niehaus Center for Globalization and Governance. James received his PhD from NYU's Department of Politics in 2019. James now studies how economic dislocation influences politics using novel data -- ranging from Twitter and YouTube to the U.S. Census -- and quantitative methods.
Angelais a data science Ph.D. student at NYU working in the Center for Social Media and Politics. Her research interests include network analysis and natural language processing.
A
That
I'm
really
excited
about
today's
presentation.
We
have
kind
of
a
three-pronged
approach:
three
different
presenters
on
the
same
topic
or
varying
degrees
of
it,
so
I'll
go
ahead
and
introduce
them.
Let's
see
here
so
first
we
have
James
James
Bisbee
James
is
a
postdoctoral
fellow
at
NYU
Center
for
social
media
and
politics.
Previously
James
was
a
postdoctoral
fellow
at
Princeton's,
nigh
Hospital
Center
for
global
global
I'm.
Sorry
for
globalization
and
governance.
A
Megan
is
also
with
us.
Megan
is
a
research
engineer
and
research
scientist
at
New,
York,
University
Center
for
social
media
and
politics,
Megan
researches,
disinformation,
content,
moderation
and
quantitative
methods
for
measuring
online
behavior
on
Twitter
Youtube,
as
well
as
Reddit
and
Angela's
with
us
as
well.
Angela
is
a
data
science
PhD
student
at
New,
York,
University,
working
in
the
center
for
social
media
and
politics.
Her
research
interests
include
network
analysis
and
natural
language
processing.
So,
thank
you
all
so
much
I
know
that
was
really
brief.
B
B
Okay,
great,
then
I'll
go
ahead
and
get
started
so
Chris.
Thank
you
for
the
introduction.
I'm
Angela,
Y
and
I'll
be
kicking
us
off
with
the
first
of,
as
you
said,
in
three
short
presentations
about
our
work
on
YouTube
at
the
center
for
social
media
and
politics,
so
I'll
be
starting
off
with
the
more
methodological
presentation
and
then
Megan
and
Jim
will
be
talking
about
some
more
substantive
work.
B
So
first
things
first,
why
do
we
want
to
study
YouTube?
Well,
YouTube
is
one
of
the
most
popular
websites
in
the
world.
It's
actually
the
second
most
popular
after
Google
and
according
to
the
Pew
Research
Center,
it's
one
of
or
it
is
the
most
widely
used
online
platform
among
us,
adults,
and
something
that's
really
notable
about
YouTube
is
that
it
has
a
recommendation
algorithm,
which
has
been
reported
to
drive
70
percent
of
the
views
on
the
platform.
B
In
order
to
do
this,
we've
developed
a
method
where
we
first
use
Reddit
data
to
obtain
video
ideology,
estimates,
and
then
we
train
a
natural
language
processing
model,
so
a
model
that
takes
in
text
Data
as
the
features,
and
then
this
model
is
basically
predicting
ideology
scores,
and
this
means
that
we
can
then
predict
the
ideology
for
videos
which
we
haven't
seen
before
foreign
and
the
key
advantage
of
this
approach
that
I
want
to
highlight.
Is
we
don't
need
human
labels
this
way?
B
This
is
something
that's
really
important,
given
the
rapidly
changing
political
landscape
right
from
month
to
month.
There's
a
lot
of
new
Salient
political
issues
and
we
want
to
develop
an
approach
which
can
quickly
adapt
along
with
that
now,
there's
a
really
there's
a
pretty
significant
subfield
of
political
science
work
where
scientists
are
estimating
ideology
of
different
groups
right.
So,
for
example,
pretty
traditionally
we
can
see
you
can
observe
how
legislators
like
vote
on
bills,
for
instance,
or
how
justices
rule
on
cases
and
then
from
observing
these
behaviors.
B
We've
looked
at
past
work
like
from
Pablo
Barbara,
where
they
use
the
following:
behavior
of
Twitter
users
to
estimate
the
political
ideology
of
those
Twitter
users
and
sort
of
in
the
same
vein.
We're
thinking
that
if
we
can
observe
how
Reddit
users,
post
YouTube
videos,
or
rather
how
YouTube
videos
appear
on
different
subreddits,
we
can
sort
of
treat
that
in
a
similar
way
right
and
also
use
that
behavior
to
try
to
discern
the
ideology
of
YouTube
videos.
B
So
to
go
in
a
little
more
detail.
Basically,
we
start
off
with
political
subreddits
and
which,
if
you're
unfamiliar
a
subreddit,
is
essentially
a
community
on
Reddit,
which
generally
centers
around
shared
interests
or,
in
the
political
case,
usually
a
shared
ideology.
So,
for
example,
if
you
look
on
conservative
right,
you're
going
to
see
a
lot
of
videos
that
presumably
align
with
the
conservative
ideology
and
so
on
and
so
forth,
with
standards
for
president
and
mutual
news.
Next,
we
look
at
the
YouTube
videos
that
are
posted
on
those
subreddits,
and
so
the
thought
is.
B
If
we
look
at
a
lot
of
different
subreddits
and
then
we
observe,
for
example,
a
video
is
getting
posted
in
the
conservative
subreddit
and
in
the
Donald
or
in
libertarian.
Then
we
might
think
that
this
video
is
especially
of
interest
to
people
who
belong
to
those
ideological
groups.
Probably
they
agree
with
the
ideas
expressed
in
that
video,
so
we
ultimately
end
up
constructing
a
matrix
with
around
60
000
YouTube
videos-
and
this
is
these-
are
the
rows
of
our
Matrix
and
then
on
the
columns.
We
have
the
political
subreddits
and
this
is
somewhere.
B
We
ultimately
end
up
with
around
800
subreddits
and
we
take
this
Matrix.
We
run
correspondence
analysis
and
then
we
end
up
with
what
we
treat
as
ideology:
estimates
for
the
YouTube
videos
and
so
to
sort
of
validate
that
we're
kind
of
getting
at
some
notion
of
ideology
in
this
plot
here
I
have
the
video
scores
grouped
by
a
channel,
and
this
is
just
a
small
selection
of
really
popular
channels.
So,
basically,
on
the
x-axis,
we
have
the
video
ideology
score.
So,
for
example,
what
you
see
here
this
violin
plot
for
the
progressive
voice?
B
B
These
videos
are
pinned
pretty
quite
far
to
the
left
and
then,
for
example,
if
you
look
at
Fox,
News
Stefan
Molino
trigger
U
H.
These
are
all
quite
far
to
the
right,
whereas
something
that's
a
little
more
Centrist
like
CNN
or
even
a
mainstream
news
album
like
MSNBC.
These
are
sort
of
more
widely
spread
and
they
seem
to
center
a
little
more
around
the
middle.
B
B
We
use
these
as
our
features,
because
we're
going
to
have
this
data
for
every
single
video
that
we
encounter
right.
So
we
don't
have
the
requirement
of
this.
Video
needs
to
be
posted
on
Reddit
in
order
to
estimate
a
video's
ideology,
we're
sort
of
taking
this
text
metadata.
That
is
totally
separate
from
this
stage
of
ideology.
B
Estimation
right,
then,
as
a
label,
we
use
the
ideology
estimate
from
Reddit
and
we
take
these
inputs
to
train
a
natural
language
processing
model,
namely
the
bird
model
which
is
going
to
Output,
continuous,
continuous
predictions,
basically
and
ultimately,
the
as
I
sort
of
mentioned.
The
main
benefit
of
this
is
that
we
can
then
predict
the
ideology
of
other
similar
videos,
even
if
we
don't
have
regida
for
them
and
for
new
videos
that
we
encounter
so
next
I'll
be
showing
you
the
performance
of
the
text
model,
and
here
on
this
plot
we
have.
B
The
red
is
for
the
ground
truth
on
the
x-axis,
and
you
can
see
this
ranges
from
around
negative
1.5
to
1.5
on
the
y-axis.
We
have
the
predicted
score
from
the
tech
from
the
text
data,
and
this
is
all
on
a
test
set,
and
here
I
have
basically
a
regression
line
which
is
fitted
to
this.
Data,
which
you
can
see
is
fairly
close
to
a
45
degree
angle
line,
which
is
what
we
would
desire
and
we
end
up
getting
an
r
squared
value
of
around
0.8
and
our
MSE
is
around
0.16.
B
B
So
here
I
have
the
accuracies,
and
these
are
binned
by
the
distance
between
the
ideology
scores
of
the
top
video
and
bottom
video.
So,
first
off
right.
Overall,
our
video
scores
agree
with
human
coders
around
78
of
the
time
and
in
this
plot,
I'm,
basically
showing
how
that
changes
as
the
distance
between
the
top
video
and
the
bottom
video
increases.
So
you
can
see
here
that
we
have
five
different
bins
and
as
we
as
the
videos
get
further
from
each
other,
we
get
a
higher
percentage
of
agreement
with
the
human
coders.
B
B
Next
I
have
a
visualization
that
kind
of
gets
at
the
same
thing
right
where
we
essentially
illustrate
that
our
accuracy
improves
as
the
distance
between
the
video
scores
increases
and
again
on
the
x-axis.
We
have
just
the
pure
distance
between
the
scores
of
the
top
and
bottom
video
and
then
on
the
y-axis.
We
have
very
simply
an
indicator
of
whether
or
not
we
agreed
with
a
human.
C
C
There
are
some
things
that
we
know
about
YouTube
in
the
qualitative
sense.
Some
research
by
other
researchers
and
journalists
posits
that
YouTube's
recommendation
engine,
which
Angela
briefly
discussed,
is
recommending
increasingly
extreme
content
to
people
the
longer
they
watch
political
content
on
the
platform.
C
C
These
studies
either
use
past
web
browsing
history
or
use
recommendations
collected
via
Bots
with
no
YouTube
history
and
because
we
know
that
YouTube
watch
history
and
YouTube
accounts
are
such
an
integral
part
of
what
gets
recommended.
We
decided
to
field
a
survey
where
we
asked
individuals
to
log
in
to
their
YouTube
account
and
follow
recommendations
and
the
way
that
we
instructed
them
to.
So
what
we
did.
C
We
asked
them
a
short
survey
on
how
they
use
YouTube.
They
could
upload
their
YouTube
watch
History
if
they
wanted
to
for
additional
compensation
and
then,
finally,
we
used
all
of
this
information
together
to
look
at
how
the
YouTube
recommendation
engine
is
recommending
ideological
content
to
users.
C
So
we
start
with
our
research
question:
does
YouTube's
recommendation
algorithm
push
users
into
ideological
Echo
Chambers
we're
going
to
show
you
a
couple
of
different
traversals
that
our
respondents
went
on
to
illustrate
this
point
so
on
the
x-axis?
Here
we
have
the
traversal
step,
which
is
how
far
into
this
task.
They
were
how
many
essentially,
how
many
videos
they'd
clicked
on
from
the
starting
video
in
the
survey
and
on
the
y-axis,
we're
going
to
be
showing
the
ideological
distribution
of
the
videos
that
they
were
recommended.
C
C
We
asked
this
individual
to
always
click
on
the
second
video
and
on
their
second
video.
They
got
the
second
distribution
and
so
on
and
so
forth.
What
we
can
see
from
this
user
is
that
generally,
when
we
asked
them
to
randomly
when
we
gave
them
the
random
assignment.
To
always
click
on
the
second
video
they've
received
a
a
wide
range
of
ideological
contents,
not
particularly
narrow,
wasn't
particularly
leaning
in
a
very
conservative
direction,
or
a
very
liberal
Direction.
C
Conversely,
one
of
our
other
respondents
recommendations
looked
like
this.
We
started
them
off
on
a
moderate
video.
Their
first
set
of
recommendations
was
fairly
diverse
and
thereafter
they
had
a
very
narrow
set
of
ideology
and
a
very
in
a
particularly
bright
leading
ideological
distribution
for
the
videos
they
were
recommended.
C
So
first
first
set
of
democratic
users.
We
can
see
that,
regardless
of
the
starting
ideology
of
the
video,
regardless
of
the
video
that
they
start
on
all
of
these
Democratic
users,
when
they
follow
the
recommendation
engine
end
up
going
towards
the
center
right
or
staying
at
the
center
right
again
for
independent
users.
We
see
something
very
similar
when
they
start
on
a
liberal
video.
They
go
towards
the
center
right
when
they
start
on
a
conservative
video.
C
C
We
see
that
when
they
start
on
liberal
videos,
their
average
variance
goes
down
on
conservative
videos.
It
goes
up
slightly
in
all
moderate
videos,
it
stays
the
same
and
for
Republican
users
we
see
that,
regardless
of
their
starting
ideology,
the
variance
of
their
recommendation,
the
ideological
variance
of
the
recommendations
goes
down.
C
We
found
that
more
time
clicking
on
the
recommendations
means
you're
getting
more
conservative,
at
least
towards
the
center
right
and
less
diverse
recommendations,
and
this
trend
towards
more
conservative
and
less
diverse
content
is
strongest
amongst
our
Republican
respondents,
which
does
match
some
of
the
qualitative
work.
That's
been
done
in
this
area
that
a
lot
of
this
Behavior,
that's
being
found,
is
being
found
by
these
more
right-leaning
communities
on
YouTube
some
limitations
of
this
method
we're
just
looking
at
a
snapshot
in
time.
C
C
Also
at
this
point,
we
are
only
looking
at
the
supply
side.
What
YouTube
was
recommending
to
people
when
we
randomized
what
video
they
could
pick.
We
have
not
yet
looked
at
what
this
looked
like
when
users
were
selecting
the
videos
they
most
wanted
to
watch
and
that's
it.
Thank
you
very
much.
I'll
be
handing
this
over
to
Jim
to
talk
about
some
of
our
other
findings
around
election
fraud.
D
D
So
the
motivation
for
the
final
project-
that's
that's
part
of
this
overarching
research
is
investigating
the
degree,
the
possible
degree
to
which
YouTube's
recommendation
algorithm
put
people
into
informational,
Echo
changes
in
which
those
who
are
most
likely
to
believe
Trump's
Narrative
of
election
fraud.
We're
also
going
to
be
having
that
narrative
reinforced
while
they
were
online
and
arguably
this
contributes
to
the
January
6th
riots
in
the
capital.
So,
like
Megan
said
this,
this
survey
that
we
conducted
in
the
fall
between
October,
20th
and
December
1st.
D
We
added
a
battery
of
questions
about
the
presidential
election.
So
in
addition
to
asking
these
respondents
to
Traverse
and
to
scrape
what
they're
being
recommended,
we
also
asked
them
what
they
thought
about
the
the
2020
US
presidential
election.
So
the
goal
of
this
is
to
see,
if
there's
a
relationship
between
a
user's
belief
in
an
election
fraud
narrative
and
whether
they
were
being
shown
things
on
YouTube
that
reinforce
that
worldview.
D
So
the
theory
is
very
straightforward.
We
know,
based
on
Google's
own
research
white
papers,
that
their
recommendation
algorithm
at
a
high
level,
is
designed
to
maximize
watch
time.
It's
designed
to
choose
videos
that
it
thinks
users
will
click
on.
So
our
assumption-
and
this
is
building
off
a
lot
of
work
recently
in
political
science,
about
ideology-
is
that
people
are
homophilus.
They
like
things
that
they
agree
with.
They
don't
like
to
be
challenged,
and
so
the
theory,
our
hypothesis.
That
still
is
out
of
this
fact,
plus
this
assumption
is
very
straightforward.
D
If
the
recommendation
algorithm
is
good
at
maximizing
watch
time,
it
might
result
in
filter
bubbles
that
have
negative
consequences
for
a
deliberative
democracy,
filter
bubbles
in
which
people
who
believe
a
conspiracy
about
election
fraud
are
going
to
hear
that
reinforced
to
the
extent
that
they
take
action
and
commit
violent
criminal
acts.
So
the
the
explanatory
variable
of
interest
for
us
is
this
measure
of
how
skeptical
our
respondents
are
about
the
2020
election.
So
we
asked
a
number
of
questions
like
I
say
between
October
20th
and
December
1st.
D
The
first
ask
them
to
indicate
How
concerned.
They
are
about
the
following
problems
right,
so
we
asked
about
the
possibility
of
fraudulent
mail-in
ballots
being
counted,
valid
ballots,
not
being
counted
whether
they
thought
there
was
widespread
voting
illegal
voting
by
non-us
citizens
or,
more
generally,
if
there
was
interference
or
influenced
by
Foreign
actors.
These
are
all
sort
of
parts
of
the
narrative
that
Trump
and
otherism.
The
conservative
media
were
spinning
to
to
create
this.
This
sense
of
a
stolen
election,
an
unfair
outcome.
D
We
also
asked
them
over
the
course
of
this
month
or
a
month
and
a
half
whether
they
knew
or
whether
we
as
a
country
knew
who
won
the
election
and
so
conditional
on
them.
Saying
yes,
we
do
know
we
asked
first,
who
is
it?
Is
it
Trump
or
is
it
Biden?
And
then
we
asked
whether
this
outcome
is
legitimate
and
then
finally,
we
also
asked
after
I
think
it
was
November.
D
Our
outcome
variable
is
an
evaluation
of
what
they
were
shown
right.
So
we've
got
this
whole
list
of
videos
that
they're
being
recommended.
We
could
rely
on
Angela's
method
to
look
at
the
ideology
of
them,
but
for
this
specific
topic,
it's
a
pretty
narrow
topic.
We
want
to
actually
get
a
little
bit
more
detailed
than
just
sort
of
a
broad
ideology,
and
so
we
use
topic
models
of
the
recommended
videos
metadata.
So
these
topic
models
estimated
via
LDA
latent
to
reach,
lay
allocation
effectively.
D
Look
for
semantic
topics
from
the
co-location
of
words
among
these
different
videos
and
without
belaboring
the
math
here
for
each
topic.
We
can
get
a
probability
of
word
conditional
on
that
topic
denoting
Phi
and
for
every
video
we
get
a
topic
loaded,
the
probability
of
it
being
topic,
K
conditional
on
a
video
Z
denoted
with
Theta,
and
so
what
we
did
after
estimating
these
topic
models
is.
We
then
inserted.
D
We
searched
for
a
set
of
keywords
that
we
had
decided
were
most
likely
to
be
about
The
Narrative
of
election
fraud,
so
things
like
dominion
and
smartmatic.
The
voting
machines
that
Trump
argued
were
were
illegitimate.
Things
like
voter
fraud,
fraudulent
ballots.
All
of
these
different
terms.
We
searched
for
among
these
different
words,
and
then
we
created
a
score
for
each
video
that
added
up
the
fees
these
these
probability
of
the
word
being
associated
with
the
topic
and
then
weighted
them
by
the
loadings
onto
the
video.
A
D
Idea
of
of
what
the
highest
scoring
topics
look
like
the
most
likely
topic,
the
topic
that
we
think
is
most
focused
on
this
Narrative
of
election
fraud
was
topic
number
108,
the
highest
scoring
terms
here
were
news,
Trump
election
Donald
Trump,
and
then
there
were
37
of
these
keywords
that
we
had
searched
for
right.
So
things
like
fraud,
Rudy,
Giuliani,
Sydney,
Powell,
recount
fake
news.
D
When
we
look
at
the
actual
videos
themselves,
we
find
that
the
highest
scoring
videos
associated
with
topic
108
are
either
from
the
White
House
and
are
our
press
briefings
around
the
time
of
the
election
where
Trump's
press
secretary
was
literally
laying
out
this
Narrative
of
a
stolen
election,
or
they
were
from
a
subsidiary
of
Fox,
News
called
NewsNow
and
based
on
these
titles,
we
can
again
sort
of
confirm
that
this
topic
is
not
only
about
election
fraud
writ
large,
but
it
is
taking
a
stance.
It's
it's
reinforcing.
D
So
it's
a
very
simple
regression:
we're
just
going
to
predict
the
proportion
of
videos
about
fraud
as
a
function
of
the
user
level,
self-reported
skepticism
and
our
hypothesis.
Our
theory
is
straightforward,
like
if
the
algorithm
Works
these
users
should
be
being
shown,
content
more
fraud,
content
than
other
users,
I'm
happy
to
talk
a
little
bit
more
about
the
specification,
we're
not
going
to
be
controlling
for
use
of
demographics
in
this,
it's
I
think
going
to
be
a
post-treatment
variable,
but
we
can
talk
more
about
that.
D
If
there's
questions-
and
this
is
going
to
be
our
main
Marquee
result
right.
So
this
is
for
topic
number
108.
Each
of
these
points
is
a
coefficient
representing
the
relationship,
the
correlation
between
users,
self-reported
skepticism
and
the
proportion
of
videos
that
they
were
shown
about
election
front
right.
So
the
top
panel
in
this
plot
is
indices,
where
we
just
added
up
all
of
the
questions
about
fraud.
All
other
questions
about
legitimacy
and
creative
indices
out
of
them.
D
We
find
that
the
combined
index
is
statistically
significant,
positive
and
it's
really
being
driven
by
the
fraud
questions,
and
this
is
reinforced
as
we
break
out
the
individual
questions
below.
So
there
is
this
dimension
of
skepticism
about
whether
Trump
should
concede
or
contest
is
unrelated
to
what
was
being
shown.
But
those
who
believe
that
the
outcome
was
illegitimate
were
being
shown
more
fraud
related
content,
while
spending
time
on
YouTube.
They
were
also
more
likely
to
think
that
foreign
governments
interfered
that
non-us
citizens
voted
and
that
fraudulent
ballots
will
be
accounted
right
so
taken
together.
D
This
is
strong
evidence
consistent
with
our
theory
that
the
recommendation
algorithm
is
showing
people
content
that
reinforces
their
worldview
and,
in
the
context
of
a
world
view
that
is
I,
think
unarguably
damaging
for
Democratic
legitimacy.
This
has
pernicious
consequences
so
substantively
we
can
translate
those
coefficients
into
estimates
of
how
many
more
videos
this
actually
is.
So
we
can
compare
the
bottom
five
percent-
the
least
skeptical
users
to
the
most
skeptical
users
and
run
that
through
our
regression
model
and
get
predictions
for
how
many
more
fraud
related
content
fraud
related
recommendations
that
were
shown.
D
It's
roughly
14
videos
and
to
put
this
in
context
over
the
course
of
this
task
that
Megan
described
they're
shown
a
little
bit
less
than
400
total
recommendations.
So,
on
the
one
hand,
14
more
videos
out
of
400
is
perhaps
not
that
concerning,
but
on
the
other
hand,
when
we
compare
the
most
skeptical
to
the
least
skeptical
right.
This
is
like
a
700
increase
in
the
fraud
related
content.
D
D
I
just
showed
you
to
one
where
we
replaced
the
topic
with
a
different
topic,
and
specifically,
we
chose
a
topic
about
a
very
popular
online
video
game
called
Among
Us,
and
for
those
of
you
who
aren't
aware,
this
video
game
just
happened
to
spike
in
popularity
right
around
the
time
of
the
US
presidential
election
right.
So
this
means
that
it's
a
very
useful
Placebo
test
for
us
if
our
Theory
holds.
D
D
I
just
showed
on
the
right
is
topic
143,
which
is
this
Among
Us
topic,
and
here
we
find
consistent
null
results
across
the
board
right,
there's
no
relationship
between
being
skeptical
about
the
U.S
presidential
election
and
how
often
you
are
shown
videos
about
among
us,
but
there's
a
very
strong
relationship
between
the
same
sense
of
skepticism
and
videos
about
fraud
itself,
so
did
YouTube
show
skeptical
users
more
content.
More
videos
about
election
fraud-
yes,
they
did
again
it's
important
to
caveat
the
findings
here.
First,
as
Megan
mentioned,
this
is
a
convenience
sample
right.
D
D
More
educated
than
than
the
US
demographics
writ
large
one
might
I
mean
we
need
to
look
into
this
more,
maybe
do
an
additional
follow-up
survey,
but
this
might
mean
that
our
results
are
a
lower
back
that
if
we
actually
did
get
older,
more
conservative,
less
educated
Americans
taking
our
survey,
they
might
be
being
shown
even
more
election
fraud.
Content.
D
A
A
I'm
sorry,
a
spy,
try
leaving
the
the
group
and
coming
back
in
if
you
can
I'll,
be
sure
to
admit
you
if
you're,
if
you're
still
having
trouble
hearing
us,
it
might
be
an
audio
issue
with
your
laptop
and
then
receipt
is
asking.
Are
you
working
with
video
or
image
information
of
videos
on
your
research
other
than
text
and
other
text
paste
other
text-based
data
metadata
information,
I
am
working
on
opinion,
mining,
emotion,
assessment
and
emotion,
profiling
of
YouTube,
Tick,
Tock
and
Instagram
videos.
A
My
research
is
mostly
working
with
the
content
of
social
media
posts,
including
video,
audio
and
Text
data,
and
their
opinion.
Scores
also.
Will
the
recording
be
available?
I
can
answer
that
last
question.
Yes,
it
will
be
available
on
the
group's
YouTube
page,
but
yeah
feel
free
to
go
ahead
and
respond
yeah.
Thank
you.
B
Yeah,
thank
you.
So
we
are
only
using
the
text
data
so
far,
I
think
we
had
we
potentially
toyed
with
like
video
using
other
information,
but
I
think
the
conclusion
for
us
was
that
it
might
be
a
little
too
expensive
like
model
wise
or
computationally,
and
not
really
worth
the
payoff,
but
we
think
I
mean
for
future
things.
It
could
definitely
be
really
interesting
to
try
to
work
with
that
data
with
us.
Yeah.
A
I
have
a
couple
questions
if
that's
all
right
and
I
apologize.
I
am
not
a
data
scientist,
so
these
are
not
very
technical,
but-
and
this
might
be
impossible
to
quantify.
But
do
you
think
that
some
of
the
reason
why
YouTube
seems
to
kind
of
push
people
to
write,
leaning
sources
or
radical
right
sources
is
there?
Is
there?
Could
there
be
more
content
out
there
that
is
leaning
in
that
direction
than
the
other
way.
C
So
yeah,
so
that's
certainly
what
some
other
researchers
have
found
just
looking
at
descriptives
on
YouTube
around
political
channels,
there's
a
paper
by
Kevin
Munger.
That
shows
that
just
the
amount
of
right-wing
content
on
YouTube
generally
just
outweighs
that
of
left-wing
content,
so
it
there
is
certainly
a
supply
element
here
that
there's
just
a
lot
of
it
on
YouTube.
C
A
C
C
I
think
that's
more
so
a
question
for
what
like
what
do?
We
think
a
good
recommendation
algorithm
looks
like.
Is
it
something
that
prioritizes
giving
people
diverse
news
feeds?
Is
it
something
that
prioritizes
giving
people
news
feeds
that
matches
their
own
ideology?
I?
Don't
know
that
we
have
a
great
answer
for
that.
I
do
think
we're
all
sort
of
generally
in
the
camp
that
ideally,
it
would
reduce
harm
potential
harm.
C
D
I
might
just
quickly
jump
in
one
thing
that
I
didn't
share,
but
for
the
the
election
note
in
the
paper,
we've
got
sort
of
a
concluding
plot,
which
Megan
put
together,
which
shows
YouTube
was
the
latest
of
all
the
major
social
media
platforms
to
crack
down
on
Election
misinformation
or
disinformation
content
right
like
Twitter,
famously
just
canceled,
Trump
right
away
and
went
on
a
real
tear,
canceling
a
bunch
of
other
accounts.
Facebook
did
shortly
thereafter.
Youtube
waited
I.
Think
until
correct
me.
If
I'm
wrong,
it
was
December
8th.
D
Right
so
YouTube
waited
until
the
the
you
know
established
institutions
of
of
democracy,
the
Safe
Harbor
deadline
being
where
the
votes
were
finally
certified
until
that
day,
and
what
we
show
is
that,
after
they
then
cracked
down
on
this
content,
there
was
an
enormous
drop
off
right.
So
this
is
oh,
thank
you
perfect
right,
and
so
what
we
see
here
this
is
the
percent
of
tweets
linking
election
fraud.
D
There's
basically,
if
we're
worried
about
conspiracy
theories
or
misinformation
or
other
things
that
can
cause
genuine
harm
to
the
fabric
of
society,
there's
two
things:
big
picture
that
you
could
too
can
do
that.
You've
asked
in
your
question
right:
they
either
change
the
recommendation
algorithm
or
they
censor
content
or
somewhere
in
between
somewhere
and
what
I
think
if
we
find
at
least
based
on
this
plot,
and
also
it's
important
to
remember
that,
like
a
recommendation
algorithm
that
shows
people
what
they
want
is
not
normatively
bad.
D
Always
it's
bad
when
what
people
want
is
misinformation
or
disinformation
right,
but
in
many
other
contexts
it
is
objectively
good
to
show
people
content
that
they're,
not
aware
of,
but
that
might
be
of
interest
to
them,
and
so
therefore
I
would
argue
the
big.
The
big
challenge
here
for
YouTube,
as
with
all
of
these
other
social
media
platforms,
is
more
on
the
moderation
and
and
censoring
very
extreme
content
right
so
that
there
just
isn't
the
supply
of
it
there
to
be
recommended,
as
opposed
to
going
in
and
being
like.