►
From YouTube: SOCCYBSEC WG: Using technology to understand and counter violent extremism and hate online
Description
Aug. 5th: Social Cybersecurity Working Group
Presenter: Vidhya Ramalingam
Title: Using technology to understand and counter violent extremism and hate online
Abstract/Description: NA
Bio: Vidhya Ramalingam is Founder of Moonshot CVE, an organization that uses technology to disrupt and counter violent extremism globally. She directs overall strategy and oversees campaigns, software development, and digital projects in over 25 countries. Under her leadership, Moonshot has pioneered new partnerships with tech companies to respond to violent extremism on their platforms, online intervention programs to pull individuals out of violent movements, and the use of automated messaging to disrupt hate groups online. Vidhya is recognized internationally for her role leading policy responses to white supremacist extremism and terrorism. Following the 2011 attacks in Norway, she led the European Union’s first inter-governmental initiative on white nationalist terrorism and extremism, initiated by the Governments of Sweden, Norway, Denmark, Finland, and the Netherlands, and launched by the EU Commissioner for Home Affairs. She regularly advises Big Tech and Heads of State on tackling white supremacy online, including recently the Prime Ministers of New Zealand and Norway. She has testified before the U.S. Congress on the global threat posed by white nationalist terrorism. Vidhya also serves on the Board of Life After Hate, an organization set up by former white supremacists to help individuals leave hate groups. In 2020, she was named an Obama Leader by the Obama Foundation.
A
So
again,
it
is
my
pleasure
to
have
vidya
talk
to
the
working
group.
Vidya
ramalingam
is
founder
of
moonshot
cve,
an
organization
that
uses
technology
to
disrupt
and
counter
violent
extremism
globally.
She
directs
overall
strategy
and
overseas
campaigns.
Software
development
and
digital
projects
in
over
25
countries,
under
the
leadership
moonshot,
has
pioneered
new
partnerships
with
tech
companies
to
respond
to
violent
extremism
on
their
platforms,
online
intervention
programs
to
pull
individuals
out
of
violent
movements
and
the
use
of
automated
messaging
to
disrupt
hate
groups.
A
Online
media
is
recognized
internationally
for
a
role
leading
policy
responses
to
white,
supremacist
extremism
and
terrorism.
Following
the
2011
attacks
in
norway,
she
led
the
european
union's
first
intergovernmental
initiative
on
white
nationalist
terrorism
and
extremism
initiated
by
the
governments
of
sweden,
norway,
denmark,
finland
and
the
netherlands,
and
launched
by
the
eu
commissioner
for
home
affairs.
She
regularly
advises
big
tech
and
heads
of
state
on
tackling
white
supremacy
online,
including
recently
the
prime
ministers
of
new
zealand
and
norway.
A
She
has
testified
before
the
us
congress
on
the
global
threat
posed
by
white
nationalist
terrorism,
where
they
also
serves
on
the
board
of
life
after
hate.
An
organization
set
up
by
former
white
supremacists
to
help
individuals
leave
hate
groups
in
2020.
She
was
named
an
obama
leader
by
the
obama
foundation.
A
B
Great,
thank
you
so
much.
Thank
you
so
much
for
the
invitation
to
join
you
all
today.
So,
as
mentioned,
my
name
is
vithia
ramalingam,
I'm
co-founder
of
moonshot.
I'm
going
to
start
just
by
giving
you
some
background
on
moonshot
and
specifically
what
we
do
to
understand
and
to
prevent
the
spread
of
violence
online,
so
we're
an
organization
that
works
with
governments
with
tech
companies
and
with
the
international
community
to
respond
to
a
whole
range
of
online
harms,
and
actually,
I
know
I
know,
as
introduced
as
a
spatter
of
moonshot
cbe.
B
We
have
actually
renamed
ourselves
to
just
moonshot
and
the
reason
is
because
we,
our
core
work,
started
with
responding
to
violent
extremism
and
terrorism
online.
That
was
very
much
the
founding
story
of
moonshot
cbe
and
then
in
recent
years
we
started
to
expand
our
portfolio
to
cover
a
whole
range
of
online
harms.
So
in
particular,
we
have
started
to
apply
our
technology
on
issues
like
gender-based
violence,
on
human
trafficking,
on
disinformation,
including
health-related
disinformation
and
a
whole
range
of
online
harms.
B
But
as
as
mentioned
in
the
introduction,
my
particular
background
really
started
in
the
fight
against
white
supremacist
and
far-right
terrorism.
That
was
my
my
personal
area
of
passion
in
setting
this
work
up.
A
lot
of
the
examples
that
I'm
going
to
focus
on
today
are
really
from
our
work
on
hate
and
extremism,
but
I'm
also
happy
to
answer
any
questions
and
also
have
any
follow-on
conversations
on
our
work
on
a
whole
range
of
other
issues.
I
I
want
to
start
first
with
some
context
today.
B
Individuals
are
posting
accessing
and
engaging
with
violent
extremist
content
online
at
a
really
alarming
rate
and
the
tech
companies
are
certainly
improving
their
efforts
to
moderate
and
remove
terrorist
content.
But
it's
it's
not
enough
to
stem
violent
extremism
and
terrorism
online
on
its
own,
and
this
is
really
for
three
reasons.
B
B
Yes,
the
post,
the
video
or
the
account
is
deleted,
but
not
the
individual
who
posted
it.
That
person
still
exists
and
they
may
either
continue
to
repost
online
or
move
to
another
platform,
or
they
may
just
be
deeply
vulnerable
and
pose
a
threat
to
the
community
around
them
and
then
the
third
reason
why
content
moderation
is
not
enough
on
its
own
is
that
there
are
still
many
spaces
on
the
internet
which
are
just
not
even
liable
for
takedowns
and
where
this
content
can
thrive,
and
I
wanna
I
wanna
give
an
example
here.
B
This
was
a
search
that
was
entered
into
google
by
an
individual
after
the
christchurch
attack
in
march
2019.
The
massacre
which
left
51
people
dead,
brenton
tarrant,
is
the
name
of
the
perpetrator
of
that
attack.
This
same
search
was
replicated
across
the
globe
in
new
zealand
in
norway
across
dozens
of
countries.
Numerous
languages
across
search
engines-
moments
like
this
are
not
liable
for
takedown
a
google
search
can't
be
removed
or
blocked
at
least
not
the
way
google
search
is
currently
constructed.
You
can
enter
anything
you'd
like
into
google
and,
more
often
than
not.
B
If
you
search
for
that
content,
google
is
most
likely
going
to
present
you
with
top
search
results
related
to
that
to
that
particular
search,
so
content
moderation.
Removal
is
really
important
and
the
tech
companies
absolutely
need
to
be
doing
far
better
on
that
than
they
are.
But
it's
not
the
solution
to
violence
and
extremism
online
alone.
B
Moonshot
is
now
a
team
of
nearly
50
people
based
out
of
both
london
and
washington
dc
and
we're
working
around
the
world,
and
it's
a
it's
a
weird
and
wonderful
mix
of
skills
that
we've
brought
together
in
our
team.
We
have
software
engineers
and
data
scientists.
We
have
a
former
detective
on
staff,
former
mental
health
nurse
practitioner.
We
have
communications
and
marketing
experts,
human
rights
activists,
it's
a
it's
a
diverse
group
that
we've
brought
together
and
diverse
skills
that
are
needed
to
do
this
work.
Now.
B
B
Now
I
want
to
show
you
some
examples
of
how
we
use
our
tools
and
technology
to
build
an
evidence
base
around
the
scale
of
violent
extremism
and
hate
online.
This
is
really
for
us,
the
first
step
before
we
can
then
deliver
programming
to
respond.
We
want
to
be
evidence-based
and
not
driven
by
assumptions
in
our
in
our
responses
online.
Now
earlier
I
mentioned
that
google
search
is
essentially
an
unmoderated
space.
B
B
B
It's
a
it's
a
it's
a
space
that
we
often
turn
to
ask
questions
and
it's
a
space
where,
especially
if
there's
social
stigma
or
if
they're,
if
someone's,
aiming
to
do
something
which
might
be
illegal,
it's
actually
a
particularly
meaningful
place
for
people
to
turn
to
to
to
use
google
search
to
access
information
on
that.
So
it's
also
a
particularly
useful
source
when
we're
trying
to
monitor
extremism
and
hate
and
understand
attempts
to
access
this
content
across
the
country.
B
What
you're
seeing
on
the
screen
now
is
a
county-by-county
breakdown
of
attempts
to
access
far-right
extremist
white,
supremacist,
hate
content
across
the
united
states
in
2019,
and
you
can
see
some
example
search
terms
included
in
the
top
right
corner
there,
but
in
2019
we
tracked
nearly
1
million
attempts
across
the
entire
united
states
to
access
content
that
was
affiliated
with
far-right
extremism
or
white
supremacy.
That
was
2019
alone
and
at
the
start
of
the
pandemic.
B
B
Here,
on
this
slide,
you
can
see
how
this
trend,
this
kind
of
increase
during
a
period
of
crisis,
was
experienced
across
the
globe.
So
you
see
a
nationwide
increase.
There
reported
in
attempts
to
access
this
content
in
new
zealand
and
a
68
nationwide
increase
across
australia
for
the
united
states,
in
particular
as
of
april
2021.
So
this
year
we
are
now
at
140.
B
Higher
engagement
with
hate
content,
far-right
content
on
google
than
we
were
pre-pandemic
level.
So
that's
just
for
the
us
alone.
So
that's
an
update
on
the
data
that
you're
seeing
here
on
this
slide.
So,
as
we
are
all
online
more
as
as
this
event
is
taking
place
online,
as
we
are
all
at
our
computers,
more
so
are
hate
actors,
and
in
countries
where
we've
been
building
longitudinal
data
sets,
it's
also
been
possible
for
us
to
then
assess
spikes
and
escalations
and
behaviors
online
as
a
result
of
any
event
or
incident
taking
place
offline.
B
So
one
example:
I
wanted
to
show
following
the
august
3rd
2019
attack
on
a
walmart
in
el
paso
texas,
that
was
an
attack
by
a
white
nationalist
who
had
been
inspired
by
christ
church.
We
were
able
to
map
a
54
increase
in
attempts
to
access
white
supremacist
content
online
nationwide
across
the
united
states
and
in
particular,
as
you
see
here,
we
saw
a
495
increase
in
attempts
to
access
content
that
was
violent
towards
jewish
communities
and,
specifically,
an
82
and
an
83
percent
increase
in
attempts
to
access
content
violent
towards
black
and
mexican
communities.
B
Most
worryingly,
we
oftentimes
don't
see
a
return
back
to
normal
levels
of
engagement.
I
don't
like
using
that
term
normal
levels
of
engagement,
because
this
behavior
shouldn't
be
normal,
but
you
know
we
have
been
looking.
We
have
been
tracking
this
longitudinally
and
what
has
been
really
worrying
is
when
you
see
those
spikes,
we
oftentimes
just
see
a
continual
escalation
in
content.
We
haven't
often
times
returned
back
to
that
original
baseline
pre-event
in
some
cases,
and
we've
been
really
monitoring
this
since
2015.
B
B
B
Well,
there
are
several
tiers
of
programming
that
we
use
to
reduce
the
risk
posed
by
extremists
and
hate
actors
online.
These
are
four
options
and,
in
the
interest
of
time,
I'm
going
to
focus
on
the
final
three
options
here,
because
I
specifically
want
to
want
to
talk
about
how
we
reach
out
to
audiences
who
are
at
risk
and
try
and
pull
them
away.
B
B
We
run
global
prevention
and
safeguarding
campaigns
to
reach
users
that
are
at
risk
of
violent
extremism
and
specifically
to
offer
them
safer
alternatives.
Now
one
example
we
partnered
with
google
several
years
ago
now.
This
was
back
in
2015
to
repurpose
advertising
technology,
to
reach
violent
extremists
with
content
which
discredits
those
ideologies
and
which
would
offer
alternatives.
So
how
does
this
work?
Well,
it's
just,
as
you
see
it
on
the
screen.
What
we're
doing
is
we're
repurposing
tools
that
were
developed
for
commercial
purposes
and
we're
trying
to
use
them
for
social
goods.
B
So
if
someone
searches
for
neo-nazi
or
white
supremacist
content,
the
very
first
piece
of
content
that
they
would
see
would
be
safer,
a
safer
alternative.
The
aim
here
is
to
ensure
that
every
person
who
is
searching
for
hate
content,
regardless
of
their
risk
level,
maybe
they're
just
someone
who's
doing
research
on
this.
Maybe
there's
someone
who's
genuinely
looking
into
it
to
join.
B
We
now
deliver
this
method
globally,
with
the
the
main
objective
being
to
ensure
that
there's
always
that
safer
alternative
available
and
increasingly
when
we're
delivering
these
methods
at
scale.
We
not
only
want
to
connect
users
in
with
safer
content,
but
we
also
want
to
connect
them
in
with
local
services,
so
services
that
they
can
access
and
engage
with,
which
might
help
them
turn
their
lives
around,
and
there
are
really
learnings
that
come
from
decades
of
offline
prevention
and
intervention
models
which
we're
just
simply
applying
online.
B
So
we're
building
off
a
very
large
evidence
base
here
and
what
we're
doing
is
transit
transitioning
matter
translating
that
into
the
online
space.
There's
a
huge
raft
of
research
from
scandinavia
and
germany
in
particular,
but
from
europe
really
over
the
past
decade,
that
has
evidenced
the
efficacy
of
one-to-one,
counseling
and
social
work
interventions
with
right-wing
extremists.
In
particular,
there's
also
a
growing
evidence
base
around
the
efficacy
of
one-to-one
counseling
with
potential
salafi
jihadists
with
individuals
that
have
been
getting
involved
in
jihadist
extremism.
B
What
we're
doing
is
working
to
apply
these
same
social
work
methods
in
the
online
space,
and
this
can
be
done
through
offering
these
individuals
psychosocial
service
content
online.
So
maybe
that's
in
the
form
of
videos
manuals
self-help
guides,
but
it
can
also
be
done
through
sign
posting
them
directly
into
a
local
service
such
as
an
online
or
an
offline
chat
with
a
counselor
or
even
a
hotline
to
call
and
we've
even
run
programs
in
which
social
workers
are
directly
reaching
out
to
internet
users
and
trying
to
start
conversations
with
them.
B
Now
in
our
global
tests,
we
consistently
find
that
far-right
extremists
are
disproportionately
likely
to
engage
with
offers
of
assistance
assistance
when
they're
offered
online,
so
the
results
you're
seeing
here
were
from
a
test
that
we
ran
around
three
years
ago.
Now,
in
the
united
states,
we
found
that
far-right
extremists
were
48
more
likely
to
engage
with
mental
health
content
than
the
general
public.
So
this
was
specifically
self-help
content
and
social
service
content.
B
If
they
were
looking
to
join
a
violent
group
like
the
kkk,
they
were
115
more
likely
than
the
general
public
to
engage
with
that
content.
We've
now
delivered
these
kinds
of
online
intervention
and
referral
mechanisms
across
the
globe,
including
in
non-western
context,
so
I'm
going
to
share
an
example
from
southeast
asia.
So
earlier
this
year
we
released
new
data
showing
some
tests
of
results.
B
Some
results
of
some
tests
that
we
ran
in
indonesia,
audiences,
engaging
with
violent
groups
online,
were
128
more
likely
to
engage
with
psychosocial
content
as
compared
to
what
we
would
call
traditional
ideological
counter
content.
So
we
tested
content
which
directly
undermines
violent.
Extremist
narratives
tries
to
deconstruct
the
narrative
and
tested
that
against
offering
content,
which
was
psychosocial
service
content
and
yeah
128
more
likely
to
engage
with
psychosocial
content.
B
So
this
trend
holds
in
nearly
every
geography
that
we
try
it
and
I
suppose
the
main
message
that
I
want
to
get
across
here
is
while
it's
tempting
to
meet
hate
and
disinformation
and
extremism
with
facts
and
counter
facts
telling
them
that
they're
wrong
or
or
trying
to
engage
in
debates
with
them.
On
the
ideology
we
tend
to
find
that
it's
far
more
effective
to
engage
with
them
through
essentially
social
work.
B
I
want
to
run
through
some
of
the
messages
that
we
were
testing,
so
these
are
some
of
the
message
themes
that
we
tested
over
the
course
of
what
has
been
a
really
kind
of
volatile
political
environment
for
the
last
year
now
you'll
note,
these
are
largely
messages
which
aim
to
deter
individuals
from
taking
violent
action.
They're.
Really
it's
really
about
pushing
positive
democratic
values.
B
Psychosocial
support
messaging-
you
can
see
here
this
messaging
around
neighbors,
it's
really
about
asking
people
to
pause
and
remind
themselves
of
their
loved
ones,
the
impact
that
their
actions
can
have
on
their
neighbors
and
their
loved
ones.
And
the
final
message
here
is
a
really
important
one
here
to
help
this
is
really
about
services.
B
We
set
up
a
range
of
services
to
offer
to
individuals
that
were
at
risk
of
violent
extremism
in
the
us,
so
in
particular
a
dedicated
24
7
helpline
service
for
people
who
who
needed
support
basically
offering
crisis
support.
What
we
did
is
we
reached
out
to
four
audiences
testing.
These
messages
we
reached
out
online
to
audiences,
engaging
with
white
supremacist
content
audiences,
engaging
with
armed
group
content
audiences,
engaging
with
anti-government
conspiracy,
theory,
content
and
then
also,
finally,
that
fourth
audience
was
human
on
audiences,
and
so
what
did
we
find?
B
The
the
most
engaging
advertisement
that
we
ran
began
with
the
message.
Anger
and
grief
can
be
isolating
so
on
average,
our
audiences
were
17
more
likely
to
click
on
that
ad
than
any
other
message
that
we
tested
in
particular
armed
group
audiences
engaged
most
with
the
calm
messages,
so
messages
as
you
see
here
that
we're
really
offering
resources
to
try
and
calm
someone
down
who
might
be
hyped
up,
angry
and
potentially
violent.
B
These
were
messages
that
said
things
like
don't
do
something
you'll
regret.
Self-Control
is
key.
We
even
offered
mindfulness
exercises
to
de-escalate
people
from
anger
and
and
q
anon
audiences,
in
particular,
engaged
most
with
what
we
call
empathy.
Empathy
anger
ads,
so
these
were
ads.
That
said,
are
you
feeling
angry
you're,
not
alone,
learn
how
to
escape
the
anger
and
move
forward
by
far
in
a
by
far
and
away?
This
is
just
a
kind
of
interesting
data
point
from
this
work
we
did.
B
The
highest
engagement
we
had
on
youtube
was
with
a
breathing
exercise
that
we
ran.
We
ran
this
as
what's
called
a
pre-roll
skippable
ad,
so
these
are
the
ads
which
many
of
you
might
be
familiar
if
you
ever
watch
youtube
videos.
These
are
the
ads
which
come
up
just
before
you,
your
the
video
that
you're
intending
to
watch,
and
you
can
choose
to
watch
it
30
or
60
seconds.
B
We
even
had
comments
from
some
members
of
our
at-risk
audience
saying
that
they
felt
better
after
watching
it
or
that
they
really
needed
it
that
day.
Now,
one
of
the
reasons
why
we're
so
interested
in
how
self-help
and
psychosocial
support
content
is
resonating
is
especially
yeah
through
the
election
through
the
inauguration
as
q,
anon,
fragmented
and
as
disillusionment
started
to
grow
online.
We
found
that
audiences,
you
know,
as
they
were,
realizing
that
the
prophecies
they
were
sold
were
false.
B
We
started
to
track
just
the
the
fraying
mental
health
of
so
many
audiences
online
people
who
had
lost
their
jobs
connections
with
their
loved
ones.
You
know
we
saw
audience
audiences
on
on
4chan
apan
saying
you
know
that
my
son
won't
speak
to
me
anymore.
I
might
my
family
has
totally
alienated
me.
They
lost
all
of
their
connections
by
buying
into
this,
and
we
started
to
see
an
increased
activity
around
expressions
of
extreme
anxiety
and
suicidal
thought.
B
So
this
was
really
worrying
and
one
of
the
reasons
why
we
continued
to
have
a
focus
on
psychosocial
support,
as
we
were
reaching
out
with
to
these
audiences
and
will
continue
to
do
so
throughout
this
year.
So
I
just
wanted
to
leave
you
with
some
examples
and
actually
I'll
stop
sharing
my
screen
there,
so
that
I
can.
I
can
also
see
you
all.
I
wanted
to
leave
you
with
some
examples
there
of
how
it
is.
B
We've
been
engaging
with
audiences
in
the
us
during
this
really
volatile
moment,
but
also
how
technology
is
allowing
us
to
scale
this.
A
lot
of
the
examples
I
gave
you
are
around
the
use
of
advertising
advertising
and
ad
tech
is
a
really
important
technology
that
we
use
to
scale
up
messaging
and
to
scale
up
personalized
messaging
to
reach
people
who
might
be
at
risk
of
violence.
But
there
are
also
other
mechanisms
that
we
use
to
try
and
reach
out
to
people.
B
The
most
important
thing
for
us
is
how
we
can
use
technology
to
send
really
personalized
messages
out
to
people
who
might
be
in
need
of
direct
support
and
connect
them
and
link
them
in
with
ideally
support
that
they
can
engage
with
in
the
real
world.
So
with
that,
I'm
going
to
I'm
going
to
end
here
and
I'm
really
happy
to
take
questions.
A
Thank
you
so
much
fiddler.
That
was
a
fascinating
talk
with
lots
of
interesting
statistics
and
you're
so
happy
that
you're
doing
this
work.
That
is
not
only
making
a
difference
technologically,
but
also
in
the
in
the
society
in
the
real
world.
I'm
sure
there
are
a
lot
of
questions
from
the
audience.
A
C
Sure
yeah,
I
was
just
curious
how
you
were
you
know
as
you're
targeting
these
groups
or
these
actors
I
mean:
how
do
you
define
that
I
mean?
Is
this
something
that's
a
database
or
is
there
something
that
you're
measuring
to
define?
This
is
gone
over
the
edge
far
enough
to
be
extreme
versus
somebody.
That's
just
angry
or
whatever.
B
Yeah,
it's
a
great
question,
so
the
the
parameters,
the
way
that
we
actually
do,
the
analytical
work
that
we
that
we
deliver
and
also
the
campaign's
work,
it's
all
through
those
databases
of
indicators.
So
when
I
flashed
up
on
the
screen
earlier,
you
saw
all
the
kinds
of
keywords
that
are
in
our
database
now,
each
of
those
keywords
we
code,
we
categorize
in
code
in
a
number
of
different
ways.
B
One
is
by
ideology,
and
when
and
and
I'm
kind
of
using
far-right
extremism
as
a
catch-all
term,
what
we
really
do
is
break
that
down
to
different
types
of
groups
and
ideologies
that
might
fall
within
a
far-right
extremist
category.
So
I
mentioned
earlier
that
there
were
four
audiences
that
we
focused
on
in
our
cross-us
work.
Last
year,
white
supremacists,
armed
group,
conspiracy,
theorists
anti-government
conspiracy,
theorists
and
q
anon.
We
did
also
do
a
lot
of
work
on
disinformation
and,
in
particular
on
anti-black
and
anti-semitic
disinformation.
B
B
That's
you
know
that
we
we
see
on
the
news,
and
so
therefore
we
can't
be
certain
that
someone
who's
engaging
with
that
term
on
on
a
social
media
platform
or
on
google
search
is
genuinely
at
risk
of
violent
extremism.
So
we
had
to
adjust
our
risk
coding
on
basically
a
daily
basis,
as
we
were
running
this
work
over
the
last
year
in
the
us.
A
C
Okay,
yeah,
so,
as
as
I
was
as
I
was
listening
to
to
your
presentation,
one
of
the
things
that
immediately
popped
out
to
me
was
how
much
of
this
are
we
trying
to
because
it
sounds
more
like
censorship
and
I
apologize.
I
come
more
from
a
political
background
than
policy,
so
I
have
my
mpa.
So
that's
why
I'm
asking
this
question
but
freedom
of
speech?
How
much
does
that
really
play
a
role
into
you,
know
people
being
able
to
post
and
say
what
they
want
online.
C
You
know
where
we're
coming
in
is
we're.
We
have
a
situation
where
literally
the
president
of
the
united
states
was
taking
off,
was
taken
off
of
social
media
and
it
it
becomes
a
a
issue
of
okay.
Well,
how
far
can
we
have
these
big
tech
companies
going
in
their
efforts
to
you
know,
take
violent
content
off
of
off
of
the
internet
where
we're
talking
about
okay.
C
Well,
this
is
the
most
powerful
man
in
the
country
and
one
of
the
ways
that
he
communicates
with
people
is
through
these
big
tech
companies
and
their
and
their
platforms.
So
how
much
can?
How
far
can
we
really
go
and
how
much
are
we
really
will?
Are
we
willing
to
give
the
tech
companies
this
power
to
limit
the
communication
of
the
president.
B
Yeah,
it's
a
it's
a
great
question
and
I
really
see
us
as
being
free
speech
activists
as
well
as
an
organization.
We
we
don't.
We
don't
really
work
on
content,
moderation,
that's
not
how
we
respond
to
these
issues.
What
we
are
doing
is
basically
trying
to
ensure
that
for
anyone
who
is
posting,
this
content
online
that
there
are
safer
alternatives
available.
So
from
our
perspective,
we
just
want
to
basically
flood
those
spaces
with
safer
alternatives.
B
Now
I
I
do
believe
that
the
tech
companies
do
need
to
do
more
when
it
comes
to
content
moderation,
specifically
on
terrorist
content.
You
know,
I
think
I
think
the
christchurch
massacre
was
a
huge
learning
moment
for
so
many
of
the
tech
companies,
just
in
terms
of
the
scale
of,
for
example,
the
video
of
that
of
that
of
that
particular
event,
the
scale
at
which
that
was
spread
and,
and
so
the
the
pace
at
which
it
was
spread.
B
I
think
it
was
a
huge
learning
curve
for
the
tech
companies,
so
I
do
think
that
moderation
is
important,
but
I
agree
with
you:
we
do
need
to
find
ways
to
be
dealing
with.
You
know
extremist
extremist
thought
you
know
when,
when
it's
not
illegal,
how
do
we
ensure
that
they're
that
we
are
still
reaching
out
to
those
audiences
and
creating
a
meaningful
opportunity
to
actually
kind
of
bring
people
along
in
a
in
a
peaceful
and
constructive
way?.
A
Thank
you
follow
up
to
that
with
a
lot
of
researchers
like
us
work
with
social
media
data,
and
we
often
find
things
that
you
would
like
to
report
to
the
social
media,
companies
or
a
regulatory
body
who
can
take
an
action
in
in
your
experience.
What
advice
would
you
give
researchers
like
us
and
the
and
the
larger
community
so
that
the
best
way
to
engage
with
social
media
companies
to
share
the
research
findings
and
perhaps
also
assist
them
with
policymaking?.
B
It's
a
great
question
and
you
know
I
wish
I
could
say
I
wish
I
could
say
that
I
had
all
the
solutions
around
how
to
engage
with
the
tech
companies.
I
I
definitely
don't
we're
kind
of
where
we're
kind
of
learning
as
we
go
as
well.
I
think
when
it
comes
to
reporting
content
and
if
you
see
things
where
you
really
just
want
to
engage
with
the
tech
companies
on
it.
Unfortunately,
the
tech
companies
are
huge,
machineries
right.
They
they're.
Essentially,
they
operate
like
governments,
they're,
huge
huge
organizations
and
so
oftentimes.
B
My
greatest
advice
would
be
to
really
work
to
build
up
relationships
with
individuals
that
work
inside
the
company
find
your
trusted
contact
because
that's
going
to
be
the
most
effective
way
to
actually
push
something
forward
at
a
strategic
level
or
at
a
when
you're.
Really
your
your
work
is
really,
you
know,
underpinned
by
expertise
and
experience
and
you're,
not
just
a
member
of
the
public
who's.
Flagging
one
thing,
you're
actually
flagging
something:
that's
of
strategic
importance.
B
I
would
really
focus
on
finding
your
one
person
within
the
the
you
know
the
public
policy
team
at
the
right
at
the
at
the
relevant
tech
company
who
you
can
build
up
that
trusted
relationship
with.
But
I
you
know
it's
we.
We
are
still
working
ourselves
on
the
best
way
to
steer
these
big
ships
in
the
right
direction,
and
I
I
don't
have
all
the
solutions.
I
will
report
back
as
we
learn
along
the
way.
A
B
Yeah
excellent
question,
and
so
we
we
have
been
doing
some
research
on
this.
It's
we
are
hoping
to
publish
some
results
on
this
later
this
year,
because
we
get
this
question
a
lot.
Our
initial
assessment
is
that
the
scale
and
that
the
volume
of
engagement
on
the
far
left-
and
this
is
you
know
really
looking
at
anarchist
movements,
in
particular
violent
anarchist
movements.
B
We
tend
to
find
that
the
volume
is
far
lower
than
on
far-right
extremism.
That
said,
we
do.
We
are
looking
to
actually
build
a
robust,
evidence-based
around
that
and
put
that
in
the
public
domain,
because,
right
now
we
don't
have
a
publicly
available
reference
point
to
push
people
towards,
so
we
are
working
on
that.
I
I
think,
if
anyone
looks
at
our
website,
you'll
see
that
it's
very
heavily
focused
on
far-right
extremism,
and
we
do
actually
need
to
provide
a
little
more
balance
on
that.
So
we'll
look
to
do
that
later
this
year,.
A
B
Excellent
question:
predictive
modeling:
this
is
the
kind
of
golden
question
that
I
often
get
asked
because
I
obviously
gave
you
some
evidence
that
we
can
see
offline
events
have
an
impact
in
the
online
space,
but
can
we
can
we
say
that
online
online
activity
will
have
an
impact
in
the
offline
space?
We
are
working
towards
developing
an
answer
to
this
question
years
ago.
When
I
was
asked
this
question,
I
would
tell
people
look
it's
going.
It
could
take
us
a
decade
to
build
up.
B
You
know
the
predictive
modeling
capability
to
be
able
to
do
that.
I
actually
think
that
in
the
last
year,
given
the
sheer
volume
of
content
online
and
also
the
sheer
volume
of
offline
events
which
have
taken
place
where
online
activity
has
been
integrally
connected
with
it,
I
actually
think
we've
in
in
terms
of
our
technology
and
our
our
data
science
teams.
I
think
we
have.
We
are
actually
getting
closer
to
that
than
I
I
would
have
even
anticipated.
B
Those
platforms
are
a
much
greater
predictor
of
offline
violence
than
twitter
than
the
same
content
when
it's
shared
on
twitter
or
on
facebook
or
the
mainstream
platforms.
But
that's
just
an
initial
highlight
from
some
very
early
work,
we're
doing
to
map
online
activity
against
real
world
events.
Again,
that's
something
to
stay
tuned.
We
should
be
publishing
on
something
on
this
later
this
year
or
early
next
year.
A
C
Sure,
thank
you.
Thank
you
very
much
and
it
was
a
great
talk.
Actually,
I've
been
following
your
work
like
for
a
few
years
now,
and
I
remember
that
you
were
working
with
google
actually
before
so
I
I
really
like
your
work,
so
my
question
is
so
in
your
key
responses
to
the
to
the
user,
behavior
leaning
towards
like
these
extremist
content.
C
How
do
you
measure
the
effectiveness
of
your
key
responses?
So
do
you
like
conduct
some
surveys
with
the
users
of
randomly
selected
users,
for
example,
and
another
question
that
I
had
was
so
the
desired
harm
that
you
showed
in
one
of
the
slides?
I
believe
it
was
related
to
the
el
paso
shooting
event
and
it
was
actually
an
offline
event.
I
believe-
and
there
were
some
it
influenced.
B
Yeah
good
question
so,
on
your
first
question
around
how
to
measure
efficacy,
so
the
beauty
of
doing
this
work
online
is
that
there
is
a
lot
of
data
you
can
work
with
if
you're
running
advertising
campaigns,
just
like
commercial
entities,
coca-cola
adidas,
just
like
they
track
and
measure
the
you
know
the
impact
of
and
the
actions
that
result
from
individuals
clicking
on
their
ads.
We
are
also
able
to
collect
a
limited
amount
of
data
around
that
now
that
data
is
limited.
It's
just
that
right.
B
It's
limited
when
I
mentioned
earlier
that
we
saw
over
500
000
times
that
someone
who
was
at
risk
was
watching
at-risk
content
watched
our
our
breathing
exercise.
All
the
way
through
that
for
us
was
a
bit
of
a
breakthrough
simply
because
getting
someone
to
watch
an
advertisement,
that's
30
seconds.
All
the
way
through
is
actually
a
really
tough
thing
to
do
even
in
traditional
marketing
campaigns,
so
that
felt
like
a
breakthrough.
But
then
the
next
question
is
what
impact
did
that
have?
B
B
So
there
are
limitations
to
the
data
that
we
can
use
if
you're
offering
someone
static
content
if
you're
linking
someone
into
a
service,
that's
where
it
gets
a
lot
more
interesting,
because
then
you
can
start
to
understand
well
how
what
what
were
those
audience
journeys
like
from
you
know
an
online
and
it's
an
interest
in
watching
extremist
content
clicking
on
an
offer
of
a
service.
Did
they
actually
then
meaningfully
engage
with
that
service?
B
That's
where
it
gets
really
interesting
is
when
you
can
kind
of
link
in
with
offline
services
and
then
your
second
question
around
your
desire
to
harm
that
particular
slide
was
tracking
certain
keywords
that
fell
within
our
category
of
desire
to
harm,
so
those
were
searches
where
someone's
entered
into
google.
I
want
to
kill
x
group
or
how
to
kill
x
group
how
to
hang
experts
those
sorts
of
keywords.
So
there
were
keywords
where
someone
indicated
an
interest
in
in
harm
or
violence
against
a
specific
community.
B
A
I
think
we
have
a
question
from
farmers,
but
thomas
is
your
question
answered
based
on
the
just
discussion,
the
discussion
we
just
had
because
it's,
I
think,
you're
asking
about
the
outcomes
of
the
life
after
hate
program
and
I
believe
they
just
responded
to
that.
Let
us
know
if
you
have
any
follow-up
question
and
we
also
have
a
message
from
sheikh
mehndeen.
C
A
couple
of
questions
that
how
many
search
engines
are
you
guys
tracking?
Are
they
being
tracked
only
when
they
even
happen
or
they're
being
tracked
like
throughout
the
year?
And
the
second
question
is:
oh
sorry.
The
second
question
is
that
what
are
the
challenges
you
guys
get
when
you
present
the
counter
narratives?
When
you
see
you
know
a
lot
of
extreme
messages
or
extreme
searches
being
going
on?
So
what?
What
are
the
challenges
you
get.
B
If
we,
if
we
set
out
to
track
a
certain
set
of
searches,
then
we'll
be
able
to
gather
that
data,
we
can't
retrospectively
gather
that
data.
So
you
know,
for
example,
with
the
question
around
far-left
extremism.
When
we
set
out
to
track
our
left
extremism,
we'll
then
have
access
to
that
data.
We
wouldn't
be
able
to
go
back
historically
and
look
at
all
of
the
far
left
searches
which
took
place
in
2019
and
benchmark
them
against
far-right
extremism.
B
So
we
have
to
do
kind
of
dedicated
dedicated
data
gathering
and
then,
in
terms
of
challenges,
did
you
mean?
Did
you
mean
I
want
to
make
sure?
I
answer
your
question
correctly.
Do
you
mean
how
how
when
we
are
when
we
are
engaging
with
extremist
content?
Is
that
challenging
for
us
on
a
personal
level?
Or
do
you
mean
what
challenges
do
we
encounter
when
we're
trying
to
engage
someone.
C
B
B
Yeah,
we
basically
just
test
we
test,
so
many
different
kinds
of
messages.
Like
I
talked
to
you
today
about
a
lot
of
the
non-ideological
de-escalation
messages
that
we're
testing.
We
have
also
test
a
lot
of
ideological
messages,
so
messages
that
try
and
deconstruct
the
narrative
try
and
you
know,
offer
alternative
alternative
messages
to
someone
who
might
be
searching
for
dehumanizing
language
and
we
just
continually
test
and
then
kind
of
refine
and
optimize
our
campaigns
based
on.
B
What's
working
the
most
one
thing
we
found
in
terms
of
ideological
messaging
throughout
last
year,
when
we
were
doing
work
around
anti-semitic,
violent,
anti-semitic,
narratives,
violent
anti-black
narratives.
We
found
that
some
of
the
most
effective
content
was
content
which
presented
basically
a
debate
where
it
was
kind
of
two
sides
of
an
argument.
It
was
about
introducing
empathy
for
someone
who's
on
the
other
side
of
the
argument
that
was
one
of
the
most
effective
narratives
that
we
tested
but
yeah.
A
Thank
you
so
much,
and
I
think
that
is
almost
time
for
us
videos.
Thank
you
so
much
again
for
being
here
and
sharing
your
great
work
with
us.
I
would
also
like
to
point
the
attention
of
the
working
group,
along
with
the
the
workshop
that
we
are
doing,
which
aligns
very
well
with
this
topic,
which
is
called
deviance
in
social
media,
and
this
is
co-located
with
ieee
big
data
conference
happening
in
december.
A
I
will
be
posting
the
link
for
the
workshop
if
you're
interested
and
if
you
have
your
students
working
on
this
con
on
this
topic,
please
feel
free
to
send
us
your
studies
papers.
They
will
be
published
with
ieee
publication
banner.
We
also
have
a
special
issue
coming
following
up
that
event,
so
there
are
many
publication
opportunities
for
the
folks
who
are
working
in
this
topic.