►
Description
Captain Iain Cruickshank is a data scientist at the U.S. Army's Artificial Intelligence Task Force and a non-resident, associate researcher with Carnegie Mellon's CASOS Institute. Iain holds a Bachelor of Science from the United States Military Academy and a Master's of Science from the University of Edinburgh which was obtained as a Rotary Ambassadorial Scholar. He is a recent graduate of the Doctor of Philosophy in Societal Computing program at Carnegie Mellon University where he was a National Science Foundation Graduate Research Fellow. Prior to starting his Ph.D. at CMU, Iain worked in various elements of the U.S. Army to include as an advisor to the Afghan National Army while with the 101st Airborne Division and as a company commander and signals intelligence officer with the 780th Military Intelligence Cyber Brigade. Iain's current research areas are computational techniques for clustering social-based multi-modal data, social-cyber security, and machine learning applications for military intelligence.
Date: 09/03/20
Presenter: Iain Cruickshank
Institution: U.S. Army Artificial Intelligence Task Force
A
Captain
crookshank
is
a
data
scientist
at
the
u.s
army's
artificial
intelligence
task
force
and
a
non-resident
associate
researcher
with
carnegie
mellon's
casos
institute,
led
by
dr
kathleen
carley
ian
holds
a
bachelor
of
science
from
the
united
states
military
academy
and
a
masters
of
science
from
the
university
of
edinburgh,
which
was
obtained
as
a
rotary,
ambassadorial
scholar.
He
is
a
recent
graduate
of
the
doctor
of
philosophy
and
societal
computing
program
at
carnegie
mellon
university,
where
he
was
a
national
science
foundation.
A
Graduate
research
fellow
prior
to
starting
his
phd
at
cmu
ian
worked
in
various
elements
of
the
us
army
to
include
as
an
advisor
to
the
afghan
national
army,
while
with
the
101st
airborne
division
and
as
a
company
commander
and
signals
intelligence
officer
with
the
780th
military
intelligence.
Cyber
brigade
ian's
current
research
areas
are
computational
techniques
for
clustering,
social
based,
multimodal
data,
social,
cyber
security
and
machine
learning,
applications
for
military
intelligence,
and
today,
ian
is
going
to
talk
to
us
about
his
work
on
multi-clustering
results
of
social
based
data.
B
Thank
you
very
much
the
introduction,
dr
agarwal.
I
just
want
to
say
again
thanks
for
having
me
today
and
let
me
speak,
and
so
today,
as
mentioned,
I'm
going
to
be
talking
about
some
research
we've
done
at
the
caseless
institute
on
this
idea
of
multiview
clustering
for
social
based
data,
so
to
kind
of
set
the
stage
for
what
I'm
about
to
talk
about
with
these
multi-view
clustering
techniques
and
why
they
might
be
important.
B
Now,
despite
all
this
new
and
these
different
types
of
data
being
generated
to
describe
human
social
behavior
fusing
that
data
into
a
kind
of
cohesive
analysis
is
still
a
challenge,
in
particular,
clustering
of
these
heterogeneous
data
that
all
describe
the
same
actors
can
be
a
real
challenge
because
of
the
heterogeneity
of
the
data
and
as
social
science.
Researchers
clustering
is
one
of
our
important
analysis
methodologies
for
human
social
behavior,
because
we
use
it
in
everything
from
community
detection
to
understanding
kind
of
like
human
behavioral
patterns,
and
so
that's
really.
B
What
I
want
to
focus
on
kind
of
with
this
talk
is:
how
can
we
take
all
of
this
heterogeneous
social,
human
behavior?
That's
captured,
digitally
now
and
use
that
to
better
understand
these
human
social
interactions
taking
place
so
to
set
the
kind
of
stage
for
how
we
do
this
first.
I
want
to
talk
about
what
I
what
I
mean
when
I
say
multi-view
data,
and
so
multi-view
data
often
occurs
a
lot,
especially
in
human
social
behavior.
B
It's
the
idea
that
different
real
world
phenomenon
can
naturally
give
rise
to
multiple
types
or
even
subdivisions
of
data
when
it
does
give
right
to
data
now.
An
important
assumption
with
this
is
that
all
of
the
actors
are
capable
of
producing
data
in
each
view
or
mode
that
we're
looking
at
their
data
in
now.
Some
more
traditional
examples
of
multiview
data
would
include
things
like
audio
and
visual
data
that
arise
from
something
like
film
or
different
sequencing
and
interaction,
types
that
arise
from
looking
at
proteins.
B
B
Like
the
idea
of
co-learning,
however,
that
you
know
it
was
kind
of
been
a
recent
summary
by
baltistratus
and
morenci
they've
highlighted
that
the
use
of
multiview
data
machine
learning
does
come
with
some
kind
of
real
distinct
challenges
and,
in
this
case
we're
looking
at
this
challenge
of
the
idea
of
fusion
the
idea
that
we
want
to
fuse
information
from
all
of
these
views
in
order
to
do
clustering.
So
that
way
we
can
better
understand
kind
of
the
human
social
behavior
taking
place.
B
B
This
is
the
idea
that
other
views
provide
complementary
information
for
a
better
clustering
of
a
view
that
would
come
from
just
the
information
in
that
view
by
itself.
So
for
those
who
are
kind
of
more
familiar
with
ensemble
methods,
especially
in
supervised
learning,
this
is
a
very
similarly
important
underlying
principle
of
ensemble
methods
as
well.
B
So
the
first
paradigm-
and
it's
still
the
most
widely
used-
is
the
early
integration
paradigm,
and
this
is
the
idea
that
we
effectively
create
feature
vectors
from
all
of
the
views
we
concatenate
those
into
one
feature
vector
and
then
treat
it
as
we
would
is
like
a
traditional
clustering
problem
now.
This
typically
is
the
least
successful
paradigm,
as
view
specific
patterns
get
kind
of
washed
out
when
they
are
combined
with
all
the
other
views
in
a
naive
way.
B
Now
it
is
worth
mentioning
there
are
some
techniques
that
would
be
classified
as
multi-view
community
or
multi-view
clustering
techniques
within
community
detection,
social
networks
already.
So
these
would
be
things
like
multiplex
network
clustering,
where
we
have
multiple
networks
defined
over
the
same
set
of
nodes,
and
you
want
to
typically
find
one
clustering
from
all
of
those
networks.
There's
multi-layer
network
clustering,
which
is
sort
of
a
more
generalized
version
of
multiplex,
and
that
the
clusters
can
actually
vary
between
the
views
or
layers
and
then
attributed
network
clustering.
B
Where
now
you
have
a
network
but
then
also
some
kind
of
data
about
each
of
the
nodes,
and
you
want
to
use
that
data
to
to
inform
the
network
clustering.
This
idea
has
also
kind
of
been
expanded
out
into
more
computational
techniques
like
graph
convolution
networks.
The
one
thing
you'll
notice
here,
though,
is
they?
The
the
key
limitation
of
these
techniques
is
they're
only
limited
to
either
all
network
views
or
one
network.
B
In
one
network
view,
they
don't
inherently
work
on
situations
where
you
could
have
multiple
network
and
non-network
views
so
they're,
not
quite
as
ubiquitous
in
their
in
the
data
they
can
tackle
as
part
of
our
work
on
multiview
clustering,
especially
with
social
based
data.
One
of
the
big
things
we
realize
is
well
clustering.
Social
based
data
is
actually
pretty
hard,
especially
from
a
multi-view
paradigm
and
there's
a
lot
of
things
that
give
difficulties
within
the
social
based
realm.
B
We
often
see
within
social
based
scenarios
that
there
will
be
some
actors
that
don't
engage
in
activities
in
certain
of
the
views,
and
so
you
could
think
of
this
as
like.
If
we
were
looking
at
twitter
data-
and
you
have
some
tweeters
that
never
retweet.
So
if
I
was
to
form
a
retweet
network,
they
would
inherently
be
isolates
and
you
couldn't
really
cluster
them
with
that
network.
B
Additionally,
the
the
view
topologies
or
the
the
vu
network
topologies
can
be
wildly
different
from
each
other.
So,
for
instance,
if
I
create
a
k-nearest
neighbor
network
off
the
text,
it's
going
to
have
a
very
different
topology
in
terms
of
density,
connectivity,
number
of
components
than
say
a
friend's
network
might
of
the
same
exact
actors.
B
And
then
one
of
the
great
difficulties
as
well
is
with
social
based
data.
Is
we
know
that
humans
are
kind
of
a
complex
organism
and
they
can
often
belong
to
more
than
one
social
circle
or
more
than
one
possible
group,
and
so,
when
we're
looking
at
what
types
of
behavior
or
interactions
we
need
to
use
to
characterize
them
into
something
say:
communities
or
usage
patterns.
It's
not
always
clear.
What
is
the
right
views
to
select
for
a
given
clustering
that
you're
looking
for
so
as
a
as
a
means
of
kind
of
addressing
this?
B
So,
to
begin
with
this
new
multi-view
clustering
technique,
we
first
had
to
devise
a
means
of
describing
clusters
in
social
based,
multi-view
data,
and
so
for
those
of
you
who
do
social
network
analysis,
you're,
probably
already
familiar
with
the
idea
of
network
modularity
originally
proposed
by
newman,
and
so
what
we've
done
here
in
this
first
function?
Is
we
took
that
idea
of
network
modularity,
but
we
adapted
it
to
a
multi-view
framework.
B
So
now
you
can
see
for
each
view
denoted
by
this
v
symbol
at
the
top.
It
can
have
a
different
weight
in
terms
of
its
impact
upon
that
modularity
score,
but
it
can
also
have
a
different
resolution
property
as
well,
and
so
this
resolution
property
is
particularly
important
for
social
based
data
because,
as
I
mentioned,
the
topologies
between
the
different
views
can
actually
vary
considerably.
B
B
B
B
First,
the
the
weights
when
we
talk
about
those
waiting
terms
indicate
the
relative
importance
of
the
views
to
the
cluster
structure,
so
this
can
be
kind
of
handy
when
you're.
Looking
back
at
your
data
and
saying
what
views
are
the
most
important
to
say
the
community
or
to
these
usage
patterns
with
how
users
use
things
well,
you
can
look
at
the
relative
difference
in
the
weights.
Those
that
are
higher
weighted
are
more
important
to
the
cluster
structure.
B
The
other
thing
we
found
is
we
found
that
the
number
of
steps
to
reach
convergence
in
terms
of
its
alternating
procedure
actually
indicates
the
strength
of
the
cluster
structure
present
in
the
data
across
all
the
views,
and
we
found
empirically
that
this
holds
not
only
for
say
our
particular
technique,
but
also
across
any
multi-view
clustering
technique
that
we
sampled
and
used.
So
it
has
some
very
nice
properties
in
terms
of
helping
you
understand
the
data
so
now
I
know
at
this
point
you're
probably
all
saying
well
yeah.
B
B
So
to
do
this
again,
we
kind
of
used
a
multiview
clustering
of
the
data
set
now.
First
to
start
out
with
the
data
we
were
looking
at
here.
This
was
covet
19
twitter
data
for
this
particular
study.
We
we
collected
data
at
the
cases
group
internally
from
1
february
through
30
april,
and
it
had
about
300
million
tweets
that
have,
on
average
about
forty
five
thousand
unique
hashtags
per
day.
B
There
are
relatively
high
number
of
tweets
that
feature
hashtags
around
fifty
percent
on
average,
as
well
as
users
that
use
at
least
one
hashtag
on
a
given
day.
It's
about
forty
percent
on
average,
but
you
can
see
in
these
bottom
two
charts
here.
The
ratios
of
tweets,
featuring
hashtags,
as
well
as
users
using
hashtags,
actually
does
vary
pretty
considerably
over
the
time
span
of
collection.
B
Despite
the
variations
in
usage,
however,
the
number
of
unique
hashtags
almost
continually
increases
over
the
entire
time
period.
So
there's
a
lot
more.
These
topical
hashtag
things
becoming
more
and
more
part
of
the
conversation
as
the
pandemic
went
kind
of
from
its
very
early
stages
into.
I
guess
what
we
would
now
consider
the
early
stage,
given
how
long
it's
gone
on.
B
So
there
are
some
distinct
variations
in
the
data
to
begin
with.
Now,
in
order
to
multi-view
cluster
this
data,
we
did
some
data
processing.
First,
we
broke
the
tweets
into
daily
collections,
so
there'd
usually
be
anywhere
from
a
few
hundred
thousand
to
a
few
million
million
tweets
on
any
given
day.
B
We
used
only
those
hashtags
that
were
used
in
three
or
more
tweets
per
day.
Generally.
If
you
had
a
hashtag
that
was
used
in
three
tweets
or
less,
it
was
often
a
misspelling
of
a
more
common
hashtag
and
then
in
order
to
really
leverage
this
multiview
clustering
idea.
We
thought
of
four
views
to
look
at
how
hashtags
are
used,
and
these
are
kind
of
come
out
of
the
literature
of
hashtag,
topical
clustering.
The
first
was
the
tweet
text
that
occurs
with
the
hashtag.
B
This
is
actually
a
common
way
to
cluster
hashtags
or
do
something
like
hashtag
recommendation.
The
next
was
the
other
hashtags
that
co-occur
within
a
tweet
with
a
hashtag.
Then
we
looked
at
the
urls,
so
this
would
be
the
websites
that
occur
co-occur
with
hashtags
in
their
given
tweets.
And
finally,
we
created
a
hashtag,
the
hashtag
sharedusers
network,
and
this
is
also
another
common
way
of
clustering.
Hashtags
in
the
literature
is
looking
at
the
shared
users
between
them.
B
When
we
clustered
each
day's
set
of
hashtags,
we
typically
found
around
80
clusters
any
of
these
topical
clusters
on
any
given
day.
Although
you
can
see
this
actually
can
vary
considerably
between
day
to
day.
Generally
speaking,
though,
we
see
kind
of
another
temporal
trend
like
we
saw
within
the
data
itself.
We
see
clusters
that
start
small
and
are
many
in
numbers,
so
many
small
little
hashtag
topical
clusters
and
then
they
kind
of
steadily
grow
in
size
with
now
a
lot
more
variance
in
the
size
of
the
clusters
within
the
data
as
well.
B
So
it
can
be
used
as
a
similarity
as
a
way
of
expressing
similarity
between
two
clustering
solutions
on
the
same
set
of
objects
and
what
we
found
when
we
looked
at
this
is
we
found
yeah.
There
are
just
some
distinct
periods
of
time
where
the
clusters
are
very
much
similar
to
each
other
on
those
days,
so
up
in
the
top
left
you'll
see
in
february.
All
those
kinds
of
clusters
that
occurred
on
topical
hashtag
usage
were
very
similar
and
then
again
later
on
towards
the
month
of
april.
We
see
that
kind
of
similar
pattern
emerging.
B
So
using
this
knowledge
that
there
are,
you
know
three
time
periods
of
clusters.
We
then
use
cluster
ensembling
to
produce
the
meta
clusters
for
those
time
periods
based
upon
the
daily
clusterings.
So
in
essence,
what
we
did
was
we
created
a
prototypical
clustering
solution
for
each
of
the
three
time
periods
using
these
meta
clusters
from
each
of
the
periods.
B
We
then
got
a
sense
of
the
discussion
topic,
so
we
see
things
like
persistent
topics,
including
u.s
politics,
trading,
the
economy,
world
news,
regional
news
technology
as
well
as,
even
though
these
were
all
collected
on
english
language
terms,
a
fair
amount
of
foreign
language
clusters
as
well.
But
then
the
things
are
also
kind
of
interesting.
Was
there
were
some
topical
clusters
of
hashtag
usage
that
are
sort
of
unique
to
different
time
periods,
so
we
saw
things
like
online
education
and
online
entertainment
appear
in
some
periods
of
the
data.
We
saw
negative
sentiment.
B
Hashtag
clusters
towards
the
chinese
government
appear
in
other
periods
of
the
data,
and
so
this
guy
was
thinking
in
terms
of
well.
Let's
look
at
these
clusters
a
little
bit
more
to
understand
this.
This
behavior
that's
occurring
with
these
hashtag
groups
and
so
to
begin
the
analysis
of
these
topical
hashtag
clusters.
We
first
analyzed
the
user
bases
of
the
different
clusters,
so
in
particular
we
looked
at
how
much
overlap
in
terms
of
the
users
there
was
for
all
of
the
hashtags
present
within
the
clusters.
B
So
in
other
words,
if
we
had
a
small
amount
of
users
using
all
of
those
hashtags,
it
would
have
a
very
low
overlap
score,
and
so
we
found
some
topical
areas
tend
to
have
less
diverse
user
bases,
meaning
most
of
the
hashtag
usage
was
done
by
a
relatively
small
number
of
users,
and
these
were
things
like,
for
instance,
this
cluster
3
and
period
1,
which
contains
hashtags
relating
to
the
chinese
government.
But
then
also
things
like
cluster
5
and
period
3,
which
are
hashtags
about
us
politics.
B
But
you
can
also
see,
though
one
they
have
a
lot
of
spread.
So
in
other
words,
these
hashtags
aren't
necessarily
clustering
based
on
just
their
volume
of
usage,
but
actually
more
topical
concerns.
But
the
other
one
is
too.
Is
you
see
these
little
dots
below
here,
which
are
the
outliers
outside
the
inner
quartile
range
of
usage?
B
And
so
we
went
and
looked
at
a
few
of
these,
these
outlier
hashtags
that
are
present
within
some
of
the
more
the
more
well
topically
defined
cluster
groups,
and
they
end
up
being
some
kind
of
interesting
ones
to
look
at
in
in,
I
guess
from
a
disinformation
or
political
science
point
of
view.
So
what
we
have
here
are
the
bottom
two
hashtags
from
period
one
cluster
six,
which
is
the
u.s
politics
focus
cluster,
and
so
these
are,
as
you
can
see
here,
two
hashtags
where
the
they're
used.
B
You
know
over
150
times
between
the
both
of
them,
but
only
by
one
user
and
in
fact
it's
the
same
user
between
both
hashtags.
It's.
This
bear
witness
central
website
here
we
see
kind
of
similar
things
as
well,
so
or
in
some
of
the
other
time
periods
and
clusters
as
well.
So
this
is
second
period
cluster,
seven,
which
is
generally
related
to
science.
There's
a
lot
of
hashtags
about
science,
related
topics
here-
and
this
one
we
see
is
this
particular
hashtag
called
coronavirus
reports
and
it's
only
used
by
five
users.
B
So
after
looking
at
kind
of
these
interesting,
not
only
hashtag
clusters
but
then
kind
of
the
user
bases,
we
then
kind
of
want
to
take
a
broader
focus
on
the
nature
of
the
clusters
themselves.
So
I'm
going
to
go
through
some
of
the
kind
of
the
more
interesting
clusters
we
found
within
our
data.
So
the
first
one
here
was
this
chinese
focus
cluster
in
the
first
time
period.
B
B
Here,
the
hong
kong
protest,
as
well
as
the
very
early
initial
stages
of
of
the
spread
of
covet
19.
another
one
that
was
especially
interesting
was
the
syrian
civil
war
cluster
that
emerged
in
the
first
time
period.
So
this
was
actually
a
relatively
small
cluster.
It
only
had
64
hashtags,
but
you'll
notice.
B
The
thing
that
was
particularly
also
interesting
from
a
technique
standpoint
is
like
I
mentioned.
There
was
very
little
overlap
in
the
user
bases
between
the
hashtags
being
used,
and
so,
if
I
was
to
say
analyze
these
hashtag
clusters
by
something
like
a
hashtag,
the
hashtag
sharedusers
network
odds
are
good.
This
cluster
would
not
have
been
found
because
they
are
used
by
a
diverse
user
base.
B
So
in
the
first
time
period,
which
is
again
around
february,
you
can
see
that
the
the
there's
a
little
bit
more
diversity
and
the
types
of
hashtags
being
used.
There's
definitely
some
on
the
right
side
of
spectrum,
but
there's
some
on
the
left
side
of
the
spectrum.
There's
a
lot
of
it
that
a
lot
of
the
text,
that's
relating
to
an
external
focus
like
talking
about
the
spread
from
china.
B
It's
originated
in
china,
it's
spreading
that
kind
of
thing,
but
as
we
move
into
the
third
time
period
into
april,
the
nature
of
the
u.s
politics,
cluster
actually
does
kind
of
shift,
and
so
what
we're
seeing
now
is
not
only
more
of
a
dominating
voice
coming
from
hashtags.
That
would
be
associated
with
the
right
side
of
the
political
spectrum,
but
a
lot
more
that
are
actually
associated
with
full-on
conspiracy.
B
B
So
for
those
who,
you
know,
are
more
familiar
with
these
kinds
of
things
or
not
urls
or
external
websites,
particularly
things
like
youtube
or
some
of
the
alternative
news
ecosystem
are
often
tweeted
or
used
in
social
media.
As
like
evidence
of
a
claim
right,
we
may
say
that
aha
see,
I
told
you
that
vaccines
are
fake
or
whatever
the
case
might
be.
Look
at
this
news
article
or
look
at
this
youtube
video
and
they
post
these.
B
First,
the
text
that
co-occurs
with
the
url
in
the
tweet,
but
then
we
also
separated
out
the
user
base
into
those
that
tweet
it
like
organically
tweet
it
or
those
who
are
just
retweeting,
a
tweet
that
happens
to
mention
it.
It's
been
some
recent
research
showing
that
those
those
two
behaviors
actually
have
qualitatively
different
meanings,
and
so
we
actually
separated
those
out
into
two
for
this
particular
study.
B
What
we
found
is
once
we
did
the
clustering.
You
can
see
that
it's
it's
about.
98.9
percent
of
all
urls
break
into
three
distinct
clusters
and
then
there's
about
60
or
so
68
70
outlier
clusters
that,
frankly,
just
didn't
have
enough
overlap
and
things
like
user
base
or
text
to
actually
be
assessed
into
one
of
the
three
major
clusters.
B
So
with
that,
we
went
ahead
and
said:
what's
the
nature
of
these
kind
of
three
url
clusters,
and
so
we
happen
to
have
at
the
casos
institute
done
by
very
laborious
work
by
researchers.
There,
they
put
together
a
series
of
different
labelings
for
different
domains
that
these
urls
can
come
from,
and
so
these
labelings
are
based
on
everything
from
fact-checking
websites
to
previous
publications,
all
this
kind
of
stuff,
any
kind
of
detail
like,
for
instance,
for
news
type.
B
If
it
has,
if
it's
one
of
these
black
fake
news
websites
versus
one
that
will
spread
conspiracy
theories,
we
looked
at
the
biases
in
the
domain
in
terms
of
their
political
biases,
whether
they're
pro
science
or
conspiracy
theory,
as
well
as
factual
ratings.
Is
this
a
domain
that
tends
to
publish
high
quality,
material
or
low
quality
material.
That
kind
of
thing,
and
what
we
see
was
that
what
we
know
of
at
least
for
the
known
domains
they
do
not
break
down
uniformly
across
the
three
clusters.
B
Talking
about
covet
19
in
the
united
kingdom,
then
there's
also
a
lot
of
it
about
what
various
groups
are
doing
to
help
fight,
coven
19
or
what
you
can
do
to
help
fight
kova,
19.
and
finally,
there's
cluster
2,
which
held
sort
of
the
lion's
share
of
kind
of
conspiracy.
Theory,
fake
news,
as
well
as
right,
leaning,
content,
and
you
see
a
lot
of
it
is
focused
around
personalities
again.
So
we
have
president
trump
bill
gates,
dr
faushi
all
file
into
this
there's
a
lot
of
it.
B
B
B
Buffly
are
actually
build
as
a
marketing
device
to
essentially
shorten
long
ugly
links
so
that
they
look
more
attractive
and
are
more
likely
to
get
clicked
on.
There
should
be
nothing
kind
of
a
priori.
That
would
say
that
you
know
one
particular
set
of
domain.
Labels
should
be
using
more
link,
shorteners
or
others,
but
what
we
did
find
is
that
there
are
some
link
shorteners
that
tend
to
be
used
with
some
content
versus
other
content,
which
was
kind
of
interesting.
So,
for
instance,
we
think
see
things
like
owly
and
buffly
link.
B
Shortening
services
are
used
a
lot
more
in
that
cluster,
too
than
they
are
in
some
of
the
other
clusters,
our
current
working
theory
as
to
why
this
might
be,
as
some
news
sites
actually
do,
routinely
use
certain
link.
Shortening
services
is
almost
like
a
business
agreement
in
order
to
better
promote
their
content,
but
we
actually
don't
know
quite
why
that
is,
and
this
is
something
we're
kind
of
looking
at
within
our
current
research.
B
So
with
that,
you
know
I
just
again
want
to
thank
you
all
for
the
chance
to
come
and
speak
today
and
just
to
highlight
again
that
we're
you
know
this.
This
concept
of
multi-view
clustering
for
social
based
data
is
kind
of
a
new
area,
and
there
are
certainly
a
lot
of
challenges
still
to
be
tackled
within
it,
but
it
does
show
not
only
kind
of
in
theory
but
also
in
practice
from
some
real
work.
B
B
C
B
It's
a
good
question,
so
some
of
the
other
areas
we're
investigating
right
now
are
looking
at
things
like
clustering
users
as
well.
Instead
of
just
social
media
artifacts,
one
of
the
other
areas,
we
kind
of
hope
you
know
that
might
be
interesting
to
look
at
is
combining
is
being
able
to
blend
some
of
the
data
from
online
and
offline
data
right,
in
the
sense
that
sometimes
we
have
very
rich,
descriptive
offline
data
and
surveys
or
behavioral
studies.
B
B
It's
not
really
known
that
if
we
take
these
techniques,
we
build
for
human
social
behavior
if
they
can
actually
work
in
wider,
wider
domains
like
if
I
backport
this
to
the
original
domains
that
gave
rise
to
the
field.
Does
it
actually
get
superior
performance
there
and
that's
kind
of
another
potential
area
to
look
at
with
this.
B
So,
for
for
the
hashtags
out
of
the
tweets,
what
we
did
to
create
the
vector
is
the
same
way
as
if
you
were
dealing
with
text.
So
you
can
imagine
you
know
a
document
by
term
matrix.
Instead,
you
have
a
hashtag
by
whatever
matrix.
So
in
this
case
you
could
say
hashtag
by
text
so
again,
it'd
be
hashtag
by
term
you
can
do
hashtag
by
user,
and
so
once
you
create
these
matrices
right,
you
have
you
know
hashtag
by
whatever
users
interacted
with
it.
A
A
You
know
almost
zero
weights
on
some
fuse
right
and
it
can
that
could
be
a
solution.
Is
there
a
way
to
go
back
and
say?
Okay,
I've
got.
I
have
this
clustering
for
this
particular
weighting,
find
a
weighting
that
now
emphasizes
the
other
views
that
were
down
weighted
is
it's
related
to
your
ensemble
clustering,
but
I
guess
I
didn't.
I
didn't
quite
understand
how
much
the
consensus
versus
complementary
approach
was
taken
with
this.
B
B
Predominantly
in
the
spectral
multiview
clustering
now
the
big
problem
with
translating
that
over
to
to
social
based
data
comes
with
the
fact
that
there
are
often
disconnects
or
isolates
or
disconnected
components
in
the
real
world
data.
When
I'm
looking
at
you
know
the
first
couple,
eigenvectors
right,
if
I'm
doing
a
spectral
clustering
like
in
weiss
and
jordan
style,
you
know
that
the
number
of
non-zero
eigenvalues
is
going
to
relate
to
the
number
of
components
within
it,
and
so
it
can
be
hard
to
get
get
out
the
right
eigenvectors.
B
If
you
have
a
lot
of
disconnects
in
the
first
place.
The
other
problem
we've.
Seen,
though
too,
is
that
oftentimes?
What
you
don't
want
to
you
want
to
avoid
prematurely
letting
a
particular
view
go
to
zero
right
away,
and
the
reason
being
is
that
sometimes
that
view
actually
does
contain
some
kind
of
useful
information
in
it.
You
don't
want
to
wipe
it
out
instantly
you
want
to,
you
know:
have
it
have
a
little
bit?
Let
that
kind
of
stay
in
there
and
get
you
some
information
from
it.
B
So
that's
that's
kind
of
first,
why
we
sort
of
adopted
that
the
waiting
scheme
going
that
way
but
yeah?
You
could
definitely
also
take
that
the
other
way
and
say
well
now,
okay,
so
having
done
that,
what
if
I
now
fix
the
weights
and
just
leave
them
fixed,
because
I
want
to
see
what
cluster
structure
I
get
out
specifically
focusing
on
these
particular
views.
You
can
absolutely
do
that.
A
C
C
B
I
can
attempt
to,
but
that's
a
that's
a
good
question.
I'm
not
really
sure
you
know
kind
of
what
the
policy
implications
should
be.
What
I
would
say
from
these
findings
is:
maybe
we've
got
some
avenues
of
research
that
could
help
inform
policy
more
like
we
know,
for
instance,
the
concept
of
hashtag
hijacking
right,
like
you
know,
hostile
actors
will
try
to
take
over
a
hashtag.
B
In
a
conversation,
I
think
from
some
of
this
research
and
looking
at
it
in
a
clustering
standpoint,
we've
maybe
come
across
more
techniques
for
how
they
do
that,
and
so
that
can
be
kind
of
incorporated
into
how
now
they
can
take
this
and
inform
policy
about
what
what
should
be
done
about
hostile
actors
taking
these
viewpoints.
It's.
B
You
know,
I
that's
been
something
I've
been
thinking
about,
but
I
really
I
definitely
do
not
have
a
good
answer
for
you
know.
Supposing
we
do
know
that
these
conversations,
like
we
saw
within
the
kova
data
shift
in
the
politics,
fear
from
one
of
being
maybe
more
healthy
political
dialogue,
the
one
that's
become
conspiracy,
theory
dominated
and
less
healthy.
B
B
Okay,
well
just
in
case,
please,
you
know,
let
me
scroll
back
up
to
my
title
slide.
If,
folks,
if
you
have
questions
that
that
come
up
later,
please
do
shoot
me
an
email
or
something,
I'm
always
happy
to
answer
questions
or
if
you
have
comments
about
things,
love
to
hear
them.
So
please
don't
hesitate
to
reach
out
and
send
me
an
email
if
folks
do
come
up
all
the
sudden.
Something
hits
them
later
in
the.
B
C
C
Registration
is
open
and
we
would
welcome
you
coming
and
attending
that
there
will
be
a
session
at
that
on
copic
19,
there's
the
disinformation
challenge
that
is
going
on
at
that
and
that
will
be
posted
online
and
the
entire
conference,
of
course,
is
virtual
this
year.
The
other
thing
alert
you
two
is
that
in
late
october
there
will
be
a
disinformation
conference
at
carnegie
mellon.
C
It
will
also
be
virtual
if
you're
interested
in
finding
out
more
we'll,
also
announce
it
here,
but
feel
free
to
contact
me
kathleen
carley
get
additional
information
and
then
the
other
thing
is
that
we'll
be
doing
a
special
issue
of
of
computational
mathematical
organization
theory,
that
is
on
disinformation
related
to
covid19
and
those
you
would
just
write
your
paper
and
submit
it
to
that
journal
and
and
select
that
special
issue.
I
believe
this
specialist
title
will
be
called
cobin.