►
Description
April 12th 2021: Data Science and Cyberinfrastructure Workshop
Presenter: Sagar Samtani, Kathy Benninger, Amarnath Gupta, and Jelena Mirkovic
Title: Cybersecurity as Big Data Science: An Interactive Workshop
Abstract/Description:
Cybersecurity Practitioners
• Overwhelmed by large volume, variety and fast-evolving data that are necessary to do their jobs
(intrusion detection, threat intelligence, bot detection, disinformation, malware evasion, etc.)
• Interested in learning and adopting AI/ML/Data Science tools but not sure how to begin.
Researchers working on AI/ML/Data Science for Cybersecurity
• Has limited (mostly unrealistic) data to work with.
• Faces challenges to bring research advances to practices.
Meanwhile, there might be data science techniques being overlooked.
So let’s bring the community together to talk about:
• Cybersecurity use-cases that could benefit from data science tools and approaches.
• Technical and non-technical challenges to treat cybersecurity as big data science.
• Plausible solutions and best practices to apply data science for cybersecurity.
• Training, education, and engagement strategies.
This INTERACTIVE workshop is meant for all of use to share thoughts!
• 73 in Cybersecurity
• 26 in Data Science
• 11 in HPC
• 7 in Other scientific domain areas
• 7 in Business Operations
• many “others” or did not answered
• From 11 countries
• Please use the poll to select reasons that you are here.
Use the following ways to share your thoughts:
• Google Doc (link in Zoom Chat)
• Zoom Chat
• Raise your ”virtual” hand in Zoom and we will call you to speak.
Panelists:
• Sagar Samtani, Indiana University
• Kathy Benninger, Pittsburgh Supercomputing Center
• Amarnath Gupta, San Diego Supercomputer Center, University of California San Diego
• Jelena Mirkovic, Information Sciences Institute, University of Southern California
• and YOU!
Topics:
• Use-cases
• Challenges
• Solutions/Best Practices
• Engagement/Training/Education
We will begin by hearing briefly from the panelists for use-cases and challenges.
A
Well,
I
wanted
to
welcome
everybody
today
to
the
cyber
security's
big
data
science
interactive
workshop,
it's
great
to
see
so
many
new
faces.
This
was
a
meeting
that
we
had
I've
been
planning
for
about
a
couple
of
months
now
to
really
try
and
get
some
interactive
discussions
between
people
to
foster
new
ideas
in
data
science
and
in
cyber
infrastructure.
A
As
far
as
ai
and
machine
learning
goes,
and
that
is
cyber
security
and
not
in
the
sense
of
of
compliance,
but
more
so
in
the
sense
of
detection
and
and
reaction
to
events.
A
lot
of
these
are
shared
with
other
areas.
I
it's
a
widely
distributed
data
field,
so
this
is
very
similar
to
things
like
smart
agriculture
or
even
large
telescope
surveys.
A
It
has
a
lot
of
security
concerns,
so
it
has
some
similarities
to
things
that
are
going
on
in
medical
professions.
It's
about
remote
detections
with
poorly
labeled
data,
which
is
really
a
challenge
for
everyone.
And
lastly,
it's
about
really
detecting
things
that
really
aren't
known
per
se
in
advance,
because
of
course,
what
we're
trying
to
find
is
the
new
things
that
are
going
on
in
the
realm
of
cyber
security
and
react
to
those.
A
But
then,
following
this
as
the
next
cyber
infrastructure,
big
data's
working
group
meeting
on
friday
may
7th
at
3
pm
eastern
we'll
be
hosting
another
session
to
just
sort
of
have
more
discussions
around
what
has
occurred
to
people
or
how
things
are
working.
After
all,
this
sinks
in
and
and
we've
had
a
chance
to
think
about
what
we've
learned
and
talked
about
today.
A
B
Thank
you
niall.
Thank
you.
Everyone
for
coming
over
here,
I'm
very
excited
to
have
this
interactive
workshop.
That's
the
key
word.
One
of
you
already
asked
about
whether
the
style
being
shared.
Yes,
the
limited
share
slide
will
be
shared,
but
mostly
we'll
be
hearing
from
you,
our
community,
so
this
workshop
will
be
recorded,
and
so,
if
you
do
have
any
concerns,
then
please
let
us
know
we'll
work
out
a
way
so
that
this
recording
will
not
be
shared
to
the
public
but
to
share
with
just
the
participants.
B
So
while
we
are
here
so
why
are
we
here
we're
here,
because
there's
a
there
has
been
a
lot
of
security
practitioners.
They
are
really
overwhelmed
by
a
lot
of
data
right,
the
the
shared
common
properties
with
other
big
data,
science
problems
and
there's
a
lot
of
use
cases
you'll
hear
quite
a
lot
from
the
panelists
today,
at
the
same
time,
they're
really
interested
in
learning
adopting
the
ai
ml
data
science
tools,
but
not
sure
exactly.
What's
the
best
way
to
begin.
B
Meanwhile,
the
researchers
are
working
on
this
area,
like
myself,
has
really
limited
and
mostly
unrealistic
data
to
work
with.
We
also
face
a
lot
of
challenges
to
bring
our
advances
to
real
practices
right
in
operation,
and
we
also
recognize
that
there's
a
lot
of
data
science
techniques
out
there,
not
the
simplest
type
that
security,
but
they
could
still
be
beneficial
and
but
could
be
overlooked
as
well.
So
we
really
wanted
to
bring
the
community
together
to
talk
about
the
use
cases
right.
B
The
challenges,
technical
and
non-technical
challenges,
the
plausible
solutions,
best
practices
right,
not
just
technical
solutions,
but
best
practice
to
to
get
it
done
right
and
how
do
we
engage
training
and
educate
people
in
this
area?
I
wanted
to
have
this
discussion
with
you,
so
it's
very
interesting.
We
have
140
registrations
as
of
maybe
yesterday
and
there's
a
lot
of
variety
of
different
interest,
backgrounds
and
areas,
and
so
on
and
so
forth,
and
we
have
people
coming
from
11
countries
very
proud
of
this
broad
region.
B
Thank
you
on
flores
and
north
east
speaking
in
the
hub
behind
the
scene
to
get
this
working
helen
tia
all
of
those.
So
we
really
want
to
use
a
variety
of
ways
to
interact
with
you
today
right.
So
one
of
them
is
the
pope
hell
if
you
can
launch
a
poll,
so
they
can
know
that
why
you
are
here
where
you
want
what
you
want
to
learn.
B
B
You
can
use
the
doom
chat
and
if
you
feel
more
comfortable
with
that
and
we'll
try
to
copy
and
paste
past
those
zoom
chat
into
the
online
google
doc,
you
can
also
raise
your
virtual
hand
if
you
don't
know
how
to
do
that.
I
guess
that
we
can
provide
instruction
in
the
chat
for
you
the
virtual
hand
and
we'll
track
them
as
much
as
we
can.
You
know
we
have
if
we
do
have
140
people
here,
it's
very
difficult
to
try
we'll
do
our
best
and
then
we'll
call
you
to
speak.
B
B
I'm
really
proud
to
have
these
penalties
all
together
to
to
share
their
thoughts
with
you,
sagar
from
indiana
university
kathy
with
the
pittsburgh
small
computing
center
ominous
is
with
a
san,
diego
community
super
computing
center
from
uc
san,
diego
and
yelena
for
information
science
institute
for
usc
and,
of
course
you.
So
I
would
really
want
to
stress
that
this
workshop
this
next
an
hour
and
20
minutes
is
not
just
about
the
penalties.
B
Presenting
you'll
see
very,
very
few
slides,
probably
no
more
than
10,
because
we
really
want
to
hear
from
you
your
thoughts
right
through
the
google
doc,
raise
the
virtual
hand
to
chat
any
ways
you
can
to
share
with
your
thoughts
right.
What
is
your
concerns?
Your
challenges,
suggestions?
You
have.
We
want
to
listen
so
the
topical
areas
as
we
I
said
earlier.
We
use
cases,
challenges
again,
technical
and
non-technical
solutions,
best
practices,
suggestions
and
we'll
talk
about
how
to
potentially
engage
further
training
education
and
will
rather
wrap
up
with
the
next
step.
B
You
can
do
from
this
workshop,
and
I
already
mentioned
about
the
working
group
monthly
weeks,
all
right
so
we'll
begin
by
hearing
briefly
from
the
panelists
for
their
thoughts
on
the
use
case
and
challenges.
So
god
is
going
to
present
very
briefly
and
then
kathy
alvarez
and
yelena
will
briefly
talk
about
their
thoughts
as
well.
Right,
I'm
going
to
unshare
my
screen
and
have
some
god
take
it
away.
C
Sagar,
thank
you
jay.
So
let
me
share
my
screen
here.
So
can
everybody
see
my
screen?
Yes,
okay,
perfect!
So
thank
you
so
much
jay,
and
also
thank
you,
florence
and
northeast
big
data
hub
for
the
opportunity
to
to
have
this
workshop
at
this
interactive
workshop
and
to
really
get
some
good
ideas
flowing
in
terms
of
this
emerging
area.
C
So
I
only
have
a
few
slides
to
kind
of
set
the
stage
with
the
other
panelists
here
I
I'm
not
going
to
speak
for
too
long
at
five
minutes
maximum,
but
I
really
want
to
kind
of
just
set
the
stage
as
to
why
cyber
security
has
a
big
data
sign
some
of
the
challenges
and
the
opportunities.
So,
as
jay
mentioned,
I'm
at
indiana
university,
I'm
an
assistant
professor
in
the
kelly
school
of
business.
C
So
I
I
think
the
the
first
question
is:
what
is
why
the
focus
on
cyber
security
we've
heard
data
science
in
many?
Many
other
contexts
as
well,
but
why
cyber
security
and
really
I'm
not
going
to
belabor
the
fact
too
much
that
cyber
attacks
really
cost
the
global
economy.
A
tremendous
amount
of
money
on
a
on
an
annual
basis
and
really
no
single
sector
is
immune
from
this
type
of
some
from
these
types
of
attacks.
C
It
really
affects
government,
industry,
academia
and
so
on,
and
I
know
that
we
have
coverage
from
about
11
different
countries
here
today,
but
within
the
us
it's
really
been
labeled
as
one
of
the
national
strategic
priorities
now
within
for
for
cyber
security
and
data
science.
Some
of
the
common
challenges
that
many
cyber
security
practitioners
and
academics
face
is
data
overload,
prioritizing
the
appropriate
indicators
ever
evolving
data
sources
and
so
on.
C
So,
even
though
the
areas
of
data
science
and
cyber
security
have
historically
been
a
little
bit
siloed
there,
they
do
have
a
very
they
have
the
potential
to
have
great
relationship
with
each
other
they're,
not
just
buzzwords
and
really.
If
I
were
to
summarize
it
down
what
is
the
role
of
data
science
in
cyber
security?
C
I
think
the
real
question
that
comes
up
in
terms
of
doing
data
science
for
cyber
security
is
that
what
is
the
data
that
is
relevant
within
cyber
security
and
there's
really
kind
of
two
broad
areas:
that
of
data
that's
accessible
within
the
cyber
security
context
and
on
the
sign
up
page
for
the
big
data
hub
for
the
for
this
particular
workshop
on
the
northeast
big
data
hub,
there's
an
editorial
link
at
the
bottom.
That
summarizes
a
little
bit
further.
C
But
really
there
are
two
major
categories
of
data
sources
within
the
co
that
can
be
used
for
for
cyber
security
applications.
One
is
internal
and
the
other
one
is
external.
The
key
delineation
is
internal.
Is
the
type
of
data
that's
within
an
organization
that
any
organization
would
have
access
to
and
external
is
outside
in
cyberspace
itself.
C
They
can
provide
a
great
insight
into
some
of
the
existing
threats
and
past
attacks
as
well,
but
the
heterogeneity
and
the
velocity
of
the
data
can
be
quite
substantial
here
and
then
there's
external
data,
so
data
sources
that
are
external
to
an
organization.
So
this
could
be
iot
search
engines,
dark
web
related
data,
karting
shops,
pace
sites,
threat
feeds
and
so
on.
It
gives
a
great
perspective
of
threats
that
are
external
to
an
organization.
This
data
is
primarily
generated
by
humans.
C
C
Thus
far,
there
are
really
four
major
themes
of
data
science
or
cyber
security
that
have
emerged.
One
is
cyber
threat,
intelligence,
disinformation,
computational
propaganda
security,
operation,
centers
and
adversarial
machine
learning,
but
as
you'll
hear
from
some
of
the
panelists
today,
some
of
the
key
limitations
within
the
space
is
that
there's
often
very
siloed
resources.
There's
often
a
lack
of
publicly
accessible
and
diverse
data
sets
that
are
realistic
as
well
as
labeled
or
balanced,
and
then
things
such
as
model
sharing
is
also
remains
a
challenge
as
well.
C
C
There's
all
the
way
from
emerging
applications
and
data
to
different
types
of
ai
based
methodologies,
such
as
multi-view
learning
and
multimodal
analysis
to
the
role
of
decision
making
within
this
particular
context,
so
having
more
automated
predictions
and
resiliency
and
those
types
of
topics,
but
we'll
delve
into
a
little
bit
more
of
this
as
we
proceed
forward.
So
with
that
I'll
go
ahead
and
stop
sharing
my
scribe,
my
slides
and
then
I
think
cat
these
up
next.
D
Thanks
sega
hi,
my
name
is
kathy
benninger.
I'm
would
like
to
thank
jay
for
inviting
me
to
participate
today.
I
am
the
manager
of
networking
research
at
the
pittsburgh
super
computing
center,
based
at
carnegie
mellon.
I've
been
a
poc
for
over
30
years.
I've
worked
in
a
range
of
cyber
infrastructure
projects
earlier
years,
compute
storage
and
video
systems,
but
most
recently
I've
been
focusing
on
on
networking
network
performance
and
emerging
network
technologies,
so
I
kind
of
come
to
it.
D
D
These
team
members
are
representatives
from
each
of
nsf's
major
facilities
who
are
responsible
for
cyber
security
at
their
respective
sites.
For
folks
who,
who
aren't
necessarily
in
familiar
with
major
facilities,
those
are
the
nsf
projects
that
are
funded
at
100
million
dollars
and,
above
so,
they're
they're
things
such
as
the
observatories,
telescopes,
ligo,
antarctic
programs
and
and
trusted
ci
works
with
them
and
facilitates
their
their
collaboration
on
cyber
security
related
issues.
D
D
D
The
part
of
the
challenge
here
also
the
major
facilities
are
attractive
targets
and
the
folks
who
work
in
the
cyber
security
roles
within
the
sites
are
are
sometimes
there's
a
separation
between
what
they're,
they're
doing
and
some
of
the
the
data
science
applications.
So
there's
some
area
of
work
that
can
be
done
in
that.
E
F
E
So
let
me
first
talk
about
the
problem,
so
we
at
san
diego
center,
we
have
a
lab
where
we
look
at
how
to
gather
data
from
various
sources
and
how
to
analyze
them
together.
So
there
was
one
question
from
andreas.
That
said,
what
kind
of
data
model
should
you
use?
Would
it
be
a
document
database,
graph
database
etc?
Right?
So
let
me
let
me
speak
to
that
with
respect
to
our
work,
so
here
are
two
cases
that
we
that
we
talk
about.
E
So
there
are
standards.
You
know
people
have
said.
Okay
here
is,
there
is
an
attack.
Can
we
characterize
the
attack
and
there
are
weaknesses,
though,
of
known
software
systems,
like
documented
weaknesses.
So
if
you
look
at
that,
nist
and
other
people,
mitre
corporation,
have
created
these
standards.
So
there
is
a
common
weakness.
Enumerator,
it's
a
cwe.
There
is
a
national
vulnerability
database
which
essentially
tries
to
document
things,
and
the
goal
is
when
you
see
a
new
incident
report
coming
not
from
the
machine
like
sagar
said,
but
somebody
puts
it
in
okay.
F
E
E
E
E
E
To
characterize
the
problem,
the
second
problem
comes
from
here,
so
this
is
social
media,
and
this
business
has
been
called
threatened.
Intelligence.
It
has
also
been
called
risk.
Intelligence
here
are
two
tweets
that
we
gathered.
We
gathered
like
3.5
million
tweets
per
day
on
various
things,
so
this
was
happening
the
first
of
january,
and
you
would
see
that
there
are.
There
are
some
agitated
people
talking
about
fighting.
You
know,
you
know
if
something
goes
wrong
in
politics
now.
The
question
is:
is
this
risky
or
is
this
not
risky?
E
E
What
we
have
figured
out
is
now
now
the
issue
about
data
models
and
so
forth.
There
is
no
one
thing
that
you
can
do.
We
have
some
of
the
data
in
relational
tables.
We
have
some
of
the
data
in
networks.
We
have
some
of
the
data
in
time
series
analysis,
and
indeed
we
have
some
of
the
data
in
text
analysis
and
we
try
to
combine
this
heterogeneous
analytics
to
see
if
we
can
figure
out.
What's
going
on.
B
B
E
E
B
F
F
They
are
hard
to
share,
so
many
research
papers
are
published
that
you
know
with
great
results,
but
the
data
that
is
being
used
is
private
data
and
no
one
can
repeat
the
results.
No
one
can
you
know,
compare
somebody
some
technique
that
they
develop
with
the
technique
that
was
published
because
we
don't
have
the
same
data,
that
is,
that
is
equally
large
and
rich.
So
there's
this
problem
of
data
sharing
that
is
pervasive
when
you
talk
about
cyber
security
data
and
some
issues
that
arise
around,
that
are
privacy
issues,
but
also
reputation
issues.
F
So
so
that's
one
issue:
how
to
share
the
data
and
then
the
other
issue
that
I'm
very
interested
in
is
how
to
label
data
because,
as
you
heard
from
other
panelists,
the
ground
truth
is
very
elusive
in
this
realm.
Usually
what
people
do
is
they
use
commercial
products
to
label
data,
for
example,
if
you're
researching
malware,
you
might
use
virustotal
to
label
it
and
say
this
is
malware.
This
is
benign,
but
even
the
commercial
technologies
are
just.
You
know,
technologies
that
that
are
trying
to
do
their
best
to
label
this
data.
F
They
are
not
100
correct,
and
so
the
question
then
arises:
if
you
develop
some
technique,
that
is
say
better
than
commercial
in
some
aspects.
How
can
you
prove
that
when
you
don't
have
ground
truth,
when
you
don't
have
the
label
data,
and
so
one
approach
that
I'm
working
on
with
with
my
colleagues,
is
looking
how
to
devise
these
probabilistic
labels
that
multiple
researchers
could
assign
to
the
same
data
set
and
then
other
researchers
could
work
with
these
labels
and
we
could
have
this
probabilistic
ground
truth.
B
Thank
you,
then.
I
think
this
is
great
insight
on
that.
There's
a
lot
of
angles
on
that,
so
I
just
post
it
in
the
chat
box,
where
you
can
put
things
into
the
google
doc
you
can
raise
hands,
please
feel
free
to
raise
hands
where
people
to
track
that
and
then
invite
you
to
speak
up.
Meanwhile,
I
want
to
share
with
you
so
the
earlier
poll
that
we
did
most
of
you
were
wanting
to
learn
more
about
what
kind
of
approach
you
get
that
says.
B
Approaches
can
help
solve
such
problems,
and
many
of
you
want
to
know
subject
that
can
benefit.
So
it's
bi-directional.
We're
really
glad
to
see
that
so
right
now
we
are
in
these
topics
of
use
cases
and
challenges.
We
really
want
to
hear
from
you
several.
You
are
typing
as
we
speak
and
as
you
look
watching
online
is
very
interesting.
Looking
at
you
putting
in
your
thoughts,
but
definitely
that
please
feel
free
to
raise
hands
helen.
B
Can
you
launch
the
second
poll,
so
the
second
poll
we're
asking
you
what
you
think
this
is
just
some
some
subset
of
the
challenges
so
and
then
again,
technical
and
non-technical
challenges?
We
want
to
hear
your
thoughts
on
that.
Take
your
time
to
express
your
thoughts
on
that,
but
at
the
same
time
we
really
want
to
hear
from
you
your
thoughts
on
use
cases
and
challenges.
Conceptually
could
benefit
from
big
data
science
or
data
science
tools.
B
B
I'll
put
a
comment
here
that
I
think
that
there's
a
indeed
a
lot
of
data
challenges.
Personally,
I
I
think
that
it's
going
to
be
very
difficult
like
in
some
other
data
science
domains.
I
don't
think
there'll
be
a
lot
of
ground.
Truth.
Labels
label
data
in
this
domain,
and
I
think
that
there's
also
a
very
fast
evolving
nature
of
adversary.
Behaviors,
as
well
as
our
networks,
our
systems
there'll,
be.
The
vulnerabilities
will
change
very
quickly
so
methodologies
in
fact
methodology.
B
B
I
will
bring
up
a
few
questions
being
posted
online.
One
of
the
is
talking
about
which
database
models
are
the
most
successful
to
tackle
emerging
cybersecurity,
threats,
time
series
database
graph
based
database,
no
sql,
no
document
kind
of
thing.
What
are
your
thoughts?
What
kind
of
database
would
be
great
for
substring
bombs.
F
I
guess
that
depends
on
the
problem
you're
addressing
in
our
work.
We
worked
a
lot
with
time
series
data,
but
you
know
it's
really.
Cyber
security
is
broad
that
it
depends
if
you're
working
with
passwords
or
block
lists
or
something
you
know
like
it
will
be
a
completely
different
kind
of
problem
that
you're
addressing.
E
Let
me
give
a
different
answer
to
that
question
and
I
think
yellow
is
absolutely
right.
It
depends
on
the
problem
right
so
because
we
try
to
integrate
data
from
multiple
sources.
We
have
developed
and
still
developing
what
a
thing
called
polystore,
which
means
there
are
many
several
there
is
a
graph
database
underneath
it.
There
is
a
time
series
database
underneath
and
these
are
all
standard
systems,
but
what
we
do
is
we
take
an
analysis
problem
and
we
make
them
look
at
all
the
different
kinds
of
data
and
exchange
information.
B
Thank
you.
I
also
see
quite
a
few
students
online
as
well
some
of
them.
I
know
I
don't
want
to
call
your
name.
I
know
that
you
know,
I
know
you
but
students
if
you
are
working
this
area
here,
any
thoughts
or
questions
here
are
a
lot
of
great
professors
here
or
practitioners
here
for
you
to
ask
your
questions.
This
is
your
opportunity
to
get
those
answers
of
your
your
work.
B
Some
other
questions
we're
talking
about
qaeda,
so
qaeda
does
have
this
network
telescope
collecting
a
lot
of
data.
I
know
sagar
has
also
has
ample
data
in
front
of
dark
web
and
vulnerability
elements.
I
have
actually
shared
with
several
links
to
a
variety
of
data
sets
that
people
have
post
online
from
us
from
canada
from
european
and
many
other
places,
so
they
are
data.
The
question
is:
are
those
data
usable
and
are
they
data
solving
the
problem
that
we
are
tackling?
B
C
Thoughts,
I
do
so
one
thing
just
to
kind
of
circle
back
on
the,
if
I
can
just
summarize
some
of
the
data
characteristic
issues
that
we've
heard
so
far
on
the
panel
and
then
I'll
delve
into
some
of
my
thoughts
in
terms
of
data
data
challenges
and
so
on.
So
I
think,
if
I
can
just
summarize
the
what
the
panel
has
said
so
far
in
terms
of
the
as
it
pertains
to
data,
I
can
really
put
it
into
three
or
four
words.
C
I
think
it
would
be
data
acquisition,
data
representation
and
then
data
pro
data
processing.
So
let
me
talk
about
both
of
them,
so
data
acquisition
is
how
do
we
acquire
realistic
data
that
is
actually
going
to
help
us
out
with
some
of
our
analytics
or
some
of
the
data
science
issues
that
we're
seeing
data
representation
is.
How
do
we
represent
the
data?
Do
we
represent
it
as
a
time
series
kind
of
issue?
Do
we
represent
it
as
a
graph?
C
Do
we
represent
it
as
a
sequence
and
so
on
and
so
forth,
and
then
the
last
bit
is
that
how
do
we
make
sure
that
the
data
is
labeled
and
it's
in
a
way
that
we
can
actually
use
it
for
some
sort
of
analysis?
I
think
those
are
some
of
the
themes
that
we're
starting
to
see
from
the
emerge
from
the
panel
they're
kind
of
synthesizing.
Some
of
the
comments
that
we've
we've
got
so
far
now,
as
it
pertains
to
you,
know
kind
of
my
perspective
on
this
I'll.
C
Just
give
example
of
the
types
of
data
that
I
collect
and
the
way
that
I
structure
and
format
them
and
then
maybe
we
can
get
some
better,
better
sense
of
from
the
other
panelists
and
from
the
audience
as
well
in
terms
of
different
use
cases.
So
me
for
me
personally,
a
lot
of
the
data
that
I
gather
is
actually
not
secondary
data
sources
in
the
sense
of
downloading
a
data
set
off
of
the
web,
such
as
kaida
or
existing
data,
sets
that
folks
may
use
those.
C
C
That
may
not
be
obvious
right
off
the
bat
I'll
give
one
example.
When
I
was
a
phd
student
at
the
university
of
arizona,
I
was
collecting
a
lot
of
hacker
forum
data
and
collecting
a
lot
of
the
data.
Well,
at
one
point,
my
virtual
machine
that
was
collecting
the
data
had
600
viruses
on
there
because
of
the
people
that
were
trying
to
attack
me
back
right.
So
these
are
some
of
the
unique
challenges
within
the
context
of
data
science,
for
cyber
security.
C
That's
unique
to
a
particular
type
of
data
set
within
within
cyber
security,
and
similarly
you,
you
may
see
some
of
those
types
of
adversarial
behaviors
as
well.
Now,
in
terms
of
the
database
structures
that
I
I
lean
on,
I
I
primarily
use
my
sequel
and
mongodb,
and
I
found
that
to
be
sufficient
for
the
type
of
work
that
I'm
doing,
but
I'd
love
to
also
get
some
thoughts
from
the
other
panelists
like
elena,
in
terms
of
some
of
the
data
sets
and
structures
that
you
use
that
you
found
to
be
useful
in
your
work.
F
B
Thank
you,
so
I
I
I
hear
a
lot
of
great
conversation
here
and
there's
also
another
parallel
conversation
on
the
chat
from
caroline
as
well
as
that,
I
think,
for
oscar
talking
about
this
point
on
real
world
problems.
Real
world
data,
real
world
feedback.
We
will
use
this
with
research
right,
so
research
advances
in
terms
of
connecting
the
between
the
research
and
the
practice
and
actually
the
core
results.
B
I
think
that
most
of
you
can
see
right
now.
The
highest
percentage
of
you
talking
about
the
mismatch
of
incentives
goes
between
the
research
versus
industry.
Application
focus,
that's
actually
one
of
the
top
outcome
of
your
process.
That
is
one
of
the
biggest
challenge
followed
by
some
others,
and
the
second
one
is
limited
collaboration
between
seven
screen
and
data
science,
experts
right
so
again
it's
the
the
the
collaboration.
D
They
are
trying
to
to
do
the
best.
I
can
day-to-day
and
dealing
with
the
cyber
security
challenges,
but
you
know
adding
adding
these
new
skills
and
just
kind
of
picking
up
with
the
science.
That's
sort
of
in
its
kind
of
early
early
phase.
Here
is
a
challenge,
so
they're
they're
interested
in
in
ways
to
ease
that
that
integration
very
much
thank.
A
Yeah,
so
I
think
this
sort
of
ties
together
with
both
the
points
of
sort
of
the
the
lack
of
integration
necessarily
between
academic
and
and
sort
of
real-world
practitioners
and
and
as
well
as
what
sounds
like
a
very
disjointed
sets
of
data
and
and
other
problems
around
that,
because
you
know
this
is
the
cyber
infrastructure
working
group.
I
was
just
going
to
put
the
bug
in
people's
ears
around
that
about.
You
know
we
talk
about.
A
But
if
we
were
to
reduce
some
of
the
hurdles
of
entry
would
really
benefit
this
two-way
discussion
of
both
research
on
data
and
publishing
algorithms,
so
that
people
can
then
validate
those
coming
in
from
outside.
Going
into
then,
what
what
would
sort
of
be
discovered
discover
discussed
as
as
a
fair
type
data
system.
A
B
Thank
you
nyle.
I
think
that's
very
interesting,
so
I'll
provide
a
little
bit
of
my
perspective.
Answering
the
question
just
rates,
so
I
think
for
the
researchers
actually
a
lot
of
times.
We
want
data,
but
we
really
don't
care
about
the
the
legal
people
what
legal
people
might
have
heard
about
the
city.
Ipa
address
the
city,
url
specific
payload
data,
some
of
those
things
that
we
don't
necessarily
care
as
much.
Of
course,
there
are
cases
we
do.
B
There
are
a
lot
of
cases.
We
don't
so
oftentimes
the
blanket
statement
about
or
we
need
this
nda.
I
think
I
was
going
to
mention
that
we
need
that
kind
of
contract.
We
need
to
anonymize
in
some
way,
there's
no
way
to
anonymize.
I
think
the
blanket
statement
make
it
so
hard
in
my
opinion
that
there
are
actually
a
lot
of
ways
to
walk
around.
So,
in
my
opinion,
if
we
as
a
community
can
list
that
I
can
provide
mind
that
here
are
the
data
and
fields
that
I
really
need.
B
The
rest,
I
don't
need
them,
and
actually
we
have
experience
been
successful
on
that
and
the
cysos
and
legal
people
say
look
at:
can
we
realistically
release
those
fields,
but
not
the
other
fields?
Right,
don't
think
about
break
blanket
statement.
I
think
that
can
leave
you
a
lot
of
concerns
that
connecting
the
industry,
the
the
actual
practical
operations
with
the
researchers.
H
H
So
just
your
thoughts
on
that
and
also
you
know,
I
can
also
also
put
in
the
chat
if
you're
interested
or
if
this
is
something
that
you
know
is
in
your
area.
Thinking
about
we're
going
to
open
up.
You
know
a
program
for
actual
sublimatic
experts
in
this
in
this
area,
which
is
already
out
so
those
two
things
that
question
and
then
I'll
also
put
in
if
you're
interested,
you
can
look
at
the
link
and
just
let
me
know,
and
we
can
email
more.
B
B
I
I
I
B
Caroline
thank
you
for
the
comment,
indeed,
that
I
think
prioritizing
the
indicator
compromise
be
able
to
predict
what
might
be
the
most
impactful
attack
is
definitely
a
very
interesting
part
of
it.
In
my
personal
experience,
one
of
the
things
that
the
analysts
in
stock
analyst
is
looking
for
is
how
do
we
help
them
to
prioritize
the
overwhelming
amount
of
data
right?
So
I
absolutely
agree
with
you.
I
think
that
the
industry
also
need
to
start
thinking
about
that.
B
How
do
we
stop
looking
at
all
the
event
logs,
all
the
network
flows,
all
the
intelligent
alerts
right
because
there's
so
so
much
of
them
and
start
thinking
about.
Can
we
look
at
a
summary
of
ongoing
attack
attacks
right
so
few
of
them
right,
instead
of
hundreds
of
thousands
looking
at
the
tens
of
them
only
or
even
within
10
prioritize
them.
So
there
are
some
works
out
there
in
industry
and
academia
in
those
those
elements
that
could
help
in
those
I
can.
I
can
chat
with
you
offline
more
on
that
as
well.
B
There
are,
I
think,
that
we
are
moving
already
start
towards
not
just
challenges
and
use
cases,
but
also
solutions
and
best
practices.
Certainly
most
of
the
audience
are
in
the
cyber
security
area,
but
I
also
hope
to
hear
from
data
science,
data
scientists
or
people
who
work
in
other
science
area
and
given
what
you
have
saw
her
so
far,
do
you
have
any
thoughts
about
hey
that
problem?
We
have
seen
it
before
right,
getting
the
users
to
use
it
that
we
have
seen
it
before
getting
people
to
share
data.
B
B
C
I
think
just
to
build
on
what
I
mentioned
a
little
bit
earlier.
I
think
the
key
themes
that
I'm
seeing
that
are
coming
up
in
the
discussions
here
great
discussions
is
related
to
the
data
access,
the
data
representation,
the
data
labeling,
the
data
management
aspects
as
well,
but
I
think
just
to
add
on
to
that
in
terms
of
what
else
we've
seen
so
far
come
up.
C
I
saw
in
the
chat
that
there
were
a
few
other
concerns
that
I
would
lump
into
a
similar
theme
as
well
in
terms
of
issues
with
data
set
access,
as
it
pertains
to
mousse,
ndas
and
and
so
on,
and
so
forth
that
that's
a
major
point
of
concern
for
some
folks
as
well.
That
may
be
a
little
bit
different
than
other
data
science
context.
I'd
love
to
get
some
thoughts
about
that.
B
Thank
you
cigar.
That's
great.
I
continue
to
track
the
a
lot
of
chat
going
on,
so
it's
like
a
so.
This
is
a
very
interesting
experience.
Right,
like
we
have
three
channels,
one
radio
channel,
that's
this
voice.
The
other
channel
is
on
the
chat
and
the
third
channel
changes
on
google
doc.
At
the
same
time,
it's
very
interesting
how
we
share
that.
We
I've
seen
silent
just
talk
about
some
conversation
with
yelena
seven.
You
want
to
talk
a
little
bit
about
your
questions.
Oh
yeah,
elena.
G
I
can
I
can
comment
if
you
want.
Can
you
hear
me?
Yes,
so
I
guess
you
and
I
were
just
chatting
about
labels
of
the
data,
and
the
point
was
the
only
point
was
making
was
that
you
could
assign
different
labels
to
the
data
and
different
evaluations.
But
my
my
point
was
the
fact
that
it
needs
to
be
dynamic
right
as
even
if
we
have
a
certain
set
of
some
certain
data
set.
That
at
one
point
gives
us
one
insight.
G
You
run
it
through
a
different
algorithm
or
you
have
some
ground
truth
later.
That
tells
you
that
another
label
needs
to
be
assigned,
so
there's
a
need
to
have
this
dynamic
labeling.
So
that
was
really
the.
I
think
the
summary
of
the
conversation
and
y'all
know
if
you
forget
to
jump
in
if
I
didn't
represent
your
point
of
view
correctly,.
F
So
what
I
was
what
I
was
talking
about
in
the
chat
there
was
a
question
about
the
probabilistic
labels
that
I
mentioned,
and
I
said
we
have
many
algorithms
for
anomaly
detection
or
for
detection
of
various
kinds
of
attacks
that
we
want
to
look
for
in
data.
But
these
are
these
algorithms
can
be
more
or
less
effective,
so,
instead
of
using
them
to
label
with
the
absolute
ground
truth,
we
can
label
by
saying
algorithm.
Xyz
said
this
is
abnormal,
and
then
we
can
have
multiple
labels
for
the
same
record.
F
Just
like
virustotal
has
many
antivirus
engines
and
then,
when
you
do
research
on
malware,
you
will
say
something
like
well.
I
selected
malware
from
virustotal
that
had
80
of
tools
label
it
as
malware,
and
I
am
going
to
work
with
that
data
set.
So,
conversely,
we
could
then,
when
we
evaluate
let's
say
the
scanning
detection,
phishing
detection,
you
know
what,
whatever
whatever
detection
algorithm
we
are
evaluating
for
different
anomalies
and
data
sets.
We
could
say
this
was
effective
on.
F
You
know:
95
effective
on
data
that
was
labeled
anomalous
by
80
of
the
labels,
so
that
would
give
us.
You
know
better
better
way
to
progress
in
case
of
uncertainty
and
then
sven
was
saying
that
the
labels
are
dynamic,
because
we
always
learn
more
about
the
data
set.
That's
absolutely
true
and-
and
you
know
more
algorithms
emerge,
so
you
can
see
this
set
of
labels
as
an
evolving
set
over
time.
J
Yes
hi,
so
I
I
did
raise
a
couple
of
points
which
I
don't
really
remember,
and
the
yeah.
But
basically
the
point
was
that,
as
a
phd
student,
we
usually
have
a
fixed
term
contract
and
well
figuring
out
these
ndas
and
setting
up
these
collaborations
with
the
companies
can
take
a
long
time.
There's
a
lot
of
red
tape
around
it
and
I
think
that's
a
really
big
issue,
because
there's
a
lot
of
research
that
is
done
by
the
graduate
students,
but
because
there's
a
high
turnover
kind
of
then
yeah.
J
These
are.
These
are
some
of
the
issues
that
well,
the
research
community
needs
to
work
on
together,
so
that
we
can
support
the
graduate
students
who
who
want
to
work
on
these
challenges.
Otherwise,
there's
a
lot
of
work,
that's
going
into
these
problems,
but
then
there's
no
good
solution
coming
out
of
it,
because
you
know
these
are
problems
that
well.
J
So
in
ai,
particularly
there
are
toy
problems
for
dif
for
for
at
least
emerging
research
topics
and
that's
how
a
sort
of
a
research
field
builds
itself
and
then
once
it's
mature
enough,
then
we
start
going
more
into
the
complex
sort
of
edge
cases,
and
that
is
something
that
I
think
we
are
sorely
lacking
in
security.
I
have
rarely
seen
toy
problems
being
appreciated,
at
least
in
academia
and
also
in
industry,
because
of
course
they
are
really
dealing
with
the
real
world
problems.
J
But
then,
because
there's
this
problem
with
access
to
data
and
then
you
don't
have
data,
but
you
have
techniques,
we
could
kind
of
work
on
on
toy
problems,
but
then
it's
not
really
appreciated.
So
it's
kind
of
like.
I
don't
really
know
what
what
solution
there
is
to
this
problem.
Yeah,
sorry
for
rambling
on.
B
E
J
So
I
think
that's
a
very
good
point,
and
I
know
that,
for
example,
for
the
malware
case,
there
are
some
open
data
sets
that
some
lab
so,
for
example,
there's
a
lab
in
prague
that
works
on
malware
and
then
they
release
network
traffic.
But
the
issue
that
I've
seen
is
that
when
we
try
to
publish
with
that
data
set,
the
reviewers
always
come
back
saying.
Well,
the
data
is
too
old.
It's
not
representative
anymore,
and
that
is,
I
think,
a
big
problem
in.
J
B
Right
ask
what
I
I
agree
with
you
understand
the
this
concern.
I
actually
share
a
few
links
data
sets
out
there,
that's
open
that
you
don't
need
to
sign
in,
and
things
like
that,
so
the
reality.
The
problem
you
mentioned.
There
are
two
side
of
stories
right,
so
one
is
that
what
kind
of
data
sets
are
out
there
available-
and
second
is
the
reviewer's
mindset
right,
so
the
I
do
agree.
I
think
that
using
realistic,
up-to-date
data
is
excellent
and
this
particular
apartment
for
cyber
security
is
a
little
bit.
B
More
exaggerated,
like
exaggerated,
is
more
amplified
because
the
subsequent
problems
are
evolving
faster,
so
our
faces
right
face,
recognition,
think
about
that
our
face
doesn't
evolve
every
week
right,
but
our
network,
our
devices,
change
every
week.
Attackers
behavior
change
every
day
right,
so
that
makes
it
a
lot
harder.
So
I
do
think
that
the
community
needs
some
effort
to
think
about
a
pipeline
in
creating
data
set
and
not
necessarily
label.
I
think
labeling
is
going
to
take
a
long
time
to
be
far
behind.
B
So
how
do
we
create
unlabeled
or
semi-labeled
data
set,
allowing
us
to
work
on
variety,
different
problems,
whether
it's
website,
fingerprinting,
whether
it's
the
log
analysis,
whether
it's
mail
analysis,
whether
it
is
whatever
right?
So
there
are
different
kind
of
data.
There
was
a
question
about
anonymization.
B
I
think
that
john
mentioned
this
I
got
commented
on
like
I
said
earlier,
I
believe.
Actually,
if
we
don't
use
a
blank
to
say
no,
this
is
not
possible.
Then
we
can
look
at
different
types
of
data
set
and
how
to
anomalize
would
be
great
about
10
years
ago
there
was
a
study
by
a
group
of
people,
including
professors
in
the
united
university
forgot,
her
name
and
talk
about
the
different
kind
of
substitute
data.
That's
needed.
B
B
I
think
that
we
want
to
move
on
to
the
next
topic
very
quickly.
We
already
went
through
one
hour,
so
I
want
to
touch
upon
the
next
topic
a
little
bit.
This
is
just
the
beginning
of
our
community
discussion,
so
we
have
chance
to
come
back
to
this.
The
topic
we
haven't
talked
a
lot
about
is
how
do
we
engage?
B
How
do
we
educate?
How
do
we
train
whether
students,
practitioners
or
me
as
a
professor
right,
so
I
want
to
be
trained?
I
want
to
be
educated
as
well
in
fact
that
over
the
past
two
years,
I've
been
educated
significantly
by
practitioners
and
thank
you
all
of
those
people
in
omnisoc
and
cacr,
and
some
other
places
that
I've
heard
from
you.
You
really
open
up
my
eyes.
So
I
think
this
this
this.
This
is
a
bi
direction
or
actually
trilateral
educational
opportunities
for
researchers,
practitioners,
students
and
so
on
so
forth.
B
D
D
E
Let
me
ask
a
quick
question:
you
know
in
many
areas
of
machine
learning
and
data
science,
you
will
see
kaggle
challenges,
so
you
put
put
together
a
data
set,
and
you
say
here
is
a
problem
we
are
trying
to
solve
now,
go
for
it.
Different
groups
try
different
things
and
everything
is
open
right.
You
can
see
other
people's
solution
when
they
are
ready
to
submit
it,
and
I'm
wondering
whether
it's
feasible.
E
B
B
That
was
one
of
the
efforts
for
that,
but
I
think
that
if
we
have
some
ongoing
effort,
right,
sustainable
ongoing
effort-
and
maybe
this
is
something
the
working
group
of
big
data
hubs-
can
help
right-
continuous
curation
collecting,
even
small
the
real
world
data
for
for
the
students
for
the
practitioners
for
the
researchers
to
continue
to
tackle,
discovering
new
things,
and,
let's
not
talking
about
label
data,
I
mean
we
could,
but
I
think
they'll
be
very
difficult.
Ask
why
you
raise
your
hand
please.
J
Yeah,
I
just
wanted
to
add
regarding
this
competition,
so
I've
seen
some
conferences
that
have
workshops
that
actually
do
organize
these
kind
of
competitions
and
they
have
a
data
set.
They
have
a
very
specific
kind
of
problem,
so
my
group
actually
took
part
in
an
adversarial
malware
detection
challenge
where
the
goal
was
so
the
framework
was
already
there.
J
It
was
a
deep
learning
based
framework
and
they
had
some
binary
pe
files
and
the
challenge
was
for
attackers
to
craft
adversarial
naval
examples
and
for
defenders
to,
without
basically
looking
at
the
attacker
files
design
a
malware
classifier
that
could
detect
adversarial
malware
examples.
So
basically,
this
was
an
environment
where
which
kind
of
simulated
the
real
world
environment,
where
there
are
attackers
that
are
crafting
malware
samples
without
knowing
what
the
defenders
are
doing
and
vice
versa,
and
that
actually
worked
really
well
and
in
the
competition
results.
J
Then
the
the
organizers
ran
the
submitted
attacks
files
against
the
defender
files,
and
then
there
was
this
this
competition
and
then
a
big
table
that
showed
the
accuracy,
values
and
so
on,
and
the
winners
were
announced.
I
think
it's
a
it's
a
good
opportunity
that
really
promotes
innovative
thinking
and
in
especially
in
in
security,
this
kind
of
stimulates
what
happens
in
the
real
world.
So
I
think
it's
a
good
idea
and
more
people
should
do
it.
B
So
I
I
really
like
a
lot
of
this
discussion.
I
I
think
that
one
of
the
things
we
will
do
at
the
end
is
what
are
the
next
steps,
so
we
can
potentially
list
a
repo
of
data
kind
of
challenges,
right
establishment
challenges
that
for
students,
even
the
past
wise,
they
might
be
posting
some
solutions
and
use
cases
scenarios.
That
would
be
great.
We
can
maintain
some
kind
of
repo
for
that.
For
that
purpose.
B
I'm
also
wondering
have
any
of
you
heard
about
any
of
these
challenges,
where
we
invite
both
practitioners
and
researchers
right,
I'm
not
saying
competing
against
each
other
that
we
found
so
because
there
are
some
very
expertise
like
human
expertise.
I
think
aiml
data
science
are
not
there
yet
at
the
same
time
there
might
be
aim
ml
data
science
approaches.
That's
very
interesting,
think
about
having
practitioners
competing
against
researchers,
I'm
not
sure
whether
those
things
exist
or
not,
and
if
you
know
please
please,
for
yourself.
E
Well,
in
some
of
the
capstone
projects
we
do
for
data
science,
students,
we
do
involve
the
industry,
so
there
is
an
advisor,
for
example.
The
the
first
problem
I
mentioned
there
is
an
advisor
from
cisco
and
obviously
cisco
is
like
in
in
the
middle
of
many
problems.
They
see
a
lot
of
problems
and
their
advice
and
and
the
way
they
sort
of
help
students
understand
the
problem.
Better
is
invaluable.
E
F
So
I
agree
with
everyone:
we
need
competitions
and
public
data
sets.
We
have
a
proposal
to
do
a
hackathon
at
sitcom.
I
don't
know
if
it's
going
to
be
approved,
but
if
it,
if
it
gets
approved,
then
that
would
be
a
good
opportunity
because
we
will
be
bringing
one
of
such
data
set
and
challenging
people
to
develop
to
find
all
the
attacks
in
data
set
in
the
past.
B
So,
on
that
note,
and
one
of
the
things
that
having
competition
and
challenges,
I
think
one
other
thing
to
make
it
really
usable
and
sustainable
is
having
something
recorded
either
the
actual
network
data
right,
as
well
as
that
human
take
humans,
take
on
the
data,
for
example,
in
rit
hosts
the
cptc,
which
is
international.
Now
collegiate
penetration
testing
competition,
I've
been
working
with
organizers
to
collect
the
data
traffic
alerts
along
with
the
competition,
and
I
think
that
ancho
reggae.
B
I
don't
think
that
she's
here
today,
I
had
a
project
with
her
a
couple
years
ago
and
she's
collecting
from
the
human
perspective.
What
do
the
team
members
in
competition?
What
do
they
think?
What's
their
strategy
right
document
that
at
the
same
time,
we
collect
the
raw
traffic
data?
These
allow
a
very
unique
kind
of
data
set
and
every
year
we
can
do
this.
B
Of
course,
the
projects
were
not
wrong
forever,
but
if
we
have
some
sustainable
process
like
this
right,
so
whether
it's
ccdc
cptc
or
some
other
challenges
that
regularly
happen,
and
then
the
organizers
allow
us
to
collect
data
both
from
the
human
side
and
the
network
side.
Right
and
every
year
we
have
renewed
data
right,
read
new
data
and
renew
data.
B
So
in
the
in
the
in
the
group
in
the
community
in
those
who
are
involved
right
now,
I
see
there
are
a
lot
of
people
that
some
of
them
are
recognizing
names
from
practitioners
and
thoughts.
Earlier
we
talked
about
some
other
type
of
challenges,
not
just
network
traffic,
but
also
this
information
and
other
types
of
maybe
phishing
website,
fingerprinting.
B
G
B
G
G
So
we're
recreating
the
project
in
pro
and
python
and
we're
in
this
now
that
the
students
understand
the
problem
now
and
they
understand
the
solution.
Now
they
can
take
it
to
python.
They
can
take
it
to
oracle
and,
what's
so
cool
about
that
is
that
they
know
the
problem
now
and
now
they
they're
just
seeking
out
what
tools
to
use
so
that
that's
my
contribution.
B
B
So
I
think
that
we
are
at
4
12..
I
do
want
to
be
respectful
everyone's
time
we
have
the
last
topic.
Really.
I
think
it's
very
important.
We
have
heard
a
lot
of
very
good
conversations
so
far
from
a
lot
of
people
on
ideas,
challenges,
solutions,
best
practices
and
things
they
could
do.
One
thing
that
we
want
to
do
is
next
step
for
this
community,
so
we
do
not
expect
this
workshop
will
solve
all
the
problems
that
we
have
had
in
the
past
20
30
years
right.
B
B
At
the
same
time,
there
will
be
in
the
google
doc.
There
are
other
bullets,
because
the
poll
is
only
limited
number
of
options,
and
definitely
you
can
share
your
thoughts.
But
more
importantly,
is
that
if
you
have
any
specific
ideas,
please
put
down
some
in
the
bullet
points
in
the
shared
document.
Thank
you.
I
think
that
florence,
you
just
share
again
on
the
google
doc.
Thank
you
very
much,
florence
so
put
down
your
ideas
and
thoughts
that
you
you
think
that
can
be
helpful
and
then
we
can
we'll
compile
it.
B
Don't
worry
about
the
organization
as
much
just
put
down
your
thoughts,
your
ideas,
if
you
do
want
to
help
explicitly
feel
free
to
put
down
your
name,
your
affiliation
and
content
information.
I
would
be
happy
to
reach
out
to
you
to
see
how
we
can
involve
you
directly
yeah.
So
as
the
polls
are
coming
in,
I
will
wait
a
little
bit.
B
We
have
about
30
something
percent
voted
at
the
moment,
but
we
wait
for
a
little
bit
a
few
more
minutes,
meanwhile,
that
I
do
want
to
hear
a
little
bit
more
about
your
thoughts
that
what
we
can
do
I
mean
we
can
actually
really
open
up
the
conversation.
What
you
think
that
we
can
do
specific
anything
you
have
heard
today
that
you
think
will
be
the
best
thing
to
do
immediately.
We
should
do
immediately
whether
it's
for
students
from
practitioners
for
any
researchers,
any
thoughts.
What
we
can
do
immediately.
I
D
I
I
have
a
whole
platform
right.
I've
got
cloudera
data
platform,
I've
got
ways
to
share
data.
I've
got
ways
to
catalog
data.
I've
got
big
data.
I've
got
machine,
learning
tools,
I've
got
python,
I've
got
scala.
I
get
like
all
the
tools
that
that
you
guys
need,
but
I
need
the
brains.
You
know
they
need
the
brains
of
the
operation
and
my
customers
are
super
interested
in
ai
and
a
lot
of
people
are
talking
about
ai
in
cyber
security.
But
it's
just
not
it's
really
unattainable
for
them.
E
E
Part
of
the
disconnect
that
we
heard
about
between
academia
and
in
the
practices
is,
you
know
very
often,
academia
will
never
get
access
to
the
kind
of
data
that
the
industry
has.
Clearly,
that
is
understandable,
but
is
it
possible
to
look
at
the
characterizations
of
those
data
and
to
create
synthetic
data?
Sets
can
be?
Can
that
be
a
kind
of
collaboration
that
can
happen
between
the
two
groups?.
B
F
So
I
think
we
had
a
lot
of
great
discussion.
There's
a
question
in
the
chat.
I
want
to
elevate
from
a
high
school
student
looking
for
suggestions
to,
I
think
about
materials
he's
just
getting
into
cyber
security
and
data
science.
So
if
people
want
to
give
some
suggestions
yeah,
I
thought
it
was
great
discussion.
It
sounds
like
it's
a
just
start
of
a
of
a
long
and
wonderful
discussion
for
us
all
to
have
as
a
community.
B
B
Meanwhile,
we'll
go
to
kathy
and
sagar
as
well,
but
I
want
to
share
with
you
a
little
bit
about
the
poll
result.
You
probably
can
see
yourself
now.
I
think
that
we
have
a
lot
overwhelming
more
than
half
60.
Something
percent
want
to
have
sheer
repositories,
share
repository
things
like
that
excellent.
Some.
B
Some
kind
of
expertise,
an
interest
catalog
for
people
right
to
find
collaborators.
I
think
that's
great
so
we'll
work
on
this
we'll
take
this
into
input
and
see
what
we
can
do
as
a
community
and
if
you
want
to
volunteer
to
help
certain
things.
I
think
that
would
be
excellent
as
well.
Florence.
You
just
made
a
comment
about
caroline
and
everyone
to
the
may
7
discussion.
I
think
now
that
was
the
plan
right.
G
Good
job,
the
other
idea
that
sounds
great
and
jay.
The
other
idea
we've
talked
about
a
little
bit
is:
if
part
of
this
group
would
like
to
work
together
to
submit
either
to
perk
or
sc,
either
a
panel
or
a
buff.
You
know
I'm
wondering
how
we
can
get
other
people
in
the
boat
with
us
if
we
think
that's
valuable
or
maybe
we're
not
ready,
but
that's
another
opportunity.
B
Right
so
I
think
that
definitely
there
are
a
lot
of
venues.
We
can
definitely
try,
like
you,
said,
perk
and
other
things,
and
I
think
sagar
also
has
something
that
he
wants
to
share
there
coming
up
as
well.
So
definitely
florence,
I
absolutely
agree
with
you.
I
think,
if
you
have
in
the
audience,
you
have
any
thoughts
on
specific
venues.
You
think
it
will
be
best
for
this
company
continue
to
have
the
conversation,
please
feel
free
to
put
them
down.
B
F
C
Yeah,
so
I
I
just
put
in
the
chat
link
to
the
editorial
that
I
was
talking
to
a
little
bit
earlier
about
a
lot
of
the
issues
that
we
talked
about
today
in
terms
of
gaps
between
practitioners
and
academics,
the
different
types
of
data
sources,
some
of
the
key
issues
related
to
data
access
representation
and
so
on
and
so
forth
in
there.
But
I
think
jay.
If
you
don't
mind
I'll,
I
can
go
ahead
and
share
some
of
those
other
opportunities
that
we
had
talked
about
as
well.
C
I
understand
that,
there's
more
than
you
know,
just
the
u.s
that's
represented
here,
but
these
programs
have
funded
topics
related
to
data
science
for
cyber
security
or
have
done
things,
and
I
just
want
to
kind
of
summarize
some
of
the
high
points
of
this
and
and
in
the
editorial
there's
a
more
comprehensive
summary
of
them,
but
there's
all
the
way
from
the
early
career
stage
to
infrastructure,
developing
infrastructure
around
this,
and
so,
for
example,
size.
Community
research
infrastructure
has
supported
various
cyber
security
related
infrastructures
out
there.
C
Core
research
related
topics
such
as
securing
trustworthy
cyberspace
and
cyber
security,
innovation
for
cyber
infrastructure,
to
transition
to
practice
and
education
as
well.
So
there
are
a
few
amount
of.
There
are
a
few
opportunities
that
out
there
that
are
actively
funding
topics
related
to
what
we
had
talked
about
today.
That
would
also
be
potential
suitable
outlets
as
well
that
that
could
be
searched
for
as
well
and
then
just
to
put
a
little
bit
of
a
plug
in.
C
We
have
a
couple
of
upcoming
events
related
to
this
as
well
more
from
the
academic
side,
but
we
have
practitioners
on
the
program
committees
related
to
this,
as
well
so
at
kdd
and
ieee
icdm,
and
we
see
our
our
leader
here
as
one
of
the
co-founders
of
the
workshop.
So
so
we
have,
you
know,
jay
and
and
dr
chen
as
part
of
the
kdd
workshop,
and
then
also
we
have
the
icdm
workshop,
that's
related
to
more
more
academic
related
work.