►
Description
Our meeting on Sep 26th, 2020 was about Machine Learning in Clojure.
Background:
https://clojurians.zulipchat.com/#narrow/stream/151924-data-science/topic/ml.20meeting.20-.20Sep.2026th
Agenda:
1:39 Machine learning using tech.ml and Fastmath
25:03 Kiran Karkera: using Weka through clj-ml
45:07 Anthony Khong: using Spark and Spark ML through Geni
1:00:18 Questions and discussion
Karkera's presentation:
https://docs.google.com/presentation/d/1jZFwjw5DiW77DqV5EHaRlVnGA3A_pWnJkQpKEgWEivk/edit
Anthony Khong's demo:
https://github.com/zero-one-group/geni-repl-demo
Daniel Slutsky's tutorial:
coming soon.
Main projects discussed:
- https://github.com/techascent/tech.ml
- https://github.com/generateme/fastmath
- https://github.com/shark8me/clj-ml
- https://github.com/zero-one-group/geni
Other projects mentioned:
- https://gtoolkit.com
- https://github.com/scicloj/notespace
- https://github.com/metasoarous/oz
- http://gorilla-repl.org/
- https://pink-gorilla.github.io/
A
So
welcome
everybody.
This
is
the
september
2020
meeting
of
the
cyclog
group.
We
are
interested
in
the
mix
of
data
science
and
closure,
and
today
we
have
a
very
interesting
meeting
and
we
will
first
start
with
daniel.
We
will
discuss
the
state
of
the
arts
of
the
machine
learning
ecosystem
enclosure.
A
We
will
discuss
how
to
make
spark
interact
with
closure
and
finally,
we
will
have
some
discussion
afterwards.
Those
who
wish
to
stay
can
stay
after
the
official
time.
We
have
a
time
we
have
a
90
minute
limit
for
the
whole
event.
That
includes
every
every
top
by
speakers,
which
we
should
last
15
minutes.
B
B
Yes,
yeah,
wonderful,
okay.
So
what
I
wish
to
do
in
the
following
few
minutes
is
share
my
user
experience
with
the
current
stack
of
machine
learning,
libraries,
enclosure
and
maybe
ask
some
questions,
and
I
guess
maybe,
since
since
we
will
have
three
talks
about
different
aspects
of
machine
learning
today,
it
may
be
good
to
to
somehow
say
what
it
is
about.
B
B
B
That
knows
how
to
predict,
and
then
you
can
ask
that
model
what
the
predictions
are
for
the
training
data
so
that
you
can
measure
how
good
it
is
with
the
training
data.
So
that
is
very
schematic.
That
is
very
high
level,
so
the
code
and
the
idea
about
this
pipeline
is
that
we
can
think
about
the
model
as
everything
we
discover
through
the
process,
including
the
the
parameters
we
chose,
including
certain
statistics
in
the
intermediate
steps,
including
everything.
B
But
this
time
everything
we
do,
we
do
in
the
context
of
already
knowing
the
model.
So,
for
example,
if
at
the
pre-processing
stage
we
were
doing
some,
we
were
computing
some
statistics
over
the
data,
then
this
time
in
the
test
data.
These
statistics
are
already
available
to
us.
So
we
do
everything
in
the
context
of
knowing
the
model,
knowing
everything
that
has
gone,
that
has
been
going
on
in
the
training
phase,
and
then
we
can
predict
them,
get
the
model
predictions
for
the
test
data.
B
Can
we
have
all
this
process,
and
can
we
have
it
with
a
beautiful
grammar
with
good
performance,
and
we
with
all
the
features
that
we
may
like
to
have
like
having
it
really
easy
for
beginners
and
big
being
able
to
analyze
the
performance
of
everything
and
check
many
variations
and
optimize
over
parameters
and
also
serialize
any
object
that
we
create
and
deploy
everything
easily?
Can
we
have
all
that
and
the
tecml
library
that
we
have
learned
about
in
previous
meetings?
It
act?
B
These
are
smile
smile
by
hi
friendly,
which
has
lots
of
machine
learning,
algorithms
and
actually
xgboost
2,
which
is
a
separate
machine,
learning
library
and
fast
math
by
who
has
lots
of
mathematics
and
machine
learning
and
clustering,
algorithms
that
we'll
be
using
and
and
also
all
the
tech
stack
of
data
type
obstructions
and
data
frames
and
tablecloth,
which
is
a
nice
grammar
around
the
tech
stack.
I
guess
most
of
us
already
know
these
libraries,
but
just
had
to
mention
that
these
are
being
used
in
the
current
example.
B
So
we
usually
begin
by
getting
the
data
and
splitting
it
to
certain
training
data
and
test
data,
as
we
mentioned
before,
and
then
we
can
look
at
the
data
and
explore
it
and
see
if
there
are
any
problems
and
I'm
skipping
most
of
these
details,
since
most
of
them
are
not
so
much
surprising
and
then
eventually
what
we
have
by
the
tecml
philosophy,
which
is
the
philosophy
of
the
tecml
library.
We
have
a
certain
pipeline
where
the
data
set
goes
through
a
long
process
of
data
transformations.
B
B
So
this
clustering
is
done
on
the
training
step,
so
we
cluster
the
all
the
location
in
in
the
city
into
places
and
in
the
test
step
in
the
test
stage.
We
already
have
these
clustering
information.
We
do
not
want
to
do
the
clustering
again,
so
this
is
done
through
metadata
and
it
is
actually
a
really
neat
idea.
I
think,
of
the
tech
ml,
maybe
the
central
idea-
and
here
I
I
did
it
in
a
kind
of
in
a
explicit
way,
which
is
a
little
bit
verbose.
B
Maybe
adding
the
clusters
to
the
metadata,
but
tecml
provides
a
certain
set
of
macros
that
make
it
makes
it
much
less
robust.
I
just
didn't
want
to
include
them
for
clarity
and
and
then
we
can
have
a
processed
training
set
and
a
process
data
set,
and
when
we
create
deposit
data
set,
we
do
it
with
the
context
of
things
we
that
we
have
learned
in
the
training
set.
For
example,
we
look
at
the
location
of
the
pickup,
where
the
taxi
picks
up
the
the
passenger,
and
then
we
cluster
it.
B
So
the
the
certain
cluster
which
is
which
matches
kind
of
a
neighborhood
in
the
city
it
is
being
able
to
recognize
it
is
done,
is
something
that
we
learned
in
the
training
data,
and
so
we
process
the
test
data
using
this
existing
context
and
and
then
it's
really
fun
actually
to
use
both
techml
and
fast
math,
the
library
and
the
techviz
library
that
allows
for
these
neat
histograms
to
look
into
the
data
to
look
into
the
process.
Data
and,
for
example,
here
is
one
column
that
we
were
created
in
the
preprocessing.
B
It
says,
based
on
your
location
around
what
speed
is
expected
for
the
trip
for
the
taxi
trip.
So
you
see,
there
is
some
pattern,
visual
pattern,
and
we
see
that
this
pattern
is
actually
quite
similar
in
the
testing
set
and
trace
set,
which
is
which
makes
sense
because
they
were
sampled
from
the
same
data
and
then
eventually
we
can
train
a
model
and
for
this
model,
what
we
do.
I
just
sampled.
B
Some
part
of
the
training
set
to
have
it
quicker
and
then
I
use
extreme
boost,
and
you
see,
the
grammar
of
tacoma
is
really
nice
in
these
details,
and
it
is
really
actually
much
cleaner
than
the
python
story
that
I
tried
to
mimic
and
then
we
can
predict
using
this
model.
We
get
predictions
and
we
can
see
how
bad
these
predictions
are
compared
to
the
actual
data
which
is
in
this
case.
B
You
know
the
the
median
average
arrow
is
the
median
absolute
arrow
is
about
300
seconds,
which
is
not
so
good
but
relatively
to
the
durations
of
trips.
It
is
decent
and
we
see
that
it
is
similar
in
testing
and
in
training,
which
means
that
we
don't
have
the
overfitting
so
to
speak,
and
then
we
can
scatter
the
actually
predicted
trip
durations
in
the
test
data
and
see
how
they
compare
to
the
actual
triplications.
And
we
see
that
there
is
some
connection.
B
So
this
was
some
flow,
and
that
is
how
the
flow
looks,
and
I
just
wanted
to
to
give
the
taste
of
it
and
and
just
say
that
I'm
amazed
by
by
what
the
already
is
in
the
in
the
stack
and
what
I
learned
from
chris
in
the
last
few
days
is
that
maybe
the
main
idea
here
is
being
able
to
to
have
a
certain
notion
of
pipeline
like
having
everything
as
a
sequence
of
operations.
And
you
see
some
of
these
operations
are
using
the
dsl
of
the
camel
and
others
are
using.
B
The
tablecloth
library
which
has
a
nice
grammar
for
data
processing
and
this
this
pipeline
has
this
magic.
That
knows
to
behave
differently
in
training
and
testing,
and
it
keeps
the
context
of
training
so
that
everything
flows
and
this
I
think,
matters
a
lot
having
a
clean
way
to
express
ourselves
matters
a
lot,
because
in
machine
learning
in
machine
learning,
you
can
have
really
subtle
errors,
really
subtle
ways
to
to
be
confused
and
to
have
a
mistake,
a
mistake
that
makes
you
believe
you
did
a
good
process.
B
A
C
D
D
B
Oh
yeah,
thank
you
for
mentioning
that.
So
here
we
were
not
using
any
python
libraries
and
we
were
using
mostly
the
jvm,
except
for
some
small
parts
which
use
native
libraries
behind
the
scenes,
but
it
was
mostly
a
jvm
story
of
java
libraries,
which
are
wrapped
in
closure
and
maybe
I'll
mention
because
you
you
mentioned
that
we
are
currently
thinking
about
creating
a
certain
learning
group
where
we
learn
together
and
that
follows
actually
your
comments
in
the
previous
meeting
that
we
wish
to
learn
together.
D
D
B
Yeah,
at
least
that
is
how
it
felt-
and
I
haven't
mentioned
fast,
mass
math
so
much
in
this
process,
but
behind
the
scenes.
Lots
of
mathematical
computations,
like
the
clustering
and
other
parts,
will
being
done
by
fast
math,
which
is
just
a
huge
collection
of
neat
closure,
wrappers
for
existing
java
libraries
for
math
and
statistics
and
machine
learning
and
yeah,
and
it
just
works.
E
Yeah,
I
had
a
question
daniel
great
presentation,
so
my
question
is
really
from
your
presentation:
it's
not
obvious
how
many
different
libraries
you
used
for
the
data
pipeline
and
you
mentioned
a
couple
of
them
right.
So
obviously
you
felt
the
need
to
use
multiple
of
them
because
one
of
them
does
not.
You
know,
solve
everything.
So
perhaps
it's
useful
to
have
a
high
level
guideline
on
what
are
the
capabilities
of
of
say,
library,
one
versus
two,
and
where
do
you
need
to
use
both?
E
Because
you
know
when
a
beginner
approaches
this
they're
they're?
Looking
sometimes
for
the
shortcut,
you
know
just
import,
one
library
and
everything
works
and
it's
it
becomes
a
bit
unclear
when
they
need
to
use
something
else.
So
it's
not
a
comment
on
your
presentation
itself,
but
it
would
help
a
beginner
if
you
can
guide
them
on
what
library
does
what
well,
so
that
they
can
pick
one.
B
Yeah,
so
maybe
I'll
just
for
a
moment
share
my
screen
again
and
let
us
look
into
it
if,
if
it
is
okay,
landlord
yeah
thanks.
So
so
let
us
look
at
the
collection
of
libraries
that
I
have
been
looking
into
and,
of
course
a
yukidan
will
add.
Another
part
of
the
story
which
is
not
here
and
and
anthony
will
add
another
one
so
yeah.
B
For
me,
the
main
part
of
the
everyday
libraries
that
I'm
using
and
so
smile
is
just
a
collection
of
algorithms
by
heifen
glee,
both
supervised
and
unsupervised,
machine
learning
and
statistics
and
other
mathematical,
algorithms
and
fast
math,
is
in
a
sense
similar.
It
is
just
a
huge
collection
of
mathematical
algorithms
and
both
have
a
closure
api.
B
That
is
having
a
certain
notion
of
array,
even
though
it
is
called
a
bit
differently
in
tech
data
type
and
being
able
to
work
with
them
and
connect
different
libraries
that
are
that
are
speaking
in
terms
of
arrays
of
data,
and
I
think
the
fact
that
things
connect
well
and
can
be
glued
together
nicely.
For
example,
that's
the
technical
data
frames
can
be
translated
comfortably
to
smile
data
frames
so
that
we
can
use
smile
with
decamel.
B
All
these
details
are
there
mainly
thanks
to
tech
data
type,
which
is
the
fundamental
part.
The
way
I
understand
it
and
take
ml
data
set
is
a
really
detailed
library
and
a
really
performant
library
for
data
frames
that
that
is
tables
of
data
and
tablecloth
is
a
nice
wrapper
around
it.
So
you
can
use
two
different
grammars,
the
one
of
the
kml
dataset
and
the
one
of
growth,
a
matter
of
of
taste.
B
I
guess-
and
you
can
express
yourself
in
two
different
ways
if
it
makes
sense
and
takamel
is
just
about
about
gluing
this
into
a
machine
learning
context
and
most
of
the
details
of
the
pipelines
we
have
been
seeing
actually
come
from
here
for
from
second
ml
data
set,
but
tecml
does
put
it
in
the
context
of
machine
learning
and
chris.
The
author
is
here
in
the
audience,
and
maybe
he
would
like
to
add
more
about
about
this
context,
and
I
hope
it
helps.
A
A
E
Okay,
so
today
I'm
going
to
talk
about
weka
and
cljml
and
it's
a
slightly
different
take
on
the
on
the
machine
learning
ecosystem.
I
hope
you
enjoy
it
about
me.
I
go
on
github
as
shocked
me
and
I've
been
programming
for
a
long
time.
I've
been
involved
with
closures
since
the
past
seven
years.
A
lot
of
it
has
been
in
the
data
science
space.
E
I
work
from
singapore
and
in
the
past
I
have
done
small
contributions
to
data
science,
libraries,
enclosure
ones
that
I
remember,
are
co-dot
matrix
sparkling,
which
was
one
of
I
know.
There
are
three
rappers
of
of
spark
enclosure
sparkling
was
one
of
them,
and
I
worked
a
bit
with
cortex,
which
is
tons
of
contributions
to
cortex,
which
is
a
machine,
a
neural
network
library
that
chris
was
leading
a
couple
of
years
back.
E
So
my
goals
today
are
to
talk
about.
You
know
first
of
all
when
and
why
should
you
consider
using
weka
and
cljml,
I
think,
from
a
practitioner's
perspective,
what
happens
when
you
look
at
any
language
or
any
ecosystem
is
that
there
are
many
ways
of
doing
something,
and
it's
not
very
clear
why
you
should
use
one
versus
the
other,
and
that's
going
to
be
the
main
crux
of
my
discussion
today,
then
I'm
going
to
do
a
very
quick
dive
into
clj
ml
itself
and
then
talk
about
suggested
usage
patterns.
E
E
You
know,
have
a
great
deal
of
choice
in
terms
of
the
tooling,
so
you
might
be
working
with
somebody
who's
good
in
python,
who's
good
in
r
and
who
can
do
a
lot
of
things.
So
there's
a
certain
amount
of
pressure
in
terms
of
justifying
why
you're
using
closure
and
any
other
machine
learning
libraries
right.
So
this
is
I'm
talking
in
production,
of
course,
when
not
when
you're
say
just
trying
out
a
library.
E
So
what
inevitably
happens
is
that
you
will
have
somebody
who
says
you
know
why
don't
you
use
python
or
r
or
azure
ml,
or
something
like
that,
and
this
has
been
my
experience.
Of
course
it
may
not
reflect
with
all
of
you
and
having
worked
in
such
projects
in
the
past,
I
have
seen
that
the
visual
tools
that
exist
for
you
for
machine
learning
have
made
tremendous
progress
in
the
past.
E
So
I
struggle
with
you
know
what
should
I
use
sometimes-
and
I
reflect
on
this
thing-
that
larry
wall
said,
which
is
that
you
know
this
is
a
long
quote
which
basically
was
meant
to
say
that
pearl
is
meant
to
do,
make
easy
things
easy
and
difficult
things
possible,
and
sometimes
you
know
the
gui
tools
versus
the
core
tools.
They
offer
very
diametrically
different
options,
so
the
gui
tools
are
good
for
some
things:
they're
good
for
quick
data
exploration,
they're
good
for
building
and
evaluating
models.
E
Some
of
them
can
even
do
deployment.
I
think
amazon
sagemaker
comes
to
mind.
On
the
other
hand,
the
the
code
based
tools
are
very
good.
At
data
cleaning
and
building
pipelines,
data
cleaning
can
be
done
by
gui
tools,
but
it
is
somewhat
painful
like
you,
don't
you
can't
do
it
as
well,
and
if
there's
something
missing
in
in
that
library,
you
really
have
no
way
around
it.
E
E
E
E
I
I
think
there
are
three
sort
of
dimensions
that
you
want
to
consider,
and
one
dimension
is
role
where,
if
you're
in
a
project
where
you
want
to
do
machine
learning
and
the
role
that
you
have
to
handle
is
really
the
full
stack
machine
learning
engineer
where
you
you're
doing
all
the
data
extraction,
data
acquisition
say
from
kafka
and
database
and
any
number
of
sources
you're
building
a
data
set.
You
spend
some
time
looking
at
the
data,
then
you
train
a
model
and
then
you
you
have
to
deploy
it
as
well.
E
When
some
teams
data
scientist
gets
to
get
very
lucky,
they
get
a
very
clean
data
set,
so
they
just
have
to
do
model
tuning
and
that's
all
they
do,
and
this
is
of
course
it
could
be
true
for
domains
very
specialized
domains
such
as
audio
and
video,
where
indeed,
the
data
is
what
it
is
and
there's
not
much
more
processing.
It's
just
you
know,
improve
the
model
by
a
little
bit
is
what
what's
done.
However,
in
most
projects,
I've
worked
in
it's
low
dimension
data,
and
that's
not
the
case.
E
The
second
is,
you
should
be
comfortable
with
using
multiple
facets
of
these
tools,
so
you
should
be
comfortable
with
using
a
gui
or
a
cli
or
a
library
based
on
what's
needed,
and
third,
of
course,
because
you're
a
closure
programmer.
You
want
to
build
a
data
pipeline
enclosure
and
in
terms
of
data,
the
sweet
spot
is
the
data
should
be
low
dimension
data,
so
not
images
not
audio,
not
video,
clj,
clg
ml
and
weka.
They
work
best
with
non.
E
You
know
traditional
models,
that's
non-neural
network
models;
they
do
have
support
for
perceptrons
and
integrations
with
deep
learning
fuji.
However,
that's
also
done
in
extension
and
for
most
models.
The
data
should
fit
in
core
memory,
there's
a
star
because
you
have
a
couple
of
ways
around
it,
which
involves
some
hacking.
You
can
use
memory
map
files
like
you
know
the
arrow
work
that
chris
has
been
doing
and
you
have
certain
classifiers
called
incremental
classifiers,
where
you
can
save
and
load
and
retrain.
E
You
know
even
after
you
know,
serializing
the
model
specifically,
and
the
third
aspect
that
I
just
got
to
know
of
today
is
that
weka
is
gpl
licensed.
So
if
it's
a
product,
you're
building
and
you
have
to
ship
it,
you
might
want
to
consider
that
becca's
gpl
and
therefore,
can
you
really
sh.
You
know
how
does
it
contaminate
the
license?
The
rest
of
your
product
is
something
that
you
should
know
about.
I'm
not
qualified
to
comment
on
that.
E
So,
let's
look
at
weka
now
and
what
it
is
so
weka
is
a
machine
learning
toolkit
written
in
java.
It's
one
of
the
oldest.
It's
development
started
actually
in
1997.
That
makes
it
more
than
20
years
old
now
and
it
is
still
actively
maintained.
It's
gone
through
some
ownership
changes.
It
has
two
two
branches:
a
stable
and
a
development
branch.
The
stable
branch
was
actually
released
in
last
year
and
the
active
branch
is
actually
actively
developed.
E
It
has
interrupt
with
r
python
and
spark,
and
it
also
has
a
a
very
active
extension
community.
So
as
an
extension,
you
can,
you
know,
develop
a
new
model,
a
new
algorithm
and
provide
it
in
a
specific
api
which
which
can
be
loaded
into
weka
and
used
and
in
terms
of
resources
to
learn.
There
are
at
least
three
online
courses
that
you
can
use
to
learn,
weka
and
there's
a
book.
There
are
lots
of
forums
as
a
side.
E
You
know
you
might
observe
that
the
book
is
called
data
mining
which
is
sort
of
this
old
school
term
for
machine
learning
which
somehow
many
people
don't
use
nowadays,
but
that's
how
old
it
is,
and
if
you
actually
start
to
learn
about
weka,
you
would
see
that
a
lot
of
the
messaging
is
around
the
visual
interface,
which
is
which
can
probably
put
you
off.
You
might
think
that
it's
only
a
visual
tool,
but
that's
not
the
case.
E
It's
got
three
interfaces,
a
gui,
cli
and,
of
course,
either
java
or
closure
libraries,
and
I
find
it
quite
useful
because
these
interfaces
support
three
different
audiences.
The
gui
supports
an
experimentation
kind
of
audience.
The
library
supports
a
data
pipeline
development
and
the
cli
is
really
useful
for
devops.
So
if
you've
got
a
trained
model-
and
you
just
want
to
orchestrate
it
using
docker,
you
know
you
don't
really
have
to
build
your
own
little
wrapper
on
top
of
a
cli
wrapper.
On
top
of
that,
how
much
time
do
I
have
left
may
ask.
E
E
You
have
a
second
stack
for
feature,
selection
and
data
set
and
feature
transformation.
To
be
honest,
I
don't
find
the
feature
transformation,
one
very
useful,
because
you
can
do
a
lot
of
that
enclosure
in
vanilla
closure
and
three.
You
have
a
bunch
of
algorithms
classification,
regression,
clustering,
pca
kind
of
stuff.
So
if
you
look
at
the
list
of
supported
classifiers
and
regressors,
it's
really
it's
a
lot.
And
frankly,
this
is
a
small
subset
of
what
is
actually
present
in
weka
itself
and
it's
really
straightforward
to
add
new
classifiers.
E
So
those
of
you
who
are
you
know
very
much
into
machine
learning-
might
observe
that
yeah
I
see
xg
boost
missing,
which
is
a
fair
observation.
Exit
boost
is
available
as
an
add-on
package.
It's
not
doesn't
yet
exist
in
clj
ml
itself,
but
it's
really.
You
know.
5-10
lines
of
code
to
to
put
it
into
cljml,
if
you
absolutely
need
it
so
I'll
jump
through
what
I
call
use
it
scenarios,
which
is
you
know,
a
textbook
usage
of
how
somebody
would
want
to
use
cldml
and
real
world
usage.
E
So
in
terms
of
classification,
I've
really
just
copied
this
from
the
readme
and
it
is
really
straightforward
to
use.
You
can
load
data
from
a
classifier
from
a
file
csv
arff.
Whatever
format
you
need,
you
indicate,
which
is
the
target
variable
in
this
case.
There's
one
column
named
class
and
then
you
can
click
create
a
classifier
by
passing
arguments.
In
this
case
you
can
say
decision
tree
and
c45
is
one
variation
of
a
dish's
decision
tree.
E
You
have
a
few
more
of
them
and
you
train
it
and
you
can
use
the
same
trained
classifier
to
basically
classify
an
instance
fairly
straightforward.
You
pass
the
classifier
and
you
pass
an
instance
a
single
instance,
and
you
get
in
this
case.
We're
using
the
iris
data
set,
you
get
a
predicted
class
for
it,
and
evaluation
is
again
fairly
straightforward.
E
You
pass
a
classifier
object
and
a
data
set,
and
you
specify
how
many
fold
cross
validation
you
want
to
do
and
you
it
will
return
an
evaluation
object
which
contains
a
bunch
of
statistics
which
are
relevant
for
that
task.
So
if
it's
regression,
you
might
not
get
correctly
classified
instances,
you'll
get
something
else.
E
What
I
find
useful
is
to
think
about
how
you
would
use
this
in
a
real
project
and
looking
at
looking
back
at
my
efforts,
I
found
this
is
what
I
do
so.
I
generally
start
by
creating
a
data
acquisition
namespace
which
acquires
data
from
multiple
sources
such
as
you
know,
kafka,
database
stream,
sometimes
files
sometimes,
and
I
create
a
data
set
or
an
arff
for
a
csv
data
set,
and
I
constantly
transition
between
the
closure
emacs,
for
example,
and
the
weka
gui.
So
I
would
do
the
data
exploration
in
weka.
E
I'd
look
at
say,
correlations
between
features
in
weka.
I
look
at
which
features
are
important,
so
weka
allows
you
to
basically
look
at
a
data
set
and
say
rank
the
features
in
terms
of
importance,
and
there
are
12
different
ways
in
which
you
can
do
that,
and
then
I
might
jump
back
to
actually
create
a
data
set
with
more
features.
And,
of
course
all
of
this
is
iterative
and
at
some
point
I
might
use
the
weka
experimenter.
E
E
You
can
also
use
it
in
lieu
of
auto
ml
I'll
just
address
that
and
finally,
from
a
project
perspective
once
I'm
I'm
familiar
with
which
classifiers
are
doing
well
and
what
what
is
the
data
set?
That's
working
well
I'll
jump
back
to
model
training
and
fine
tuning
in
closure
itself,
and
gradually
I
will
stop
using
the
gui,
so
the
experimenter
for
those
who
have
not
seen
it
is
a
gui
tool
where
you
can
provide
multiple
data
sets
here.
There's
just
one
and
you
can
test
it
on
multiple
algorithms.
E
A
Very
much
kiran
for
your
time
since
your
talk
is
very
similar
to
anthony's,
and
since
you
took
a
little
bit
longer
than
was
expected,
I
think
he
will
be
also
very
useful
to
have
the
questions
related
to
this
talk
after
we
hear
anthony's,
so
we
can
compare
both
libraries
and
both
closure,
wrappers
sure.
As
I
said
now,
we
will
hear
anthony
who
will
talk
about
how
to
use
spark
and
spark
ml
through
guinea
anthony.
F
Sure,
let
me
show
my
face
hi
everyone.
Let
me
share
my
screen
if
I
could
okay,
so
I
thought
today
what
I'm
going
to
do
is
share
with
you
how
I
work
with
guinea
on
a
daily
basis.
I
spoke
about
it
in
the
last
meeting
where,
basically,
in
summary,
it's
a
it's
a
closure
data
frame
library
that
runs
on
spark
right.
It's
a
spark
wrapper,
but
it
has
its
own
grammar
kind
of
thing.
F
So
today,
I'd
like
to
do
a
live
demo,
which
is
hopefully
nothing
goes
wrong.
Fingers
crossed
and
the
objective
today
is
to
show
you
the
kind
of
fast
rapper
feedback
that
you
can
get
with
guinea
right.
It's
the
important
thing
is
not
so
much
what
we
do
with
the
data
in
this
session,
but
note
the
number
of
evals
and
the
the
query
speed
that
that
comes
with
it
right.
F
That's
that's
what
I
want
to
show,
and
secondly,
I
want
to
sort
of
dispel
this
myth.
That
spark
is
only
for
big
data
right
because
it's,
I
think,
even
for
a
lot
of
power
users
and
spark
they.
They
say
this,
but
really,
I
think
it's
amazing
for
small
data
too
so
caveat.
Don't
take
any
of
this
seriously.
Please
I'm
going
to
be
using
some
free
data
on
kaggle
and
yeah.
F
What
that's
going
to
do
is
going
to
step
you
inside
a
closure
rebel
and
it's
going
to
require
all
the
guinea
stuff
for
you
and
basically,
you
can
just
connect
via
the
nrepo
through
your
text
editor.
So
on
kiran's
note
you
know
about
requiring
multiple
libraries.
This
is
the
ultimate
shortcut
right.
You
you're
going
to
require
zero
libraries
with
this.
So
you
know
you
can
play
around
with
this.
That's
pretty
much
what
you
expect,
but
I'm
going
to
connect
to
the
n
repo
session.
F
With
the
text
editor,
I
use
new
vim
and
conjure,
so
I
just
like
having
the
the
the
repli
log
on
the
right
hand,
side
right
and
I'm
going
to
start
loading
the
data
again
no
requires
it's
been
required
for
you
and
just
doing
that.
That's
the
the
spark
data
set
right
and
you
can
pretty
much
do
whatever
you
you.
F
You
expect
a
data
frame
library
to
be
able
to
do
count
is
the
number
of
rows
right
shape
as
number
of
rows
and
columns
print
schema
is
you'll,
see
the
columns
and
also
the
the
the
types
and
whether
or
not
they're
nullable,
and
if
you've
been
using
spark.
This
would
look
very
familiar
to
you
right,
but
since
we're
in
closure,
I
thought
you
know
it'd
be
nice
to
work
with
columns
that
are
in
kebab
cases
right.
F
So
we
can
do
that
with
kebab
columns
and
they'll
just
rename
the
the
the
columns
for
you.
We
can
have
a
look
at
the
data,
but
if
we
do
this,
you
can
see
that
it's
the
output
is
a
bit
truncated.
So
that's
not
very
nice,
so
we
can
do
vertical
extent
right
and
we'll
be
able
to
see
the
data
more
clearly.
So
it's
only
showing
20
rows
and
we
see
that
invoice
stock
quote
they're.
F
Just
numbers
and
description
is
just
a
bunch
of
words
and
the
other
stuff
pretty
much
what
you
expect.
So
I
think
I'm
done
with
that.
I'm
gonna
call
that
invoices
all
right,
I'm
just
gonna,
evaluate
that
and
I'm
going
to
do
some
exploratory
data
analysis
right
and
the
first
thing
whenever
I
get
a
new
data
set.
First
thing
I
do
is:
I
want
to
check
whether
or
not
any
of
the
columns
represent
like
the
primary
key
right
like
there's
a
unique
value
for
that
column.
F
F
So
we
said
that
we'll
we
had
like
a
million
rows,
but
then
this
is
only
50
000
right.
So
that's
not
quite
that
that
column
right,
but
what
we
can
do
is.
We
can
also
count
this
thing
stock
code-
maybe
that's
the
one
so
there's
only
five
thousand,
so
that's
not
really
the
one
and
we
probably
don't
wanna
type.
All
the
columns
in
so
one
thing
we
can
do
is
just
map
g
columns
of
inverses,
something
we
can
do
so.
F
So
if
we
do,
invoices
count
we'll
have
a
million
sixty
thousand
things
and
but
then,
if
we
do
distinct
count,
we'll
have
a
million
and
thirty
thousand
things.
So
we
can
just
go
back
up
and
see,
let's
just
drop
duplicates,
and
I
think
we
can.
We
will
say
that
that
that's
okay,
so
next
I
want
to
look
at
this
column
description
right.
The
description
column
looks
interesting.
F
So
that's
just
a
shortcut
to
tell
you
hey
that
that's
like
a
white
hanging
heart
tea
light
holder
appears
5740
times.
So
I
see
that
on
the
third
entry
there's
now,
so
I'm
just
going
to
remove
that
again.
Closure
idiom,
remove
null
description
and
we'll
eval
again
and
see.
Okay,
so
that's
that's
been
taken
care
of,
but
maybe
I
don't
want
to
look
at
the
description
right.
I
want
to
look
at
the
descriptors,
like
the
actual
words
that
make
up
the
description.
F
Now,
instead
of
having
every
row
being
an
array
of
descriptors,
I
want
every
row
to
be
an
element
of
the
descriptor
right
and
so
that's
the
explode
operation.
So
we
can
say
with
column
descriptor,
which
is
the
explode
of
descriptors,
so
we
can
do
that.
So
if
I
get
rid
of
this
and
show
you
the
count,
it
should
tell
me
a
million
something,
but
then
once
I
exploded,
we
should
get
like
4
million
something
so
that
every
row
in
on
average
has
like
four
or
five
descriptor
words.
F
And
now
that's
that's
been
exploded
and
I
can
go
back
again
and
say:
select
descriptor
and
value
cons
and
show
to
see.
What's
the
most
common
descriptor
right,
we
can
see
that
okay
bag
is
very
common
set,
is
common,
but
office
may
be
something
that
we
don't
want
to
see
there.
It's
not
really
a
description
and
there's
the
empty
string
there.
One
thing
we
can
do
is
just
let's
just
remove
all
the
length
that's
less
than
three
okay,
let's
keep
evaling
okay,
so
that
looks
a
bit
better.
F
At
this
point,
I
think
you
kind
of
get
the
idea
of
how
guinea
allows
you
to
get
started
very
quickly
and
also
be
able
to
iterate
through
different
ideas
very
quickly
as
well.
It's
designed
so
that
it's
it's,
it's
short
to
write
right,
and
it
has
a
lot
of
these
idioms.
So
at
this
point
I
want
to
go
to
another
script
that
I've
made.
F
Basically,
the
the
first
two
is
the
same
as
before,
with
a
little
bit
more
cleaning
step,
and
maybe
what
I
want
to
do
with
these
transaction
data
is
that
I
want
to
do
a
recommender
system
right.
I
just
want
to
train
on
a
collaborative
filtering
model
just
like
in
the
netflix
challenge,
where
you
have
members
movies
and
their
ratings
this
time
around.
F
It
expects
numbers
of
ids
right
instead
of
strings
so
that
first
we
need
to
convert
the
descriptor,
which
is
a
sentence
to
descript
our
id,
which
is
just
going
to
be
a
number
and
there's
a
nice
feature
step
for
that
on
spark
as
well.
That's
called
the
string
indexer
and
we
can
chain
these
two
stages
using
a
pipeline.
F
What's
going
to
happen
is
that
during
training
you'll
pick
up
all
those
descriptor
and
assign
it
a
number
and
then
during
test
time
you
remember
what
the
descriptors
are
and
what
corresponding
numbers
will
be
assigned
to,
and
here
I'm
going
to
fit
and
I'll
be
able
to
take
once
the
model
is
fit
I'll,
be
able
to
take
the
last
stage
which
is
corresponding
to
the
alternating
least
squares
model.
And
then
I
can
do
some
item
recommendation
and
I'm
going
to
do
all
of
that.
Now,
let's
give
that
a
go.
F
So,
just
to
show
that
I'm
not
bluffing
right
like
this
is
really
training
right.
It's
using
all
the
cores.
It's
going
to
take
a
bit
less
than
one
minute.
The
data
frame
I
think,
contains
800
000
rows.
So
it's
considered
to
be
you
know,
spark
is
pretty
performant
that
way
and
we'll
just
let
it
finish
there
you
go,
it
just
dies
down.
So
that's
finished
right.
So
what
I
want
to
show
is
this
right
recommend
items.
F
Oh
excuse
me:
I
need
to
define
the
als
models
and
now
we
can
just
do
recommend
items.
What
that's
going
to
do
is
that
for
every
customer
id
it's
going
to
recommend
a
descriptor
right,
so
maybe
this
customer
id
relates
red
and
then
we're
gonna
recommend
bags
and
lights
as
well.
Who
knows
it's
again?
It's
not
meant
to
be
useful.
F
A
A
A
G
B
E
E
I
I
wouldn't
know
if
it
is
worthwhile
actually
trying
to
connect,
because
it
appears
that
these
are
two
different
audiences.
The
kind
of
people
who
want
to
work
with
the
rebel,
as
opposed
to
somebody
who
wants
to
work
with
with
the
gui
and
and
frankly,
weka
also
has
the
ability,
just
like
rapidminer
and
nime,
to
actually
build
the
whole
pipeline
in
a
gui.
E
D
You
know
where
to
you
know,
step
in
and
and
do
something
custom.
If
you
want
to
replicate
that,
so
I
think
it's
a
useful
thing
now,
not
everybody
will
need
it
so
again
like
what
how
git
does
and
maybe
even
you
know,
the
linux
installation
process
does
where
you
can.
It's
typically
by
default,
collapsed
and
you
can
expand
it
to
see
what
commands
are
being
run.
D
E
D
H
I
I
I
have
a
question
or
comment
rather
like,
as
in
this
coming
from
someone
who
is
trying
to
explore
like
the
data
science,
libraries
as
a
beginner,
then
when
else
I
mean,
I
understand
that
clojure
tends
to
have
a
lot
of
like
libraries
that
seem
to
be
like
their
stock
development,
but
they
still,
they
are
still
definitely
usable.
But
so
when
I,
when
I
explored
when
I
saw
a
clj
ml
like
when
I
saw
the
last
day
of
update
in
2017,
I
was
like.
Oh,
this
is
probably
not
being
used
anymore.
E
Yeah,
that's
a
good
point
and
I
think
it's
a
double-edged
sword
in
the
sense
that
I
believe
cljml
has
everything
out
of
the
box.
I
mean
it's
been
around
for
a
really
long
time
and
it
doesn't
need
you
know.
The
only
reason
you
would
need
to
update
clg
ml
is
that
if
there's
some
new
library
that's
come
out
and
you
want
to
add
support
otherwise
beyond
that,
just
like
weka
it
has
you
know
so
many
classifiers
that
you're,
like
you
know,
do
I
really
need
to
try
30
classifiers,
but
that's
how
it
is
so.
E
But
I
take
your
point
about
documentation,
I
mean
sometimes
improving
documentation.
I
mean
just
seeing
the
last
commit
date,
sort
of
gives
people
the
idea
that
you
know
it's
old.
It's
probably
you
know
broken
around
the
edges,
which
is
not
true,
but
I
don't
know
if
there's
a
way
to
you
know
make
it
more
appealing
and
I'm
open
to
suggestions
around
that.
J
So
one
of
the
fundamental
design
principles
of
clojure
that
isn't
shared
by
scala
is
that
we
don't
need
to
update
our
libraries
once
they're
written
there's.
A
lot
of
clojure
libraries
that
are
very
old
and
haven't
had
a
commit
for
four
or
five
years
potentially
and
are
still
extremely
useful,
and
the
reason
that
that
works
is
because
rich
made
a
lot
of
design
decisions
very
carefully
so
that
he
chose
very
conservative
pathways
with
the
language
and
the
language
has
not
changed
them.
J
So
you're
used
to
languages
like
python
or
scala
or
these
other
languages,
where
you
know
with
scala.
If
they
update
the
compiler,
then
every
single
scala
library
has
to
change
now
to
use
the
new
compiler
but
and
to
a
lesser
extent
java,
but
with
clojure.
It's
been
backwards
compatible
for
quite
a
long
time.
So
one
of
the
assumptions
that
new
people
have
coming
into
the
ecosystem
is
man.
J
There's
all
these
libraries
that
haven't
been
updated
for
a
long
time,
just
like
you
said,
but
I
urge
you
to
think
you're
in
a
different
ecosystem
and
some
of
the
design
principles
that
ecosystem
mean
that
as
a
library
author,
I
don't
have
to
update
my
libraries
anymore,
and
I
have
a
few
libraries
that
are
quite
old
but
they're
still
really
useful
so
that
that's
another.
Take
on
that
kind
of
perspective.
A
Oh,
thank
you
very
much
for
your
comment.
Chris.
I
think
it's
very
important
to
stress
that
traits
of
the
closure
landscape
of
the
closure
library
landscape.
It's
it's
very,
very
important
to
to
know
that,
because
sometimes
those
who
are
not
so
fluent
in
closure,
we
forget-
we
don't
forget
that
and
I
think
it's
a
very
important
point.
A
E
E
D
So
this
may
be
a
very
kind
of
like
a
newbie
question
kind
of
a
thing
sometimes,
and
it's
part
of
the
you
know
some
of
you
who've
been
working
in
data
science
or
machine
learning,
and
all
this
take
a
lot.
I
mean
you
take
things
for
granted
and
that's
what
comes
you
know
as
an
expert
you
you
take
it
for
granted,
but
sometimes
it
is
these
basic
questions
that
trip
you
up
and
it's
like.
How
do
you
set
up
your
environment?
How
do
you
get
started
on
this
someone's
using
something?
D
Am
I
doing
the
right
thing,
and
this
goes
to
the
the
I
think
you
covered
that
in
the
first
part
right
where,
when
what
is
a
good
use
for
cljml
or
weka
kind
of
thing,
when
should
you
use
it
and
all
that?
So
it's
part
of
that
thing,
you
know
I'm
saying
hey.
Should
I
even
bother
to
use
this
if
I'm
doing
work
most
of
my
exploration
on
just
using
a
laptop,
and
I
don't
have
access
to
you-
know
a
an
aws
cloud
environment
or
something
like
that.
E
A
F
Yeah,
so
I
definitely
understand
that
the
that
spark
has
this
reputation
right
of
being
used
only
for
distributed
computing.
It's
only
being
used-
and
you
know
when
you
have
like
a
massive
cluster
and
when
you
have
like
a
massive
data
set
and
stuff,
doesn't
fit
in
core
right
and
and
your
memory
so
based
on
my
experience,
this
hasn't
been
the
case
right,
like
I
use
guinea
every
day
on
the
desktop
on
my
laptop,
and
it
works
a
lot
nicer,
at
least
for
me
right,
oh
and
and
obviously
I'm
biased.
F
You
know
I've
been
using
pandas
a
lot.
I've
been
using
dusk,
a
lot
for
me.
The
experience
with
guinea
and
spark
is
a
lot
better,
so
yeah
definitely
use
it
on
your
laptop
and
on
sort
of
like
getting
started
right,
like
I
think
at
least
we
invested
a
lot
of
time
on
on
and
trying
to
make
it
as
easy
as
possible.
F
So
the
entire
thing
with
the
guinea
cli
is
that
you
should
just
be
able
to
start
then
and
there
right
and
to
install
that
you
need
three
bash
commands,
so
download
move
it
to
your
path
and
make
it
executable.
That's
a
bit
like
line
right
and
then
it
just
starts
and
you're
you're
good
to
go
yeah.
So
it's
usable
on
the
laptop.
F
A
I
I
have
a
a
question
which
is
also
for
for
both
first
kieran
and
then
anthony.
I
remember
there
was
some
discussion
about
the
license
under
which
weka
is
distributed,
which
is
gpl2.
If
I
remember
correctly,
and-
and
there
was,
if
I
remember
correctly-
there
was
someone
who
said
he
or
she
didn't
use
weta
because
of
the
license.
E
A
A
A
A
In
any
case,
if
it's
mit,
so
it
should
be
compatible
with
the
gpl.
Actually,
I
think
the
problem
is
just
when
one
has
to
build
the
the
jar.
Let's
say,
given
that
claudia
has
epl,
like
eclipse
alliance,
highlight
license
it's
a
little
bit
complex
to
combine
your
own
code
with
closure
and
with
java,
which
is
under
the
gpl
with
dashboard
exception.
So
that's
probably
the
the
most
tricky
part,
at
least
as
far
as
I'm
gonna,
I'm
concerned.
A
F
A
J
J
I
think
in
this
case
it
was.
Oh
man,
sorry,
if
I'm
speaking
about
things
you
don't
understand,
but
it
was
a.
It
was
an
executable,
and
so
we
shipped
the
specific
library
as
a
shared
library.
We
kept
it
as
a
shared
library
and
we
shipped
a
whole
build
system
so
that
a
user
could,
for
instance,
rebuild
a
portion
of
that
shared
library
and
then
relink
it
into
the
main
program.
J
Now
you
have
to
ship
to
a
user
or
allow
a
user
to
change
in
some
way
and
that
that
definition
there
has
changed,
depending
on
which
version
of
the
gpl
you're
actually
linking
to
and
and
kind
of
a
lot
of
things
in
there.
So
the
gpl
is
definitely
something
where,
if
you're
making
money
with
the
software,
you
need
to
consult
a
lawyer
who
specializes
in
it
to
tell
you
what
the
right
way
to
do
it
and
the
right
way
to
do.
It
may
very
well
eliminate
your
competitive
advantage
in
your
field.
K
K
K
But
I
was
kind
of
left
confused
because
you
were
putting
things
into
other
things
and
is
there
a
way
to
think
about
those?
What
do
they
take
in
and
what
do
they
take
out
and
if
we
have
the
time,
perhaps
you
can
just
share
your
screen
again
and
explain
a
few
of
the
functions.
For
instance,
g,
slash,
remove
and
ml
translator.
F
F
So
if
you've
got
experience
working
with
spark
data
sets,
they
are
yeah,
they're,
they're,
just
straight
off
translations.
So
these
really
are
just
spark
wrapping
spark
methods,
plus
some
other
extra
stuff,
for
example,
like
spark,
doesn't
have
to
remove
right,
so
you'd
have
to
do
a
filter
and
then
something
like
not
or
something
like
that.
That's
what
you
would
do
in
spark,
but
then
enclosure
does
have
removed.
So
I
added
remove
right
yeah.
F
F
F
Yeah,
that's
that's
right!
So,
if
I
get
rid
of
all
of
that,
so
invoices
is
just
another
spark.
Data
set
right
and
then
select
is
just
another
spark
data
set
and
yeah.
So
this
is
meant
to
resemble
the
kind
of
method
chaining
that
you
you,
that
you
have
and
scala
and
which
I
think
works
very
well
as
well
in
closure.
F
F
It
would
be
enclosure,
so
actually,
what
happens?
What's
interesting
in
spark
is
that
this
thing
gets
recomputed
every
time
so
yeah.
So
there
is
an
absolutely
no
mutation
happening
here,
yeah,
so,
if
you're
doing
all
of
these
stuff
actually
every
time
I
call
it
it
gets
recomputed.
So
it's
actually
going
to
read
the
csv,
it's
actually
going
to
rename
it
what's.
F
What
happens
underneath
is
that
it
creates
a
computational
graph
until
I
have
an
action,
so
show
is
an
action
or
a
count
as
an
action,
but
otherwise
this
is,
I
think,
what
haskell
people
call
a
thunk
right.
It's
just
a
computational
graph
like,
but
it's
a
plan
to
do
something
as
soon
as
you
say
show,
then
it
does
something
that,
and
you
can
do
like
cache,
which
just
means
like
okay,
actually
save
it,
so
that
the
next
time
it's
faster,
that
that
happens.
F
So
I
I'm
guessing
yeah.
I
guess
it's
kind
of
a
long-winded
way
of
saying
that
I
guess
definitely,
this
all
the
spark
peculiarities,
all
the
spark
features.
All
the
spark
properties
are
here
right
and
you
kind
of
have
to
understand
the
how
spark
works
in
order
to
be
quite
effective
in
this,
and
and
that's
also
by
design
right,
because,
just
like
how
you
know
enclosure,
sometimes
you
need
to
understand
how
how
java
works-
and
I
think
you
know,
if
you're
leveraging
a
particular
ecosystem.
K
A
A
So
if
I
see
that's
the
one
okay,
if
that's
the
only
question,
then
we
can
take
that
last
question
and
and
then
officially
finish,
because
we
are
already
past
time
and
at
least
officially,
of
course
we
can
stay
longer
if
we
wish
to,
but
I
would
like
to
avoid
forcing
someone
to
stay
if
he
or
she
doesn't
want
to
so
just
to
just
to
be
sure.
Kiran's
question
is
the
only
question
we
have
left
right.
A
E
Sure,
I
think
so
I
have
actually
two
questions
and
I'll
try
and
make
it
quick.
The
first
one
is
that
there
have
been
other
spark
rappers
in
closure.
So
what
was
your
motivation
for
writing
this,
and
the
second
question
is:
how
would
you
advise
somebody
to
use
a
non-spark
pipeline
versus
a
spark
pipeline
and
you've
shown
clearly
that
it
is
fast
enough,
but
there
would
obviously
be
some
trade-offs
in
using
spark
in
a
single
machine.
E
F
F
The
main
thing
is
that
all
these
four
libraries
they
target
spark
rdd
apis.
So
that's
the
lower
level
spark.
So
you
don't
have
this
nice
spark
sql
interface,
which
I
think
you
know
so
I
used
to
work
with
scala
spark
a
lot
for
about
two
years.
You
use
spark
sql
for
90
95
of
the
time
and
then
the
rest.
You
know
it's
combination
of
ml,
rdds,
yeah,
so
they're
missing
out
the
most
important
part
of
the
spark
ecosystem.
F
I
feel
spark
api
and
it's
not
to
say
anything
bad
like,
like,
I
think,
spark
sequel
is
they
got
released
after
these
libraries
were
actively
being
worked
on
so
that
that
would
be
the
answer
for
question.
One
and
second
question
is
about
sorry.
Would
you
can
you
remind
me?
Is
that
like?
When?
Would
you
use
this
well
like
in
what
situation
is
that.
E
F
No
actually
like
everything
I
showed
you
is
as
not,
it
happens,
on
a
single
desktop.
It's
not
a
distributed
cluster
at
all.
So
spark
has
these
modes
right,
there's
distributed
mode
and
there's
local
mode
as
well.
So
no
like
it
works
on
your
laptop.
It
works
on
the
desktop.
It
works
on
the
cluster
same
code
like
no
modification
at
all
right
and
actually
with
ram
like
I
found
that
ram
usage
will
spark
us
a
lot
a
lot
better
than
pandas.
F
J
I
think
so
one
is
there's.
There
is
a
consideration
that
I
found
with
spark
and
that
it
generally
use
makes
heavy
use
of
disk
in
various
ways
that
you
may
not
necessarily
predict
and
I've
had
it
eliminate
the
disk
space
on
my
machine
when
I
was
playing
with
it
potentially
carelessly,
but
it
it
does
write
a
lot
to
temp
files
when
you're
doing
various
things.
J
So
if
you're
gonna
use
it
in
like
a
kubernetes
situation,
you
just
got
to
kind
of
think
of
how
you're
going
to
mount
your
containers
and
that
kind
of
thing
so
that
it
has
disk
space
available
to
it,
and
that
very
well
could
be
how
I
misconfigured
it
or
lots
of
other
things.
But
the
the
naive
configuration
of
spark
data
frames
is
can
be
disk
io
intensive,
but
that
that
I
think,
is
something
you
would
not
hit
on
a
laptop,
and
I
don't
think
it's
a
consideration
for
the
desktop.
J
J
F
J
J
F
A
A
So
I
think
we
can
officially
end
this
event.
Many
of
us
will
stay.
Of
course,
no
one
is
forced
to
stay
if
he
or
she
doesn't
want
to.
But
again
we
we
thank
everybody
for
participating
and
coming
today
and
hearing
these
very
interesting
thoughts
about
machine
learning
enclosure,
and
we
really
are
looking
forward
to
see
you
in
our
next
event.
Thank
you
very
much.
H
Hey
just
quickly
before
you
continue,
this
was
so
great,
absolutely
amazing
stuff.
Thank
you
for
for
the
speakers.
I
can't
wait
to
get
back
home
to
my
machine
and
play
around
with
these
libraries
and
also
thanks
for
the
thanks
for
the
moderation.
These
are.
These
are
so
nice,
but
yeah.
I
need
to
go
now
so
bye,
bye,
bye,.
D
B
And
yeah
so
so
recently
we
have
been
having
some
chats
about
having
like
maybe
a
central
place
for
tutorials,
so
that
can
be
one
way.
We
have
an
a
github
organization
where
everybody
can
put
their
projects
if
they
wish
to
put
them
there,
but
personal
repos,
also,
both
I
guess
to
me.
It
may
take
a
little
bit
more
time
to
complete
the
tutorial
and
I
wish
to
share
it
there
under
the
psych
lodge
with
an
organization.
B
B
B
Oh
she
she's
calm.
Now
I
hope
yeah.
So
so
here
we
begin
yeah.
We
have
an
empty
namespace
and
some
some
empty
view
at
the
browser,
and
we
can
start
adding
different
kinds
of
closure
elements,
anything
that
we
can
have
in
a
closure
namespace.
So
one
of
them
can
be
a
string
that
renders,
as
marker
something
like
that,
and
we
can
update
the
view
and
we
can
see
in
the
view
that
we
have
it
in
the
browser.
But
we
keep
looking
at
the
repel
and
editor
right.
B
A
B
B
A
B
F
B
B
B
D
B
Yeah,
so
gorilla
reppel
is
a
really
maybe
one
of
the
first,
probably
the
first
notebook
written
in
closure.
It
is
a
project
written
by
johnny
hudson
and
it
is
written
in
closure
and
javascript
and
it
provides
a
full
client-side
experience
where
you
can
edit
and
manipulate
your
notebook
in
the
browser,
and
it
has
really
neat
ideas
about
creating
composable
views
of
your
data
that
we
cannot
go
into
in
the
moment.
But
there
are
really
really
conceptual
conceptually
beautiful
ideas
there
about
the
way
you
visualize
you,
visualize
closure
data
structure
and
gorilla.
B
Effel
has
stopped
being
under
development,
and
now
there
is
a
new
project
called
ping
gorilla
by
our
friends.
Andreas
and
florian
and
pink
gorilla
is
actually
a
huge
ecosystem
of
closure.
Libraries
for
a
very
much
complete
experience
of
notebook
like
of
a
notebook-like
tool
in
the
browser,
and
it
is
written
in
closure
and
closure
script,
and
there
is
a
lot
going
on
there
and
the
central
notion
is
having
a
really
decent
browser
experience.