►
Description
On Oct. 30th, 2022, we had our second study session about NLP in Clojure.
This time, the session focused on building NLP machine learning pipelines involving Clojure & Python in two ways: DVC, and metamorph.
Towards the end, we also saw some work-in-progress additions to scicloj.ml, combining its grid-search algorithms with DVC.
Carsten Behring, who is author of Metamorph and the scicloj.ml platform, presented most of the session.
João Santiago, who is involved in the DVC community and is the author of the dvthis R utility for DVC, supported the session with an intro to DVC.
Carsten Behring's blog post and code:
https://scicloj.github.io/blog/predict-real-vs.-fake-disaster-tweets/
A few of the tools and libraries mentioned:
https://dvc.org/
https://github.com/clj-python/libpython-clj
https://github.com/jcpsantiago/dvthis
https://github.com/scicloj/metamorph
https://github.com/scicloj/metamorph.ml
https://github.com/scicloj/scicloj.ml
https://simpletransformers.ai/
A
This
is
the
second
time
that
this
group
of
data
and
closure
enthusiasts
are
meeting
to
talk
about
natural
language
processing.
So
yeah,
hello,
thanks
for
joining
us,
and
maybe
since
there's
only
a
few
people
here,
we
could
do
just
a
really
brief
intro
go
around
and
explain.
Basically,
just
like
you
know,
30
seconds
who
you
are
and
why
you're
here
so
I
can
start
I'm,
Kira
I'm,
a
software
developer
I
do
mostly
web
development,
but
happened
to
stumble
into
a
job,
doing
web
development
for
an
open
data.
A
Publishing
company,
which
kind
of
led
me
into
this
world
of
working
with
data
and
now
I'm
working
on
a
project
to
create
educational
resources
for
the
closure
data
community.
So
a
kind
of
QuickBook
of
examples
that
will
hopefully
serve
as
like
a
reference
for
how
to
combine
all
of
the
different
libraries
so
something
that
could
be
added
to
on
top
of
all
the
really
excellent
API
docs
that
already
exist
for
people
who
are
looking
to
just
like
solve
a
problem
so
very
similar
to
like
the
our
cookbook.
A
B
Hello,
my
name
is
Carson
Billy
I'm
data
scientist
I
have
a
background.
This
is
a
software
engineer
and
I
try
to
to
do
my
due
to
this
data
scientist
using
as
much
closure
as
possible,
so
I'm
very
much
interested
in
the
machine
learning
aspect,
but
as
well:
interoperability,
visual
tools
as
well
to
Tool,
which
allows
me
to
do
my
my
job
a
lot
of
times
using
closure
and
I'm
very
happy
to
contribute
to
to
increase
that
percentage,
because
closure
is
a
fantastic
language.
Thank
you.
C
Hi
I'm
Ian
I
work
with
closure,
I
work
for
flexiana
and
with
a
a
few
different
teams,
programming,
mostly
enclosure,
but
occasionally
in
other
languages,
and
that's
what
attracted
me
to
tonight's
session.
It's
like
integrating!
What
do
you
say
connecting
closure
and
python,
and
that's
a
current
problem
that
we
are
facing
I'd
like
to
know
more
about
that
personally.
I
have
a
great
interest
in
the
way
that
closure
is
taught
and
I
have
this.
C
This
strong
feeling
that
if
closure
was
taught
earlier,
it
would
be
of
great
benefit
to
both
the
closure
community
and
the
students
that
are
fresh
learning
programming.
That's
something
that
I
want
to
explore.
More
of
and
I
kind
of,
like
the
the
visual
side
of
the
data
science
aspect
of
it
so
combining
that
enclosure.
It's
like
interested
in
lots
of
different
directions:
they're
all
kind
of
meet
up
with.
What's
going
on
here,
I'm
going
to
nominate
Ash.
D
Hello,
I'm
Ash
my
background
software
engineering,
I'm
working
in
a
research,
a
university
research
project
in
Scotland,
and
this
research
project
is
to
do
with
basically
creating
a
recommender
system.
That's
going
to
help
people
I'm
going
to
help
recommend
treatment,
Pathways
to
try
to
rehabilitate
people
getting
back
into
work,
getting
back
into
society
and
we're
working
with
a
data
set
from
an
Icelandic
Corporation
and
we're
using
some
machine
learning
and
I'm
interested
in
closure,
I'm
interested
in
DVC
and
I'm
glad
to
be
here.
That's
me.
E
Hello,
I'm,
I
I,
think
you
know
me
so:
I'm,
Tori,
Anderson
and
I
am
a
web
development
engineer
at
University
and
I
spent
a
lot
of
time.
Training
students
in
closure,
so
I
love,
I'm,
a
definite
believer
in
teaching
students
and
my
background
is
in
natural
language
processing
and
I'm,
currently
doing
research
in
control
theory
as
well.
So
that's
the
data
science
plug
there.
So
closure
is
great.
I've
been
using
that
for
a
while
foreign.
F
So
that's
me
Santiago
I've
I'm,
working
into
in
the
anti-fraud
team
of
fintech
in
Berlin,
so
my
my
daily
job
is
mostly
data
science,
just
doing
algorithms
for
fraud
models
in
real
time
and
so
on.
We
use
mostly
R,
but
for
some
of
the
tasks
like
I
was
saying
before
in
our
pipeline
we
did
have
some
closure
involved
and
now
I'm
actually
building
my
team
up
with
closure,
not
from
data
science
out
but
from
the
engineering
in.
F
So
we
are
everything
related
to
model
deployment,
model
operations,
model,
Registries
and
so
on
model
alerting
everything
is
being
done
in
closure,
and
so
we
are
slowly
building
the
team
there.
Otherwise
I've
only
been
using
closure.
You
know
as
a
foresight,
projects
and
hobbies
and
stuff
and
I'm
actively
trying
to
change
that.
F
G
That
system
is
implemented
in
both
closure
and
and
python.
What
brings
me
here
in
particular,
is,
while
I
don't
know
anything
about
natural
language
processing.
We
part
of
our
database
system
is
an
automated
data.
Modeling
component,
that
is
filled
with
closure,
Python
and
DVC,
so
I
figure
coming
here.
I
might
learn
something
so
I'm
very
much
looking
forward,
who
hasn't
gone,
Daniel
I
mean
I
know.
Did
you
introduce
yourself
yet.
H
Thank
you
and
I'm
Daniel
I
do
statistics
mostly
I'm
a
statistician,
but
recently,
at
my
day,
job
I
did
have
to
do
some
natural
language
processing
and
it
is
so
much
different
from
what
it
used
to
be
and
I'm
so
much
overwhelmed
by
new
ideas
and
the
people
around
me
are
teaching
me
a
lot
and
yeah.
We
did
have
a
little
bit
of
opportunity
to
use
DVC
and
it
was
delightful,
and
you
know,
since
carton
blog
post
about
people.
I
They're
working
on
trying
to
pick
up
closure
a
bit,
but
mostly
just
trying
to
adapt
to
using
emacs
is
my
main
editor
or
well,
not
editor,
Computing
environment,
so,
but
so
I've
liked
a
little
bit
spinning
up
with
closure.
But
I
was
attracted
to
the
fact
that
we're
you
know
today,
there's
going
to
be
discussions
about
interface
between
Python
and
closure.
That
sounds
very
interesting
to
me.
A
Okay,
anyway,
we
will
carry
on
with
maybe
a
brief
intro
about
today's
topics
from
Carson,
so
we're
gonna
learn
about
yeah,
basically
connecting
closure
with
python
and
DVC,
which
basically
all
I
know
about.
It
is
a
sort
of
like
version
control
system
for
pipelines
or
for
machine
learning
models,
or
something
like
this.
So
person
is
going
to
give
us
a
little
overview
and
then
we'll
get
into
some
more
details
and
have
some
time
for
discussion
so
I'll
hand
it
over
to
you
thanks.
So
much
Carson.
B
So,
thank
you.
I
want
to
talk
about
python
closure,
Inca
up
using
DVC
the
data
Version
Control
I.
Look
at
the
agenda
just
three
points:
closure
very
short
closure
for
data
science.
Then
you
see
the
two
use
DVC
for
model
training
and
then
a
little
bit
of
a
comparison,
because
we
have
in
DVC
a
concept
of
pipelines,
and
some
of
you
have
looked
at
ski
closure,
metamov,
there's
another
concept
of
pipelines
and
just
to
contrast,
the
the
two
please
interrupt
me
at
any
moment.
B
B
Okay,
closure
for
data
science,
but
I
want
to
the
message
I
want
to
bring
here
is
that
in
in
data
science,
that
in
general
relies
on
a
very
fast
evolving
Universe
of
existing
models
and
algorithms.
A
big
part
of
that
is
rooted
in
Academia
still
Python
and
R
dominate
the
field
since
years,
and
that
will
not
change
a
lot.
There's
no
Java
there,
but
it's
a
clearly
certain
class.
Maybe
Julia
comes
now,
but
we
talk
about.
B
So
that
means
that
we
want
part
of
our
job
is
to
basically
Google
for
people
who
have
SIM
similar
problems
to
have
a
look,
how
they
have
solved
the
problem,
and
that
is
then
the
next
try
and
we
repeat
until
we
gave
up
or
or
we
have
have
a
good
enough
solution.
So
there
is
somehow
lots
of
trial
and
error.
We
are
copy
paste
what
others
people
have
done,
and
that
means
interop.
B
B
There
are
two
ways
of
interop.
There
is
one
way
which
I
call
in
process
interop.
So
that
means
that
in
some
form
we
have
a
really
an
interop
Library
which,
for
the
case
of
python
and
and
R,
are
called
lip
python,
cha
and
closes
are
so
basically
there
you
write
a
closure
code
and
behind
the
scenes
it
gets
transformed
into
python
or
our
codes
in
that
form,
so
I
call
it
in
process,
maybe
even
technically
the
right
word,
but
you
write
at
the
end.
B
Only
kind
of
closure
both
are
very
compatible,
but
not
hundred
percent
in
the
sense
of,
even
though
that
they
that
lip
python,
for
example,
is
using
a
python
process
and
clauses
are,
is
using
an
R
process.
They
do
things
differently
than
the
majority
of
other
people
are
doing,
because
so
it's
a
little
bit
of
a
different
runtime.
There
are
very
compatible.
Not
100.
I
would
call
them
a
99
person
compatible,
but
there
are
different
runtimes
and
they
are
far
less
tested.
B
B
F
So
the
the
first
thing
I
would
like
I'm
going
to
show
you
is
a
library
I
wrote
to
for
interacting
between
DVC
and
R,
because
it
has
a
couple
of
examples
already
just
to
show
how
this
is
used
in
practice
and
I
think
seeing
an
example
of
how
it's
used.
You
can
also
understand
like
why
I
would
use
this
in
the
first
place
and
why
what
Carson
is
saying
makes
so
much
sense.
F
We
originally
wanted
to
use
DVC
because
of
these
Version
Control
aspects
of
it
I
mean
the
whole
tool
is
called
Data
Version
Control
DVC,
but
in
the
end
it's
a
lot
more
and
I.
Actually
what
I
use
the
most
is.
The
other
part
that
is
not
data,
Version
Control,
so
to
speak.
So
DBC
has
two
big
components:
it
has
this
data
Version
Control
part,
but
it
also
has
this
pipelining
part
and,
as
as
you
will
see
like
almost
everything
in
data
science
is
about
dags.
You
have
graphs
everywhere.
F
You
have
things
that
have
been
painted
on
other
things
and
steps
that
depend
on
previous
steps,
which
I
guess
for
a
lot
of
us
doing
closure.
It's
just
it's
obvious
right:
we
learn
a
lot
of
threading
macros.
Now,
just
you'd
have
to
expand
this
into
whole
stages,
and
so
so
this
is
my.
This
is
my
library,
it's
a
bunch
of
utilities.
What
I
want
to
show
you
is
just.
This
is
an
example
right,
so
usually
data
science,
workflow
machine
learning.
Workflow
would
have
some
stages
like
this.
F
You
need
to
fetch
data
somewhere,
you
need
to
clean
it
transform
it.
You
have
to
tune
some
model
and
train
a
model,
and
then
you
want
to
produce
some
plots,
some
metrics
to
show
stakeholders
or
to
yourself,
and
you
want
to
have
an
actual
model
in
the
end.
You
can
obviously
do
all
of
this
in
a
single
notebook,
if
you're,
using
only
python,
for
example,
but
even
if
you're,
using
a
single
language
like
right
now
at
Billy
in
my
company,
we
are
only
using
R
for
the
fraud
team.
F
We
don't
you
don't
use
closure
anymore,
because
we
don't
need
this
PDFs
that
we
had
prefer
to
scrape,
but
even
here
it
makes
sense
to
have
each
of
these
steps
separate
because
think
of
them
as
pure
functions.
Right,
like
this
stage,
only
does
fetching
data.
So
if
there's
a
bug,
we
only
go
there
and
we
only
fix
faction
data.
F
All
of
the
steps
after
this
are
untouched
right-
and
this
is
the
really
smart
part
about
DVC-
is
that
it
creates
this
graph
of
dependencies
and
it
only
runs
the
stages
that
are
invalidated
every
time
you
make
a
change.
So
if
you
only
change,
let's
say
the
produce
plot
stage,
which
is
usually
the
last
thing
you
do,
you're
not
going
to
run
the
whole
pipeline
again,
it
just
runs
the
produced
plot
stage
yeah.
F
How
would
this
look
in
practice
is
a
bit
like
this,
so
DVC
in
itself
is
quite
simple:
it
only
produces,
or
you
only
have
to
interact
mostly
with
two
files,
there's
obviously
like
a
CLI
interface
and
so
on,
but
I
will
not
get
into
that.
The
the
DVC
yaml
file
is
where
you
define
stages,
so
you
would
say
you
know,
I'm
having
here
a
stage
called
hello.
F
World
I
just
says,
run
this
script
with
r
there's
a
dependency
on
the
actual
script,
so
that,
if
I
change
something
in
the
script,
the
whole
the
the
whole
pipeline
after
this
stage
gets
invalidated
and
DVC
will
run
that,
for
you
and
I'm
also
saying:
there's
an
output
right.
So
there's
a
there's.
A
file
here
called
random
numbers,
that's
being
saved
into
data
and
intermediate,
so
any
stage
that's
dependent
on
this
file
will
also
subsequently
be
invalidated,
and
so
on
so
a
very
simple
example.
F
F
F
F
So
what
what
DVC
does
is
just
saving
this
file
and
then
you
can
push
your
actual
file
somewhere
else.
So,
for
example,
you
can
have
in
GitHub
the
DVC
log
file
saying
this
is
the
hash
of
the
file
that
I
need
and
then
the
actual
file
lives
in
an
S3
bucket
somewhere
or
in
a
Google
drive
or
in
Dropbox.
This
is
actually
what
we
do
at
Billy
to
deploy
models
to
production.
F
The
only
thing
we
have
to
do
is
merge
to
master,
and
then
we
have
another
script
that
runs
DVC
and
says:
hey
fetch,
all
the
models
that
are
available
and
it
grabs
them
from
S3
and
then
deploys
those
files
into
the
prod
service.
So
we
never
have
to
deal
with
very
large
files
being
saved
in
in
git.
Another
cool
aspect
of
this
is
that
you
can
always
go
back
in
time
and
check
what
happened
with
that
specific
pipeline
right.
F
F
It
will
not
work
because
I
have
nothing
configured,
but
if
you
would
run
this
command,
you
would
pull
all
of
the
files
that
are
in
the
remote
storage,
with
the
correct
hashes
and
now
you
can
run
the
pipeline
yourself
and
you
will
get
exactly
the
same
results
as
I
am
getting
so.
This
is
also
super
important
in
data
science,
not
being
reproducible
and
transparent
and
auditable
to
understand
what
is
going
on
and
what
files
change
and
when
now
just
you
know
as
an
example,
let's
say
we
want
to
write
a
new
stage.
F
So
what
you
would
do
is
you
would
simply
add
you
know.
Let's
say
this:
is
the
hello
world
two
stage
I
cheated
right,
so
I
already
have
a
script
there
and
I
say
I
want
to
this
stage.
One
needs
to
run
the
hello
world
again
pipeline
or
our
script.
Sorry
and
I'm
saying
it's
dependent
on
the
script
which
is
stored
in
stages.
F
F
F
All
right,
let
me
show
you
this,
so
we
did
DVC
Repro
here
and
so
you
see
DVC
detected.
No,
not
this
one
yeah
running
stage.
So
the
first
stage
didn't
change
hello
world.
We
didn't
do
any
change
to
the
to
the
script
or
the
outputs.
So
it's
not
running
anything
which
is
very
efficient,
but
it
found
there's
a
new
stage
now
called
hello
world
2.,
it's
running
it
and
you
can
see
here.
We
are
just
saying
that
we
are
pulling
in
the
results
from
the
previous
stage.
F
We
are
saving
it
into
a
variable
and
then
we
are
just
passing
it
into
a
big
string
and
printing
the
string
here.
So
these
are
very
you
know,
basic
examples,
but
now
you
can
expand
this
into
thinking.
You
can
have
the
stage
for
fetching
data
from
a
DB.
Then
you
train
the
model.
You
do
all
sorts
of
things
in
production.
In
my
company
we
have
around
14
stages
because
you
need
to
do
a
lot
of
fetching
from
different
sources.
F
We
also
have
multiple
models
running
in
the
same
pipeline,
and
so
instead
of
having
you
know,
one
fraud
model,
for
example,
going
from
top
to
bottom.
We
actually
have
sort
of
a
for
Loop
training,
multiple
ones,
always
in
the
same
and
the
same
pipeline,
and
as
you
see,
this
is
not
specific
to
data
science.
This
is
not
specific
to
machine
learning.
It's
a
very
abstract
way
of
saving
intermediate
data
and
running
stages
in
a
graph
that
you
need.
F
So
if
you
want
to
interrupt
for
whatever
reason
like
I
had
to
interrupt
with
closure,
just
to
parse
like
to
do
OCR
on
some
PDF
files,
it
has
nothing
to
do
with
machine
learning,
but
closure
was
the
best
tool
for
the
job,
so
I
created
a
stage
using
a
babashka
script.
To
do
this
and
DVC
doesn't
care
as
long
as
you
can
run,
he
recommend
using
bash
or
whatever
shell
you
have
DVC
will
do
it
and
that's
kind
of
my
super
fast
intro
to
DBC.
B
Okay,
if
there's
no
questions,
I
go
ahead.
Okay,
as
I
said
before
these
in
process,
interoper
yeah,
they
work
99,
but
not
100.
So
it's
good
to
have
a
plan
B.
So
that
was
my
basic
way
why
I
wanted
to
use
something
like
DVC
I
was
as
well
only
looking
at
the
pipeline
part,
but
then
I
learned
as
well.
The
data
versioning
part
so
now
I'm
convinced
completely
about
DVC,
but
I
came
to
it
because
I
wanted
to
have
a
way
to
run
fewer
python.
B
Just
in
case
I
mean
this
one
percent,
income
compatibility
and
the
python
and
closes
I,
don't
do
the
job.
So
yes,
as
I
was
saying,
complex
python,
libraries
which
doing
optimize
number
crunching
are
maybe
in
the
most
unstable
the
most
difficult
for
the
python
CJ
enclosure
closes,
are
because,
yes,
we
have
native
C
code,
we
have
GPU,
we
have
multi-core,
we
have
distributed,
and
we
have
these
absolutely
relied
working
in
the
python
CJ
and
closes
I.
Think
is
a
big
challenge,
but
machine
learning
model
training
is
there.
B
So
the
training
part
of
machine
learning
is
often
native
code
GPU
multi-core
distributed,
because
we
want
to
have
these
things
fast,
so
there
we
should
use
DVC.
So,
overall,
as
it
was
presented
in
DVC,
the
whole
training
of
a
model
can
be
split
in
these
server
steps.
Each
step
is
a
skills,
a
shell
script,
as
we
see
so
the
execution
of
a
CSG
file
or
of
a
python
file
or
an
R
file,
there's
no
difference
for
DVC.
B
B
So,
for
example,
the
interop
which
I'm
going
to
talk
about
then
today
mainly
is
a
different
type
of
interoper.
It's
not
really
python,
CG
or
enclosed
closures
are.
It
is
to
have
the
different
programming
lens
in
the
same
pipeline,
so
closure
mainly
hopefully,
and
where
needed
the
other
length,
the
other
languages.
So
it
would
be
the
opposite
of
what
Santiago
was
presenting
now.
My
aim
would
be
to
have
a
pipeline
where
I
do
everything
in
closure,
except
the
few
things
which
I
cannot
do
in
closure.
B
For
example,
the
data
pre-processing
I
would
do
now
enclosure
because
a
tablecloths
or
that
Tech
data
set
there
are
full
feature
to
compare.
So
it's
basically
no
reason
not
to
use
those,
and
then
we
can,
for
example,
write
the
pre-processed
data
as
aerofiles,
but
not
to
CSV.
Csv
is
bad,
but
we
can
do
Arrow
files
and
then
the
next
step
is
a
python
script
which
reads
the
error
file,
and
then
it
does
the
thing
so
train
a
model,
for
example.
In
this
situation,
the
required
python
code
at
the
end
will
be
very
short.
B
I,
don't
know
which
don't
do
python
our
prefer
closure,
but
it
can
be
a
few
lines
probably
copied
from
somewhere,
because
if
you
go
for
any
example,
how
to
do
a
certain
model
training
in
Python
these
few
lines,
you
need
to
put
there
in
this
fight
everywhere.
There
always
so
there's
a
lot
of
examples
to
to
to
reuse,
so
you
can
basically
avoid
to
Learn
Python.
If
you
don't
want
to,
because.
H
F
Because
you
said
it
so
so
quickly,
it
doesn't
sound
like
a
big
deal,
but
the
what
you
say
here
write
the
people
says
data
as
error
files.
This
is
essentially
the
only
limitation
we
have
when
you're
doing
interrupt
with
this
DVC
model
with
DVC
framework
right,
because
as
long
as
the
as
a
different
language
you're
using
can
speak
the
same
file
format.
Yes,.
K
J
B
Yeah
yeah,
no,
it
is
true,
you
are
right
and
that
is
rather
new
because
let's
say
if
I
would
have
looked
at
that
two
years
ago,
there
was
a
certain
moment
where
I
did
not
find
a
data
format
except
CVS
CSV,
which
is
supported
by
the
three
languages
closure
python
R
there
was
none.
There
was
no
table
format
on
disk
supported
between
these
three
languages
very
recently
error,
so
you
did
either
CSV,
which
all
the
risks
that
the
different
CSV
implementations
do
things
slightly
different.
B
So
you
especially
on
text
files
they
operate,
but
at
least
since
a
year
or
something
this
is
about
to
be
solved,
and
even
arrow
is
very
efficient,
so
I
think,
but
you
are
perfectly
right.
I
said
it
in
a
line,
but
one
year
ago
or
two
years
ago,
this
would
have
been
a
more
of
a
headache.
Yes,
yes,
I
agree,
yes,
that
this
became
important
and
maybe
there's
even
a
goal.
Good
and
future
for
that
it
was
Arrow
supports
as
well
a
kind
of
in-memory
sharing
of
the
data,
so
I
think
very
soon.
B
It
will
be
even
possible
to
have
our
our
steps
in
in
different
shell
scripts,
but
they
point
to
the
same
memory
location.
So
we
don't
even
need
to
copy
the
data
I'm,
not
sure
if
that
is
already
working
like
that,
but
that
is
one
of
the
ideas
of
arrows.
So
basically
the
file
the
the
data
will
be
somewhere
in
the
realm
of
the
computer
towards
point
to
the
same
file
kind
of
in
memory.
So
you
don't
even
need
to
write
them
to
disk
anymore.
B
So
if
we
do
these
things,
then
we
are
100
compatible
with
standard
python
because
we
run
a
standard
python
interpreter.
So
we
are
get
it's
guaranteed
that
all
these
python
libraries
are
working
in
the
same
way
in
our
pulley
pipeline.
Then,
if
we
would
use
python
alone,
then
before
we
dive
a
little
bit
more
into
the
the
code,
I
don't
know
how
many
of
you
are
familiar
with
the
ski
closet,
metamorph
concept
of
pipelines.
B
We
use
in
both
the
word
pipelines,
I
I,
don't
want
to
give
an
introduction,
but
just
for
the
people
who
haven't
heard
it
so
these
ski
closure
machine
learning
library
which
which
I'm
developing
is
a
certain
kind
of
a
pipeline
as
well,
which
maybe
the
simplest
way
to
think
about
that.
It's
an
extension
of
the
closure
shredding
macro,
so
you
pass
a
kind
of
context
around
and
that
context
contains
the
Full
state
of
your
data
processing
in
the
full
state
is
not
only
the
data
file
but
as
well
some
auxiliary
data.
B
So
instead
of
saving
these
auxiliary
data
in
in
in
vast
enclosure,
you
can
save
them.
Inside
of
this
context,
she
passed
from
us
to
the
other.
That
is,
this
concept
of
of
a
ski
closure
ml
pipeline.
Very
I
just
want
to
compare
these
two
because
yeah.
There
are
two
different
things
with
the
same
name
pipeline,
but
they
are
technically
very,
very
unrelated,
so
they
have
basically
nothing
to
do,
which
is
either
just
both
code
pipelines.
B
We
need
to
be
careful
a
bit
there,
so
the
device
DVC
pipelines
as
explained
before
they
have
steps
and
each
step
is
to
sell
script
so
I
could
them
out
of
process
plan.
No,
because
there's
no
single
process
which
executes
the
pipeline
each
each
one
is
a
new
shell
script
and
they
communicate
between
each
other
by
writing
data
to
to
disk.
We
need
to
set
explicitly
the
dependencies
as
we
have
seen
it
and
then
DVC
decides,
but
to
what
to
run
in
which
order.
B
So
this
is
the
concept
of
the
of
this
dependency
graph.
Santiago
was
talking
about,
so
we
tell
the
thing
which
are
the
inputs
and
the
outputs
of
each
step.
We
can
write
the
steps
in
any
order
into
this
yaml
file
and
it
will
detect
the
order
and
it
will
detect
automatically
what
needs
to
be
rerun.
That
is
the
DVC
pipeline.
This
key
closure
ml
library
has
optionally
a
concept
of
metamorph
based
pipeline.
So
again
it's
a
kind
of
shredding
macro.
B
It
runs
in
process,
it
runs
always
the
full
pipeline
and
in
sequential
order
yeah.
So
these
two
have
nothing
to
do
which
one
one
or
the
other.
What
they
have
to
do
is
that
both
support
hyper
parameter,
search
of
an
ml
Pipeline
with
pre-processing
modeling
prediction
and
model
evaluation
steps,
so
you
can
either
do
that.
That
I
would
say
is
the
takeaway
from
that
in
in
machine
learning.
B
There
always
comes
this
problem
of
hyper
parameters
such
which
means
that
you
want
to
rerun
the
same
steps
over
and
over
and
over
with
slightly
different
parameters,
and
you
want
to
keep
somehow
track
which
has
the
best
Matrix
out
of
that,
and
they
are
in
both
pipeline
versions.
They
support
that,
but
in
very
different
forms.
One
is
in
process
sequential
the
other
one
is
out
of
process
non-sequential
explicit
dependencies
so,
but
they
both
are
meant
to
solve
the
same
issue.
B
This
hyper
parameter
search
of
of
an
ml
pipeline,
so
the
first
DVC
supports
this
explicitly.
It's
an
explicit
feature
of
the
tool,
because
it's
a
machine
learning
platform
or
machine
learning,
experimentation,
platform
out
of
process,
and
it's
basically
supported
by
run
these
DVC
pipeline
so
run
the
share
script
several
times
so
from
outside
I
tell
it.
The
parameters
in
a
certain
parameter
file
and
then
I
just
rerun
this
these
share
scripts
again
and
again
and
again
and
dvdc,
then
by
itself
it
starts
the
hyper
parameters
and
performance
matrices
persistently.
B
That
is
a
big
thing
here.
So
what
each
time
I
run
a
pipeline,
the
the
outputs
of
the
inputs,
but
as
well
the
the
hyper
parameters
I
have
used
and
the
the
model
performance
I
get
out
of
that
they
are
stored
automatically
by
DVC
I.
Don't
need
to
do
anything
for
that,
so
in
this.
In
this
context,
the
PVC
goes
under
this.
This
word
of
it's
a
machine
learning
experimenting
platform,
because
that
that
is
what
DVC
is.
It
is
yeah
a
machine
learning
experienting
platform.
B
There
are
several
for
that,
and
DVC
is
simply
one
of
them
and
ski
close
JL
pipelines.
On
the
other
side,
they
support
as
well
in
process
hyper
parameter
search,
but
this
is
not
persistent.
If
my
closure
to,
if
my
closure
process
ends,
everything
will
is
gone,
so
the
maximum
kind
of
I
can
do
it's
just
that
I
run
it,
and
then
it
prints
me
on
the
screen
which
we
are
the
best
hyper
parameters
and
then
I
can
keep
that
for
for
something
and
apply
it
somehow.
B
B
The
best
that
I
came
now
to
by
by
using
this
a
little
bit
is
the
best
practice.
I
think
is
not
to
combine
the
two
forms
of
pipeline
I
think
so
so
we
need
to
decide
if
we
start,
let's
say
a
new
machine
learning
thing
either.
I
do
metaphor:
metamorph
pipelines
and
I
I
know
that
I
do
in-process,
hyper
parameter,
search.
B
I,
know,
I,
know
that
I
do
closure
only
eventually
using
lip
pie
and
crg,
because
that
I
would
somehow
consider
still
in
process
with
in
process
or
I,
say
no
I
use
DVC
I
use
the
DVC
based
type
a
parameter,
search
and
I
do
a
because
I
probably
need
to
do
a
polyglot
pipeline
I.
Think
it's
a
decision
which
needs
to
be
taken
in
the
beginning
either
or
the
other
key
closure
ml
still
makes
sense
in
both
as
these
metamorph
pipeline
Concept
in
ski
glow,
jml
is
optional,
so
we
don't
need
to
use
it.
B
B
While
I
still
learn
machine
learning
as
a
as
a
concept,
then
maybe
I
don't
need
I,
don't
need
DVC,
but
if
I
want
to
produce
a
good
model
than
any
TVC,
and
because
this
Persistence
of
experiment
results
but
as
well,
the
other
features
of
DVC
like
the
data
versioning
and
the
versioning
of
of
the
code
it
does
as
well.
It
just
keeps
track
of
code
changes
that
the
San
Diego
didn't
mention,
so
DVC
keeps
track
of
code
changes,
data
changes,
hyper
parameter
changes
and
Metric
changes.
These
four
things
together.
B
It
keeps
track
of,
and
the
relationship
between
the
between
these
four
migration
from
one
to
the
other
approach
doesn't
work,
neither
in
the
sense
of
because
if
you
start
with
the
metamorph
pipeline
approach,
and
then
you
should
go
to
DVC,
you
need
to
rewrite
your
pipeline,
because
the
the
the
way
the
hyper
parameters
are
given
to
the
to
the
in
the
different
form
is
completely
different.
So,
to
a
large
extent,
you
need
to
change
the
the
what
you
did
before.
So
it
is
a.
It
is
a
decision
which
needs
to
be
taken
on
the
beginning.
B
B
Will
that
mean
that
I
stop
using
metamorph
pipe
metamorph
pipelines?
I
said
maybe
yes,
I
think
the
answer
is
yes,
because
the
the
big
selling
point
why
we
have
developed
metamorph
pipelines
was
for
doing
in
Pros.
It
says
hyper
parameter,
search,
yes,
but
but
with
DVC
we
don't
do
in-process
hyperbolement.
As
such,
we
do
out
of
process
hyper
parameter
such
so.
B
B
On
the
other
side,
it
doesn't
matter
in
the
approach
of
DVC.
These
things
they
somehow
run
in
the
background.
There's
a
concept
of
queuing.
There
will
be
soon
a
kind
of
distributed
running
of
these
things.
Yes,
there
is
a
performance
penalty,
but
I
think
in
practice.
You
get
you
press
a
button
to
execute
you
queue.
Then
you
go
to
better
when
you
have
your
results.
That
is
the
way
people
can
you
I
would
use
DVC.
B
So
I
will
probably
stop
to
a
certain
way
using
the
metamorph
pipelines,
because
I
will
do
now
DVC
from
always
from
First,
first
day
onwards,
there's
maybe
a
mixture
of
the
two,
because
there
is
something
in
DVC
of
a
feature
which
eventually
allows
to
mix.
That
is
a
good
idea.
I'm
not
sure
so.
Dvc
has
a
feature
to
register
persistently
in
process
created
snapshots.
So
we
what
we
can
do
in
DVC
as
well.
B
We
can
write
a
loop
in
closure
or
in
Java
over
our
hyper
parameters
and
then
in
after
we
have
done
one
model
training
in
the
loop.
We
somehow
we
write
some
marker
files
to
disk
and
then
DVC
was
somehow
detect.
It
now
should
register
this
step
into
its
registry
of
of
into
its
persistent
database
of
of
metrics.
B
So
it
is
a
little
bit
similar
to
what
deep
learning
libraries
do
in
any
case,
because
if
you
use
a
deep
learning
library
like
tensorflow,
they
always
have
this
concept
that,
in
a
training
run,
it
creates
several
several
versions
of
the
of
the
model.
It
writes
it
to
disk.
It
evaluates
the
hyper
meet
us,
so
it
dries
wild.
B
B
This
thing
requires
a
python
API
to
be
used.
This
is
first
class.
They
have
four
seen
to
do
that
in
all
languages.
By
writing.
Let's
say
these
marker
files
on
disk,
so
this
is
documented.
So
there's
a
certain
way.
You
need
to
write
certain
files
on
disk
and
this
would
work
as
well.
So
maybe
there's
a
way
to
combine
the
advantages
of
the
two,
so
the
in-process
pipelines
and
still
use
DVC.
F
All
right,
it's
just
because
in
in
what
we
do
in
my
company
is
actually
mixed
both.
So
we
are
not
using
metamorphosis
in
r,
but
we
run
the
hyper
parameter
search
Within
an
R
script,
while
everything
else
is
in
DVC
right.
So
we
still
do
everything
in
process
all
the
parts
that
take
a
lot
of
time
and
everything
else
is
just
externalized.
So
we
have
one
stage
which
is
just
hyper
parameter
search.
We
then
restore
a
file
with
the
results
of
that
and
any.
J
F
C
F
F
B
B
B
So
there's
no
price
in
it.
You
just
participate
for
for
kind
of
fun.
It
is
a
classical
supervised
text.
Classification
problem
binary
text
classification,
so
you,
if
you,
if
you
don't
know
the
training
data,
it
is
basically
the
text
and
the
the
class
assigned
to
it
real
or
fake,
and
then
you,
you
train
a
model
for
that
in
a
supervised
way,
and
the
model
should
then
predict
real
or
fake
on
unseen
and
see
text.
B
If
you
do
that
in
Python,
the
current
state
of
the
art
is
a
is
a
class
of
nap
models
which
are
called
Transformer
models,
yeah
deep
learning
models,
I'm
going
to
go
in
the
details
of
that
they're,
the
state
of
the
art
there's
now
the
the
there
came
with
10
tensorflow
or
another
python
based
deep
learning,
libraries
there's
now
a
rather
new
wrapper
around
that
of
our
own
distance,
which
is
simple
Transformers.
It's
called
The
Library.
B
They
are
eventually
Java
Solutions
upcoming
to
yeah,
because
this
Transformer
is,
let's
say
it's
a
fact
of
algorithms.
So
nobody
they
could
be
re-implemented
in
Java,
I
think
so
far.
Nobody,
nobody
did
that
yeah.
They
exist
in
in
Python
and
if
you
use
them
from
other
languages,
you
basically
use
wrapping
approaches,
so
I
think
R
can
use
them
as
well
via
the
the
r
python
integration
or
we
use.
We
can
use
them
potentially
in
on
the
gvm,
so
enclosure
with
slip,
python
COG,
but
I
was
that
was
this
model.
B
So
it's
just
this
python
Library
simple,
Transformers
I
was
in
the
one
percent
area
of
libraries
which
are
not
working
well
with
lipid
and
CLG
that
we
had
a
lot
of
debugging
sessions
with
Chris
Nuremberg.
On
that
we
we
came
to
somehow
a
stable
situation,
but
it
is
a
complicated.
It
is
a
difficult
library
for
to
get
it
fully
supported
with
lip.
B
Python's
ERG
and
the
what
you
get
as
failures
is
very
is
very
unsatisfactory
because
you
get
gdm
crashes,
so
there's
absolutely
no
way
unless
you
are
a
gvm
expert
to
even
start
to
debug
that
so
it
is
very,
very
nasty
Beast
and
the
ways
we
improve,
that
the
stability
of
that
that's
what
it
went
down
to
doing
things
with
with
the
jills
or
the
global
interpreter,
lock
of
python
disabled
a
multi-processor
do
certain
things
in
a
different
way.
And
finally,
it
boils
down
that
Lupine.
B
Crg
is
not
the
same
as
a
as
a
pure
python
interpreter
is.
Is
it
because
it
does
a
form
of
running
pipe
which
is
called
embedded
python,
so
other
people
use
as
well
embedded
python,
but
it
is
not
what
most
people
are
using
so
that
it's
just
not
a
lot
tested
so
I
have
a.
We
will
see
a
DVC
Pipeline
with
three
steps.
We
will
process
the
data
that
is
enclosure.
B
We
train
the
model
that
will
be
in
Python,
then
we
run
the
prediction
and
there
I
will
use
your
closure
using
lip,
pi
and
crg.
So
in
training
and
run
prediction,
it
will
both
be
the
simple
Transformers
library
but
the
third.
So
the
prediction
step
is
not
critical
at
all.
It
runs
very.
It
runs
without
any
problem
via
the
python
Cog.
It
was
the
training
it
was
not
working
well,
so
these
three
steps
they
communicate.
We
are
writing
reading
data
files
on
disk,
as
far
seen
by
DVC
voila
that
we
will
look
at
now.
J
B
Somebody
wants
to
comment
something
you
know
back
to
the
screen,
seeing
some
comments
here,
clarifying
with
arrow
hijack
this
feature
to
train
the
same
model
yeah
exactly
exactly
this
is
the
error.
Ones
was
asking
Arrow
Apache,
yes,
yeah,
okay,
there
no
other
comments.
Then
we
look
a
little
bit.
How
much
time
is
it
we
still
have
officially?
Yes,
we
are
fine
in
time,
27
minutes.
J
B
Okay,
so
we
will
start
by
looking
at
the
pipeline
the
concrete
pipeline
we
gonna
use
to
participate
in
this
cargo
competition.
So
we
have
the
stages.
As
said
before,
I
have
three
stages:
one
stage
is
pre-process,
so
this
is
closure.
It's
a
closure
script
doing
something.
I
said
that
the
dependencies
of
that,
because
I
need
to
tell
it
the
dependencies
and
the
odds,
so
it
can
construct
these
dependency
graphs.
B
Why
Arrow
yeah
it
was
this
thing
that
I
was
hope
that
I'm
hoping
I'm
still
convinced
that
to
working
with
arrow
between
between
closure
and
python
is
less
problematic
than
CSV,
because
CSV
at
the
end
is
untyped,
so
you're,
always
in
the
in
the
mercy
of
of
the
of
the
concrete
CSV
implementation.
If
writing
from
closure,
CSV
and
reading
that
from
python
will
not
lose
something
or
we
do
things
slightly
different
and
I.
Think
and
I
yeah
I
arrow
is
meant
to
solve
this
problem.
B
B
Yeah,
of
course,
dates
is
a
difficult
one
eventually,
but
it
is
meant
to
solve
this
problem,
so
I
I'm,
putting
a
lot
of
hope
that
over
over
the
next
time,
these
libraries
they
will
improve
as
well,
and
that
we
get
a
very,
very
good
interoperability
between
the
between
the
the
street
between
all
programming
languages.
Let's
say,
then:
we
have
a
train
step
since
we
run
a
python
process
here.
So
this
is
pure
python.
B
B
B
B
B
So
the
code
is
traced
on
two
levels,
either
as
jit
commits
and
on
this
level
and
there's
maybe
a
little
bit
of
things,
what
that
what?
But
it
works
together
with
jit
and
it
traces
and
it
traces
the
parameters
and
the
way
parameters
are
handled
in
DVC.
Is
that
that
doesn't
there's
a
convention
about
a
certain
file
which
need
to
exist
on
disk
and
the
default
name
is
yaml.
B
B
B
B
This
is
as
well
from
the
level
of
the
of
the
of
the
pipeline.
Here
is
it's
a
closure
call.
My
dependencies
are
the
is
the
output
right
and
it
is
the
code
itself
and
it
produces
a
file
which
which
I
can
send
direct
to
to
kaggle,
so
it
does
the
inference
of
the
it
does.
It
runs
the
prediction,
so
it
uses
the
model
which
was
trained
before
to
run
predictions
on
I
didn't
declare
a
dependency
to
the
cargo
thing.
That
is
a
good
point
here.
B
It's
not
important,
I
think
because
the
that
is
one
of
the
things
the
code
doesn't
do
automatically.
So,
if
I
open,
for
example,
this
was
pretty
cargo
I
think
it's
reading.
It's
writing.
It
reads
this
and
that
didn't
declare
that.
So
that
is
one
of
the
things.
Of
course
you
need
to
keep
aware
of.
It
will
not
detect
that
my
code
is
reading
this
file,
so
I
somehow
forgot
to
declare
it
here.
It
means
if
that
file
would
change.
It
would
not
rerun
the
code.
What
it
should
do
so
I
forgot
it's
test.coz.
K
B
C
K
J
B
No,
no,
no
I,
don't
need
no,
no
I
need
to,
because
I
only
need
the
model.
At
this
point
in
time,
I
read
the
model.
I
only
use
the
model.
I
don't
need
the
aerophytes
anymore
here,
because
this
contains
the
model
because
the
model
was
written
to
disk.
So
my
data
is
here:
I,
don't
need
them
here,
I
needed
I
need
it
I
need
it
here.
That's
correct
here:
I
need
them
here.
That
was
in
the
Train.
B
B
So,
let's
briefly
look
at
the
code.
That
is
very
short.
We
look
at
these
three
different
pieces,
so
we
have
these
pre-process
file,
which
is
com,
which
is
com,
which
is
only
closure
as
we
require.
We
only
need
a
tablecloth.
We
require
this
Arrow
Library
and
we
do
here.
This
is
the
main
method
which
is
called
from
my
thing
here,
so
here
I
call
it
this
function
here
and
this
function
is
this
one?
So
what
I
do
I
read
the
training,
data
and
yeah
I?
B
Have
here
not
a
very
good
naming
thing
I,
basically
from
kaggle
I
get
a
train
CSV
file,
but
the
training
I
need
the
train.
I
need
to
split
and
train
and
test.
We
always
need
to
be
confusing
with
the
names
here
now,
because
Kyrie
gives
me
two
fats.
It
gives
me
one
five,
which
contains
the
truth.
That
is
the
train
file
and
it
gives
me
another
file
on
which
I
need
to
predict.
B
B
J
B
I'm
getting
I
always
do
it
like
that.
I
always
use
this
thing
here.
So
we
see
it's
a
data
set.
It
contains
the
text
column.
So
we
are
an
LLP
problem
that
contains
two
other
two
other
columns,
because
the
text
they
have
a
location
and
a
keyword,
and
we
can
of
course
imagine
the
location
of
a
disaster
is
maybe
important
to
predict.
If
it's
a
fake
or
a
real
disaster,
it's
clear
if
the
location
would
be
Moon.
B
K
B
Data
you
see
here,
lots
of
it
so
can
I
see
them
here,
yeah.
So
there's
some
keywords
here:
some
people
gave
basically
keywords
to
their
to
their
tweets
because
it's
come
somewhere
from
Twitter
I,
don't
even
know
how
they
relate,
because
if
it's
Twitter
there's
only
the
tweet
anyway,
this
is
the
one
column
and
the
other
column
is
the
location.
B
There's
a
lot
of
missings
as
well.
I
know,
there's
no
missing.
No
sorry,
I'm
hoping
the
wrong
one.
That
was
the
text,
there's
no
business
here.
We
have
as
well
a
lot
of
missing
data,
and
if
it's
not
missing,
then
we
see
things
like
the
other
locations,
think
that
at
the
completely
random
stuff,
so
everybody
types
as
he
wants.
B
It
looks
to
me,
like
a
lot
of
garbage,
that
you
cannot
really
do
a
lot
of
that
and
the
only
thing
the
code
is
doing.
It
adds
them
to
the
text
because
I
do
a
simple
Transformer
model
and
the
simple
Transformer,
more
former
model
can
only
take
text
as
an
input.
One
text
thing
I
cannot
give
you
tables
or
something
a
simple
Transformer
model
needs
text.
So
what
I
do
I?
Basically,
add
these
two
columns
I
add
them
into
the
text.
B
So
my
final
text,
which
the
model
is
using
for
for
training,
is
the
treat
the
Tweet
text,
plus
this
keyword
and
the
location
I
just
append
it
I
append
it
to
the
text
and
then
I
hope
that
the
Transformer
model
will
use
it
for
for
something,
because
that
is
the
way
these
NLP
models
of
this
Transformer
class
work.
They
can
only
work
with
text,
they
cannot
work
with
other
features,
voila
and
then
I've
write
these
things
to
I.
Do
a
split,
I
split
this
train-
and
this
is
by
default,
2080
or
80.
B
B
Apply
this
is
now
using
the
simple
Transformer
Library.
So
in
my
if
I
run
this
code,
I
need
to
be
in
a
virtual
environment,
classical
python
stuff,
where
these
Library,
the
simple
Transformers
libraries
installed,
I,
need
to
use
pandas
and
Pi
Arrow
feather.
This
gives
the
support
of
reading
Arrow
files.
So,
but
then
I
read
here
my
yaml
things
here:
yeah.
B
Working
because
my
I
think
now
the
the
fights
are
in
Json
format,
somehow
I
I
swapped
there
something
anyway
the
code
here
it
would
use
these.
It
would
read
these
files
that
we
saw
before
this
poweramps
file
and
basically
it
would
take
all
the
all
the
parameters
from
here
and
pass
them
to
the
call
to
train
the
model.
B
So
I
read
here
this
I
I
read
you
these
arguments.
They
go
into
the
variable,
params
and
then
I
make
one
overall
map.
You
know
a
dictionary.
We
have
pythons.
We
talk
about
dictionaries.
I
have
some
other
parameters
which
I
don't
want
to
hyper
hyper.
They
are
not.
They
are
parameters
to
the
model
training,
but
I
don't
want
to
optimize
them.
B
One
part
of
the
parameters
is
which
model
am
I
using,
so
I
use
a
model
called
bet.
No
a
model
type
bet,
and
the
model
name
bet
tiny.
This
points
to
registered
models
in
kind
of
in
a
model
registry
which
comes
together
with
which
is
working
supported
by
tensorflow,
hugging
hugging
hugging,
face
Hub
or
something
there
are
thousands
of
models
there,
and
each
model
has
basically
a
name,
a
unique
identification.
B
So
in
that
way,
I
can
basically
try
a
lot
of
moderates
by
just
giving
put
in
there
named
it
and
the
library
does
everything
automatic
it
downloads,
the
model
which
might
be
gigabytes,
so
it
can
take
long
and
all
that,
but
it
does
that
all
automatic,
and
that
is
one
of
the
hyper
parameters
which
concrete
model
do
I
want
to
use,
do
I
want
to
use
bad
or
Alberta
or
any
of
those
voila,
and
then
I
run
my
training.
So
one
line
of
code
train
the
model
I
have
my
data
frame.
B
The
two
Arrow
files
is
one
for
training.
Data
and
I
evaluate
the
model
while
I
run
the
training,
so
I
have
evaluation
data
here.
So
again,
there's
this
confusion
about
the
do.
We
call
these
things:
evaluation,
test
or
validation
data.
There's
a
lot
of
there's,
no
there's,
not
a
clear
convention
how
these
things
should
be
called
and
I
get
an
evaluation
result,
which
is
basically
my
metrics
I,
have
chosen
here
at
a
very
specific
certain
metrics
to
measure
how
good
my
model
is,
and
that
gets
that
I
write
to
this
file.
B
B
B
So
this
all
this
is
produced
via
the
model
is
training,
so
this
is
the
model
five.
This
is
the
trend
model
as
such
a
binary
file
rather
big,
but
it
does
as
well
some
checkpointing
so
why
it's
running
it
writes
different
versions
of
the
model
into
different
directories,
just
in
case
it's
crashing,
although
you
don't
lose
all
of
the
all
of
the
work,
because
if
this
is
running
for
days,
if
it's
crashing
in
the
last
round,
then
at
least
we
have
the
models
of
before
it
always
saves
as
well.
B
The
evaluation
results
so
I
would
not
need
to
do
that.
In
any
case,
if
I
have
this
is
this
is
the
full
specification
of
the
model
which
was
created
so
if
I
want
to
use
some
python
code,
which
runs
predictions
I
need
to
give
it
this.
This
directories
all
these
files,
and
that
is
then
what
happens
in
the
last
step.
B
B
B
We
do
a
kind
of
requiring
here
in
this
form.
I,
don't
know
if
you
have
used
the
python
CG
before.
So,
if
you
did,
if
you
do
this
type
of
stuff,
your
python
API,
it
looks
like
closure
now,
we've
write
something
like
that,
but
in
reality
this
is
now
a
python,
a
python
class,
but
I
can
use
it
in
this
in
in
closure
in
closure
syntax.
So
what
this
code
is
doing?
It
reads
as
well.
These
parameters
I'm
just.
J
B
Yes,
I
need
to
get
the
model
type
or
the
top
of
it.
Okay.
What
happened
before
is
that
I
I
have
now
on
disk
I
have
my
train
model,
but
I
need
to
to
run
prediction.
I
need
to
read
it
from
disk,
so
I
need
to
create
an
instance
of
this
classification
model,
and
this
apparently
needs
this
model
type.
So
I
need
to
tell
it
the
type
of
the
model
which
would
be
bad
and
then
I
point
to
the
place
on
this
Square.
B
This
best
model
was
stored
by
the
training
run,
then
I
load,
my
training
data
I
need
to
rerun
the
same
pre-processing
no,
and
that
is
one
of
the
things
which
key
close
jml.
The
metamov
pipeline
helped
a
little
bit
for
that.
The
the
the
I
need
to
make
sure
here
that
I
run
exactly
the
same
pre-processing
on
this
data,
then
I
was
using
on
the
on
the
on
the
training
data.
B
Anyway,
now
I
have
my
my
test.
Data
and
I
just
run
on
this
python
model.
I
run
predict
with
the
test
data
and
I
get
my
prediction
yeah.
Here
we
are
restarting
the
Nifty
parts.
So
apparently,
there's
some
options
in
lipid
and
CLG
to
get
this
thing
more
stable
to
do
to
wrap
that
in
some
Azure
context
or
not
I,
don't
know
if
this
would
be
needed
here,
maybe
not
anyway.
I
have
now
my
prediction
and
I
write
a
prediction
to
my
file.
B
So
at
the
end
I
get
an
output
file
which
I
sent
to
cover,
so
I
send
this
file
to
kaggle,
and
then
it
will
give
me
a
score
on
the
leaderboard
because
it
will
basically
cargo
knows
the
truth.
All
of
this
I
make
a
prediction.
The
prediction
is
that
it's
the
type
or
it's
the
ID
of
the
text
somehow,
and
it
will
tell
me
where
I'm
on
the
leaderboard,
so
in
this
way
I
can
participate
fully
in
cargo
competitions
from.
B
B
Ir
there's
some
clothing
in
here.
That
is
one
of
the
features
which
is
supported
as
well
by
by
DVC,
so
it
can
plot
certain
things
which
you
always
use
in
machine
learning,
and
this
is
the
this
is
the
training
curve
do
I
have
that
on
disk
I
think
I
have.
If
I
can
have
a
look
at
that,
because
one
of
the
questions
we
always
get
at
the
end
of
machine
learning
that
my.
B
K
B
But
this
produces
a
learning
curve,
so
this
is.
This
is
where
this
is
the
way
how
we
data
scientists
now
say:
that's
these
models
generalize,
so
because
what
we
see
here
that
the
the
training
loss
goes
down
all
the
time,
but
this
is
normal.
If
the
model
architecture
is
complex
enough,
my
training
loss
will
always
go
down
to
zero,
but
at
the
certain
moment
we
will
start
to
overfit,
and
this
moment
this
moment
is
somewhere
here.
So
the
moment
the
evaluation
loss
goes
down
goes
up
again,
which
is
somehow
here
here.
B
It
starts
to
go
up
again
here
we
start
to
overfit
the
model,
so
that
means
that
the
very
best
model
is
not
the
one
which
the
lowest
training
loss,
but
it's
probably
this
one.
So
no
there's
some
ways
to
automatically
stop
the
training
at
these
points
in
time,
but
this
so
after
400
steps.
This
model
here
would
now
be
the
best
one
with
I'm,
not
sure,
if
I
saved
that,
hopefully
I
have
saved
it.
Normally
the
tensorflow
library
does
that
so
it
saves
every
step.
B
B
That
means
that
every
X
steps
it
runs
the
evaluation
we
see.
It
is
running
at
this
point
in
time
at
this
point
in
time
at
this
point
in
time.
At
this
point
in
time,
it
runs
the
evaluation
and
each
time
it
does
so,
it
saves
the
model
to
disk
and
the
best
model
of
all
of
that
will,
at
the
end
end
in
this
folder
base
model
right,
but
there
are
other
versions
of
the
model,
but
this
one
is
the
best.
So
this
is
some
way
of
of
interpreting
the
the
training.
J
B
D
I
know
that
DVC
has
a
couple
of
python
apis.
You
know
if
it's
got
python,
API
dtcc.api
to
access
parameters
and
things
like
that.
It's
also
got
a
DVC
live
API
to
do
plotting
and
things
I'm,
just
wondering
I
noticed
you
didn't
use
them.
I
was
wondering
if
that
is
that
for
any
particular
reason,
because
I
guess
you
could
use
them
through
lib,
python
closure
or
maybe
maybe
even
there's
a
nice
closure
wrapper
around
some
of
those
DVC
apis
I'm,
not
sure.
B
I'm
not
really
I'm
the
the
tool
by
itself;
it
is
command
line
oriented
now,
so
that
means
it.
It
doesn't
require
the
usage
of
these
python
libraries
why
they
are
then
even
there.
It
is
a
command
line
tool
and
it
is
com,
programming,
language
independent.
What
is
true,
I
can
only
imagine
that
there
are
some
libraries
which
basically
automate
the
command
line.
Interface
I'm,
not,
but
I
would
not
know
why
I
would
use
it.
B
I
mean
in
fact,
I
show
you
the
directory
I
use
DVC
in
this
way,
exp
run
that's
what
I
do
that
runs
the
experiments
now
or
I.
Do
things
like
PVC
exp
show.
This
will
show
me
the
main
purposes.
Yeah
so
I'm
wondering
what
the
what
the
other
one,
what
you
were
talked
about,
the
DVC
live,
that
is
a
commercial
tool
which
they
did.
The
company
makes
money
from,
but
it's
separated
from
the
open
source
DVC.
B
D
B
K
F
A
nearest
use
case
for
that
Ash,
which
is
deployments
in
production,
like
imagine,
if
you
have
you,
have
a
server
running
python,
deploying
your
models
and
you
have
DVC
Library
installed
there.
You
could,
theoretically
simply
send
a
command
like
your
request
and
say
this
is
the
model
that
I
need
you
to
run
and
your
server
would
actually
just
check.
Is
this
loaded?
No,
then
just
get
it
from
DVC,
because
then
you
can.
You
can
use
nothing.
This
is
a
good
idea,
but
you
could
technically
use
EVC
in
this
way
in
a
production
server.
F
So
just
continuously
update
your
model
by
using
their
API,
which
is
something
you
cannot
do
using
r
or
closure
today.
You
would
need
to
do
this
like
in
some
other
way,
but
but
that's
one
way:
I
guess
that
Ash
is
just
saying
using
the
API
from
from
python
or
away,
not
necessarily
by
replacing
what
you
get
from
the
command
line,
which
is
already
very,
very
extensive.
But
there
are
some
use
cases
which
you
know
it's
convenient
to
use
the
python,
API
I,
don't
think
I,
don't
use
it
either.
F
K
B
Yeah,
maybe
for
automation,
so
instead
of
you
know,
if
you
have
a
lot
of
qcli
commands
so
type
them
all
the
time,
instead
of
typing
them,
and
then
we
have
one
example
for
that,
where
I
did
something
like
that?
Yes,
that
might
be
the
use
case
for
the
API.
But
let's
say
by
principle
and
I
was
as
well
doing
some
comments.
There
I
think
they
have
really
the
aim
to
stay
programming.
B
Language
independent,
would
hope
so
and
I
would
hope
so
that
that
is
the
case
and
that
well
yeah,
and
that
is
via
the
command
line
interface.
So
the
commander
interface
is
the
official
entry
point
to
the
to
DVC.
Maybe
there
are
some
python
libraries
which
make
certain
things
easier
or
if
something
is
very
repetitive,
but
that
you
can
probably
do
as
well
with
bash
automation.
You
know
the
special
automation.
You
just
write
a
little
best
script,
which
does
stuff
yeah,
so
yeah
I
haven't
used
any
python
guy
for
for
DVC.
D
B
But
I
have
seen
now
that
you're
telling
me
there
is
a
one
way
to
say
the
parameters
is
a
simple
file
and
another
way
is
something
far
more
complex.
If
you
have
parameters
which
depend
on
each
other,
you
can
express
that
in
Python.
That
is
true
yeah.
So
you
can
basically
because
it's
true
that
the
parameters
as
they
have
worked
so
far.
You
cannot
say
this
parameter
should
be
five,
but
only
if
these
other
parameter
is
between
7
and
10..
Something
like
that.
B
You
cannot
express
in
these
Json
and
yes,
there's
a
way
to
have
the
parameter
file,
a
real
python
script.
That
is
true
yeah,
but
they
have
made
an
extension
there
which
allows
to
do
that
on
the
on
the
on
the
CLI
level,
and
it's
called
so.
This
Hydra
is
a
kind
of
specification
language,
and
that
would
be
again
without
python
because
that
you
would
again
do
on
the
pipe
on
the
level
of
the
of
the
level
of
the
CLI
or
some
specific
specification
files
which
I've
written
in
these
Hydra
language
or
something.
B
Okay,
if
they're
still
five
minutes,
I
want
to
show
some
fun
because,
regarding
the
hyper
parameter
search,
what
I
would
in
practice.
However,
practice
do
a
hyper
parameter
search
of
this.
That
would
basically,
if
I,
do
that
purely
by
the
command
line.
I,
don't
know
if
this
is
really
a
good
example
here,
but
let's
say
I
need
to
show
a
little
bit
too
at
this
time,
so
you're
going
to
see
the
command
line,
so
I
move.
B
So
these
hyper
parameters
it
be
it.
It
happens
very,
very
quick
by
only
having
two
three
or
five
hyper
parameters,
but
by
doing
let's
say
a
full
grid,
search
over
them
to
end
very
easily
in
hundreds
of
thousands
of
different
combinations,
and
let's
say
the
pure
command
line
way
to
do.
That
is
that
you
needed
to
type
100
times
here
run
and
between
all
these
runs.
B
You
need
to
change
the
parameter
file,
so
I
will
do
something
I,
let's
say:
I,
do
the
queue
now,
so
it
puts
it
in
the
queue
because
it
does
I.
If
I
do
the
DVC
XP
run,
it
runs
the
me
immediately,
so
I
need
to
wait.
I
can
put
it
in
a
queue,
so
it's
only
queued
for
running,
but
it
doesn't
execute
it
yet.
So
if
I
do
something
like
that,
it
returns
immediately
and
then
I
now
will
change
the
file
and
I
would
put
it
again
in
the
queue.
B
K
B
B
To
do
that,
100
times
is
already
boring,
no
100
times
here
the
file
and
then
or
for
the
the.
So
how
but
of
course,
so
how
can
we
automate
that
that
is
maybe
one
of
the
things
that
API
can
do,
and
but
I
can
do
that
as
well
in
closure
by
basically
automating
the
so
the
here
I
can
I.
Just
can
shell
out
to
this
road
I
can
just
shell
out
to
this
command
all
the
time.
B
It
looks
a
little
bit
like
that.
So
this
is
these
new
namespace,
which
is
not
published
yet
so
in
a
certain
way.
It
would
be.
It
would
be
secular,
jml,
supporting
DVC
for
this
feature
thing
in
here.
So
what
we
would
do
Let's
ignore
this
part.
We
don't
need
to
read
the
file,
so
what
I
can
do,
then?
In
practice,
I
can
Define
my
search
space
using
existing
features
of
tech
email.
They
were
already
existing
in
the
old
Tech
ml
Library,
which
is
these
grid
search
approach.
B
The
principle
is
that
it
divides
first,
these
search
space
in
big
steps,
so
it
even
though
it
has
defined
here
that
this
should
be
brought
down
in
100.
It
will
first
give
me
minus
one
and
one
which
is
the
two
extremes
and
that
will
split
it
in
the
middle
and
that
it
will
so
it
does.
Basically,
it
gives
me
first
the
splits
of
that
coarse
grained
before
it
goes
to
the
fine
grain,
so
that
means
in
practice.
B
I
can
take
the
first
X
of
these
hundred
thousand
and
they
cover
already
a
big
part
of
my
search
space.
So,
basically,
by
doing
if
by
doing
something
like
that,
so
I
run
these
functions,
so
ball
grid
search.
This
one
returns
me
now
a
list
of
maps,
so
I
can
just
do
that,
like
that
a
list
of
maps
where
this
basically
this
thing
is
very-
is
variating.
So
this
variable
here
goes
from
minus
one
to
one
the
other
Factor
Decay
rate.
B
It
will
go
for
minus
one
to
one,
and
this
learning
rate
goes
between
these
two,
but
first
in
the
big
steps
and
then
in
the
in
the
so
it's
a
little
bit
instead
of
doing
all
the
hundred
thousands,
which
would
be
a
full
Rich
search
over
my
over
my
space
of
of
parameters,
I
can
take
the
first
100
and
I
cover
the
most
important
part
of
my
grid
space.
So
there's
a
certain
optimization
here,
so
I
can
calculate
this
one.
B
So
then
I
get
10
I
have
now
10
Maps
a
vector
of
10
maps
in
here
in
this
one
I
can
then
send
to
this
function,
and
it
will
be
basically
the
one
that
I
showed
for
by
hand.
So
it
will
write
to
this
is.yaml
file,
which
is
the
content
of
that
clicking
one
value
here,
picking
one
value
there,
and
then
it
runs
this
command
at
the
result.
I
will
get
a
q
of
10
runs
with
the
adapted
parameters,
so
to
demonstrate
that
I
can
know
the
queue
give
us
DVC
Q.
B
B
B
Exactly
I
have
experimenting
that
before
so,
if
I
run
now
this,
it
will
always
write
the
part.
The
Yama,
the
parameter
file
and
queue
and
experiment
with
that
and
DVC
will
remember
all
that
it
takes
a
bit
because
it
really
runs
these
DVC
Q
command,
which
yeah,
it's
not
immediate.
So
to
do
that
for
10
000,
even
the
queuing
of
the
thing
would
take
a
few
hours,
but
now
I
have
a
queue
of
10
runs
and
I
can
run
them
in.
B
B
B
D
K
K
A
A
A
A
All
of
the
stuff
we've
talked
about
will
be
like
usually
we'll
post
a
summary
of
the
these
meetings
on
the
somewhere
on
the
cyclist
website.
Probably
on
the
closure.
Verse
event
would
be
one
place
to
check
we'll
like
leave
a
comment
or
something
where
you
can
find
the
recording
of
this
and
whatever
other
resources
we
talked
about
or
used
and
yeah.
If
yeah
we'll
see
okay
studying
thanks
so
much
for
joining
us
yeah.