►
Description
Presented by Cristóbal Valenzuela Barrera, and Anastasis Germanidis
Explore and experiment with the creative possibilities of machine
learning with this hands-on session exploring how to use and train your
own models with RunwayML.
Cristóbal Valenzuela is a Chilean-born technologist and software
developer. He is a co-founder of RunwayML. Previously, he was a
researcher at New York University mainly working on the development of
ml5.js.
Anastasis Germanidis is a software engineer and artist interested in
generative media. He's currently co-founder and CTO at Runway, where
he’s focusing on making machine learning accessible to designers,
artists, and other creators. His artwork has been shown internationally
across the US and Europe, including at Ars Electronica Festival,
Festival de Cannes - Le Marché du Film, Megaron - The Athens Concert
Hall, and the The Museum of Moving Image, and featured in The Telegraph,
Die Zeit, WIRED, Fast Company, and Mashable, among other outlets.
GitHub Satellite: A community connected by code
On May 6th, we threw a free virtual event featuring developers working together on the world’s software, announcements from the GitHub team, and inspiring performances by artists who code.
More information: https://githubsatellite.com
Schedule: https://githubsatellite.com/schedule/
A
A
Today,
we're
gonna
show
you
some
very
cool
stuff
with
runway.
Now,
if
you
do
no
runway
mail,
you
can
head
down
to
run,
Rimmel
become
and
done
with
this
offer.
Today,
we're
gonna
be
playing
with
a
lot
of
creative
things.
You
can
do
with
mature
learning,
Widow
without
curry,
Kogan
experience
and
so
to
get
things
started.
If
you
don't
have
runway,
you
can
just
go
ahead
and
download
runway
from
runway
melt,
calm
until
Iran
that
down
better.
A
So
today,
we're
gonna
go
over
kind
of
like
the
overview
of
runway,
using
some
of
the
few
models
that
we
have
in
the
platform
and
then
we're
actually
gonna
train
our
own
custom
models
using
using
the
platform
so
to
get
them
started,
make
sure
to
download
runway
and
I'm
gonna
go
ahead.
I
already,
don't
know
it
so
I'm
gonna
go
ahead
and
start
this
runway
is
the
desktop
application
that
runs
on
all
platforms,
so
you
have
Mac
Linux.
A
It
will
be
available
for
you
and
once
you
download,
once
you
create
an
account
and
sign
in
you'll,
see
this
I'm
gonna
I'm
gonna
explain
what
all
these
are
on
during
the
session,
but
to
get
things
started.
First,
I'll
go
into
the
model
directory,
so
so
the
model
directory
is
basically
a
space
for
you
to
browse
and
discover
a
very
peripheral,
interesting
machine
learning
models
that
you
can
use
for
a
variety
of
different
purposes.
So
we
have
152
models
right
here
and
you
can
browse
them
over
here.
A
There
are
for
a
lot
of
different
use
cases.
You
can
filter
them
by
case
the
community.
For
instance,
image
synthesis
motion
capture
text
base
models.
You
can
use
recognition,
you
can
search
by
data
types,
the
way
you
fit
data
into
the
models
or
by
the
ones
that
are
trending
or
most
popular.
This
is
a
very
cool
space
if
you
want
to
just
like
discover,
what's
possible
if
you're
new
to
my
to
learning
and
you
wanna
like
learn
about
of
the
range
of
possibilities
of
things
you
can
do
so
will
encourage
everyone
to
check
this.
A
If
you
want
to
like
understand
most
of
all
things
you
can
do
with
my
to
learning
and
if
you're
really
familiar
with
the
topic,
if
you
bring
know
what
kind
of
like
what
you
can
do
with
ml
this,
some
of
the
models
inside
or
inside
here
will
be
very
familiar
to
you.
So
today
we're
going
to
demo
two
models,
specifically
we're
gonna,
be
demoing
image,
generative
model
that
allows
you
to
create
custom,
generative
images
and
we're
gonna
be
demoing
text
generation
model
that
allows
you
to
create
custom
tags,
and
so
it's
gonna
be
super
fun.
A
So
let
me
show
you
one
one
of
the
models
that
I
run
a
demo
today.
This
is
a
model
I
trained
on
runway,
so
you
can
search
here.
This
is
a
model.
Next
click
here
you
can
see
I'm
learning
more
about
the
model
itself
you
can
learn
about
who
made
it
in
this
case,
I
did
some
tribution
around
kind
of
like
model
itself
in
kind,
that's
kind
of
the
architecture
that's
in
use.
You
can
see
a
lot
of
other
stuff,
the
repo
itself,
so
every
model
has
a
corresponding
github
repo.
A
So,
just
to
kind
of
like
demo
how
this
works-
and
you
know,
you'll
get
a
better
sense
once
it's
running,
I'm
gonna.
Add
this
to
a
new
workspace,
so
I'm
gonna
go
by
the
default
name,
really
good
name
and
now
I
mean
runway
stuff,
like
workspace
view,
and
this
is
the
main
view
of
how
you
use
this
material
models
in
runway,
and
so
the
first
thing
I
did
was
I
went
ahead
and
click
start,
and
what
this
does
is
basically
starts
career,
running
the
session
or
running
the
model.
A
For
you
to
start
using
and
now
once
the
model
is
it's
ready,
we're
gonna
go
ahead
and
select,
vector
and
vector
is
the
way
you
interact
with
the
model.
As
so,
as
you
know,
every
single
model
has
different
ways
of
interacting
with
you
with
it,
and
this
is
specifically
one
very
particular
one
that
we're
super
excited.
That's
different
from
any
other
kind
of
like
interface,
that
I'll,
let
honest
s
is
explain
how
it
works.
Yeah.
B
In
this
case,
the
develop
students
serve
each
tile
in
this
vector
grid
corresponds
to
a
different
vector
that
we
passed.
The
models
generate
a
different
image
and
tiles
that
are
close
together
correspond
to
vectors
that
are
close
together.
So
that
means
that
they
generate
outputs,
are
similar
to
each
other
and
do
further
build
an.
B
B
We
will
see
that
the
output
saturated
by
the
model
are
very
similar
to
each
other,
but
if
we
slightly
increase
that
value,
we
can
get
still
similar,
but
much
more
different
include
some
before
and
this
way
we
can
associate
li
explorer.
I
so
I
certainly
explore
the
space
of
possible
image
from
model.
We
can
start
from
an
image
that
looks
interesting
and
then
find
images
that
look
similar
to
that
originally
mentioned.
Associatively
I
arrive
at
the
Altamonte
myself.
A
Thanks
for
that,
intro
that
was
great
until
explaining
how
the
late
an
expert
that
we
haven't
run
works
I
would
encourage
everyone
to
to
play
with
it,
explore
it
look
just
like
zooming
in
and
out
and
seeing
the
possible
and
rate
of
images.
Again.
These
are
all
generated
image
by
the
model
that
we
train
them.
They
were
actually
gonna
train
a
bit,
so
you
can
see
how
they're
works.
A
A
I'm
gonna
go
ahead
and
show
you
guys
this
and
so
you're
familiar
to
the
mature
learning
world
you'll
probably
have
her
about
GPT.
It's
a
model
that
you're
opening
I
will
is
I
think
almost
a
year
ago,
or
so
it's
very
good
at
creating
text
generating
tax,
it's
another
language
processing
model,
and
in
this
case,
what
you
can
do
in
wrong
ways.
A
You
can
retrain
that
GPT
model,
with
a
specific
data
set
that
you
have,
and
in
this
case
we
kind
of
like
to
get
for
a
different
spin
and
we
use
the
g2
model,
but
instead
of
training
like
normal
text
or
English
text,
we
train
it
on
JavaScript
code,
and
this
is
a
really
interesting
data
set
that
we
actually
took
from
github,
which
is
reco
they
gave
here,
is
also
gonna.
Do
and
I
studies
I,
think
you
have
more
more
information
about
the
co
search
net
that
was
used
to
train
this
model.
Ma'am
yeah.
B
But
we're
gonna
do
something
different
and
kind
of
use.
This
data
set
not
for
its
intended
original
purpose,
just
by
taking
all
the
code
snippets
from
the
data
set
and
then
training
a
text
animation
model
on
those
code
snippets
in
order
to
generate
new
code
snippets
that
hopefully
are
syntactically
Colet
and
synthetic
incorrect.
A
So
with
that
in
mind,
I'm
gonna
go
here:
I'm
gonna
show
you
how
it
works
first
and
then
I'm
gonna
go
ahead
and
train
these
from
scratch.
So
you
can
follow
along
and
try
this
on
your
own
I'm
gonna
first
write,
so
you
can
see
how
we
can
play
with
this
model
once
it's
trained,
so
you
see
here
you
get
the
idea,
we're
like
piling
up
or
grouping
up
models.
This
is
the
one
way
we're
using
before
I'm
gonna
click
run
here
and
now.
A
We're
gonna
have
a
different
interface,
and
this
is
very
cool
for
runway,
because
every
model
has
a
different
set
of
parameters
in
the
different
ways,
different
ways
of
using
those
and
as
as
you
know,
in
the
model
directory,
there
are
models
for
a
variety
of
different
things
and
use
cases.
You
can
find
models
for
object.
Detection
will
like
we'll
use
images
to
kind
of
like
detect
objects
in
the
scenes.
A
A
B
A
This
is
one
thing
one
thing
I
will
mention
eventually
or
gonna
mention
it
now.
At
some
point
is
this:
you
can
interact
with
models
through
runway
through
this
visual
interface
or
this
do
we
we
don't
need
into
kind
of
like
program
or
access
anything,
but
you
can
also
have
a
programmatic
API,
so
you
can
deploy
this
model
through
runway
and
manage
it
through
through
a
remote
endpoints.
You
can
string
data
in
and
out
of
runway.
This
allows
you
to
create
very
interesting
or
complete
creative
applications
of
Blatz,
slugbots
or
I.
A
Don't
know
the
Twitter
box,
or
maybe
you
can
like,
send
the
string
and
the
body
will
reply
with
something,
and
this
is
for
every
single
model
main
runway.
Every
single,
modern
runway
allows
you
to
host
it
in
a
network
and
use
it
through
an
API
am
so
that's.
That's
very
cool.
I'm
gonna
show
that
if
we
have
time
for
that,
I'm
gonna
go
ahead
now
and
stop
this
gp2
model
and
I'm
gonna
actually
show
you
how
to
train
these
two
models
in
runway,
and
so
in
assesses.
B
B
We
have
the
preprocessdataset
functionality
that
it
basically
allows
you
to
crop
the
images
in
your
data
set
to
to
align
them
together,
so
that
the
stock
and
we
have
to
deal
with
mass
variety
and
in
the
data
solidity
strain
with
stock
and
the
outputs
of
Soglin
tend
to
be
tend
to
have
a
lot
to
do
with
how
consistent
the
data
said.
The
original
dataset.
A
A
B
A
And
this
is
this
is
basically
saving
you
a
lot
of
time.
If
you
have
a
large
data
set,
you
need
to
kind
of,
like
preprocessor,
select
specific
regions
of
interest
for
you
to
train
a
model.
You
can
use
the
alter
curve
function.
To
basically
do
that
process
automatically
for
you,
so
I'm
gonna
go
ahead
and
choose
the
since
we're
gonna,
recreate
the
sneakers
models
that
we
trained
to
generate
new
imager,
so
I'm
gonna
go
ahead
and
select
as
an
assessment
Stein.
A
B
B
B
So
this
determines
how
long
we
will
perform
training
for
so
I
leave
said
in
the
training.
The
model
slightly
becomes
better
at
understanding
space
of
images.
That
straight
up,
specifically
in
this
case,
Salman,
is
what's
called
genitive
adversarial
Network,
and
this
means
that
we're
training,
two
models
with
training,
a
generator
in
the
screen
never
and
we,
and
during
the
course
of
training
the
generator,
tries
to
create
fake
images
that
look
like
damaged
in
the
dataset.
B
Well,
the
discriminator
tries
to
distinguish
between
images
that
are
created
by
the
generator
and
images
that
are
in
the
original
dataset
and
through
this
iterative
game,
both
the
generator
becomes
better
at
generating
images
that
look
more
and
more
like
the
original
image
of
the
dataset
and
a
discriminator
becomes
better
at
telling,
apart
which
he
must
have
faked
and
which
were
out.
Indeed,
we
finally
only
use
the
generator,
but
this
process
allows
us
this
process.
Odd
result.
Training
has
created
some
of
the
most
impressive
results
in
recent
years
in
generating
images,
text
or
audio.
A
Awesome,
thank
you
for
that
intro.
So
now,
with
that
in
mind,
I'm
gonna
go
ahead
and
click
start
training
cool
and
now
we
are
basically
training
our
own
gun,
our
own
stall
gun
model
that
we'll
be
able
to
generate
new
images
based
on
our
data
set
and
the
patterns
that
it
finds
in
the
data
set.
So
as
an
assassin
was
mentioning,
this
might
actually
take
some
hours,
I
think
for
3,000
steps.
It
takes
like
2
hours
to
process
or
comes
like
train.
A
So
since
this
is
a
cooking
show,
we
already
have
the
bake
ready,
so
I'm
gonna
go
ahead
and
show
you
how
this
looks
after
has
been
trained.
I'm
gonna
have
some
fun
exploring
how
it
works,
so
this
is
model
that
we
trained,
but
after
he
has
it's
finished
the
process.
So
you
see
here,
we
have
3000
steps
completed.
This
is
the
fit
score,
which
is
an
indicator
of
how
well
the
model
perform
unless
I
said.
I
maybe
want
to
mention
anything
here.
B
A
When
you're
training,
it's
always
important
to
look
at
this
number
to
see
how
good
your
model
is
being
is
performing
on
your
data
set
and
one
very
cool
thing
that
you
can
do
REME
is
you
can
monitor
the
progress
of
how
the
model
is
actually
going
through
the
face
of
training
and
going
through
the
samples
of
your
idea
to
understand?
What's
in
it,
and
what
you
can
see
here
is
that
remember.
We
started
before
our
actually
trained
intern
that
were
running
here.
A
We
started
with
a
checkpoint
of
faces
because
an
Assessor
was
saying
that
was
just
easier
and
more
kind
of
like
faster
to
train
in
this
case.
I
started
this
model
instead
of
faces
with
a
checkpoint
of
cars,
and
these
are
generated
cards.
So
what
you
see
here
is
the
progression
of
the
model
translating
from
cars
into
sneakers,
which
is
which
is
I.
I,
always
find
really
really
compelling.
I'm
interesting
to
export
have
the
visual
exploration
of
transferring
from
one
so
much
into
another.
B
So
this
is
a
what's
in
the
interpolation,
and
the
idea
is
that
we,
as
we
explained
before
stager,
uses
a
random
vector
to
generate
outputs.
So
what
we
do
is
we
start
with
from
from
a
random
vector?
Then
we
slowly
tweak
it
just
to
get
like
consecutive
frames
are
of
letter
of
similar
images,
starting
from
one
vector
to
another
vector,
and
then
we
explore
the
entire
space
between
those
vectors,
and
this
leads
to
a
very
interesting
exploration
of
all
the
of
all
the
images.
A
And
this
is
always
I
always
like
to
look
at
this
when
training
I
feel
like
it
percents
how
well
your
model
data,
it's
really
cool,
to
see
those
animated
transitions,
so
you
can
say
that
model
expert
with
it
and
you
can
also
view
the
model
and
use
it.
So
as
we
started
the
session,
I
show
you
all
how
to
use
this
model,
so
this
is
actually
the
model
who
trained
on
the
model
we
just
before
to
generate
these
sneakers
that
you
saw
over
here.
A
Okay,
we
have
like
around
8
more
minutes,
so
I'm
gonna
go
ahead
and
show
you
guys
how
to
train
the
JavaScript
generator
that
we
showed
you
before
this
one
again
from
scratch.
Let's
go
create
new
and
in
this
case,
I'm
going
to
click
here,
train
a
new
text
model.
Let's
go
by
the
defaults,
you
can
always
change
the
name.
I
never
do
so
now.
A
In
this
case,
we
have
represented
with
a
different
set
of
options
for
data
sets
remember
before
we
had
only
images,
in
this
case
your
training
at
space
model
or
an
NLP
based
model.
So
we're
not
gonna
see
this
thing.
That's
that
we
saw
over
here
when
we're
training
this,
because
these
data
sets
are
not
gonna,
be
able
to
fit
into
the
model
for
tax,
so
I'm
gonna
go
ahead
and
use
the
JavaScript
code
school
search
that
we
talked
about
before
you
can
see
here
a
bunch
of
samples,
so
this
is
real
JavaScript.
A
This
is
the
source
text.
The
models
using
to
to
Train
I'm
gonna
go
ahead
and,
in
the
same
manner
that
we
did
with
stalkin
before
I'm
gonna
go
ahead
and
click
Next.
I
know
this
is
different.
Now,
when
Isis
is
we
see
a
bunch
of
different
new
options
over
here
we
have
a
prompt
selection
and
an
initial
pre-trained
model.
Should
we
explain
what
this
is
doing.
B
Yeah,
so,
in
a
similar
manner
with
how
we
started
training,
we
also
using
transfer
learning
in
this
case
and
transfer
learning
again
means
starting
from
a
model,
has
been
trained
on
the
really
large
corpus
data
and
then
fine-tuning
it
in
our
as
much
smaller
hope
that
we
provide
specifically,
in
this
case
original
52.
More
model
was
trained
on
8
million
different
webpages
across
a
script
from
across
the
web,
and
that's
too
many
to
complete
and
it
cost
some
something
on
the
order
of
tens
of
thousands
of
dollars.
B
Specifically
here
there
is
a
few
different
models
and
the
main
difference
of
them
is
that
the
number
of
parameters
that
they
have
and,
generally
speaking
and
smaller
the
model
master,
it
is
to
train,
like
each
steps
instead
takes
less
time,
but
the
more
parameters
the
model
has,
then
the
more
the
better
it
becomes
with
an
understanding
and
presenting
the
rich
data
set
and
the
better
the
results
tend
to
be
so.
In
this
case,
we
can
just
pick
a
medium
because
we're
interested
in
getting
the
best
possible
results.
A
B
Yeah,
so
these
prompt
text,
people
use
as
a
way
for
us
to
understand
how
the
model
is
getting
better.
We'll
just
give
you
the
same,
prompt
at
different
steps
and
see
how
it
becomes
better
at
completing
that
prompt.
So
we're
gonna
pass
that
same
initial,
prompt
to
the
model
at
different
steps
and
then
be
able
to
look
over
time.
How
it's
becoming
better
at
generating
Texas
closer
to
the
result
is
exactly.
A
Coupe,
so
with
that
in
mind,
I'm
gonna
just
go
ahead
and
click
start
training,
and
now
we
are
training.
Our
own
gptt
model
train
our
own
JavaScript
data,
set
that
we
took
actually
from
over
here
again,
if
you're
just
joining
this,
we
are
showing
how
to
train
your
own
gbt
to
model
or
text
generation
models
to
runway,
using
the
training
model
interface
that
we
have
right
here
which
allows
you
to
do
all
kind
of
interesting
stuff
with
or
without
coding.
A
So
again
we
are
showing
you
this
and
it
might
take
some
time
I
think
for
this.
It
might
take
like
an
hour
so
two
hours
you
need.
We
only
gave
it
like
a
thousand
steps,
but
you
can
always
kind
of
like
monitor
this.
Also,
you
can
just
close
run
me
and
come
back
to
it
later
and
you
get
an
email
once
the
model
is
ready
and
you
can
just
use
it
and
I'm
gonna
go
here.
I'm
gonna,
go
to
you.
I'm
gonna
show
you
the
final
result.
A
A
This
is
an
indicator
of
how
well
the
model
did
over
the
training
session,
and
you
can
see
here
it's.
This
is
the
the
prom
that
we
gave
it
at
the
very
beginning
from
Kapoor's.
This
is
a
text
from
shows,
and
this
is
the
result
of
the
journey
model.
I
mean.
Actually,
this
is
nice.
I
added
like
comments,
inline
comments
to
the
model
I
mean
it
kind
of
like
matches
some
of
the
actions
functions.
So
that's
that's
interesting
to
see
some
of
the
semantics
of
the
model
understands
the
data.
B
A
We
have
two
more
minutes
left
and
before
we
go
I
want
to
show
you
guys,
another
very
cool
feature
that
we
recent
really
amazing
runway,
which
is
which
is
like
this
allows
you
to
interact
with
runway
through
all
the
models
that
we
have
in
the
platform
through
a
GUI,
but
as
I
mentioned
early
on,
you
also
have
this.
So
let's
say
we
train
the
GPT
to
JavaScript
models,
and
now
we
have
our
custom
modeled
Jared
code,
I'm
gonna,
keep
here
close
this
model,
and
this
is
super
interesting.
What's
what
this
is
doing?
A
It's
basically
allowing
you
to
host
this
model
through
a
remote
endpoints.
You
can
see
here,
I
can
choose
whatever
I
want
JavaScript
generator,
and
this
will
allow
us
to
create
a
remote
endpoint
that
we
can
connect
through
any
application.
A
web
application,
a
phone
application
on
iOS
Android,
a
node
server,
any
kind
of
application
that
you
have.
You
can
stream
data
in
and
out
of
that,
so
I'm
gonna
go
here
and
if
you
Coast
and
now
you
can
see
here,
I
have
my
model
being
hosted
on
the
Internet
to
a
permanent
URL.
A
You
can
configure
how
this
works
over
here.
You
can
set
it
up
and
there's
a
lot
more
to
explore.
I
think
we
ran
out
of
time,
but
please
feel
free
to
reach
out
to
us.
If
you
want
to
explore
to
learn
more
about
hoodies
runway,
make
sure
to
download
it
into
so
start
playing
with
it,
and
if
you
have
any
questions,
just
feel
free
to
reach
out
to
us
at
any
point,
and
if
this
is,
do
you
wanna
say
something
with
you
before
we
choose
it?
Thank.