►
Description
Presented by Yangqing Jia, Director, Facebook AI Infrastructure (Facebook).
One of the greatest advantages for leaders in AI innovation comes from the ability to quickly bring cutting-edge research to large-scale production. Earlier this year, Facebook announced the roadmap for PyTorch 1.0. Continuing in the open source spirit of its predecessors, PyTorch 1.0 is being developed completely in the open with contributors from major academic and industry research institutes, utilizing open source platforms and tools such as GitHub, Jenkins, Conda, and Python PIP. PyTorch 1.0 combines the scalability of Caffe2 with PyTorch’s flexible front end to create a multi-faceted deep learning system that delivers an accelerated end-to-end research-to-production experience. Come learn about PyTorch 1.0, and how it can help you embed AI into your workflow with strong GitHub community support.
About GitHub Universe:
GitHub Universe is a two-day conference dedicated to the creativity and curiosity of the largest software community in the world. Sessions cover topics from team culture to open source software across industries and technologies.
For more information on GitHub Universe, check the website:
https://githubuniverse.com
A
And
thanks
so
much
guys-
and
it's
really
great
to
be
here
and
it's
great
to
be
talking
a
target
up
universe,
I've
been
a
github
user
for
since
grad
school
and
it's
really
great
to
be
given
a
talk
here
and
so
I
worked
in
the
eye,
and
then
you
know
like
before
we
talked
about
AI
can't
want
to
share
one
story.
So
what
you
seen
here
is
Sir
Alexander
Fleming.
His
discovery
of
penicillin
is
probably
one
of
the
greatest
manifestation
of
human
intelligence.
A
He,
basically
that
has
saved
millions
of
life,
and
then
it's
really
thrilling
for
me
today
to
see
that
AI
is
actually
starting
to
change
the
way
we
think
about
how
to
do
drug
discovery
and
all
those
kind
of
a
conventional
industry.
We
have
been
pushing
the
human
intelligence
to
such
a
level
of
that
since
become
so
complex.
We
cannot
really
start
reasoning
and
stuff
like
that,
and
this
is
where
AI
can
basically
help
quite
a
bit
here
and
over
the
last
few
years,
we've
heard
a
lot
about
AI.
A
It
has
become
an
essential
tool
for
many
fields
in
our
life,
but
young,
just
computer
science
and
images
and
videos
and
stuff
like
that,
and
that's
one
of
the
really
exciting
things
to
work
in
this
field
and
one
of
the
things
that's
pretty
interesting
is
we've
heard
about
AI
for
such
a
long
time,
but
how
is
AI
changing
now
software
engineering
before
we
go
to
research?
I,
don't
want
to
talk
a
little
bit
about
the
implications
of
that
now.
A
So
one
of
the
classical
form
of
computer
vision
is
how
to
classify
objects
in
the
images
now
in
a
conventional
sense.
What
we
do
is
we
try
to
mimic
what
human
brain
does
or
what
we
do
by
writing
logic,
and
we
know
that
edges
like
if
there's
a
white
pixel
there's
a
black
pixel,
there's
an
edge
edges
are
really
important
features
and
we
in
the
early
days
of
computer
vision.
A
We
basically
started
trying
to
write
logic
that
allows
us
to
extract
edges,
get
some
local
statistics
out
and
use
those
kind
of
features,
primitive
features
built
with
logic
to
do
classifications.
So
now,
if
you
look
really
carefully,
you
can
kind
of
see
this
round
stuff
that
we
always
like
and
like
trunks
and
things
like
that.
So
it's
a
reasonable
feature
and
that's
as
good
as
we
can
do
with
logic.
A
Don't
really
need
to
write
all
those
logic
and
essentially
what
we
are
doing
for
edge
detection
is
basically
slide
a
little.
What
we
call
filter
across
the
whole
image
and
George
of
trying
to
find
some
local
characteristics,
whether
it's
an
edge
on
distraction
or
that
direction
or
is
a
color
contrast
and
things
like
that,
and
it
was
like
why
don't
I
build
a
more
deeper
structure
by
stacking
these
kind
of
commutations
one
by
one.
A
We
later
call
those
convolution
and
pull
in
and
stuff
like
that,
and
it
was
like
why
don't
we
just
throw
a
bunch
of
data
in
it
and
let
the
quote-unquote
training
algorithm
figure
out
what
kind
of
things
there
should
be
and
so
that
design
kind
of
stuck
and-
and
we
got
pretty
good
progress.
On
top
of
that.
Essentially,
we
stopped
writing
logic
and
we
start
to
basically
write
in
a
general
schema
to
say
what
the
computation
should
look
like
and
have
the
computers,
together
with
data,
to
figure
out
what
exactly
there
are.
A
Well,
not
very
surprisingly,
this
actually
maps
to
our
human
brain.
Quite
well,
because
when
we
look
at
the
neurons
that
are
in
our
brain,
so
we
by
looking
at
stuff,
we
we
first
reached
retina
and
then
immediately
ritenour
goes
to
a
region
what
we
call
LGN.
That
starts
basically
using
neurons
to
process
information.
What
happens
over?
A
There
is
basically,
the
neurons
are
forming
a
natural
layered
structure
where
the
neuron
in
the
previous
layer
process
the
input
from
the
from
lower
and
then
passes
on
the
Distilled
information
to
the
other
layers
and
stuff
like
that
and
is
kind
of
similar
to
what
we
observe
in
our
deep
neural
networks
that
we
do
association
with.
So
we
know
that
in
human
brains,
actually,
these
are
pretty
popular
like
in
across
all
the
stack
and
that
helps
us
to
recognize
objects.
A
It
got
us
a
pretty
good
result,
so
is
essentially
with
hard-coded
features,
were
able
to
get
about
26
percent
that
has
been
plying
for
quite
a
bit
and
then
was
alex
net,
which
is
the
new
new
net
structure
were
able
to
break
a
break
yet
really
well.
Essentially,
what
we
found
is
that
in
the
AI
field,
we
start
basically
replacing
the
logic
that
we
had
in
the
early
days
to
basically
use
a
more
adaptive
approaches,
write
a
model,
half
d
train,
and
I
wouldn't
figure
it
out
now.
A
That
leads
to
the
next
question,
which
is
where
writing
code.
In
the
past.
We
take
the
code,
we
compile
it
and
that's
the
binary
widget
we
deliver
right.
We
take
that
part
pretty
complex
and
compile
it
now
in
the
AI
world,
since
we're
actually
doing
modeling.
This
code
is
basically
just
the
scaffolding.
It
doesn't
do
anything
beyond
saying
what
the
model
should
look
like
and
essentially
the
compilation
stage
becomes
more
complex
or
more
data
dependent.
A
Essentially,
we
need
to
take
that
algorithm,
compile
it
first
in
a
conventional
fashion
and
then
start
training
to
finally
get
what
kind
of
parameters
there
should
be
and
to
get
a
quarter-point
model
that
we
deploy.
Now
we
start
doing
these
concerns
with
really
crappy
hardware
back
in
grass
two
days
or
back
in
you
know
like
back
a
few
years
ago.
They're,
basically
just
you
know
like
desktop
machines
that
are
usually
restart
engines,
like
that,
it's
kind
of
amazing
that
we
got
so
far.
A
Nowadays
the
models
become
so
complex
that
we
are
starting
to
get
much
more
competition
need
so
for
a
typical
image
classification
tasks.
Where
we
talk
about
such
as
the
alex
net
model,
we
usually
run
it
academically
on
about
1.2
million
images
and
the
typical
model.
Today
involves
about
one
exaflop
of
computation.
What
that
means
is
assuming
that
everyone
can
shell
out
their
piece
of
paper
and
start
doing
calculations
one
second,
one,
one
calculation
and
we
assemble
the
whole
people.
All
the
people
in
san
francisco
basically
say:
let's
do
it.
A
It
takes
about
four
thousand
years
to
train
one
such
model
and
that's
gonna
be
a
bit
crazy.
So
nowadays
we
often
employ
a
really
large-scale
systems,
such
as
clusters
of
machines,
GPUs
and
things
like
that
and
in
the
last
few
years
the
hardware
world
has
actually
developed
really
well
on,
and
we
have
you
know:
Amida
CUDA,
GPUs,
AMD,
GPUs
and
a
lot
of
hardware
accelerators
that
are
coming
up
in
the
day
in
these
days
to
help
us
do
the
competition
more
efficiently.
Essentially,
what
they
are
doing
is
they
take
the
input
data
they
take.
A
The
module
and
I
basically
say
I'm
gonna
be
chewing
those
model
trying
to
figure
out
based
on
the
training
information
that
we
get
from
those
data.
What
the
model
should
go
and
gradually
the
after
salt,
like
many
iterations,
the
model
evolves
on
to
a
direction
that
gives
us
the
best
result.
So,
essentially,
when
we're
doing
the
computation,
the
conventional
definition
of
compilation
starts
fading
away
and
we
basically
start
to
redefine
computation
as
compilation
as
we
do
the
computation
on
a
bunch
of
training
data
with
our
models
with
data,
and
we
talk
about
data.
A
The
best
hope
is
that
it's
that
it's
a
gold
mine
right
if
we
can
get
more
data,
if
you
can
get
more
insight
out
with
it
double
great
in
a
path
to
the
constraint-
is
that
we
cannot
have
a
person
going
in
to
look
at
this
huge
amount
of
data
and
try
to
make
logic
out
of
it.
It's
really
difficult
for
us
to
do
so.
There's
actually
one
example
in
the
CT
image
or
medical
area,
saying
that
doctors
get
more
and
more
experienced
as
they
see
more
and
more
x-ray
images.
That's
white!
A
That's
how
the
doctors
trained,
but
doctors
do
not
really
have
the
ability
to
train
themselves
night
and
day
computers
have
so.
The
question
is
basically:
are
our
algorithms
powerful
enough
to
choose
through
a
bunch
of
data
and
still
allowing
us
to
get
higher
than
high
quality,
now
want
to
face
what
researchers
had
the
same
exact
question
and
we
embarked
on
a
path
of
verifying
this
idea
about
image
classification
tasks,
so
we're
basically
saying
well
an
in
academic
world.
We
basically
take
the
1.2
million
images
and
we're
done
with
it.
A
What
if
bisque,
is
throw
much
more
data
on
to
the
same
model
without
any
model
itself?
The
model
basically
says
you
know.
This
is
the
way
we
can
do
compeition,
but
can
bigger
data
improve
the
performance
now?
The
way
we
get
data
is
there's
a
huge
amount
of
public
images
out
there
with
hashtags
annotations
on
the
site
text
and
things
like
that.
A
So
we
found
that
with
larger
images
with
large
amount
of
images,
the
performance
keeps
increasing,
there's
kind
of
like
a
a
pretty
good
gospel
to
us,
because
that
basically
means
that
our
models
are
powerful
enough
and
we
can
basically
continuing
and
benefit
from
the
large
and
large
amount
of
data
that
are
coming
in
and
I'll
talk
about
the
effective
data
a
little
bit
later.
Essentially,
what
happened
was
that
were
basically
replacing
human
reasoning,
basically
looking
at
data
and
try
to
write
programs.
That
exactly
represents
the
logic
of.
A
If
this,
then
that,
if
then
is
this
than
that
to
basically
write
a
model
and
have
large
and
large
amount
of
data
to
help
us
distill
the
knowledge
to
finally
improve
the
prediction,
accuracy
or
any
kind
of
task
that
we
want
AI
to
do,
and
essentially
what
we
were
observing
is
that
soft
engineering
in
AI
era
started
to
become
not
just
coding
and
compile
and
run
the
code.
It
becomes.
A
A
cycle
of
you
know
like
better
algorithms,
how
to
enable
researchers
to
write
algorithms
that
deals
with
images,
text,
recommendation
systems
and
things
like
that,
combined
with
a
larger
pool
of
competition
power
that
we
are
getting
from
our
harder
partners.
You
know
GPUs
and
hardware
accelerators,
combined
with
a
large
amount
of
data
that
will
help
us
to
utilize
the
representation
power
in
those
in
those
algorithms
to
eventually
deliver
the
AI
system
and
that's
what
we
were
observing
and
were
like
wow.
A
The
key
factor
here
is
basically
to
combine
the
intelligence
of
humans
and
researchers
to
provide
new
models,
explore
new
models
and
the
large-scale
computation
power
that
we
have
to
verify
those
ideas
and
to
deliver
those
into
products.
So
definitely
I
is
a
pretty
important
part
in
Facebook
right
and
we
basically
use
those
kind
of
dogs
evasions
to
to
help
us
improve
the
product.
Now.
A
One
of
the
example
is
we
have
a
massive
amount
of
translation
need
and
with
billions
of
users,
there's
got
to
be
more
than
one
language,
people
speak
and
in
fact,
hundreds
of
languages.
So
in
the
past
we
basically
use
rule-based
systems
and
wasn't
that
great,
it's
kind
of
basically
doing
word
replacement
in
some
way
and
nowadays
using
the
neural
net.
Approaches
were
actually
able
to
get
much
better
performance
because
the
models
are
quite
flexible,
they're
kind
of
state
of
art
and
it's
much
better
than
a
conventional
rule-based
system.
A
Translation
of
Facebook
is
actually
taking
quite
a
lot
of
computation
power
and
it's
fearing
quite
a
lot
of
applications.
So
we
are
supporting
about
more
than
45
languages.
We
are
doing
about
2,000
directions
from
English
to
French,
from
Chinese
to
German,
from
German
to
you
know
like
other
languages
stuff
like
that
all
you
know
every
day
we're
seeing
about
6
billion
computations
translations
and
flowing
through
our
system.
A
Now,
machine
learning
at
Facebook
actually
takes
multiple
forms
from
the
conventional
SVM
and
DBT
approaches
to
the
more
modern
modular
perceptions,
come
convolutional,
neural
nets
for
images
and
videos,
and
recurrent
neural
Nets
for
text
languages.
Stuff,
like
that,
it's
a
it's
pretty
massive
in
the
sense
that
every
day
earlier
in
April,
were
recording
about
200
trillion
predictions
per
day
prediction,
meaning
that
taken
an
image
and
in
production
or
taking
a
user
input,
and
do
some
recommendation
isn't
like
that
today,
we're
already
seen
a
pretty
great
growth
and
we're
probably
recording
about
300
trillion.
A
So
in
half
a
year,
it's
already
grown
50
percent
or
something
like
that.
All
these
commissioning
applications
are
basically
cured
by
a
path
of
going
from
prototype
ideas.
What
kind
of
models
want
to
build
to
train
them
on
large-scale
data
and
then
to
product
eyes?
That
and
as
it's
a
very
new
field,
we
found
that
there's
a
lot
of
direction
from
research
or
exploration
to
production,
going
on
it's
an
actual
dilemma,
because
researchers
want
maximum
flexibility
and
product
Swann's,
scalability
stability,
and
you
know
like
efficiency.
So
a
facebook.
A
We
basically
tried
running
an
ad
hoc
approaches
to
gain
ideas
about
what
are
the
key
principles
for
us
back
in
early
in
2018.
What
we
are
doing
is
we
have
two
pretty
excellent
platforms
on
one
called
pi
torch
and
one
called
kv2
that
Hargis
research
and
product.
At
the
same
time,
let
me
go
back
a
bit,
so
hydro
is
really
great
for
research,
because
it
gives
us
a
very
flexible
Python,
like
environment,
to
do.
A
So
this
you?
What
we
are
basically
trying
to
push
really
hard
is
to
combine
the
flexibility,
a
torch
and
the
standard
of
Onyx
and
the
power
back
kind
of
cafe
to
and
combine
them
in
one
big
framework
to
iron
out
all
those
overhead
of
going
from
one
part
to
another
part
going
to
another
part,
and
we
call
that
whitewash
1.0
and
it's
a
completely
open-source
system.
That's
actually
able
to
give
github
and-
and
we're
really
excited
to.
A
Basically,
you
know
like
share
this,
stepwise
as
we
are
developing
it
share
it
with
the
community
as
we
discovered.
Basically,
there
is
the
two
side
of
priorities
that
were
trying
to
bridge
and
we
try
to
categorize
them
as
these
two
on
efficiency,
annotations,
one
is
develop
efficiency
when
we're
writing
some
modules
or
when
we
write
to
doing
some
air
research,
we
want
seems
to
be
really
debuggable
an
interactive
one
to
basically
say
I
want
to
print
out
something
to
me.
A
The
results,
look
at
it
and
then
improve
my
model
and
it
better
be
very
simple
to
use.
We
don't
really
want
to
write
the
assembly
and
for
a
lot
of
the
research
ideas
as
long
as
we
can
make
them
more
intuitive.
We
can
basically
have
a
better
human
iteration
about
what
kind
of
new
models
we
can
build
on
the
infrastructure
side.
What
we
want
to
enable
is
infrastructure
efficiency.
How
do
we
run
competition
efficiently?
How
do
we
reliably
carry
out
those
competitions?
A
How
do
we
scaled
it
to
a
large
amount
of
machines
and
how
to
make
them
cross-platform,
because
in
many
cases,
when
we're
actually
deploying
on
thousands
or
tens
of
thousands
of
machines,
then
even
one
or
two
percent
of
the
accuracy
is
pretty
critical
for
us
and
so
on
the
developer
efficiency
side.
Then,
basically,
the
question
is:
how
do
we
actually
write
the
models
or
in
some
way
authoring
right?
So
the
good
thing
about
white
Forge
is
or
basically
trying
really
hard
to
make
it
as
pythonic
as
possible.
A
A
lot
of
the
things
are
pretty
debuggable
because,
as
you're
writing,
Python
programs,
one
of
the
best
debugging
tool,
is
just
to
a
print.
In
many
earlier
frameworks.
It
turned
out
to
be
that
not
that
possible,
but
as
long
as
we're
saying
in
the
Python
learned,
it
gets
really
nice.
The
downside,
of
course,
is
you
know,
like
any
spice
and
to
run-
and
it's
a
bit
hard
for
us
to
do
after
ization
in
parallel,
and
that
was
actually
one
of
the
challenges
in
the
old
school
Python.
So
now
we
call
hydrogen
classic.
A
We
tend
to
call
this
the
Eco
mode.
Basically,
as
you
are,
writing
your
program,
you
get
results
out
and
well
augment
in
fight
watch
the
classical
Python
impact
which
one
with
another
mode
called
script
mode.
Script
basically
says
the
model
is
a
program
written
in
the
format,
that's
kind
of
similar
to
Python,
but
really
under
the
hood.
What
we're
doing
is
you're
doing
yet
declare
in
a
model
without
rounding
it
and
until
you
basically
say,
run
this
program.
It's
not
been
like
all
those
intermediate
results
are
not
there.
A
It's
kind
of
similar
to
write
in
a
C
program
or
the
conventional
AI
framework.
What
that
allows
you
to
do
is
basically
to
write
your
model
in
a
very
nicely
constrained
environment,
guaranteed
to
be
shippable
on
to
the
product
side
and
in
the
past
people
write
those
gum
problems
in
their
own
dsls,
with
like
protobufs
or
json
or
stuff
like
that
and
in
the
scripting
mode.
We
make
it
as
pathetic
as
possible
so
that
you
don't
have
too
much
mental
overhead
to
learn
that
mode
and
to
basically
write
models
on
a
front-end.
A
A
So,
let's
talk
about
how
that
looks
like
essentially
what
you're
doing
in
a
eager
mode
or
basically
Python,
is
that
you
write
a
function
and
you
basically
say
there's
some
operations
from
the
import
instance.
Like
that,
the
good
thing
is,
you
can
basically
do
debugging.
You
can
put
a
print
over
there
and
because,
when
we
are
reaching
print
all
those
competitions
already
been
done,
then
you
can
get
a
content.
You
can
do
debugging
and
stuff
like
that.
Now
want
to
maintain
that
capability
and
then,
essentially,
you
can
basically
say
I
want
to
trace
this.
A
This
function
by
passing
in
an
input
and
then
run
it
once
and
then
record,
what's
been
computed
inside
there.
It
can
be
one
step,
it
can
be
multiple
steps
and
ideally
it
can
be
even
contained
in
for
loops
and
control
flows
and
things
like
that.
What
a
disclaimer
we're
not
having
fully
available
control
flows
yet,
but
that's
the
direction
that
we're
working
towards.
So
basically,
once
you
are
having
a
function,
you
can
basically
call
trace
and
JIT
to
get
the
model
out
of
it.
A
Basically,
if
you
want
to
write
a
model
that
is
more
complex,
but
you
know
what
you're
doing
and
you
don't
want
too
much
flexibility
and
you
want
to
basically
say
and
layout
the
computation
exactly
as
I
described,
including
contact
control
flows
and
things
like
that.
You
can
write
it
in
the
script
mode.
What
that
gives
you
is,
you
can
have
more
complex
control
flows
in
there.
You
don't
need
to
rely
on
tracing
it
with
sometimes
don't
work
to
get
a
models.
A
I
want
to
call
a
tracing
module
and
those
interactivity
interactivity
allows
us
to
basically
provide
a
pretty
good
flexibility
for
the
users,
whether
their
writings
models,
which
are
pretty
stable
or
rnm.
Models
which
are
kind
of
crazy,
was
very
complex
control
flows.
We
are
still
developing
tracing
jet
and
the
Python
library,
but
we're
already
putting
it
online.
If
you
go
to
peta.org,
you
can
either
try
it
out.
A
Installation
is
pretty
easy,
either
pipe
or
condo,
or
things
like
that
and
web
tutorials
and
documentation
for
you
to
try
out
the
more
detailed
stuff
of
it,
so
that
basically
allows
us
to
address
the
from
of
developer
efficiency.
Pretty
well.
We've
got
pretty
good
feedbacks.
There
was
one
gentleman
in
Twitter
which
who
joked
you
should
talk
to
a
doctor
to
see
if
python
is
fit
for
you
and
how
about
infrastructure
efficiency
so
a
Facebook.
We
run
things
at
a
very
large
scale,
so
we
look
into.
A
We
look
quite
a
lot
into
the
scalability
problem
now
last
year.
One
thing
that
we
did
pretty
pretty
interestingly,
is
we
try
to
see
how
much
we
can
speed
up
the
imagenet
training,
the
typical
model
that
we
use
for
computer
vision
so
about
when
in
2012,
when
Alex
Khrushchev,
we
started
training.
One
imagenet
model
takes
about
six
days
on
two
GPUs
and
that's
a
little
bit.
That's
okay,
but
a
little
bit
slow
for
research
use
cases,
because
if
a
researcher
want
to
try
an
idea,
wait
for
six
days,
that's
really
crazy.
A
So
we
try
to
speed
it
up,
and
the
last
year
were
actually
able
to.
You
know
like
speed
this
up
to
training
one
model
under
an
hour.
So
that's
a
pretty
impressive
speed
up.
We
were
pretty
surprised
that
it
was
very
well
received
by
the
community
and
people
started,
piling
on
and
basically
pushed
for.
A
Destruction
today,
I
think
fast
AI,
for
example,
in
San
Francisco
is
able
to
Train
imagenet
in
about
18
minutes
on
the
public
clouds,
and
you
can
pretty
much
try
it
out
yourself
on
media,
for
example,
is
able
to
Train
I
think
under
six
minutes,
which
is
impressive.
This
is
like
already
a
10
times
improvement
over
our
results
as
well.
The
key
components
under
these
computer
training
or
large-scale
training
is
what
we
call
high
performance
computation
if
you're
working
in
National,
Labs
or
weather
systems,
you
might
have
already
heard
about
the
word.
Mpi
message.
A
Passing
interface
is
probably
something
that's.
You
know:
I
developed
by
a
government
large-scale,
conventional
computer
and
stuff
like
that,
and
it
focuses
on
scam
tation,
and
we
found
that
AI
competition
is
much
similar
to
that.
You
have
a
bunch
of
really
powerful
machines,
for
example,
GPUs,
nowadays
really
popular
and
available
on
public
house
as
well,
and
you
want
to
basically
squeeze
out
every
little
bit
of
compeition
power
out
of
it
either
a
network
or
the
wall.
Compute.
A
One
thing
that's
pretty
important
for
us,
too,
is
to
basically
allow
the
missions
to
communicate
as
efficient
as
possible
among
each
other
and
an
MPI
and
provides
very
good
primitives
on
top
of
that-
and
we
basically
built
on
top
of
these
ideas
very
efficient
communication
primitives
to
do
so,
we
have
an
open
source
library
called
glue.
That's
also
a
connector
to
PI
torch
that
we
use
to
drive
those
competitions.
Essentially,
what
happens
is
with
multiple
GPUs
or
multiple
machines.
We
are
almost
able
to
get
linear
speed-up
so
effectively.
A
If
have
one
GPU,
you
run
at
say
this
amount
of
time.
If
you
have
eight
GPUs,
you
have
close
to
eight
times
faster
computation
for
images.
That's
how
we
do
it
and
it's
kind
of
not
so
surprising
that
we
can
achieve
these
kind
of
accuracy
on
other
applications
as
well,
so
for
translation
that
we
talked
about
and
just
now
it's
was
distributed
training.
A
It
allows
us
to
basically
get
our
training
algorithms
much
faster,
going
from
about
300
minutes
to
30
minutes
and
assuming,
if
you
want
to
join
2,000
directions
of
bi-directional
translation
that
really
helps
us
to
very
quickly
iterate
and
flush.
The
models
to
new
versions-
and
this
also
allows
researchers
to
look
into
the
new
models
and
model
iterations
as
well
so
with
the
fast
iteration,
were
able
to
get
pretty
good
performance
gains
with
just
video
training
and
researchers
working
together.
A
We
have
a
fair
seek
that
allows
you
to
basically
do
text
understanding
and
stuff
like
that,
without
too
much
trouble,
we
provide
you
free
trained
models.
If
you
want,
you
want
to
do
a
light
touch.
We
provide
you
training
scripts.
If
you
want
to
train
your
own
models
and
stuff
like
that,
one
pretty
amazing
thing
is
we
basically
do
all
those
kind
of
development,
especially
potage,
completely
online,
on
github?
Well,
we
actually
didn't
believe
that
we
could
do
it
because
we
were
like
wow.
A
That
allows
us
to
basically
partner
with
external
friends
really
well.
So
a
few
highlights
about
what
kind
of
things
people
are
building
on
top
of
Pi
George
today
so
fostered
a
I
created
a
very
nice
high
level
library
for
you
to
basically
do
adopt
and
use
data
modules
without
having
to
learn
completely
from
front-end
or
back-end
on
the
AI
software
stack.
If
you're
doing
things
like
published
programming,
then
there's
a
library
called
pyro,
auto
open-source.
A
That
makes
quite
a
lot
of
publicity,
programming,
Bayesian
inference
and
things
like
that,
a
little
bit
easier
if
we're
doing
Gaussian
process
in
the
past,
the
libraries
of
Gaussian
processes
a
little
bit
hard
to
deal
with,
and
but
there
are
really
nice
libraries
developed
by
third-party
friends
called
GP
Gorge
that
allows
us
to
do
easily
formation
translation,
not
only
we
are
doing.
It
were
also
kind
of
like
enabling
a
lot
of
other
research
institutes
and
universities
to
do
those
research.
So
they
also
publish
the
results
and
there's
a
really
nice
library
called
open.
A
Enemity,
open
natural
machine
translation,
that's
kind
of
similar
to
fseek,
but
provides
you
in
like
a
variety
of
tools
and
new
researches
to
fight
out
and
on
computer
vision,
side
and
UC
Berkeley
has
pretty
nice
library
called
psycho
again
using
the
idea
of
general
adversarial
network
for
you
to
do
pretty
interest
in
experiments,
while
it
has
really
good
reorders
cases,
but
you
can
also
play
with
it
was
pretty
cute
tools
such
as
painting
a
horse
to
a
zebra
and
stuff
like
that.
That's
one
of
the
results
really
nice
results
from
Berkeley.
A
Basically,
you
take
the
image
of
a
horse
and
then
you
just
like
trainer
model
that
converts
that
to
a
zebra
without
too
much
like
painting
knowledge.
So
that's
pretty
much
what
we're
trying
to
do
with
pay
towards
1.0.
The
goal
is
that
we've
got
really
good
experience
using
it
for
research.
We've
got
pretty
good
experience
deploying
that
into
products,
and
we
would
like
to
basically
present
those
kind
of
tools
to
you
and
figure
out
and
and
together
figure
out
what
we
should
do
as
an
open
ecosystem.
We
started
fighters
1.0
about
May
this
year.
A
There's
a
lot
of
things
on
the
fly.
If
you
go
to
github,
you
can
see
that
even
we
ourselves
are
having
debates
about
the
design
here
and
their
instance
like
that,
but
that's
the
beauty
of
open
source.
We
just
like
put
everything
out
there
and
we
are
basically
putting
out
a
preview
version
for
you
to
try
it
out.
Some
of
them
will
be
on
the
fly.
Some
of
the
features
will
be
a
bit
unstable
and
please
bear
with
us.
A
You
can
go
online
and
to
basically
Pedro
org
and
get
a
preview
and
out
of
it
with
different
versions.
You
know
like
if
you
do
Python
no
problem,
if
you
did
Python
3
no
problem,
if
a
different
CUDA
versions
no
problem
at
all-
and
we
really
appreciate,
if
you
could
try
it
out,
send
feedback
to
us,
send
issues
basically
say
hey:
this
is
working.
Can
you
can
you
figure
it
out
and
together
I
think
we'll
be
able
to
build
a
much
better
Ecosystem
both
for
research
and
for
production
use
cases?
A
You
can
find
more
information
from
the
pet
org
webpage,
which
hosts
all
these
open-source
potage
content
contents
and
from
kotofey
comm,
where
we
regularly
share
news
about
what
kind
of
new
libraries
we
are
pushing
out,
what
kind
of
publications
we
are
having
and
we'll
have
to
share
as
much
information
as
we
have
with
you
as
an
open
community
with
that
that
concludes
my
talk
and
I
really
appreciate
your
time,
and
this
we
are
great
to
be
here
at
talking
about
get
up
universe.
So
thanks
so
much.