►
Description
My guest this time is Francisco Webber, founder and General Manager of artificial intelligence startup Cortical.io. Francisco presented at the O’Reilly AI conference on an approach to natural language understanding based on semantic representations of speech. His talk was called “AI is not a matter of strength but of intelligence.” My conversation with Francisco was a bit technical and abstract, but also super interesting.
The show notes can be found at https://twimlai.com/talk/10.
Subscribe!
iTunes ➙ https://itunes.apple.com/us/podcast/this-week-in-machine-learning/id1116303051?mt=2
Soundcloud ➙ https://soundcloud.com/twiml
Google Play ➙ http://bit.ly/2lrWlJZ
Stitcher ➙ http://www.stitcher.com/s?fid=92079&refid=stpr
RSS ➙ https://twimlai.com/feed
Lets Connect!
Twimlai.com ➙ https://twimlai.com/contact
Twitter ➙ https://twitter.com/twimlai
Facebook ➙ https://Facebook.com/Twimlai
Medium ➙ https://medium.com/this-week-in-machine-learning-ai
A
Hello
and
welcome
to
another
episode
of
we'll
talk,
the
podcast,
where
I
interview,
interesting
people,
doing
interesting
things
in
machine
learning
and
artificial
intelligence.
I'm
your
host
Sam
Charrington.
Once
again,
the
recording
you're
about
to
hear
is
part
of
a
series
of
interviews.
I
recorded,
live
from
the
O'reilly
AI
and
strata
conferences
in
New,
York
City,
my
guest.
This
time
is
Francisco
Weber,
who
is
the
founder
and
general
manager
of
artificial
intelligence.
A
Start-Up
cortical,
IO
Francisco's
presentation
at
O'reilly,
AI
was
called
AI
is
not
a
matter
of
strength,
but
of
intelligence
to
set
the
stage
for
my
conversation
with
Francisco
recall
that
in
the
last
interview,
Pascal
Fung
noted
how
recent
advances
and
natural
language
understanding
have
been
based
largely
on
ignoring
language
structure
and
focusing
on
statistics.
Well,
in
this
interview,
you
hear
Francisco
argue
that
the
next
advance
in
NLU
will
come
from
shifting
our
attention
from
statistical
models
to
models
based
on
a
more
sophisticated
model
of
the
brain,
a
warning
in
advance.
A
This
conversation
is
very
technical
and,
moreover,
rather
abstract
don't
be
afraid
to
listen
to
it
a
couple
of
times
to
allow
the
idea
as
an
opportunity
to
sink
in
you'll
find
this
week's
show
notes
at
twin,
Olay,
Icom,
slash
talks
last
ten,
you
might
be
particularly
interested
in
a
link
to
Francisco's
presentation,
slides,
which
are
helpful
to
review
alongside
the
podcast
and
now
on
to
the
show
hello.
Everyone
here
at
the
O'reilly
AI
conference
and
I
am
with
Francisco
Webber,
who
gave
a
great
talk
earlier
on.
B
Clio
is
actually
my
third
company
I
previously
worked
in
the
field
of
patent
information,
which
is
also
a
sort
of
complex
natural
language
issue,
and
that
was
basically
where
I
learned
of
the
limitations
of
the
current
systems
and
that
motivated
me
to
actually
try
and
find
something
substantially
different.
Okay,.
B
So
cortical
is
all
about
two
things
in
fact,
so
one
thing
is
a
theoretical
framework
that
we
have
discovered,
I
would
say
and
that
we
explore
that
is
about
how
the
human
brain
to
be
more
specific.
Actually,
the
neocortex
supposingly
handles
language
information,
and
the
other
is
that
we
basically
use
this
theoretical
framework
to
also
create
a
real
technology
that
we
basically
offer
to
the
markets.
Currently,
this
is
mainly
for
enterprise
customers.
B
They
have
a
lot
of
problems
out
there
and
so
say.
The
effort
to
revenue
ratio
makes
us
work
there
for
a
while,
but
we
do
also
have
a
public
API
where
basically,
everybody
can
play
around
with
our
technology
for
free
yeah.
So
that's
that's
what
the
company
sort
of
does
and
we
try
to
find
a
really
alternative
way
to
deep
learning
and
to
the
more
traditional
ways
of
statistical,
modeling
and
machine
learning.
B
For
the
moment,
our
approach
doesn't
actually
use
any
statistics
so
which
might
not
be
the
case
in
the
future.
So
there
are
some
motivations
to
maybe
team
this
up
with
deep
neural
networks,
or
so
so
that's,
but
in
fact
it's
not
our
specialities
or
I.
Leave
this
to
orders
to
try
out
what
we
basically
do
is
that
we
have
solved
I
would
say,
or
we
have
found
a
solution
to
the
famous
representational
problem
that
exists
in
natural
language
understanding
since
decades.
B
Basically,
a
very
fundamental
issue
is
that
basically
says
if
you
find
out
how
to
represent
language,
which
means
text
at
some
point
in
a
way
that
you
can
actually
compute
with
it.
Then
many
of
the
big
problems
like
ambiguity,
like
vocabulary,
mismatch
all
the
traditional
problems
we
have
in
NL.
You
basically
are
solved
in
in
one
in
one
approach
and
that's
what
I
actually
presented
today
is
that
by
this
little
shift
in
how
to
generate
the
features
everything
falls
in
place
afterwards.
You
know
in
a
very
convenient
and
and
most
importantly,
efficient
manner,
so.
B
So
our
approach
basically
is
founded
on
the
work
of
Jeff
Hawkins,
who
is
a
researcher
in
in
in
the
area
of
cortical
processing?
So
he
works
on
finding
out
how
the
human
neocortex
actually
processes
data
they
assayed
data
in
general,
because
one
of
his
findings
was
that
regardless,
what
kind
of
data?
So
might
it
be
sound
hearing
or
seeing
or
touching
all
of
that
data
when
it
comes
to
the
neocortex,
looks
the
same?
It
is.
It
has
the
same
format,
which
is
a
what
is
called
a
sparse
distributed
representation.
B
So
it's
like
a
large
vector
of
binary
features
where
you
have
like
two
of
two
to
five
percent
of
those
features
are
actually
set
to
one
and
all
the
rest
is
zero
and
everything
is
encoded
into
such
a
such
an
SDR,
and
that
was
basically
our
first
goal
is
to
find
a
systematic
unsupervised,
because
otherwise
it's
not
doable
in
practice
a
systematic
unsupervised
way
of
converting
text
into
such
a
an
SDR.
Ok,.
A
Now
I've
heard
a
couple
of
times,
even
at
this
event,
there
were
a
couple
of
comments
that
were
made.
That
was,
one
of
them
was
even
I.
Think
in
the
keynotes
this
morning
there
was
a
comment
about
how
you
know
what
we've
got
with
neural
nets
or
don't
have
the
complexity
and
the
nuance
available
to
express
what's
actually
happening
in
the
brain
and
and
in
another
talk,
the
the
kind
of
follow-on
statement
was
so.
Therefore
we
shouldn't
try.
A
B
I
mean
fundamentally,
what
we
use
as
neurons
nowadays
has,
in
fact,
very
little
to
do
with
real
neurons
right,
so
it
was
an
abstraction
that
was
made
like
3040
years
ago
on
a
compared
to
today
on
a
very
rudimentary
understanding
of
what
neurons
actually
do.
Nowadays,
we
know
more.
We
know
that,
for
example,
the
actual
learning
happens
through
the
building
and
unbuilding
of
synapses
between
them
and
if
you
actually
model
a
neuron,
not
not
chemically,
so
it's
not
about
sort
of
creating
all
the
molecules
that
are
there,
because
that's
something
that
nature
uses.
B
Yes,
so
nature.
You
know
in
evolution,
you
always
have
the
components
from
the
previous
evolution
evolutionary
state,
and
you
have
to
play
with
this
kind
of
LEGO
bricks
and
do
something
which
sometimes
looks
a
bit
inefficient.
But
what
is
key
on
the
other
side
is:
what
is
the
mechanism
that
those
real
neurons
create,
and
that
is
what
Jeff
Hawkins
actually
has
figured
out
and
is
about
to
even
figure
out
in
more
detail,
and
so
certain
aspects,
like
the
the
sparse
binary
representation,
are
actually
key
for
this
to
work
properly
and
by
working
on
text.
B
So
our
approach
was
basically
okay.
If
Jeff
is
right
with
his
theory,
everything
he
says
about
the
general
way,
how
the
cortex
processes
has
also
to
be
true
for
language
as
the
language
is
generated
by
the
cortex,
and
so
we
basically
took
his
theoretical
framework
as
a
set
of
constraints,
and
we
tried
to
say
okay,
if
that
is
the
limitation.
B
But
then
ahead
at
some
point,
while
I
was
listening
to
his
talks,
reading
his
book
on
intelligence
and
so
and
it
was
literally
sort
of
taking
a
shower
and
in
a
second
I,
had
this
visual
idea
so
to
say
how
how
this
could
happen,
because
it
boils
down
to
a
sort
of
visual
aspect
in
the
sense
that,
as
a
necessity,
we
have
to
find
a
representation
where
two
words
that
mean
similar
things
have
to
actually
look
similar
and
when
I
say
look
there.
Sdrs
have
to
be
similar
and
literally
similar.
B
So
in
fact,
and
that's
also
what
the
brain
is
doing
to
put
one
word
representation
on
top
of
the
other
word
representation
and
by
measuring
the
overlap.
How
many
of
the
bits
actually
stay
at
the
same
position?
You
get
two
things.
One
is.
How
related
are
those
words,
and
the
second
is
by
looking
where
the
overlap
happens
within
this
representation,
because
this
is
a
two-dimensional.
B
It's
like
a
bitmap
with
128,
x,
128,
pixels
and
like
2
percent
of
those
16
thousand
bits
are
set
to.
1,
therefore,
are
like
pixels
dark
pixels.
If
you
want,
and
so
it's
actually
a
visual
thing-
and
you
can
try
this
out
on
our
website
when
you
take
two
words
that
are
sort
of
have
a
common
context,
or
so
you
can
actually
literally
see
that
they
look
similar,
yeah
and
interestingly,
I
mean
it's
it's
hard
to
know,
or
maybe
even
impossible
to
absolutely
decode
what
it
means.
B
But
if
you
as
I
have
done
stare
a
lot
into
these
representations,
you
end
up
seeing
the
differences
like
in
a
blink
of
an
eye.
You
might
not
know
the
details,
but
to
identify
that
two
words
are
similar
in
this.
Representation
takes
a
million
seconds
yeah,
and
that
is
already
a
a
hint.
So
to
say
that
shows
what
the
major
gain
is
of
this
approach,
which
is
efficiency
and
that's
what
our
brain
is
famous
for.
B
So
I
know
that
on
the
deep
learning
in
the
deep
learning
community
things
like
precision
and
so
are
the
key
metrics,
and
they
are
of
course
important.
It's
like
having
classes
that
are
blurry.
Nevertheless,
at
the
very
end,
the
choice
of
algorithm
is
not
so
much
on
the
precision,
but
it
relates
to
down-to-earth
energy
efficiency.
Yeah
I
mean
the
brain
works
with
something
like
10
watts
or
something
so
I
don't
even
want
to
know
how
much
power
the
GPU
servers
eat
up,
and
that
is
already
a
very
good
hint
of
how
well
a
certain
approaches.
B
If
and
that's
why.
I
chose
the
title
also
of
brute
force,
because
statistics,
especially
if
you
do
statistics
of
large
combinatorial
spaces
like
like
language
I,
mean
you
basically
can
create
an
indefinite
number
of
combinations
of
words
to
make
meaningful
sentences
so
to
do
a
statistics
on
such
an
open
system.
It's
a
real
hard
work,
because
you
have
to
provide
endless
examples
to
have
like
a
micro
bit
of
a
semantic
payload
in
your
representation
yeah,
and
it
works
up
to
a
certain
point.
B
No
question
I
mean
the
statistical
systems
work,
but
what
you
see
is
that,
in
order
to
make
the
model
a
little
bit
smaller
or
to
gain
a
tenth
of
a
percent
in
precision,
you
have
to
put
a
lot
of
effort
in
yeah
so
from
to
get
from
sixty
to
sixty
one
percent.
Precision
you
might
even
double
the
effort
like
going
from
1
to
60
is
the
same
as
from
60
60
to
61.
Yeah
drag.
A
You
drag
you
back
a
little
bit
just
it
sounds
like
understanding.
The
Jeff
Hawkins
stuff
is
important
to
understanding
what
you
guys
are
doing
to
some
degree.
Yes,
so
they
so
he's
defined
this
sparse
data
representation,
this
SDR
and
is
there
also
a
different
concept
of
a
neuron
that
underlies
that
absolutely.
B
So
he's
modeling
a
real
neuron,
but
on
the
functional
level,
so
he's
also
modeling
a
neuron
if
you
want
but
he's
modeling,
everything
that
is
relevant
within
the
neuron
for
processing
data
is
part
of
his
model
and
everything
that
might
be
housekeeping
building
up
proteins
and
stuff
like
that
is
not
part
of
the
actual
data
processing
layer
and
therefore
not
represented
there.
So
he's
basically
tried
to
expand
the
simplistic
concept
of
neural
net
neuron
to
become
a
real
neuron,
and
sometimes,
if
you
face
the
problem,
the
way
it
is.
B
A
A
B
B
Everything
and
therefore
it
is
highly
unprovable,
if
not
impossible,
that
by
having
an
initial
state,
a
you
can
predict,
which
are
the
let's
say,
physical
possible
next
steps
and
that's
what
the
processing
relays
on
the
fact
that
not
like
in
statistics
after
state
a
any
state
could
happen
because
I
need
to
do
the
statistics
for
it.
But
the
reality
is
that,
after
a
step
a
there
is
a
certain
set
of
steps
which
have
all
to
be
possible
in
reality
and
what
we
learn
as
walking
brains.
If
you
want,
is
what
are
those
potentials?
B
What
are
the
potential
outcomes
and
how
many
hints
from
the
initial
state
could
point
me
to
the
right
next
state
and
that's
in
the
end,
what
the
brain
is
doing.
Yeah
the
brain
is
nothing
more
than
a
sequence
learning
engine
that
does
prediction
based
on
what
it
has
seen.
So
far,
and
if
you
think
through
the
that
on
a
let's
say,
philosophical
level,
you
will
find
out
that
you
basically
can
solve
or
explain
everything
we
do.
B
B
Yes,
the
processing
happens
in
the
processor
and
the
RAM
is
just
a
dormant
store,
yeah
and
the
brain
obviously
does
this
differently
and
and
and
and
the
other
aspect
is
that
the
prediction
is
in
fact
the
condensed
intelligence,
because
the
more
I'm
right
in
predicting
the
more
I'm
intelligent
and
by
the
way,
I
mean
there.
You
know
there
have
been
very
behavioral
ways
of
looking
at
intelligence.
B
That's
the
reason
why
the
dog
looks
intelligent
to
the
dog
owner,
because
the
dog
owner
knows
the
dog
and
knows
what
predictions
the
dog
is
making
about
things
and
is
right
in
doing
so,
and
therefore
the
dog
indirectly
so
to
say,
looks
it
more
intelligent
to
the
owner
than
to
everybody
else,
yeah.
So.
B
The
SDR
is
all
about
getting
a
an
explicit
representation
of
the
state,
so
that's
the
other
difference
in
in
the
world
of
brains
and
STRs.
You
only
work
with
what
is
called
semantically
grounded
information,
so
every
bit
in
representation
of
the
SDR
actually
corresponds
to
something
real
and
concrete.
B
If
you
have
now
a
representation
that
gives
you
the
same
phenomenon,
namely
that
two
bits
that
are
set
to
one
stay
close
to
each
other,
it's
easy
to
guess
that
they
are
related
and
they
are
part
of
the
same,
maybe
subunit,
of
the
system.
The
only
thing
you
have
to
be
sure
is
that
the
data
that
is
provided
is
actually
inherently
semantic,
so
it
has
to
be
part
of
a
system
in
a
very
abstract
level,
so
the
world
is
a
system.
B
Therefore,
any
data
that
I
can
hear
or
see
or
so
about
the
world
is
semantic
because
there
are
rules
of
physics,
rules
of
biology
and
so
on,
and
the
same
thing
is
true
for
language,
language
is
data
that
is
inherently
tied
together
by
a
framework
of
grammar
of
syntax
and
all
these
aspects.
We
know
since
a
long
time,
but
we
have
we
had
the
problem
on
how
to
actually
store
these
mechanisms
and
the
realities
don't
store
the
mechanism,
but
just
were
the
examples
just
or
the
detailed
information,
the
explicit
information
and.
B
So
in
our
approach,
we
declare
the
semantic
atoms
in
language.
To
be
words,
I
mean
there
are
like
smaller
units
like
full
names,
for
example,
but
they
have
no
meaning
by
themselves.
So
the
first
time
you
actually
have
a
meaning
is,
when
you
have
a
word
and
all
the
subsequent
meaning
of
a
sentence
of
a
paragraph,
a
document
and
utterance
even
comes
out
of
the
sequence
of
those
words,
and
so
what
we
do
is
basically
we
convert
every
single
word
into
such
a
sparse
representation.
B
We
call
this
because
it's
a
hard
to
say
we
call
this
a
semantic
fingerprint,
and
the
interesting
thing
is
that,
through
the
way
how
we
convert
the
semantic
fingerprint
you
take
advantage
of
some
of
the
properties
that
are
inherent
of
sparse
binary
vectors,
for
example,
can
make
a
union
of
as
many
sparse
vectors
as
you
want,
and
you
don't
lose
information.
Yes,
you
can
always
say
from
an
unseen
vector
if
it
was
part
of
the
union
or
not.
If
you
try
to
do
the
same
thing
with
the
dense
representation,
let's
say
the
the
ASCII
encoding.
B
You
have
8
bits
and
every
possible
combination
corresponds
to
another
character.
If
you
make
a
union
of
a
couple
of
them,
no
way
to
say
what
was
the
initial
part
and,
as
I
said,
the
the
generation
of
this
pattern
is
done
in
a
way
that
every
single
pixel
of
our
fingerprints
corresponds
to
an
explicit
learned
context,
and
you
can
in
fact
reward
a
context.
It's
a
context
which
is
basically
technically
it's
a
bag
of
words.
If
you
want
it's
a
bag
of
words
of
utterances
in
which
the
word
occurred,
mm-hmm.
A
B
The
what
we
do
is
in
principle:
we
generate
the
atoms,
which
is
a
fingerprint
for
a
word,
but
if
I
want
to
create
a
fingerprint
for
a
sentence,
I
just
convert
every
single
word
into
its
fingerprint
I
make
a
stack
and
aggregate
them
together.
Let
me
can,
over
of
all
of
them
and
then
I
have,
depending
on
the
location
on
the
fingerprint.
You.
B
Exactly
exactly
yeah,
that's
the
reason
why
we
have
to
stay
on
the
sparse
site,
yeah
and
if
you
make,
for
example,
a
union
of
let's
say
ten
words
that
are
in
a
sentence.
You
of
course
fill
up
the
the
representation.
Therefore,
after
making
the
Union,
we
we
do
what
we
call
responsive
Phi.
We
introduce
a
threshold
to
cut
away
everything
that
fills
the
fingerprint
on
more
than
the
2%,
and
so
we
end
up
with
a
fingerprint
for
a
sentence
that
has
basically
the
same
topology.
B
A
B
So
there
is
the
topology
and
there
is
that's
the
name
why
why
we
call
it
semantic
folding
there?
Is
this
semantic
space
folded
into
the
representation,
so
the
way
how
we
do
this?
How
we
generate
the
word
as
the
ours
is
that
we
take
a
collection
of
documents
which
are
the
reference
documents,
that's
for
a
human
that
would
be
everything
you
ever
read
and
heard
all
language
elements
that
you
have
exposed
to
and
we
digest
them,
and
we
do
this,
of
course,
using
machine
learning,
because
we
are
not
like
humans.
B
B
A
B
It
actually
represents
is
a
semantic
space,
because
it's
it's
it's
it's
the
the
fundamental
of
the
representation
and
if
you
just
do
the
math
of
selecting
300
bits,
which
is
about
close
to
5%
of
16,000
bits,
the
number
of
combinations
you
can
do
is
like
the
number
of
stars
in
the
Milky
Way.
So
it's
a
huge
community
oral
space
and,
as
you
know,
we
have
not
the
same
assumption
as
in
statistics
that,
in
principle,
every
word
could
be
combined
to
every
other
word
yeah.
B
B
It's
it's
it's
a
natural
development
that
we
have
started
NLP
and
information
retrieval
with
so
called
document
vectors.
Everything
will
sort
of
derived
from
a
document,
and
we
found
out
over
the
years
that
word
vectors.
The
representation
of
individual
words
seems
to
be
more
appropriate.
Nevertheless,
there
is
a
fundamental
but
crucial
difference
so
were
to
act.
Like
other
word,
embedding
mechanisms
use
they
try
to
do
dimensionality
reduction
and
they
end
up
with
a
dance
vector
and
to
put
even
more
on
it,
a
dance
vector
of
double
or
float
numbers
so
sort
of
computationally,
expensive
representations.
B
We
don't
do
a
dimensionality
reduction,
we
might
even
to
an
increase
the
dimensionality
at
some
point
if
you
want,
but
we
make
it
a
sparse
representation.
So
we
have
sparse
binary
vectors
versus
dense
floating-point,
whether
the
double
vectors
yeah,
which
already
sort
of
gives
you
a
hint
on
where
the
efficiency
will
be
all
right
right.
A
A
B
B
When
I,
when
I
show
this
and
people
see
the
the
fingerprints
on
the
screen
and
how
they
interact
and
how
they
overlap,
you
can
see
in
their
faces
I
understand
this
yeah.
You
don't
need
to
know
anything
about
machine
learning,
or
so
it's
so
intuitive.
But
if
I
imagine
to
sort
of
follow
a
description
that
is
purely
verbal,
then.
B
Yeah,
so
the
the
rest
basically
was
that
I
gave
a
number
of
practical
examples
where
we
apply
this
and
I
can
cite
a
few.
For
example,
we
do
pro.
Let's
say
we
have
certain
prototype
ways
of
solving
typical
problems,
and
what
is
the
case
is
that
we
solve
all
of
them
with
one
unique
operator
which
is
similarity
yes,
so
we
only
had
the
only
sort
of
verb
we
have
in
our
universe
is,
is
similar
or
is
not
similar,
and
so
one
thing
you
can
do,
of
course
is
search
yeah,
so
you
can
and.
A
B
So
we
actually
measure
this
by
calculating
the
overlap
between
two
fingerprints,
which
is
the
most
generic
way.
I
mean
we.
We
do
offer
a
number
of
distance
metrics.
As
I
said,
this
is
a
metric
space,
so
we
have
different
ways
of
calculating
a
distant
metrics
like
a
Euclidean
distance
and
others,
but
I
have
to
say
that
in
fact
the
pure
overlap
count
is
fully
sufficient
to
get
the
result.
All
of
it
and
it's
very
computationally,
efficient
yeah.
So
one
of
the
prototypes
as
I
said,
is
search.
Imagine
you
have
a
collection
of
documents.
B
You
convert
each
of
the
documents
into
a
fingerprint.
You
have
a
user
who
types
in
a
language
based
query,
I'm,
looking
for
information
about
red
spot
cause,
you
create
a
fingerprint
of
that
query
and
you
just
match
how
much
overlap
you
have
between
all
the
documents
and
the
query,
and
you
rank
all
your
documents
according
to
the
size
of
the
overlap,
very
generic,
it's
it's
it's
a
real
search
mechanism.
So
what
you
get
is
really
all
the
balanced
aspects
that
you
have
in
a
document.
B
B
The
point
is
that
we
have
even
brought
this
to
a
step
further,
because
we
were
able
to
not
only
train
in
different
languages,
the
semantic
spaces,
but
to
also
topologically
align
them
and,
as
a
result
and
I
gave
the
example
in.
In
my
talk
we
take
the
word
philosophy
in
English.
Has
a
certain
representation,
and
the
word
philosophy
in
French
has
the
same
representation.
So
the
better
patterns
are
the
same,
and
what
this
means
is.
B
The
word
the
words
with
the
same
meaning,
regardless
of
the
language,
have
the
same
fingerprint
right.
So
a
second
prototype
where
I
could
give
you
an
example,
is
classification,
so
our
classifiers
are
actually
just
fingerprints.
I,
don't
need
to
train
my
classifier
if
I
say
I
want
to
get.
Let's
say
all
the
tweets
about
mobile
phones.
I
can
take
the
word.
B
B
Definitely
you
should
be
surprised
yes
and
no
I
mean
people
who
know
two
languages
are
able
to
do
this
in
the
same
way
yeah.
So
there
has
to
be
a
let's
say,
mathematical
way
of
doing
this,
and
the
point
is
that
we
aligned
the
two
semantic
spaces.
So
we
have
one
topology
that
we
generate
in
one
language
and
we
can
then,
with
a
with
a
pure
dictionary,
lookup
mechanism,
which
is
the
dumbest
way
of
doing
a
translation
right.
B
We
can
convert
all
the
distributed
snippets
in
the
vocabulary
of
the
other
language
and
use
the
same
distribution
that
we
have
trained
with,
for
example,
the
English
methods
and
therefore
you
have
now
the
convenience
to
listen.
Let's
say
to
the
Twitter
firehose
and
regardless
of
what
language
message
comes
along,
you
can
filter
it
with
an
English
example
and
I've
done
that,
just
to
give
you
a
a
feeling
on
efficiency,
I've
done
that
in
real
time
on
the
fire
hose
with
my
notebook
yeah.
So
it
was
sort
of
running
locally.
A
Through
the
I'm
trying
to
run
through
the
physical
analogy
or
the
biological
analogy
of
this
like
in
you
know,
the
the
notion
here
is
that
you've
kind
of
extracted
this
model
that
you
know
more
closely
represents
what's
happening
in
the
brain,
then,
and
and
you
can
you
have
this
kind
of
transfer
ability
across
languages
I'm?
Is
there
some?
A
You
know
again
we're
kind
of
way
beyond
you
know
the
pale
of
what's
actually
gonna
happen,
but
like
is
there
something
like
you
take
the
you
know,
some
part
of
the
brain
from
someone
who
learned
Chinese
and
you
transplant
it
into
a
person.
You
know,
and
then
you
know
who
has
some
other
part
of
the
brain
that
is
kind
of
symbolically
to
English,
and
they
could
then
you
know,
translate
on
the
fly
yeah.
B
B
Another
language
has
always
in
his
head
to
map
from
the
one
region
into
the
other
region.
Now,
interestingly,
there
is,
and
I
showed
this.
Also
in
my
presentation,
there
is
sort
of
new
research
in
in
brain
science
that
supports
our
representation
strongly,
so
they
were
able
to
do
an
fMRI
study
to
be
precise.
So
there
has
been
an
earlier
version
of
this
experiment
where
people
were
exposed
to
words,
and
then
they
made
like
snapshots
from
their
fMRI
activity
and
what
they
found
out.
B
B
So
this
is,
as
you
say,
Wow,
but
it
comes
better
and
they
have
sort
of
trained
a
machine
learning
algorithm
to
make
this
transfer
to
correlate
the
picture
with
the
word
that
the
persons
have
been
hearing
and
the
absolutely
unbelievable
thing
is
that
let's
say
you
have
been
in
the
fMRI
first,
the
model
has
been
trained
on
your
images
to
map
to
certain
terms.
If
now
you
present
this
very
same
model,
the
images
taken
from
my
brain,
it
will
still
recognize
the
terms
properly
and
then.
B
Has
been
done
in
English,
I'm
I'm,
pretty
sure
that
even
if
I
would
do
this
with
with
the
Portuguese
meaning
of
your
English
terms,
it
still
might
work
out.
But
but
why
not?
The
fact
is
that,
obviously,
if
two
individuals
have
been
grown
up
sufficiently
similar
from
a
cultural
point
of
view
yeah,
so
we
both
went
to
school
for
more
or
less
the
same
time
we
more
or
less
read
the
same
stuff
we've
heard
about
the
world.
B
In
the
same
way,
the
representation
ends
up
being
similar
across
individuals
and
in
the
end
it
makes
a
lot
of
sense.
I
mean
just
imagine
if
we
would
really
be
wired
completely
different
from
one
to
each
other.
It's
it
would
be
very
hard
to
have
a
simple
conversation
yeah
and
in
fact,
if
you,
if
you
do
the
the
investigation,
for
example,
I'm
pretty
sure
again,
this
is
just
guessing,
but
the
fMRI
pictures
from
I
don't
know
some
distant
tribe
living
somewhere
in
the
Amazonian
jungle.
They're.
A
B
So,
as
I
said
earlier,
we
are
very
strong
with
this
approach
in
doing
similarity,
calculation
and
therefore
classification
and,
as
you
might
know,
in
business
natural
language
processing,
nearly
all
problems
can
be
reduced
to
one
or
several
classification
problems.
Okay,
so
we
do
all
sorts
of
things.
Yeah
I
mean
companies
who
say
we
want
to
classify
our
inbound
emails
in
product
requests,
complaints
and
I,
don't
know
looking
for
a
person
and
individual
in
the
company
and
believe
it
or
not.
B
I
haven't
seen
any
working
machine
learning
solution
for
that
problem
out
there
I
mean
I've
been
visiting
like
150
companies
over
the
last
two
years.
Of
course,
trying
to
sell
our
stuff
right,
but
I
haven't
seen
a
working
solution
for
simple
I
mean
this
is
really
one
of
the
most
basic
issues
you
could
have,
and
nearly
nobody
is
actually
using
technology
for
that,
because
the
the
the
statistical
approach
has
a
lot
of
noise
that
comes
in
has
false
positives,
which
is
by
the
way,
the
the
biggest
problem
in
business,
and
we
solve.
B
A
B
So
we
have
customers
in
the
domain
in
a
lot
of
customers,
for
example,
are
in
the
banking
domain.
There
we
solve
problems
like
compliance
monitoring
or
your
customer
activities
or
automation
of
business
processes
that
depend
on
some
text
input.
At
some
point,
we
have
consumer
good
companies
who
want
to
know
how
to
segment
their
customers.
For
example,
we
have
manufacturing
industry
where,
for
example,
in
technical
products,
the
documentation
that
the
manual
of
the
product
is
so
complicated.
B
Good
example
is
car
industry,
for
example,
modern
cars
so
complicated
that
if
something
breaks
you
need
to
visit
the
manual
or
to
find
out
what
is
this
funny
light?
Meaning
there
is
this
dangerous
or
can
I
just
continue,
and
people
can't
find
anything
because
they
have
the
problem
that
the
person
in
the
car
doesn't
speak
the
technological
language.
So
an
example
that
I've
that
I've
learned
is
the
query
where
do
I
find
a
donut
in
in
the
US
I
didn't
know
that
before,
but
obviously
the
donut
is
this
fair
wheel.
B
That
is
sometimes
pretty
good
hidden.
So
if
you
look
for
donut
in
the
in
the
manual,
you
probably
don't
find
it
yeah,
and
there
is
a
lot
of
these
issues.
Yeah
I
mean
to
be
even
a
more
extreme
case,
a
person
speaking
only
Spanish,
driving
a
US
car
and
being
unable
to
actually
find
the
the
right
answer
could
use
our
system
to
sort
of
pose
a
Spanish
query
and
be
pointed
to
an
English
page,
for
example.
B
So,
as
I
said,
I
mean
in
principle,
we
have
solutions
all
across
the
domain.
We
can
do
things
like,
for
example,
you
have
a
LinkedIn
profile.
You'll
describe
yourself
in
your
LinkedIn
profile.
I
can
make
a
fingerprint
of
your
profile
and
if
I
do
a
fingerprint
of
my
profile,
we
probably
have
a
lot
of
overlap,
as
we
are
interested
in
the
same
kind
of
topics
traditionally
to
make
matching
of
people
in
HR.
C
A
B
You
can
go
there.
You
find
a
white
paper
where
you
get
basically
a
more
in-depth
introduction
to
the
whole
approach.
You
find
access
to
a
public
rest
api
that
you
can
play
around
it's
trained
on
Wikipedia
and
English
Wikipedia
data.
You
can
then
even
spin
up
an
instance
containing
the
software
on
Amazon
or
Asia
to
play
around
if
you
have
more
proprietary
data,
so
that
you
that
you
want
to
use-
and
of
course
you
can
contact
us
if
you
need
help
to
sort
of
get
started,
I
mean
the
problem.
B
Is
that
many
of
us
who
have
been
struggling
using
conventional
tooling?
Sometimes
it
needs
a
little
bit
of
help
to
sort
of
get
the
right
angle
on
how
to
solve
something
yeah.
So
we
do,
for
example,
offer
a
keyword,
extraction
functionality.
You
can
throw
in
a
text
and
you
get
like
the
ten
most
important
keywords
out
of
it
and
I've
observed
that
many
people
try
to
systematically
extract
keywords
and
then
try
to
do
some
magic
with
that
and
I
just
told
them.
Okay,
that
keywords:
you
need
them.
B
A
A
Alright,
everyone,
that's
it
for
today's
show.
Please
leave
a
comment
on
the
show
notes,
page
at
twilly,
Icom,
slash,
talks
last
10
or
tweet
to
me
at
at
sam
Charrington
or
at
swim
le
I
to
discuss
this
show
or
just
reach
out.
Let
me
know
how
you
liked
it
thanks
so
much
for
listening
and
catch
you
next
time.