►
From YouTube: HTM Layers and Columns Meetup
Description
Join the conversation on HTM Forum: https://discourse.numenta.org/
This is a live-stream of the HTM Layers and Columns Meetup held on Nov 3rd in SF at Crown Plaza.
https://www.meetup.com/preview/numenta/events/243501858
The slides can be accessed here:
Jason Toy - https://www.slideshare.net/jtoy/touchnet-preview-at-numenta
Jeff Hawkins - https://www.slideshare.net/numenta/numenta-brain-theory-discoveries-of-20162017-by-jeff-hawkins
Marcus Lewis - https://www.slideshare.net/numenta/recognizing-locations-on-objects-by-marcus-lewis
For further discussion on the content of this meetup, see our forum at https://discourse.numenta.org/t/htm-layers-and-columns-meetup-nov-3-san-francisco-airport/2830
B
E
A
E
A
F
G
A
A
A
A
What
about
a
bacterium?
Some,
do
you
think
that
plant
is
intelligence
like
an
oak
tree,
something
plants
of
theirs?
Are
some
people
in
the
class?
Tell
it's
at
the
point
being
what
is
intelligence
so
I
just
like
to
point
that
out,
like
everybody,
has
a
different
idea
of
what
intelligence
is
you
never
know
until
you
talk?
Okay,
let's
do
something
more
fun.
What's
the
answer?
A
C
A
A
Do
you
think
that
sparrows
are
conscious,
slugs,
who
think
slugs
are
conscious,
love
any
fewer
hands?
What
about
trees?
Okay,
so,
based
on
this,
you
might
conjecture
that
some
things
might
be
intelligence
without
being
conscious
person
just
something
to
think
about.
Okay,
anyway,
if
you
don't
know
me,
I'm
Matt,
Taylor
I'm,
the
community
manager
for
them
into
open
source
HTM
work
thanks.
A
For
coming,
it's
good
to
have
an
event
again,
we
haven't
had
a
community
event
in
a
long
time.
So
thanks
everybody
for
coming
out
we're
live-streaming
this
in
case,
you
didn't
hear
my
other
two
warnings.
So
there's
already
several
people
watching
this
right
now,
so
we're
gonna
have
three
talks
tonight.
The
first
is
going
to
be
from
Jason
toys
and
I'll.
A
Let
you
do
your
introduction
in
a
moment,
then
we're
gonna
have
Jeff
Hawkins
our
co-founder
of
the
Mensa
come
and
talk
about
some
new
theories
that
we
have
in
a
paper
that
we
just
published
about
columns
and
the
cortex.
The
last
talk
will
be
from
Marcus
Lewis
who's,
a
research
engineer
and
he's
going
to
talk
about
a
location
in
the
brain
and
how
location
which
represented
in
the
brain.
But
first
of
all,
please
him
a
big
hand
for
Jason
toi
who's
going
to
talk
about.
I
K
I
I
I
K
I
And
just
put
hands:
had
people
come
in
and
put
hands
into
a
black
box
and
just
try
to
figure
out
what
the
objects
were.
Just
what
touch
and
so
I
figured
that's
something
that's
doable
and
that's
the
one
through
this
much
greater
version,
but
I
started
off
with
a
hand
that
could
go
to
the
box
and
can
touch
stuff
and
so
I'm
gonna
release
the
initial
version
in
the
next
couple
weeks.
So
just.
I
C
J
I
So,
on
top
of
that,
you
know,
another
expiration,
Helen
Keller
has
been
inspiration,
know
most
people
know
her,
but
she
was
born
around
her
fifty
years
ago.
She
was,
she
was
born
normal,
but
after
18
months
old
she
became
blind
and
deaf,
and
so
she
didn't
have
her
full
consciousness
and
her
full
mind.
Developed
I
was
until
around
11
years
old,
where
she
her
teacher.
She
had
a
teacher
who
was
trying
to
teach
her
how
to
become
a
member
of
society
to
be
intelligent.
I
You
know
her
teacher
taught
her
the
symbol
for
water
just
by
putting
water
on
her
hand
and
then
writing
a
simple
then,
after
that
she
she
learned
what
that
there
were
concepts
and
went
crazy
learning
concepts
because
she
wanted
to
she
wanted
to
get
to
ice
cream.
She
she
had
this
craving
for
ice
cream
and
she
wanted
to
get
that,
and
so
she
is
no
symbol.
I
But
what's
interesting
if
she
didn't
know
about
herself,
she
was
like
a
basically
an
amorphous
blob
floating
through
the
environment
and
just
reacting
to
things,
and
it
wasn't
until
that
her
teacher,
she
learned
a
concept
for
AI
and
me
that
she
knew
that
there
is,
you
know,
a
self
separate
from
this
environment
so
separate.
You
know
Jesus
agent,
acting
in
the
environment.
That's
that's
the
second
thinkin
it
can
feel
both
herself
and
and
the
environment,
and
she
talks
about
I've
read
all
her
books
she's.
I
She
wrote
a
book,
but
he
talks
about
in
the
books
how
her
whole
life
is.
Basically
it's
through
the
hand
and
to
touch
him
through
sensing
and
just
touching
things,
and
so
it's
very
interesting
I
recommend,
if
you
have
time
to
read
it,
but
it's
wanting
to
show
two
quotes
from
books.
So
one
of
them
every
object
is
associated
in
my
mind
with
textual
qualities
which
combined
and
countless
ways
give
me
a
sense
of
power
of
beauty
and
then.
I
See
only
a
very
small
portion
of
a
surface
and
that
was
passed
my
hand
continuously
over
it
before
my
touch
graphs.
The
whole
is
more
important,
however,
to
remember
that
my
imagination
is
not
tethered
to
certain
points,
locations
and
distances.
It
puts
all
the
parts
together
simultaneously
as
if
it
saw
or
knew,
instead
of
feeling
them.
I
So
thinking
about
all
these,
you
know
both
intelligence,
Helen,
Keller
and
some
of
the
research
that's
been
happening
now.
You
know
like
right
now,
a
lot
of
has
been
explosion
and
right
now,
that's
just
due
to
it's
usually
an
image
problem.
Staying
convolutional
accomplishing
their
own
networks
on
the
table
networks.
Those
are
all
image.
Processing
techniques,
there's
also
LTM,
which
do
some
time
to
india,
but.
I
Models
and
these
ARS
are
really
focused
on
static
models,
training
ones,
so
I
couldn't
get
a
binary
back.
Then
you
pass
data
through
it
like
a
classification
regression,
so
they're
not
really
they're,
not
really
meant
for
time.
Ls
p.m.
Towson
time
components
to
it,
but
my
mind
is
really
tacked
on
on
top
of
the
the
cord
algorithms
there's
there's,
of
course,
there's.
K
I
So
in
looking
at
these
algorithms
and
thinking
about
these
problems,
you
know
I
did
to
me
in
my
mind.
It
seems
like
you
know
time.
Is
it
it's
a
critical
core
component?
That's
not
really
at
the
core
of
these
algorithms,
and
so
you
know,
I
want
to
develop
this
testbed
touching
that
that
really
really
takes
gives
you
the
ability
to
researchers
and
and
developers
really
test
out
time,
components
and
also
motor
sensory
systems,
because
you
know
some
of
the
some
of
the
talks
were
going
to
hear
later
later
than
discussed.
I
It's
more
detailed,
but
you
know
if
you
look
at
the
example
of
Helen
Keller,
you
know
she's
moving
her
fingers
to
really
identify
what's
HAP.
So
if
you
think
about
like
an
object,
job
she's
moving
her
fingers-
and
you
know
that
hats
happening
over
time.
She
has
to
control
her
her
hand,
and
so
that's
using
her
motor
sensory
neurons.
That
system
and
she
has
to
be
aware
of
where
her
body
is
and
where
the
object
is
and
where
and
where
they
both
are
in
the
world.
So.
I
Components
that
have
not
really
been
studied
a
lot,
and
so
the
focus
of
touch
men
is
to
really
develop
these
algorithms.
It's
a
testbed
for
algorithms
and
so
I
just
want
to
share
this
quote
an
unknown,
but
potentially
large
fraction
of
animal
and
human
intelligence
is
a
direct
consequence
of
the
perceptual
and
physical
richness
of
our
environment
and
is
unlikely
to
arise
without
it.
I
So
touchnet
is
this
project.
It
consists
of
four
parts:
it's
it's
a
database
of
3d
objects
to
touch
and
interact
with
it's
say:
evolving
simulation
and
it's
an
expendable
API
and
in
their
benchmarks
of
collection,
benchmarks
to
measure
and
track
the
progress.
So
the
data
set.
These
are
these
are
examples
of
the
data
that
exists
in
the
environment.
It's
some
pyramids,
pencils,
cylinders,
circles,
cubes,
they're,
all
parametric
parametrically,
designed
so
procedurally
generated
all
the
code.
B
I
I
J
I
I
I
I
So
I
ended
up
using
this
engine
called
bullet
and
PI
bullet.
It's
a
considered
a
secret
weapon,
but
it's
so
it's
a
very
robust
physics.
Engine
that's
been
around
for
over
15
years.
It's
used
in
many
games
movies
that
you
guys,
probably
I've,
seen
or
played,
and
it's
very
easy
to
install
and
so
I
don't
use.
Bullet
Bill
is
a
C++
engine
and
I
write
everything
in
Python,
so
I
use
the
Python
bindings.
But
the
end
in
itself
is
a
it's
a
physics
engine
that
is
rigid
and
soft
body
simulation.
So
a
cloth
solid.
I
Since
2003
and
it's
now
being
developed
more
Google
used
for
robotics
and
learn,
so
this
is
what
I
want
to
try
to
simulate
right
now.
So
this
is
this.
Is
this
diagram
is
a
picture
of
a
skin
and
so
far
as
we
know
that
there
there
are
four
types
of
sensory
neurons
on
the
skin.
There's
the
Ruffini
particles,
Merkel
styles,
tactical
popsicles
and
pacinian
corpuscles.
They
measure
pointing
edges,
temperatures
little
vibrations
stretching
on
the
skin.
I
I
Very
large
that
they're
taking
so
right
now,
you
know
it's
nowhere
near
that
what
I
have
instead
is
robotic
hand
with
a
200
by
200
sized
square.
So
like
a
bit
like
an
image,
was
by
high
on
the
tip
of
the
index
finger
and
that
index
thing
there
when
it
goes
towards
the
objects
and
touches
it,
it
will
get
zeros
and
ones
when
the
raid
zero
one
zero,
meaning
note
option,
one
meaning
touch.
I
I
K
E
I
I
K
C
I
Said
it's
an
evolving
simulator!
My
my
goal
is
to
it's
to
release
the
first
one
and
hopefully
that
will
generate
interest
from
other
scientists
and
neuroscientists.
You
know
how
to
really
implement
or
a
better
model
of
what
the
hand
would
look
like
and
so
I
want
to
work,
but
with
some
some
people
an
open-source
and
try
to
get
better
and
better.
So
you.
I
I
So
I
want
to
talk
about
the
the
API
for
this,
so
it
was
really
important
for
me
to
make
this
as
simple
and
easy
to
work
with,
as
I
mentioned,
trying
to
use
some
of
those
physics
simulators
that
we're
all
pretty
painful
to
install
I
talked
to
actually
a
lot
of
the
original
developers
for
many
other
projects.
You
know
some
of
them
take
days
to
install
or
the
install
instructions,
don't
even
work,
and
so
that's
a
real
turnoff
of
people
are
trying
to
build
stuff.
I
J
I
So
the
api's
are
based
off
of
a
reinforcement,
learning
style
API,
which
just
means
that
you
know
if
this
is
the
nomenclature
for
more
traditional
machine
learning,
but
having
an
agent.
That's
where
we
would
write
our
code
and
then
the
environment
is
touch
net.
That
responds
that
you
know.
That's
the
environment.
The
agent
has
to
choose
some
action,
that's
moving
their
finger
or
the
hand
is
one,
those
26
actions
that
action
gets
passed,
the
environment
and
then
there's
a
state
which
is
the
the
array
for
that
represents
the
touch
and
then
the
reward.
I
Ai,
which
is
another
company,
doing
research
in
longer
term
machine
learning
they
do
learning,
and
so,
if
you,
if
any
of
you
have
done
code
coding
in
an
open,
AI
or
just
plugging
the
code
and
the
whole
thing
should
run
so,
there's
a
quick
example
of
the
opening
eyes.
Example,
you
know
it's
very,
very
short
code
and
ours
is
almost
almost
the
same
thing
so
import
the
environments,
render
it
and
then
line
tenure,
I
just
choose
a
random
action
and
then
line
11.
I
G
I
So
another
piece
of
this
is
that
benchmark
I
mentioned
so
in
in
more
classical
machine
learning.
You
know
we
have
regression
classification,
classification,
being
you
know,
the
image
sense,
give
it
a
bunch
of
images
it
trains
on
and
then
give
it
some
brand
new
images.
Is
this
a
dog
or
cat?
It's
gonna
tell
you,
that's
that's.
K
I
A
lot
of
businesses
use
machine
learning
in,
in
my
case,
I
built
a
benchmark,
which
is
it's
also
off
the
classification,
but
it's
blind
because
the
hand
it's
not
getting
any
event,
it's
getting
it's
a
hand,
that's
trying
to
touch
right,
and
so
so
the
goal
is
the
same
as
regular
classification.
Try
to
try
to
tell
us
is
this
object,
a
dog?
Is
it
a
box
of
sphere,
in
our
case
a
pencil
and
and
so
because
this
is
a
reinforcement,
learning
style,
environment,
I
used
the
reinforcement,
learning
algorithm
I?
I
What
I
do
is
I
the
example
I
wrote
is
really
it's
it's
to
deep
neural
networks.
One
is
a
convolutional
neural
network,
which
is
for
images
and
the
other
one
is
a
a
reinforcement,
learning
algorithm,
called
actor
critic
and
so
I'm
training,
one
that
learns
how
to
classify
the
touch
data
is
this:
what
is
this
class?
Is
it
a
dog
and
then
the
actor
critic
algorithm?
All
that
does
this
learns
how
to
move
the
hand
towards
the
other,
so
I
just
trained
them.
Concurrently,
I,
don't
know.
I
If
I
did
it
right
because
I'm
not
a
real
reinforcement,
learning
person
and
I
wrote,
I
wrote
the
code
hi
torch
and
tensorflow.
These
are
two
popular
machine
learning
frameworks,
but
mostly
probably
no
now
I
just
got
word
from
last
night,
so
the
accuracy
is
incredible.
Four
percent,
you
know:
that's
that's
really
little
numbers
that
weeds
out
of
you
know
out
of
a
hundred.
It
only
gets
it
gets
four
right.
So
it's
basically
chunk,
but
I
expect
that,
because
I'm
using
a.
K
J
I
I
I
These
two
live
this
line
these
lines
here
they
just
set
up
the
optimizers
and
here's.
Here's
where
the
actual
stepping
through
the
environment
is
happening,
so
I
select
an
action
when
it
starts
it.
Does
that
random
thing
I
showed
you
as
it
gets
smarter
to
handle,
will
become
smarter
and
not
go
crazy
and
maybe
just
use
a
single
finger
and
go
forward
towards
the
toward.
I
I
I
I
said
so,
the
whole
goal
of
this
is
to
really
be
a
benchmark,
a
tool
set
for
researchers
to
develop
and
test
out
these
new
algorithms
and
and
then
benchmarks
for
people
to
test
this
out
and
see
how
they
perform
against
the
other
algorithms.
And
so
you
know,
I'd
love
to
see
an
HTM
version.
I
haven't
played
with
the
latest
HTM
code,
but
years
ago,
I
played
with
it.
I
would
love
to
see.
If
you
know.
K
F
F
D
I
Fingers
right
now,
we
can
feel
the
difference
between
wood
and
sandpaper
and
rocks
and
stuff.
So
that's
the
high
level
what
it
is,
and
you
know
it's
a
pretty
ambitious
project,
especially
for
me:
I.
Don't
really
I'm,
not
an
expert
in
any
of
this
kind
of
stuff,
but
generalists
so
kind
of
barely
works,
I'm
going
to
open
source
the
whole
thing
in
about
a
week
or
two
and
yeah
I'm.
Looking
for
anyone
who
wants
to
help
contributors
in
neuroscience,
math
now
and
I'm,
looking
also
for
anyone
who
wants
to
maybe
fund
this
too.
I
B
I
Of
freedom
I've
been
tweaking
the
physics
engine
a
bit
and
so
the
fingers
right
now
they're.
They
can
go
basically
full
range
motion,
all
the
way
the
hand
fingers
can
go
all
the
way
backwards.
So
that's
over
200
degrees
a
little
bit
too
far.
So
I
need
to
tweak
that
in
terms
of
simpler,
I
mean
I
think
what
I
have
now
is
already
the
simplest
hand.
So
I
don't
want
to
go
any
simpler.
I
mean.
I
I
Get
a
lot
of
feedback
hearing,
hopefully
during
this
period,
nerd
socks
and
that
changes
to
improve
it
so
I,
don't
see
it
becoming
simpler.
I
see
it
becoming
a
lot
more
sophisticated.
So
you
know,
as
I
mentioned
right
now,
it's
just
it's
200
by
200
array
of
touch
sensors,
but
you
know
the
the
skin
the
whole
body.
You
know,
like
the
question
that
asked
earlier
about.
I
What's
intelligent
and
what's
conscious,
you
know
to
me
that
answer
is
everything
that's
alive,
because
everything
that's
alive
has
some
type
of
interaction
with
the
environment
and
that
interaction
usually
is
with
a
touch
or
motor
sensory
neurons
or.
However,
you
want
to
call
it.
You
know
most
things.
E
I
Problem
yeah
I
mean
that
that
could
be
interesting.
I
mean
my
longer-term
goal
is
that
you
have
I've
had
this
dream
that
I
want
to
have
like
a
full
Helen
killer,
robot
roaming
inside
of
this
environment
feeling
her
way
around
and
becoming
it
tells
it
that
way.
So
you
know
I'm
trying
to
work
my
way
up
to
that,
but
you
know
they
break
right.
Now.
There's
no
use
cases
like
for
business.
Yet
for
this,
but
you
know
I
see
this
longer
term
is.
I
E
I
You're
right-
and
you
know
maybe
we'll
see
what
happens
it
could
be.
This
whole
thing
doesn't
work
at
all,
but
I
think
that
it's
worth
trying
and
yeah
I
mean
the
hand
and
I
can
people
can
relate
to
the
hand,
because
it's
kind
of
it's
humanoid
and
does
another
reason
for
doing
is
just
because
people
can
relate
to
it
more.
C
I
G
M
Since
your
substitution,
that
you
could
have
I
mean
this
woman,
she
learned
the
world
through
touch.
That's
it
right.
No
sight,
no
sounds,
and
yes,
her
model
the
world
was
a
gun
cannot
afford
the
year
in
my
model,
the
one
she
knew
all
the
same
things
we
did.
She
won't
say
concept.
She
learns
all
the
things
about
language
and
about
the
world
that
no,
it
works,
and
yet
she
only
did
that
through
touch
and
that
gets
to
the
universal
properties
of
how
the
neocortex
works
because
doesn't
matter
what
your
sensory
organ
is.
M
M
M
M
M
Our
goal,
that
might
say
we're,
engineer
MIT.
We
want
to
be
biologically
accurate,
we're
not
interested
for
inspiration.
We
really
want
to
know
the
biological
theory
about
how
the
neurons
in
your
head
work
and
it
should
be
testable
empirically,
that
is
through
neuroscience
and
things
and
also
the
relationship.
It's
an
ambitious
goal
and
that's
my
primary
goal.
We
have
a
secondary
goal
which
is
generally
secondary,
which
is
we
believe
the
work
we're
doing
is
going
to
enable
a
technology.
That's
going
to
be
the
foundation
for
the
future.
M
Beyond
what
we
have
today
and
some
of
the
leaders
in
AI
our
base
is
singing
provocative
brains
or
engineering.
So
that's
what
we
do
to
enable
our
technology
and
that
ones
are
open
source
to
the
great
chocolate
pen
and,
if
you're
wondering
what
our
businesses
well,
there
isn't
much
of
a
business
today,
but
the
business.
M
Tonight
now
this
is
a
this.
Is
a
video
before
I
talk
about
my
manual
version
of
this.
My
model
of
the
brain,
and
it
consists
of
about
70%
of
the
volume
of
the
brain,
is
the
existing
ring
of
the
thing
around
the
top,
probably
using
a
handheld
mic.
We
use,
and
so
the
old
brain
is
sort
of
stuck
up
in
the
middle
of
their
emotions.
M
You
know
sex
and
drives,
and
things
like
that,
but
the
new,
your
cortex
is
this
big
thing
that
wraps
around
the
rest
of
it
and
in
only
mammals,
have
a
neocortex
non
mammals.
Don't
have
it,
and
in
humans
is
quite
large
if
you
could
remove
it
from
the
rest
of
the
brain
in
our
min
flat,
which
you
can't
do,
but
it
could
look
like
this
is
about
this.
M
Size
is
luke
dinner,
napkin,
it's
about
the
size,
it's
a
little
thicker
than
two
millimeters
thick,
but
not
much
thicker
than
this,
and
the
new
projects
is
divided
into
regions
of,
do
different
things,
languages
such
thought
processes
and
all
kinds
of
vision,
and
so
on,
but
anywhere
you
look
in
the
neocortex.
It's
there's
this
complex
architecture
and
it's
remarkably
the
same
everywhere.
There
are
some
differences
here
and
there,
but
it's
unbelievable
how
similar
it
is
and
that
detailed
architecture
in
the
matter.
What
part
it's
doing
well,
it
looks
like
a
language
here
in
touch.
M
It
looks
another
identical
and
that's
true,
even
if
you
go
look
at
other
animals
and
others
other
sensors,
you
know
rats
whisking
and
things
like
that.
So
there's
this.
The
basic
theory
has
been
years
that
there's
a
common
algorithm.
That's
underlying
everything
than
your
projects
does
that
can
be
applied
to
all
these
different
things
and
our
brains
got
baked
because
we
replicated
that
algorithm.
So
now,
you're
gonna
see
this
cool
video
of
it
and
that
happening
which
is
gonna.
M
And
then
it's
going
to
show
you
here
was
I
didn't
mention
these
columns
now
show
you
the
columns
and
then
your
cortex
don't
look
physical
like
that.
They
don't
exist
as
physical
entities,
they're
small
things
with
many
colleges
that
is
not,
but
there's
a
structural
sort
of
functional
column.
That
neuroscience
is
all
sort
of
agreeing.
Most
of
us
agree
that
that
exists
and
that
information
is
processed
in
this
comp.
M
That
information
comes
into
this
little
thing
up
and
down
this
column
and
that's
going
on
parallel
columns
all
around
there's
a
couple
hundred
thousand
of
those
in
your
brain,
and
if
you
look
at
one
of
those
columns
from
one
part
of
the
brain
and
the
other
part
of
the
brand
or
complex,
they
look
identical.
So
that's
our
tool
really.
If
you
want
to
send
any
other
projects,
you
want
to
understand.
What's
going
on
in
that
little
block
tissue
and.
M
If
you
go
swimming
again,
I
didn't
okay,
if
you,
if
you
go
to
a
neuroscience
textbook,
you'll
find
this
very
simple
diagram
says:
well,
here's
a
picture
of
a
column
and
it's
got
different
layers
in
it.
So
they
typically
say:
there's
six
layers
of
cells,
labeled
l1
through
l6,
and
they
say
the
information
comes
in
through
sensors
and
goes
to
layer.
Four,
that's
the
primary
layer.
Layer
four
protects
the
layers
really
and
then
layer
three
is
the
output.
That's
just
do
this
in
every
neuroscience
of
tech
department,
it
kind
of
makes.
M
M
M
Can
you
see
so
this
is
sort
of
a
common
feature
or
cartoon,
and
this
is
the
real
thing
we're
trying
to
figure
out
and
it's
much
more
complicated
than
the
simple
things
that,
and
so
we
spend
our
days
in
detail
going
through
these
things,
trying
to
figure
out
how
they
work.
Now
the
media
we've
made
some
good
progress.
We
have
two
papers
that
I
can
see.
We
that
came
out
in
the
last
two
that
are
sort
of
the
cornerstone
of
our
work.
There's
more
papers.
M
You
can
see
all
these
things
that
the
meant
about
coms
less
papers,
but
these
two
are
sort
of
like
the
cornerstone
on
the
Keystone
papers.
For
us,
one
came
out
March
of
last
year
called
white
noise
and
thousands
of
synapses,
the
theory
of
sequence
memory
in
your
cortex
and
the
second
one
that
just
came
out
two
weeks
ago
called
a
theory
about
columns
in
the
neocortex
enable
learning
in
the
structure
of
the
world.
These
are
all
published
in
frontiers
in
neural
circuits.
The
first
paper
introduces
two
major
concepts.
The
first
is
a
neuron
model.
M
Talk
about
that
and
then
we
introduced
how
those
real
neurons
work
in
layers.
Now,
how
would
you
put
thousands
of
them
in
any
single
layer
itself,
one
of
those
top
layers
and
how
we
will
learn
temporal
patterns
that
will
learn
sequences,
and
so
it's
about
sequence
memory,
but
we
think
that's
a
core
thing,
because
it's
a
predictive
sequence
memory,
because
the
next
elements
of
changing
in
time,
based
important
speakers
phenomenon
for
your
man
just
came
out
two
weeks
ago-
starts
with
the
old
one
left
and
it
shows
how
columns
in
the
neocortex
models
talking
them.
M
M
The
locations
are
coming
into,
your
brain
is
completely
changing.
It's
really
different,
you
perceive
the
world
is
stable
and
you
perceive
the
world
as
it
is
not
as
it's
changing
on
your
sensors,
just
like
mitosis,
this
brain
of
this
cup
here,
I
perceive
it
even
though
my
centers
are
looming
is
simply
the
same
and
then
the
poor
thing
is
this
magic
missing
ingredient.
Now,
if
you've
read
the
paper,
you
know
what
this
is,
but
it
can't
read
the
paper
I'm
going
to
keep
it.
We
actually
discovered
I
think
something
extremely
important
for
neocortical
theory,
I.
M
M
Which
you
prefer
a
or
then
this
Mike,
this
one
there
we
go,
then
you
knock,
you
add
these
together,
you
put
it
through
some
nonlinear
function
and
you'll
get
your
output,
so
these
neurons
typically
have
skill
or
inputs
and
scalar
outputs
in
me
and
that's
how
they
work.
This
is
not
a
neuron
real
reason.
I
like
this
at
all
this
way
they
thought
this
100
years
ago,
but
that's
not
true.
M
M
Between
five
and
thirty
thousand
synapses,
never
less
than
a
thousand
that's
a
lot
of
synapses.
The
artificial
neurons
don't
have
anything
like
that.
Less
than
10%
of
the
hole
we
call
proximal
near
the
cell
body
and
those
can
make
mostella
become
active.
They
can
generate
a
spike
over
90%
of
them
are
distal
either
out
here
on
this
implicit
branch
of
here,
and
they
don't
seem
to
be
able
to
make
the
cell
fire
at
all.
M
M
That's
fine,
just
discovered
if
you
these
dendrites,
are
active
processes
and
if
you
have
a
somewhere
around
15
synapses
become
active
at
the
same
time,
within
a
few
milliseconds
and
meter
bonds
and
40
microns
of
each
other,
then
it
generates
a
dendritic
spike,
which
is
different
than
the
action
potential
comes
out
of
the
cell
body,
but
the
generation
by
child
is
going
to
dendrite
and
goes
to
the
body.
Now
it
does
something,
and
so
these
are
like
got
a
coincidence,
detectors
now.
M
It's
interesting
that
the
dendritic
spike
itself
is
not
sufficient
to
make
lists
they're
on
fire.
It's
like
hey
information
to
do,
but
it's
not
enough
to
do
anything
with
it.
Our
proposal,
in
our
theory,
is
and
I'm
very
confident
this
is
correct.
Is
that
what
it
does
is
what
we
know.
This
is
true,
then,
in
polarizes
the
cell
changes
the
internal
functions
ourself
or
something
recently
long
period
of
time,
and
that
cell
is
now
as
closer
to
firing,
and
it's
like
a
prediction.
M
The
self
is
going
to
predict
that
it's
going
to
become
active
sooner
and
we
hold
advantage
of
this
process
is
when
a
cell
is
in
this
predictor
state,
when
it
does
become
active
through
the
feed
program,
but
becomes
active
a
little
bit
quicker
and
it
beats
up
other
cells
and
that's
the
key
to
the
mechanism.
So
we
modeled
this
in
the
agency
neuron.
This
was
in
the
2016
paper
where
we
say:
okay,
a
neuron
actually
has
a
whole
set
of
coincidence.
Detectors
one
of
them
is
other
feed-forward.
Proximal
ones
are
the
seven
patterns.
M
And
most
of
us
power
to
be
polarizing
the
neuron
or
putting
a
stone
to
a
particular
state.
So
that's
that's
a
shin
neuron
model.
We
don't
know
the
thing
about
our
normal
model
is
in
learning
many
years.
People
thought
that
learning
was
the
changing
of
the
wait
on
the
efficacy
of
the
synapse,
and
some
of
that
occurs,
but
that's
not
what
most
learn
your
courage.
We
now
know.
Biologists
know
the
most
little
learning
occurs,
because
these
two
notches
are
disappearing
in
being
formed
all
the
time
they
sometimes.
M
It
just
gets
more
permanent,
and
so
we
model
this
in
our
there
on
with
us,
which
goes
from
zero
one,
the
more
you
train
the
neuron,
the
number
this
number
goes
up
at
some
point:
you
go
from
the
the
synapse
because
you're
not
connected
to
connected,
so
the
weight
goes
from
zero
to
one
into
binary,
synapse
and
that's
closer
to
the
biology.
It's
a
much
more
powerful
way
of
learning
than
modifying
the
weight
of
the
synapse.
This
is
a
really
stupid.
They
don't
they're,
not
very
you
can't
put
them.
M
M
We've
done
work
on
this
per
years,
but
this
is
a
figure
from
that
paper,
which
shows
that
if
you
have
a
a
cartoon
version
would
be
thousands
in
there
on
a
little
dr..
Nam's
is
as
that
we
can
show
him
the
kind
of
representation
you
get
in
this
thing,
learning
sequence
is
basically
when
a
neuron
becomes
active,
it
looks
for
connection,
performs,
connections
or
other
neurons
are
active
a
moment
before,
and
so
basically,
every
neuron
is
trying
to
predict
when
it's
going
back
in.
That
is
the
essence
of
a
sequence.
M
And
okay,
instead
of
neurons
and
what
we
cylinder
and
a
very
powerful
sequence
memory.
So
that
was
all
in
that
first
paper
now
I'm
going
to
talk
about
some
paper
that
came
out
two
weeks
ago
and
the
problem
with
here
now.
This
is
very
much
to
justify
here.
I'm
gonna
start
with
this
problem,
we
asked
we
wanted
to
ask
the
question:
how
does
the
coordination
any
predictions
we
first
talked
about?
How
to
make
predictions
of
sequence
is
like
a
melody
and.
M
When
you
sense
something
so
we
we
looked
at
vision,
we
get
lost
up
when
we're
thinking
about
touch,
and
now
here's
think
about
this.
I
don't
have
to
look
at
this
I.
Have
this
thing
in
my
hand,
I
know
this
cup
I
drink
from
it
every
day,
I
watched
it
before
drug
shortages
and
I'm,
touching
you
with
my
hand
or
my
fingers
and
as
I
move.
My
fingers
over
this
object.
I
predict
when
I'm
going
to
feel
I
can
even
imagine
so.
M
I
move
this
finger
up
to
the
top,
and
so
I
mean
I
make
that
prediction
before
I
do
it
now
that
condition
is
happening
for
every
little
part
of
my
skin
constantly
all
the
parts
of
my
skin
there's
most
these
predictions
you're,
not
aware
of,
but
if
anything
was
different
there,
something
on
noodle
about
the
cup
I
would
notice
it.
So
we
can
be
certain
of
the
brain
is
making
any
predictions
now,
here's
yourself!
What
does
it
take
to
make
this
good?
M
How
would
a
piece
of
cork
ex-associate
if
my
finger
green
you're
running
bases
its
in
there,
make
this
prediction?
Well,
first
thing:
it
has
to
know:
one
thing
has
to
know
the
object.
It's
touching
cuz
to
make
the
minute
I
would
make
a
different
prediction
for
this
copper
from
the
brain
of
an
electron
or
something
like
that.
Fine,
even
on
the
object,
I
also
need
to
know
where
the
finger
is
going
to
be.
M
M
That's
not
where
the
finger
is
knowledge
of
my
body.
This
way
is
the
man's
oven
here
here
here
here
predictions
that
I'm
more
the
object
and
finally
happen.
You
have
to
know
the
orientation
of
the
finger
on
the
object
and
I'm
not
going
to
talk
about
that
more
in
it,
but
the
third
ingredient.
So
this
led
us
to
we
used
a
sort
of
property
revealed
you
here's
a
picture
of
another
slice
of
the
cortex
these
those
six
layers.
M
This
is
not
that
basic
circuitry
I
mentioned
before
right,
where
the
input
comes
in
from
sensors
and
goes
to
layer
four
layer
for
projection
layer.
Two
three
layer,
two
three
becomes
the
output.
This
is
your
classic
basic
cortex
I've
shown
that
layer
choose
meself,
send
these
long
time,
job
horizontal
projections
to
two
other
columns.
M
Now,
there's
another
thing:
that's
known
from
the
anatomy
which
I'm
going
to
add
onto
here
that
roughly
50%
of
the
synapses
other
connections
to
the
layer,
four
cells
come
from
layer,
six
egg,
we're
not
about
now
remember
there's
only
like
seven
or
eight
percent
of
the
synapses
are
coming
from
the
actual
sensory
like
the
finger
or
the
eyeball,
or
something
like
that
is
so
good.
Half
of
them
are
coming
from
the
other,
this
other
location
in
that
column.
Well,
what
we're
going
to
say
it
is
we're
going
to
say
that
is
the
location.
M
That
is
the
signal
where
that
is
telling
you
where
this
sensory
input
is
relative
to
the
object.
That
is
being
sensed
so
in
the
case
of
the
cup
I
just
talked
about
is
where,
on
the
cup
that
this
finger
is
about
the
touch
when
I
move
my
finger
there.
If
I
was
looking
at
this
object,
and
that
was
a
common,
a
column
in
the
visual
cortex
there
same
thing
would
happen.
It
sounds
a
little
magic.
I
know
this.
It
sounds
right,
but
that
section
of
the
cortex
is
associative.
N
M
Matter
where
the
object
is
and
where
my
fingers,
that
were
the
signal
remember
all
to
my
body-
that's
that's
being
an
extension
thing
and
that
turns
in
neuroscience
in
tune
is
called
allocentric,
meaning
the
other
etc.
On
the
other.
We
are
then
going
to
proposed
that,
when
the
input
comes
in
to
this
region,
layer
four
is
gonna
form
a
written
representation,
that's
unique
to
both
the
location
and
the
feature.
You're
saying
this
isn't
end
at
that
location,
on
the
object
and
if
I
was
on
the
location
would
be
would
be
encoded
differently.
M
And
that
information
is
being
passed,
the
next
languages
out
there,
and
now
they
are
we're
going
to
say
as
the
object
and
what
it's
doing
is
determined
pooling
in
neuroscience.
It's
essentially
learning
to
recognize
all
the
different
feature,
location
pairs,
and
it's
like
a
can
file.
Isn't
really
that
simple
is
saying
this
coffee
cup
consists
of
a
set
of
features
and
locations
and
I'm
going
to
remember
them
and
reformists
table
representation
to
them.
M
I
said
if
I
see
any
one
of
those
big
duplications
and
that
representation
of
encounter
and
so
I
just
this
is
the
same
thing.
I
just
said
every
column
determines
is
now
the
centric
location
of
input,
essential
moves
and
the
sense
it
was
a
common
suppose
in
different
future
locations
in
the
objects.
One,
that's
your
eyes
moving
or
your
fingers
are
moving.
The
output
leg
pools
those
specifications
and
it
becomes
stable
over
moments.
The
other
is
getting
be
stable,
even
though
the
inputs
change
in
all
of
them
and
output
is
just
the
object.
M
And
what,
if
you
do
this
in
a
single
column,
can
learn
complete
hundreds
of
computers?
Even
a
hundred
thousand
words
can
with
hydrogen
plate
on
it,
the
shape
and
the
morphology
of
them.
It's
going
to
be
more
than
that,
but
we
didn't
talk
about
that
as
a
baby.
Now
this
thing
at
this
location
thing
one
of
the
gentleman
the
audience
here,
because.
M
M
M
Cortex
and
heavily
studied
people
won
the
Nobel
Prize
for
it,
but
it
sounds
a
very
similar
problem,
interesting
the
magic
tricks
that
the
neurons
do
to
do
this,
and
we
look
for
that
and
figure
the
same
here
instead
of
FINA
green
cells.
Do
they
tell
you
where
you
are
in
the
my
body
is
relative
to
this?
We're
not
doing
that.
We
just
want
to
figure
out
where
our
senses,
our
relatives,
objects
it's
very
similar
product,
so
market,
that's
with
Marcus
is
talk.
M
He's
been
working
on
that
income
of
really
great
ideas
lament
then
I'm
going
to
continue
on
which
in
the
paper.
So
then,
if
you
have
multiple
columns
like
three
fingers
on
your
hand
and
you're
touching
the
object
three
points
at
the
same
time.
Well,
we
have
three
different
engine
features
you're,
going
to
three
different
locations,
each
column
on
its
own,
not
with
one
sensation,
but
they
all
are
guessing
and
they
can
say
well,
one
column
can
say
well,
this
could
be
a
coffee
cup.
M
M
M
I'm
going
to
make
sure
we
assume
the
location
without
the
touch,
the
location
invokes
a
set
of
possible
features
that
might
occur
at
a
location
like
that
and
then,
when
it
touches
it
actually
gets
the
feature.
It
said
that
unless
consistent
with
one
of
the
ones
I
so
I
know,
this
is
a
feature
of
that
location
and
cousins
at
the
other.
They
are
the
ultimate
layers
as
well.
It
could
be
any
one
of
these
three
objects,
it
could
be.
A
cup
could
be
a.
M
Have
a
rounded
edge
like
that,
then,
when
you
move
to
the
next
location,
it
gets
a
new
location,
it's
okay,
I'm
predicted.
This
is
where
my
new
locations
come
to
me.
If
it
features
you
can
get
their
future.
This
is
up
to
the
top
layer.
It
says:
okay,
that's
consistent
with
me.
Come
from
the
camera
is
inconsistent
with
the
ball,
so
it
starts
narrowing
down
the
set
of
hypotheses
and
he
goes
to
the
third
location
and
get
a
new
location
feature.
M
Proper
representation
with
three
touches,
then
this
scene,
configuring,
is
not
not
going
as
on
the
ball.
It's
a
cup,
that's
the
basic
idea.
If
you
do
that
with
three
columns
at
once.
Here
you
are
grasping
it
and
you
touch
the
hand
on
three
columns
at
once
through
the
three
fingers
at
once.
Each
one
gets
its
own
location.
M
This
is
just
a
data
set,
there's
nothing
else
to
it,
and
it's
for
robotics
and
so
on.
So
we
work
through
that
and
here's
our
ears
our
simulation.
This
is
not
just
a
million
the
immunity
we
put
sensors
on
and
of
each
finger.
We've
made
very
simple
sensors,
which
detect
curvature
at
various
levels
of
curvature
on
the
end
of
the
tips
of
the
fingers
and
convert
those
in
the
sparse
vector-
and
we
did
this
with
three
columns.
It's
by
2025.
M
J
M
M
M
M
Had
to
go
monitor,
there's
a
lot
of
other
things
going
on,
there's
a
lot
of
other
data
about
how
cell
responses
are
modulated
by
movement
and
are
so
location
signals.
This
is
all
kind
of
crispy
nothing's
like
it's
not
like.
You
know
the
definitive
thing
like
that
pendants
border
armacham
cells,
but
this.
M
Earlier
there
is
a
strange
called
grid
cells
which
solve
a
very
similar
problem
and,
and
they
figure
out
where
your
body
is
in
an
environment
where,
as
opposed
to
where
your
finger
is
on
either
it's
a
very
similar
problem,
I'm
not
going
to
go
through
the
difficulty
of
this
problem
is
tonight
and
explorers
of
the
solution,
but
nature's
just
coming
really
clever
ways
of
solving
this
problem.
So
you
know
that
neurons
can
do
this
and
and
then
we
said,
let's
look
at
the
grid.
Styles
inspiration
how
the
cortex
does
that
alright.
M
So
in
summary,
things
have
done
things
you
published
over
the
last
year.
There's
a
lot
of
stuff
proceeded
that
we
just
didn't
swipe.
We
weren't
publishing
in
scientific
journals
at
the
time
is
we
have
a
new
neuron
model?
My
model
is
biologically
more
realistic
and
more
powerful.
It
recognizes
honeymooning
powers.
M
Yeah
most
of
the
parents,
with
the
noise
of
predictive
state
and
learning,
is
being
the
grocery
or
synapses
some
of
the
people
who
studying
the
world
and
would
live
in
more
sizes
living
the
world
with
neurons
love
this
theory,
because
it
gives
them
an
explained
reason
why
these
things
exist
and
we're
starting
to
collaborate
with
them.
But
we've
been
introduced
a
cellular
layer
models
which
shows
how
you
basically
predictive
models
of
sensory
input,
and
it's
all
about
time
base
changes.
M
You
first
apply
the
temporal
sequences
like
Nellie's
in
the
world,
and
now
we
provide
sensory
motor
sequences
same
basic
model.
Then
we
can
do
three
images.
We
didn't
find
it
in
there,
but
we
notice
that
there's
an
elephant
replication
must
be
in
every
column
and
that
for
every
common
ones,
include
models
of
objects
and
quickly
and
in
the
numbers
one
leave
on
this
pot
here
this
this
discovery,
the
all
center
location,
I,
see
changing
everything
in
my
mind,
it
does
frisson
tells
us
that
concentrators
are
far
more
powerful
than
cream.
M
M
M
M
And
I
think
huge
implications
for
robotics
machine
intelligence
and
the
intelligence
in
general.
I
can't
make
a
definitive
case
on
this.
This
is
how
our
work.
It
turns
out
that
everything
we
do
is
some
place
in
all
of
our
plots.
All
of
our
perceptions
have
a
place
like
this
circuitry
that
exists.
I
told
you
about
for
determining
the
location
exist.
M
Everything
the
cortex
does
is
being
manipulated
in
a
spatial
context
in
manipulated
in
spatial
context
and
I,
don't
think
even
build
a
fluid
robotics
or
until
the
machines,
without
it
I'm
not
saying
what
that's
going
to
happen
on
this
thing,
that's
I
think
that's
going
to
happen,
so
I
think
it's
really
exciting
than
that
point.
I
do
too.
So
that's
it.
A
M
M
K
M
The
question
answer
around,
so
the
layers
are
really
misleading.
What
really
matters,
of
course,
is
what
kind
of
neurons
and
who
they're
connected
to.
If
I
took
a
bus
from
neuron
in
there,
too
and
I
moved
it
down
to
layer
four,
but
I
kept
all
of
its
connections.
It
would
still
be
the
same
neuron.
It
was
still
been
doing
something
that
lectures
us.
In
fact,
there
is
a
genetic
Maps
that
we
learned
about
recently,
where
the
neurons
in
the
layers
are
all
scrambled
up
to
the
layers
are
all
over
the
place.
M
Together,
but
it
doesn't
really
matter
where
the
layers
begin
or
end,
it
really
matters.
What
cell
types
when
we
really
talk
about
twelve
different,
saying
he's,
told
them
types
of
criminal
cells
in
terms
of
their
connections
like
there's
these
ones.
That
cannot
come
here
here
here
and
there's
another
one
looks
just
like
that.
Connects
the
difference
really
Tree
of
Life.
M
We
don't
do
that.
The
Porsche
goes
really
really
telling
people
out
there
who
do
this
experimental
work,
and
there
are
hundreds
of
papers,
letters,
so
they
have
all
kinds
of
techniques.
They
can
sustain
themselves,
so
they
they
fill
the
style.
So
you
can
see
the
entire
cell
in
the
image
it
they
put
dyes
in
there
they
go
backwards,
they
comprise
them
to
go
forward,
they
slice
them
up
and
put
electron.
M
M
M
The
difference
is
what
are
your
inputs
are
coming
from
and
now,
if
I
keep
my
region's
connecting
the
reasons
about,
if
I
have
a
reason
that
skin
touch
it's
going
to
form
representation,
two
columns
that
are
combination
of
those
two
and
if
I
get
something
else
in
that,
so
there's
actually
there's
other
inputs
not
aware
of
other
things,
the
cortex,
and
so
the
reasons
are
basically
defined
by
what
their,
what
their
inputs
are.
That's
it
that's
highly
specialized
I'm
connected
to
the
I
I
become.
D
D
M
M
And
they
said,
birds,
the
birds,
they
study
and
they
said
there's
something
very
matters:
they
don't
call
it
cortex,
but
there's
the
ball
infections
and
things
they
haven't
turned
all
of
associate
blobs,
but
they
seem
to
match
my
phenomena.
They
are
very.
Similarly,
the
bird
has
a
dimple
and
ball,
which
is
changing
up
a
little
stable,
seem
to
be
doing
pulling,
and
so
it
was
eyes
they
were
saying.
M
N
The
this
sensing
identifying
an
object
and
a
hollow
centric
and
all
that
missus
to
me
it
misses
the
HTM
part
of
it,
but
I
thought
there
was
this
sort
of
temporal
thing
that
was
in
in
man,
explained
at
the
five
o'clock
talk
about
how
you
have
the
you:
have
the
proximal
in
the
end
in
troll
this
or
difficult.
If
I
got
it
right,.
M
M
The
inputs
are
constantly
changing
temporal,
though
there's
not
a
temple
thing
on
there's
a
constantly
changing
and
just
like
my
speech
is
temporal.
That
mechanism-
and
this
is
temple
to
the
input
to
change
I
still
won't
have
a
stable
representation.
I
want
to
know
it's
a
cup,
it
does
thanks
to
magic.
Your
eyes
are
moving
you're
processing
temporal
streams
when
you
still
have
to
form
a
stable
representation
of
the
world,
because
the
road
that
is
not
changing.
M
A
L
We
just
learn
to
make
sure
that
everybody's
paying
attention
and
after
the
food
and
like
Jeff
was
mention
about
magic.
I
have
been
just
magic
since
I
was
a
kid
so
I
what
I
do
and
try
to
understand
how
brain
attention
is
being
deflated,
I'd
like
to
show
you
something
there
so
I
have
this
board
and
there's
a
one
black
dot
here
and
pay
close
attention,
because,
on
the
other
side,
I'll
show
you,
we
have
a
four
black
dots
and
I.
L
K
L
Scientist,
you
know
what's
going
to
have
attack,
what
do
you
know
what
I
know
you're
thinking
what's
happening
so
I'll?
Do
it
again
so
I
show
you
this
side
I'll
tell
you
what
I'm
doing
your
brain
is
spinning
in
the
gaps,
you're
you're,
looking
at
a
pattern
and
you're
filling
it
in
and
you
know
you're
thinking,
there's
a
dot
yet
right
and
of
course,
like
you
know,
if
I,
if
I
switch
this
around
like
here,
you're
thinking
you
know-
and
there
is
so
like
I
move
again
like
here
and
two,
there
are
only
two
dots.
A
A
Look
good
good
enough,
and
everyone
can
hear
me:
okay,
everyone.
My
name
is
Marcus
I
work
for
Numenta
and
I'm
here
to
present
some
of
our
latest
research
on
location.
This
is
unpublished,
research
that
it
builds
on
top
of
the
paper
that
Jeff
just
presented.
This
presentation
is
called
recognizing
locations
on
objects.
Today,
I'm
going
to
start
by
describing
the
problem
in
context.
E
A
E
A
A
That
I'm
going
to
talk
about
the
magical
cells
of
it
just
marketed
the
grid
cells
and
those
give
us
some
nice
clues
into
how
the
brain
processes,
locations
and
finally,
a
layout
of
a
solution
that
is
inspired
by
those
principles,
partly
the
problem
and
context,
detecting
locations
relative
to
objects.
Consider
the
inputs
to
your
brain.
A
Your
body
is
basically
this
array
of
sensors
and
your
retina
is
like
an
array
of
sensors
I've
shown
a
picture
of
a
coffee
cup
landing
on
that
already,
but
with
brains
is
really
important
to
focus
on
the
fact
that
sensors
move
and
movement
is
kind
of
very
native
to
the
system.
The
brain
is
very
good
at
handling
movement,
it's
very
good
at
handling
sensations
that
are
separated
by
movements.
The
easiest
example
is:
why
don't
you
look
at
something
your
eyes
are
always
moving
around
when
you're
looking
at
this
face.
Your
eyes
are
doing
this.
A
Just
talk
about
this
briefly,
that
three
to
five
times
per
second,
your
eyes
just
start
around
and
yet
you're
not
aware
of
this.
The
input
to
your
brain
is
in
both
movement
and
both
movement
and
good
mood,
and
you
just
sense
a
stable
image,
and
so
the
brain
is
very
good
at
making
sense
of
these
sensory
motor
senses.
A
Another
aspect
to
focus
on
on
the
inves
to
the
brain
is
that
this
array
of
sensors
is
made
of
both
independent
moving
sensors
and
so
and
so
coffee
cups,
don't
just
land
on
the
array
in
some
rigid
way
they
can.
They
can
land
on
your
array
of
sensors
in
many
different
ways,
and
what
these
two
aspects
have
in
common.
Is
that
they're
examples
of
what
the
brain
isn't?
Learning?
A
It's,
not
your
brain
can't
just
be
memorizing
sensory
motor
sequences,
because
when
you
look
at
something
your
eyes,
don't
just
darker
your
eyes,
don't
repeat
the
same
process
every
time
and
when
you've
got
an
object,
it's
going
to
land
on
the
features
are
going
to
arrange,
unto
your
sensory
array
in
a
novel
way,
almost
every
time,
even
if
the
object
is
familiar
and
so
somehow
the
bridge,
somehow
the
brain
bridges,
it's
yeah.
It
somehow
takes
in
sensory
motor
sequences
and
from
that
builds
a
model
of
objects,
and
our
latest
paper
which
Jeff
has
been
discussing.
A
It
worked
backwards
from
the
idea
of
a
model
of
objects,
and
it
gave
definition
to
what
is
a
model
of
objects
and,
as
just
pointed
out,
a
model
of
objects
is
well
here's,
a
coffee
cup
and
the
organism
that
has
these
moving
centers
as
this
organism
moves
its
sensors
over
the
object.
It
built
a
model
of
the
object
by
learning
features
at
locations
for
every
sensation.
A
When
is
learning
the
object?
It
learns
two
things:
it
learns
where
the
sensor
was
relative
to
the
object
and
what
it
says
and-
and
there
you
heard
me
say
the
words
relative
to
the
object,
its
allocentric.
That's
just
says
these
locations
are
relative
to
the
body.
It's
not,
whereas
my
hand,
relative
to
my
body
is
where
is
my
hand
relative
to
the
object.
It's
this
blue
location,
and
this
is
very
important
because
it
allows
you
to
reuse
this
model
and
many
other
contexts.
You
can
generalize
over
body
position.
A
If
you
sense
this
object
from
somewhere
else
or
if
you
sends
it
somewhere
else
relative
to
your
body,
you
can
continue
using
this
same
model
and
you
can
continue
building
on
the
same
model,
and
so
those
those
problems
which
are
laid
out
of
independent
moving
sensors
are
not
really
problems
anymore.
They
become
fairly
easy
to
solve.
When
you
add
this
notion
that
when
you
had
the
idea
that
every
sensor
has
a
location,
we
have
one.
N
Correlation,
so
when
you
make
the
first,
you
make
the
first
sense
since
you
touch
it
one
place,
then
the
next
place
you
tension
is
has
to
take
into
account
the
prior
place.
So
there's
a
temporal
thing
that
I
don't
hear
anybody
talking
about,
but
it
seems
to
me
like
it
could
be
there.
The
temporal
part
of
it.
It
says,
motors
sequences
yeah.
A
A
A
A
See
why
it's
called
that
as
the
rat
moves
around
this
box,
they
draw
a
little
tick
every
time
of
the
cell
spikes,
and
so,
whenever
this
works,
they
draw
a
tic
where
the
rat
was
standing
at
that
point,
and
you
will
see
that
it's
not
totally
random
with
a
higher
sample
size.
You
see
it
as
clustered
into
these
specific
locations
that
are
equally
spaced
and
they
form
a
sort
of
grid.
A
A
This
picture
is
just
showing
a
single
cell,
though
this
is
showing
the
firing
field
of
a
single
grid
cell.
If,
on
this
electrode,
you
record
the
nearby
cells
you'll
find
the
same,
hexagons
shifted
to
the
right
or
shifting
shifted
down,
and
if
you
record
a
large
number
of
cells
in
this
general
area,
you'll
record
enough
that
they
can
totally
cover
this
space.
They
can
totally
cover
it.
The
box
and
in
the
next
slide
I'll
show
sixteen
cells
tiling
and
covering
this
whole
box.
A
In
this
picture
there
are
sixteen
cells,
and
currently
cell
7
is
active.
Cell
7
is
fire.
If
the
rat
walks
to
the
right,
then,
when
I
should
what
this
way,
if
it
is
you're
right,
then
cell
eight
will
fire
if
it
wants
further
to
the
right
cell
five
home
fire,
and
so
you
get
this
mini-map
or
this
mini
tracker.
That
is
keeping
this
red
dot
just
moves
with
the
rats
body
and
if
it
walks
off
the
end
of
the
map,
it
just
revolves
around.
A
Interestingly,
in
a
room
before
that
arrives
back
into
a
familiar
room.
This
the
grid
cells,
will
fire
in
the
same
place
as
they
fire
forward,
but
if
you
put
them
in
a
different
room,
that
has
the
same
dimensions,
but
this
is
sufficiently
different.
This
cells
will
fire
in
a
different
place
in
this
experiment.
A
F
C
A
Into
modules
there
are
a
previously
we
were
looking
at
the
results
of
a
single
electrode.
You
report
under
themselves,
but
if
you
put
electrodes
into
various
parts
of
the
answer,
Ardex
you'll
find
you
cells.
They
have
very
similar
properties,
they're
still
print
cells,
but
the
space
that
they
cover.
The
title
pattern
covers
larger
spaces,
it'll
be
a
bigger,
hexagon
or
it'll,
be
a
rotated,
hexagon
and
what's
cool
about
this.
Is
that
this
combination?
A
That's
kind
of
magical
about
this
idea
is
that
is
that
this
system
only
knows
how
to
know
me
needs
to
know
how
to
move
the
red
dots
around
on
a
module
by
module
basis,
and
by
doing
that,
it
learns
how
to
move
around
this
enormous
space
of
possible
locations.
It
only
needs
to
know
how
to
do
this,
and
then
it
can
suddenly
move
around
many
many
unique
locations.
A
G
A
Is
my
moving
sensor
relative
to
an
object,
and
these
are
very
analogous
problems,
but
there
is
one
sort
of
big
difference:
I
mean
that's
that
you
have
multiple
sensors,
and
so
the
task
at
hand
is
to
take
through
the
principles
of
grid
cells,
but
also
come
from
the
fact
that
there
are
multiple
movement,
sensors
and
somehow
make
them
coordinate,
and
that
brings
us
to
the
solution
which
I'll
describe
this
recall.
Normalize
vote,
I'm
gonna
start
with
the
two
layer
model
that
Jeff
was
talking
about.
I
have
two
moving
sensors.
A
There's
this
two
layer
model
that
involves
it
in
choleric,
which
is
learning
features
at
locations,
learning,
feature
location
pairs
and
an
object
is
just
a
set
of
feature
location
pairs
in
the
columns
vote
on
what
object
they're
sensing,
but
here
I'm,
going
to
stop
taking
this
location,
this
allocentric
location
as
an
input
I'm
going
to
compute
it.
It's
the
location
of
the
feature
relative
to
the
object,
as
shown
in
the
diagram
and
the
only
other
input
I'm
going
to
give
the
system
is
the
location
of
the
feature
relative,
but
great.
A
The
first
way
is
by
sensory
cues
using
sensory
cues
to
recall
the
locations
so
may
have
this
driving
dimension
from
the
sensory
input
so
that
it
can
cause
the
system
to
recall
locations.
When
you
sense
a
feature,
your
brain
as
core
problem
will
say,
it
will
ask
yourself,
where
have
a
sense
this
feature
before,
and
it
will
activate
a
union
of
these
location
pesky
hours.
It
will
activate
all
of
the
possible
locations
that
this
could
that
this
could
indicate,
and
every
sensor
is
going
to
do
this.
A
Every
sensor
is
going
to
recall
all
of
the
locations
that
it
might
be
sensing,
and
now
the
remaining
task
is
a
fairly
simple
one
in
summertime.
What
needs
to
happen
now
is
you
have
and
I'll
show
a
simulation
of
this
soon
where
you'll
see.
There
are
many
active
cells
here,
and
there
are
many
active
cells
here
and
somehow
they
need
to
talk
to
each
other
and
choose
locations
that
are
coherent
with
each
other
and
it
can't
just
work
like
the
ones
up
at
the
object
later,
because
every
location
or
every
sensor
has
its
own
location.
A
G
A
To
have
them
normalize
this
location
they're,
going
to
take
it
and
using
using
the
location
using
the
blue
love
issue
which
they
they
don't
know,
but
they
know
that
list
of
possibilities.
They
take
the
list
of
possibilities
of
this
and
they
take
this
and
they
convert
it
into
a
list
of
possibilities
of
this
of
the
location
of
the
body
relative
to
the
object.
Every.
K
A
On
that
green
location-
and
this
is
very
analogous
to
the
second
way-
that
grid
cells
compute
locations,
the
second
way
they
compute
locations-
is
they
say
they
take
it
and
they
take
this
distance,
they
take
the
distance
of
the
movement
and
they
and
they
shifted.
They
transform
it
on
that.
This
is
going
to
take
this
location
and
shifted
by
this
distance
and
just.
A
Knowing
any
of
any
of
these
two
locations,
you
can
figure
out
the
third
one,
just
that
you
think
of
it
as
math.
It's
just
a
simple
vector
addition
of
our
subtraction,
and
this
is
the
entire
solution.
The
final
aspect
of
it
is
you
vote
on.
The
body's
location
is
relative
to
the
object.
Each
sensor
is
going
to
vote
on
this,
so
there
two
parts
each
sensor
recalls
the
list
of
possibilities
and
then
the
sensors,
both-
where
is
the
body
relative
to
the
object,
hey
Marcus,
yeah
I've,
a
couple
questions,
that's
a
good
question.
A
Topological
relationships
to
each
other
print
cells,
yes,
and
no
within
a
module.
No
that
hasn't
been
found,
but
modules
are
topologically
arranged,
so
cells
within
a
cells
near
each
other
tend
to
be
in
the
same
module
but
cells
near
each
other.
Aren't
necessarily
it's
when
I
was
showing
that,
like
the
red
bump
moves
with
the
rat,
the
red
bump
might
actually
be
hopping
around
various
cells
within
a
small
area
and
in
following
question
is
what
you're
talking
about
here?
A
The
solution
that
you
just
presented
is
this
implemented
or
is
it
just
theory
a
simplified,
a
double
I'll
show
you
it
in
30
seconds.
Think
there's
one
last
thing
I'll
say
is
when
you
you'll,
remember:
I
talked
about
the
modules
how
they
allow
you
to
have
unique
locations
for
every
environment.
Similarly,
if
you
have
unique
locations
for
every
object,
it
makes
the
system
very
powerful
and
I'll
show
this
in
this
demo
in
this
demo
and
is
a
simulation.
A
A
Two
things
I
want
to
point
out:
there
are
three
cortical
columns,
the
corresponds
to
each
Center
and
they
look
like
that
and
then
I
only
show
the
cells
that
are
active
in
each
layer.
Each
population
of
cells
I
show
a
little
black
triangles
to
denote.
Connect
itself
and
I
show
a
little
kitchen
next
to
each
population
of
cells,
which
is
decoded.
It's
a
it's
classified.
It's
saying
what
is
the
meaning
of
this
set
of
active
cells
in
the
main
thing.
I
want
to
show
you-
and
this
is
when
this
robot
reaches
out
in
senses
AEE.
A
The
first
thing
that
happens
is
it
activates
an
immense
number
of
cells,
it
activates
every
sensor,
activates
old,
the
possibilities.
It
senses
this
one
senses
name,
so
it
activates
all
of
the
possible
locations
all
of
the
places
where
it
has
ever
since
day
and
this
one
senses
B.
Similarly,
all
the
locations
that
sense
P,
so
it
starts
by
activating
a
union,
but
then
with
just
a
little
bit
of
time,
passing
it
narrows
to
just
the
locations
that
are
consistent
across
all
of
the
sensors.
But
so
the
mental
image
here
is
big.
A
Union
narrows
this
solution
very
much
takes
advantage
of
unions
and
to
describe
why
this
works
in
a
visual
way
that
works
really
well.
For
me,
I'm
gonna
bring
back
this
big
space
of
locations
that
comes
from
Krita
cell
modules.
When
ever
you
learn
an
object
whenever
this
organism
learns
an
object,
it
puts
it
somewhere
in
this
space.
It
says:
hey.
This
is
a
new
object.
I'm
gonna
just
choose
a
random
location
in
this
space
and
I'm
going
to
put
the
object
there
I'm
going
to
associate
this
set
of
locations
with
this
set
of
features.
A
A
It
uses
the
space
to
create
maps
of
objects
and
just
so
I'm
clear
these
are.
These
are
totally
different.
Maps
the
the
distance
between
them
isn't
beautiful
is
not
saying
that
this
object
is
this
far
from
that
object,
so
the
solution
there
is
women.
Senses
cos
object,
each
sensor,
votes
on
where
the
body
is
on
this
map,
and
so
this
top
sensor
says
I'm
sensing
a
that
means
the
body
could
be
here
here
here
here
here
or
here.
A
The
second
sensor
is
something
B,
so,
as
I
might
be
here
here
here
here
here
are
here
those
six
locations.
And
finally,
the
third
sensor
goes
through
the
same
process,
and
the
system
now
knows
that.
There's
only
one
possibility
that
could
have
caused
these
three
inputs
and
by
making
that
observation,
the
system
infers
the
location
of
the
object
all
of
the
sensors
and
it.
A
The
object,
and
so
to
go
back
to
this
diagram
I
showed
at
the
beginning,
which
was
stating
the
problem
as
sort
of
displayed
this,
as
as
sort
of
a
serial
process
of
determining
the
location
relative
to
the
body
relative
to
the
object
and
then
proceeded
onward.
But
it's
also
useful
to
look
at
this
as
a
holistic
process
and
the
holistic
system
does
this
I?
A
Basically,
this
is
a
neural
model
that
learns
objects
in
a
way
that,
when
it
senses
a
set
of
features,
it
asks
itself.
Do
you
might
remember
any
objects
that
contain
this
arrangement
of
features.
This,
quite
literally,
is
what
this
network
mean
does
and
it
uses
the
answer
to
this
question.
To
figure
out,
where
is
the
location
of
each
sensor.
A
And
and
in
the
process,
it
recognizes
the
object.
So,
to
summarise,
location
is
probably
one
of
the
brain's
main
primitives.
It's
one
of
the
main
things
that
the
brain
works
with
grid
cells
are
make
a
strong
case
for
this,
and
just
from
analyzing
the
task
of
building
a
sensory
motor
system,
it
really
solves
a
lot
of
problems
just.
A
Grid
cells
suggest
an
approach
to
location,
and
that
is
that
cortex
has
some
space
of
locations
that
I
can
process,
and
these
these
places
can
be
I
mean
a
grid
cell
module
is
a
space
of
locations,
it's
a
small
space
and
the
combination
of
them
can
create
a
big
space
of
locations
and
then
cortex
Maps
objects
into
this
space
when
it
lands
the
object.
And
finally,
with
this
general
framework,
cortex
can
recognize
these
locations
in
three
steps.
A
A
To
the
object
location
feature
down,
there,
they're
all
knees
over
here
and
you
see
I've
drawn
them
into
these
little
squares
and
those
squares
are
grid.
Cell
modules
are
they're,
inspired
by
grid
cell
modules.
Do
not
actually
internal
cortex
there,
but
they're
the
same
type
of
they
use
the
same
type
of
mechanism
and
in
this
visualization
to
show
this
off
when
I,
when
I,
when
I
hover
over
these
cells,
it
shows
the
firing
fields
of.
A
So
just
that
type
of
back
into
where
the
hexagons
come
now.
At
the
same
time,
these
aren't
Pentagon's.
These
are
squares
because
there's
just
not
a
theoretical
reason
in
our
heads
that
hexagons
would
be
necessary,
it's
more
just
what
biology
shows
and
the
reason
the
reason
this
theorized
for
that
is,
as
that
I
can
show
a
picture.
I'll
just
describe
the
natural
packing
density
for
of
circles.
If
you
just
take
a
bunch
of
circles
put
them
together,
they
naturally
stack
in
tax
accountants.
The
easiest
example
of
this
is
a
crate
of
oranges.
H
A
M
M
M
But
I
should
point
out
that
it
matches
quite
a
good
point
here.
The
n-terminal
consciousness,
an
older
structure.
Do
you
think
about
it
as
soon
as
the
animals
started
moving
if
they
ever
wanted
to
get
back
to
where
they
started
from,
they
had
to
have
a
solution
to
the
problem.
The
location
and
the
animals
can
get
back
to
places
falling
paths
they
never
taken
before
they
can
longer
all
through
the
woods
and
know
how
to
exactly
back
to
where
they
came
from.
This
is
the
problem.
M
A
Yes,
they
have
tried
to
catch
rephrase
that
are
you
asking
whether
grid
cells
responses
are
their
best
to
be
firing
pills,
yeah,
they've,
tried
and
I
have
not
found
any
studies
of
3d
grid
cells.
When
interesting
thing
is
some
people
think
that
they
have,
because
there
was
a
study
where
they
found
written
cells
in
bats
and
people
get
excited
when
they
hear
that,
but
in
the
experiment
the
bats
were
actually
walking.
A
Another
possible
theory
that
some
people
believe
is
that
cortex
has
learned
to
co-opt
a2d
too
eager
in
cells
to
process
a
3d
world.
There's
a
recent
paper
on
the
head,
Direction
cells
that
that
basically
had
Direction
your
sense
of
direction
which
I
haven't
gotten
into,
but
they
work
very
closely
with
good
installs.
M
How
is
just
so
exciting
to
talk
about
this?
You
can
play
around
this
yourself
that
you
can
either
his
own
time
that
you
get
the
mill
of
money
and
in
its
dark
and
walk
around
and
ask
yourself
do
I
know
where
I
am
I
have
a
sense
where
I
am
can
I
I
feel
when
I
walk
down
my
hallway
a
little
bit
your
ability
to
do
that,
tells
you
everything
and
your
ability
to
know
where
you
are.
H
A
Say
this
I
would
say
this
model
doesn't
try
this
out
that
yeah
I
was
trying
to
show
just
the
minimum
pieces
to
illustrate
this
new
principle
and
and
in
some
sort
of
scale.
Invariance
would
require
some
new
connections
here
or
some
some
new
mechanism
and
I
really
intentionally
kept
this
as
small
as
there
to
really
focus
on
a
single
principle.
D
A
E
So
you've
explained
how
green
cells
are
used
to
map
the
location
of
our
our
sensors,
where
you
say
basically:
okay,
the
animal
thinks
the
sensors
here
in
Delta,
2
and
you've
shown
us
in
your
visualization
how,
in
a
3d
model,
you
say
like
the
magic
system
you
were
discussing
in
folks
where
the
body
is
relative.
To
that.
Yes,
is
the
location
signal
in
terms
of
like
the
actual
neurons
firing
in
your
bottle.
Isn't
isn't
the
same
like
that?
You
have
a
similar
representation
for
the
location
of
the
body.
A
I
well
in
here,
one
of
them
the
location
of
feature
relative
to
the
object,
looks
like
this
and
the
location
of
the
body
relative
to
the
object
looks
like
this
they're.
Both
they
look
the
same,
but
that
comes
from
shifting
each
of
these.
It
comes
from
shifting
each
of
these
sets
of
active
cells
by
the
number
encoded
by
this
active
cell
to
get
these
actor
cells,
but
then
they
look
very
similar,
they're
speaking
the
same
language.
In
some
sense,
let's.
J
J
Have
some
kind
of
you
put
on
these
VR
goggles?
Then
you
use
your
hands
to
go
up
and
down
or
something
like
you're
flying
but
wondering
what
what
would
you
do
in
the
VR
type
situation?
You
know
who
has
the
if
your
grid
cells
are
kind
of
able
to
tell
that
you're,
either
higher
or
lower?
Well,
one
answer
to
that
is
that.
A
A
A
G
C
M
I
just
wanna
remind
ourselves
we're
not
studying
for
it's
those
pursuing
the
questions
and
we
were
not
staying
on
the
cortex
works,
and
so
these
are
interesting
questions
for
good
cell,
but
we're
not
there
to
Aaronson
good
cells
in
the
cortex.
It's
clearer,
certainly
from
touch
from
the
tactile
sense
that
we
move
our
fingers
in
three
dimensions.
So
this
is
nice
to
work
in
in
these
somatosensory
cortex,
but
we
have
to
be
careful
which
are
McMinn
cells
but
Marcus,
say
grid
cell
modules,
but
we
mean
that,
like
they
were
collected
so
money,
but.
B
M
A
L
D
A
N
A
A
A
Don't
have
a
good
answer
to
that.
What
I
can
talk
about
is
recognizing
a
location
and
amazed,
but
solving
amazes
just
kind
of
beyond
here
this
isn't
generating
the
behavior
of
any
kind.
This
is
more
just
sensing.
What's
up,
there
is
localizing
itself.
This
could
localize
itself
in
a
maze
that
it's
been
in
for
by
using
a
sequence
of
inputs
and
its
own
movement
to
determine
where
it
is
this.
It
could
do
that,
but
this
isn't
this
can't
solve
mazes.
A
It
just
would
require
you
to
add
more
pieces.
We
have
to
solve
more
of
the
part
of
Sakhalin,
keep
in
mind.
We
don't
have
a
wet
lab
or
anything
at
events
up.
We
are
sort
of
relying
on
the
neuroscience
research
community.
To
do
continuing
research
will
take
you
the
opportunity
to
prod
them
in
two
directions.
We
would
love
to
see
them
go,
but
for
the
most
part,
these
researchers
are
reading.
A
Stuff
that
they
can
find
and
trying
to
figure
out
the
intelligence
works
in
the
cortex,
so
I'm
going
to
cut
it
off
here.
You
guys
can
hang
out
and
be
cool
until
10:00.
Then
I'll,
kick
you
all
out
of
ten,
and
so
then
thank
you
guys.
Everybody
for
coming.
This
has
been
a
really
nice
beat
up,
see
a
lot
of
new
faces.
Thanks
to
our
speakers,
Jason
toy
and
Jeff
Hawkins
and
Marcus,
Lewis
and
I
hope
you
guys
have
a
nice
night
and
drive
safe
and
I'll
be
around.