►
From YouTube: Thousand Brains Hangout with Jeff & Subutai
Description
About our latest paper "A Framework for Intelligence and Cortical Function Based on Grid Cells in the Neocortex": https://numenta.com/neuroscience-research/research-publications/papers/a-framework-for-intelligence-and-cortical-function-based-on-grid-cells-in-the-neocortex/
Text Q&A on HTM Forum at https://discourse.numenta.org/t/thousand-brains-hangout-with-jeff-subutai/5077
The MIT talk Jeff refers to: https://numenta.com/company/events/2017/12/15/mit-center-for-brains/
A
Okay,
thanks
for
joining
us,
this
is
a
thousand
brains
hangout
with
Jeff
Hawkins
in
Suva,
ty
Ahmad,
we're
all
from
the
Mendte
I'm
Matt
Taylor,
and
we're
going
to
talk
about
our
most
recent
paper
framework
for
intelligence
and
do
some
QA.
So
if
you're
just
watching
this,
there's
a
there's,
a
discussion
on
our
forum,
that's
linked
in
the
show
description
down
there
and
there's
also
a
link
to
the
paper
down
there.
If
you
want
to
read
it
so
that
should
provide
all
the
context
that
we
need
for
this
discussion.
A
A
C
A
D
F
Yes,
can
you
hear
me?
Yes,
yes,
okay!
Thank
you
very
much
so
so.
My
plan
now
is
to
be
actually
working
on
my
master's
thesis
on
an
idea
with
HTM
and
basically
the
core
idea
is
to
try
and
do
anomaly
detection,
but
in
the
cross-correlation
space
of
multiple
metrics
and
there
I
think
that
it
could
be
very
useful
analogy
with
the
object.
G
F
E
D
D
You
want
a
channel
I
think
we've
explored
with
a
few
different
ideas.
There
I,
don't
think.
We've
really
settled
on
anything
I
think
the
old
theory
has
been
moving
quite
fast,
and
so
we
haven't
really
focused
on
the
continuous
learning.
That's
thanks
so
much
but
there's
some
general
ideas
and
even
on
the
forum,
there
are
some
ideas
about.
D
Somehow
that
there's
a
new
object,
the
same
way
that
in
the
temporal
memory,
a
lot
of
bursting
kind
of
triggers
learning
up
new
sequences,
you
could
potentially
do
something
like
that
with
with
the
columns
paper
as
well,
but
I
wouldn't
say
we
really
explored
it
or
simulated
this
yeah
too
much
but
surprises
that
the
signal
always
for
for
learning
new
things.
There's.
H
Another
idea
we've
explored,
to
which
we
haven't
really
taken
very
far-
that
that
any
individual
network
can
be
doing
inference
being
trying
to
recommend
this
existing
objects
and
learning
simultaneously
sort
of
on
alternate
phases
of
a
cycle
and
there's
a
lot
of
evidence
for
this
system
from
some
parts
of
the
brain,
where
you
literally
every
phase
of
a
cycle
you're
you,
the
the
neurons
switch
between
assuming
that
you're
learning
something
new
and
then
assuming
that
you're
trying
to
infer
something
that
was
already
learned.
That
is
crazy.
That
sounds.
H
H
That
may
not
be
happy
for
you,
but
because
we
feel
that
there's
a
solution
there,
we
don't
know
it's
where
the
subitize
suggestion
when
I
just
mentioned,
but
we're
trying
to
get
to
sort
of
the
basic
mechanisms
down.
First
before
we
decide
exactly.
Okay,
is
this
continuous
learning
happening
exactly
how.
H
B
D
A
E
F
Think
that
you
alluded
a
little
bit
into
this
with
with
framework
paper
with
a
thousand
brains
paper
when
used
won't,
you
talk
about
the
ranking
of
grid
cells.
Maybe
we
could
rephrase
the
relevant
question
as
what
exactly
triggers
the
reentering
of
grid
cells?
What
makes
and
a
new
environment
actually
new.
H
A
H
Working
on
it
suppose
that
there's
a
problem,
one
of
the
things-
that's
there's,
there's
literature
about
this
in
my
grid
cells
in
the
entorhinal
cortex
related
to
its
you
know
the
good
that
grid
cells
are
driven
by
several
different
factors.
One
factor
is,
of
course,
they're
they're
updated
by
motor
command.
So
that's
how
they're
known
to
do
it,
but
they're
also
they're
anchored.
It's
believed
their
anger
by
sensory
input,
and
so
one
of
the
theories
is
the
ones
that
we
sort
of
subscribe
to
is
that
play
cells.
H
Once
you've
learned
once
you've
learned
a
connection
between
play
cells
and
grid
cells
that
the
play
cells
are
constantly
angry.
Indeed,
bit
cells,
it's
it's
constantly
happening,
you'll
get
a
new
play
cells
and
it's
constantly
trying
to
react
or
the
grid
cells,
but
then
there's
a
question
of
well
what,
if
it
can't
and
then
how
does
it
decide
to
pick
a
random
anchor?
H
A
A
F
That
well
I
slide
polo
up
in
the
same
area.
Is
that
in
turn
a
this
type
of
paper,
you
mentioned
that
during
learning
the
location
layer
doesn't
update
in
response
to
sensory
input.
So
I
was
simply
wondering
there.
If
this
separation
of
learning
and
inference
thing
is
that
all
neurologically
plausible,
or
simply
an
artifact
of
the
model,
what.
H
H
A
H
That
that
idea,
that
is
occurring
on
different
oscillatory
cycles,
comes
from
empirical
observation
that
that
there's
evidence
empirical
evidence
that
diet
is
actually
what's
going
on.
We
didn't
make
that
up.
I
wouldn't
have
never
thought
of
that.
So
there
is
the
evidence
that
cells
go
through
these
two
different
phases
and
different
actual
activation
cycles
on
two
different,
like
in
the
in
tirana,
cortisone
the
different
phases
of
the
theta
cycle.
Now
we
don't
know
if
that's
happening
in
the
new,
your
cortex,
but
that's
that
there's
strong
evidence
for
that.
D
F
A
A
G
A
H
A
H
And
occupies
the
majority
of
v1
just
it
was
one
of
the
largest
regions
in
your
cortex
in
the
human.
So
it's
not
like
it's
a
small
thing.
It's
a
there's,
a
huge
amount
of
processing
going
on
with
the
phobia
and
I,
don't
see,
there's
no
inherent
reason
to
to
change
the
thousand
brains,
theory
or
the
frameworks
theory
at
all,
but
they
that
the
framework
does
not
rely
on
that.
It
doesn't
rely
on
a
phobia.
There's
other
animals
that
don't
have
a
phobia
rats
have
vision
without
a
foe
fuel
they
have
opposing
eyes
largely
posting
eyes.
H
The
theory
doesn't
really
care
about
those
that
those
differences-
all
it
says
is
you
have
a
sensor
arrey
and
the
sensory
is,
is
observing
different
parts
of
an
object
than
a
phobia
wouldn't
be
doing
that
I'm.
Looking
at
the
camera
in
front
of
me
right
now,
it's
actually
occupying
a
small
part
of
my
visual
field,
but
it's
I
see
it's
details,
and
so
the
different
parts
of
my
trophy
will
be
attending
the
different
parts
of
that
camera
and
they
would,
as
a
group,
the
different
columns
would
be
voting
on
what
that
object
is.
H
If,
if
this
object,
the
camera
occupied,
like
you
know
huge
part
of
my
visual
field,
that
might
be
difficult
to
see
what
it
is,
but
the
fact
that
it's
small
and
they're,
not
quite
a
small
part
of
my
visual
field,
and
it
allows
me
to
do
object
reference
recognition
at
a
long
distance.
So
I
can
see
things
that
are
actually
you
know
just
certain
the
details
and
things
that
are
very
far
away.
I,
don't.
H
H
H
If
you're,
not
a
Braille
reader,
you
try
to
discern
those
patterns.
You
know
it's
really
hard.
Yeah
I
know,
but
that's
like
saying:
if
I'm,
not
a
Russian,
speaker
and
I,
hear
Russian
I
don't
understand,
it
would
be
like
if
I
never
really
learned
to
see
and
also
make
my
vision,
I
really
can't
see
either
right.
H
So
you
have
to
train
a
train
to
recognize
these
patterns
by
my
point
is
that
their
finger
is
like
a
phobia
because
it's
an
area
of
high
acuity,
maybe
you
could
discern
very
small
differences
and
the
area
represented
by
your
finger
in
the
cortex
is
relative
or
very
large,
compared
to
say
the
back
your
hand
or
something
like
that.
It.
H
It
depends
what
you
mean
by
see.
The
vision
comes
through
your
eyes,
and
those
are
the
patterns
are
going
to
the
brain.
I
think
the
general
consensus
of
brain
researchers
is
that
your
perception
of
the
world
is
a
is.
There
is
really
the
model
that
you
have
into
the
world,
so
you've
built
an
internal
model,
the
world.
That's
what
the
frameworks
paper
is
all
about.
How
do
you
build
models
of
the
world
and
when
you
perceive
what
you
sense
is
really
based
on
that
model?
It's,
and
that
doesn't
mean
it's
wrong.
You
know
it's
fake.
H
A
G
Okay,
there's
one
I
posted
on
the
on
the
forum,
and
maybe
it's
been
answered
on
all
the
parts
I'm
not
always
up
to
date.
One
of
the
things
I,
don't
really
understand
is
that
so
when
you,
when
you
look
at
an
object,
you
obviously
have
to
make
a
model
and
when
you
touch
an
object,
you
have
to
make
a
model.
G
But
it
seems
to
me,
like
you,
have
a
tremendous
amount
of
hardware
in
every
cortical
column
and
it's
very
useful
for
looking
and
perhaps
even
hearing
things
orientate
yourself
based
on
what
you
hear,
but
of
for
a
lot
of
other
things.
I
don't
really
understand
how
this
is
useful
or
well
I.
Guess
you
understand
what
I
mean,
but
I
don't
say
it's
not
right!
I!
Don't
say
it's
wrong,
but
doesn't
make
much
sense
to
me
and
I
don't
get
it
yeah.
H
H
One
of
the
facts
is:
is
everywhere
you
look
in
your
cortex.
The
architecture
is
extremely
similar.
There
are
differences,
but
the
similarities
are
remarkable
and
the
differences
are
more
tweaks,
apparently
than
dance,
fundamental
differences.
It's
and
many
parts
in
your
cortex.
If
you
look
at
them
in
great
detail,
you
cannot
discern
that
what
they're
doing
or
how
they're
different
than
another
part,
and
so
the
you
know
the
one.
The
basic
themes
of
neuroscience
is
that
there
is
this
common
circuitry
that
does
everything
and
and
that's
true
of
language
areas,
really
language
areas.
H
They
look
remarkably
similar
as
the
touches
and
visionaries
and
so
on.
It's
it's
incredible,
and,
and
so
this
for
a
long
time,
it's
believed
that
there's
some
underlying
computation
that's
done
everywhere.
That
somehow
applies
to
all
the
different
things
in
your
cortex
dots.
There
is
there's
little
evidence
suggest
that's
not
true,
and
so
now
we've
attacked
this
from
two
different
parts.
H
That's
a
very
powerful
idea,
and
it's
and
even
beyond
what
we've
written
so
far,
it
looks
like
you
can
explain
the
vast
majority
of
the
circuitry
in
the
neocortex,
so
we
didn't
really
get
into
that
in
the
frameworks
paper,
but
our
next
paper
is
kind
of
get
into
that.
So
now
we
say
to
ourselves:
okay!
Well,
if
that's
true,
how
would
it
apply
the
language
and
how
would
it
apply
to
other
things?
At
the
same
time,
the
people
have
been,
and
we
we
referenced
some
of
these
in
our
in
the
front
paper.
H
There
are
people,
who've
been
looking
at
fMRI
data,
which
suggests
that
there
are
grid
cells
in
the
New
York
cortex,
while
people
do
quote
high
level
tasks,
thinking
about
things
and
this,
and
so
they
not
like
how
do
I
sense,
what
something
is
receive.
Something
is,
but
when
I'm
thinking
about
birds
or
I'm
thinking
about
sort
of
mental
cognitive
tasks,
they
find
evidence
that
grid
cells
underlying
that.
So
there's
some
empirical.
H
But
it's
saying
yes,
some
high-level
thought
processes
are
also
somehow
built
on
grid
cells
and
that
we
mentally
map
out
things
in
the
world
in
a
space.
So
they
would.
The
classic
example,
I
think
was
that
the
dollar
paper,
where
they
talk,
people
think
about
birds
in
the
different
attributes
of
birch
and
when
you're
thinking
about
the
different
attributes
of
birds,
they
have
evidence
that
you're
assigning
them
to
location
spaces
or
locations
in
a
space.
H
You're
not
aware
you're
doing
this,
but
that's
how
you
categorize
data
about
something
yeah,
the
birds
are
taller
or
smaller
different
attributes.
You
put
them
on
these
dimensional
accesses
and
it
looks
like
grid
cells
are
modifying
them.
So
there's
a
lot
of
evidence
we're
just
so
triangulating
on
this
and
then
you
can
say
well
how
does
that
really
apply
to
something
like
language?
Well,
we
don't
really
know,
but
the
evidence
which
is
just
it
does,
and
one
way
to
you
might
think
about.
You
can
think
about
words
as
objects.
H
We
said
in
the
Friendly's
paper,
those
reference
frames
don't
have
to
correspond
the
physical
things
in
the
world.
You
know
they
just
like
the
birds.
You
know
we
put
these
birds
on
these
sort
of
reference
frames.
It's
don't
really
correspond
to
locations
in
the
world,
but
we
we
built
a
reference.
We
built
the
brain
seems
to
build
this
reference
frame
for
how
to
place
knowledge
and
we
move
through
it.
H
So
we
took
an
attempt
at
describing
this
the
best
we
could
in
a
discussion
section
and
there
in
the
frameworks
paper
and
about
and
we're
not
the
only
people
starting
to
talk
about
this.
So
it
is
an
interesting
question.
Your
your
your
questions
are
correct.
No
one
really
knows
exactly
how
this
works
yet,
but
the
evidence
is
very
strong
that
everything
we
do
is
built
on
reference
frames
and
if
you
haven't
read
it
go
back,
you
know.
The
best
I
can
explain
is
what
we
wrote
in
the
frameworks
paper
about
it.
H
Well,
we
gave
these
references.
We
talked
about
this,
but
we
also
admit
we
don't
really
understand
it,
but
it
seems
like
that's
going
to
be
part
of
the
answer
and
it
doesn't
seem
to
be
something
else.
There's
no
other
magic
things
going
on
elsewhere
in
the
cortex
would
say:
oh
yeah
language
works
differently.
It
don't
seem
to
be
that
way.
H
A
J
J
H
H
Yeah
well
everything
we
do
assumes
that
cortical
columns
are
very
similar.
We
don't
make
any
assumptions
about.
That's
that's
standard
neuroscience
dogma,
so
we
don't.
We
don't
have
any
evidence
that
the
columns
between
are
different,
but
there
is
a
very
interesting
question
as
where
are
the
grid
cells
and
exactly
what
is
their
structure
in
a
cortical
column
and
I
could
talk
about
this
for
hours
because
we're
spending
a
lot
of
time
on
I'm
spending
a
lot
of
time
on
it,
so
I
can
give
it
I,
don't
know
how
much
we
want
to
go
into
this.
H
Okay,
so
let
me
let
me
give
you
the
sort
of
a
big
picture
of
this.
Okay
grid
cells,
of
course,
were
discovered
not
in
the
neocortex,
but
in
the
hip
and
the
hippocampal
complex
or
in
the
internal
cortex
and
grid
cells
in
the
in
tirana,
cortex
represents,
you
know,
they've
been
studied,
mostly
in
rats
running
around,
in
mazes
or
rooms,
and
and
in
that
situation
the
grid
cells
represent
a
2d
space,
a
two-dimensional
space
where
the
rat
is
on.
H
You
know,
rats
don't
fly
to
the
space,
they
did
they
kind
of
stay
on
the
ground
and
they
move
around
in
2d,
and
so
everything
that's
been
written
about.
Grid
cells
is
about
2d
representations
of
space
and
that
system
was
evolved
to
represent
a
location
of
an
animal
in
a
two-dimensional
environment.
Now
we
have
hypothesized
that
the
same
basic
mechanisms
exist
in
the
neocortex,
but
the
neural
cortex
isn't
necessarily
dealing
with
2d
spaces.
H
We
move
in
3d
spaces
objects
have
three
dimensions,
they
might
even
we
might
even
be
modeling
higher
dimensional
spaces,
but
at
a
minimum
we
know
they
were
modeling
three-dimensional
spaces.
So
how
does
a
2d
dimensional
grid
cell
represent
3d
spaces?
We
have
a
paper,
that's
being
written
by
a
couple
of
our
researchers
right
now,
which
is
very
close
to
being
submitted
about
this
very
topic
about
how
you
could
represent
higher
dimensional
spaces,
using
two-dimensional
bits
or
modules,
and
this
is
this
is
your
question
in
a
moment.
H
So
what
this
tells
us
is
that
you
need
to
have
to
represent
a
three-dimensional
space.
You
need
to
least
have
multiple
2d
grid
sub
modules
that
some
sense
slice
up
the
3d
space
differently,
a
2d
you
can
think
about
2d
grid.
So
module
is
representing
a
projection
of
3d
space
under
2d
to
achieve
good
cell
space.
H
Until
you
need
a
you
need,
more
than
one
slice
through
3-dimensional
space
to
represent
3-dimensional
space,
you
can
represent
it
with
multiple
2d
modules
that
are
basically
intersecting
the
3d
space
at
different
projections,
and
so
that's
one
cool
thing.
We
know
that
it's
definitely
gonna
be
different
than
you
know,
cortex
and
then
an
Tirana
cortex.
It's
it's!
It's
I'm,
currently
working
on
the
idea
that
it's
possible
that,
in
the
neocortex
grid
cell
modules,
are
one
dimensional.
H
And
there's
some
evidence
to
suggest
this
might
be
true,
and
she
said
what
does
I
mean?
Basically,
if
I
want
to
represent
a
3d
space
or
a
2d
space,
I
have
to
have
a
whole
bunch
of
1d
modules
that
are
interested
that
are
basically
projections
of
the
3d
space
onto
a
1d
line,
and
hopefully
that's
that
is.
H
You
can
imagine
that
in
your
head,
what
that
means
so
meant
as
much
of
the
movement
through
3d
space
would
not
would
be
reflected
on
all
these
1d
modules,
because
they
don't
all
move
depending
on
the
projection,
so
I'm
moving
perpendicular
to
the
1d
module.
It's
not
going
to
reflect
that
change,
but
some
other
1d
modules
book.
So
this
is
a
long
question
to
stay.
H
In
a
cortical
column,
we
believe
there
has
to
be
multiple
grid
cell
modules,
so
in
one
square
millimeter
for
example,
there
have
to
be
deduced
logically,
have
to
be
more
than
one
grid.
Cell
module
has
to
be
multiple
ones,
especially
if
they're
1d,
but
even
if
the
2d
they
have
to
be
multiple
ones.
They
have
to
represent
different
projections
in
3d
space
and
and
then
we
know
something
about
how
these
physically
look
in
an
to
Rhino
cortex.
There's
a
nice
paper
came
out
recently
by
David
tank
at
Princeton,
I
think.
H
Each
one
could
be
a
unique
grid
cell
module
and
that
together
they
represent
that
entire
space
and
they
might
be
1d
grid
cell
modules.
This
this
is
this
there's
some
evidence
for
this.
It's
got
a
lot
of
its,
but
it's
very
speculative
still,
but
it's
elegant
in
some
ways.
If
it's
not
in
the
many
columns,
it
still
has
to
be
divided
up
somehow
and
many
a
cortical
column
has
to
have
multiple
original
modules
that
are
acting
independently
slicing
up
space
in
different
ways.
D
Cells
update
their
representation
based
on
motor
commands,
so
wherever
the
grid
cells
are,
they
should
be
receiving
some
sort
of
a
motor,
coffee
or
motor
command
coming
in
and
they're,
only
a
couple
of
layers
in
the
neocortex
where
that
happens,
and
the
other
anatomical
constraint
is
that
we
think
there's
this
sort
of
back-and-forth
between
the
location,
representation
and
the
sensory
or
the
place
cell
and
log
representation.
So
there
has
to
be
some
strong
kind
of
the
current
connectivity
between
to
the
sensory
layers
and
the
grid
cell
layer.
D
H
They're
they're,
you
know
we
all
we
have
to
say.
Should
it
could
I
actually
feel
really
confident
about
that
I
think
they're,
really
in
the
layer,
six
and
we've
know
which
cell
tops
they
are,
but
it
is
obviously
theory
so,
but
you
know
we
can
still
put
different
levels
of
confidence
on
these
things,
so
I'm
I'm
very
confident
that
they're
those
you're
sick
cells,
but
it
could
be
wrong
but
I'm,
very
confident
then,
but
other
things.
Well,
you
know
this
is
the
thing
I
just
mentioned
about
many
columns.
A
A
A
H
H
We've
been
focusing
purely
on
the
neuroscience
side
lately
and
I
am
continuing
to
focus
of
the
neuroscience
night,
so
I
can
talk
about
what
we're
doing
on
the
neuroscience
side
subitize.
Now
it's
all
about
doing
this
together.
There's
no
acrimony
here
at
I
and
other
researchers
are
starting
to
focus
on
how
to
apply
some
of
what
we've
learned
to
machine
learning
techniques,
sort
of
going
back
in
that
direction.
So
Billy
wants
to
talk
about
that
more
yeah.
D
D
But
you
know,
of
course,
I'm
continuing
to
be
extremely
interested
in
the
neuroscience.
But
you
know,
Numenta
has
always
had
this
kind
of
two-pronged
mission
of
understanding
the
neuroscience
side
of
it
and
then
trying
to
see
if
the
principles
that
we
learned
from
the
neuroscience
can
be
applied
to
practical
problems
and
to
and
to
machine
intelligence.
And
we've
done.
D
So
the
kind
of
the
research
direction
that
I'm
pursuing
a
little
bit
now
is
to
take
some
of
the
concepts
that
we've
worked
on
from
the
neuroscience
and
apply
them
more
directly
to
to
neuroscience
and
I.
Think
in
in
that
research,
which
is
still
very
speculative
and
exploratory.
At
this
point,
I
think
there
are
basically
two
components
to
it.
If
I
look
at
everything,
we've
done,
I
think
there's
like
two
fundamental
pieces.
D
One
is
kind
of
a
representational
component
and
a
lot
of
you
on
the
forum
know
about
how
much
we
rely
on
sparse,
distributed
representations
and
the
properties
of
STRs
and
deep
learning
systems.
Don't
really
embody
STRs
today,
they're
primarily
dense
representations.
So
the
question
is,
you
know:
can
we
embody
SDRs
into
deep
learning
systems
and
machine
learning
systems
and
take
advantage
of
some
of
their
properties,
and
the
second
part
of
it
is
just
looking
at
the
cortical
column
as
a
structure.
D
If
you
look
at
a
deep
learning
system
or
neural
network
today,
it's
extremely
simplistic
feed
forward
structure,
whereas
the
cortical
column
structure
is
a
lot
more
complex.
So
can
we
take
that
structure
and,
along
with
SDRs,
improve
machine
learning
and
deep
learning
to
embody
everything
that's
in
this
kind
of
common
algorithm
with
a
common
cortical
micro
circuits?
So
that's
a
very
quick
description
of
the
kind
of
the
research
I'm
just
really
starting
on.
H
He's
got
he's
getting
really
interesting
results
already
so
I'm
going
to
focus
on.
If
you
don't
mind,
I
can
just
I'm
going
to
what
my
work
for
this
year
is
still
on
the
biology
side
and
I'm
trying
to
fill
in
all
these
missing
pieces
of
a
cortical
column
and
specifically
the
role
of
orientation
which
is
like
head
Direction
cells
and
equivalent
to
play
cells
which
and
so
I'm
working
on
the
idea
that
I
actually
mentioned
a
year
ago
and
I
talked
together
at
MIT,
but
I'm
back
to
it
with
a
vengeance.
H
Now
is
that
in
a
cortical
column,
there's
actually
two
different
sensory
motor
inference
mechanisms
being
done.
One
is
movement
through
space
is
what
what
the
frameworks
paper
talks
about
a
lot
and
that's
how
you
have
grid
cells
and
moving
through
space,
and
the
other
is
a
sensory
motor
mechanism
which
has
to
do
with
orientation
or
or
changing
orientation
to
an
environment
and
so
and
that's
a
produces
with
the
equivalent
of
play
cell,
so
I
think
the
cortex,
but
to
fill
out
the
framework
and
many
of
the
details
we
can
understand.
H
A
cortical
column
is
doing
two
types
of
inference.
At
the
same
time,
one
is
angular
a
movement
which
is
your
orientation
to
the
world
and
that's
figuring
out
like
like
play
cells.
What
what?
Where
am
I
based
on
my
sensory
input?
And
then
there
is
the
movement
through
space,
which
is
a
more
of
a
linear,
sensory,
motor
and
friends
and
I.
Believe
you
can
map
these
two
inference
mechanisms
precisely
on
two
different
cortical
layers
and
adding
orientation,
and
it
really
fills
out
the
complier,
the
complete
picture
of
what
a
cortical
column
does.
H
That's
a
that's.
A
first
of
all
play
cells
are
in
the
hippocampus
right.
That's
where
the
term
comes
from.
These
are
cells
in
the
hippocampus.
We
think
there
are
equivalent
cells
in
the
in
the
neocortex,
although
we
have
not
really
talked
about
them.
As
such,
we
didn't
mention
that
in
the
in
the
framers
paper.
So
how
are
they
selected?
It's
more.
It's
it's
there's!
Well,
here's
one
way
to
think
about
it
there,
but,
first
of
all
what
do
place
cells
do.
Play
cells
represent
some
sensory
input
that
encode
your
location.
H
So
it's
like
when
an
animal
is
in
a
particular
location
based
on
the
century
it
puts
around
the
animal
these
place.
Cells
represent
that,
but
they
don't
they.
They
represent
independent
of
the
orientation
of
the
animal.
So
it's
not
like
I
see
something
in
front
of
me.
It's
like
there's
something
relative
to
the
room,
so
the
place,
those
don't
change
when
the
animal
changes
its
orientation
to
the
room.
H
H
But
it
could
be
whiskers,
it
could
be
vision,
it
could
be
hearing,
it
doesn't
really
matter
it's
long
as
I
sense,
something
that
I
can
then
turn
it
into
a
representation
of
the
location
in
the
room.
Based
on
that
one
thing,
so
there's
it's!
It's
not
really
critical
to
what
senses.
You
sense
it's
more
critical
to.
How
do
you
do
the
sensorimotor
inference
and
that's
a
long
topic
I?
Don't
think
the
actual
features
are
really
that
important.
It
could
work
of
any
kind
of
sensory
modality.
A
Okay,
I
want
to
promise.
We
were
to
answer
the
forum
questions,
so
let
me
go
through
these
and
because
we
only
got
about
15
more
minutes,
because
that
might
generate
more
topics
and
I'll
get
to
the
rest
of
the
chat
stuff.
So
so
someone
who's
asking
about.
Is
there
any
relationship
between
grid
cells
and
the
the
orientation
stripes
or
bands
that
we've
observed
in
human
Beisel
papers
yeah?
A
H
I
remember:
early
I
was
saying
that
working
on
the
hypothesis,
the
grid
cells,
are
there's
a
little
modular
mini
column.
You
know
each
mini
column
in
that
in
the
hoobmobile
model
of
v1
has
a
specific
orientation,
so
the
next
one
the
lines
are
for
nitration,
the
next
mini
column,
idealized
a
different.
H
Stimulus
at
federal
citations,
but
also,
very
importantly,
those
cells.
Many
of
those
cells
respond
to
motion,
so
they
actually
not
just
orientation
but
they're.
Actually,
that
line
is
moving
this
way,
or
this
way
that's
what
they
prefer.
I
won't
have
time
to
explain
all
of
this,
but
that
is
exactly
the
signal
you
would
need
to
to
update
and
create
a
one
dimensional.
It's
our
module
that
movement
command.
It
would
tell
you
which
way
the
bump
should
move
on
a
mundum
oechler
its
own
module.
It's
already.
H
They
are
visual
features,
but
they
actually,
the
movement
defines
those
the
metrics.
We
need
to
create,
with
some
modules,
an
orientation
module
head,
Direction
cells
approval.
So
that's
an
interesting
idea
that
we
I
don't
know
if
anyone
else
has
ever
thought
of
before
I
said
early,
it's
very
speculative
but
I'm,
working
on
that.
Okay.
A
D
I
think
we
were
talking
about
this
a
little
bit
earlier.
There's
many
different
aspects
to
invariance,
but
I
would
say
you
know
this
whole
idea
of
having
a
location
signal
within
a
cortical
column
came
from
the
I
came
in
part
from
thinking
about
invariance
and
and
what
and
the
idea
of
reference
spaces.
So
if
you
think
about
you
know
what
invariance
is
you
want
to
have
some
sort
of
a
signal?
That's
stable.
D
While
you
are
sensing
different
aspects
of
the
same
thing,
that's
sort
of
one
way
you
can
think
about
invariance
and
in
order
to
do
that
for
an
object
if,
as
I'm
sensing,
an
object,
I
have
to
have
a
representation
of
an
object.
That's
in
the
reference
frame
of
the
object
itself.
That
way,
my
the
output
of
my
system
of
our
system
can
be
invariant,
regardless
of
the
pose
of
this
object
relevant.
You
know
relative
to
me
so
grid
cells
and
the
location
signal
by
encoding
relative
positions
of
features
within
the
reference
frame
of
the
object.
D
H
The
probably
the
biggest
one
right
I
mean
essentially
you're,
going
from
some
presentation
on
a
two-dimensional
sensory
array
or
that
your
fingers,
your
eyes
or
something
like
that
and
you're.
Turning
into
an
internal
representation
which
is
completely
independent
of
your
pose
relative
to
that
object,
it's
a
3d
model
of
the
object
and
doesn't
matter.
You
know
what
you
have
a
3d
model
is
the
object.
A
3d
model
is
invariant
to
any
other
position
and
orientation
to
anything
else.
I
think
one
other
aspect.
D
D
A
H
A
H
H
Reference
frames
and
different
modality,
and
so
yeah
and
and
as
point
I
soon
as
I
pointed
out
the
key
thing
about
invariance
is
you
have
a
stable
representation
while
inputs
are
changing?
That
is,
in
some
sense
the
definition
of
invariance,
and
we
propose
there's
this
very
specific
mechanism
for
that
which,
I
think
is
pretty
good.
This
is
the
temple
puller,
which
is
in
the
college
paper
in
the
columns
plus
paper
and
we're
I'm,
very
confident
that
that's
basically
happening.
Okay,.
H
H
H
You
go
back
to
the
neuron
paper,
the
sequence
memory
paper.
We
we
laid
out
a
very
detailed
model
of
the
neuron
and
how
the
dendrites
work
and
what
they're
computing
and
part
of
that
was
that
they
have
to
detect
these
coincident
patterns
on
a
dendritic
branch,
and
the
biology
tells
us
that
those
synapses
have
to
be
active
within
a
few
milliseconds
of
each
other.
So
there
needs
to
be
you'd
like
to
have
some
sort
of
synchronizing
ability
to
get
the
action
potentials
arrive.
At
the
same
time,
it's
supposed
to
scatter
it
over
time
and.
H
H
The
the
basic
belief
is
that
there
are
cycles
in
the
brain
and
these
cycles
will
the
cells
will
tend
to
fire
on
the
peaks
of
these
cycles
and
not
following
the
troughs
of
these
cycles
and
therefore
they,
if
they're,
going
to
admit
a
spike,
they
tend
to
do
at
the
same
time
and
then,
but
this
question
an
ism:
is
there
traveling
long
distances
and
there's
there's
delays
and
the
delays
would
be
different,
so
now
they're
not
going
to
arrive
at
the
same
time?
It's
a
good
question.
I,
don't
have
any
answer
to
it.
H
Not
hard
to
imagine
how
this
is
so
much,
it's
not
known
about
some
of
this
that
maybe
the
dendrites
are
as
critical
as
we
people
think
they
are
maybe
there's
local
dynamics
which
make
these
things
happen.
There
are
some
many
synapses
have
this.
The
tropic
response
means
that
they
lead
to
a
long
term
to
polarization
that
would
bridge
these
time
gaps.
So
there's
lots
of
possibilities,
but
it's
not
an
area
that
we
focused
on
hey
the.
A
A
A
H
In
fact,
some
of
the
research
which
Marcus
and
Scott
used
to
come
up
with
the
displacements
was
literally
and
we
referenced
this
in
the
paper
literally
came
about
from
people
trying
to
figure
out
how
we
navigate
how
you
know
how
to
get
from
point
A
to
point
B
in
the
same
space.
So
now
we
have
this
mechanisms
which
we
were
trying
to
figure
out.
The
object,
compositionality,
but
clearly
could
also
do
navigation
within
the
same
space.
So
so
now
we
have
these
two
dual
ideas
here.
H
One
is
like
between-
and
this
is
very
clearly
written
in
them
in
the
frameworks
paper-
that
this
concept
of
space
missiles
could
do.
Both
of
these
things
could
say:
hey
here's,
how
I
get
from
point
A
to
point
B
in
one
object,
space
and
here's
how
I
relate
to
different
points
into
a
difference
of
reference
frames.
Now,
as
we
go
forward
in
time,
it's
that
one
of
those
still
works
really
well,
that's
the
how
to
get
from
point
A
to
point
B
how
to
generate
behavior.
H
The
compositionality
one
is
starting
to
have
some
problems
or
we're
struggling
with
trying
to
get
the
details
working
so
there's
issues
of
orientation
and
scale
that
we
haven't
quite
figured
out
how
to
get
working
in
the
displacement
as
an
object.
Compositionality
problem
so
I'm
now
far
more
comfortable
that
the
displacement
cells
exist
and
they're
doing
movement,
I'm,
calm,
I'm,
confused
now
exactly
how
they're
doing
the
composition,
Elleni,
and
maybe
we
might
move
to
slightly
different
mechanism
for
that.
H
Maybe
we'll
separate
them
out,
there's
two
different
things,
so
we
wrote
them
as
this
displacement
cells
could
do
both.
That
may
still
be
true,
maybe
not,
and
but
I
do
know
they
could
do
movement.
So
this
is
an
area
we're
trying
to
it's
very
difficult
to
think
about,
but
we're
trying
to
really
get
to
the
core,
how
we
do
object,
composition
exactly.
How
does
it
deal
with
these
problems
of
orientation
and
meaning
like
magic?
We
use
the
coffee
cup
example.
We
said,
oh
there's
a
logo
on
the
coffee
cup.
H
Well,
we
didn't
really
address
what
happens
of
the
logos
or,
and
it
changed
it
in
orientations
that
coffee
cup
we
didn't
address
that
we
didn't
address
how
the
couple
of
logo
wraps
around
in
three
dimensions
on
the
coffee
cup.
We
didn't
really
address
with
the
issue
of
how
the
scale
of
the
local
can
change
on
the
coffee
cup.
So
there's
a
lot
of
things
with
displacement
cells
that
Billy
didn't
address
those
issues
we
pointed
those
out
in
the
paper.
We
made
it
clear
like
it.
H
A
A
Right
this
displacement
cells.
You
know
you
think
of
him
as
a
movement
between
two
points,
not
the
two
points,
but
the
movement
between
the
two
points:
right,
yeah
and
then
so
Mark
Brown.
How
does
the
local
grid,
in
a
mini
column
square
with
the
known
repeating
grid
patterns
across
the
internal
cortex
yeah.
H
H
H
Yeah,
it
could
be
the
exactly
the
same
thing
in
neocortex,
so
you
might
have
a
two-dimensional
grid,
so
much
Ivan
we
haven't
eliminated.
That
possibility.
That's
the
first-order
assumption,
in
which
case
you'd
have
cells
that
repeat
the.
If
I,
if
I
you
know
as
I
move
over
objects,
they
would
repeat
and
in
the
same
sort
of,
but
now
in
a
you
know,
two-dimensional
projection
of
a
3-dimensional
space
so
which
is
a
little
bit
odd
to
think
about,
but
imagine
if
I
could
just
move
some
through
some
space
continuously
relative
to
some
object.
H
D
Right
I
think
one
thing
that
came
out
of
the
work
that
Mirko
and
Markus
are
working
on
is
that
the
dimensionality
of
the
grid
cell
modules
is
kind
of
independent
of
the
dimensionality
of
the
location
space.
So
you
can
take
any
dimensional
location
space
and
represented
with
almost
any
dimensional
grid
cell
modules,
as
long
as
you
have
enough
of
them,
and
you
get
these
random
projections
that
do
it.
So
you
kind
of
divorce,
the
two
of
them
to.
H
Same
thing
happens
of
orientation
by
the
way
we
think
there's
an
orientation
of
your
finger
to
the
cop.
Just
like
the
rat
has
into
the
room
you
can
think
of
rot
in
the
room.
The
orientation
is
a
one-dimensional
vector
at
the
head,
Direction
cells-
they
just
it's
like
you
know,
there's
just
one:
it's
an
angular
representing
angular
position
and
it's
one-dimensional
and
if
you
go
all
the
way
around
then
you're
back
to
the
same
cells
again.
So
it's
a
repeating
pattern,
but
it's
a
closed
space
because
you're
doing
I
angle
movement.
H
But
how
would
I
represent
my
orientation
of
my
finger
to
this
cup?
That's
not
a
one,
dimensional
orientation,
there's,
there's
all
kinds
of
movements
here:
I
can
do
while
on
the
same
location
of
the
cup,
but
it's
different
orientations,
and
so
even
there,
if
I
represented
orientation
with
1d
orientation
modules,
I
would
need
multiple
of
them
to
represent
the
orientation
of
my
finger
to
this
cup.
So
it's
the
same
basic
problem,
and
so
this
is
a
fact.
A
H
H
H
There
are
cells
and
basically
layer,
2
3,
and
there
are
cells
in
a
certain
cells
in
layer,
5
and
those
are
the
only
that's
a
subsidiary
of
five,
and
only
only
those
are
the
only
two
cell
types
that
project
long
distances
and
the
current
theory,
which
is
goes
a
little
bit
beyond
what
was
in
the
frameworks
paper,
is
that
in
this
we
we
in
the
frame
bush,
but
we
talked
about
one
of
those
being
the
we
actually
in
the
columns
paper.
We
talked
about
it.
Initially,
we
modeled
it
in
the
columns
paper.
H
No
substrate
is
long-range
axons
in
layer,
2,
3
2,
other
cells
in
layer,
2
3
anywhere
in
the
cortex.
That
might
be
modeling
the
same
object
and
you
all
you
have
to
do
is
take
a
population
of
cells
and
another
population
through
sparse
populations
and
you'd,
say:
okay,
we're
both
learning
the
coffee
cup.
Now,
let's
form
these
long-range
connections,
basically
just
associate
this
pattern
with
that
pattern
and
they
can
do
that
from
an
hundreds
of
different
patterns.
And
now,
when
you
see
this
pattern,
it's
going
to
invoke
that
pattern
over
here.