►
Description
Subutai Ahmad goes over voting in the Thousand Brains Theory.
In the first of two research meetings, he lays the groundwork for understanding how columns vote in the theory by unpacking the ideas in our "Columns" paper. First, he presents the hypothesis of the paper on how cortical columns learn predictive models of sensorimotor sequences. Then, he explains the mechanisms behind a single cortical column and how it learns complete objects by sensing different locations and integrating inputs over time. In the next research meeting, he will review voting across multiple columns.
Columns paper "A Theory of How Columns in the Neocortex Enable Learning the Structure of the World": https://www.frontiersin.org/articles/10.3389/fncir.2017.00081/full
Other paper mentioned: “The columnar organization of the neocortex” - https://academic.oup.com/brain/article-pdf/120/4/701/17863573/1200701.pdf
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a framework called the Thousand Brains Theory of Intelligence that will be fundamental to advancing the state of artificial intelligence and machine learning. By applying this theory to existing deep learning systems, we are addressing today’s bottlenecks while enabling tomorrow’s applications.
Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://numenta.com/news-digest/
Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter
Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta
Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/
Our Website:
https://numenta.com/
A
B
A
Dendrites
and
explained
you
know
how
that
stuff
works,
and
here
I'm
gonna
focus
on
voting
based
on
you
know,
requests,
and
the
reason
I
did
dendrites
in
part
was
because
I
think
some
of
that
material
is
important
to
understand
exactly
how
voting
works
or
at
least
our
proposal
of
voting.
But,
as
I
started
putting
this
together,
I
realized
there's
so
much
to
this
just
to
even
understand
voting.
I
hope
people
will
get
it.
I'm
not.
I
don't
know,
but
there's
a
lot
to
understand.
A
The
latter
oh
yeah,
okay,
to
convey
voting.
I
really
have
to
sort
of
explain
you
have
to
kind
of
understand,
cortical
columns
and
how
we
approach
this
whole
idea
of
sensory
motor
models
and
then
to
understand
the
details.
You
have
to
understand
the
anatomy
and
how
neurons
work
and
all
of
this
stuff-
it's
just
again.
B
B
A
So
so
what
I
did
here
is
go
back
to
the
columns
paper
and
we
had
we've
had
a
bunch
of
slides
in
the
past
and
posters
and
things
like
that,
and
so
I
slapped
together
a
bunch
of
material
from
past
presentations.
So
I'm
hoping
there's
a
consistent
flow
here,
but
there
might
not
be
but
really
to
understand
voting.
You
know
you
need
to
understand
this
paper
and
then
to
really
understand.
A
B
A
A
You
probably
did,
and
it's
slightly
different
from
what
you
talk
about
normally
in
your
talks,
but
here
the
idea
is
suppose
you
reach
your
hand
into
a
black
box
right,
so
you
can't
see.
What's
in
there,
all
you
can
do
is
feel
what's
in
there
and
you're,
trying
to
determine
what
object
is
in
the
box,
for
example,
a
coffee
cup
and
there's
two
different
scenarios.
A
A
There
could
be
100
possible
objects,
let's
say,
but
you
get
some
sensation
and
you
can
you
can
narrow
down
what
it
is
you
can
you
know
it's
not
a
tennis
ball,
for
example,
you
know,
but
you
know
it's
probably
something
spooned
and
curvy
with
just
one
touch,
but
you
don't
really
know
exactly
what
it
is
and
to
recognize
what
it
is.
You
would
move
your
finger
around
get
a
number
of
different
sensations
and,
as
you
move
things
you
know,
the
idea
of
what
that
object
is.
A
Will
coalesce
in
your
head
and
pretty
soon
you'll
be
able
to
recognize
uniquely
what
it
is,
and
so
does
that
scenario
make
sense
and
there
it's
important
to
note
that
you're
getting
one
sensation
at
a
time,
you're
integrating
the
evidence
you
had
from
the
past
into
your
next
set
of
hypotheses,
and
the
motion
is
actually
critical
by
moving
things
around.
You
understand
where
the
relative
position
of
different
point
sensations
are
and
they
so
to
distinguish
a
copy
cut
from
some
other
ceramic
object.
C
A
Your
fingertip
would
you'd
be
clueless
yeah.
Imagine
I
created
some
weird
object.
That's
made
out
of
components
of
a
coffee
cup,
but
just
completely
bizarre
locations.
You
would
know
that
as
soon
as
you
start
moving,
but
if
you
just
looked
at
the
sensation,
you
wouldn't
be
able
to
tell
so
that's
the
main,
that's
sort
of
one
scenario.
A
Should
mention
from
a
cortical
column
perspective
the
sensation
you're
getting
from?
Let's
say
a
fingertip
is
going
into
one
cortical
column
and
it's
integrated
over
time.
The
second
scenario
is
the
multiple
cortical
column
scenario.
Suppose
you
put
your
entire
hand
in
and
you
grasp
the
object
now
you're
recognizing
the
object
with
multiple
fingers.
It
could
be
two
fingers,
it
could
be
all
five
fingers,
it
could
be
two
hands,
but
the
more
fingers
you
use
the
quicker
you're
going
to
recognize
it
in.
B
A
Cases
just
one
grasp
is
enough.
You
can
instantly
tell
what's
happening,
but
remember
that
each
cortical
column
is
just
getting
one
of
those
sensations.
So
no
single
particle
column
has
all
the
information,
so
you
have
to
integrate
the
information
or
accumulate
the
integration
information
across
multiple
cortical
columns
in
order
to
make
that
inference,
and
that's
primarily
what
the
voting
algorithm
is
doing
even
within
the
single
cortical
column,
there's
still
a
voting
thing
going
on
within
a
single
column,
so
we'll
go
into
that.
So
actually
the
same
algorithm
is
used
same
voting.
A
C
A
C
C
C
D
A
Touch
as
the
example,
but
the
model
is
not
specific
to
touch.
This
is
something
conversations
with
other
people.
People
often
get
confused
about
this.
It
applies
to
all
inputs
to
all
columns
everywhere
in
the
neocortex,
and
so
what
the
actual
input
is
can
vary
dramatically
from,
depending
on
where
you
are
in
the
neocortex.
But
is
this
it's
the
same
micro.
B
C
C
Hearing
is
a
little
bit
less
obvious,
but
but
what's
clear
is
from
the
anatomy
of
the
brain,
it's
the
same
right
and
but
it's
you
know
it's
a
topology
of
your
of
the
representation
of
your
your
skin
and
your
brains.
Right,
it's
very
clear.
Oh
each
part
of
my
skin
has
its
own
little
column.
Arranged
in
this
little
monk
of
the
body
has
its
own
columns
looking
a
little
part
of
the
visual
scene.
C
Just
like
you
know,
this
is
like
vision,
but
the
anatomy
suggests
it
is
and
we
haven't
spent
a
lot
of
time
trying
to
analyze
vision,
but
there's
a
lot,
that's
known
about
that
anatomy,
but
it's
clearly
operating
on
the
same
principles
but
in
a
different
sort
of
dimensional
space.
So
we
don't
really
understand.
A
It,
but
it's
kind
of
the
same,
the
core
of
this
is,
is
being
able
to
make
predictions
as
you
do
motor
actions
being
able
to
predict
the
sensory
input.
That's
coming
in.
So
that's
at
the
core
of
a
lot
of
this,
and
that
has
been
shown
even
in
the
auditory
cortex
that
david
schneider
and
others
have
done
experiments
where
they
can
record
from
auditory
cortex
and
show
that
there
are
neurons
that
are.
C
You
want
to
think
about
like
the
sensor.
You
can
think.
Okay,
the
vision
is
an
array
of
you
know:
sensory
sensors
and
your
skin
is
an
array
of
sensors.
The
array
of
sensors
in
the
ear,
of
course,
is
the
polkadot,
which
is
a
you
know,
a
strip
of
material
where
you
have
frequencies
represent
high
frequencies,
one
at
low
frequencies
at
the
other
end.
So
that's
that
is
the
topology
of
that
sensory
organ.
C
It's
like
it's,
it's
the
same
basic
process
but
less
intuitive
as
to
how
you
how
it
performs,
but
it
is
definitely
a
sensory
motor
system
and
not
only
that
when
you
hear
you're,
often
turning
your
head
and
cocking
your
ears
and
you
don't
even
know
you're
doing
this.
But
every
time
you
move
your
head
when
it's
a
little
bit,
you're,
changing
the
sounds
and
and
and
that's
how
you
differentiate
things
when
you're
having
trouble
understanding
something
you
turn
your
head.
You
turn
it
like
this.
C
A
C
Yeah
definitely
like
the
core
of
the
whole
thing:
those
aren't
a
little.
You
know
they
changed
completely
dramatically,
so
yeah.
All
of
that,
and,
of
course
everywhere
you
look
at
the
cortex,
the
part
of
the
vital
cortex
projects
back
to
the
part
of
their
brain
and
moves
the
eyes
and,
of
course
the
somatosensory
cortex
is
right
next
to
the
part
that
moves
your
limbs
and
auditory
cortex
projects
back
to,
like
the
part
of
your
your
spinal
cord,
which
turns
your
head,
so
there's
clearly,
essentially
what
it
likes
to
do.
D
C
B
C
I
think
marcus
did
a
lot
of
presentation
on
this
one
there's
different
senses
that
are
active
when
you're
moving
your
finger
along
something
versus
you're,
touching
something
there's
multiple
ways
to
think
about
that.
But
one
is
that
even
even
that
flow
is
a
way
of
telling
your
brain,
how
the
how
the
sensor
is
moving
when,
when
there's
a
there's
a
signal
coming
back,
says:
oh
yeah,
this
thing's
moving
across
your
skin
there's
a
certain
velocity,
that's
not
to
recognize
something
moving
in
the
world.
That's
to
tell
you
how
your
finger
is
moving.
E
A
Column,
getting
sensation,
not
multiple,
so
and
it
integrates
that
information
over
time
so
as
you're
reaching
into
the
black
box,
and
you
send
something
and
you
send
something
else
and
you
send
something
else.
That's
three
sensations
coming
in
temporal
sequence,
into
a
single
cortical
column
and
it
has
to
integrate
all
of
those
into
this.
C
C
But
we
don't
want
to
think
about
that.
We
want.
We
want
to
understand
how
a
single
column
would
work
right
and,
and
it
should
work
independently
and
it'd,
be
just
like
touching
a
teeny
little
part
of
your
finger.
You
know,
just
like
you
know,
on
the
monkey's
hand,
six
you
know
six
pounds
wide
or
something
like
that's
less
acuity
than
on
your
fingertip.
A
C
A
A
C
First,
we
keep
going
that
one
there
you
can
see.
I
I
don't
remember
exactly
this
figure,
but
imagine
those
numbers
are
representing
different
columns,
yeah
and
and
to
be
honest,
a
lot
of
these
columns
overlap
somewhere
have
a
bigger
area
in
a
smaller
area.
This
is
true
in
vision.
Two,
you
might
have
a
in
b
one.
You
have
a
column,
that's
getting
input
from
a
very
small
part
of
the
retina
and
then
b2
is
getting
in
from
a
larger
part
of
the
retina.
C
So,
to
be
honest,
when
you
just
activate
one
spot,
you
might
be
getting
multiple
columns
of
different
sizes,
so
you
can
look
there
he's
got
like
you
know,
one
two,
three,
nine
eight
or
so
all
around
that
tip
of
that
finger
or
the
three
four
five,
six
seven
so
they're
actually
multiple
columns
that
are
overlapping
at
this
point.
So
technically,
yes,
they
would
be
voting,
but
we
don't
want
to
think
about
it
that
way.
Initially,
we
don't
understand
what
a
single
column
does
and
then
how
they
vote.
No.
E
C
C
Do
recordings
move
a
little
bit
more
and
so
on
and
see
so
a
single
probe
over
time
could
would
transcend
six
columns
and
that
what
they
noticed
was
all
the
cells
in
one
millimeter
of
area
all
respond
to
one
patch
of
skin,
and
then
they
abruptly
changed
to
in
column
two
to
all
the
columns
in
that
next
millimeter
representing
the
next
patch
over
and
so
on.
This.
A
Left
figure
might
be
also
useful
here,
the
similar
you
know
the
probe
is
coming
in
this
way,
and
so
now
he's
recording
from
you
can
see
my
screen,
multiple
cells
within
a
single
cortical
column.
So
this
is
like
all
one
cortical
column,
so
multiple
cells
within
a
single
cortical
column
and
as
he's
recording
from
there,
you
see
that
it's
corresponding
to
the
same
patch
of
skin.
A
C
A
This
is
part
of
the
trick
of
doing
these
experiments
there's
a
whole
part
in
how
to
do
this.
Probably,
and
you
can,
if
you're,
not
careful,
you
can
get
all
these
antibodies
and
stuff
that
form
around
it
and
right.
You
know,
because
the
scar
tissue
is
trying
to
heal
itself,
all
sorts
of
stuff
happens,
so
they
have
to
do
this
properly.
C
C
B
Another
question
to
the
the
slides
you
have
so
there
you
say
those
two
options
are
basically
the
same
voting
mechanism,
but
the
first
one
kind
of
seems
a
lot
more
active
to
me.
So
you
can
actively
decide
where
to
move
next
to
resolve
the
most
uncertainty,
for
example,
when
you
need
to
know
when
do
I
have
enough
information
to
take
the
vote
or
to
be
certain
that
there's
this
object
and
with
the
second
one
you
kind
of
have
one
sensation
from
multiple
places,
but
it's
not
like
actively
sampling
the
environment.
A
C
I
made
earlier,
I
don't
think
we
should
call
the
first
situation
voting.
It's
it's.
I
think
you
know
sympathizing
the
same
neural
someone.
The
first
century
is
not
voting.
It's
sensory
motor
integration,
the
cortex,
is
deciding
where
to
move
the
finger.
That's
part
of
its
algorithm.
The
second
situation
is
really
I
at
any
point
in
time.
Each
column
knows
something
they're
trying
to
they're
trying
to
resolve
their
differences.
C
C
Sort
of
like
the
fundamental
premise
here
is
that
quarter
columns
are
identical
or
very,
very
close
to
the
functionality.
Each
one
is
a
complete
system.
Therefore,
we
need
to
understand
how
a
single
one
works
or
multiple
ones
work,
there's
no
magic
that
occurs
when
you
go
from
one
to
two.
You
know
it's
not
like
you
have
each
one
has
to
stand
on
its
own.
A
C
C
A
B
A
A
We
had
kind
of
described
a
little
bit
last
time
and
if
you
look
at
the
cortical
anatomy,
you
know
we're
saying
the
sensory
feature
comes
into,
let's
say
layer
force
and
that's
what
a
sensor
feature
but
there's
another
big
input
coming
in.
That's
almost
half
of
the
input
into
layer,
four
coming
in
from
lower
layers,
and
so
you
know
when
we.
D
A
Discussing
this
this
was
you
know
this
was
an
unexplained
anatomical
fact
at
that
point
at
that
point
in
time,
right
what
is
going
on
in
here
and
so
what
jeff
sort
of
hypothesized
is
that
suppose
we
had
a
motor
related
contact
signal,
let's
say
allocentric
location.
We
talked
a
lot
about
this
since
then,
but
at
that
time
it
was
just
some
motor
related
contact
signal.
Suppose
it
comes
in
here
and
acts
the
same
way
as
these
lateral
signals
did
in
the
temporal
memory.
C
It's
a
little
confusing
to
call
it
motor
related
here,
because
it's
it's
really
a
location
cycle
and
the
location
signal
is
updated
by
motor
information,
so
just
think
of
it
as
a
point
in
some
space-
and
this
is
how
we
describe
this
in
the
neuron
paper-
we
now
just
be
clear.
We
now
have
a
more
refined
idea
of
how
this
works.
Marcus
has
been
talking
about
that
so,
but
so
this
is
just
describers
in
that
paper,
but
it's
still,
the
ideas
are
still
correct.
C
Not
a
motor
command.
At
one
point,
we
tried
that
it
doesn't
work.
That
was
our
little
robot
arm.
We
didn't
remember,
and
the
big
insight
was
a
motor
command
could
work,
but
it
it
has
a
combinatorial
explosion
problem.
The
big
insight
was
that
it's
a
location
command,
it's
imposed.
It's
where
the
sensor
is
relative
to
the
object
is,
is
the
key
item,
and
initially
no
one
thought
that
you
could
have
a
representation
of
where
the
sensor
is
relative
to
the
object
in
the
object
reference
frame.
C
A
C
A
C
You
know
where
you
are
close
your
eyes
and
start
walking
very
quickly.
You
have
a
lot
of
uncertainty
really
quickly
and
you
start
getting
nervous
like.
Oh,
I
might
be
walking
into
a
wall
here.
So
so
it
does
do
this.
This
path,
integration,
right,
your
your
body,
takes
his
movement
command
and
updates
your
sense
of
location.
Oh,
I
now
moved
away
from
the
bed,
I'm
out
towards
the
door
of
the
room,
but
very
quickly.
The
noise
gets
really
large
and
you
start
being
uncertain
where
you
are.
A
But
if
you
make,
you
know,
touch
reach
out
and
touch
this
one
thing
you'll
often
instantly
know
where
you
are
now
and
that's
because
you
have
a
sense
of
rough
sense
of
where
you
are
and
by
reaching
out
in
you
know,
you
make
a
prediction
of
what
you
might
feel.
Maybe
you
make
many
predictions
if
you
feel
the
corner
of
the
bed
instantly,
you
know
where
you
are
in
the
bedroom.
Another.
C
Simple
example
of
this:
it's
really
fascinating,
you're
walking
in
the
dark,
and
all
you
see
now-
is
the
slightest
hint
of
a
corner
of
the
wall,
and
if
I
put
this
into
a
vision
system,
I'd
have
no
idea
what
it
is,
but
because
I
know
I
have
a
model
of
the
room
that
slight
hint
of
the
corner
of
the
wall
does
exactly
what
super
tight
just
says
tell
them.
I.
C
Now
and
so
in
the
smallest
little
reflections
in
your
room
or
little
glooms,
or
something
will
anchor
you
again
so
this
is
why
you
know
computer
vision
and
very
like
like
we
have
this
amazing
ability.
We
think
we're
seeing
things
we're
not
really
seeing
things
we're
just
re-anchoring
our
own
location.
At
times,.
A
And
maybe
one
subtle
point
there
is
like,
if
you
felt
the
edge
of
your
bed,
you
might
not
know
exactly
where
you
are,
because
that's
still
ambiguous,
you
kind
of
roughly
know
where
you
are,
but
you
don't
know
if
you
feel
the
point
of
your
bed,
that's
more
unique
and
you'll
know
instantly
where
you
are,
and
this
algorithm
will
work
the
same
way
so
which
feature
you
feel
will
determine.
You
know
how
quickly
you
arrive
at
the
location,
yeah.
A
As
long
as
you
know
where
you
are
so
just
to
recap
in
the
paper
itself,
we
don't
detail
the
location
signal
in
this
paper
and
we've
talked
about
it
a
lot
since
then,
and
we
can
have
a
separate
meeting
on
that.
If
you
want,
but
in
this
in
this
paper,
you
can
think
of
it
just
an
sdr
that
represents
the
allocentric
location
of
the
sensor
of
the
object
and
just
to
be
clear
what
you
know
when
we
say
alice,
centric
location,
the
location
is
in
the
reference
frame
of
the
object.
So
what?
A
A
F
F
So
column
versus
mini
column,
a
cortical
column
is
the
set
of
cells
receiving
the
same
sensory
input
spanning
all
layers.
I
know
a
mini
column
is
a
subset
of
a
particle
column,
but
you
said
that
a
mini
column
is
also
all
these
cells
are
also
responsive
to
the
same
sensory
input.
So
is
the
sensory
mini
column,
a
subset
of
the
sensory
input
to
the.
A
A
A
A
C
C
He
said
he
actually
proposed
that
this
has
been
a
perennial
confusion
for
people.
He
said
the
algorithmic
unit
of
the
cortex
is
the
mini
column,
and
these
are
very
skinny,
really
skinny
little
things,
and
he
says
several
hundred
millicolumns
together
form
a
cortical
column
which
is
really
defined
by
the
input
bishop.
C
C
A
A
Think
avi
had
a
similar
confusion
yesterday
when
I
was
talking
to
him-
and
this
is
the
kind
of
thing
that
I
just
don't
even
think
about
it's
so
natural
to
me-
yeah-
that
I
didn't
even
realize
it
was
like
distinction
and
confusion.
It's
you
can
see
why
you
know
every
like
every
phrase
here
has
like
an
hour.
C
D
E
A
C
A
Try
to
lower
the
yeah
and
the
other
thing
that
we
could
do
at
some
point
I
mean
in
the
paper.
I
wrote
down
exactly
the
equations
that
correspond
to
everything
and
we
could
just
review
equations
that
there
there's
no
ambiguity,
but
it's
you
know
it's
actually
all
notation,
there's
nothing
difficult
about
the
equations.
It's
just
all
notations
right.
C
People
who
study
the
new
york
times
they're
used
to
these
kind
of
diagrams,
and
so
you
have
to
think
about
your.
I
don't
think
that
confused
neuroscientists
to
study
the
neocortex,
but
I
can
see
why
it'd
be
confusing
to
other
people
yeah.
Well,
let's
do
it
just
a
quick
distinction
here:
does
the
mini
column
spin.
C
Diagram
here
is
representing
one
layer:
okay
right
so
within
one
layer,
you'll
find
like
in
layer,
four
you'll
see
a
set
of
cells,
maybe
like
10
15
or
something
like
that.
That
all
responds
to
the
same
input
and
in
other
ways
they
do
different
things.
The
actual
physical
mini
column
goes
across
all
layers,
okay,
but
so
it's
like
you
can
you
can
imagine
many
columns
got
120
cells
in
it.
Roughly
that's
about
what
it
does.
It's
really
skinny.
C
A
C
A
C
B
B
C
Clearly,
if,
if
you're,
actually
sliding
between
locations
on
an
object,
you're
gradually
moving
across
the
surface,
the
representation
will
also
gradually
move
and
drive
the
change.
We
didn't
try
to
capture
that.
We
didn't
know
how
any
of
this
worked
at
the
time.
So
we
did
a
very
crude
approximation.
A
C
A
F
A
C
There
are
a
lot
of
new
ideas
in
this
paper
and
and
so
lots
of
them
and
in
the
previous
papers
too,
we
didn't
really
delve
into
how
this
motor
related
context
get
created.
Or
what
does
it
look
like?
We
didn't,
we
didn't
know
at
the
time
it
was
just
a
placeholder,
so
we
picked
random
sdrs,
but
this
whole
mechanism,
how
a
layer
cells
explaining
what
why
why
the
cells
in
the
mini
column,
I
have
the
same
feedback
response
properties.
Why
they're
connected
like
this?
Why
there's
why?
C
The
green
hour
only
connects
to
the
proximal
synapses
and
the
blue
across
connection
to
the
distal
synapses.
Now
that
I
had
an
explanation,
there
really
were
no
concretex
or
theories
about
how
any
of
this
stuff
worked.
So
there's
a
lot
of
new
ideas
here
and
we
that's.
We
focused
on
that
we're
focusing
on
those
mechanisms
in
this
box.
B
Yeah
yeah,
it
makes
totally
sense,
but
I
was
just
wondering
like
in
principle:
your
model
should
be
able
to
generalize
if,
if
the
inputs
are
in
like
a
meaningful
space,
so
if
I
have
a
location
here
and
then
here
and
then
I
get
a
new
one
in
the
middle
it
just
because
of
the
input
having
this
meaningful
relationship,
the
column
model
should
be
able
to
do
something.
I
think.
C
A
A
A
So
if
I'm
moving
around
in,
if
I
stick
my
hand
in
the
black
box
and
I'm
sensing,
a
coffee
cup,
the
object
layer
is
going
to
have
a
single
sdr
that
represents
that
coffee
cup
it's
going
to
be
stable,
whereas
the
bottom
layer
is
going
to
change.
As
I
move
my
finger
around,
I
don't
know
if
that
was
clear
anymore,.
A
B
A
C
C
C
We
no
longer
believe
that
we're
learning
features
at
locations.
We
derive
the
models
actually
as
features
relative
to
each
other,
and
we
build
that
model
by
knowing
where
my
finger
is
relative
to
those
features.
So
we
do
need
to
know
where
the
finger
is
relative
to
the
feature.
It
loves
it
to
the
object,
assuming
we
need
to
know
where
the
fingers
relative
to
the
object,
but
the
actual
model
that's
built
is
not
a
collection
of
feature.
Object,
feature
feature,
location,
feature,
location.
C
A
Exactly
yeah,
if
you
didn't
have
the
location
signal
yeah
at
all
right
then,
and
in
this
case
you
would
just
accumulate
a
bag
of
features,
so
yeah
pointy
thing
a
flat
thing,
a
curvy
thing
and
that's
it
and
that
can,
if
you
change
the
rearrangement
of
those,
you
wouldn't
know
the
difference
right.
But
once
you
have
a
location
signal,
whether
it's
allocentric
location
or
the
relative
location,
then
then
you
know
it's
not
a
backup
feature
right
right
and
then
the
paper
we
actually
tested
against
backup
features
is
the
default
model.
C
C
F
C
A
A
Okay,
I'll
walk
through
one
example,
that's
in
the
paper,
so
I
think
it's
like
the
simplest
example.
I
could
think
of
here
suppose
there
are
two
different
objects.
This
cube
and
this
prism
thing
wedge
and
let's
say
the
top.
One
has
three
different
features:
f1
is
this
little
corner
here?
F3?
Is
this
other
corner
and
f2?
Is
this
vertical
line
here?
The
wedge
has
also
three
features:
f1
and
f3
are
shared,
so
they're
both
little
points
here
and
then
f4
is
a
diagonal
plane
here
and
so
that's
a
different
feature.
A
A
A
A
Okay,
so
here's
what
might
happen
after
the
first
sensation.
So
let's
say
you
reach
out
again
you're
going
into
this
dark
black
box
and
you're
touching
something
with
a
single
fingertip,
and
so
you
sense
feature
one
okay
and
so
the
sensory
input
that
comes
in
corresponds
to
these
many
columns
that
become
active.
A
A
A
A
A
Remember
that
f3
was
in
both
of
these
objects,
but
now
that
we've
coalesced
onto
this
representation
of
the
object,
even
when
you
send
something
that,
on
its
own,
would
be
ambiguous.
The
representation
of
the
top
stays
focused
on
on
the
particular
unique
object,
identity
or
the
str
okay.
Is
that
the
questions
here
is
this?
Okay,
as
I
was
describing
it,
I
started
what
I'm
saying
is
ambiguous,
but
hopefully
it
was
clear
enough.
A
A
So
imagine
this
coffee
cup
is
in
some
black
box
again
and
you're.
There's
some
more
subtleties
about
that's
coming
into
this
description
here,
I'm
going
to
move
my
fingertip
or
this
fingertip
is
going
to
move
and
so
and
sense
something
okay.
So
it's
moving
towards
this
guy
here
and
so
it's
predicting
the
location,
it's
predicting
the
allocentric
locations.
Imagine
there's
some
some
system
somewhere,
that's
taking
that
movement
command
and
doing
a
reference
frame,
transformation
and
converting
it
into
a
location
that
represents
this
3d
location
on
in
the
reference
frame
of
the
cloud.
A
Okay
and
how
that
actually
happens
is
that
topic
of
the
stuff
that
we're
discussing
and
what
you
know
jeff
talks
about,
as
we
call
about
reference
rate
and
transformations
here.
Imagine
you're
making
this
movement
and
you
have
this
transformation,
and
now
you
can
predict
the
location,
that's
going
to
be
on
the
in
the
reference
frame
of
that
cup,
so
that
movement
signal
happens
before
you
ever
get
sensory
input
right
so
that
location
is.
A
So
now,
let's
say
you
touch
it,
you
you
get
an
actual
sensory
input
that
sensory
input
is
going
to
be
a
subset
of
all
the
ones
that
you
predicted,
because
in
the
beginning
you
don't
know
what
object.
It
is
you're
predicting
a
whole
bunch
of
stuff,
and
but
you
get
an
actual
sensory
input
coming
in
corresponding
to
the
side
of
that
coffee
cup,
and
so
these
cells
will
now
become
active,
will
inhibit
everything
else.
These
predictions
will
just
die
away,
so
these
these
cells
will
stay
active.
A
This
in
turn
will
get
be
fed
to
the
next
level,
and
now
all
of
the
cells
that
are
consistent
with
with
the
sensory
input
at
that
location
will
will
become
active
and
let's
say
there
are
three
separate
objects
that
are
consistent
with
that.
So
you'll
get
a
union
of
three
different
objects
at
the
output
layer:
okay,
this
coffee
cup,
the
soda
can
and
the
tennis
ball.
Okay,
let's
say
they
all
have
this
curve
sensation
and
with
one
touch
you
don't
know
which
you've
eliminated
everything
else
we
don't
know
which
of
these
three.
A
A
A
What
I've
neglected
to
mention
here
is
during
learning
you're,
going
to
form
associations
between
all
of
the
cells
that
correspond
to
the
same
object:
okay,
so
they're
sort
of
reinforcing
each
other
plus
you
get
sensory
input.
The
actual
sensed
feature
location
pair
from
the
previous
layer
coming
in,
and
so
only
the
cells
that
have
both
lateral
support
and
support
from
below
are
going
to
stay
on.
The
rest
are
going
to
be
inhibited
okay,
so
this
is
like
a
temporal
k-winner.
A
C
A
C
C
A
Talking
about
you
know
what
actually
generates
this
but
yeah,
but
your
intuition
is
right.
I
mean,
if
you
can
feel
something
really
unique,
then
you
would
instantly
coalesce
onto
that
object,
okay
and
then
we'll
take
a
third
motion
to
the
edge
there
and
same
thing's
gonna
happen.
You're
gonna
predict
the
feature
location
pairs
down
here,
that's
gonna!
A
What
you'll
actually
sense
will
be
a
subset
of
that
and
those
cells
are
gonna,
become
active
through
the
same
mechanism.
We
talked
about
last
week
that
output
gets
fed
above
and
the
exact
same
mechanism
I
talked
about
a
minute
ago
is
going
to
now
narrow
down
into
a
unique
sdr
that
represents
just
this
coffee
cup,
so
the
cells
which
had
support
from
the
previous
iteration
through
lateral
connections
plus
the
incoming
input.
A
E
A
C
In
this
case,
you
can
think
of
it's
like
voting
over
time.
Yeah
one
vote
from
one
century
input.
I
get
another
vote
from
another
sentiment,
but
I
get
another
vote
from
another
century.
10
percent
change,
but
so
the
column
is
sort
of
voting
over
time
to
decide
what
what
hypothesis
or
what
underlying
object
is
consistent
with
all
those
votes
and
then
the
next
stage
is
you
have
multiple
columns
doing
the
same
thing,
but
they
can
vote
amongst
themselves
as
well
yeah,
so
they
eliminate
possibilities
by
voting,
coincidentally,
as
opposed
to
sequential.
A
The
really
beautiful
thing
I
like
is
that
you
think
about
just
from
a
single
neurons
perspective:
it's
it's
basically
the
exact
same
thing
I
talked
about
last
week,
single
neurons,
just
getting
inputs
coming
in
from
all
over
the
place
and
it's
integrating
it
into
the
and
there's
this
k-1
or
take-home
circuit.
That's
really
at
the
end
of
it.
That's
all
that's
going
on
it's
very
simple!
It's
just
the
way
you
connect
it
up
and
the
way
you
create
these
networks
into
very,
very
different
functions
and
very
powerful
functions.
E
A
C
C
So
yeah,
I
wouldn't
get
hung
up
on
that
kind
of
stuff.
It's
like
you
know
if
you
reach
your
hands
in
black
box
and
touch
something
and
then
walk
away
and
come
back
later
and
reach
your
hand
in
the
back,
you
probably
won't
have
any
difficulty
with
integration,
and
I
there's
this
kind
of
idea.
While
one
of
my
head
sounds
like
a
a
working
memory,
I'm
keeping
I'm
I'm
thinking
about
it,
but
not
moving
anything
for
a
while.
That's
like
a
separate
issue.
Most
of
the
time
you
don't
realize,
you're
doing
this
at
all.
C
I
could
reach
over
to
my
side
here
and
pick
up
this
coffee
cup.
I
even
know
I'm
picking
up
a
coffee,
but
I
may
not
even
know
where
it
and
is
you
know.
I
touched
something,
and
I
touched
it
again
and
also-
and
I
know
not
only
what
it
is
but
where
it
is
relative
to
me-
and
so
you
know
most
of
the
time
these
are
very
floating
non-thoughtful,
not
thinking
about
it.
It's
just
you're
doing
this
every
waking.
Second,
every
time
you
touch
anything
you're
essentially
doing
this.
E
C
Yeah
yeah
and
I'm
just
saying
it's
not
the
main:
it's
not
the
main
way.
We
do
it.
You
know
there
is
this
idea
of
working
remembering
you
could
keep
something
in
mind.
That's
why
you're
trying
to
right,
but
I
mean
it's
it's
I
kind
of
like
it.
How
we
do
certain
creative
thoughts
is
that
we're
able
to
pull
things
together.
E
A
E
A
Yeah,
I
hope
I
try
to
convey
that
kind
of
the
simplicity
of
it
all
at
the
underneath
it,
and
that's
one
of
the
reasons
why
I
think
we
should
we
need
to
focus
a
lot
on
the
neuron
model
and
how
to
really
think
about
the
neuron.
The
same
thing
with
learning
it's
very
simple.
At
the
end
of
the
day,
it's
just
local
to
the
neuron.
A
It's
like
you
know,
a
touring
machine
is
really
simple,
but
an
operating
system
is
super
complex,
but
at
the
end
of
the
day,
it's
the
starting
machine,
there's
intricacies.
Let's
put
it
that
way,
right,
yeah
anyway,
of
course,
I
don't
mean
to
say
that
details
are
not
important.
It's
just.
I
I
like
to
also
think
of
it
as
being
at
the
end
of
the
day,
kind
of
simple.