►
From YouTube: Niels Leadholm on Grid Cells and Reference Frames in an Image-based Setting - October 5, 2020
Description
Niels Leadholm, a visiting researcher, discusses some ideas for further research on how to apply the object recognition implemented in Numenta's 2019 paper ""Locations in the Neocortex: A Theory of Sensorimotor Object Recognition Using Cortical Grid Cells"" to images. In particular, the research would explore whether the strengths of object reference frames and grid-cell encoding can be leveraged in an image-based setting.
Read paper here: https://www.frontiersin.org/articles/10.3389/fncir.2019.00022/full
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a cohesive theory, core software technology, and numerous software applications all based on principles of the neocortex. Our innovative work delivers breakthrough capabilities and demonstrates that a computing approach based on biological learning principles can do things that today’s programmed computers cannot do.
Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://tinyurl.com/NumentaNewsDigest
Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter
Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta
Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/
Our Website:
https://numenta.com/
B
Yes,
so
the
the
kind
of
main
idea
is
essentially
to
build
on
what
was
done
in
the
using
cortical
grid
cells
to
perform
object,
recognition
paper
and
then
try
and
replicate
basically
those
results,
but
with
actual
images
and
and
kind
of
show,
it's
it's
flexibility
and
being
able
to
do
that,
and
so.
In
order
to
kind
of
do
this,
then
the
the
basic
idea
is
to
say:
okay
well,
starting
with
images,
simple
images
like
mnist
or
fashionedness.
B
How
can
we
derive
some
meaningful
sdrs
to
then
feed
into
the
system
of
object,
recognition
and
so
to
do
this?
I've
just
started
with
a
simple
cnn
architecture:
convolutional
neural
network,
which
you
know
kind
of-
has
this
standard
structure
of
convolutions,
followed
by
max
boolean,
where
it's
kind
of
32
channels
and
64
channels
and
then
some
hidden
layers
that
are
fully
connected
to
perform
classification
and
then
so.
B
B
And
so
then
the
idea
is
this
is
then
the
kind
of
features
that
can
then
be
used
in
the
for
kind
of
grid
cell
object
recognition
where
at
each
one
of
these
25
locations
there
is
an
sdr
and
then
so.
This
was
just
kind
of
the
idea
to
do
this.
Obviously,
there's
then
an
assumption
that
these
sdrs
are
actually
going
to
have
some
useful
information
about
the
inputs.
C
Yeah,
I
I
I
but
I
have
a
question
about
it
and
it
might
be
a
stupid
question.
You
know,
because
I
don't
I'm
not
very
fast
with
these,
but
you
know
the
idea
here
ultimately
is
you
know
you
have
some
some
features
at
some
point
in
a
reference
frame
right
and
but
you
haven't
introduced
that
yet
all
you
have
is
at
the
moment,
all
you
have
is
its
location
on
the
image
or
something
like
that
which
is
not
a
proper
location
on
the
object.
C
B
B
C
B
A
C
B
Normally
be
the
case,
got
it
and
then
yeah
so
then
the
question
is
okay.
Well,
I've
kind
of
derived
these
features,
but
are
they
actually
helpful
so
starting
with
mnist?
So
it's
a
64
by
5x5
kind
of
vector
of
binary
features
and
then
so
I
can
feed
these
into
a
linear
classifier
like
a
multi-layer,
perceptron
and
then
a
decoder.
B
B
B
That
was
missed,
yeah,
okay,
okay,
if
I
just
take
one
of
the
sdrs
this
one
and
feed
that
to
a
linear
classifier,
then
that
gets
kind
of
close
to
60
accuracy,
so
not
very
strong.
But
if
I
feed
all
of
the
sdrs
to
a
linear
classifier,
it
actually
does
quite
close
to
the
kind
of
original
network
and
kind
of
shy
of
if
I
feed
them
to
a
multi-layer
perceptron.
B
So
I
kind
of
try-
and
I
I
break
down
these
results
again
later
on
in
kind
of
a
table
to
show
to
kind
of
summarize
this,
and
then
the
decoder
actually
does
quite
a
good
job,
reconstructing
the
inputs
from
the
original,
so
kind
of.
In
conclusion,
there
is
definitely
information
in
these
features.
It's
not
so
trivial
that
just
given
one
sdr,
you
can
kind
of
perform
good
classification.
B
And
then
I
did
the
same
thing
with
fashion
numbness
and
it's
basically
the
same
result
just
slightly
different
but
so
kind
of.
In
summary,
the
kind
of
the
way
I'm
interpreting
this
is
the
kind
of
the
actual
binarization
process
where
I
go
from
k
winner,
take
all
to
enforcing
that
the
features
are
either
zero
or
one
causes
this
drop
in
classification
accuracy
from
the
original
and
then,
in
addition
to
this
drop.
B
So
that's
kind
of
that's
that
side
of
things
and
then
so
in
terms
of
kind
of
looking
forward
so
yeah.
The
main
thing
I
wanted
to
start
was
looking
at
was
just
object
recognition,
but
this
leads
to
kind
of
an
interesting
situation
where
you
kind
of
have
sort
of
general
versus
particular
digit
recognition.
B
And
so
what
I'm
going
to
kind
of
start
with
is
is
just
the
kind
of
the
simplest
case
where
I'll
have
like
kind
of
possibly
yeah
ten
canonical
examples
and
then
performing
classification
on
those.
But
when
I
move
on
to
kind
of
wanting
to
classify
nines
in
general
or
with
anesthesia
in
general
yeah,
I
imagine
it
will
require
some
tweaking
of
the
the
kind
of
hyper
parameters
of
the
the
grid
cell
algorithm
to
to
get
it
to
kind
of
robustly,
classify
these
actually
and
actually
yeah.
B
B
So
yeah,
so
my
next
step
is
to
kind
of
actually
look
at
implementing
this
object
recognition,
and
then
I
thought
I
just
discussed
some
of
the
ideas
I
have
for
time
permitting
what
I
could
explore
afterwards
and
so
kind
of
one
of
the
things
that's,
and
so
this
kind
of
gets
to
your
question
earlier
jeff
about
like
why.
Why
even
kind
of
do
this,
with
the
grid
cell
modules,
so
kind
of
one
of
the
interesting
properties
of
the
original
system?
B
Is
this
kind
of
one
few
shot,
hebbian
memorization
of
feature
location
pairs,
and
what
I'm
interested
to
see
is
if
this
is,
which
is
kind
of
yeah,
very
much
part
of
the
system
in
the
original
favorite.
But
this
is
still
possible
in
the
kind
of
same
hyperparameter
space
that
enables
robust
recognition
of
digits
so
kind
of.
Once
once
I
found
that
hyperparameter
space,
where
okay,
it
can
classify
nines
and
ones
and
sevens,
can
it
still
do
this
in
one
or
a
few
shots,
or
does
it
now
actually
require
many
more.
B
But
what
I
think
would
then
be
interesting
is
because
I'll
be
asking
it
to
predict
an
sdr,
but
I
can
then
take
those
sdrs,
including
the
one.
That's
predicted,
feed
it
into
the
decoder
and
then
actually
see
what
they
kind
of
completed.
Image
looks
like
and
then
compare
that
to
if,
if
a
random
sdr
is
put
in
instead
just
to
get
kind
of
a
sense
of
yeah,
how
again
kind
of
how
good
is
the
system
when
kind
of
dealing
with
real
real
data?
B
And
then
this
could
also
be
compared
to,
for
example,
a
recurrent
neural
network,
where
the
kind
of
expectation
is
that,
although
it'd
be
fine
as
long
as
you
always
feed
the
features
in
the
same
order.
If
you
start
feeding
the
features
in
a
new
order,
it
will
likely
be
stumped,
whereas
the
system
should
be
robusted.
C
B
C
B
Is
more
interesting,
yeah,
okay
and
then?
Lastly,
this
is
kind
of
not
something
I've
thought
too
hard
about
yet,
but
bringing
in
some
of
these
kind
of
hierarchical,
binding
ideas
from
what
I've
been
doing
on
my
phd,
where
kind
of
the
features
I've
shown
you
so
far
are
derived
from
this
max
pooling
layer.
But
then
the
idea
here
is
that
there
would
be
kind
of
multiple
levels
of
abstraction,
and
then
so
you
know
possibly
the
the
max
point.
Ones
are
better
for
doing
the
object,
recognition
part.
C
So
yeah
it's
interesting.
I'm
just
trying
to
understand
this
in
context
of
other
way.
I
think
about
things
what
you
know,
one
of
the
things
when
we
when,
if
you
were
presented
with
an
mnist
character,
that's
not
obvious
to
you
and
there's
there's
a
bunch
that
are
kind
of
like
yeah.
What's
that
and
and
so
what
we
tend
to
do,
and
we
tend
to
do
any
object,
we
don't
recognize.
Is
we
attend
to
different
components
of
it?
Small
parts
and
we
scan
them.
A
C
Know,
and
and
by
looking
at
the
look
at
the
relationships
of
sub
components,
it
can
remind
us
of
something
we
know.
So
I
don't
understand
what
this
letter
is.
But
I
see
oh
that
looks
relative
to
this,
but
like
it
might
like
to
be
a
seven
or
this
relative
to
something
else,
might
be
a
three
or
something
like
that,
and
so
that
we
kind
of
go
through
this
serial
attentional
process.
C
When
we
see
something
we
it's
unfamiliar
to
us,
whether
it
doesn't
have
to
be
you
know,
it
could
be
any
time
I'm
presented
with
something
new.
That's
what
we
do.
We
talked
about
this
quite
a
bit,
and
so
it
doesn't
seem
you're
capturing
that
aspect
of
this
and
I'm
not.
I
don't
mean
you
should
I'm
just
just
making
sure
I
understand
this.
It's
like
it's.
It
would
be
like
the
way
we
would
learn
a
new
letter.
C
If
I
was
teaching
a
child
a
new
character,
I
would
I
would
point
out
the
different
components
and
I'd
say:
oh
well,
it
throws
a
cross
and
there's
a
down
thing
and
there's
another
thing
and
there's
a
round
thing,
and
so
we
sort
of
teach
it
as
these
sub-components
thoroughly
presented
and
then
only
after
you've
trained
it.
That
way,
can
you
then
say?
Oh,
I
recognize
it
in
the
same
way
like
we
might
train
a
word.
C
B
C
Kind
of
attending
to
yeah
okay,
so
I
guess
I
wasn't
really
clear
about
that.
So
you're,
hey!
Obviously
you
can
process
the
features.
It
doesn't
really
matter,
they're
just
sitting
there
in
a
buffer
someplace.
So
the
question
is:
when
you
train:
how
do
you
go
about
it
and
yeah?
So
you
think
you're
capturing
that
then.
B
Serially,
it's
not
going
to
do
them
and
it's
so
it
would
almost
be
like
it
would
almost
be
like
being
presented
an
object,
a
novel
object
that
you've
never
really
seen
before.
So
you
you
just
get
a
you
know,
150
milliseconds,
to
look
at
it
or
whatever
yeah,
and
then
it's
gone,
but
then
in
your
heads,
you're
able
to
kind
of
parse
through.
B
C
I
mean
because
we
don't
think
the
brain
never
does
anything.
You
know
we
have
this
like
yo.
We
can
do
this,
one-shot
learning,
we're
so
quick,
but
we
don't
really
right
we're
really
quick,
but
we
do
it.
Certainly,
so
I
think
capturing
that
is
important.
I
think
another
interesting
thing
you
might
just
think
about,
and
perhaps
we
already
have
is
you
know
what
happens
if
you
present
a
character
which
is
really
ambiguous?
You
know,
outside
of
eminence,
I
don't
know,
give
it
something
completely.
C
You
know,
then
you
know
you
give
it
a
half
and
half
you
know
in
theory,
the
order
in
which
you
go
through
it
in
a
brain
would
say:
hey
I've
attended
to
these
three
features
that
looks
like
x
and
I've
attended
these
three
features.
It
looks
like
y
because
it
doesn't
look,
you
know
the
thing
is
new
and
obvious,
and
then
you
could
say:
hey
it's
a
bit
like
accents
a
bit
like
y,
and
that
would
be
very
clear.
C
C
In
some
sense,
it's
a
generalization
mechanism
right,
yeah,
you're,
you're,
finding
out
subsets
of
the
thing
you're
looking
at
are
like
this
and
subsets
are
like
that,
and
so
you
can
go
back
and
forth.
Saying
like
yeah,
I
can
see
why
it's
like
x
and
I
can
see
why
it's
like
why
it's
not
just
bam.
Here's
an
image,
I'm
confused
or
it's
you
know,
it'd
be
more
like
hey.
When
I
look
at
it
this
way,
it's
clearly
xml
look
at
this
way.
It's
clue!
C
B
C
C
C
A
B
A
A
Pixel
of
the
first
quarter,
I
think
it
should
be
identified
from
that.
I
see
so
so
in
case
of,
like
am
I
correct,
to
understand,
like
there's,
there's
really
not
not
really
much
spatial
information
in
in
each
of
these
sdrs,
the
the
sdr
is
like
this
is
quite
analogous
to
to
the
paper
you're
talking
about.
We
call
it
columns
plus
locations
in
the
neocortex
where
like
where
this
is
really
saying
like
at
this
location,
on
the
image.
What
is
the
feature?
Sdr?
A
A
D
Given
that
it's
convolutional
there
is
spatial
semantics
in
how
it's
laid
out,
so
I
don't
think
you
did
it's.
Basically,
the
the
spatial
location
is
implicit
by
where
it
is
in
that
array.
Even
if
it's
you
know,
if
you
look
back
out
to
it,
it's
you
know,
maybe
looking
at
you
know
a
25
by
25
region
or
you
know,
whatever,
whatever
the
you
know,
because
you're
going
back
a
couple
of
levels
of
convolution,
so
there
is
spatial
information.
D
In
whatever
extent
things
are
being
aggregated
through
the
linear
classifier
that
you
know
the
upper
left-hand
corner
plus
something
that
was
like
that.
That
combination.
If
I
wait
that,
then
it
says:
okay,
that's
that's
a
dominant
feature,
indicative
of
something
you
know,
so
I
I
think
you've
got
both.