►
Description
Broadcasted live on Twitch -- Watch live at https://www.twitch.tv/rhyolight_
A
I
started
with
the
question
of
I,
wanted
to
understand:
3d
orientation,
better
and
I.
Think
I've
got
a
useful
question
for
understanding.
3D
orientation
is,
if
you
just
have
a
simple,
continuous,
attractor
Network,
that's
representing
3d
orientation
representing
and
updating
it.
What
would
it
look
like
in
the
sense
that,
like
we
know
that
a
2d,
continuous
attractor,
a
link
to
2d
location
on
this
representing
location,
is
just
trivial?
A
It's
just
a
sheet
of
cells,
the
pumps
moving
over
it,
the
3d
location,
one
would
probably
like
a
naive
one
would
just
be
a
3d
sheet
of
cells
or
3d
volume
of
cells
above
the
equivalent
version
for
3d
orientation.
I,
didn't
know
what
it
would
look
like
and
I
felt
that
I
would
come
away
smarter
if
I
did
know
what
it
looks
like
and
it
would,
but
just
in
general,
be
something
that
may
be
a
tool
that
we
reach
out
and
need
to
use
somewhere
in
one
of
the
places
where
we
were
presenting
orientation.
A
A
Other
say
Laurie
the
we're
the
center
of
the
sphere.
Let's
say
you
have
like
a
default
orientation
or
like
the
wrath
on
top
of
the
sphere
facing
north
is
its
default
orientation.
That's
here
at
my
center,
all
of
the
other
orientations
of
the
wrath
of
the
agent
can
can
be
reached
by
rotating
about
some
axis
at
some
amount
up
to
180
degrees.
A
So,
just
to
read
this
off.
The
center
of
this
is
like
it's
kind
of
a
default
orientation.
The
direction
of
the
vector
is
the
the
rotation
axis
of
the
agent.
The
length
of
it
is
how
much
is
rotated
in
the
interesting
interesting
thing
about
this
sphere.
Is
that
all
opposite
points
on
the
surface
of
it
are
the
same?
A
C
A
C
You
know
how
could
tell
arranged
just
like
that,
and
the
question
is
you
know:
I
can
imagine
you
can
feel
like
Rin
solid.
You
can
physically
how
this
can
continuous
cells
and
it
pumps
moving
between
them
and
and
it
keeps
repeating
itself
and
it
all
works.
No,
the
cells
that
are
moving.
You
see
these
beautiful
space
yeah
and
the
questions.
Not.
Obviously
you
could
have
the
bottom
of
envision
that
you
could
physically
move
in
contiguous
space
here.
C
C
Just
wondering,
I
wonder
this
question
about
with
orientation,
whether
it's
possible
to
have
a
similar
type
of
setup
or
not,
where
you
could
argue
the
ball
moves
back,
but
if
I
just
have
a
continuous
sheet,
could
the
bump
continue
on
by
multiple
copies
of
the
same
thing?
Could
that
same
property
that
would
work
with
grid
cells?
Work
here,
I
guess
depression.
It's
not
obvious
to
me
because
my
intuition
would
say
not,
but
but
I
don't
really
know.
Do
you
have
a
sense
of
that
I
don't
know
no.
C
C
C
A
A
And
you
have
those
cells
distance
themselves
from
each
other
maximally
and
they're
tuning.
They
will
not
be
uniformly
spread
through
this
feeder.
This,
the
sphere
does
not
correctly,
just
not
correctly,
just
like,
isn't
that's
not
equivalent
to
randomly
placing
them
in
this
sphere.
They're
gonna
be
kind
of
more
center
to
the
center,
or
something
like
that.
A
So
just
we
shouldn't
go
too
deep
into
this,
but
I
just
had
to
State
for,
like
just
for
the
record,
the
actual
way
to
visualize
3d
orientation
space
in
a
way
that
keeps
all
in
metric
the
metric
relations
to
them
clear.
It
is
the
area
of
the
top
half
of
a
4d
hyper
sphere,
which
can
which
I
mean
here
I'm
just
showing
a
series
of
spheres
as
the
fourth
dimension
changes,
and
all
this
is
just
to
say,
you
can
take
this
set
of
shells,
the
set
of
spheres
and
pack
them
into
a
3d
sphere.
A
C
A
forty
or
forty-five
mystery:
okay,
fine,
that's
good,
but
I'm,
trying
to
imagine
what
would
that
be
equivalent.
Now,
how
could
I,
let
that
in
in
actual
neurons
and
what
they
look
like
I'm,
going
back
to
like
the
whole
I'd,
given
any
columns
in
the
slab
I,
don't
think
it's
not
an
irrelevant
question!
It's
an
important
question
and
I
just
want
every
time
easier
to
sell.
That's
what
I'm
imagining
about
something
else,
yes
or
about,
though
the
distribution
in
this
sort.
A
Sure
so
I
guess
before
I
just
show
the
picture
is
so
it
kind
of
answered
to
this.
What
does
the
can
look
like?
Well,
the
answer
is,
if
you
take,
if
you,
if
you
choose
one
cell
of
the
continuous
attractor
that
represents
through
the
orientation
and
just
like
Center
it,
but
if
you're
sorting
cells,
another
particular
I
just
choose
one
cell
center,
it,
the
rest
of
orientation.
Space
appears
as
a
sphere
around
that,
but
there's
not
inherently
a
right
correct
center.
So
it's
kind
of
strange
and
I'll
show
the
visualizations
of
it
here.
C
Because
something
you
said
the
right
beginning,
what
I
just
want
to
keep
these
ideas
on
the
table
is
that
so
Jesus
imagined
you
know
a
location
like
a
2d
location
and
continuous
chapter
model,
but
actually
it's
not
I
mean
it's
like
because
it
because
it's
because
they
continue
to
dr.
Milo
repeats
you
don't
really
get
location
you
have
to
and
the
way
we've
got
location
people
do.
You
have
to
have
different
modules
of
different
it.
So
it's
funny
that
the
actual
physical
cells
that
we
know
grid
cells
are
not
good
at
representing
location
because
they.
A
C
C
C
A
A
C
A
D
C
A
This
default,
like
the
the
starting
orientation
kind
of
standing
at
the
North
Pole,
with
the
red
facing
to
the
right
and
moving
a
certain
direction
in
the
sphere
and
showing
these
axes
as
like
other
rotations
of
the
agent
moving
along
this
axis.
This
top
axis
is
like
rotating
all
on
the
Palouse
on
rotating
on
that
axis,
or
so
it's
rotating
on
the
blue
plane
similar
with
this
axis.
It's
like
rotating
on
the
pink
plane.
This
is
like
rotating
on
the
blue
plane
and.
A
Like
so
like
a
cell,
that's
centered
right
here
represents
the
disorientation
a
cell.
That's
right
here
represents
this
orientation
and
okay
I'll
temporarily
hide
those
little
pictures,
I'll
bring
them
back
soon,
but
now
I'm
gonna,
shut
ups
could
have
put
checkboxes
here,
but
instead
I
have
to
type
true
and
false
I'll
go
ahead
and
show
a
bump
of
activity
moving
through
it.
So
here's
a
bump
I'll
show
it
moving
as
the
agent
moves.
C
A
And
so
and
start
a
bunch
of
movements,
I
must
have
pressed
there.
We
go
so
now.
The
agent
is
moving
just
randomly
over
oh
through
orientation.
Space
and
this
bump
of
activity
is
moving
retracting.
Its
orientation
and
it's
I'll
show
pictures
a
little
bit
that
makes
this
clearer,
but
right
now,
you'll
see
that
the
bump
is
Janet
is
usually
arcing
from
position
to
position.
It's
not
going
in
a
straight
line
from
position
to
position
well.
C
A
C
C
C
C
C
A
These
lines
from
this
orientation
to
this
orientation
are
becoming
curved,
which
is
to
say
under
this
view,
if
you
moved
from
this
orientation
that
this
orientation
is
going
to
arc
is
going
to
make
this
kind
of
arc
shape,
but
under
when
it's
when
it's
centered,
it's
going
to
be
more
straight
again.
Meanwhile,
since
I'm,
you
can
see
now
that
what
I'm
doing
right
now
is
I'm
rotating
the
view
along
this
axis.
A
So
this
lines
staying
straight
here,
I'm
just
really
trying
to
get
a
little
bit
of
an
intuition
for
what
this
attractor
looks
like
or
how
everything
is
connected
and
now
I'll
go
ahead
and
show
one
more
picture.
I'm
gonna
make
that
so
reorienting,
because
it's
making
my
CPU
sad
I'll
go
ahead
and
I'm
gonna
go
back.
A
I'm
gonna
do
one
more
thing
now:
I've
been
showing
this
agent
moving
around
this
sphere
and
this
bump
moving
with
the
agent
now
I'm
going
to
add
one
more
thing
that
occurs
after
every
movement:
I'm
going
to
recenter
the
bump
I'm
gonna
move
the
but
I'm
gonna.
Basically,
I'm
gonna
change
the
view
so
that
the
bumps
at
the
center,
so
I'll
go
ahead
and
do
this.
A
C
A
C
A
C
A
There's
always
a
little
bit
of
weirdness.
That
there's
always
going
to
be
some
point
on
the
surface
of
a
sphere
that
is
icy.
You
could
call
the
singularity
we
call
it
anyway.
The
point
is
that
if
you,
if
you,
if
you
draw
arrows
over
the
surface
of
a
sphere,
there's
going
to
be
some
point
where
the
arrows
seven
point,
hot
and-
and
it
makes
it
feel
like
orientation-
is
inherently
like
this
hard
thing
where
our
orientation
is
inherently
something
that's
messy
like
that.
But
what
I
wanted
to.
C
C
A
Any
any
type
of
representation-
that's
rings,
I
think
you
can
say
anything
that
breaks
3d
orientation
into
multiple
populations
like
breaks
it
into
multiple
variables.
It's
going
to
have
weakness
of
this,
but
that,
but
that
weirdness
is
an
artifact
of
the
fact
that
you're
breaking
it
up
like
that,
and
that
was
one
thing
I
want
you
to
understand
this.
If
there's
that
is
this,
is
this
fundamental
to
orientation
that
you
have
these?
The
South
Pole
is
always
broken
or.
A
Yeah
and
under
this
view
like
maybe
it
may
be,
this
changes
by
animal-
maybe
who
knows
maybe
changes
by
part
of
the
brain,
but
if
you
do
find
this
ring,
if
you
do
find
head
Direction
cells,
for
example,
in
an
animal,
it's
it's
so
possible
that
the
animal
actually
has
one
of
these
3d
attractors
and
the
resurrection
cells
are
really
just
reading
it
out
and
because
the
Unruh,
because
this
3d
version
just
works,
it
requires
a
lot
of
cells.
I
don't
have
the
exact
numbers
on
this
here.
A
B
C
A
I
mean
the
nice
thing
here
is
that
this
is
much
fewer
cells,
then
they're
always
being
used.
The
one
point
that
the
paper
that
we
were
basically
talking
about
from
Kate
Jeffrey's
lab
one
thing
it
brings
up
is
that
the
weird
thing
about
having
a
full
3d
attractor
for
orientation
is
that
many
of
those
orientations
are
going
to
be
very
rarely
visited,
and
so
it's
just
strange
to
just
have
the
exhausting
neural
material,
keeping
that
all
correctly
connected
and
everything
so
yeah.
A
A
A
Moving
over
the
coffee
cup.
Are
you
representing
the
location?
Are
you
representing
like
if
you
feel
like
the
laser
shooting
after
eye
and
hitting
the
cup?
Are
you
representing
that
location
or
are
you
representing
where
your
is
relative
to
the
cup
and
I
just
wanted
to
lay
out
one
coherent
view
of
all
of
this?
That
does
represent
the
location
of
the
sentence
feature,
but
in
a
different
way,
is
kind
of
what
we
talked
about
in
the
cosine
coaster
last
year.
A
The
six
for
our
doing
our
normal
thing
that
we
talk
about
it's
it's
a
location
of
a
sensor,
for
example
a
touch
sensor
relative
to
some
basic
feature
like
a
cylinder
or
or
a
cube
or
or
a
handle
which
might
be
a
little
bit
of
stretch,
but
the
scope
so
yeah.
This
is
just
the
basic
model
for
paper
locations
in
the
neocortex,
writing
orientation.
A
Platform
yeah
now
one
thing
I'll
throw
in
I
didn't
know
how
to
did
no
orientations.
I
just
drew
like
a
the
volume
of
a
sphere
somehow
about
volume
of
a
sphere.
That's
like
supposed
to
be
like
this,
but
this
this
circle
might
actually
be
broken
into
multiple
subpopulations.
My
orientation
could
be
represented
in
different
ways:
okay,.
C
A
So
it's
conceptually
on
a
high
level
that
these
labels
will
apply
to
all
of
those
to
all
of
those,
no
matter
how
you
swap
in
those
details,
so
we
in
this
model
the
child
object
in
the
parent
object
or
the
feature
and
the
object
both
have
their
own
location
space,
their
own
reference
frames
and
say:
we've
talked
about
6a,
representing
the
location
and
the
features
reference
frame
also
know.
There's
a
child
objects,
reference
frame,
the
location
of
the
center
6b
as
location
of
that
sensor.
A
C
A
A
Yet
yeah,
basically,
we've
talked
about
if,
if
you
were
leaving
out
Lorient
ation
completely,
this
would
certainly
be
this
displacement
in
a
strict
sense
of
the
word.
Anyway.
We
could
talk
more
about
that.
What
he
does
scaling.
No
there's,
no
there's
no
scaling
in
here.
However,
there's
no
scaling
in
here.
However,
if
you
just
like
you
just
change
the
implementation
details
inside
the
boxes,
suddenly
there
would
be
scaling
and
these
labels
would
still
apply.
A
So
the
thing
I
wanted
to
point
out
yeah
is
that
when
I
first
laid
this
out,
I
could
describe
this
as
a
transform
and
later
I
realized
another
other
language
for
describing.
This
is,
if
you're,
representing
the
transform
between
these
the
displacement
between
these
another
way
of
saying
that
is
you're
representing
the
location
and
orientation
of
the
feature
relative
to
the
object.
Yes,
which
is
another
way
of
saying
the
location
of
the
sensed
feature.
C
A
C
C
D
Well
so
one
question:
I
have
maybe
it's
the
same
thing
silks.
So
you
have
this
position
of
the
sensor
relative
to
the
feature
yeah,
and
so
you
know,
if
I'm
looking
at
a
feature
and
it's
getting
closer
or
further
away
from
me.
That
position
is
changing.
Yeah
same
thing,
sensor,
relative
to
the
object
that
the
object
is
moving
closer
further
away,
keeps
changing
yeah,
but
the
feature
itself
relative
to
the
object,
the
position
that
never
done
doesn't
stretched.
D
A
C
A
C
All
right
so
I
don't
understand
how
to
interpret
that
in
this
picture.
Right
now
we
say:
okay,
the
thing
that
that's
being
used
to
predict
the
input,
the
lighter
form
is
also
being
used,
the
scale
and
and
yet
layer
6b
does
not
seem
to
be
doing
that.
There's
often
as
though,
as
far
as
we
know,
there's
no
and
roll
like
that.
So
somehow
we
had
this
sort
of
one
thing
which
is
need
to
have
scale
then
letting
the
other
woman
just
alright
I'm
thinking
I
mean
we
did
have
to
right.
There.
C
A
B
C
A
C
C
A
B
C
C
C
C
So
it's
a
total
sort
of
sub
diffuse
of
some
sort.
That's
going
on
here,
whether
you're
acting
in
one
way
but
actually
everything's
scale.
Both
the
movements
of
scales
and
features
are
scale,
and
so
the
cortex
doesn't
know
the
difference.
This
is
copacetic
with
your
idea.
Maybe
that's
a
prototypical
business.
C
C
A
C
C
C
A
So
I
guess
my
second
motivation
for
drawing
all
this
was
to
show
this
in
context
and
saying
that,
like
okay,
we
have
to
we
kind
of
know
two
fundamental
use-cases,
two
fundamental
questions
for
orientation
location.
One
of
them
is
what's
going
on
inside.
One
of
these
boxes,
however,
power
location
and
orientation
represented,
and
the
second
is
how
to
multiple
of
these.
How
do
you
detect
the
transform
between
them
and
determine
one
from
the
other
etc
and
anyway
I'm?
C
This
rectangle,
and
what
are
these?
How
do
these
layers
actually
talk
to
each
other?
How
would
not
relations
actually
I
don't
know.
This
is
the
fact
that
I
just
I
want
to
keep
emphasizing.
It
appears
that,
if
you
think
about
the
classic
orientation
that
people
think
about
it,
then
that's
somehow,
you
know
goes
through
these
layers
right
and
so
these
guys,
but
there's
somehow
linked
together,
where
that's
not
true
of
these
things.
So
that's
just
something
to
keep
in
mind.
C
If
that's
true
yeah,
just
you
know
when
you're
talking
about
a
singer,
so
at
the
end
of
this
yeah,
the
other
thing
when
you
talk
about
the
4d
representation,
it
struck
me
remind
me
of
something:
I
don't
know.
If
it's
even
has
anything
to
do
with
this
enough,
but
reminding
me
that
we
think
about
how
there's
these
orientation
columns
right
in
v1
and.
C
D
D
C
A
A
A
C
Nothing
to
wear
a
super-tight
I
have
all
these
different
orientations,
these
movement
directions,
but
in
the
end,
the
many
column,
equivalent
representation
for
it
and
when
he
comes
to
oh,
that's,
the
orientation
I
want
is
for
learning,
but
these
individual
styles
or
something
like
that
yeah,
because
this
is
sort
of,
is
what
you
need
the
movement.
But
it's
not
I
mean
it's
not
what
you
want
for
object,
rotation.
C
C
C
C
A
Now
I
would
be
able
to
figure
out
how
to
connect
up
these
layers
so
that
they
perform
a
full
coordinate,
transform
such
that
with
what
we've
documented
so
far.
We
could
do
this
coffee
cup
example,
but
we
can't
do
the
briefcase
example,
because
the
handles
being
handle
is
rotating
and
I
could
be
able
if
there
were
the
right
thing
to
do
right
now,
I
could
I
couldn't
implement
this
so
that
it
does
both
the
coffee
pump
in
the
briefcase.
A
That
said,
I
already
know
what
it
I
don't
know
the
fact
that
I
know
I
can
get
it
to
work.
I,
don't
know
what
information
I'll
get
out
of
getting
it
so
anyway,
I'm
putting
this
all
out
there
that
we
had
to
space
to
orientation.
Questions
answered,
what's
going
on
inside
your
what's
going
on
between
these
often
when
I'm
presenting
at
these
I'm,
not
mentally
equipped
to
figure
out
what
I'm
doing
next,
because
I
put
all
my
focus
on
this.
C
D
A
Naturally,
do
it
as
yeah
the
union
of
all
later
five
well
for
one
thing:
each
each
layer,
five
each
feature
relative
to
object,
is
specific
to
the
object.
So
you
could
classify
the
object
based
on
this
yeah.
You
could
have
another
population
cells.
Maybe
there
is
a
layer,
2
and
layer
3,
one
of
them
feature
one
of
them's
object.
There
they're
always.
A
Yeah
I
mean
another
question
that
is
kind
of
specific
to
this
diagram,
but
another
question
is
yes
I.
This
puts
a
large
demand
on
layer,
four
layer,
six
on
this
connection,
to
learn
a
lot
of
stuff
if
I
take
one
way,
I
think
about
this
whole
model
is
I'm,
just
taking
kind
of
a
classic
view
of
computer
vision
and
encoded
it
into
neural
into
neurons.
A
So,
but
because
it's
because
it's
kind
of
like
this
intuitive
thing,
people
have
reached
that
conclusion
before
and
for
like
I
could
list
people
to
talk
about
both
Mar
and
Biederman.
One
thing
they
had
and
they're
they're
primitives,
like
they're
cylinders
or
whatever
is
they
assumed
the
system
could
like,
could
kind
of
skew
them
or
reshape
them
in
different
ways.
Like
you
see
a
new
set,
a
new
cylinder
to
spent
in
a
slightly
different
way.
How
do
you
represent
that?
A
So,
like
another
area
of
focuses
in
our
diagram,
it's
like
what
is
going
on
in
this
layer
sites
layer
four:
could
you
give
it
some
more
flexibility
of
some
kind
working
like
Miss
shape,
I,
don't
know
I.
Can
they
could
take
one
of
these
and
do
a
skinnier
tone
there,
or
something
like
that?
Neural
mechanisms
for
jion's
narrow
mechanisms
for
these
parameterised
features.
I,
don't
even
know
where
to
go
with
that,
but
that's
another
that
that's
another
way
to
really
make
this
work
in
the
real
world.
Well,.
C
C
To
know
this
part
of
this
object
is
like
a
motorcycle
on
these
parts
over
here
or
like
a
flower,
and
it
depends
of
which
order.
Would
you
attention
them,
because
some
sub
structure
will
be
similar
to
something
else?
I
guess
I'm,
saying
that
part
of
that
answer
that
I
think
is
going
to
fall
attention
on
the
back
when
you
buy
sequential
attention
of
features,
I
know,
I,
think
that's.
C
C
D
C
D
C
C
And
paper,
and
so
on,
is
growing
evidence
that
may
not
be.
Might
it's
gotta
be
so
that
else
going
on
and
I
made
the
point
that
orientation
are
the
single
wing
attractors
insufficient,
something
else.
Perhaps
now
we
know
there's
something
else
like
this.
This
shows
a
little
sphere
or
the
gravity
vector
or
whatever
we
want
to
call
it,
and
I
also
thinks
I
want
to
be
even
understand.
The
location,
I
think.
C
C
A
One
one
thing
that
I
put
into
the
paper
locations
in
the
neocortex
was
I
thought.
This
was
interesting
and
just
worth,
including
so
I
made
sure
to
include
it
was
that
the
model
when
it
has
only
one
module,
still
were
yeah
one
one
module
for
location
so,
like
the
layer,
four
layers
they
excrete
up
just
one
of
these
are
they
circling
us?
C
A
C
D
C
C
C
I
mean
it
does
feel
like
though
we
even
I
can
consciously
imagine
the
feeling
I'm
going
to
get
when
I
move
my
fingertips
in
the
location
and
so
that
I
can,
like
you
generate
in
mind,
I
took
about
this
to
generate
a
non
silent
prediction.
There's
a
there's,
an
ability
to
say
on
is
the
feature
I'm
really
expecting
just
we
never
resolved
this
issue
that
there's
silent
predictions
lights
in
the
temple
memory,
which
I'm
sure
they're
going
on,
but
then
also
conscious.
A
Just
Answer
one
question:
you
said
the
capacity
would
might
fall
a
lot
when
you
had
only
one
module.
It
falls,
but
not
by
a
lot
capacity.
Does
not
this
capacity
skills
sub
linearly
with
number
of
modules,
because
it's
not
the
representation
representational
capacity
skills
exponentially,
but
the
union.
C
A
C
C
C
C
Model,
this
is
the
same
cell,
regardless
of
this
three-dimensional
thing,
but
we
shall
become
the
strength
of
it
could
vary
depending
on
so
imagine
now.
I
have
six
phase
clusters,
as
this
tank
showed
we
might
have
16
attractors
and
and
yet
they
all
behave
correctly,
but
more
active
than
others
at
any
time.
C
C
D
3D
the
wireframe
and
apparently,
as
you
move
around
it
is
completely
confusing
okay,
because
it's
you're
moving
around
in
40
and
we're
not
it's
similar
to
what
you're
going
to
show
here.
But
if
you
keep
doing
that
for
a
while,
apparently
it
just
clicks
and
you
get
it
and
then
suddenly
you,
you
have
a
completely
predictable
map
up
this
4d
hypercube.