►
From YouTube: Numenta On Intelligence Episode 14: Florian Fiebig
Description
In this episode, host Matt Taylor chats with Numenta Visiting Research Scientist Florian Fiebig. Florian is a recent graduate from the KTH Royal Institute of technology in Stockholm, Sweden with a PhD in computational neuroscience. His PhD thesis focuses on Hebbian learning networks and he regularly presents his work at Numenta research meetings[1]. Florian’s thesis is titled, “Active Memory Processing on Multiple Timescales in Simulated Critical Networks with Hebbian Plasticity.”[2]
Show Notes
• 1:05 Intro to Florian
• 2:41 Florian’s background and what led him to Numenta
• 3:06 Continuous learning
• 9:30 Does deep learning have anything similar to Hebbian learning?
• 11:36 Different types of plasticity in Hebbian learning
• 11:55 Long-term Potentiation (LTP)
• 14:38 “So it turns out: Short-term potentiation is not always short-term potentiation”
• 15:47 Two fast forms of plasticity: facilitation and augmentation
• 17:57 Homeostatic mechanisms: the Bobcat example
• 19:41 Let’s talk about working memory
• 21:21 Associative nature of memory
• 26:46 The brain as a massive filter
• 28:16 Episodic memory vs. semantic memory
• 30:38 Non-declarative memories
• 32:47 How does the transfer process of initially acquired memory into something that is longer lasting work?
• 35:05 The keys to remembering: repetition and relevance
• 37:28 Attractors and dynamical systems
• 44:15 The cortical attractor theory of neocortex or neocortical memory
• 45:26 The binding problem
• 55:00 Closing
Download the full transcript of the podcast here.[3]
Subscribe to Numenta On Intelligence: iTunes, Stitcher, Google Play, Spotify,
RSS
[1] https://numenta.com/blog/2019/07/31/the-livestream-experiment-update
[2] http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1263428&dswid=-4633
[3] http://numenta.flywheelstaging.com/assets/pdf/numenta-on-intelligence-podcast/NOI-episode-14-conversation-with-florian-fiebig.pdf
A
Florian
Phoebe
is
a
research
scientist
at
Numenta.
Recently
graduated
from
the
kth
Royal
Institute
of
Technology
in
Stockholm
Sweden,
with
a
PhD
in
computational.
Neuroscience
he's
been
sharing
with
us.
His
work
on
hebbian
memory
networks,
which
he
describes
in
his
PhD
thesis
linked
in
the
show
description
he's
recently
been
featured
in
our
research
live
streams.
Talking
about
synapses
and
plasticity.
You
can
find
upcoming
live
streams
featuring
florian
by
subscribing
to
the
Numenta
youtube
channel.
His
thesis
not
only
provides
new
memory
theory,
but
also
provides
a
general
introduction
to
memory
and
learning
in
cortex.
A
It
is
called
active
memory
processing
on
multiple
time
scales
and
simulated
cortical
networks
with
hebbian
plasticity.
I
won't
make
you
remember
that
you
can
find
a
link
for
it
in
the
show
notes
where
you
can
download
a
PDF
for
free
thanks
for
listening
to
than
Aminta
on
intelligence,
podcast
and
I
hope
you
enjoy
this
interview
with
a
neuroscientist.
A
I
have
been
spending
a
good
amount
of
time,
reading
your
PhD
thesis
and
the
three
papers
in
the
back.
A
lot
of
the
things
that
have
I'd
like
to
ask
you
about
come
from
this
thesis
for
those
listening.
It's
called
active
memory,
processing
on
multiple
time
scales
and
simulated
cortical
networks
with
heavy
and
plasticity.
That's
a
mouthful
in
the
show
notes,
so
you
guys
know
it's
freely
accessible
in
the
IDF
format.
It's
really
good!
Honestly.
A
A
B
The
good
thing
is
just
like
just
like
Jeff
I've
gone
through
this
experience
of
so
coming
from
a
different
field
before
and
then
falling
madly
in
love
with
neuroscience,
and
then
you
know
yeah,
like
squeaking
your
way
into
the
into
the
field,
and
so
I've
had
this
experience
for
like
one
year,
you
know
dedicated
trying
to
push
into
neuroscience
and
try
to
learn
about
no,
not
to
me
and
electrophysiology
and
whatnot.
You.
B
My
background
is
I'm
an
engineer
so
I
studied
at
the
Technical
University
humbug,
humbug,
general
engineering,
science
pretty
much
every
branch
of
engineering
that
is
just
because
I
couldn't
get
enough
of
it.
So
can
be
continuing
electric
engineering
process,
engineering,
mechanical
systems.
You
know
it's.
B
Little
bit
with
not
the
simplifications
per
se,
I
like
simplified
models,
but
the
refusal
to
look
at
the
brain
for
an
inspiration,
and
so
I
was
aware
of
Jeff
Hawkins
and
his
work
for
quite
some
time
and
eventually
figured
out.
But
if
I'm
gonna
do
this,
which
I'm
gonna
have
to
do
it
myself,
yeah
and
so
I,
sort
of
retrained
myself
like
on
the
side
in
neuroscience
and
started,
sneaking
into
neuroscience
lectures
and
asking
people
in
the
back
row
for
the
password
for
the
course
webpage
Wow.
B
B
B
No
sword
I
got
a
bunch
of,
like
you
know,
heavy
rocks
of
sorts
like
like
artistic
things.
One
of
them
was
actually
very
nice
as
like
this
inset
stone
into
a
glass
sculpture
of
a
head,
so
it
looks
like
you're
peering
inside
the
brain
and
forgot
the
name
of
the
artist,
but
it's
actually
quite
beautiful
and
you
get
that
after
student
I
still
should
do
like
a
proper.
B
Like
ceremony,
like
the
PhD,
you
know
graduation
ceremony,
there
was
one
now
that
I
missed,
of
course
yeah,
because
I
went
here,
but
you
know
they
don't
take
that
away
from
you.
You
can
let
go
whenever
and
get
a
ticket
and
technically
my
PhD
is
also
a
dual
degree,
so
I
get
one
from
the
University
of
Edinburgh
as
well,
and
then
you
get
killed.
So
that's
one
more
reason
to
show
up
to
get
myself
a
proper
killed,
excellent
well.
A
Speaking
of
your
your
doctorate
thesis
so
I
I
haven't
read
all
this,
but
I
read
enough
to
think
several
times.
This
guy
should
be
working
for
us
so
many
times,
I
was
like
that's
exactly
how
we
think
the
brain
should
work.
So
I
love
that
you're
here
and
let's
talk
about
some
of
this
heavy
and
learning
stuff,
so
we're
gonna
get
into
some
of
the
nitty-gritty
of
neuroscience
in
this.
This
is
podcast
about
neuroscience.
So,
let's
so,
let's
do
this.
A
B
But
the
point
is
that
fire
together
obviously
means
that
you
need
to
have
some
metrics,
some
measurement
by
which
you
can
observe
whether
they
are
actually
active
together.
And
so,
since
there
are
two
neurons,
you
always
have
what
is
called
the
pre
synapse
and
the
post
sign
apps.
So,
and
so
you
need
to
track
activity
on
two
sides
of
the
equation,
and
then
you
also
want
to
measure
how
many
times
they're
active
together
right.
So
you
want
some
sum.
B
B
How
closely
in
time
and
how
closely
together
in
intensity
and
like
different
learning
rules,
capture
different
aspects
of
that
right
and,
of
course
it
matters
a
lot
because
they
some
learning
rules,
have
very
tight
temporal
cones
like
STP
spike,
timing-dependent
plasticity,
kernels
tend
to
be
very
narrow
so
like
if
they're
spikes
on
the
pre
and
postsynaptic
site
don't
occur
within.
Like
you
know,
10
20,
milliseconds
and
I
won't
be
much
of
a
synaptic
change.
A
B
Experimentation
right
so,
like
you
have
a
you,
have
neurons
that
you've
grown
in
addition,
there's
a
synapse
between
them
and
now
you
want
to
investigate
how
that
china
signup
changes
with
with
activity
yeah,
and
so
then
we
do
protocols
that
are
a
lot
better
at
capturing
sort
of
rate
relationships
rather
than
the
individual
spike
time
right.
It's
that's
why
it
also
matters
to
understand
that
there's
so
many
different
forms
of
plasticity.
A
B
The
interesting
realization
is,
of
course,
it's
it's
not
just
a
zoo
in
terms
of
mathematical
models.
That's
abide
by
this
broad
idea
of
have
been
learning
that
there's
somehow
caption
correlations,
but
there's
also
a
zoo
by
molecular
mechanisms.
That's
poor
back
those
yeah,
those
ideas,
there's
many
forms
of
plasticity
have
been
learning
is
obviously
just
one
of
them,
but
there's
several,
you
know
possible
mechanisms
that
could
implement
something
like
that
in
the
brain
is.
A
B
But
in
fact,
there's
a
lot
of
messaging
between
them,
not
just
you
know,
pre
synapse,
sort
of
releasing
new
transmitter
that
then
gets
detected
and
leads
to
change
on
the
other
side,
but
there's
also
retrograde
signaling
and
so
there's
a
lot
of
communication,
but
it's
all
very
local,
which
is
a
small
volume.
So
you
don't
get
these
big,
full
network
kind
of
analysis
things
you
can't
reshape
the
whole
network.
You
can
only
do
it
from
the
smallest
element
up
and
see
what
emerges
when
you
change
the
local
rule
right
so.
B
And
that's
what
makes
it
a
little
bit
hard
to
wrap
your
head
around,
because
we
are
so
used
to
designing
systems
top-down
yeah,
more
than
bottom-up
yeah.
Absolutely
that's
why
all
so
many
of
these
things
require
you
know,
building
things
small
and
then
it's
testing
how
how
they
actually
behave
when
because
we
might
not
be
able
to
predict
it
a
priori,
sometimes
for
very
simple
Network
I
can
feel
doubt
because
you
can
predict
these
things.
B
Like
you,
you
know
what
the
distributions
of
weights,
for
example,
are
going
to
look
like
certain
conditions,
and
you
know
what
kind
of
you
know
the
dynamical
structures
can
get
formed
or
not.
But
as
soon
as
you
go
have
any
amount
of
biological
detail,
you
can
so
many
nonlinearities
and-
and
you
know
so
many
differential
equation,
yeah
that
interact
with
it
with
with
another
that
it
gets
impossible
to
predict
much.
So
that's
why
all
of
my
work
is
very
heavy
simulation
science
right
now.
A
B
Is
at
the
right
yeah,
so
long-term
potentiation,
it's
always
super
important
word
and
neuroscience
often
abbreviated,
LTP,
and
so
because
no
sciences
have
been
asking
this
question:
what
does
it
take
to
change
synapse?
You
know
in
a
lasting
way,
like
yeah.
How
do
we
learn
alright?
So
how
do
we
learn
things?
You
know
that
stick
with
us,
I
don't
know
just
to
buffer
the
last
three
words
that
were
just
said
right,
but
you
know
how
do
I
remember.
B
You
know
the
name
of
my
grandfather
or
something
yeah,
and
so
so
so
it
turns
out
there's
some
very
good
experimental
sort
of
findings
and
protocols
that
are
very
nice
and
predicting
how
to
potentiate
synapse
is
long-term.
The
problem
with
many
of
those
is
that
they
require
these.
You
know
long
tetiny
by
this
long
stimulus
bursts
rates
that
are
just
outrageous
for
a
for-real
brain
and
part.
B
Some
of
an
abeyance
of
the
namibian
and
in
part
my
thesis
makes
the
argument
that
particularly
heavy
and
ones
might
be
super
useful
for
expanding
things
like
working
memory,
and
so
it
feels
to
me
like
a
lot
of
the
research
are
still
you
know
reasonably
new,
because
it's
not
really
you
use
the
term
long
term
or
short
term
yeah.
But
the
problem
is
that
many
of
these
forms
of
plasticity
are
not
really
necessarily
time-dependent
and
more
activity
dependent
right
activity.
A
B
Out
you
can
do
get
like
a
short
term
potentiation
using
very
simple
paradigms,
but
if
there's
any
amount
of
ongoing
activity,
you
know
just
a
couple
of
X
or
spikes
that
potentiation
will
very
quickly
go
away,
whereas
if
you
then
silence
that
cell
culture,
you
cut
all
the
inputs
go
away
for
six
hours,
you
come
back
and
then
you
test
just
with
a
presynaptic
ping
right.
You
just
send
one
spike
through
and
they
take
what
comes
out
on
the
other
side.
Sign-Ups
is
still
strong,
so
it
turns
out
short
term.
B
A
B
Of
course,
these
chemical
cascades
can
be
complicated.
It's
not
really
just
you
know,
calcium,
but
it's
often
sort
of
the
initiating
mechanisms
all
for
calcium,
and
so
so
then
you
really
want
to
understand.
It's
not
really
time
dependent,
plasticity,
it's
more
like
calcium,
dependent,
plasticity,
yeah,
that's
more.
B
B
Right,
homeostatic
mechanisms
are
super
important.
They
keep
the
balance
on
many
of
these
systems
that
otherwise
would
run
away
to
weird
indefensible
states.
Yeah
and
I
mean
very
rarely.
Do
we
get
a
glimpse
into
that,
like
in
epilepsy,
when
you
have
runaway
excitation
these
waves
of
activity
that
just
wash
over
the
brain
and
active
every
every
neuron
I,
often
get
asked
by
friends?
Sort
of
you
know
what
about
this.
B
You
know
like
how
much
of
our
brain
is
active,
like
this
popular
miss
that
we
only
use
a
little
part
of
our
brain
yeah
and
then
I
tell
them
look
I've
seen
three
times
in
my
life,
I've
seen
people
with
a
hundred
percent
brain
activity.
It's
not
pretty
to
look
at
I.
Do
have
to
be
careful
that
these
people
don't
swallow
that
tongue
yeah.
A
Nothing
doing
anything
useful,
it
seems
you
know
I
always
think
about
this
I
used
to
do
you
know
what
a
bobcat
is.
It's
a
it's,
a
it's
a
small
tractor
right
and
it's
got
a
scoop,
so
it's
sort
of
like
they
call
it
a
backhoe
or
loader
in
the
Midwest,
but
the
controls
you
sit
in
it.
It's
just
like
one
seat
and
it's
like
a
small
car
right
and
it's
got
a
loader
in
the
front.
Usually
just
move
dirt
or
drill
holes
with.
A
A
Because
you're
on
tracks,
or
sometimes
it's
wheels,
but
so
there's
a
state
you
can
get
in
very
easily.
If
you
stop
too
abruptly
both
of
the
sticks
go
forward
and
then
you
go
forward,
move
back
and
then
you
go
forward
and
back
and
forward
fact
Warren
Buffett
and
the
only
way
you
can
release
from
that
is.
Let
go.
B
A
A
Talk
about
working
memory,
which
a
lot
of
your
thesis
is
about
working
memory,
short-term
memory
versus
long-term
memory,
so
I
found
this
fascinating
because
you
know
I've
been
thinking
a
lot
about
where
pathways.
What
pathways
you
know
the
different
streams
and
in
your
thesis,
you
talked
about
short-term
memory
and
is
it
the
medial,
prefrontal
cortex.
B
Yeah
I
mean
frontal
cortex
in
general,
like
this
is
involved
in
all
these
top-down
things,
but
particularly
for
sort
of
the
kind
of
semantic
decorative
working
memory-
maintenance
it
also
lateral,
prefrontal
cortex
has
been
described
to
be
important,
but
that
maps
onto
if
you
map
that
onto
rodents
that
they
don't
have
it
also
lateral
prefrontal
cortex.
So
there
you
talk
about
the
medial,
prefrontal
cortex
so
sometimes
gets
a
little
bit
tricky
when
you're
pooling
studies
from
different
animals.
A
A
B
We
do
this
all
the
time
like
it
turns
out.
We
have
some
of
our
working
memory.
Mechanisms
are
like
really
quite
specialized
for
certain
things
like
so
cognitive
scientists
have
long
described,
for
example,
is
this
auditory
phonetic?
You
I
think
it's
called
so
this
idea
that
you
can
even
without
much
understanding
you
can
loop
phonetic
sequence
of
some
sort
so,
rather
than
thinking
of
these
five
items
that
you
want
to
buy
and
like
visualizing
them
in
front
of
your
mind,
you
just
speak
the
words
right
or
you
build
yourself.
B
You
know
like
a
little
rhyme
or
something
oh
sure,
offering
things.
I
also
know
milk,
egg,
toast
and
I.
Don't
know
cheese,
butter,
toast
cheese,
but
I'm,
okay,
those
cheese
like
I
can
repeat
bottom
of
Actos
cheese
faster
than
I
can
even
think
of
this
yeah.
They
would
look
like
in
the
packaging
on
this
shelf
in
the
supermarket.
B
B
A
B
B
That's
typically
how
you
would
prescribe
it
in
today.
Note
worse
is,
though,
that
that
is
a
very
new
idea,
like
the
idea
that
long-term
I,
miss
associative
have
has
long
been
around,
has
been
shown
in
these
LTP
experiments
that
you
know
you
get
binding
of
Co
activated
units
and
if
you
anti
activate
them
in
an
anti
correlated
way,
that
was
actually
uncouples
like
this
nothing
deep
potentiation
of
synaptic
depression.
It's
called
led
instead
of
LTP,
that
is
long
been
known,
but
the
idea
for
working
memory
was
for
for
a
long
long
time.
B
There
would
be
some
kind
of
persistent
activity
signal
so
rather
than
a
system
that
buffers
these
items
and
some
synaptic
changes
why
people
would
think
there's
some
kind
of
reverberating
activity
that
self-perpetuate
somehow
it
mapped
very
nicely
onto
early
models
of
neural
networks
like
these
hopfield
networks,
which
had
these
beautiful
attractors.
So
they
can
very
once
you
let
them
once
you
kick
them
off.
They
can
be
good
for
Wow.
B
The
problem
is
just
that
you
need
dedicated,
attractors,
follow
these
items
and
you
can
think
of
a
billion
different
things,
and
if
you
get
to
the
permutate
of
complexity
of
all
of
these
things
you
can
think
of,
and
then
it's
vaguely
queer
where
it
quickly
becomes
clear
that
you
would
much
rather
have
a
scratch
board
of
some
sort.
Yeah
it
can,
you
know,
put
things
down
and
then
gets
quickly
gets
erased.
First,
you
don't
have
a
dedicated
network
for
all
the
working
memory
things
you
might
want
to
keep
shorting
on
your
mind,
yeah.
A
B
Yeah
and
a
fast,
healthy
and
mechanism
would
be
one
way
to
recruit
them
into
some
doing
something
useful
for
some
time
yeah,
but
because
it's
a
fast
mechanism,
it
also
means
they
can
be
relatively
quickly
overwritten
yeah,
but
it
might
just
be
the
temporal
bridge
that
you
need
in
order
to
kick
off.
You
know
more
lasting
that
may
change
this
order
just
to
perform
the
task
it
oftentimes.
We
don't
actually
need
the
long-term
memory.
B
B
Think
of
the
brain
is
like
this
massive
filter
right,
so
you
have
like
all
these
impressions
that
you
could
perceive
and
a
tiny
percentage
of
that
is
even
you
know,
perceptible
to
the
kinds
of
organs
that
we
have.
Yes,
the
only
see
certain
you
know
wavelengths,
and
we
only
hear
certain
frequency
bands,
okay
and
then
out
of
all
the
things
that
you
could
perceive
only
a
tiny
thing
passed
through
your
attentional
gate
so
and
so
you're,
aware
of
even
less
than
what
you
actually
perceive
yes,
I
mean.
B
You
know
the
last
sentence
that
I
just
used,
and
so
a
tiny
fraction
of
these
things
then
become
something
that
you
might
remember
tomorrow
and
of
all
the
things
you
remember
from
yesterday.
You
know
you're
gonna
forget
most
of
it.
It's
very
unlikely
that
you
know
much
of
that
will
be
left
after
a
decade.
Yeah.
A
B
A
B
Of
the
fascinating
things
we
have
this
episodic
memory,
where
we,
you
know
they
remember
the
exact
event
and
what
happened
and
this
person
walked
in
and
said,
yeah.
But
we
also
have
this.
You
know
much
more
semantic
memory
and
sort
of
things
that
stick
around
and
things
we
learned
to
sort
of
deep
connect
from
the
context
and
in
fact,
one
of
the
most
interesting
discoveries
that
I
made
early
on
sort
of
that.
What
I
thought
was
super
interesting
about
memory.
B
Research
was
the
fact
that
we
have
all
these
different
systems
and
they
don't
seem
to
be
connected
very
well
in
the
sense
that
you
can
have
one,
but
not
the
other,
like
you
can
like.
Knowing
how
to
ride
a
unicycle
right
does
not
mean
you
know,
might
you
you
might
have
a
memory
of
when
you
learned
how
to
ride
the
unicycle,
but
actually
they're
completely
different
systems,
and
you
can
yeah.
A
B
Could
teach
him
all
kinds
of
interesting
motor
tasks,
but
you
would
never
remember
any
any
of
it
in
the
episodic
sense.
Read
better
yeah,
you
could
train
him
to
be.
You
know,
happy
for
something
or
to
be
afraid
of
something
might
feel
conditioning
or
to
teach
a
motor
task,
like
you
know,
writing
backwards
in
a
mirror.
Things
like
that.
B
It's
hard
to
learn,
but
you
can
learn
it,
but
the
interesting
thing
is,
of
course
you
would
get
better
like
everybody
else,
because
he
had
these
systems
that
are
necessary
for
learning
these
things,
but
it
did
not
have
the
hippocampus,
which
is
you
know
this
super
important
bridge
into
a
long-term
of
the
episodic
consolidation
of
memories.
So
you
could
leave
the
room.
You
know
come
back
two
hours
later
and
you
would
not
even
know
that
he
had
done
that
task
yeah
and
actually
proved
its
performance
on
it.
Yeah.
That's.
B
B
B
A
B
A
B
B
B
A
B
And
one
of
the
fascinating
you
think
so
we
can
of
course
ask
in
that
I
liked
it
much
in
my
early
work
is:
how
do
these
transfer
processes
work
right?
How
does
something
go
from
this
sort
of
initially
acquired
memory
into
something
that
is
longer-lasting?
And
so
that's
where
all
these
interesting
particular
when
it
comes
to
like
spatial
memory,
but
also
just
you
know,
episodic
memory
in
some
sense.
There's
this
strong
involvement
of
hippocampus,
which
we
can
read
out
and
rodents
way
nicely.
B
You
write
and
you
get
all
this
interesting
evidence
of
like
this
strong
replay
during
sleep,
where
whole
episodes
get
compressed
into
like
these
short
effector
cycles.
Like
a
hundred,
you
know
seconds
shock,
wave
ripples,
so
he's
fast
oscillations
riding
on
a
slower
wave
and
they
compress
the
whole
movement
sequence
of
the
Rhoden
that
you
saw
before
you
know
like
behaving
for
20
seconds
or
something
compress
it
down.
B
It
is
this
very
short
burst,
with
the
exact
same
neurons,
activating
in
the
exact
same
sequence,
but
not
just
once
or
twice
hundreds
of
times
Wow
and
that
obviously
got
people
thinking
is
that
causally
related
to
their
memory,
and
then
people
started
meddling
with
it.
So
if
you
know
suppress
these,
so
you
can
have
a
microcontroller
listening
to
those
and
then
target
interrupts.
B
Big
argument
that
that
modelers
have
have
been
making
and
how
exactly
strong
this
is
sort
of
an
interventionist
sensor
that
you
can
prove
that
yeah
I
like
intermittently,
you
know
blocking
to
come.
You
know
Jake
modulation
of
plasticity,
for
example
yeah.
But
that
is
true
that,
like
the
presence
of
dopamine
at
like
these
synapses
can
can
make
the
synapses
a
lot
more
plastic,
meaning
they
are
stronger
and
the
apophysis
that
is
my
first
paper
makes
is
that
these
stronger
synapses
will
then
be
much
more
capable
of
reactivation
and
because.
A
B
Yeah,
and
so
in
that
sense
the
really
question
is,
but
if
you
want
to
remember
something
is
really
tube
to
two
plans
right
plan:
a
is
just
repeated,
a
lot
right,
make
it
sort
of
strong
by
repetition,
repetition
of
it.
That's
why
you
practice
your
you
know
vocabulary
not
just
once
or
twice
you
go
through
your
card,
see
no
idioms
and
that
increases
the
odds
that
they
will
get
consolidated
right,
because
it
doesn't
matter
that
you
know
now
what
analysis
that
you
know.
You
know
two
weeks
enough.
B
B
A
A
Any
I
mean
there's
there's
a
reason
for
any
performance
artist
or
athlete
that
when
they
perform
something
well,
it
feels
really
good
right.
Wait
because
you
get
immediate
feedback
from
night
before
you're,
paying
attention
to
the
music
you're
playing
or
the
ball
that
you're,
hitting
or
whatever,
and
when
everything
works,
it's
a
great
feeling
or
yeah
flow
States
not
beautifully
if
they
are
beautiful
things
serendipity.
A
Let's
talk
about
attractors
for
a
little
bit
right.
The
tractors
are
tricky
to
describe
so
not
coming
from
a
mathematics
background.
It
was
very
hard
for
me
to
understand
what
an
attractor
even
is
or
what
a
dynamical
system
even
is
I,
don't
know
if
you
can
help
explain
that
or
not,
because
it's
a
hard
thing
to
explain.
I
think
I.
B
Can
you
know
I
think
I
mean
put
like
you
know,
highly
simplified
right,
an
attractor
is
quite
simply,
if
you
have
a
system
with
a
lot
of
elements,
there's
obviously
the
the
space
in
which
that
system
can
be
in
terms
of
all
the
combinatorics
right.
All
the
different
elements
can
be
in
a
you
know
to
the
combined
span
a
wide
space
of
what
the
configuration
it
is.
You
call
it
a
high
dimensional
space
yeah,
it's
like
it
high
dimensional
space
right.
B
B
When
you
connect
these
nodes-
and
you
associate
some
things,
then
you
can
have
sub
structures
within
that
and
more
than
that
is
that
the
when
you
have
sort
of
some
number
of
connected
nodes,
let's
call
them
by
for
simplicity
sake,
then,
when
you
activate
just
some
of
them,
they
might
recruit
the
rest
of
that.
What
we
call
an
Osama,
oh
yeah,
yeah.
B
So
whenever
you
get
close
to
the
state
that
is
encoded
in
an
attractor
and
it's
that
particular
configuration
of
activation,
then
the
network
activity
will
be
attracted
to
that
correlation,
meaning
it's
gonna
change
through
the
interactions
of
the
elements
into
a
configuration
that
is,
that
attracted
state
right.
The
cool
thing
about
that
is
that
you
can
have
many
of
those
in
a
big
Network.
You
can
embed
tens,
hundreds,
thousands
of
attractors,
and
they
rely
on
connections
between
your
nodes
right
these
nodes
and
the
brain
would
be
neurons
right.
B
So
you
would
get
some
kind
of
a
population
code
where
different
configurations
of
active
neurons
together
represent
something.
And
then,
if
you
activate
just
a
tiny
piece
or
you
get
close
in
in
the
activity
space
to
one
of
the
encoded
attractors,
then
the
network
would
fall
into
those
attractors.
Very,
very
you
would
and
then,
as
many
researchers
to
us
that
is
well
I'll.
Give
you
a
cue
and
you
retrieve
the
memory
right.
That's
my
attractors
are
useful
memory
models
because
there
it
feels.
A
B
So
that
already
gets
a
little
bit
complicated
because
of
course,
when
there's
many
fuzzy
objects
that
might
mean
at
any
given
point
you
might
fall
bunch
of
different
attractors
right
and
then
becomes
relevant.
Well,
biasing
elements:
do
you
have
nice
or
maybe
also
you
know,
see
a
certain
color
or
certain
shape
or
something.
B
The
little
weird
thing
about
sort
of
like
these
dynamic
attractors
is
that
they
don't
just
fall
into
a
place
like
the
simplest
forms
of
attractor
networks.
Like
so-called
hopfield
net
works
right
yeah,
they
have
a
number
of
embedded
attractors
and
as
soon
as
they
enter
that
attractive,
the
network
is
stable
there.
You
know
it
will
not
change
over
time
right
in
neural
networks
and
in
the
currents
of
biophysically
detailed
spiking,
no
networks
that
I
have
built
attractors
dynamically
destabilize.
That
does
not
mean
that
the
delay
attractor
is
deleted
from
the
memory.
B
That
is
called
synaptic
depression,
and
so
you
can
use
those
two
elements
to
build:
a
system
that
can
fall
into
an
attractor,
your
activated
for
a
while
and
then
automatically
be
pushed
out
of
it,
because
the
resources
that
are
necessary
for
maintaining
that
item
inactive
non,
get
depleted
yeah,
which
then
means
the
network
is
free
again
to
hold
it
around
and
find
some
other
attractor.
Does.
B
B
No,
so
so
the
idea
is,
there's
only
one
thing
on
your
mind
at
any
given
moment
in
time.
You
can
jump
between
different
things
yeah,
but
there's
always
sort
of
a
foreground
of
activity
right
which
we
obviously
relate
to
from
the
conscious
experience
of
always
attending
to
something.
Yes,
there
can
be
external
right.
B
When
you're
thinking
of
your
own
memories
and
what
you
might
want
to
think
of
right-
and
so
it
turns
out
that,
even
though
the
brain
is
like
this
massively
parallel
system
with
all
these
neurons
that
are
active
in
any
given
time,
yeah
you're
still
playing
it
like
like
a
keyboard.
So
that's
an
only
one.
You
know
configuration
active
at
any
given
point
in
time,
even
though
it's
capable
of
many
different
ones,
and
some
of
them
are
built
to
transition
like
when
you're
learning
sequences,
yeah.
A
B
Series
of
three
tones
activates,
the
next
activation
next
activates,
the
next
sequence-
memory
right;
the
thing
that
the
HTM
normal
no
matter,
we
Lee,
excels
at
yeah.
These,
these
kinds
of
memories
are
also
a
form
of
attractor
because
the
activity
is
pulled
forward
into
something
plays
out
right.
It's
just
that.
Attractor
is
not
one
fixed
point
yeah,
because
that
point
immediately
evolves
into
something
else.
Yeah.
It's.
B
That's
like
essentially
called
the
cortical
attractor
theory
on
a
new
cortex
of
neocortical
memory,
the
idea
being
that
you
have
these
distributed
representations
or
specific
brain
states
that
actually
get
it
coated.
Well,
so
there's
strong
connections
between
them
right
and
they
are
competing
for
activation
at
any
given
moment
in
time.
A
B
So
there's
a
lot
of
in
the
one
of
the
problems
of
of
my
work
right
and
the
the
field
that
I
work
in
is
that
it's
very
it's
very
interdisciplinary
right.
So
there's
ideas
from
cognitive
science,
where
the
binding
problem
comes
from,
which
I
get
to
in
a
second
there's,
obviously,
a
evidence
on
all
kinds
of
things
and
details
that
constrain
models.
When
you
built
a
MIDI,
Ted
wells
and
then
there's
the
whole
computational
side.
B
How
do
you
build
these
systems
and
neural
network
simulators
and
how
you
put
these
things
together,
yeah
and,
as
a
consequence,
my
work
tries
to
speak
to
all
of
these
different
groups
and
I
want
to
build
systems
that
are
reasonably
defensible,
but
then
also
do
something
useful
computationally
and
that
also,
you
know
sort
of
demonstrate
what
we
can
do
in
terms
of
brain
simulation.
Yeah.
A
B
Interesting,
as
has
some
utility
all
right,
and
so
the
the
the
one
of
the
kinds
of
of
binding
problems
is
this.
What's
called
roll
filling,
so
I
think
it
uses.
Example
in
that
paper
of
telling
you
that
the
the
the
name
of
my
parrot
may
be
Charlie
Nylund,
so
you
know,
charlie
normally
has
a
name
of
somebody
right.
You
might
have
a
friend
called
Charlie,
it's
just
a
label
right.
B
Now
tell
you
my
doctor
telling
you
that
the
name
of
my
pedestal,
you
won't
be
that
you
can
tell
me,
in
fact
it
won't
be
surprised
if
I
tell
you
that
you
know
Charlie
can
fly
and
you
might
even
be
able
to
tell
me
now
that
Charlie
can
probably
speak
because
you've
now
connected
these
elements.
Yeah
and
the
interesting
thing
is
you
don't
have
a
long-term
memory
representation
of
that
yeah
I
guess.
A
B
Literally,
just
learned
that
and
the
kinds
of
long-term
memory
that
we're
talking
about
with
LTP
that
takes
at
the
very
least
hours
right
to
express.
So
you
cannot
have
brain
structure
that
represents
a
flying.
Pirate
called
Charlie
all
right,
but
you're
still
capable
of
making
that
association
very
quickly.
All
right.
B
Tell
what
college
you
just
see
well,
who
say
it's
a
spades!
Okay
cool
so
turns
out
that
associative
binding
can
be
very
quick,
and
then
you
really
have
to
ask.
Where
does
that
binding
take
place
and
one
of
the
ways
that
people
used
to
think
about
that
in
terms
of
persistent
activity?
Is
that
all
of
these
things
get
activated
together?
So
they're
named
Charlie
right
without
your
friend
that
has
that
name
and
the
idea
language.
A
B
There's
all
these
wonderful
studies
about
activity,
silent
working
memory,
yeah
have
come
out
in
in
recent
years
of
people,
who've
been
looking
for
Huntly
for
the
signatures
and
finding
them.
So
you
can
read
out
the
content
of
the
working
memory,
but
there's
the
substantial
period
when
you
distract
people
or
when
people
have
to
shift
their
attention
to
multiple
items
that
have
to
hold
on
to
where
the
entire
activity
signature
of
that
working
memory
item
gets
lost,
you
can
no
longer
detect
it.
Yes,
as
soon
as
you
provide
a
clue,
it's
back
again,
yeah.
B
B
And
the
only
place
in
the
brain
where
information
can
go,
that
it's
not
sort
of
active.
You
know
in
the
neural
activity
well
in
the
changes
in
the
in
between
the
neurons,
the
silences
yeah.
So
there
must
be
very
fast
right
because
again,
I
flesh
it
your
card
put
it
down.
You
already
know
what
it
was
yeah.
B
There
must
be
very
fast
buffers
that
might
be
activity
dependent
and
reasonably
fast,
non
activity,
dependent
synaptic
changes
and,
if
they're
going
to
be
associative,
they're
likely
heavy
in
and
some
the
form
yeah
and
the
cool
thing
about
those
very
fast
forms
of
having
plasticity
as
a
mechanism
to
solve
the
spiny
problems.
To
put
things
together,
yeah
right
is
that
you
don't
have
to
encode
the
for
context.
You
already
know
what
a
parrot
is.
A
B
B
Somehow
need
to
put
them
together,
yeah
and
the
brain
can
build.
You
know
like
these
powerful
associations
across
neocortex,
but
again,
if
it's
long
term
structures
like
long
term
memory,
that's
going
to
take
time
and
if
we
buy
the
story
of
long-term
memory,
consultation
during
deep
sleep
in
the
campus
that
might
takes
hundreds
of
reactivations
doing
deep
sleep,
so
you
need
some
other
mechanisms,
while
you
awaken
behaving
to
do
to
solve
this
binding
problem
yeah,
and
so
the
solution
that
we've
come
up
with.
B
Is
this
idea
that
prefrontal
cortex
does
not
actually
hold
the
full
content
of
working
memory?
All
that
it
needs
to
do
is
to
hold
the
temporary
short-term
index
with
which
those
things
can
be
linked
and
I've
shown
in
my
work
that
the
connectivity
that
is
required
for
that
can
actually
be
very
low.
You
don't
need
a
lot
of
silences
yeah,
particularly
when
the
items
that
you
want
to
bind
together
very
strongly
encoded
yeah.
So
you
already
know
that
geometric
shape-
and
you
already
know
that
you
know
phonetic,
that
that
word
or
that
sound
right.
B
B
A
A
B
I
do
this
example
right
where
I
have
have
a
very
brief
cue,
which
activates
one
long-term
representation
through
sort
of
sensory
input.
Then
that
thing
is
activates
an
index
and
prefrontal
cortex
and
retrieves
an
associative
long-term
memory
in
a
brain
area
that
might
not,
you
know,
might
be
far
away
like
we're
talking
centimeters
away
right
so
where
that's
very
unlikely
to
be
direct
connections
right
between
the
things
you're
wanting
to
associate.
But
we
know
that
the
working
memory
is
reasonably
universal.
You
can
of
all
the
things
you
can
have
an
octa
memory.
B
B
It's
sort
of
position
is
a
little
bit
like
that
right,
because
if
you
want
to
observe
it,
if
you
test
the
silence
a
lot
by
a
lot
of
pins
to
see
what
its
strength
is
then
you're
eroding
whatever
you
know,
it's
a
coded
state
was
interesting.
So
silence
is
like
you
know
the
way
that
working
memory
might
get
preserved,
not
activity,
interesting.
A
B
People
are
fighting
over
the
nomenclature
for
this
fight,
so
what
is
STP
ShoreTel
plasticity?
Even
some
people
are
now
arguing.
Well,
if
you
know
it
might
be
STS
P,
because
you
want
to
differentiate,
you
know
different
classes
of
that,
and
then
some
people
call
it
label
long-term
potentiation,
because
it's
lasting,
but
it's
fragile
yeah
of
new
times,
and
it's
a
little
bit
like
the
nomenclature
on
inhibitory
neurons.
B
B
A
Absolutely
well
Florian,
it
was
a
pleasure.
I
should
remind
Watchers
or
listeners
that
if
you
want
to
hear
more
from
Florian,
you
can
go
to
our
YouTube
channel
than
the
mentor
youtube
channel
where
we
have
live
research
meetings
fluorines
given
a
couple
of
those
recently,
as
you
mentioned
earlier,
and
they'll
probably
be
more
a
little
gonna
be
more
yeah
yeah,
so
you
can
get
more
Florrie
in
there
thanks
again,
fist
bump
we're
doing
all
right
thanks
for
watching
everybody.
Thank
you.