►
Description
Guest speakers Johannes Leugering and Pascal Nieters talk about their work on neural computation with dendritic plateau potentials. Johannes first frames the problem of sequence processing and makes the case that a neural model based on active dendrites and dendritic plateau potentials would help solve the problem. Pascal then explains their recent work on the computations in a neural model with segmented dendrites and one with stochastic synapses. He concludes the presentation by discussing the implications of this model. The team asks questions and discusses.
Preprint v4: https://www.biorxiv.org/content/10.1101/690792v4.abstract
Paper at NICE workshop: https://dl.acm.org/doi/10.1145/3381755.3381763
Presentation at NICE workshop: https://www.youtube.com/watch?v=qLaq1m0xVuQ
Presentation at the Computational Cognition Workshop: https://www.youtube.com/watch?v=kVyY776m1PM
Other papers mentioned:
“Functional clustering of dendritic activity during decision-making”: https://elifesciences.org/articles/46966
"Embedded ensemble encoding hypothesis: The role of the “Prepared” cell": https://onlinelibrary.wiley.com/doi/full/10.1002/jnr.24240
"Local glutamate-mediated dendritic plateau potentials change the state of the cortical pyramidal neuron": https://journals.physiology.org/doi/abs/10.1152/jn.00734.2019
"Compartmentalized dendritic plasticity and input feature storage in neurons": https://www.nature.com/articles/nature06725
"Functional clustering of dendritic activity during decision-making": https://elifesciences.org/articles/46966
"Data-driven reduction of dendritic morphologies with preserved dendro-somatic responses": https://elifesciences.org/articles/60936
0:00 Johannes Leugering
30:30 Pascal Neiters
55:25 Q&A
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a framework called the Thousand Brains Theory of Intelligence that will be fundamental to advancing the state of artificial intelligence and machine learning. By applying this theory to existing deep learning systems, we are addressing today’s bottlenecks while enabling tomorrow’s applications.
Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://numenta.com/news-digest/
Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter
Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta
Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/
Our Website:
https://numenta.com/
B
C
D
D
I
think
for
the
last
three
years
more
or
less
by
now,
so
it
started
when
we
were
both
in
oceanwork
university
and
I
have
since
moved
to
phonofy
institute
for
integrated
circuits
in
the
south
of
germany.
But
yeah
topic
is
very
interesting,
so
we
are
still
working
on
this
three
years
later,
and
so
we
prepared
the
slides
to
take
roughly
half
an
hour.
So
there
should
be
plenty
of
time
for
you
to
interrupt.
Ask
questions.
D
The
word
is
the
order
in
which
these
three
syllables
are
set
for
these
three
phonemes,
the
precise
timing,
however,
matters
less
probably
yeah
there,
there
might
be
deviations
between
different
speakers
and
also
I
personally
when
I
say
the
same
word
three
times.
I
might
have
slightly
different
timings
between
all
of
these,
but
still
we
can
reliably
detect
this
and
it
seems
to
be
a
very
general
thing
that
well
animals
and
humans
as
animals.
We
are
able
to
do,
and
it's
not
just
the
thing
with
auditory
processing.
D
The
same
sort
of
problem
also
happens
when
you
look
at
navigation
tasks.
So
if
you
think
about
the
environment
of
an
animal
that
is
tiled
by
play
cell
populations,
if
an
animal
moves
through
this
environment,
it
also
will
sequentially
activate
different
populations
of
these
place
cells,
and
that
will
lead
to
a
very
distinct
pattern
of
activation
over
time
that
will
encode
in
the
order
in
which
these
are
activated.
D
Then
we
detected
a
subsequence
of
the
sequence
that
we're
interested
in,
and
that
starts
another
time
frame,
and
now
we
are
looking
for
the
next
event
that
might
occur,
but
it
has
to
occur
in
this
time
frame.
So
in
this
case
imagine
the
third
third
event,
in
this
case
the
third
phoneme,
for
example,
or
the
third
receptive
field
that
we
entered
here.
D
That
has
to
happen
in
the
time
frame
given
by
the
second
event
that
has
happened,
and
only
if
all
three
of
them
happen
in
the
right
succession,
we
would
say
it
was
the
full
sequence
abc
that
we
are
looking
at
and
from
a
computer
science
perspective.
This
is
a
this
is
a
solvable
problem
that,
for
example,
the
finite
state
automaton
could
solve.
D
So
imagine
we
have
some
initial
state,
it's
a
starting
state,
and
then
there
is
some
event
that
happens
this
event
a
and
then
this
will
turn
the
finite
state
automaton
into
another
state,
into
a
waiting
state
where
it's
now
waiting
for
the
input
b
and
if
nothing
happens,
then,
after
some
time,
this
final
state
machine
is
going
to
time
out
and
it's
going
to
revert
back
to
the
initial
state
because
nothing
happened.
It
just
randomly.
D
Someone
said
power,
but
it's
it's
not
the
full
sequence
that
we're
looking
for,
but
while
we're
in
this
waiting
state,
if
now
the
next
event
happens
this
b
event,
then
we
go
into
the
next
waiting
state.
And
now,
if
we
finally
hear
the
event
c
yeah,
then
we
hit
the
complete
sequence
and
at
this
point
the
sequence
detector
would
then
fire
would
say:
okay,
we
detected
the
sequence
abc
and
and
that's
what
we're
after
right
and
then
it
would
fire
and
revert
back
to
the
initial
stage.
D
D
This
is
a
problem
that
we
got
interested
in,
and
so
we
looked
into
the
literature
for
quite
some
time
to
figure
out
how
this
had
been
discussed
before,
and
we
found
there's
a
lot
of
research
on
this
problem
in
general
because
it
occurs
in
many
different
contexts.
So
one
of
the
most
obvious
things
is
in
audio
processing
or
language
understanding.
There
are
these
different
time
scales
in
which
we
need
to
integrate
information.
D
On
the
same
time
scales,
it
suggests
that
yeah
there's
really
seem
to
be
biological
mechanisms
operating
on
these
different
time
scales
and
if
you
do
psychophysics
experiments
with
people
trying
to
tease
apart
what
is
and
isn't
important
for
people
to
be
able
to
detect
these
sequences.
These
patterns
is
words.
It
turns
out
that,
yes,
as
we
might
expect
the
order
in
which
these
events
occur
is
very
important
for
perception.
D
D
D
Invariant
and
people
have
studied
this
and
studied
different
modalities,
and
it
seems
that
there's
kind
of
like
a
theme's
recent
evidence,
converging
that
the
hippocampus
might
not
just
be
involved
in
the
sort
of
task
for
navigation,
but
also
for
other
modalities
that
it
might
actually
be
a
bit
more
of
a
general
principle
that
there
is
sequence,
processing
or
processing
of
sequential
information
happening
there.
And
I,
like
the
summary
here
from
for
the
role
that
this,
these
sequences
play
for
for
processing
in
the
in
the
hippocampus.
D
So
they
say
the
neural
sequences
play
a
key
role
in
information
processing
and
support
the
formation
of
initial
memory
traces.
So
for
the
hippocampus
as
well
as
for
audio
processing,
the
sequences
seem
to
be
quite
important
and
there's
a
lot
of
other
examples.
I
mean
I
don't
want
to
go
through
all
of
them
and
the
slide
is
already
like
a
lot
of
like
in
your
face.
D
I
think
they
are
much
more
critical
or
they
seem
to
play
a
much
more
central
role
for
information
processing
than
one
might
have
thought
some
years
ago,
because
it
turns
out
that
there
are
many
more
of
these
actively
generated
events.
Then
there
are
somatic
spikes,
so
these
plateau
potentials,
for
example,
integrity
spikes.
D
These
active
processes
can
disrupt
a
lot
of
behaviors
and
it
can
also
disrupt
learning
and
other
fundamental
properties
of
of
this
neural
network.
So
in
forward
this
evidence
yeah
all
of
these
all
of
this
research
for
us
implies
that
there
is
a
lot
of
support
that,
first
of
all,
the
sequence
processing.
This
detection
of
sequences
is
a
really
important
problem
that
needs
to
be
solved
and
biology
with
these
active
dendrites
might
have
the
tools
that
we
need
to.
D
So
imagine
this
event
a
here
just
triggers
on
some
population
of
recurrently
connected
neurons,
a
yeah
and
this
population
can
just
turn
on
for
some
period
of
time.
So
it
basically
just
implements
the
first
part
of
this
finite
state,
automaton
that
I
showed
in
the
beginning
and
if
we
know
the
second
population
of
neurons
that
receives
input
from
this
first
one,
then
we
can
set
it
up
such
that
just
input
from
the
first
population.
D
Here
is
not
enough
to
make
this
population
turn
on,
but
if
we
additionally
get
the
input
that
we
are
waiting
for,
that
would
be
enough
to
turn
this
return
population
on.
Then
it
will
stay
on
for
some
period
of
time,
and
we
can
repeat
this
game
now
by
adding
another
neuron.
This
neuron
is
now
waiting
for
input
from
this
population,
as
well
as
this
external
input,
and
only
when
it
receives
both
of
them.
The
neuron
will
fire.
D
So
this
way
we
build
now
a
kind
of
recurrence
left
feed
forward
network
here
that
would
be
able
to
solve
the
task,
and
we,
of
course
we
could
simplify
this.
We
could
just
say
as
just
a
recurrent
network
that
could
be
able
to
solve
it,
and
we
could
also
train
a
network
like
this
yeah
and
it
would
be
able
to
solve
the
task.
D
And
if
we
do
this
for
all
of
the
inputs
in
the
sequence,
we
can
tune
it
so
that
for
all
of
these
different
inputs,
they
will
all
arrive
at
the
soma
at
exactly
the
same
time.
And
then
the
neuron
only
needs
to
be
some
coincidence.
Detector
to
see.
Okay,
like
we
got
these
three
inputs
here
at
the
right
times,
so
we
detected
the
sequence,
but
this
also
has
a
problem
right.
So
so
this
sort
of
solution
would
mean
you
can
detect
one
very
specific
sequence,
but
suddenly
the
timing
invariance
is
gone.
D
So
a
sequence
that
is
200
milliseconds
long
would
be
very
difficult
to
build
with
the
neuron
like
this,
so
passive
dendrite
dynamics
and
the
transmission
delays,
while
in
theory
they
could
work
for
us.
They
didn't
seem
like
a
very
convincing
solution
for
for
that
sort
of
problem,
and
that's
why-
and
this
is
now
after
a
long
excursion,
we
kind
of
get
to
the
the
point
that
we
are
most
interested
in,
and
that
is
the
active
dendritic
processing
and
I
think
a
simple
solution
to
this
problem.
Using
active
dendrites
looks
like
this.
D
D
But
since
the
dendrite
segments
that
are
neighboring
each
other
electrically,
coupled
the
higher
membrane
potential
will
not
stay
here,
it
will
diffuse
through
the
through
the
neuron
and
the
way
we
model
it
is
by
just
saying
okay.
So
if
this
dendrite
segment
is
active,
it
will
have
a
weakly
depolarizing
effect
on
the
next
segment.
D
So
it
will
make
this
segment
more
likely
to
fire
when
it
receives
no
synaptic
input,
and
so
that
means
that
in
this
time
window,
if
we
now
have
the
synaptic
input,
the
synaptic
input
will
trigger
this
dendrite
segment
on
and
we
can
play
the
same
game
again
now.
This
dendrite
segment
is,
of
course,
electrically
coupled
in
both
directions,
and
so
it
will
depolarize
now
the
zoma
a
little
bit
and
it
will
fully
depolarize
the
upstream
dendrites
is
there's
some
asymmetry
in
the
production,
but
that's
maybe
not
so
so
interesting
now.
D
The
interesting
part,
I
think,
is
that
if
this
dendrite
segment
now
depolarizes
the
sama,
then
the
soma
becomes
receptive
to
the
next
input
c
here
and
when
this
arrives,
it
could
trigger
a
spike
at
the
summer.
But
if
this
would
arrive
without
any
of
these
other
inputs
arriving
prior
to
the
it
would
not
be
enough
to
make
the
neuron
fire.
A
Yeah
quick
question:
I
mean
with
the
way
the
plateau
potential
stuff
works
that
even
the
event
a
would
depolarize
the
soma
for
several
hundred
milliseconds
up
to
10
to
20
minutes
like
it
doesn't
need
to
go
through.
You
don't
need
to
have
the
depolarization
at
b.
Just
a
itself
is
enough
to
depolarize
the
solar.
D
Yeah,
that's
I
I
would
say
this
is
a
matter
of
debate.
I
would
say
so.
It
depends
where
you
depolarize
the
soma.
So
sorry
where
you
depolarize
the
dendrite,
so
our
assumption
was,
if
you
look
at
the
so
once
you
depolarize
a
part
of
the
dendrite,
and
you
look
at
the
passive
so
now
we're
looking
at
the
passive
spread
of
the
membrane
potential
that
comes
from
this
one
plateau
potential
and
the.
If
you
look
at
the
how
quickly
this
potential
should
decay.
D
C
Because
of
the
there's,
a
dendritic
spike,
an
nmda
spike
which
will
travel
to
the
soma
not
actively,
though
it
does
actively
propagate
it's,
not
as
it's
not
as
reliable
as
a
sodium
action
potential.
But
it's
pretty
reliable
and
there's
some
debate
about
that.
But
under
many
conditions
it
does
reach
the
summer.
A
B
There's
also
some
work
losing
a
little
bit
sight
of
all
the
citations
now,
which
group
exactly
was,
but
they
did
exactly
this
sort
of
it's
2019.
It's
an
e-life
paper.
Definitely,
how
much
is
each
segment
or
how
much
are
segments
of
a
dendrite
electrically
coupled
towards
each
other,
and
they
sort
of
probe
the
dendrite
in
that
way,
and
they
do
actually
find
quite
a
substantial
compartmentalization
depending
on
the
neuron
they
use
this
on
I
mean
this
modeling
work
mostly.
C
That
makes
sense
but-
and
I
think
we'd
agree
with
that-
but
yeah
the
whole
point
of
that
that
is
sort
of
these
passive
electrical
properties
yeah,
but
but
an
nmda
spike.
It
transcends
that
and
and
actively
travels
along
the
dendrites
and
reaches
the
soma
not
again
there's
some
debate
about
under
what
conditions,
but
in
many
conditions
that
occurs
so
again.
It's
whether
you're
talking
about
passive
electrical
properties
or
active
propagation.
B
Well,
I
I
guess
I
guess
our
reference
points
are
is
there's.
I
think
it's
a
thruster
review
or
something
where,
specifically
in
there,
that
platoons
are
localized
events
and
you
do
find
measurements
of
calcium.
So
this
is
this
new
sort
of
two-photon
microscopy
where
they
image
the
calcium
densities
along
the
dendrite,
and
you
can
see
the
calcium
on
active
dendritic
events.
This
is
one
of
the
papers
johanna
cited
earlier.
B
D
There's
also
other
studies
that
talk
about
like
how
the
active
processes
can
facilitate
or
can
be
facilitated
by
activity
more
downstream
and
so
on.
So
it's
like,
I
I
think,
there's
a.
I
would
concede
that
there
is
a
conflicting
amount
of
evidence.
I
would
say
so
I
I
don't
think
it's.
I
mean,
okay,
if
you're
saying
that
for
you
it's
a
conclusive
thing
that
the
dendritic
spikes
will
always
depolarize
the
neuron
at
the
sama.
D
B
D
A
I
think
one
of
the
areas
to
be
careful
about
here
is
that
if
you
look
at
the
apical
dendrites,
those
dendritic
things
do
not
travel
to
the
soma
unless
there
is
a
major
calcium
event,
but
the
basal
dendritic
segments
those
those
there.
When
you
have
active
dendritic
events,
they
do
travel
to
the
soma
for
a
sustained
depolarization.
D
Yeah,
I
think
we
were
only
looking
at
ethical.
I
mean
the
thing
is
the
mechanism
that
we
are
talking
about
here.
So
if
you,
if
you
suspend
disbelief
for
a
moment-
and
you
just
think
about
how
this
coupled
segments
would
interact
this,
the
similar
mechanism
would
be
possible
in
different
places.
So
it
might
not
even
just
be
a
middle
neurons.
They
also
have
similar
like
actively
generated
events
locally
and
other
types
of
neurons
as
well.
D
A
D
Yeah,
okay,
then
I
will
say
for
now:
we
just
we
continue
as
if
we
all
agreed
that
there
are
these
localized
events
right
and
electrically
coupled
compartments,
where
there
is
a
neighborhood
interaction.
That
would
happen
also
just
through
the
passive
spread
of
membrane
potential,
from
from
one
segment
to
the
next
right.
D
So
if
that's,
if
that
happens,
then
you
would
have
a
mechanism
like
this,
where
you
would
actually
need
a
couple
of
different
events,
so
it
would
not
be
sufficient
to
just
have
so
in
this
in
the
village
that
you
were
saying
now,
where
any
actively
then
generated
event
in
the
dendrites
would
have
a
depolarizing
effect
on
the
soma,
then
you
can
basically
ignore
this
metal
b
and
then
you
directly
go
from
a
to
c.
Then
the
sequences
become
a
lot
simpler.
D
You
basically
just
have
the
you,
have
a
plateau
potential
which
induces
an
upstate
in
the
stoma
and
makes
the
neuron
receptive
to
a
somatic
input
yeah.
Then,
then,
we
are
looking
at
only
sequences,
a
b,
which
is,
I
think,
more
similar
to
other
modes.
I
think
it's
also
more
similar
to
the
hdm
model,
and
so
I
think
this
is
where
the
connection
then
comes
in,
but
now
we
will
just
go
all
in
on
this
locally
coupled
standard
segments.
C
And
I
might
have
misunderstood
what
you
just
said,
though
I
didn't
understand
what
you
said
there.
So
one
of
the
things
I
was
thinking
about
is
like
okay,
when
you
think
about
the
model.
This
way
it
seems
like
it's
limited
to
the
to
the
the
length
of
the
sequence,
if
you're,
if
you're
relying
on
dendritic
segments
in
sequence-
I
don't
know
if
you
just
addressed
that
through
comma,
because
I
didn't
understand
it,
but
is
that
a
limitation
here
or
not.
C
Like
the
main
sequences
can
last,
you
know
very
very
long
if
you
think
about
a
melody
predicting
the
next
element,
the
melody
it
can
be,
quite
quite
long
or
or
a
motor
sequence
like
like
a
very
complex
motor
sequence,
you
might
perform.
So
when
we,
when
we
thought
about
this
problem,
we
always
we
said
to
ourselves.
Well,
there
has
to
be
almost
unlimited
length
to
sequences,
because
they've
been
quite
long-
and
I
just
understand
is-
is
by
assuming
this
dendritic
properties.
C
D
I
would
say
that
that
happens
on
the
population
level.
Like
you
connect
neurons
together,
so
you
can
out
of
small
sequences.
You
can
build
longer
sequences.
I
think
that
is
the
general
principle
that
you
always
have
to
do,
but
on
the
small
scale.
So
here
we
are,
if
you
think,
if
you
think
about
this-
like
it's
a
very
simplistic
view
here,
but
if
you
think
about
like
each
segment
has
a
time
window
of
100
milliseconds
and
you
you
stack
a
chain
of
like
three
five
six.
D
B
But
there
is
an
interesting
thing
about
platoons.
I'm
sure
you're
also
aware
that
they
can
be
prolonged
at
least
there's
some
evidence
for
it
that
repeated
activations
can
keep
dendritic
segments
active,
but
still
that
doesn't
get
us
to
the
long
sequences
yeah.
I
think
I'll
take
over
now
janus
correct.
B
So
the
should
we
should
work
right
now.
Wait
give
me
a
second
okay,
so
now
to
sort
of
the
the
meat
of
our
model
here,
and
indeed
we
did
not
concentrate
as
much
yet
on
networks
and
longer
sequences,
etc.
But
we
really
drilled
down
onto
into
this
sort
of
what
you
can
do
with
dendritic
trees.
If
the
thing
you're
working
with
is
compartmentalized
dendritic
segments
that
interact
with
each
other
on
sort
of
a
plateau
time
scale,
and
for
that,
let's
just
imagine
a
slightly
more
complicated
neuron
right.
B
So
we
now
have
three
more
distal
branches
here
and
let's
go
through
a
quick
sequence
to
see
what
happens
in
our
assumption,
so
a
would
activate
segment.
A
b
would
activate
x,
second
b
and
so
on.
All
these
three
segments,
the
platos
on
which
wouldn't
interact
that
much,
but
then
what
they
would
do
is
they
would
elevate
the
voltage
in
c
and
make
it
more
likely.
B
That
c
also
emits
a
plateau
and
for
the
purposes
of
this
presentation
we
will
always
assume
strict
gating,
which
is
of
course
too
strong
right,
a
very
heavy
synaptic
spy.
A
synaptic
input
at
c
would
activate
a
plateau,
no
matter
what,
but
with
elevated
voltage
and
mba
spikes
are
more
likely.
Our
negate
channels
are
more
likely
to
displace
magnesium
blocks
and
initiate
platoons.
B
B
We
don't
need
b
and
d
and
then
c
would
activate
e
and
so
on,
and
we
have
sort
of
the
sequential
overlapping
of
platoons
and
what
this,
what
we
can
do
with
this
is.
We
can
have
our
little
sort
of
steel
a
little
bit
from
from
logic
in
our
notation
and
say
what
this
neuron
in
this
way
would
do.
It
would
recognize
a
sequence,
a
or
b
or
d
and
then
c,
and
then
e
right
and
the
sort
of
key
first
functionality
here
is
that
you
sort
of
rank
order
by
overlapping
the
plateaus.
B
B
B
Lastly,
the
the
big
structural
thing
we
can
do
with
playing
around
with
this
model
is:
we
can
just
move
the
segments
a
little
bit
so
now
say
we
have
two
segments
connected
to
the
soma.
Let's
imagine
once
the
bazel
segment
wants
a
trunk
was
the
apical
dendrite
and
then
we
have
two
segments
in
the
apical
part
of
the
dendritic
tree,
for
example,.
B
And
in
this
case,
we
again
go
through
the
same
sequence:
a
b,
nothing
much
happens
c
doesn't
get
any
input,
but
when
d
gets
input
and
activates
the
plateau,
it's
directly
connected
to
the
soma,
which
is
why,
in
this
case
at
this
symbol,
e
would
actually
be
accepted
and
we'll
get
an
output
and
the
same
and
for
the
second
sequence,
a
c
e.
Nothing
much
would
change
still
this
dendritic
tree
computes
a
different
function
over
these
ordered
inputs
now,
and
we
sort
of
have
to
represent
this
in
our
little
formula.
D
We
didn't
jump
in
to
relate
this
to
the
point
that
jeff
made
before
so.
Maybe
at
this
point
you
can
then
see
why
why
I
was
saying
that
I
think
actually
having
this
dependency
might
make
it
more
powerful,
because
in
this
sort
of
structure,
the
fact
that
each
dendrite,
the
each
dendrite
segment
would
depend
on
the
activity
of
the
upstream
dendrite
segments.
It's
connected
to
would
allow
us
to
make
these
a
bit
more
complex
expressions.
I
mean
this
neuron
might
be
a
bit
weird.
C
Well,
yeah,
but
but
you
know,
if
you're
familiar
with
our
work,
then
you
can
see
how
at
the
population
level
that
that's
not
an
issue,
but
we
can
get
back
to
that
later.
But
so
this
is
flexible.
But
then
you
have
to
introduce
your
inhibitory
neuron
to
basically
make
this.
So
it's
not
a
general
computational
element,
but
it's
a
sequence.
Memory
of
a
specific
sequences,
because
again
here
a
and
b
are
equivalent
so
you're
going
to
talk
more
about
the
inhibitory
neurons
and
how
they
decide
when
to
become
active.
B
B
C
C
C
But
again,
to
get
the
specific
to
get
the
specific
sequence:
you'd
have
to
basically
concatenate
the
different
dendrites
in
order.
There
is
a
fairly
strict
limit
on
the
branching
structure
of
dendritic
trees.
Yes,
they
don't
go
very
deep,
so
again
limits
what
an
individual
neuron
could
learn
in
any
possibility.
Absolutely
yeah,
okay,.
D
I
think
this
is
this
is
one
of
the
limitations
that
we're
also
wondering
about
in
in
that
sort
of
model,
you
get
more
more
refinement
on
the
top
part
like
because
it's
a
branching
structure.
It
gets
wider
towards
the
top,
so
you
have
more
segments
that
are
further
apart
and
that's
so
it's
easier
to
initiate
a
sequence
with
a
lot
of
different
symbols,
but
as
you
move
towards
the
summer
it
would
it
becomes
more
and
more
restrictive.
C
Yeah
so
yeah,
so
the
way
we
the
way
we
dealt
with
this,
we
assumed
that
there
was
a
neural
population
that
represented
each
each
element
and
and
it's
a
sparse
activation
of
a
neural
population.
So
there
was
a
essentially
neurons
could
participate
in
many
many
different
elements
and
therefore
they
could
participate
in
many
different
places
and
sequences.
So
you
got
around
that
limitation.
B
We
will
get
to
this
actually
in
the
next
step.
Okay,
because
the
thing
that
we
obviously
haven't
talked
about
at
all
is
where
do
these
symbols
ap
et
cetera,
come
from
and
how
options
generated?
I
mean
the
for
the
specific,
initiate
initiation
of
plato
potentials.
The
biology
is
relatively
clear.
You
require
depolarization,
and
you
require
glutamate
binding
to
nmda
receptors.
They
open
up
the
cascade
of
channels
opens
up
and
you
get
the
plateau
potential.
B
That
we
sort
of
wanted
to
add
here
is
that,
let's
look
specifically
at
how
petals
would
be
generated
for
one
specific
segment
and
the
thing
that
we
have
included
in
our
model
is
that
all
our
synopsis
are
stochastic.
That's
pretty
well
documented
that
this
is
just
just
the
case.
Pre-Synaptic
transmitter
release
is
stochastic
and
quantal
and
whenever
an
afferent
sort
of
an
input,
neuron
spikes,
the
segment
on
which
it
projects,
the
the
bit
of
dendrite,
on
which
it
projects
doesn't
always
see
that
spike
right.
B
So,
yes,
sort
of
this
interaction
of
what
we
require
for
a
plateau
to
be
initiated
is
a
bunch
of
our
input.
Neurons
have
to
spike
and
then
a
sufficient
number
of
those
have
to
be
transmitted,
and
then
this
little
picture
here,
for
example,
we
just
for
simplicity.
We
reduce
the
numbers
here,
so
there
would
be
nimble
population
with
five
spikes
emitted
in
a
short
time
interval
so
that
the
epsps
can
overlap
and
the
pluto
potential
could
be
generated.
B
Two
are
actually
not
transmitted
just
because
happenstance,
but
this
would
still
be
a
larger
than
our
threshold,
so
plateau
would
be
initiated,
whereas
in
the
second
case
there
are
three
neurons
active
in
our
input.
Populations
which
would
suffice
to
emit
a
plateau
to
startup
initiate
a
plateau
potential,
but
again
one
is
not
transmitted,
which
means
that
actually,
no
plateau
potential
is
generated,
and
you
can
describe
this
relation
by
a
binomial
probability.
Distribution.
B
You
kind
of
want
to
react
to
with
a
plateau
potential
in
theory.
But
what
this
gives
you
is
the
probability
of
responding
to
a
plateau.
Potential
is
directly
proportional
to
how
large
the
input
volley
is
the
if
you
receive
20
spikes,
you're
much
more
likely
to
initiate
a
plateau
versus
you're
receiving
10
spikes
etc.
And
you
sort
of
have
this
hidden,
hidden,
continuous
variable
in
there
right
and
then
we
can
apply
that
to
our
plateaus
to
our
dendritic
trees
as
well.
So
the.
C
Are
you
are
you
taking?
Is
this
a
a
property?
I
mean
you
could
argue,
the
same
would
occur
even
if
synapses
were
not
stochastic,
you
could
just
say.
Oh
I'm.
I
have
so
many
axons
that
are
coming
in
and
if
they
all
a
certain
number
of
them
send
a
spike
that
gets
transmitted.
Then
then
we
reach
our
plateau.
So
so
are
you
showing
just
how
this
will
work
if
with
sarcastic
synopsis
or
somehow
stochastic
stamps
is
playing
an
essential
role.
B
B
We
can
talk
about
it
afterwards
again,
so
you
combine
these
relatively
simple.
We
decided
that
either
of
these
two
segments
emitting
a
plateau
would
be
enough
for
the
for
us
to
consider
this
little
sub
tree
here
to
emit
a
plateau
right.
So
it's
p
of
a
or
b,
and
we
have
a
little
logic
again
and
then
for
sequential
segments.
Now
we
actually
have
to
get
rid
of
the
temporal
component
to
be
able
to
analyze
it
in
any
way.
B
So
I
mean
this
is
this
is
just
a
bit
of
pushing
pushing
letters
around?
The
fact
is
that
you
can,
and
you
can
replace
as
well
the
these
little
p
of
a
or
b's,
or
this
p
of
a
and
b's
with
inserting
all
these
little
probability
distributions
of
threshold
crossing.
And
then
you
get
these
well-known
formulas
for
sort
of
all
events
and
end
events,
and
you
can
apply
this
recursively
and
get
a
really
sort
of
monster
of
a
formula
out
for
this
one,
that
it's
not
really.
B
It's
not
fantastic,
that
you
get
a
long
formula
for
this
one.
The
cool
thing
is
that
the
expected
value
of
the
entire
neuron,
responding
as
a
sort
of
the
key
message
here
is
proportional
to
each
of
the
individual
segments
crossing
a
threshold.
Given
a
bunch
of
volley
events
and
the
probability
of
each
of
the
neurons
crossing
a
threshold
is
directly
proportional
to
input
volume,
sizes.
A
B
A
D
So
I
think
I
think
the
point
here
here
would
be
that
you
can,
you
can
say,
what's
the
probability
of
the
neuron,
to
respond
to
a
given
sequence,
a
c
e,
for
example.
Assuming
the
sequence
is
there
right,
then
you
can
say:
okay,
that's
the
probability
that
the
first
segment
will
respond
to
the
first
signal.
D
But
if
you
increa
include
this
probabilistic
transmission,
you
could
get
into
a
situation
where
an
ensemble
of
a
couple
of
these
neurons
will
have
a
greater
response
because,
like
they,
each
of
them
independently
will,
with
some
probability
transmit
and
then
so.
You
can
infer
like.
If
and
if
you
have
a
couple
of
neurons
that
are
looking
for
the
same
sequence
and
half
of
them
got
activated.
Then
it's
an
indicator
that
there
is
like
stronger
evidence
for
a
sequence
than
if
only
a
quarter
of
them
turned
on
yeah
right.
B
Okay,
I
think
yeah,
that's!
Basically
it
we
sort
of
try
to
simplify
our
our
modeling
approach
and
sort
of
the
key
messages
to
take
away.
Is
that
we
think
if,
at
least,
if
you
believe
in
that
the
segmented
dendrites
are
possible
in
this
way,
where
they
interact
with
plateau
potentials
in
a
relatively
localized
way.
You
get
these.
B
There
are
clearly
a
lot
of
open
questions.
Networks
was
just
johannes,
just
mentioned
it
already.
You
can
get
the
expected
value
of
individual
neurons
read
out
in
an
ensemble
if
you
just
have
many
of
them,
because
there
is
no
reason
to
assume
that
sarcastic
synapses
are
in
any
way
dependent
on
each
other
on
different
neurons,
so
that
would
be
an
independent
sample
sort
of
the
same
probability
distribution.
B
B
B
We
don't
have
a
super,
clear
question,
answer
to
that
question,
yet
just
intuitions
and
then
the
other
very
big
topic
that
we
left
out
completely
today
is
how
do
you
learn
these
networks?
I
think
this
also
touches
on
the
inhibitory
synapse,
because
learning
that
is
actually
a.
If,
if
you
wanted
to
learn
that
in
the
context
of
the
types
of
models
we've
talked
about,
is
actually
presumably
quite
difficult.
B
How
would
you
know
when
you
have
to
switch
something
off
basically,
but
the
easier
problems
we
already
inherit
is
that
there's
good
evidence
from
this
busting
group
that
at
least
ltp,
so
potentiation
of
synaptic
weights
is
really
incredibly
local.
If
you
deal
with
active
dendrites
I.e
there,
even
if
you
take
the
back
propagating
action
potential
out,
which
is
in
theory,
meant
to
enable
the
synapse
to
learn
something
about
what
the
soma
actually
emits
in
the
end
doesn't
seem
to
be
that
important.
B
And,
in
our
view,
you'd
sort
of
have
to
balance
the
activation
probability
of
pathos
correctly,
because
segment
c,
for
example,
in
our
earlier
model,
would
be
able
to
block
all
activity
arriving
at
the
soma.
So
if
the
probability
at
segment
c
to
generate
a
plateau
is
too
low,
nothing
will
ever
happen.
That's
bad
and
in
the
same
way,
if
it
just
goes
active
all
the
time,
probably
also
a
bad
idea.
B
And
lastly-
and
this
is
very
clear-
I
think
from
from
our
presentations
that
we
rely
incredibly
on
where
synthesis
are
placed
in
the
wt3
versus
what
the
particular
weight
of
on
individual
synapse.
Is
this
clearly
implicates
structural
plasticity?
So
the
question
of
where
isn't
it
should
be?
There's
lots
of
evidence
that
this
happens
in
the
brain
in
many
different
ways,
but
we'd
really
kind
of
have
to
introduce
it
into
our
model
and
we
can't
rely
on
any
gradient
descent
type,
full
back
prop
rules
or
anything
like
that.
B
I
think
in
in
this
case-
and
this
is
really
the
type
of
classicity
that
would
have
the
greatest
impact
on
the
types
of
functions
we
can
learn
efficiently
right.
So
that's.
I
would
like
to
conclude
with
sort
of
our
grand
hypothesis
that
we'd
argue
that
neurons
with
active
and
segmented
dendrites
would
detect
sequentially
ordered
inputs,
but
they
do
so
with
timing
and
variance.
B
There's
some
more
information
here
we
have
the
fourth
version
of
our
preprint
here.
It
turns
out
to
be
not
very
easy
to
publish,
as
well
as
a
paper
at
the
nice
workshop
and
two
presentations,
hundreds
and
they
gave
at
other
opportunities
if
you're
interested
in
more
information.
Thank
you
very
much.
B
A
B
But
the
the
experiment
that
I
can't
record
from
the
top
of
so
I
don't
think
we,
I
would
remember
it
if
it
if
we
found
it
before
is,
is
really
what
we'd
really
need
is
that
these
sort
of
localized
interaction
really
does
happen.
There
is
a
clue
path
paper
from,
I
think,
20
15
14,
or
something
like
that,
which
is
a
computational
model
in
the
basal
dendrite,
where
they
measured.
They
were
interested
much
more
in
the
plasticity,
but
they
were
interested
in
active
nmda
events,
because
that
also
drives
plasticity.
B
There's
this
fantastic
e-life
paper
that
I
will
also
look
up
quickly
and
send
to
you
in
terms
of
this
compartmentalization
of
sort
of
electric
coupling
around
the
dendrite,
which
comes
up
with
dendritic
trees,
that
sort
of
segmentation
of
dendritic
trees
that
are
roughly
in
the
complexity
area
that
we've
shown
today
some
more
complicated,
some
much
simpler
yeah.
So
the
the
this
nmda
a
decade
of
the
nmda
spike
paper
actually
has
almost
the
graphic.
B
I
think,
if
I
for
just
like
almost
this
thing
at
the
very
very
end
of
the
of
the
paper,
so
they
hypothesize
this
already
right.
So
as
a
as
an
idea,
this
is
not
fundamentally
new.
There's
literally,
I
think
they
have
on
one
axis
and
it's
a
plateau
plateau
right.
This
is
what
they
what
they
end
their
paper
with,
so
it
sort
of
skirts
it
all
the
time.
B
D
So
what
happens?
If
we
only
send
the
burst
of
like
a
volume
of
spikes
to
the
to
the
most
distant
dendrite
segment,
to
the
one,
a
bit
closer
to
the
one
at
the
sama
versus?
We
only
stimulate
the
one
far
apart,
or
we
only
stimulate
one
in
the
middle
and
really
just
go
through
all
of
these
different
permutations
and
excluding
a
lot
of
these
other
distracting
factors
that
might
also
be
in
there
that
I
I.
B
Haven't
seen
I
mean
there
is
the
one
thing
that
might
be
closest
to
verifying
this
type
of
sequence:
detection
along
the
dendrite
with
active
dendrites,
but
it's
not
a
plateau
measurement
is,
I
think,
branco
and
heiser.
I
think
basically
measured
that
you
can,
on
a
relatively
fast
time,
start
with
active
gender.
It's
get
this
preferential
sequence,
sequential
activation
along
it
into
the
cable
right.
You
can
get
abc
and
you
have
a
much
higher
likelihood
of
a
response.
If
you
activate
in
that
order,.
B
Yeah,
but
I
mean
some
of
the
citations
you
you
might
have
seen
this
like.
This
is
extremely
new
work
and
experimenters
are
getting
super
inventive,
so
this
this
this
group
that
that
is
now
in
vivo
imaging.
So,
first
of
all,
it
seems
to
me
that
many
of
these
studies
that
come
up
with
all
this
there's
actually
a
lot
of
active
dendrites
going
on
in
the
brain
in
vivo
studies,
and
they
don't
seem
to
often
occur
in
in
vitro
as
much
so.
B
A
B
D
And
it's
a
theoretician,
it's
very
difficult.
I
mean
you
probably
went
through
the
same
phase
two
right
and
it's
very
difficult
to
navigate
the
biological
evidence
out
there,
because
sometimes
you
also
for
for
every
phenomenon.
You
find
there's
also
like
the
direct
contradiction
of
that
too
yeah
yeah.
C
So
my
general
rule
is,
you
know
you
have
to.
You:
have
to
read
at
least
a
half
a
dozen
papers
to
start
making
a
judgment
of
what's
right
or
wrong
because
they
are
contradictory
at
times
or
they
use
different
experimental
techniques
or
they
use
different
animals,
different
preparations
of
animals
and
so
on
very
difficult.
I
have.
B
B
C
C
If
you
think
about
a
melody,
the
intervals
between
the
individual
elements
and
melodies
are
very
important
and
that's
true
of
certain
movements
as
well,
like
you
know,
learned
movements
and
so
and
then
you
also
have
to
be
ability
to
speed
things
up
and
slow
them
down
both
for
playback
and
inference,
and
so
you
have
to
have
a
some
sort
of
knowledge
about
that.
So
I
just
didn't
know
much
have
you
thought
about?
C
Don't
answer
this
yet,
but
have
you
thought
about
the
the
issue
of
how
you
would
encode
specific
timings
and
how
you
would
be
able
to
speed
them
up
and
slow
them
down
in
addition
to
the
environments
that
you
mentioned
and
then
the
second
question
is
a
broader
question
which
is
like.
Well,
I
don't
know
how
familiar
with
our
htm
sequence
memory,
but
I'd
love
to
hear
your
criticism
of
it.
C
B
Okay,
so
I
think
you're
touching
on
on
a
very
interesting
point
there
with
the
timing
during
some
part
of
our
sort
of
research
journey
on
this
topic.
We
really
thought
about
like
this
is
such
a
simple
model.
What
we
I
mean,
johannes,
is
at
the
integrated
circuits
institute.
Now,
that's
not
coincidence,
we're
interested
in
building
this
type
of
designing
hardware
for
this
type
of
algorithm
as
well,
but
then
you
immediately
go
into
applications
and
applications.
It
will
always.
It
will
sometimes
be
relevant
to
timing,
and
sometimes
it
won't
be.
B
How
do
you
get
it
back
if,
if
you
lost
it
so
within
the
scope
of
this
model,
I
think
it's
interesting
that
not
all
energetic
events
are
platoons,
they're,
also
spikes,
nitrogen
spikes
or
whatever
sodium
spikes
rather
and
those
might
be
much
more
suited
to
solving
these
types
of
problem.
How
sort
of
our
takeaway
from
this
is
that
the
diversity
of
neurons
is
probably
not
an
accident,
so
our
intuition,
I
I
think,
I'm
speaking
for
both
of
us
is
that
you
might
want
a
different
solution
for
that
type
of
problem.
B
If,
if
if
timing
is
really,
if
precise
timing
is
really
important,
you
might
need
some
other
ingredients
on
top
of
the
mix
sort
of
right
you
just
the
point,
uranus
seems
by
now,
at
least
from
a
neuroscience
perspective-
seems
a
little.
I'm
not
sure
where
you,
where
you're
going
to
end
up.
You
have
to
really
have
to
rely
on
the
network.
B
To
do
everything
I
don't
think
that's
completely
correct,
but
when
you
change
the
neuron,
I
don't
think
there's
a
one-year-old
model
to
rule
them
out
all
that's
sort
of
my
takeaway
here
I
think
platoons
can
be
very
valuable
computationally.
You
can
do
lots
of
interesting
stuff.
You
can
do
a
little
bit
more
theory
on
these
little
expressions
that
we
presented
and
it
sort
of
gives
you
really
an
insight
of
how
computation
might
come
about
in
this
way.
B
But
you
lose
the
timing
and
sometimes
you
want
it,
and
it
might
be
that
you
just
need
a
different
tool
for
that.
The
the
one
thing
I
would
add
is
that
when
we
are
talking
about
emitting
synchronous
volleys
is
I
mean.
The
hippocampus
people
also
have
this
debate
all
the
time,
whether
the
sort
of
coordinated
output
of
spikes,
which
are
also
well
timed
right
and
they
pre-process
and
post-process
spikes,
and
they
have
all
this
intricate
variations
in
the
timing.
B
In
the
end,
they
encode
order,
but
they
have
to
change
the
timings
and
they
have
distributed
about
rhythm
makers
or
internally
generated
rhythms,
and
I
think,
with
those
kinds
of
model,
our
model
is
still
compatible
because
you
can,
for
example,
if
you
have
an
external
rhythm
clock,
if
that
would
exist,
which
I
don't
think
it
does.
But
in
theory
you
can
of
course
have
neurons
in
upstates
and
then
just
go.
Basically
query
them
right,
send
a
spike
see
if
the
spy
comes
out,
and
then
you
get
your
nicely
timed
outputs.
D
Topic
yeah.
They
also
like
different
dirty
tricks.
One
could
play
we
were
thinking
about
it
like,
for
example,
inhibition
might
be
on
a
different
time
scale,
and
then
you
could
potentially
also
trigger
some
rebound
spikes
by
basically
like
inhibiting
a
plateau
for
some
time,
and
then
the
activity
comes
back,
and
this
way
you
could
like
delay
something
and
make
precise
delays.
D
You
could
build
weird
clock
circuits
that
generate
these
pulses
and
then
you
detect
the
specific
sequences
of
clock
pulses,
but
this
is
more
like
esoteric
stuff
and
we
didn't
want
to
go
into
this
direction
like
first
think
about
the
basic
principles.
But
yes,
we
focus
more
on
the
detection
of
sequences.
The
generation
of
sequences
is
a
bit
more
difficult,
and
I
think
this
is
where
you
then
need
to
think
about
transmission
delays.
D
We
have
completely
ignored
the
temporal
the
passive
dynamics
of
the
dendrite
right.
We
only
talked
about,
like
coincidence,
spikes
triggering
event
thing
goes
on,
but
of
course
it
doesn't
happen
instantaneously.
There
are
some
like
there's
still
the
passive
integration
and
maybe
the
passive
integration
like
this
exponential
kernels.
They
are
really
useful
for
the
precise
timing.
D
Maybe
it's
recurrent
population
activity,
hard
to
say,
yeah
and,
and
maybe
relating
the
second
question,
so
you're
asking
for
a
comparison
to
the
htm
model.
I
think
so
I
I
like
the
htm
model.
I
think
it's
very
nice
and
I
think
it's
it's
in
broad
strokes.
It
is
very
compatible
to
this.
I
think
the
difference
is
I
mean
you
have
to
correct
me
if
I'm
misrepresenting
the
model,
but
for
for
us
the
htm
model
was
not.
D
I
think
we
didn't
perceive
it
as
an
as
an
attempt
of
like
a
biological
model.
That
explains
the
direct
mechanism,
but
it's
like
also.
You
have
the
time
step
operation.
You
have
some
simplifying
assumptions
in
there,
which
I
think
are
very
elegant,
like
the
the
dendrite
is
the
sort
of
pattern
detector
and
you
have
the
different
branches
for
detecting
different
patterns
and
so
on.
This
is
similar.
I
think
we
were
interested
in
really
like
this
local
mechanisms,
so
how
how
is
the
plateau
generated?
D
D
I
think
if
we
go
in
the
direction
as
pascal
I
mentioned
you,
you
want
to
actually
build
something
in
hardware.
Then
all
of
this
complexity
is
very
unpleasant
and
you
you,
if
you
as
soon
as
you
start
thinking
about
them,
how
can
we
break
down
the
neuron
like
this
complex
neuron
that
we're
showing
here
like
a
tree
like
this?
How
can
you
break
it
down?
I
think
you
will
find
you
probably
can
just
replica
replicate
this
by
building
a
tree-like
structure
out
of
a
lot
of
html
neurons.
C
C
So
we
focus
on
the
dendritic
spike
element
which
we
believe
is
very
well
documented
in
the
neuroscience
literature
and
you
focus
on
a
different
element
of
dendrites.
I
just,
but
I
don't.
At
least
I
don't
know
if
we
achieved
it,
but
our
goal
is
definitely
to
not
be.
It
was
to
be
very
biologically
accurate.
The
htm
model
our
mind
is,
is
we're
proposing
that
actual
world
norms
do
this,
but
I
can
see
that
how
we
end
up
with
different
places,
but
but
I
appreciate
that
feedback-
that's
very
helpful.
B
Yeah,
it
feels
to
me
like
we.
We
definitely
I'm
not
sure
how
I
can't
presume
it
for
us,
it
was
very.
It
is
still
quite
difficult
to
publish
all
this
for
the
stuff,
even
though
we
did
have
very
good
feedback
along
the
way.
But
if
you
get
like,
we
had
nice
reviews
as
well,
but
we
also
frequently
had
very
not
very
nice
reviews.
B
One
sort
of
very
nice
thing
that
the
first
time
I
encountered
the
htm
model
was
actually
in
a
hackathon
where
a
group
of
students
which
we
did
together
with
the
I
guess,
the
german
equivalent
of
the
cdc
and
one
group
of
students,
decided
to
use
your
the
python
packages
that
you
provide,
and
I
think
the
the
the
outlier
detection
or
the
sort
of
anomaly
detection
thing,
and
I
was
extremely
successful
in
analyzing
flu
data
and
they
had
the
best
results.
So
that
was
really
really
nice.
B
So
the
sequence,
let
that
to
this
point
and
that
I
thought
was
very
elegant.
The
one
thing
that
tripped
me
up
and
which
I'm
not
super
sure
about
is
that
it
seemed
to
rely
sometimes
on,
like
very
particular.
This
thing
has
to
spike
and
trigger
this
right.
It's
a
very
state
automata,
like
almost
mechanism
on
the
level
of
neurons,
transmitting
spikes
to
other
neurons,
and
that
was
the
thing
that
felt
a
little
bit.
I'm
not
super
sure.
If
I
got
this
right,
I
mean
we
can't
throw
stones
here.
B
So
so
I
guess
my
question
is:
I'm
not
sure
I
understand
the
question
actually
do
you
think
the
hdm
model
straightforwardly
would
just
work
with
the
way
we've
presented.
It
here,
sort
of
a
sort
of
ensemble
that
sends
a
bunch
of
spikes
that
are
probabilistically
transmitted
and
that's
the
signal
well.
C
To
be
well,
I
think
in
our
case
there's
always
a
distribution
right.
Nothing
is
represented
by
a
single
neuron
ever
and
so
it's
always
a
distribution.
The
distributions
are
quite
unique
and
they
can
be
quite
noisy.
So
if
I'm
saying
oh,
this
two
percent,
if
I
have
a
population
of
5000,
neurons
and
let's
say
100-
are
active
at
any
point
in
time.
Well
and
I'm
I
need
to
detect
20
of
those
to
reliably
detect
the
pattern.
Well,
then,
you
can
have
a
lot
of
noise.
You
could.
C
C
So
it's,
but
that's
what
gets
my
question
earlier,
because
we
don't
rely
on
as
a
computational
principle,
but
we
rely
on
it
as
a
constraint
that
says
yeah.
The
system
has
to
be
extremely
robust
to
all
these
different
types
of
probabilistic
influences
or
just
you
know,
noise
type
influences.
So
I'm
not
sure
if
that
gets,
that
your
question
there.
A
A
C
000
neurons,
you
can't
show
you
know
40
synapses
per
segment,
you
know,
and
so
you
make
these
very
simple
drawings.
We
saw
three
active
neurons
and
three
active
synapses
or
whatever
it
was.
But
of
course
I
think
in
the
body
of
the
paper-
and
I
think,
especially
in
the
2016
paper,
super
tight
did
a
nice
little
write-up
about
the
mathematics
of
this
there's
a
box
a
table
on
that,
and
so
you
know
trying
to
get
people
to
say
you
know
it's
like
these
big
populations
that
are
doing
all
these
things,
but.
A
That's
really
helpful,
yeah,
that's
something
you
could
leverage
as
well.
I
think
you
know
basically
what
the
mathematics
says
if
you
have
a
very
high
dimensional
input,
but
you're
only
sampling
from
it
you
so
the
dendritic
segment
only
has
20
or
30
synapses
from
a
much
higher
dimensional
population.
A
You
can
get
these
sort
of
robustness
properties
and
that's
something
you
can
easily
incorporate
in
your
model
as
well.
You
know
as
long
as
the
underlying
presynaptic
population
is
large
enough,
and
but
the
dendritic
segment
is
just
a
small
subset
of
that.
Picking
up
on
patterns,
you
automatically
get
a
lot
of
these
properties.
I
think.
B
I'd
be
interested
if
we
like
convinced
you
even
to
a
slight
degree
that
there
might
be
something
useful
about
stochastic
synopsis
because
from
if
you
read
the
literature
on
it,
as
at
least
for
some
time
there
is,
there
was
a
bunch
of
debate.
Whether
plasticity
should
really
change,
synaptic
release,
probabilities
or
whether
it
should
change
change,
synaptic
ways
or
efficacies,
and
that
implies
that
there
is
a
computational
function
and
you
do
see
these
nice
little.
B
So
the
transmission
probabilities
are
clustered,
for
example,
if
if
there's
a
population
of
a
thousand
neurons
or
so,
and
they
all
project
on,
say,
100
and
they
tend
to
find
phytosynchronous
volumes
onto
this
specific
segment,
you
will
find
the
cluster
of
spines,
on
which
the
all
the
contexts
have
very
similar
release
probabilities.
So,
but
overall
in
the
neuron,
the
release
probabilities
are
actually
quite
heterogeneous
and
not
homogeneous
at
all,
so
they're
widely
distributed.
C
So
there
seems
to
be
something
to
it
when
you
say
their
release
probabilities.
What
are
you
referring
to
exactly?
I
mean
I
mean
you
mean
you
know.
First
of
all,
as
far
as
I
understand,
axonal
spikes
are
very
reliable.
You
know
they
will
get,
they
will
get
to
their
destination,
so
we're
just
talking
about
releasing
of
a
transmitter
at
the
synaptic
clef
and
the
the
stochastic
mix.
C
There
is
largely
due
with
the
the
very
recent
history
of
the
synapse
more
so
it
doesn't
have
vesicles
that
are
waiting
to
be
dispersed,
as
opposed
to
I
mean
it,
it's
it.
Well,
I
guess
maybe
the
short
answer.
Your
question
is,
I
I've
never
read
anything
which
is
convincing
to
me
about
the
the
the
advantages
of
these
of
synapses,
not
always
releasing
vesicles,
but
it
has
more.
It
almost
always
has
to
do
with
you
know.
Basically
what
is
the
recent
history
some
some
synapses?
C
They
decrease
over
time.
So
if
you
have
a
series
of
spikes
in
a
row,
then
the
release
probability
goes
down
and
some
actually
goes
up,
but
but
I've
never
seen
a
convincing
argument
to
me
that
oh
this
is
a
that
that
there's
the
randomness
of
them.
If
there
is
such
a
thing,
is
it
really
essential?
So
it's
a
short
answer,
your
question,
but
I've
never
seen
anything
convincing
myself.
D
I
think
we
have
to
look
in
our
references
because
I'm
pretty
sure
that
we
looked
at
a
couple
of
studies
that
report
very
different
transmission
probabilities
and
they
can
get
quite
low.
Actually
up
to
the
point
that
you
could
even
argue
that
silence
synapses,
that
don't
release
anything
are
like
transmission
probability,
zero,
and
then
you
might.
C
Actually
have
a
continuum
right
yeah.
We
think
we
think
actually
those
are
going
to
be
essential
in
certain
learning
situations,
because
I
mean,
if
you
think
about
synapses,
are
your
basic,
your
method
of
learning.
Well,
you
can
form
your
synapses
pretty
quickly,
but
pretty
quickly
is
in
the
matter
of
you
know,
you
know
tens
of
minutes
and
that's
not
fast
enough
for
a
lot
of
the
learning
we
have
to
do
so.
C
Silence
can
have
to
give
you
the
ability
to
essentially
go
from
a
zero
transmission
to
a
very
a
positive
transmission
in
a
very
quick
metabolic
cascade,
which
could
you
know
essentially
be
instant
learning
or
very
close
to
instant
learning.
So
and
then
you
know
intentionally,
you
see
a
lot
of
silent
synapses
in
the
hippocampus,
where
that's
where
you
need
to
have
very
fast
learning
for
certain.
C
D
For
us
they
are
the
extreme
end
of
the
spectrum
with
probability
zero,
but
I
guess
like
yeah,
you
can
interpret
it
differently
and
I
don't
think
there
is
really
yeah
it's.
I
don't
think,
there's
that
much
direct
evidence
that
you
could
conclusively
say
this
is
definitely.
This
is
how
it
works.
Like
again,
there's
different
arguments.
We've
seen
where
some
people
say
that
the
transmission
probabilities
are
very
high,
but
it's
in
one
population
of
neurons
we're
different,
it's
very,
very
low.
They
are.
Are
they
homostatically
regulated
or
not?
Are
they
just
a
chance
occurrence?
D
C
But
anyway,
from
a
theory
point
of
view
I
haven't
found
any
compelling
yet
I
mean,
and
you
can
look
at
all
this
data
and
you
have
to
just
you
know.
As
we
said
earlier,
you
can't
just
read
the
data
and
accept
it.
You
have
to
say:
okay,
well,
there's
different
ways.
You're
going
to
interpret
this
and
some
people
say
well
they're
stochastic.
Therefore
it
must
be.
C
You
know
important
or
you
could
say
well,
they're,
not
as
sarcastic
as
you
think
they
are,
and-
and
maybe
we
just
have
to
deal
with
it-
the
fact
that
they're,
you
know
we're
dealing
with
a
little
biological
stuff
that
isn't
always
you
know
routine
little
things
they're,
not
always
reliable,
so
the
system
has
to
work.
Despite
that,
that's
the
way
I
want
to
look
at
it.
Yeah.
D
First,
I
think
the
intuition,
I
think
the
only
thing
that
kind
of
made
sense.
So
for
us,
it
was
also
like
adding
noise
for
from
an
engineering
point,
doesn't
help
normally
right.
It
makes
things
worse,
adding
noise
unnecessarily,
with
one
exception.
I
think
that's
this
idea
of
like
dithering.
Some
input,
like
you,
have
a
fixed
threshold
and
you
have
a
signal,
and
if
you
add
noise
to
it,
you
you
can
get
out
of
the
deterministic
detector.
C
Wouldn't
and
that
that
may
be
true
too,
but
that,
but
that's
essentially
again
compensating
for
perhaps
as
a
a
non-ideal
world
right
you
know
so
anyway.
These
are
all
the
interesting
ideas
so,
but
I
I
like
what
you're
doing
you
know
I've
always
I
know
there's
there
really
is
evidence
that,
under
certain
conditions
in
certain
neurons,
the
order
along
the
dendrite
matters
and
we
we
don't
pay
attention
to
that
really,
and
I've
always
wondered
you
know,
is
it
going
to
be?
Is
it
important
or
not?