►
Description
To learn more about Brains@Bay, visit out Meetup page: https://www.meetup.com/BraIns-Bay/
Brains@Bay Meetups focus on how neuroscience can inspire us to create improved artificial intelligence and machine learning algorithms. In this meetup, we focus on the role of active dendrites in learning from a neuroscience and a computational perspective.
First, Matthew Larkum (Larkum Lab) gives us an overview on the main advances in active dendrites over the last two decades.
Then, Illenna Jones (Kording Lab) presents her new, exciting research on computational power of biological dendritic trees.
Finally, Blake Richards (Linc Lab) argues that dendrites solve an implementation problem for brains, but they are not necessary at the algorithmic level, and the importance for ML is largely with respect to neuromorphic chips.
Link to meetup: https://www.meetup.com/BraIns-Bay/events/272662486/
0:00 Matthew Larkum - The Role of Active Dendrites in Learning
45:55 Illenna Jones - Can single neurons solve MNIST?
1:15:08 Blake Richards - The Role of Dendrites in Machine Learning
1:44:17 Discussion / Q&A
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a cohesive theory, core software technology, and numerous software applications all based on principles of the neocortex. Our innovative work delivers breakthrough capabilities and demonstrates that a computing approach based on biological learning principles can do things that today’s programmed computers cannot do.
Subscribe to our Weekly News Digest for the latest news about neuroscience and artificial intelligence:
https://tinyurl.com/NumentaNewsDigest
Subscribe to our Monthly Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter
Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta
Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/
Our Website:
https://numenta.com/
B
B
So,
just
to
give
some
background
brains
at
bay
is
a
machine
learning
neuroscience
meetup
that
started
last
year
in
the
bay
area
it's
now
online
and
the
idea
is
to
discuss
brain
machine
learning
algorithms,
both
from
the
perspective
of
neuroscience
and
machine
learning
researchers.
So
we
had
previous
meetups
on
lateral
connections.
Predictive
processing
have
been
learning,
sparse
representation
and
continual
learning
and
we'll
keep
doing
this
every
time
with
a
different
topic.
So
you
have
some
feedback
or
suggestions
for
next
topics
or
speakers.
Please
leave
a
message
at
the
meetup
or
send
me
an
email.
B
So
in
our
agenda
today
we
have
matthew,
larkin
from
larkin
lab
and
then
lena
jones,
from
cutting
lab,
blake
richards
from
link
lab.
And
finally,
we
have
a
discussion
panel
and
where
we're
gonna
exchange
some
ideas,
the
talks
will
be
recorded
and
made
available
later
on
youtube,
and
I
wanted
to
ask
the
penalties:
does
anyone
have
an
issue
requiring
the
talk,
so
is
that
okay
tonight
we'll
end
our
break?
B
B
B
B
So
just
give
a
brief
buyer
if
you
don't
mind
so
professor
matthew
larkin,
I
got
from
your
cv,
so
let
me
know
if
something's
wrong,
I
graduated
from
sydney
university
was
a
postdoc
for
six
years
in
the
laboratory
of
nobel
prize
winner,
bert
segment
and
he's
now
a
principal
investigator
of
the
larkin
lab
at
the
neuroscience
research
center
in
humboldt
university
of
virginia
now,
professor
matt
is
recognized
as
one
of
the
leading
experts
in
the
whole
role
of
think.
Writing
learning
and
we're
really
happy
to
have
you
here.
C
C
C
Click
on
that
screen,
so
I
can
move
on
so
the
the
the
topic
is
the
role
of
active
dendrites
in
learning
and
and
so
on,
the
website.
It
says
first
matthew
a
leading
expert
on
dendrites
thanks
for
that
super
tight,
we'll
give
an
overview
on
the
main
advances
in
active
dendrites
over
the
last
two
decades
and
that
in
30
minutes.
C
So
I
thought
about
exactly
how
you
you,
you
present
all
the
main
advances
in
active
dendrites
over
the
last
two
decades,
and
I
decided
that
that's
going
to
be
a
difficult
one,
but
I
got
some
help
from
super
tai.
Who
said
it
would
be
great.
He
gave
me
this
email,
I
think,
was
last
night
of
the
night
before
it'd
be
great.
If
you
could
review
the
main
issues
in
a
way
that
is
accessible
to
a
variety
of
backgrounds
before
describing
your
latest
work.
C
Well,
what
I
thought
I'd
do
to
rather
than
go
over
all
the
main
advances
was
to
stick
to
the
topic
and
and
give
just
what
I
think
are
the
the
two
main
advances
or
the
top
two
main
advances
in
my
opinion
and
number
one
in
my
opinion
would
be
nba.
Spikes
and
number
two
is
a
more
recent
thing,
behavioral
time
scale,
plasticity
or
btsp
as
it's
becoming
known
and
then
hopefully,
I've
still
got
time
to
talk
about
my
data
and
I'll
I'll
talk
about
I'll.
C
Try
to
keep
on
topic
to
the
active
dendrites
are
learning
have
to
admit
that
there
was
a
little
bit
of
a
tinge
of
regret
in
focusing
on
active,
dendrites
and
learning,
not
not
because
there's
I'm
not
excited
by
this
topic.
In
fact,
quite
the
opposite.
I
think
that
we're
we're
we're
now
closing
on
something
that's
new
for
the
lab
and
really
important
for
us,
but
2020
has
been
a
good
year
for
us
in
terms
of
closing
a
whole
lot
of
projects.
C
These
are
a
spread
of
the
things
that
have
come
out
this
year
and
really,
I
feel
like
I
could
give
a
talk
on
any
one
of
these
papers
for
half
an
hour
an
hour,
and
none
of
these
topics
are
on
learning
and
it's
nonetheless-
and
I
should
say
we
haven't
actually
published
anything
on
on
dendrites
and
learning
at
this
point,
but
don't
despair.
We
do
actually
have
a
a
pre-print
on
bio
archive
and
that's
what
I'll
talk
about
now.
C
So
I'm
really
grateful
and
honored
in
a
way
just
to
have
been
in
this
kind
of
familiar.
Nevertheless,
well
I
should
say
two
of
these
have
already
left
the
lab
this
year
now,
oya
and
jan
have
gone
off
to
start
their
own
labs
and
by
the
way
they're
looking
for
post-docs
and
phd
students.
If
there
are
any
people
in
the
audience
who
are
interested
but
but
I'll
focus
on
it
on.
C
As
I
said,
a
study-
that's
unpublished
at
this
point
that
is
on
dendrites
and
learning
and
and
highlights,
I
think,
also
the
role
of
layer,
one
in
in
the
cortex
and-
and
this
has
been
the
work
of
guy
doran
and
jion
shin
jian
shinzo,
a
phd
student
in
the
lab
okay.
So
just
a
bit
of
background
and
going
over
actually
more
than
last
two
decades.
C
I
I,
in
my
opinion,
a
really
important
player
in
the
in
the
story
of
what
becomes
now
nmda
spikes
was
bartlett
mel
and
here's
an
early
paper
from
butler
mel
and
christophe
koch,
who
were
looking
at
what
they
called
sigma,
pi
learning
and-
and
that
really,
I
think,
captures
the
the
notion
that
if,
if
you
include
dendrites
in
the
computational
process,
then
you
can
do
more
than
just
add
up
inputs.
You
can
you
can
do
multiplicative
functions
and
you
can
combine
these
with.
C
C
I
think
a
really
present
review
on
on
information
processing
in
dendritic
trees.
This
particular
picture
on
the
right
is
used
pretty
well
in
every
conference
on
dendrites
and
he
was
basically
highlighting
the
fact
that
nobody,
at
least
at
that
time,
took
account
of
the
real
shape
of
neurons
and
nobody
seemed
to
care
about
the
possibility
that
there
were
really
important
computations
going
on
in
dendrites
and
an
interesting
anecdote
right
at
the
end
of
that
paper.
C
Because
all
of
a
sudden,
you
could
ask
what
are
the
computational
properties
of
dendrites
and
probe
that
question
experimentally
and
that
kicked
off
a
whole
lot
of
subsequent
studies
and
and
for
my
money.
Actually,
ironically,
after
what
butler
was
saying
in
the
very
same
year,
I
it.
I
would
say
that
the
experimentalists,
to
some
extent
took
over
and
and
unfortunately,
there's
been,
in
my
opinion,
a
dearth
of
of
modelers
and
theoretical
neuroscientists,
those
those
that
are
speaking
today
accepted
and,
of
course,
numenta.
C
Well,
so
I
I
visited
new
mentor,
I
think
in
2015,
and
we
had
some
really
fantastic
discussions
and-
and
I
think
it
was
already
clear
actually
from
jeff's
book
on
intelligence
was
that
2004.
I
think
that
that
that
he
already
was
focused
on
trying
to
derive
properties
from
the
brain
and
to
understand
the
brain
and
and
to
learn
from
that,
which
I
think
makes
jeff
stand
out
in
a
in
a
crowd,
because
it's
it's
it's.
I
think
these
these
properties
derived
from
single
neurons.
C
That
is
unusual.
But
here
in
this
particular
figure
you
can
see
the
the
standard
model
for
a
for
a
neuron,
that's
used,
and-
and
it's
really
in
my
opinion-
it's
surprising
just
how
many
models
rely
on
on
this
formalization
of
a
neuron,
given
what
we
know
about
neurons
after
now,
two
and
a
half
decades
of
of
recording
from
their
dendrites
and
their
computational
properties,
and-
and
I
I
I
sense
that
that
new
mentor
and
jefferson
and
zukai
and
others
are
now-
are
now
probing
this
and
making
inroads
into
that.
C
You
can
see
in
a
way
these
are
multiple
clusterons.
If
you
like
that
are
added
to
the
to
the
dendrites
and
and
is,
is
showing
that,
if
you
take
care
of
space,
then
you
have
a
whole
lot
of
new
dimensions
or
parameters
or
possibilities
that
a
single
neuron
can
do,
and
I
think
elaine
is
going
to
talk
a
lot
about
that.
C
I
don't
exactly
know
what
her
talk
is,
but
given
the
title
that
sounds
likely
so
that,
in
that
this
paper
that
I'm
alluding
to
jeff
and
super
tight,
say
it's
common
to
think
of
a
neuron
as
computing
a
single
weighted
sum
of
all
of
its
sinuses.
This
notion
sometimes
called
a
point
neuron.
That's
this
thing
in
a
forms,
the
basis
of
almost
all
artificial
neural
networks,
active
dendrites,
suggest
a
different
view
of
the
neuron.
Where
neurons
recognize
many
independent,
unique
patterns.
C
C
I
should
say-
and
it's
still
I
think,
not
a
a
term
that
that
you
can
just
drop
into
like
company,
and
you
have
to
justify
this
and
and
and
and
insist
that
that
it's
that
it's
even
a
thing
that
that
exists,
despite
what
I
would
say
is
manifest
seven.
So
I'll
go
through
some
of
that
evidence
now.
But
in
this
same
paper
before
I
do
that.
There's
this.
C
What
it's
showing
is
the
the
attention
that
jeff
and
super
tight
put
on
recognizing
that
the
ap
called
dendrites
actually
have
a
function
and
also
that
there
there
are.
There
are
important
inputs
coming
to
this
part
of
the
dendrite,
so
we'll
get
back
to
that
in
the
second
part
of
this
talk
so
nmda
spikes.
C
They
really
came
on
the
scene,
but
from
this
paper
around
2000
and
jackie
schiller
was
the
the
driving
force
and
she
later
combined
with
with
bartlett
mill
a
lot
in
in
and
still
does
on
on,
trying
to
describe
how
what
consequences
this
have.
But
essentially
an
nmda
spike
is,
is
a
spike
that
comes
from
the
fact
that
nmda
channels
are
actually
also
voltage
sensitive.
C
But
of
course,
they
are
also
at
the
same
time,
receptors
and
and
that's
what's
so
important
about
an
nba
spike
that
there
are
two
gates
or
two
conditions
that
have
to
be
on
for
them
to
work.
But
when,
when
these
two
conditions,
that's
to
say
depolarization
and
transmitter
molecule
glutamate,
but
when
these
receptors
bind
glutamate
and
there's
depolarization
now
they
can
act
as
voltage-sensitive
channels
and
they
can
go
awol
and-
and
you
can
get
a
spike.
C
But
now
the
important
point
about
it
requiring
a
transmitter
is
that
it
can't
spread
like
say
a
sodium
spike
or
a
calcium
spike,
because
it
can
only
spread
as
far
as
the
transmitter
is
available.
So
it's
really
tracking
the
the
the
limit
of
of
the
synaptic
input
and
that's
where
clustered
input
comes
in
so
unlike
sodium
spike,
which
will
just
keep
on
propagating
as
long
as
the
density
of
sodium
channels
is,
is
high
enough.
C
So
let
me
let
me
go
really
quickly
through
the
this
is
work
that
I
then
did
in
collaboration
with
jackie
schiller,
and
we
started
to
look
to
see
whether
or
not
you
could
get
nmda
spikes
in
the
tufts
of
the
big
pyramidal
cells
of
the
of
the
neocortex,
and
you
can
see
we're
doing
really
difficult,
dendritic
patches
with
extracellular
stimulation
of
the
tissue.
This
in
slices
and
trying
to
see
whether
or
not
these
tough
dendrites
could
could
evoke
nmda
spikes,
and
I
won't
dwell
too
much.
C
We
could
block
it
with
a
with
a
blocker
for
an
mda
channel,
so
we
could
show
that
it's
nmda,
sensitive
and
importantly
in
this
case,
which
is
the
gold
standard
way
of
showing
an
nmda
spike,
we
could
do
it
by
uncaging.
That's
what
this
this
blitz
of
light
is
is
indicating
we
could
uncage
glutamate
and
and
cause
the
these
spikes
to
occur,
even
when
we
blocked
every
other
channel.
C
That's
why
it's
the
gold
standard,
so
so
the
only
way
for
this
to
occur
is
is,
is
through
nmda
channels
because
it
couldn't
be
through
any
of
the
the
say,
normal
calcium
channels
or
sodium
channels.
Okay
and
needless
to
say,
that's
a
difficult
thing
to
do
in
vivo.
So
it's
very
hard
to
show
that
that
to
do
the
gold
standard
experiment
as
it
were
in
a
you
know
in
viva
behaving
context,
so
trying
to
encapsulate
the
properties
of
the
pyramidal
neuron.
C
With
this
in
mind,
what
we
were
finding
was
that
the
there's
basically
three
kinds
of
spikes
in
a
pyramidal
neuron
there's
the
sodium
spike,
which
is
the
one
that's
that
all
models
refer
to.
This
is
basically
the
output
spike
and
then
there's
a
castle
spike,
which
I
really
haven't
had
a
chance
to
talk
about.
But
this
is
a
really
important
driving
force
that
causes
bursts
in
the
neurons
in
the
apical
dendrite.
C
Essentially,
we
we
would
put
forward
a
model-
something
like
this-
we're
not
quite
sure
how
many
compartments
you
should
have,
but
there
should
be
interaction
between
the
the
two
main
initiation
zones
for
sodium
and
calcium
and
then
lots
of
spots
on
the
dendrite
that
that
could
evoke
nmda
spikes
and
and
just
as
jeff
and
supertag
were
saying
in
their
paper.
What
you
expect
from
this.
C
Randomly
synapses
onto
different
compartments
of
of
the
apical
dendrite
here,
the
tough
dendrite
and
they
become
basically
patent
detectors.
That's
what
this
combination
lock
is
supposed
to
indicate.
So,
basically,
some
this
cell
could
know
whether
a
particular
eight
inputs
came
to
the
cell
and
and
it
could
have
an
enormous
influence
on
on
the
output
of
this
neuron.
Unlike
the
standard
model,
which
doesn't
care,
which
particular
eight
synapses
come
and
usually,
eight
would
not
be
enough
to.
C
C
So
it's
like
a
bond
going
off
within
the
within
the
in
the
cell
and
it's
basically
able
to
catch
these
these
patterns
and
it's
all
dependent
on
clusters.
So
the
question
becomes:
do
we
see
these
clusters
in
vivro
when
we
seized
nmda
spikes
and
now
really
fast,
because
I'll
still
otherwise
really
run?
Out
of
time
the
we
have
been
able
to
show
that
there
are
nba
spikes
in
vivo.
C
I
think
there's
such
an
aversion
to
saying
nmda
spike
that
that
people
find
ways
around
it
in
the
literature.
And
so,
if
you
pay
attention
to
see
whether
or
not
something
actually
probably
has
found
an
nmda
spike,
I
think
this
is
in
a
way,
a
tragedy,
because
the
the
the
word
calcium
spike
for
me
is
a
reserved
word
for
a
a
kind
of
spike
that
comes
in
the
apical
dendrite
and
spreads
all
the
way
into
the
tuft
and
calling
an
nmda
dependent
spike.
C
A
a
an
mda
dependent,
calcium
spike
is
a
bit
like
calling
if
you
would
do
two
photon
imaging
of
calcium
at
the
cell
body,
looking
for
activity
and
finding
that
it's
that
you're,
really
using
the
calcium
in
that
case,
to
to
indicate
that
there
are
sodium
spikes
at
the
cell
body
and
it'd,
be
a
tragedy,
I
think,
to
say
that
they're
gtx,
sensitive,
calcium
spikes
at
the
cell
body,
I
mean
really
they're
sodium
spikes
that
engender
some
calcium.
That
comes
with
them.
C
I
think
it's
really
important
that
that
we,
even
if
we
don't
agree
on
the
word,
nmda
spike,
that
we
agree
on
something
that
designate
this
as
a
different
phenomenon
to
a
calcium
spike
and
and
that
what
we're
trying
to
capture,
I
think,
is
the
locality
of
this.
The
space
matters
and
the
the
particular.
It
means
that
particular
synaptic
inputs
could
matter
if
it's
an
nba
spark,
okay
and
I'll
just
whiz
through
this
there's,
there's
actually
now
way
more
than
I
could
have
time
to
show
in
terms
of
evidence
for
clustering
aviva.
C
I
think
it's
it's
among
the
congress
anti,
certainly
at
the
the
gordon
dendrites
conference,
for
instance,
it's
it's
a
it's
a
done
deal
and
everyone's
convinced
that
clustering
occurs
and
and
therefore
the
nba
spikes
should
occur
and
that
this
has
effect
on
plasticity
and
learning.
So
that's
the
quick
summary
and
and
just
really
quickly,
also
because
I
promised
I
think
the
other
thing
that's
happened
in
the
meantime.
That's
really
important
in
terms
of
learning
and
dendrites
is
there's.
C
This
thing
called
btsp,
and
this
started
with
albert
lee
and
michelle
brecht
who
were
doing
recordings.
These
were
heroic
recordings
using
patch
clamps
from
a
mounted
fixation
device
onto
the
head
that
allowed
them
to
do
wholesale
patch
recordings
in
freely
moving
animals
and
and
when
albert
went
to
genealogy
farm
from
berlin.
C
He
started
doing
these
amazing
experiments
where
he
put
the
animal
in
a
track.
Well,
these
are
quite
conventional
experiments,
except
that
that
he
was
doing
a
wholesale
recording
from
a
ca1
neuron
at
the
same
time.
But
but
the
animal
was
running
around
on
a
conventional
track
and
you
can
see
that,
as
as
happens,
the
they've
discovered
here
again
a
place
cell,
meaning
there's.
C
If
ca1
cells
tend
to
spike
in
a
particular
place
and
every
now
and
again
or
quite
frequently,
actually
they
would
find
a
cell
where,
for
the
first
few
laps
it
it
had
no
place
and
that
that
happens
quite
a
lot
cell
has
no
particular
place.
But
in
this
case,
because
they
were
the
whole
cell,
they
could
actually
depolarize
the
neuron
and
they
could.
They
could
make
this
neuron
a
a
place
cell
at
at
some,
in
this
case,
random
place
on
the
track
and
and
the
the
you
have
to
stop.
Remember.
C
So
one
has
to
imagine
that
there
is
some
input
to
this
neuron
or
else
this
is
some
sort
of
spooky
action
at
a
distance,
but
but
the
but
the
point
I
think
the
only
reasonable
explanation
of
this
is
that
there's
some
dendritic
input,
that's
basically
invisible
at
the
cell
body
and
and
they're
waking
it
up
by
depolarizing
the
cell,
and
the
other
amazing
thing
about
this
is
that
the
cell
goes
completely
wild
when
at
this
place.
So
that
means
that
there's
a
tremendous
non-linearity.
C
This
is
much
more
than
than
just
a
30
change
in
a
few
synapses.
This
is
making
the
cell
go
from
from
completely
silent
to
to
jumping
out
of
the
dish.
So
it's
well
in
this
case
it's
in
the
brain,
but
but
but
it
goes
completely
wild
at
this
point
and
and
that
speaks
I
think,
to
a
a
huge
dendritic
non-linearity
occurring
so
at
gennalia
farm.
He
clearly
spoke
to
jeff
mcgee,
who
refined
this
experiment.
C
Jeff
mcgee,
I
think,
is
one
of
the
finest
researchers
I
know
of
and-
and
I
guess
this
came
easily
to
him-
to
think
of
how
to
adapt
this
experiment.
So
you
didn't
have
to
do
this
kind
of
heroic
approach.
They
then
had
fixed
the
animal
on
a
treadmill.
They
now
had
a
track
essentially,
and
they
could
repeat
these
experiments
and
sure
enough
they've
been
able
to
get
this
experiment
going
regularly
and
they
can
even
induce
this
now
with
with
with
with
plateaus.
C
In
this
case,
they're
spontaneous
or
in
this
case
they're
induced,
sometimes
they're
spontaneous
other
times
they
can
induce
the
the
plateau
and,
and
then
the
cell
goes
completely
nuts
at
a
particular
place
and
they
can
even
choose
where
that
place
is.
So
I
think
this
is
this
is
the
closest
we
come
then
to
one
shot
learning
and
it's
something
really
out
of
the
park.
C
And
then
the
question
is
what
happens
when,
when
input
comes
to
either
to
this
box
or
to
this
box
or
both
at
the
same
time,
and
that's
what
we've
been
concentrating
a
lot
on,
but
in
the
meantime,
what
I
think
became
clear
since
I
moved
to
berlin,
which
was
in
2012,
is
that
this
really
is
very
close
to
memory
in
some
sense,
because
it's
existing,
because
it's
it's
basically
what
you're,
what
your
brain
is
generating.
It's
got
to
have
something
to
do
with
your
previous
experience
and
therefore
memory
is
somehow
built
into
this.
C
The
other
thing
to
say
is
that
there's
lots
of
feedback
coming
from
lots
of
different
areas,
including
higher
order
thalamus,
and
I
won't
have
a
lot
of
time
to
dwell
on
that.
But
basically
we
can
show
through
these
are
imaging
now
of
of
transgenic
animals
or
virally
infected
animals
where
we
can,
where
we
can
basically
express
fluorescent
proteins
in
the
axons
of
particular
neurons
in
other
areas
of
the
brain,
and
then
you
can
see
them
forming
dense
projections
to
layer,
one
here
of
some
at
a
sensory
cortex.
C
The
last
projection
that,
because
presumably
the
idea
would
be
that
that,
if
the,
if
the
consolidation
due
to
the
the
activity
of
the
hippocampus
is
has
to
happen
in
the
neocortex,
then
there's
some
information.
That's
important!
That's
coming
along
this!
This
last
projection,
and-
and
here
you
can
see
us
some
some
experiments
in
which
we've
we
put
yfp
or
a
fluorescent
protein
in
the
peri
rhino
cortex,
and
you
can
look
in
the
neocortex
over
here
and
see
a
density
in
layer,
one
and
and
sure
enough.
C
C
This
is
one
you
know
where
you
stuck
your
electrodes,
so
you
don't
have
the
the
the
normal
problem.
One
has
with
memory
that
it's
really
distributed
over
the
whole
brain.
At
least.
We
expect
some
changes
to
happen
near
our
electrode
and
and
secondly,
the
animals
learn
this
extremely
quickly,
so
in
adjust
in
a
matter
of
ten
trials
and
they're
off,
and
then
they
can
do
this
really
well.
We
could
show
that
it
was
hippocampal
dependent
and
that
if
you
blocked
the
the
periodontal
cortex,
you
could
also
block
this.
This
behavior
okay.
C
So
moving
right
along
the
first
thing,
we
did
was
to
to
record
how
animals
learnt
this
task,
and-
and
this
is
just
kind
of
a
learning
curve
where,
if
they,
if
they
correctly
respond
to
the
stimulus
by
licking
for
the
reward,
then
then
it
goes
up
if
they
incorrectly
lick.
It
goes
down.
So
in
this
case,
you
can
see
that
they
learn
after
just
a
few
trials
and
and
a
control.
Animal
just
continues
to
get
it
right
for
the
rest
of
the
session.
C
And
that
means
we
can
suppress
the
axon,
and
that
means
we
can
do
this
whole
experiment
and
just
block
the
effect
of
the
perional
cortex
on
the
neocortex,
without
affecting
anything
else
without
affecting
the
action
of
the
periaronal
cortex
on
other
areas
and
so
on
and
so
forth.
So
we're
just
blocking
the
perional
cortex
and
a
bonus
is
that
we
can
put
the
drug
in
layer
one
of
the
cortex.
C
So
we
can't
even
ask
what
layer
is
important
here,
although
that's
not
necessarily
so
important,
because
the
projection
from
the
perional
cortex
anyways
goes
predominantly
to
layer
one
well.
Lo
and
behold,
when
we
put
the
chemical
which
happens
to
be
called
cno
into
layer,
one
of
the
cortex
here
and
block
the
peririnal
influence
on
on
the
cortex.
The
dam
animals
can't
learn
this.
It's
and
in
fact
it's
it's
a
little
bit
more
interesting
than
that
they're,
not
like
untrained
animals.
They
just
always
get
it
wrong,
they're.
C
They
they
kind
of
get
it
and
then
they
stop
getting
it
and
they
kind
of
get
it
and
they
stop
getting.
It
looks
a
little
bit
like
anti-grade
amnesia.
That's
really
hard
to
pin
down,
but
still
they
basically
they're
not
able
to
really
get
on
top
of
this
task.
Just
and
and
you've
got
to
think
about
what
this
is
just
by
putting
this
chemical
in
layer,
one
of
one
column
of
of
the
neocortex
we
can.
We
can
stop
the
behavior
of
the
animals,
it's
just
off
the
wall.
C
So
that's
what
the
the
collective
statistics
looks
like
and
and
last
thing,
which
is,
I
think,
just
as
important
as
all
the
rest
of
the
things.
I've
just
said
is
that
if
you
train
the
animal
first
and
then
you
put
in
the
the
then
you
block
the
periorenal
cortex
to
the
neocortex,
then
the
animals
do
perfectly
well.
C
In
fact,
they
do
even
better
because
at
this
point,
they're
doing
even
better
than
normal
control
animals,
because
they've
been
trained
a
lot,
but
but
basically
that
you
can't
block
their
their
performance,
which
means
that
the
perional
cortex
is
not
affecting
their
perception
of
this.
It's
it's
only
affecting
their
learning
of
this.
So
we
looked
further.
We
we
did
juxta
recordings.
C
That
means
basically
extra
cellular,
but
touching
recordings
from
the
cell
bodies
of
the
perihrinal
neurons,
and
we
want
to
know
what
do
they
do
during
the
task
and
basically,
when
the
animal
recognizes
you
stimulating
the
brain,
these
guys
go
crazy
and
they
start
burst
firing.
If,
if
the
animal
doesn't
recognize
it
then
nothing
happens,
that's
this
is
like
several
trials
in
a
particular
neuron.
You
can
see
all
the
the
neurons
going
crazy,
and
this
is
what
this
is.
C
Basically,
the
stats
over
many
neurons,
so
in
in
hits
you
get
you
get
a
big
change
in
the
in
the
fire
in
the
neuron
and
you
get
even
bigger
change
in
the
burstiness
of
the
firing.
I'm
afraid
I
run
out
of
time
to
dwell
on
that,
but
but
burst
burst
become
a
really
important
thing
in
this
study,
and
then
we
did
the
same
kind
of
recordings
from
the
from
the
7th
century.
C
C
So
so
it's
clear
that
the
the
input
to
to
the
neocortex
is
affecting
its
the
ability
to
change
the
firing
of
the
of
the
neurons,
even
though
the
firing
is
kicked
off
by
stimulating
s1
itself.
So
there's
a
there's,
an
electrode
in
s1,
which
presumably
is
kicking
off
the
firing.
But
if
you
don't
get
that
re-entry
input
during
learning
from
the
peribinal
cortex
and
you
can't
change
the
firing
of
the
units
and
the
same
thing
with
bursts.
C
So
there's
a
certain
fraction
of
these
are
modulated
by
their
bursts,
are
modulated,
I
should
say,
or
their
burstiness
is
modulated
and
not
when
you
block
the
the
the
periodical
input
so
going.
I
really
have
to
rush
here
sorry
about
this,
so
we
did
two
photon
imaging
from
the
dendrites
of
of
of
the
seminar
sensory
cortex
to
see
what's
going
on
in
the
parameter,
neuron
dendrite,
because
as
soon
as
you
see
burstiness,
you
should
think.
Oh,
maybe
the
dendrites
are
being
hit
and
sure
enough.
C
We
can
see
these
are
responses
in
the
dendrites
at
about
this
focal
plane.
That
means
around
about
where
you
would
normally
find
calcium
spikes.
You
can
see
in
hit
trials,
there's
very
often
a
response
in
the
dendrites,
and
we
can
see
that
these
different
populations
of
dendrites
coming
out
there's
some
that
go
crazy
with
lots
of
calcium,
some
that
actually
reducing
calcium
and
some
that
that
are
not
responsive,
and
these
were
the
stats
on
that.
C
C
Excuse
me,
I
guess
you
don't
see
that
okay,
so
when
we
recorded
from
the
cell
bodies
of
the
same
neurons,
well
the
same
depth
neurons
different
experiments,
we
get
exactly
the
same
statistics
for
on
cells,
that's
to
say,
firing
increasing
at
the
cell
body
decreasing
is
the
same
kind
of
proportion
and
and
not
responding,
and
the
question
was:
is
this
really
causal
and
that's
where
this
this
subject
gets
really
hard?
How
do
you?
How
do
you
show
that?
C
If
we
block
the
dendrites
of
the
of
the
pyramidal
neurons
and
the
other
thing
that
we
could
do
is
put
channel
adoption
into
then
and
dendrite
targeting
interneurons,
which
happen
to
be
called
somatostatin
neurons,
they're,
just
a
a
type
of
inhibitory,
neuron,
that's
known
to
target
the
apical
dendrites
and
again.
In
this
case,
we
can
really
effectively
block
the
learning
of
the
animal.
If
we
just
turn
on
the
light
and
stimulate
these
neurons,
while
the
animal
is
doing
the
behavior
so
summarizing
all
of
that
we've.
C
These
other
structures-
or
maybe
we
shouldn't
just
call
it
error
detection,
but
but
doing
something
special,
let's
say
forming
the
criteria
for
which
you
should
use
for
for
consolidating
memory
and
then
all
they
do
is
signal
layer.
One
to
say
now
is
the
time
to
consolidate
memory.
So
that's
that's.
If
you
like,
a
offering
a
a
a
a
way
of
conceiving
what
these
structures
are
doing
and
why
the
architecture
of
the
brain
is
the
way
it
is
and
effectively
what
that
would
mean.
C
Is
that
all
that
you
can
do
with
these
kinds
of
memory
is
effect
context
or
feedback,
so
you're
only
allowed
to
to
gate
the
effect
of
those,
and
you
don't
want
to
get
these
forward
connections
because
you
want
to
leave
the
feature
intact.
So
the
feature
should
stay
the
same.
It
should
just
react
to
a
new
context.
All
right
so
here
are
finished,
and
I
want
to
say
thank
you
to
all
the
people
involved
so
of
the
things
I
described
directly.
A
A
B
Alaina,
I'm
sorry
so
thanks
for
asking
yeah
I'll
give
a
brief
video
from
you.
So
elena
jones
is
a
neuroscience
phd
candidate
at
the
university
of
pennsylvania.
She
works
in
according
to
a
lab
with
coronado
courting.
She
was
previous
previously
affiliated
with
john
hopkins
stanford
and
dartmouth
college
and
her
research
focused
on
understanding
the
non-linear
computations
in
dendritic
integration,
while
guided
by
the
underlying
principles
of
deep
learning
and
ai,
and
we
are
really
happy
to
have
you
here,
alana
and
excited
to
hear
about
what
you
have.
E
E
E
You
can
define
it
in
terms
of
its
role,
so
what
do
neurons
do
to
mediate?
Behaviors
so
say
if
you
are
to
have
a
series
of
neurons
that
once
there
is
an
input
to
one
neuron,
there's
another
neuron
that
receives
that
output
and
so
on,
until
there
is
a
behavior
at
the
very
end,
if
you
were
to
remove
one
neuron
from
that
causal
chain
and
that
behavior
is
no
longer
there,
then
you
might
say
that
that
neuron
is
necessary
for
that
behavior
and
may
be
possible
for
it
to
mediate
that
behavior.
E
But
I'm
not
the
first
to
approach
neurons
as
infidel
devices
as
the
audience
should
know,
and
after
the
wonderful
talk
by
professor
larkin,
but
in
1943,
one
of
the
some
of
the
earliest
computational
neuroscientists
mcculloch
and
pitts.
They
produced
the
they
formulated
the
one
of
the
first
neuron
models,
which
is
the
mcculloch
pitts
neuron
model,
where
it
can
be
simply
summarized
as
the
linear
summation
of
inputs
in
weighting
those
inputs
and
putting
them
through
an
output
non-linearity.
E
E
E
So,
how
do
we
ask
what
neurons
can
do?
I
will
be
turning
to
neuron
modeling
and
I
will
talk
about
two
general
procedures
for
how
neuron
modeling
is
typically
done.
Well,
the
first
procedure
is
how
a
neuron
model
is
typically
done
in
a
traditionally
done,
and
then
I
propose
a
another
procedure
that
my
research
is
going
to
be
based
on
with
differing
benefits.
E
E
If
we
were
to
set
up
these
two
setups
empirical,
the
empirical
recording
based
setup
and
the
task
based
neuron
modeling
setup,
you
can
see
that
instead
of
a
somatic
current
or
a
gen,
an
input
current,
you
can
have
dendritic
task
inputs,
which
I'll
be
talking
I'll
clarify
in
a
second
and
instead
of
an
empirical
chord
recording
as
your
output.
You
have
instead
of
correct
classification.
E
So
what
kind
of
task
am
I
talking
about?
I'm
talking
about
a
binary
classification,
task
of
images,
and
I
do
binary.
I
choose
binary
classification
because
going
back
to
how
we
were
defining,
how
the
neurons
input
output
function,
the
output
of
a
neuron
insofar
as
we're
concerned
is
binary.
There
is
either
presence
or
an
absence
of
an
action
potential.
E
So
if
we
look
here,
we
have
several
different
image.
Data
sets
I'll
focus
here
on
mnist,
so
mnist
is
a
data
set
of
handwritten
numbers
from
numbers:
zero
through
nine
and
they
are.
There
are
10
classes
in
this
data
set.
However,
right
here
I
have
listed
only
two
classes
and
that's
what
I'll
be
using
for
my
binary
classification.
E
I
choose
these
classes
based
off
of
their
linear
separability.
I
used
a
linear,
classifier,
a
linear,
discriminant
analysis
to
determine
which
two,
which
pair
of
classes
were
the
least
literally
linearly,
discriminate,
tuple
and-
and
I
found
that
three
and
five
were
the
most
confused
between
the
two
and
lastly,
these
mnist
fashion,
eminist
e-m-ness
in
kuzu
cg
mnist.
E
I
have
padded
from
their
28
by
28
28
by
28
dimensions,
to
32
by
32
to
make
them
a
10,
24
dimensional,
and
I
also
use
c410
and
street
view
house
numbers
which
are
larger
in
size
and
usps,
which
is
much
smaller,
so
I've
defined
what
my
input
is,
which
is
these
images
with
high
dimensional
images,
and
I
also
defined
what
my
output
is,
which
is
these
binary?
The
binary
classification
of
these
classes,
so
three
and
five
would
correspond
to
zero
and
one,
and
so
now,
what
do
I?
E
E
So
what
I
do
is
that
I'm
trying
to
given,
I
have
a
task-based
neuron,
modeling
setup,
I'm
trying
to
understand
I'm
trying
to
understand
it's.
How
the
dendritic
sparsity
constraint
has
an
impact
on
performance
or
accuracy
in
this
task,
and
what
I'll
be
doing
is
I'll,
be
comparing
it
to
a
linear
point,
neuron
model,
or
rather,
what
I'll
be
doing
using
really
is
a
a
proxy
to
this
point,
neuron
model,
which
is
a
linear,
classifier,
linear,
discriminant
analysis
and
also
comparing
it
to
a
two
layer
fully
connected
neural
network.
E
They
perform
the
one
tree
performs
about
the
same
as
the
fully
connected
neural
network.
But
if
you
notice
here,
their
accuracies
are
actually
quite
close
to
50,
which
is
chance,
and
our
threshold
for
removing
trials
that
did
not
train
well
may
not
have
removed
every
trial.
That
did
not
train.
Well,
that's
creating
a
higher
variability
in
these
results.
So
I
wouldn't
actually
trust
these,
but
it
because
it's
so
very
difficult
to
interpret.
E
E
Great
thank
you.
So
what
else
could
impact
single
neuron
computation?
We
know
that
neurons
have
thousands
of
synapses
and
what
we
also
know
is
that
presynaptic
axons
can
synapse
up
to
approximately
four
times
per
postsynaptic
neuron.
So
that
means
that
there's
at
about
four
repeated
inputs
to
the
same
neuron,
and
we
also
know
that
in
vitro
multi-synaptic
batons
where
print
synaptic,
axons
synapse
multiple
times
onto
the
same
postsynaptic
neuron,
when
these
presynaptic
batons
onto
the
same
dendrite
increases
approximately
six-fold
after
ltp
induction,
which
implies.
E
I
think
that
learning
may
impact
replication
of
synopsis
and
we
would
think
that
learning
is
involved
with
neuron
computation
or
neural
computations
involved
with
learning.
So
the
general
hypothesis
that
comes
away
with
this
is
that
perhaps
repeated
synapses
may
also
impact
single
neuron
computation.
E
How
do
we
test
this?
Well,
we
can
modify
the
one
tree
apparatus
that
I
have
suggested
before
to
take
repeated
synapses.
So
if
we
look
to
the
left
here,
here's
the
cartoon
of
the
neuron
again
with
multiple
subtrees
and
each
subtree
is
receiving
the
same
exact
input,
one
two,
three
four
across
all
sub
trees
and
and
when
we
translate
that
to
the
deep
learning
model,
that
of
a
that
has
dendritic
dendritic
tree
constraints.
E
We
get
a
k-tree
neuron
model
where
each
sub-tree
in
this
cartoon
neuron
is
represented
as
another
as
a
as
an
added
one
tree,
making
a
k
tree
for
our
model.
So
where
k
is
equal
to
the
number
of
sub
trees
in
the
model.
One
thing
I
would
like
to
point
out
is
that
the
soma,
which
had
only
received
one
input
in
the
one
tree,
receives
now
k
inputs
in
the
k
train.
E
We
also
want
to
consider
that
the
k
tree
when
we
add
subtrees
increases
the
number
of
parameters
and
there's
a
general
rule
of
thumb
and
deep
learning.
The
more
parameters
you
have,
the
better
the
performance
of
your
deep
learning
network
kind
of
so
we
wanted
to
control
for
this.
So
what
we
did
was
we
modified
the
fully
connected
network
to
match
the
k
tree
parameter
size.
So
these
expressions
here
determine
are
are
equal
to
the
number
of
parameters
that
each
of
these
models
have.
E
So
the
k
tree,
where
n
is
equal
to
the
size
of
the
input
and
k,
is
equal
to
the
number
of
sub-trees
result
in
2k
times
the
quantity
of
n
minus
one
parameters
and
the
fully
connected
neural
network
where
it
receives
an
inputs
and
has
a
hidden
layer.
Size
h
has
h
times
the
quantity
of
n
plus
one
parameters
and
when
you
set
h
to
be
equal
to
twice
the
number
of
sub
trees
or
2k,
then
these
do
not
quite
equal,
each
other
or
but
approximately
equal
to
each
other.
E
So
in
some
cases
this
is
not
the
case
so
for
kmns
and
svhn,
and
in
one
surprising
case
and
cfar10,
the
performance
of
the
kr
actually
exceeds
that
of
the
fully
connected
neural
network,
though
the
fully
connected
neural
network
seems
to
decrease
in
performance
as
you
increase
the
hidden
layer
size.
So
the
takeaway
from
this,
the
two
takeaways
from
this
is
that
synaptic
repeats
can
improve
k-tree
performance
and
the
and
this
performance
can
also
match
that
of
the
fully
connected
neural
network
that
is
comparable
in
size.
E
So
some
discussion
and
takeaways
I
want
to
address
some
assumptions
and
limitations,
so
one
general
limitation
to
this
is
that
I
I'll
just
make
a
blanket
statement.
This
is
not
a
realistic,
I'm
not
starting
from
a
point
of
realism,
biological
realism,
I'm
starting
from
a
point
of
abstraction
and
moving
toward
biological
realism.
E
E
E
This
is
a
special
case
of
sparse
neural
network.
The
k-train
the
sparse
network
can
perform
at
the
level
of
the
dense
network
using
nearly
the
same
amount
of
free
parameters,
and
future
work
is
to
compare
the
k-tree
sparse
network
to
other
types
of
sparse
networks
such
as
randomly
connected
sparse
networks
or
prunes.
First
networks,
harkening
back
to
the
lottery
ticket
hypothesis,
the
one
used
in
the
lottery
hot
ticket
hypothesis
paper
and,
lastly,
the
relevance
to
neuroscience.
E
We're
bringing
a
new
approach
where
we're
considering
bringing
deep
learning
approaches
toward
neuron
modeling,
where
we
judge
neural
model
performance
instead
of
traditionally
trying
to
match
neuron
physiology
data,
and
our
results
suggest
that
repeated
input,
suitability
and
tradition
improve
its
ability
to
perform
complex
computations.
So
it
may
be
a
important
takeaway
for
modelers
to
also
consider
the
possibility
of
repeated
inputs.
E
Some
future
work
we
would
like
to
do
is
to
introduce
more
biological
constraints
to
our
abstracted
model,
as
well
as
jumping
to
taking
a
leap
into
biological
realism
by
training,
biophysical
neuron
models
to
do
image.
Classification
kind
of
like
a
paper
that
came
out
of
the
cigarette
this
year
last
year.
So
with
that,
I
thank
the
courting
lab
very
supportive.
B
E
E
B
Yup,
I'm
here
right
so
with
a
freak
so
like
richard
reporter
richard
graduated
from
the
university
around
the
university
of
oxford
and
he,
the
principal
investigator
link
lab
and
a
professor
at
the
university
of
toronto.
He
explores
the
neurobiology
of
learning
and
memory
with
a
great
goal
of
understanding
how
experience
alters
snap
connections
in
the
brain
and
blake
is
a
very
well
known
researcher,
both
in
neuroscience
and
machine
learning
world,
and
I
myself
had
the
pleasure
of
watching
him
speak
at
several
occasions
in
conference,
and
we
are
really
happy
to
be
here.
F
F
Now
what
I've
planned
for
today
is
to
actually
try
to
inject
a
little
bit
of
controversy
into
the
lovin
by
suggesting
that
the
direction
in
which
we
can
get
help
from
dendrites
is
not
necessarily
from
neuroscience
to
ai.
It's
actually
ultimately
kind
of
the
other
direction
from
ai
to
neuroscience,
and
that's
where
dendrites
are
important,
but.
F
F
Now,
let's
just
right
off
the
bat
say
something
that
I
think
at
this
point
in
time
is
broaching
on
non-controversial
within
the
neuroscience
community.
Dendrites
are
clearly
important
for
the
brain
I
mean
just
based
on
basic
biophysical
principles.
We
know
that
dendrites
will
determine
what
the
computational
capabilities
of
an
individual
neuron
are
and,
furthermore,
from
the
time
of
ramoni
kahal,
whose
illustration
we
see
here,
we've
known,
that
different
neurons
in
different
parts
of
the
brain
have
very
different
dendritic
structures,
and
this
surely
relates
to
the
different
computational
roles
of
these
different
neurons.
F
So,
as
a
neuroscientist,
you
know
one
would
be
crazy
to
ignore
dendrites,
even
though
that
was
what
we
did
for
many
decades,
as
matthew
pointed
out
until
we
developed
the
experimental
tools
necessary
to
really
work
with
them.
But
you
know,
for
the
sake
of
this,
talk,
we're
not
necessarily
interested
in
just
understanding
the
brain.
We're
interested
in
whether
or
not
dendrites
can
help
us
to
build
brain-inspired
ai.
F
Now,
in
order
to
answer
that
question,
I
think
it's
important
to
differentiate
between
the
question
of
algorithm
and
implementation
and
this
harkens
back
to
mars
levels,
but
I'm
not
going
to
go
over
mars
levels
too
much.
I
will
just
clarify
here
that
there
are
ultimately
two
ways
in
which
brains
can
inspire
artificial
intelligence,
so
the
first
and
the
one
that
really
arguably
has
had
the
most
traction
so
far
to
date
is
actually
algorithm
development.
F
We
can
build
artificial
intelligence
that
uses
brain-like
recipes
in
order
to
process
information
and
do
calculations
and
that's
effectively
what
artificial
neural
networks
are.
They
are
an
algorithm
level
model
of
how
the
brain
works,
because
they
capture
the
key
steps
involved
in
neural
calculation,
namely
parallel
distributed
processing
across
a
large
number
of
simple
units.
F
F
And
this,
of
course,
you
know,
has
to
do
with
neuromorphic
chip
development,
which
a
lot
of
people
are
interested
in
but
which
to
date,
hasn't
actually
sort
of
taken
off
outside
of
academic
circles,
and
I
think
that
it's
you
know
I
I
would
propose
and,
as
I
said,
I'm
not
100
sure.
I
fully
agree
with
myself
here,
but.
C
F
I
do,
for
the
most
part
that
dendrites
have
actually
the
most
potential
for
helping
ai
with
respect
to
number
two
chip
development
and
not
necessarily
with
regards
to
the
question
of
algorithmic
development
and
the
reason
I'd
say.
That
is
because
I
think
that
ultimately,
dendrites
are
solving
implementation
problems
for
the
brain.
They
are
not
key
to
the
algorithmic
properties
of
the
brain.
They
are
the
way
in
which
the
brain
successfully
implements
algorithms
that
could
theoretically
be
implemented
in
other
ways.
F
So
let's
actually
talk
about
what
dendrites
really
achieve
for
the
brain,
and
now
matthew
and
alana
have
already
done
a
great
job
of
highlighting
some
of
the
interesting
things
that
dendrites
can
do,
but
I'll
kind
of
give
a
pat
lib
answer
here
for
a
moment,
and
that
is
that
there
are
two
principal
things
that
dendrites
solve
for
the
brain.
The
first
is
that
they
solve
just
the
packing
issue
right
I
mean
where
we
all
tend
to
think
of
those
funny.
F
Computer
rendered
images
of
neurons
floating
in
space
little
electrical
signals
passing
between
them
and
stuff,
but
of
course
the
real
brain
is
a
crowded
place
and
if
you're
actually
going
to
pack
thousands
of
synapses
onto
a
neuron,
you
need
space
to
do
it
so
right
off
the
bat
dendrites
are
a
way
to
provide
more
physical
sites
to
be
synapsed
onto
and
to
allow
for
greater
connectivity
here,
you're
looking
at
an
image
from
a
rainbow
mouse
now.
This
is
a
particularly
dense
area.
F
F
So
it's
ultimately
important
at
the
algorithmic
level
to
stage
certain
calculations
so
that
they're
done
in
isolation
from
each
other
up
until
a
certain
point
and
then
combined
and
also
to
apply
the
right
non-linearities
at
the
right
moment
and
dendrites
help
us
to
achieve
this.
So
if
we
take
the
model
that
matthew
described
where
we
have
here,
a
pyramidal
neuron
like
a
real
pyramidal
neuron,
illustrated
on
the
left.
F
Its
dendrites
have
these
interesting
nonlinear
properties,
either
driven
by
sodium
channels
or
nmda.
Sorry,
by
nmda
channels.
Sodium
channels
are
for
the
axon,
by
nmda
channels
or
by
calcium
channels,
and
we
can
view
these
non-linear
channels
in
these
dendrites
as
a
as
giving
us
this
sort
of
layered
non-linear
computation.
So
we
have
a
series
of
nonlinear
computations
that
can
act
together
in
isolation
from
the
other
computations
that
are
occurring,
and
this
gets
at
the
point
of
space.
F
The
fact
that
when
dendrite,
when
synapses
are
located
close
to
each
other
on
in
on
a
dendritic
branch,
they
are
necessarily
going
to
be
involved
in
an
initial
calculation
together.
So
we
have
these
these
different
branches
that
group
different
computations
and
which
can
allow
for
a
series
of
non-linearities
via
nmda
channels,
and
then
these
will
be
combined
and
integrated
by
either
the
cell
body,
where
sodium
non-linearities
will
determine
spiking
activity
or
at
the
apical
trunk,
where
calcium
non-linearities
will
determine
bursting
potentially
and.
F
Now
I
want
to
give
you
today
an
example
of
how
I
think
this
is
really
important.
So
from
a
neuroscience
perspective,
I
just
want
to
reiterate
that
this
is
like
absolutely
key
and
an
example
of
how
this
is
key
is
the
question
of
how
we
can
combine
error
signals
with
other
information,
whether
it
be
sensory
or
motor
or
whatever.
F
F
You
want
that
error
signal
to
be
combined
with
your
other
information
in
an
appropriate
manner.
You
don't
want
to
just
add
them
together.
Linearly,
if
you
do
that,
if
you
just
add
your
signal
x
with
your
error
e,
then
you're
going
to
get
a
confused
computation
as
it
were.
Where
now
the
neuron
starts
treating
the
error
signal
as
if
it
was
part
of
the
sensory
representation.
F
F
Just
by
separating
the
signals
in
time,
so
you
could
have
your
signal
x
occur
and
then
your
error
occur
later,
but
the
difficulty
that
you
still
face,
if
you
do
this
temporal
segregation
of
the
signals,
is
that
you
are
still
going
to
not
be
able
to
dif.
It's
still
going
to
be
difficult
to
differentiate
the
two
signals
at
the
biophysical
level.
F
F
Dendrites
solve
these
problems
for
us
by
allowing
us
to
receive
things
like
an
error
signal
at
an
alternate
location
on
the
neuron
that
is
segregated
and
so
that
the
computations
that
are
going
to
happen
with
the
error
can
potentially
happen.
In
isolation
before
they
get
combined
with
the
sensory
information,
moreover,
the
non-linearities
that
exist
in
the
dendrite
will
allow
us
to
actually
have
the
cell
spike
in
different
ways
to
have
different
activity.
Waveforms
that
are
indicative
of
dendritic
activity
and
they're
by
potentially
error
signals.
F
So
matthew
mentioned
this,
you
can
have
a
situation,
for
example,
where
a
basic
bottom-up
input
to
a
neuron
will
generate
a
single
spike.
But
then,
when
you
combine
that
with
some
top-down
input,
you
get
a
complex
spike
or
a
burst,
or
something
like
that,
and
that
can
have
a
different
signal
for
your
plasticity
mechanisms.
F
Indeed,
we
see
this
happen
in
the
the
classic
example
of
this
is
in
the
cerebellum,
so
cerebellar
purkinje
cells
have
these
beautiful,
dendritic
trees.
We've
all
seen
them
many
times
before,
and
these
dendritic
trees
receive
inputs
from
the
parallel
fibers
that
carry
us,
carry
various
motor
information
and
potentially
sensory
information
as
well,
and
then
there's
this
interesting
projection
called
the
climbing
fiber,
which
actually
wraps
around
purkinje
cell
dendrites
in
a
one-to-one
fashion.
F
So
one
climbing
fiber
to
one
purkinje
cell
and
when
you
activate
the
parallel
fibers
you'll,
just
get
a
standard
spike
from
these
purkinje
cells.
But
if
you
activate
them
together
with
the
climbing
fibers,
you
get
this
comp.
What
they
call
a
complex
spike
out
of
the
purkinje
cells
and
what's
been
shown,
experimentally,
is
that
if
you
run
a
synaptic
plasticity
experiment,
then
if
you
just
stimulate
the
parallel
fibers,
you
will
get
an
increase
in
the
synaptic
weights
from
those
parallel
fibers
up
to
some
ceiling.
F
But
if
you
stimulate
them
together
with
the
complex
fibers,
you
then
decrease
those
weights
and
you
get
ltd,
and
this
complex
fiber
activation
and
the
complex
spike.
Sorry,
the
climbing
fiber
activation
and
the
complex
spike
that
it
initiates
is
critical
for
this.
In
the
absence
of
this,
you
don't
get
this,
and
so.
In
other
words,
the
neuron
is
using
the
non-linear
activation
that
the
complex
spike
induces.
F
Sorry
that
the
climbing
fibers
induce
in
the
dendrites
it's
using
that
nonlinear
activation
to
differentiate
the
error
signal
from
the
climbing
fibers
and
the
inputs
from
the
parallel
fibers
and
we've
proposed
that
this
is
something
that's
happening
in
pure
middle
neurons
in
the
neocortex
as
well.
So
I
don't
have
to
go
over
this
too
much
because
nothing
discussed
it,
but
basically
we
have
in
pure
middle
neurons.
These
you
know
two
different
sort
of
dendritic
areas:
the
the
basal
tree
and
the
distal
apical
tree.
F
I'm
going
to
ignore
the
oblique
dendrites
here
for
a
moment
and
between
these
two
sets
of
dendritic
branches
which
are
receiving
the
feed-forward
and
feedback
inputs
respectively
to
some
degree
it's
biology.
So
it's
always
messy.
We
have
this
active
calcium
zone
which
can
induce
nonlinear
activations.
F
Now
this
has
been
shown
to
some
degree.
I
mean
matthew,
described
some
of
the
interesting
experiments
that
way
with
behavioral
time
scale,
plasticity
showing
that
you
can
induce
this.
This
effect
there,
but
we
have
recently
proposed
a
kind
of
mathematical
version
of
this
so
based
on
a
number
of
experimental
findings,
including
the
more
recent
ones,
but
also
looking
further
back
into
the
past
and
and
also
based
on
matthew's
work.
We
proposed
in
a
recent
paper
that
I
I
list
here
and
I'm
happy
to
provide
I'll
share.
F
We
propose
the
following
learning
rule,
which
we
call
burst,
dependent,
synaptic
plasticity,
and
the
idea
is
effectively
that
we
have.
This
is
the
weight
change
between
neurons
I
and
j.
This
is
the
learning
rate
is
going
to
be
the
only
equation
I
run
through
here,
but
just
to
get
you
to
understand
the
rule.
F
The
basic
idea
is
that
if
you
have
presynaptic
inputs,
then,
if
the
postsynaptic
cell
fires
a
single
spike,
you
are
going
to
get
a
ltd
occurring,
whereas
if
the
postsynaptic
cell
fires
a
burst,
you
are
going
to
get
ltp
occurring.
So
this
learning
rule
is
what
expresses
that,
because,
basically
here
we
have
the
postsynaptic
burst,
which
is
a
one
or
a
zero,
and
this
is
com.
This
is
we
subtract
from
this
the
running
average
of
burst
probability.
F
So
what
happens
is
if
the
neuron
bursts?
This
term
becomes
positive
and
you
get
ltp.
If
it
doesn't
burst,
then
this
entire
term
is
negative
because
b
is
zero,
and
so
you
get
ltd
and
then
otherwise.
We
have
here
just
something
indicating
that
something's
occurred,
postsynaptically
and
a
presynaptic
eligibility
trace,
and
you
can
capture
a
lot
of
experimental
findings
with
this
learning
rule.
F
So
this
here
is
showing
the
low
level
input.
So
what
we're
doing
is
we're
training
the
system
to
do
xor,
so
we
provide
either
this
group
of
neurons
with
a
pulse
of
current
or
this
group
of
neurons,
with
a
pulse
of
current
or
both.
So
we
get
zero
zero
one,
zero
zero
one,
one
one,
and
if
we
we
do
this
whole
setup
and
then
we
provide
a
teaching
signal
to
the
apical
dendrites
of
these
guys.
F
It
just
says
like
basically,
okay,
we
want
you
to
activate
we're
going
to
give
you
a
strong
input
when
you
should
be
giving
an
output
here
of
one
in
order
in
response
to
one
zero
or
zero
one
or
we're
going
to
give
you
a
low
output
in
response
to
zero,
zero
or
one
one
over
time
in
a
totally
continuous
manner.
The
network
will
learn
to
do
xor
for
you,
and
this
just
falls
out
for
free
from
the
way
that
the
burst,
depending
first
dependent,
synaptic
plasticity
rule
interacts
with
the
non-linearities
in
the
dendrite.
F
So
here
you
see
the
output
of
the
network
before
training
and
after
training,
and
the
only
thing
we've
done
to
train
this
up
is
provide
this
teaching
signal,
which
is
current
injected
into
the
upper
level.
Apical
dendrite
and
the
reason
that
works
is
because
of
the
way
the
dendrites
all
communicate
with
one
another.
So
if
they
project
back
to
the
apical
dendrites
of
these
neurons,
what
you
end
up
getting
is
that
this
teaching
signal
gets
reflected
in
the
activity
of
the
these.
These
neurons,
and
so
here
bp,
refers
to
burst
probability.
F
So,
even
though
input
one
and
input
two
are
ultimately
seeing
the
same
data,
they
end
up
diverging
in
terms
of
how
they
respond,
and
this
is
critical
because
standard,
heavy
and
mechanisms,
for
example,
will
always
treat
the
same
inputs
the
same
way
and
it
doesn't.
There
is
no
teaching
signal
involved,
so
you
can't
differentiate
these
different
cases,
but
with
the
non-linearities
that
the
dendrites
induce
you
now
get
a
teaching
signal
for
free
and
you
can
actually
train
these.
These
things
up
to
do
a
variety
of
things.
F
We
contrast
that
with,
for
example,
node
perturbation,
which
is
a
classical
approach
that
instead
just
looks
at
basically
correlations
and
activity
with
an
external
reward
signal,
and
it
sucks,
and
it
also
does
better
than
than
feedback
alignment,
which
is
something
a
friend
of
mine,
tim
willycroft
had
hit
upon
a
few
years
ago,
which
is
where
you
just
use
fixed
feedback
weights,
and
you
just
do
back
prop
and
still
it's
it's
not
great
for
this
similar
story
with
imagenet.
So
you
can
train
up
these
networks.
F
F
Does
it
doesn't
even
do
this,
so
my
point
is
ultimately
that
the
properties
of
apricot
dendrites
provide
a
means
for
combining
error
signals
with
other
signals
appropriately,
and
this
actually
could
allow
neurons
to
follow
loss
gradients
and
in
doing
so,
this
solves
a
potentially
tricky
implementation
problem
for
the
brain.
How
do
you
differentiate
your
error?
Signals
from
your
sensory
signals
in
a
way
that
allows
you
to
actually
learn
appropriately
from
the
error
signals.
F
The
other
question,
though,
is
what
about
algorithms.
So
what
about?
If
we
just
simulated
dendrites,
if
so
so,
rather
than
actually
trying
to
build
like
physical
dendrites
on
a
chip,
we
just
simulated
dendrites
for
the
sake
of
achieving
the
same
algorithmic
capabilities
that
neurons
have
in
order
to
get
us
closer
towards
ai,
that's
more
human-like
or
animal-like.
Even
you
know.
Theoretically,
this
isn't
a
crazy
proposal.
We
know
from
the
no
free
lunch
theorems
from
wilpert
and
mcgregor
that
you
cannot
have
a
general
purpose
learning
algorithm.
There
is
literally
no
such
thing
that
exists.
F
If
you
look
at
the
performance
of
any
learning
algorithm,
a
general
purpose.
Learning
algorithm
that
is
kind
of
good
at
any
task
will
also
kind
of
suck
at
every
task
simultaneously,
and
the
only
way
you
can
get
really
good
performance
is
by
having
algorithms
that
are
specialized
towards
particular
type
of
learning
tasks.
F
So
when
we
think
about
the
group
of
learning
tasks
that
we
might
want
to
learn
in
ai,
arguably
a
lot
of
what
we
want
to
do
with
ai
intersects,
a
fair
bit
with
the
sorts
of
things
that
humans
are
particularly
good
at
and
even
macox
and
mice
and
drosophila
to
some
extent
in
any
animal.
So,
theoretically
we
could
use.
F
That's
helping
us
to
understand
the
computations
that
are
occurring
in
the
brain
and
potentially
the
algorithmic
constraints
that
exist,
but
that
doesn't
necessarily
mean
that
you
actually
have
to
simulate
dendrites
in
order
to
achieve
those
algorithms
once
you've
identified
the
algorithm
and
once
you
understand
the
algorithm
that
the
brain
is
using,
you
can
implement
it.
However,
you
want
the
analogy
that
I
would
use
is
is
with
flight
and-
and
I
use
this
also
to
correct
a
poor
form
of
the
analogy
that
sometimes
gets
shared
around.
D
F
And
indeed,
the
earliest
design
of
airplanes,
the
people
who
are
building
those
they
were
inspired
by
birds
and
they
help
birds,
help
them
to
understand
the
need
for
creating
lift
and
how
to
do
that.
But
then,
once
they
understood
the
algorithm
that
birds
were
using,
they
didn't
have
to
use
flapping
wings.
They
could
use
other
means
of
achieving
this
algorithm
and
indeed,
jets
ended
up,
providing
a
much
better
way
of
achieving
lift.
So
I
would
argue
that
it
might
end
up
being
something
similar
with
dendrites.
F
But
does
that
mean
we
want
to
simulate
dendrites
for
the
ai,
I
would
argue,
probably
not.
There
will
probably
end
up
being
better
ways,
given
our
particular
technology
setups
to
achieve
the
same
algorithms
that
dendrites
can
achieve
for
these
artificial
systems
that
we
have
so
to
kind
of
sum
up.
If
you're
a
neuroscientist,
I
think
you're
crazy
to
ignore
dendrites.
F
A
A
I
guess
one
thread
that
was
kind
of
running
through
some
of
the
discussion
is,
you
know,
matthew
and
and,
like
you
talked
about
kind
of
learning
through
apical
feedback
and
apical
dead
rights,
but
there
are
you
know
there
are
other
learning
mechanisms
that
are
also
present
in
pyramidal
neurons,
that
don't
necessarily
incorporate
apical
fitbit
feedback,
and
you
know
how
do
you
think
about
that
and
is
that
are
those
critical?
Are
they
do
they
exist?
A
A
Yeah
exactly
or
you
know,
stuff
I've
seen
is
like
an
nda
spike.
A
basal,
an
nba
spike,
followed
by
a
back
action
potential,
could
cause
a
very
localized
form
of
plasticity.
Independent
of
you
know,
calcium
spikes,
those
kind
of
things
and
how
do
those
kind
of
fit
into
your
overall
model
of
of
learning?
So.
C
So
right
at
the
moment,
we
haven't
we're
we're
agnostic
about.
What's
going
on
in
the
in
the
particular
experiments
I
was
showing
and
in
principle
could
be
any
of
those
things.
I
think
if,
if
something
like
btsp
is
a
play,
then
then
we're
looking
at
something
quite
different,
although
they
they
do
claim
that
there
is
a
plasticity
change
in
their
experiments.
In
fact,
it's
built
into
the
the
name.
C
For
me,
it
smells
like
that
to
me:
that's
what
my
intuition
would
be,
but,
but
we
don't
know
and
then
and
then
of
course
it
could
be
any
or
all
of
the
ones
you
mentioned
so
so
it
might
be
a
a
complex
mix
of
those
and
and
and
not
to
leave
out
also
the
the
synapses
that
inhibitory
neurons
as
well.
So
so
there's
there's
quite
a
lot
of
possibilities.
C
In
fact,
and-
and
I
I
from
my
perspective,
the
importance
of
of
where
we're
at
now
with
this
study
is
that
we
know
where
to
look
so
so
if,
if
there
is
something
special
going
on,
I
think
we've
got
the
the
paradigm
where
we
can
isolate
it
and
find
out.
If
these
things
are
in
fact
happening
in
that
location.
A
Yeah,
the
experimental
paradigms
are
pretty
amazing,
should
be
able
to
answer
some
of
these
questions.
Let's
see
another
set,
a
specific
question
for
you:
is
there
any
change
in
the
resting
membrane
potential
when
peririnal
input
is
blocked
and
or
the
parallel
input
is
changing?
The
conductance
state
of
the
cortical
pyramidal
neurons.
C
So
the
the
the
baseline
firing
goes
up
during
the
task
in
a
in
a
control
animal
and
if
you
and
and
and
this
increases
over
learning-
and
if
you
block
the
learning
with
of
the
period
input,
then
then
by
and
large
the
baseline
doesn't
shift
so
yeah.
There
are
changes
in
the.
So
I
guess
that
I
didn't
get
a
chance
to
dwell
on
it.
C
But
but
what's
going
on,
is
that
there's
a
there's,
a
a
micro
stimulus,
meaning
an
extracellular
stimulus
of
the
of
the
tissue
in
layer,
five
that
is
repeated
in
a
cycle
once
every
10
seconds,
or
so,
if
the
animal
licks
in
a
in
a
window,
that's
too
close
to
the
stimulus,
then
the
whole.
The
whole
trial
is
aborted.
So
sometimes
it's
there's
no
stimulus.
But
basically
this
is
turning
around.
C
A
G
B
F
One
of
the
important
things
to
remember
is
that
there
are
likely
multiple
learning
rules
impinging
on
the
brain
and
I
think
there's
this
funny
tendency.
We
have
in
neuroscience
to
desire
a
sort
of
physics-like
theory
of
everything,
but
I
don't
think
that's
you
know.
Larry
abbott
said
this
to
me
a
few
years
ago
and
I've
really
internalized
it.
F
F
C
C
A
Yeah
one
of
the
maybe
I
can
ask
the
converse
of
the
the
high
level
question
here-
is
that
you
know
apical
dendrites
have
been
implicated
in
you
know
in
matthew,
in
your
lab
and
just
perception
by
itself,
without
any
learning
as
well,
whereas
I
think
blake
you're
gonna
your
model,
you
know,
do
you
have
a
place
for
apical
dendrites?
That
is
not
learning
related.
That
is
just
purely
an
inference
in
perception.
A
Yeah,
let
me
ask
a
high-level
question
for
all
three
of
you.
You
know:
have
you
seen
any
that's
from
the
q
a
thing
here?
Have
you
seen
any
work
that
tries
to
model
network
dynamics
with
model
neurons
that
incorporate
features
like
nmda
spikes,
backfiring,
btsb,
inhibitory
neurons?
Is
it
even
possible
to
conduct
such
work
now
or
are
we
missing
crucial
experimental
data
for
it.
C
I
would
I
can't
really
be
conclusive
about
machine
learning
world
because
I'm
enormous,
but
but
but
I
I
guess
that
the
key
here
is
to
is
to
at
least
from
an
experimentalist
point
of
view
is
to
find
out
what
the
what
learning
rule
or
each
of
these
things
translates
to
and
and
and
that
kind
of
going
along.
The
way
that
that
blake
was
saying
in
his
talk
in
the
end.
C
I
should
say
looking
at
the
the
nmda
spike
world
and
looking
at
just
how
hard
it
is
to
see
any
plasticity
at
all.
There
was
a
so
jackie.
Schiller
has
a
couple
of
papers
that
show
some
interesting
plasticity,
but
but
in
one
case
she
had
to
have
bdnf
present
to
get
some
plasticity
at
the
basal
dendrites
and
in
another
case
it
was
a
very
strange
kind
of
flip
of
for
low
frequency
stimulus
of
the
tuft.
C
There
was
a
sudden
change
in
plasticity
that
is
possibly
related
to
potassium
channels
and
and
is
is
just
very
strange
at
this
point
doesn't
look
like
anything
we
see
in
a
normal
learning
context
and
then
there's
btsp
as
well,
for
which
we
also
don't
know
the
the
substrate.
So
again
my
it
smells
to
me
like
there's
a
there's,
there's
some
a
factor
that
we're
missing
that
hopefully
we'll
stumble
across
or
we'll
figure
out
and
and-
and
I
would
guess
that
that's
transformative
will
be
transformative
to
that.
C
If
that,
once
you
would
know
what
that
factor
is
then
they'll
be
well
that'll
be
a
big
day,
whether
or
not
as
blake
suspects.
That
would
just
mean
a
simple.
You
know
tune
tune
the
model
a
little
bit
this
way,
but
but
basically
nothing
to
see
here
kind
of
thing.
It's
just
a
biological
anecdote
or
whether
or
not
that's
conceptually
going
to
change
the
algorithm
as
blake
puts
it
it's.
I
guess
I
would
still.
I
wouldn't
want
to
call
it
at
this
point.
A
Yeah
alana
you've
done
a
great
job,
answering
all
the
questions
on
the
q
a
so
maybe
I
can
ask
a
very
sort
of
high
level
one
for
you.
You
know
what,
where
do
you
want
to
go
next
with
your
work
and
are
there
other
properties
of
dendritic
function
that
you're
trying
to
integrate
in
or
where?
Where
do
you
want
to
take
your
work?
Next.
E
E
I
think
my
next
step
is
to
try
to
look
at
the
activation
functions
of
the
the
this
kind
of
in
this
more
morphological
constraint,
set
up
and
try
to
understand
how
you
know.
Perhaps
if
I
were
to
make
that
a
bit
more
biologically
realistic
how
that
could
potentially
impact
performance
as
well
and
some
other
things
that
I
kind
of
really
want
to
happen.
Is
this.
E
It
comes
from
this
general
concept
of
you
know,
building
a
ruler,
so
the
task
based
modeling
is
in
a
way
allowing
me
to
measure
in
some
way
how
well
something
is
able
to
do
a
particular
kind
of
task.
I
want
to
further
pursue
that
kind
of
a
way
of
thinking
in
trying
to
determine
the
limits
of
how
any
particular
dendritic
property
or
how
a
dendrite
might
be
able
to
perform
a
particular
kind
of
function
or
computation,
and
I
I'm
still
working
on
how
to
implement
that.
A
C
Just
a
couple
of
thoughts,
on
the
one
hand
it
it
feels
a
little
bit
like
the
the
turing
machine
analogy
to
computers
that
I
think
that's
kind
of
analogous
to
the
the
claim
that
that
that
blake
is
making,
essentially
that
all
computers
can
be
whether
the
turing
machine
is
universal
and
and
eventually
we
learn
nothing
except
engineering
by
by
improving
the
chips
and-
and
it
all
boils
down
to
to
the
same
algorithm.
In
the
end,
I
think
that's,
I
hope
that's
that
sort
of
captures
the
main
idea
that,
in
the
end,
there's
nothing.
C
And
that
I
I
I
think,
that's
that's
quite
likely
in
some
sense
that
that
it
all
is,
there
would
be,
let's
say,
multiple
substrates
or
ways
to
implement
dendrites.
That
doesn't
mean
that
you
have
to.
There
has
to
be,
you
know,
really,
obviously
simulation
or
even
analogous
to
to
a
dendrite,
and
I
think
he
was
asking
nevertheless
the
right
question:
do
we
do
we
learn
anything
algorithmic,
there's
still
some
chance.
C
And
basically
this
was
there
was
workshop.
It
must
have
been
six
seven
years
ago
now
and
asking
the
same
question,
and
I
remember
that
the
general
feeling
coming
out
of
that
was
well.
It
all
boils
down
to
learning
rules
and
cost
functions,
and
that's
what
the
the
biologists
should
be
looking
at
and
because
they're
not
used
to
the
to
the
machine
learning
world
unless
they're
they're,
focusing
on
on
those
things
and
what?
What
kind
of
grates?
For
me
with
that.
C
That
view
is
that
one
of
the
things
that
seems
to
be
least
to
a
naive
observer
of
the
machine
learning
world
missing
in
most
machine
machine
implementations
and
perhaps
actually
not
in
a
in
a
hierarchical
network?
But
but
what
seems
to
be
missing
is
is
a
what
I
would
want
to
call
a
semantics,
that's
built
into
the
actual
brain,
but
but
isn't
usually
in
a
in
a
neural
network
in
a
neural
network.
It
really
doesn't
matter
if
which
neuron
is
1047.
C
It's
it
doesn't
matter
if
it
next
week,
it's
1065,
whereas
in
the
brain
it
really
does
matter
that
the
the
columns
have
a
a
function
and
that's
built
in
genetically
by
how
they're
wired.
And
although
that
you
could
rewire
that
it's
it's
not
rewired,
and
so
that
means
that
that
there's
a
there's
a
a
consequence
of
changing.
Let's
say
getting
your
your
scalpel
and
taking
out
this
part
of
the
brain
and
that
part
of
the
brain
and
therefore
there
will
be
consequences
to
to
rules
that
are
implemented
in
particular
parts
of
the
brain.
C
C
C
So
if,
if
there
are
learning
rules
that
really
need
to
be
segregated
and-
and
you
dare
not
just
let
happen
everywhere
all
over
the
network,
but
that
it
really
matters
that
the
prefrontal
cortex
has
let's
say
the
possibility
to
fire
for
three
seconds
at
a
time
and
and
the
learning
rules
have
to
be
specific
to
that.
But
but
in
sensory
cortex
there
has
to
be
a
two
component
type
sequence
of
events
and
that
that's
really
important
to
the
to
the
learning
argument
and
so
on
and
so
forth.
C
G
I
I
think,
since
you
mentioned
the
asked
me
I'll,
take
a
stab
at
that
too.
I
mean
I.
I
think
that,
like
your
position,
that
you
know,
we
don't
have
to
implement
these
things
that
way.
This
is
a
bit
of
a
straw
proposal,
because
I
think
we
all
agree
in
the
end
that
once
you
understand
how
some
system
works,
you
have
a
lot
of
variability
in
how
you
implement
those
algorithms.
F
If
I
may
comment
on
that,
because
I
also
unfortunately
have
to
run
for
another
meeting,
but
because
I
think
this
is
is
a
key
point
that
you've
just
made
jeff-
and
I
I
want
to
clarify
the
position
that
I'm
taking
here
so
and
this
also
it
relates
to
this.
What
matthew
was
saying.
I
I'm
not
suggesting
that
ai
should
ignore
the
brain.
Nor
am
I
suggesting
that
we
have
already
got
everything.
We
need
an
ai
and
it's
all
good
to
go.
F
When
we
better
understand
the
algorithms
of
the
brain,
we
can
then
use
that
information
to
inform
the
development
of
ai,
but
when
we
do
that,
when
we
take
the
algorithms
that
we
have
uncovered
from
the
brain
and
we
go
to
implement
them
in
ai,
I'm
not
convinced
that
we're
going
to
have
to
implement
those
algorithms
with
dendrites.
Even
if
dendrites
were
key
to
our
understanding.
The
algorithms.
G
I
think
I
totally
agree
with
you.
I
I
I'm
100
agree.
I
just
I
just
know
from
experience
that
not
everyone
is
is
as
enlightened
as
you
are,
and
they
will
they
a
lot
of
people
just
jump
to
the
next
conclusion,
which
you
already
dismissed
was
that
maybe
we
don't
have
to
pay
attention
to
these
things,
and
it's
just
surprising.
I
often
have
to
remember
people
that
remind
people
that
way
just
just
before
you
go.
I
just
just
I
want
to.
I
just
want
to
make
one
more
point
about
the
algorithms.
G
You
know,
there's
a
huge
part
of
neuroscience
these
days
focused
on
algorithms
that
are
in
the
hippocampal
complex.
These
are
place
cells,
good
cells,
head
direction,
cells,
very
specific
types
of
cells,
very
specific
type
of
operations,
a
whole
set
of
algorithms
and
slew
of
algorithms
that
are
really
interesting
and
fascinating
and
complex
that
you
know
that
aren't
in
any
of
the
standard,
ai
neural
networks
today,
and
so
that's
just
a
hint
of
you
know
how
far
we
are
from
knowing
all
these
algorithms,
oh
totally,
yeah
yeah.
So
anyway,
I
think
I'm
agreeing
with
you.
G
C
There's
another
level
on
which
you
might
find
that
it
does
turn
out
that
you
want
to
implement
something
like
a
dendrite
and
call
it
something
like
a
dendrite,
and
that
would
be
in
the
realm
of
let's
say
it's
object.
Oriented
coding
turns
out
to
be
a
good
way
to
organize
your
thoughts
when
you
code,
without
necessarily
changing
the
algorithms
under
that
that,
are
that
it
may
be
that
by
organizing
the
structure
of
your
network
into
with
segregated
inputs
onto
dendrites.
C
D
G
F
A
E
I
mean
I,
I
very
much
appreciate
the
discussion
and
it's
been
a
pleasure
to
be
part
of
it.
I
would
guess
that
I
I
just
think
that,
if
we're
going
to
try
to
understand
where
things
are
coming
from
how
they
fit
together
and
things
like
that,
we
really
need
to
think
about
constraints
in
that,
especially
if
we're
going
to
be
applying
or
borrowing
from
deep
learning
or
using
deep
learning
to
apply
using
deep
learning
to
apply
to
neuroscience.
E
We
need
to
use
neuroscience
to
constrain
our
deep
learning
approaches,
at
least
to
better
understand
where,
like
what
is
actually
happening
in
neuroscience,
if
not
necessarily
to
you
know,
make
deep
learning
better.
You
know
I
I
I'm
a
neuroscientist,
first
and
foremost,
deep
learning
scientists.
Second,
so.