►
From YouTube: Brains@Bay Meetup - Exploring Neuromodulators and How They Might Impact AI (Apr 13, 2022)
Description
Follow-up Q&A and slides will be posted soon - check the meetup page for updates.
Slides: https://numenta.com/resources/videos/brainsatbay-neuromodulators-and-ai
Link to meetup: https://www.meetup.com/Brains-Bay/events/284481247/
In this Brains@Bay meetup, we discuss how the principles of neuromodulators in the brain can lead to more flexible and robust machine learning systems.
First, Skrikanth Ramaswamy (Newcastle University) gave an overview of the biological organizing principles of neuromodulators in adaptive cognition and highlight the competition and cooperation across neuromodulators.
Then, Jie Mei (the Brain and Mind Institute) discussed ongoing research on bio-inspired mechanisms of neuromodulatory function in DNNs, and propose a computational framework to inspire new architectures of “neuromodulation-aware” DNNs.
Finally, Thomas Miconi (ML Collective) talked about his work on evolving neural networks, endowed with plastic connections and reward-based neuromodulation, based on a computational neuroscience framework.
To learn more about Brains@Bay, visit our Meetup page: https://www.meetup.com/Brains-Bay/
0:00 Introduction
1:39 Skrikanth Ramaswamy: A primer on neuromodulatory systems
33:19 Jie Mei: Implementing multi-scale neuromodulation in artificial neural networks
1:07:54 Thomas Miconi: How to evolve your own lab rat
1:33:22 Discussion + Q&A
- - - - -
Numenta is leading the new era of machine intelligence. Our deep experience in theoretical neuroscience research has led to tremendous discoveries on how the brain works. We have developed a framework called the Thousand Brains Theory of Intelligence that will be fundamental to advancing the state of artificial intelligence and machine learning. By applying this theory to existing deep learning systems, we are addressing today’s bottlenecks while enabling tomorrow’s applications.
Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://numenta.com/news-digest/
Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter
Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta
Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/
Our Website:
https://numenta.com/
A
Okay,
all
right
so
hi.
Everyone
welcome
to
another
brains
at
bay,
I'm
charmaine!
I
work
in
amenta
and
I'm
the
organizer
for
this
event,
so
for
our
new
members
also
welcome
we're
an
online
brain
inspired
machine
learning,
meetup
with
the
goal
of
fostering
the
study
and
development
of
machine
learning,
algorithms
heavily
inspired
by
neuroscience
research.
So
today
we're
going
to
be
talking
about
how
the
principles
of
neuromodulators
in
the
brain
can
lead
to
more
flexible
and
robust
machine
learning
systems
and
we're
super
thrilled
to
have
three
great
speakers
with
us.
A
If
you're
interested
in
watching
one
of
our
previous
meetups
or
you
have
to
leave
early
today,
we
do
record
all
of
our
sessions
and
we'll
post
them
on
naventa's
youtube
channel
and
I
usually
post
them
within
like
a
day
or
two
and
after
I
post
this.
Today's
meetup
recording
I'll,
also
post
a
link
to
the
meetup
page
and
if
you
have
any
feedback
or
suggestions,
just
feel
free
to
message
me
through
the
meetup
page
or
you
can
message
me
with
the
email
below
so
before.
We
officially
start.
A
If
you
have
any
questions
for
the
speaker,
please
ask
them
using
this
q
a
feature
in
zoom.
It
might
look
a
little
different
here
since
I
have
like
the
host
view,
but
then
you'll
see
the
little
icon
underneath
in
the
zoom
toolbar
and
for
and
there
you
can
ask
your
questions
or
you
can
also
vote
on
existing
questions.
A
So
with
that
being
said,
I'd
like
to
introduce
our
first
speaker,
dr
srikhan,
from
zami,
so
sri
directs
the
neuro
circuits
laboratory
at
newcastle
university,
which
focuses
on
understanding
how
neuromodulators
shape,
learning
and
cognition
in
neurocortical
circuits.
He
was
formerly
a
group
leader
in
the
blue
brain
project.
At
the
federal
institute
of
technology,
which
he
led,
the
effort
in
modeling
synaptic
transition
and
neuromodulation
in
the
brain
sri
and
may
recently
collapsed
on
a
review
that
focuses
on
how
the
principles
of
neuromodular
neuromodulation
might
benefit
deep
learning
networks,
which
is
what
we'll
be
talking
about
today.
A
B
Yeah
yeah
so
yeah,
I'm
really
thrilled
to
act
as
moderator
here
on
this
session
on
neuromodulators
and
ai,
I'm
really
thrilled
to
have
the
three
of
us
three
may
and
thomas
here.
I
think
this
is
the
perfect
trio
to
discuss
this
topic.
I
myself
don't
know
a
lot
about
neuromodulators,
so
you
know,
reading
the
review
paper
that
you
guys
wrote
was
really
impactful
to
me
and
you
know
I'll
have
questions
and-
and
you
know,
try
to
moderate
some
of
the
questions
that
come
up
as
well.
C
Okay,
all
right,
so
so
thanks,
charmaine
and
and
subutai
for
this
invitation
it's
great
to
be
here
and
to
talk
about
how
neuromodulators
might
impact
ai,
so
the
first
part
of
today's
meetup
I'll
try
and
focus
a
bit
on
on
the
biology
of
neuromodulatory
system.
So
I
I
decided
to
to
kind
of
put
together
a
little
primer
to
to
to
provide
an
introduction
on
on
neuromodulatory
systems.
C
So,
let's
start
with
with
a
little
motivation
on
the
synaptic
brain
as
I
like
to
call
it.
So,
the
the
human
brain
consists
of
about
86
billion
neurons
that
are
connected
by
about
100
trillion,
synapses
so
10
to
the
12
synapses
and
to
to
to
make
these
numbers
a
a
little
more
complicated
or
this
whole
affair
more
complex.
C
These
neurons
and
synapses
are
are
continuously
being
acted
upon
by
certain
signaling
chemicals
called
neuromodulators
that
that
really
dynamically
reconfigure
or
change
activity,
states
of
neurons
and
synapses
to
bring
about
myriads
of
possibilities
at
the
at
the
network
level.
So
what
are
these
neuromodulators
or
or
what
is
neuromodulation?
C
In
addition,
they
also
have
temporally
extended
effects
on
target
neurons,
synapses
and
micro
circuits
in
different
brain
regions.
So
this
is
a
a
little
cartoon
of
the
of
the
human
brain
that
actually
describes
the
various
subcortical
neuromodulatory
projections
subcortical,
because
the
the
top
most
part
in
this
cartoon
is
actually
the
cerebral
cortex
and
most
of
these
neuromodulatory
nuclei
are
actually
located
below
the
cerebral
cortex.
C
So
there
are
five
major
ascending
neuromodulatory
systems,
ascending
simply
because
there
are
neurons
that
release
certain
homed
in
in
the
subcontinental
regions,
and
they
have
these
very
long
range
axons
that
that
really
climb
up
and
project
across
the
brain
so
include
including
cerebral
cortex
and
the
hippocampus
and
and
virtually
any
any
brain
region.
So
these
axons
just
ascend
from
these
neuromodulated
nuclei
and
really
project
all
over
the
brain.
C
So
the
I
like
to
call
these
the
big
five
neuromodulators,
so
acetylcholine,
so
cholinergic
neurons,
are
mainly
housed
in
the
basal
forebrain
or
the
septal
nucleus
of
the
hippocampus
are
the
so-called
peduncular
quantine
nuclei
dopamine.
The
so-called
pleasure
molecule
that
we've
all
heard
about
dopamine
is
released
from
neurons
in
in
an
area
called
nucleus
accumbens
or
are
the
striatum,
the
ventricle
tegmental
area,
or
even
the
substantia,
so
serotonin
is
released
from
a
neurons
in
the
so-called
dorsal
raphe
nucleus.
C
C
So
there
are
five
big
neuromodulatory
systems
that
project
from
subcontinental
regions,
all
the
way
to
the
cortex
and
and
and
many
other
brain
regions.
So
to
to
give
you
a
a
little
idea
for
what
these
neuromodulators
do
and
and
and
how
they
they
act
on
different
neurons.
So,
let's
consider
the
the
cerebral
cortex
right,
which
is
which,
as
I
said,
sits
on
on
the
very
top
of
the
brain
right.
It's
it's
like,
like
the
outer
bark
of
a
tree
right.
C
So
a
tiny
crumb
of
the
cerebral
cortex,
which
is
roughly
the
size
of
a
grain
of
sand
or
a
pinhead,
consists
of
about
30
000
neurons,
and
these
30
000
neurons
could
be
further
classified
into
55
morphological
types.
It's
something
like
the
the
diversity
of
flora
in
the
amazonia
right,
so
so,
depending
on
on
the
on
the
shapes
of
axons
that
are
actually
shown
in
blue
and
dendrites
that
are
actually
shown
in
red.
C
C
But
we
know
by
by
virtue
of
the
geometry
of
axons
and
dendrites,
that
this
3025
combinations
of
synaptic
connections
is
not
geometrically
viable,
so
there
are
about
2000
or
so
that
are
geometrically,
viable
and
experimentally.
We
only
know
the
the
physiology
or
the
function
of
these
2000
or
so
synaptic
combinations.
C
We're
certainly
nowhere
there.
Yet
I
would
say
we're
still
at
the
tip
of
the
iceberg
like
skimming
the
surface
and
there's
a
lot
of
work
to
be
done
to
really
try
and
understand
how
these
neuromodulators
impact,
neuronal
and
synaptic
function.
So
just
to
give
you
another
idea
of
the
the
so-called
diversity
of
neurotransmitters.
C
So
I
already
mentioned
glutamate
and
gaba,
and
these
other
neuromodulators
acetylcholine
non-adrenaline
serotonin,
dopamine
and
histamine.
So
they
act
through
specific
receptors
that
are
mostly
distributed
on
the
dendrites
and
possibly
also
the
axons
of
of
these
55
neuron
types.
Let's
say
I
mean
just
just
to
to
to
keep
things
tractable,
so
if,
if
the,
if
the
neuromodulators
or
or
neurotransmitters
could
be
considered,
the
key
the
locks
could
be
these
different
receptors.
C
So
the
the
keys
only
open,
specific
locks
and
once
the
lock
and
key
combination
is
set.
Only
then
is
the
gate
opened
for
something
to
happen
to
a
neuron
right
so
depending
on
the
kind
of
receptors
which
could
be
broadly
categorized
as
ionotropic
or
metabotropic,
which
really
is
based
on
the
time
scales
of
of
their
physiology
or
their
function.
So
ionotropic
receptors
have
very
fast
time
scales
of
operation.
C
In
the
order
of
a
few
to
tens
of
milliseconds,
whereas
metabotropic
receptors
are
much
slower
in
their
operation,
so
they
they,
they
tend
to
operate
in
the
range
of
hundreds
of
milliseconds
right.
So
so,
depending
on
the
neuromodulator,
the
receptor
type
and
the
kinetics
of
of
the
receptors,
there
could
be
very
marked
effects
at
the
neuronal
and
the
synaptic
or
the
network
level.
So
so
this
is
another
little
cartoon
that
I
made
to
to
to
just
highlight
the
the
vast
diversity
of
receptors
neuromodulated
receptors
that
are
found
in
cortical
neurons.
C
C
So
what
this
map
is
trying
to
show
is
is
is
the
correlation
of
receptor
expression
and,
and
something
like
like
this
makes
it
a
lot
more
tractable
to
to
try
and
understand
this
combinatorial
explosion
of
what
receptor
types
are
expressed
in
in
what
neurons,
and
is
it
necessary
to
really
measure
all
of
their
physiology
are
by
just
looking
at
at
the
coexpression
of
receptors
and
and
this
matrix.
That
seems
to
predict
that
if
one
receptor
type
is
expressed,
then,
what's
the
probability
of
another
receptor
type
belonging
to
another
neuromodulator.
C
C
So
this
is
a
very
broad
overview
of
some
salient
functions
of
neuromodulators
like
serotonin.
That
projects
from
the
raphe
nucleus
to
part
of
the
cortex
seems
to
to
to
really
play
an
important
role
in
thresh
assessment
or
or
the
so-called
paradigm
of
fight
or
flight,
for
example,
whereas
acetylcholine
that's
released
from
the
basal
forebrain
in
the
neocortex,
for
example,
seems
to
encode
attention
right.
So
our
ability
to
concentrate
our
ability
to
focus
on
something
our
ability
to
stay
attentive.
C
So
I
I
wanted
to
reinforce
this
this
message
again,
that
neuromodulatory
systems
actually
act
spatially
and
temporarily
across
neurons,
for
example,
in
the
cerebral
cortex
to
actually
regulate
cortical
states
or
network
activity
or
network
transitions.
In
the
context
and
to
me
this
is
like
a
relay
race
where
one
neuromodulator
sets
the
stage
for
another
neuromodulator
to
take
over,
and
this
is
a
constant
process.
That's
ongoing,
as
long
as
as
an
organism
is
alive
right.
So
the
pattern
is
always
constantly
being
passed
to
bring
about.
C
All
all
these
multi-scale
effects
of
neuromodulators
come
together
to
to
do
something
at
the
at
the
network
level.
Right
so
again,
to
give
you
some
more
examples,
the
functions
of
neuromodulators
are
multiscale,
so
they
could
impact
synaptic
plasticity.
They
could
impact
network
oscillations.
They
could
impact
the
growth
of
neurons
so
like,
for
example,
acetylcholine
in
a
brain
region
called
the
hippocampus.
C
That's
responsible
for
learning
and
memory
plays
salient
roles
in
in
in
modulating
all
these
levels
of
organization,
and
this
is
another
little
table
to
to
try
and
and
drive
home.
The
fact
that
this
lock
and
key
combination
is
very
important
and,
depending
on
the
kind
of
receptor
type
that's
activated
and
where
they're
distributed
a
neuromodulator
like
acetylcholine,
can
have
really
a
potpourri
of
effects
in
the
in
the
hippocampus.
C
But
if
you
add
dopamine
into
the
bath
while
doing
these
experiments,
then
that's
really
going
to
modify
this
time
window
for
the
induction
of
of
stdp,
not
just
dopamine
pretty
much.
Every
neuromodulator
would
do
that,
depending
on
on
what
brain
region
it's
acting
on,
and
this
is
finally
an
example
of
of
a
of
a
relatively
understudied
neuromodulator
histamine
and
what
it
can
do
to
neurons
in
the
hippocampus.
C
So
in
the
plane,
vanilla
case,
pyramidal
cells
don't
find
as
much
in
hippocampus
also
fast
spiking
into
neurons,
but
adding
histamine
to
the
bath
actually
enhances
their
finding
rate
and
adding
histamine
to
the
bath,
also
actually
fine,
tunes
or
sharpens
the
power
spectrum
of
network
oscillations.
So
here
in
this
study,
histamine
was
found
to
actually
sharpen
network
oscillations
in
the
so
called
gamma
band.
These
are
high
frequency
network
oscillations
that
are
believed
to
be
a
direct
readout
of
attention,
learning
and
memory.
C
So
some
conclusions
here
on
the
functional
diversity
of
neuromodulators,
so
the
same
neuron
can
be
targeted
by
multiple
neuromodulators
and
depending
on
the
kind
of
neuromodulator
and
the
receptor
type
that's
targeted.
The
effects
can
be
completely
different,
so
neuromodulators
can
produce
qualitative
changes
in
the
intrinsic
properties
of
neurons,
for
example,
transform
a
tonic
firing
neuron
into
a
phasic
bursting
neuron
atomic
fighting
neuron
could
be
a
neuron.
That's
fighting
action
potentials
at
a
very
high
frequency
and
adding
a
certain
neuromodulator
could
completely
change
the
identity
of
this
neuron.
C
Like
change
dr
jekyll
to
mr
hyde,
convert
a
a
neuron,
that's
tonically,
fighting
into
a
neuron,
that's
physically
fighting
so
reduce
the
filing
frequency
of
action
potentials
and
the
same
neuromodulators
within
the
network.
Can
differentially
regulate
cell
types
and
their
synaptic
connections
so,
depending
on
the
cell
type
again,
the
very
same
neuromodulator
can
have
very,
very
different
effects,
so
some
take
home,
organizing
principles
of
neuromodulators,
so
most
neuromodulatory
systems,
project
from
subcontinental
regions.
C
They
release
neuromodulators
from
a
certain
loci
subconical
loci
that
broadcast
to
larger
areas
within
the
brainstem
thalamus
on
the
cortex.
So
neuromodulatory
systems
are
reciprocally
connected
with
the
frontal,
cortex
and
parts
of
the
limbic
system
that
actually
regulates
behavioral
and
emotional
responses.
So
whenever
we
feel
happy
or
feed,
we
feel
low,
any
behavioral
or
emotional
change
is
always
due
to
the
function
of
of
a
certain
neuromodulator
and
lo
and
behold,
neuromodulators,
don't
ever
act
in
isolation,
so
they
always
function
in
cooperation
or
a
position
to
bring
about
these
changes
in
brain
state.
C
B
Thank
you,
srikant.
That
was
fantastic.
That
was
a
great
overview.
We
have
a
couple
of
quick
questions
on
that
on
the
q
a
maybe
we
can
ask
that,
so
we
actually
have
two
sid
hearts
the
difference
in
hearts
here
so
first
sidhart,
what
what's
the
latency
on
the
effects
of
the
neuromodulators
relative
to
neuron
firings?
I
think
you
touched
on
that
a
little
bit,
but
maybe
you
can
expand.
C
C
But
if
metabotropic
receptors
are
involved,
then
this
latency
is
really
going
to
be
in
the
order
of
tens
or
hundreds
of
milliseconds.
So
it
depends
on
the
kind
of
receptors
that
are
activated
and
where
they
are
spatially
located,
for
example,
on
the
dendrites.
If
receptors
are
are,
are
located
on
distributes,
then
of
course
there's
going
to
be
a
longer
latency,
but
if
they
are
located
more
proximally,
then
the
latency
is
going
to
be
shorter.
B
D
If
I
can
interrupt
a
bit
of,
I
think
your
question
is
quite
interesting
to
take
a
look
at
alan.
This
text
work
because
he
really
studied
you
know
like
how
this
a
scenario
contact.
The
location
can
affect
the
ferry,
the
cell
and
the
computational
rules,
and
he
actually
has
a
really
interesting
case
of
he
reduced
around
200
different
compartments
into
three
and
having
comparable
functions,
and
I
think
I
can
share
the
link
to
that
paper
and
I
think,
based
on
your
recent
research
on
dendritic
computing.
B
C
C
D
In
addition
to
this,
I
think
one
interesting
example
of
people
can
take
a
looking
at
this
septalamic
part
and
also
like
the
different
dynamic
compartments.
For
example,
if
you
have
the
like
one
really
great
examples
of
the
release
of
the
thalamus,
for
example
the
vision
part
of
it
and
in
in
the
relay
cell
and
into
neurons
within
the
thalamus,
there
are
different
ways
that
are
you
know
the
neurons
are
are
contacting
each
other.
D
B
Things,
let's
take
one
more
question
and
then
the
rest
we
can
handle
in
the
discussion.
I
think
you,
this
is
more
about
the
contextual
impacts
of
neuromodulators,
and
I
think
you
touched
on
this
a
little
bit
in
the
review
paper.
But
the
question
here
is
our
neuromodulator
is
applied
based
on
some
sensory
input,
like
fear
or
arousal,
are
the
neuromodulators
that
are
constantly
active,
regardless
of
any
sensory
input.
C
C
I
think
the
interaction
between
these
neuromodulators
is
so
fundamentally
important
because
they
really
set
the
stage
and
they
provide
like
a
platform
or
the
foundation
for
other
neuromodulators
to
to
kick
in
and
and
therefore
this
whole
idea
of
co-release
and
co-transmission
of
neuromodulators
is
is
very
important
in
in
my
opinion,
but
this
is
not.
This
is
something
that's.
C
That's
certainly
not
not
easy
to
study,
because
although
we
say
that
okay
dopamine
is
involved
in
reward
or
our
or
our
serotonin
or
neuroadenyline
is
involved
in
fear
conditioning
we're
only
looking
at
one
neuromodulator,
but
that
doesn't
mean
that
others
aren't
involved.
It's
just
that.
We
have
no
way
of
not
of
simultaneously
monitoring
multiple
neuromodulators
at
once,
although
there
is
some
progress
being
made
there,
but
I
think
the
day
we
are
able
to
monitor
multiple
neuromodulators
simultaneously
is
when
will
be
a
game.
Changer
yeah.
A
Yeah,
let
me
just
quickly
introduce
you
so
maze
of
postdoc
at
the
brains
in
my
institute
from
university
of
western
ontario,
with
a
phd
in
medical
neuroscience,
may
brings
together
her
deep
learning
and
neuroscience
expertise
to
make
machines
smarter
and
she's
also
written
extensively.
Extensive
publications
in
both
fields,
she's.
Also
the
first
author
of
the
review
paper
we're
looking
at
today,
and
we're
really
happy
to
have
you
here.
A
D
You
for
the
for
introducing
me-
and
I
guess
I'm
gonna-
take
over
cherie's
stage,
because
this
is
something
that
we
want
to
see
like.
Actually,
I
saw
some
questions
that
are
related,
that
I
can
answer
using
my
presentation,
and
so
I
was
not
like
yeah,
but
I
want
to
leave
it
for
later,
and
so
my
talk
is
actually
two
three
parts,
because
I
wanted
to
introduce
the
framework
first
and
then
some
of
the
results
we
are
obtaining
from
it.
D
So
the
first
thing
I
want
to
do
is
just
a
review
of
a
couple
of
studies
that
are
in
this
related
field
and
we're
going
to
provide
like
actually
actually
a
direct
comparison
between
your
modulation.
In
the
number
systems
and
that
in
the
dns-
and
we
want
to
compare
like
the
studies
that
focus
on
artificial
intermodulation
or
use
part
of
it
to
promote
their,
you
know
learning
objectives
and
we
compare
the
tasks
they
have
achieved.
The
implementation
of
neural
networks
in
these
studies,
as
well
as
what
they
have
observed.
D
And
after
that
we
wanted
to
discuss
the
proposal
we
have
made.
You
know
in
more
detail,
it's
a
multi-scale
framework
for
your
modulation,
indians
and
for
more
details
and
read
the
paper,
and
I
hope
today
I
can
give
a
really
like
interesting
primer
on
that
and
like
make
you
learn
this.
You
know
you
know
like
in
in
real
and
get
some
real
details
and
we
wanted
to
add
the
spatial
temporal
specificities
at
different
scales
and,
in
the
meantime,
we
want
to
present
some
results.
D
We
have
by
implementing
such
a
framework
and
if
I
still
have
some
time,
I'm
gonna
just
give
a
world
to
santa's
introduction
to
the
current
work,
where
your
priority,
if
you're
having
and
the
relevance
and
what
are
the
next
steps
of
of
this
framework.
So,
okay.
So
all
the
figures,
almost
figures.
I'm
presenting
there
are
in
other,
you
know
paper
already
so
like
for
this.
You
can
refer
to
the
paper.
I
know
this.
D
There
are
a
lot
of
details
and
if
you
have
any
any
questions
or
confusions,
just
let
me
know
and
for
all
the
other
figures
we
have
there,
they
will
be
released
in
the
pre-print
gonna
submit
during
the
month.
Okay,
so
first
we
wanted
to
make
sense.
So
what
is
a
biological
nervous
system
implementing
the
ways
of
new
modulation
and
what
have
we
done
for
deeper
networks?
So
sharia
has
already
mentioned.
There
are
spatial
temporal
scales
affecting
different.
D
You
know
vicinities,
and
I
want
to
take
it
over
from
here
and
actually
describe
some
of
the
tasks
we
have
achieved
using
such
a
multiscope
design,
so
both
the
temporal
and
spatial
scale.
So
we
can
see
that
a
new
modulation
actually
have
an
effect
at
a
really
low
cost
sub
neuronal
level,
for
example,
it
can
modify
the
alternative
kinetics.
This
is
what
acetylcholine
does
and
if
we
go
a
bit
up,
it
also
performs
morphological
modification
of
dendrites.
D
And
after
that
we
also
have
a
functional
reconfiguration
of
neural
networks
or
micro
circuits
in
a
more
biological
term,
and
I
think
there
are
like
a
lot
of
experiments
on
this,
especially
done
by
e
mater,
and
she
published
a
couple
papers
on
this
about
how.
If
you
have
the
newer
same
in
your
network,
applying
different
your
modulators
by
different
kind
of
path,
you
can
fundamentally
change
the
output
of
the
same
in
your
network.
So
this
is
an
interesting
way.
D
We
can
kind
of
modulate
and
modify
the
function
of
the
same
neural
network,
and
this
is
how
we
are
spanning
across
the
temporal
spatial
scale,
in
biological
nervous
system,
by
having
your
modulators
and
having
them
kind
of
a
guiding
or
modulating
different
tasks.
And
in
the
meantime,
if
you
compare
a
bit
of
you
know,
these
are
for
the
first
studies.
We
have
a
bit
of
a
covered
and
discussed
in
a
paper.
You
know,
but
it's
not
it's
not
all
inclusive,
and
there
are
some
word
studies
in
this
field.
D
Of
course
I
think,
but
unfortunately
not
so
many
more.
I
think
there
are
a
couple
of
two
or
three
more
papers
coming
out
of
this
year
and
we
are
choosing
this
for
because
it's
for
representing
implementations
of
for
different
tasks-
and
you
know
they
have
also
observed
the
different
behavioral
benefits
of
having
new
modulators
involved
in
deep
neural
networks.
D
So,
but
in
the
meantime,
you
can
see
that
they're
kind
of
colliding
on
the
spatial
scale
and
as
most
of
these
studies,
they're
covering
a
smaller
spatial
scale
and
there's
no
like
super
tiny
network,
or
you
know,
like
a
really
large
network
that
are
being
tested.
So
if
we
really
discuss
you
know,
dissect
the
details
of
these
papers,
we
can
see
that
all
these
papers
have
achieved
a
lot
of
behavior
benefits.
D
We
have
a
new
modulation
enabling
learning
of
more
patterns
and
associations
and
in
the
meantime,
we
also
have
papers
that
are
dedicated
to
alleviating
catastrophic,
forgetting
a
long-standing
issue
in
continual
learning
using
this
modularity
base
in
your
network
and
in
the
meantime,
what
we
have
already
briefly
covered
is
that
the
neural
networks
have
implemented
their
like
of
a
colliding
spatial
scale.
For
example,
we
have
mostly
these
neural
networks.
D
In
these
studies,
they're
relatively
small
and
there's
only
one
study
by
actually
by
me
connie,
our
third
speaker,
you
know
they're
testing,
this
synaptic,
you
know
neuromodulation,
driven
snapping
properties
in
a
larger
neural
network
of
for
for
language
modeling
and
in
the
meantime,
what
we
can
see
is
that
these
papers
are
using.
You
know,
moderatory
properties,
of
deep
neural
networks,
or
you
know
kind
of
using
this
as
a
main
objective
investigation
to
achieve
different
goals.
D
D
If
the
new
modulation
can
help
us
be
more
adaptive
and
in
the
meantime
we
have,
although
other
other
objectives,
including
you
know,
in
enable
state
dependent
behaviors,
for
example,
like
in
a
certain
behavioral
context,
in
a
certain
season
of
a
simulated
year,
if
the
agent
will
learn
a
different
association
between
the
environment
and
the
food.
So
this
is
something
we
have
been
achieving
using
neuromodulation.
We
have
seen
like
people
really
define
your
modulation,
different
ways
and
implement
on
different
scales.
D
You
know
this
proposal
of
biological
nervous
systems
and
their
muscle,
linear
modulation,
to
a
scale
that
is
of
greater
spatial
temperature
specificity
and
to
see
if
we
have
a
parallel.
You
know
near
modulation
on
going
at
different
spatial
scales.
If
you
can
improve
further,
the
performance
of
your
network
will
help
it
blur.
So
this
is
this.
This
is
objective
of
us
reporting
proposing
a
multi-scale
framework
for
of
neuromodulation,
and
this
is
yeah.
D
This
is
actually
you
can
also
find
them
in
our
paper
and
with
the
more
like
thorough
description
of
what
it
does.
But
what
I
wanted
to
say
is
that
this
is
the
core
of
our
proposal.
It's
a
model
scale
artificial
neuromodulation
unit,
and
what
it
does
is
that
we
divide
this
into
four
spatial
skills.
Of
course,
there
are
spatial
skills
in
between
these,
and
these
are
just
for
you
know,
principle
demonstration
purposes
and
also
to
inspire
people
to
think
a
bit
more
about
spatial
specificities
of
these.
D
So
we
have
them
from
the
whole
network
scale
that
are
controlling
the
hyperparameter
configuration,
for
example,
the
learning
rate
and
drop
out
based
on
the
the
input
from
the
environment,
as
well
as
the
internal
state
of
the
agent
and
in
the
meantime,
if
you
have
like
a
connectivity
between
neurons
across
different
layers.
What
about?
If
we
have?
You
know
different
kind
of
scaling
and
adjustments
of
connectivity
by
introducing
like
low
coverage,
global
plasticity
and
in
the
meantime
we
can
also
see
that
for
different
neurons.
D
You
know
at
a
different
spatial
scales
or
different
layers.
We
may
have
neurons
of
different
types
by
introducing
you
know
a
cell
type,
specific
modulation
onto
these
neurons,
so
one
of
the
papers
we
have
described
to
that
paper
did
that
by
having
you
know
a
different
different
scaling
factors
of
new
modulation
onto
these
neurons
and
below
that.
D
We
may
also
have
different
dendritic
configurations
for
these
neurons,
depending
on
where
they
are,
and
I
think
some
of
papers,
for
example,
a
super
tight
paper
and
the
recent
paper
by
obliteratures
have
also
discussed
how
we
can
use
this.
You
know
dendritic
properties
of
single
neurons
to
achieve
different,
computations
context,
based
learning,
as
well
as
our
error
propagation.
D
So
this
is
our
you
know,
proposal
regarding
how
we
can
inject
different
level
of
of
the
neural
modulation
onto
a
different
networks
and,
in
the
meantime,
what
I
think
that's
important,
also
how
we
are
studying
it,
how
we
are
monitoring
the
effects
of
on
your
modulation
and
the
performance
of
the
new
network.
So
this
is
the
next
part
of
our
talk.
So
first
thing
I
wanted
to
highlight
is
that
for
this
framework
we
want
to
consider
the
internal
and
external
stage
of
the
environment
and
also
the
agent,
and
we
use
these
to.
D
You
know,
control
the
hyperparameter
update
and
everything
else
in
between
so,
for
example,
in
in
animals,
of
course,
in
living
organisms
we
always
observe
from
the
environment
at
a
certain
at
a
certain
sampling
rate
and
in
the
meantime,
for
the
biological
brain.
We
also
have
internal
copy
or
replicate
of
our
current
state,
the
cognitive
state,
behavioral
state
and
also
you
know,
internal
properties
so
for
the
brain
that
we
are
not
aware
of.
D
So
this
is
something
called,
sometimes
called
the
discharge,
and
so,
if
we
have
this
kind
of
access
to
all
these
properties,
we
have
a
better
knowledge
of
what
the
world
is
bringing
us
and
what
is
our
exact
next
action
and
we
can
have
a
quite
a
contextual
input
based
on
these
properties.
So
this
is
the
first
important
feature
we
want
to
introduce
in
this
framework.
A
second
feature
is
that
we
may
have
a
sampling
of
model
performance
and
behavior
at
a
different
spatial
scale
and
temporal
scale.
D
So
what's
interesting
about
this,
is
that,
for
example,
some
certain
animals
or
living
organisms
learning
is
a
bit
of
we
all
know.
The
learning
is
continual,
and
sometimes
not
non-symmetric,
because
for
rodents,
for
example,
when
they're
first
introducing
into
into
one
sphere
into
my
environment,
they're
more
nervous
and
they're
more
like
obsessive
about
exploring
the
surroundings
to
know
if
there's
any
hidden
danger.
D
So
this
is
a
state
that
are
super
alert
and
aware
of
the
environment,
the
surroundings,
and
then
they
learn
about
the
surroundings,
a
bit
better
and
know
how
to
avoid
danger
or
obstacles.
They
are
more
relaxed,
and
this
is
more
about
the
relaxed
states.
Of
course,
in
our
brain
we
also
have
different
cognitive,
behavioral
states.
We
have
more
overstate,
we
have
more
attention
state
and
we
have
more
relaxed
states.
So,
of
course,
at
these
different
states
we
are
learning
at
different
speeds
and
paces.
So
what
about?
D
If
we
study
all
these
different
properties
and,
of
course,
for
isometric
paces
of
learning-
and
we
observe
that
how
our
brain
or
how
the
system
optimized
based
on
a
local
loss
function
and
then
in
the
meantime,
we
monitor
the
global
learning
trajectory.
We
may
have
kind
of
these
going
in
as
synchronous
ways
but
are
interesting.
We
can
see
how
we
are
optimizing
towards
a
local
role,
for
example,
a
sudden
change
in
the
in
the
destination
and
a
global
global
goal,
which
is,
for
example,
obtaining
a
long
long
sequence
of
rewards.
D
So
this
is
the
second
point
I
want
to
discuss.
The
third
point
is
that
we
may
have
a
test
task:
dependent
evaluation
of
outcomes.
For
example,
in
the
current
study,
we
are
using
a
spatial
learning
task.
Of
course
we
are
evaluating
lots
of
the
model
and,
in
the
meantime,
we
are
evaluating,
for
example,
the
ambulation
error,
how
the
ambulance
track
is
deviating
from
the
the
track.
D
Data
is
actually
in
the
ground
truth
and
if
we
are
looking
into
other
tasks
or
other
learning
properties
of
the
the
agent,
for
example,
in
a
social
cognition
task.
So
if
repetitiveness,
we
behave
reciprocity
of
the
behavior
and
the
rewards
obtained
by
the
end,
so
these
are
something
we
are
more
focused
on
if
you're
switching
from
this
task
to
a
different
task.
D
So
of
course
we
are
kind
of
a
design
of
the
things
we
want
to
observe
or
the
metrics
are
really
important
because
we
may
have
in
the
future
our
task
dependent
evaluation
of
neural
network.
We
may
also
have
a
new
network,
that's
suitable
for
learning,
from
a
set
of
tasks.
So,
after
accompanying
all
these
scales
of
you
know
of
all
these
biological
and
spatial
and
gender
scales,
and
as
well
as
these
new,
you
know,
proposals
regarding
sampling,
behavior
and
violation
of
behavioral
outcome.
D
We
wanted
to
introduce
you
to
some
of
our
growing
work
at
the
lab
about
introducing
your
neuromodulation
rear
design
at
the
network
and
connecting
the
levels
as
well.
As
you
know,
learning
or
update
all
these
parameters
based
on
the
current
errors.
That's
our
deviation
from
the
environment.
So,
let's
get
started
and
our
tasks.
The
test
we
use
is
navigation,
open
field,
because
navigation
tests
is
long
studied
and
we
also
have
a
really
biologically
detailed
understanding
of
how
this
works
in
in
different
living
organisms
for
them.
D
Currently,
we
are
simulating
after
rodents,
emulation
trajectory
featuring.
You
know,
featuring
continuous,
featuring,
continuous
exploration
and
in
the
future
we
also
introduce
another
modeling
organism:
that's
the
primates
and
they're
learning
from
different
strategies,
of
course,
and
so
for
this
test.
We
wanted
to
estimate
the
current
location
of
the
the
animal
and
also
you
know.
D
So
in
this
model
we
have
three
different
layers,
so
we
have
the
recurrent
layer
which
is
processing
continuous
input
from
the
environment.
So
we
have
a
starting
location
where
our
agent
is
starting
from
and
in
the
meantime,
we
have
access
to
its
direction
and
and
with
movement
as
well
as
velocity,
and
we
are
processing
this
using
an
ssl,
stm
module
and
across
time,
and
then
we
input
all
these.
D
We
input
this
transformation
into
a
dense
layer,
and
then
we
have
up
layer
of
two
different
types
of
neurons,
replacing
symbol
and
head
direction.
Symbol,
and
our
goal
of
this
test
is
to
prove
is
to
provide
an
understanding
of
how
you
know,
based
on
the
estimated
location
of
the
agent,
so
how
the
different
cell
types
are
varying
and
the
loss
of
this
unit
study
is
the
difference
between
the
play
cell
activity
and
hydrogen
cell
activity,
the
predict
version
and
the
actual
version.
D
So
this
is
a
task
and
we
are
training
the
model
for
a
total
of
a
thousand
epochs
and
we
use
a
dropout
also
like
a
biologically
intuitive
term
of
dropping
certain.
You
know
neuronal
connectivity
at
random
to
reduce
or
or
memorizing
the
patterns.
Okay,
so
then
we
are
adding
the
neuromodulatory
linear
marginally
spared
input
at
different
levels,
so,
for
example,
at
the
local
level.
This
is
something
spared
actually
by
the
mikoni
paper.
We're
adding
a
heavy
plasticity
term.
Innovate
updates.
D
D
So
we
have
a
learning
schedule
based
on
that
and
of
course,
these
representative
modulation,
at
the
network
level,
indicated
by
the
learner,
scheduling
and
also
connectivity
level,
for
example,
the
having
connected
a
plasticity
as
well
as
a
drop
alpha
probability
so
so
far,
so
we're
adding,
of
course,
we're
adding
these
levels
of
modulation,
one
by
one
to
observe
what
they
are
giving
us
individually
and
also
you
know
in
combination.
That's
why
we're
having
around
four
or
five
or
five
different
model
variants
in
this
study?
Okay,
so
yeah
so
for
the
five
variants.
D
We're
studying
this
study,
so
the
base
model
is
the
mekong.
Sorry,
it's
a
it's
a
the
model,
partly
inspired
by
the
benino
paper,
and
then
we
add
the
hebbian
connectivity
and
for
the
third
version
of
the
model
we
have
having
plasticity,
plus
the
drop
out
adaptive
dropped
out
and
after
that
we
have
added
the
two
different
learning
rates,
because
we
are,
we
wanted
to
explore
how
this
affects
the
way
the
model
learns.
So
these
are
the
results
we
currently
have.
D
So
we
have
the
loss
curve
and
also
the
arrow
emulation
curve.
So
from
the
loss,
we
can
see
that
we
have
a
faster
convergence.
We
are
introducing
different
model
versions.
For
example,
if
we
compare
the
base
model
to
other
models,
we
can
see
that
at
the
beginning,
it's
more
comparable,
but
after
around
400,
500
epochs
it.
Actually,
you
know
for
the
pace
model.
It
plays
hold
a
bit
more
at
around
500
epochs
and
pretty
much.
D
It
keeps
at
this
level
until
the
end
of
the
training,
but
for
all
the
other
versions,
including
heavy
and
self-modulated,
update
of
dropout
probability
and
learning
rate.
You
can
see
that
the
laws
continue
to
improve
until
the
end
of
the
training,
which
says
that
the
the
actual
self
self-fairing
or
cell
activity
and
the
predict
ones
are
closer
and
closer.
So
what
about?
If
we
evaluate
this
test
performance
by
using
the
error
formulation?
D
So
this
is
the
difference
between
the
the
the
given
trajectory
the
ground
truth
and
the
predicted
trajectory
or
the
location
of
the
of
the
agent.
So
we
can
see
that
what's
interesting
about
this
is
that
we
have
actually
a
faster
learning
at
speed
between
zero
and
200
epochs
in
all
the
versions-
and
this
goes
well
in
alignment
with
the
loss
version
of
evaluation
and
by
the
end
we
can
see
that
the
air
formulation
across
this
fiber
is
really
hard
to
see
the
difference
here.
D
But
actually
there
is
a
numerical
difference
across
the
five
versions
and
we
have
seen
continuous
improvement
by
adding
all
these
modern
components,
and
we
see
that
the
base
model
actually
achieved
around
a
at
500
epochs
aeroformulation,
that's
comparable
to
what
we
cheap
using
the
last
version
of
the
model
where
we
have
drop
out
adaptive
drop
at
a
learning
rate
at
around
50
epochs.
That's
quite
interesting
means
that,
at
the
beginning
of
learning,
these
new
models
near
moderate
version
learning
significant,
faster
compared
to
the
base
model.
D
So
this
is
something
we
have
observed
at
a
thousand
epochs
of
training.
The
first
thing
is
that
we
have
gradually
reducing
loss
and
in
the
meantime
I
mean,
although
laws
do
not
always
convert
to
a
smaller
error,
we
still
have
error.
That's
reducing
until
you
know
until
the
end,
and
we
can
see
that
oh
okay
for
this,
for
this
task.
We
can
see
that
adding
this
newer
module
component
actually
helped
us
to
train
the
models
faster
as
well
as
to
achieve.
D
You
know,
a
smaller
error
by
the
end
of
training.
So
this
is
the
big
thing
we
have
observed
by
adding
all
these
levels
in
your
modulation
and
after
that,
we
wanted
to
understand,
what's
actually
occurring
during
the
first
two
hundred
epochs,
because
we
have
seen
a
rapid
learning
during
this
beginning
trials
and
we
wanted
to
understand,
what's
actually
happening
from
neuronal
point
of
view.
The
first
thing
we
have
solved
is
that
at
the
baseline
there's
pretty
much
a
random
random.
D
You
know
random
trajectory
and
at
200
epochs
we're
comparing
this
with
the
baseline
there's
already
a
big
improvement
looking
at
the
the
ambulatory
track-
and
this
is
what
we
have
seen
at
around
300
epochs
in
different
models.
So,
for
example,
for
the
base
model,
the
error
is
around
0.19
meters
and
what
for
the
model,
with
the
heavy
plasticity,
only
is
around
0.70
meters
and
for
the
model,
with
the
hyphen
plus
studies
and
and
drop
out.
Only
it's
around
0.12
meters
and
I
think
with
the
learning,
is
around
0.11
meters.
D
D
You
know:
increased
increased
ambulation
error
in
the
blade
in
the
base
model,
and
the
second
thing
we
want
to
understand
is
what
is
the
underlying
cell
activities,
for
example
like
in
the
fully
connected
layers
where
the
the
units
that
are
representing
or
encoding
the
spatial
learning
behavior
are
are
changed
during
this,
especially
the
first
two
hundred
epochs.
So
for
this
you
wanted
to
describe
the
properties
of
the
unit
activity
in
this.
So
that's
why
we
are
discussing
a
bit
of
a
different
cell.
D
You
know
I'm
giving
a
primer
of
all
this
like
different
cell
patterns
and
their
activities
in
the
brain.
So
what's
about
what's
interesting
about
the
spanish
study
initially,
is
that
they're
the
first
one
to
propose
that
there
are
actually
biological
plausible
behavior
occurring
at
the
fully
connected
layer
within
the
neural
network?
And
we
are
borrowing
this
as
as
a
benchmark
to
start
with,
and
we
are
aware
of,
there
are
different
varying
patterns
of
cells
in
the
fully
connected
layers,
especially
for
place
like
red-light
and
hydration
water
cells.
D
So
for
these
cells,
this
is
something
we
have
seen
commonly
in
animals.
For
example,
when
the
animals
emulating
randomly
in
a
in
a
box
or
in
open
environment,
we
can
see
that
there
are
regular
hexagonal
theorem
patterns
synched
by
three
cells,
for
example,
when
the
animals
here
or
here
or
here,
the
the
current
spine
grid
cells
in
the
brain
actually
fair,
like
crazy,
and
then
the
great
cells
they're,
projecting
the
plate
cells
and
then
the
plate
cell
marked
as
a
certain
location
with
the
environment,
for
example.
In
our
case.
D
Actually,
the
there
is
a
theory
that
vector
combination
of
the
grid
cell
is
actually
mapping
to
the
plate
cell
and-
and
there
are
a
lot
of
theory
about
why
this
is
happening
and
still
we're
trying
to
answer
this
question.
So
the
other
subtypes
are
the
border
cells
and
in
the
head
reaction
cells.
So
border
cell
marks
the
border
of
our
over
certain
field,
for
example
in
this
field,
when
the
animal
is
at
the
bottom
right
side,
border
of
this.
D
D
So
this
is
a
bit
of
hard
to
demonstrate
how
it's
all
fair,
because
it's
only
like
when
we
are
having
we're
heading
to
toward
different
directions:
the
cell
phone
chains,
for
example,
this
cell,
has
a
preference
toward
170
degrees
or
so
and
of
course
this
is
how
it's
demonstrated.
In
our
case,
it's
a
bit
of
hard
to
see
how
fairing
looks
like
for
different
angles.
D
So
there's
angular
analysis,
I'm
not
going
to
describe
that
in
details
and
if
you
have
any
questions
regarding
how
this
is
done,
please
just
drop
me
a
line,
but
this
is
how
a
direction
cell
looked
like
in
our
in
our
version,
and
we
wanted
to
understand
how
the
cells
activities
are
evolving.
You
know,
especially
during
the
starting
starting
epochs
of
the
training,
so
we
have
something
called
a
grid
grid
greatness
score.
That's
measuring
how
the
cell,
how
hexagonal
like
pattern?
The
cell
is
varying.
D
So
we
have
measured
the
risk
score
of
different
cells
in
different
experimental
versions.
We
can
see
that
the
grid
score
of
the
mirrors
we
have
in
a
fully
connected
layer.
It's
actually,
you
know,
change
the
drastically
in
between
the
first
cup
of
epochs
and
up
until
300
epochs
after
that
stabilize
a
bit
and
it's
kind
of
comparable
until
the
end.
So
this
is
our
first
operation
and
of
course,
what
we
have
seen
from
these
studies
is
that
we
have.
D
We
have
a
different
level
of
neuromodulation
that
we
sell
the
initial
evolution
of
the
gray
cell
activity
or
the
greatness
of
the
of
the
cells
are
a
bit
of
difference,
so
this
can
be
significantly
modified
by
the
protocol
we
are
using,
and
then
we
give
a
deep
dive
into
that.
So
again,
this
is
the
same
figure
and
we
wanted
to
look
into
how
across
different
number
of
epochs
the
greatness,
the
brain
cell
distribution,
the
great
score
distribution
of
the
cells,
have
changed.
D
For
example,
these
are
two
protocols
where
we
have
the
fully
flashed.
You
know
neuromodulation
onto
the
cells,
including
habian,
plasticity,
plus
the
adaptive
drop
out
and
learning
rescheduling,
and
we
can
see
that
actually,
when
we
have
different
learning
rescheduling,
you
know
to
start
with
the
the
initial
grid.
Score
of
the
cell
is
actually
having
yielding
a
different
distribution
across
the
two
versions.
D
For
example,
we
can
see
that
if,
for
for
a
smaller
learning
rate,
the
result
scores
picked
a
bit
more
and
for
our
for
a
larger
learning
rate,
it's
pretty
much
like
there's
a
wider
spread
for
the
brits
out
located
for
distribution
at
the
beginning
of
the
training.
So
I
think
this
is
playing
a
different.
D
This
is
playing
a
role
in
a
way
why
the
learning
trajectory
are
a
bit
of
different
in
the
two
first
200
trials
to
first
200
epochs,
and
the
second
thing
is
that
we
can
see
that
just
in
alignment
with
the
grid
score,
you
know
change
across
different
epochs.
We
can
see
that
after
the
first
100
epochs,
the
distribution
of
risk
score
is
a
it's
a
state
relative,
relatively
unchanged
and
up
until
the
thousand
epochs.
D
This
is
pretty
much
like
a
noisy
picture
of
how
the
cells
are
fairing
and
the
cell
does
not
have
any
preference
in
this
field
and
there's
no
clear
classification
of
the
cells.
And
after
that,
we
are
involving
a
certain
pattern
of
cell
self-fairing,
especially
by
already
some
about
by
around
around
100
epochs
and
200
epochs.
D
We
can
already
see
established
cell
classification
and
you
know
for
different
classes
of
cells
evolving
from
evolving
from
the
fully
connected
layer,
and
this
is,
of
course
not
fully
comparable
to
what
we
have
seen
by
the
end
of
the
training,
where
there
is
actually
a
clear
pattern
of
great
cell.
The
greatness,
as
well
as
the
border
cell
preferences
and
stuff
like
this,
but
of
course,
that
200
epochs.
We
can
already
pretty
much
prototyping
this
classification
and
tell
which
this
12
will
be
like
by
the
end
of
the
training.
So
this
is
something
quite
interesting.
D
D
D
We
will,
as
a
next
step
of
course,
look
into
how
the
cell
proportion
is
is
distributed
in
the
real
brain
and
if
we
can
kind
of
enable
one-to-one
projection
from
from
from
this
certain
cell
plants
or
how
that
affects
our
model
and,
in
the
meantime,
there's
some
there's
a
slight
decrease
in
number
of
head
direction,
cells
and
a
significant
decrease
in
water
cells.
There
are-
and
so
currently
we
are
looking
into
the
biological
implications
of
all
these
changes.
D
But
one
typical
message
from
here
is
that
actually
adding
your
modulation
at
different
levels
can
change
how
the
cells?
You
know
how
the
proportion
of
different
classes
of
cells
and
their
varying
properties,
for
example,
as
you
can
see,
if
you
compare
the
base
model,
you
know
the
the
grid
cell
fills
the
hexagonal
pattern
compared
to
that
to
the
version
we
have
all
the
new
modulation
involved.
We
can
see
that
actually,
the
the
field
of
fairing
of
hexagon
pattern
is
actually
a
lot
smaller.
So
that's
the
interesting
observation
we
have
so
far.
D
The
cell
activity
have
changed
when
they're
introduced
in
different
protocols
and,
in
the
meantime,
as
laws
do
not
always
map
to
a
greater
model
performance.
We
wanted
to
use
other
measures.
For
example,
the
average
emulation
to
understand
how
you
know
the
model
performance
and
the
end
results
are
changing,
they're,
adding
they're
having
different
protocols.
D
So
if
you
have
any
questions,
I
know
this
is
a
lot
of
great
details.
We
have
covered
today
and
it's
it's
hard
to
kind
of
leave
a
more
complex,
firm
presentation,
because
there
are
a
lot
of
things
I
want
to
discuss
and
present.
So
if
you
have
any
questions
regarding
the
results
or
the
kind
of
the
protocol
of
the
study
and
how
we
did
all
these
experiments,
just
let
me
know
I
can
can
reach
out
to
me
on
twitter
or
by
email.
So
I'm
looking
forward
to
your
questions.
Thank
you
so
much!
B
You
know
scaffold
where
you
tie
in
all
the
detailed
properties
of
neuromodulators
and
detailed
aspects
of
kind
of
deep
learning,
experimentation
and
so
on
and
and
then
to
see
the
experiments
which
I
hadn't
seen
before,
showing
that
sort
of
layering
these
neuromodulatory
principles
can
actually
successfully
improve
grid
cell
formation.
I
think
that
really
ties
in
well
with
what
strikeon's
point
was
earlier,
how
these
things
sort
of
work
in
combination,
so
I
do
have
more
specific
questions
and
just
on
the
chat,
it
also
says
you
know
amazing.
B
Slides
may
such
a
great
summary
of
so
many
research
papers
and
concepts
so
yeah.
That
was
really
great.
You
covered
a
lot
of
stuff
in
the
interests
of
time.
I
I
think
maybe
let's
move
to
thomas
and
then
we'll
get
to
some
more
specific
questions,
because
I
think
much
of
thomas's
stuff
is
very
adjacent
to
some
of
the
experiments
you
you
showed
as
well.
D
Together
yeah,
especially
the
for
the
heavy
and
fastest
part
thomas
proposed
this
framework,
I
think
two
years
ago,
and
we
just
we
changed
it
somehow
in
our
version,
we
were
like
largely
inspired
by
that
yeah
and
I
think
thomas
has
a
really
like
better
and
more
detailed
presentation
than
I
do
on
this
part,
so
yeah.
Thank
you.
E
B
D
E
E
Here,
I'm
going
to
show
an
example
of
how
near
modulation
or,
more
precisely,
a
very
specific
type
of
neural.
Modulation
can
be
applied
to
artificial
neural
networks
to
enhance
their
learning
abilities,
and
so
this
project
starts
from
an
observation,
which
is
that
animals
can
do
something
which
is
pretty
nice,
which
is
that
they
can
automatically
learn
novel
cognitive
tasks,
by
which
I
mean
they
cannot
automatically.
E
That
is
they
don't
have
an
external
learning
algorithm
training
them.
They
don't
learn
by
backprop
or
gradient
descent.
They
have
their
own
internal
machinery,
which
allows
them
to
learn
pretty
complicated
things.
They
can
learn
novel
tasks.
At
least
they
can
learn
tests
that
they
have
never
seen
before
in
their
life
or
even
in
some
cases
in
their
evolution.
I'm
going
to
show
an
example
in
a
minute,
but,
more
importantly,
they
can
learn
cognitive
tasks,
that
is
animals.
E
Don't
just
learn:
simple
stimulus
response
associations,
stimulus
reflexes,
like
if
you
see
this,
do
that
they
can
learn
more
complicated
tasks
in
which
they
need
to
discover
memorize
and
make
use
of
a
certain
piece
of
information
for
every
new
instance
of
the
task,
in
short
tasks
that
require
working
memory.
And
that's
what
we're
interested
in
in
this
project.
E
E
So,
every
time
the
monkey
must
find
out
through
trial
and
error,
which
of
the
two
objects
is
associated
with
food,
and
so
here
in
the
middle
you
see
the
performance
of
the
monkey
as
time
goes
by.
Initially
you
see
that
the
monkey
does
pick
out
which
of
the
two
objects
is
associated
with
the
food.
But
it's
not
very
good.
E
It
makes
a
lot
of
mistakes,
but,
as
time
goes
by
and
as
this
goes
as
it
goes
through
more
and
more
episodes,
it
gets
better
and
better
at
the
task
until
by
the
end,
that's
the
thomas
curve.
Here
the
monkey
only
needs
one
single
trail
to
determine
exactly
which
of
the
two
objects
has
the
food,
and
that
of
that
is
nearly
perfect
performance,
in
other
words,
simply
by
being
exposed
to
many
episodes
of
the
task.
E
Now
there
are
many
other
examples
of
learning
based
memory,
guided
tasks
in
the
literature
one
example
that
is
quite
interesting
is
those
recent
studies
in
which
they
have
nice
learning
memory,
guided
navigation,
right,
yeah,
nice
learning,
memory,
guided
navigation
in
virtual
reality.
That
is
a
project,
a
virtual
reality
maze
on
a
screen
and
the
mouse
has
to
navigate
through
that
and
perform
a
memory.
It
did
a
navigation
task
in
this
virtual
maze.
E
Now,
clearly,
no
task,
no
mouse
has
ever
performed
this
task
in
the
evolution
of
mice.
I
think,
and
yet
the
mice
can
run
this
task
pretty
well,
which
shows
that
they
can
really
learn
truly
novel
tasks
that
they
have
not
seen
before
during
their
evolution
and
again
those
are
nice,
not
primates,
so
there's
quite
a
range
of
animals
you
can
do.
Those
memory
can
learn
those
novel
memory,
guided
tasks.
E
I'm
just
showing
a
few
examples
here,
but
there
are
many
more
and
they
all
propose
many
different
methods
for
solving
this
problem,
but
they
all
share
a
common
structure,
a
common
general
organization,
which
is
that
you
have
the
you
know
the
artificial
agents
performing
the
task
performing
one
episode
of
the
task
here.
That
will
be
the
six
trials
of
one
episode
with
the
same
objects
and
the
same
object
being
associated
with.
E
So
during
that
it
performs
the
task
and
then
between
episode,
you
optimize
the
learner
so
that
it
gets
better
and
better
over
successive
episodes
of
the
task.
In
other
words,
you
have
two
learning
loops
at
different
levels
embedded
into
each
other.
The
terminology
is
that
you
have
the
inner
loop.
That
is
the
within
episode
learning,
during
which
the
monthly
website
the
agent
performs
the
task
it
learns
when
it
needs
to
learn
to
perform
the
task
here.
That
will
be
the
identity
of
the
rewarded
objects
and
then
between
episodes.
E
E
Now,
here
already,
we
can
see
a
difference
between
what
happens
in
meta
learning
in
artificial
systems
and
what
happens
in
animals,
which
is
that
in
animals,
you
also
have
this
autoloop
supposedly,
but
while
the
monkeys,
the
the
actual
monkey,
performs
in
our
learning
with
its
own
working
memory,
the
outer
loop,
the
improvements,
the
the
optim,
the
optimization
of
the
system,
it
doesn't
occur
through
an
external
algorithm.
It
doesn't
occur
through
back
probe
or
gradient
descent.
It
occurs.
E
Now
this
ability,
this
internal
machinery,
it
did
not
appear
by
magic,
it
was
evolved,
that
is,
evolution
has
carefully
optimized
the
innate
structure
of
the
brain
in
order
to
undo
it
with
the
adequate
levels
and
amounts
and
organization
of
connectivity,
plasticity,
neural
relation,
etc.
So
that,
when
exposed
to
a
new
novel
task
and
just
stimulating
rewards
from
this
task,
animals
can
actually
learn
to
perform
those
tasks.
Just
from
stimulating.
What's
known,.
E
E
E
So
how
are
we
going
to
do
that?
The
model
that
we're
going
to
use
the
type
of
network
that
we're
going
to
use
is
a
simple,
recurrent
neural
network.
That
is,
it's
not
like.
The
deep
multi-layer
neural
networks
that
may
was
showing
beforehand
is
just
a
set
of
fully
connected
neurons,
which
receive
stimuli
and
also
reward
information
and
produce
responses
by
the
activity
of
their
neurons.
E
Now,
an
additional
subtlety
is
that
the
connections
between
those
neurons
they
come
with
an
innate
weight
that
is
a
standard,
normal
neural
network
weights
as
classical
neural
networks.
But
in
addition
to
that,
every
connection
also
has
a
certain
amount
of
plasticity
that
is
for
each
connection.
There
is
both
an
innate
weight
and
a
certain
amount
of
plasticity.
E
Now
some
connections
are
going
to
be
very
plastic
they're,
going
to
change
a
lot
as
a
result
of
exposure
to
stimulating
walls.
Others
are
not
going
to
be
very
plastic
at
all
they're
going
to
have
very
low
opacity
and
remain
constant
and
fixed
at
their
innate
weight
and
evolution
is
going
to
play
on
those
two
parameters
for
each
connection.
Those
are
the
two
structural
parameters
of
the
network,
which
evolution
will
optimize
to
endure
the
network
with
automatic
learning
abilities.
E
E
It's
a
simple,
descriptive,
but
effective
model
of
the
activity
of
dopamine
on
synaptic
plasticity
on
heavy
and
semantic
plasticity.
It's
been
applied
to
learn
many
different
things,
including
learning
of
songs
in
songbirds
learning
of
movement
and
reaching
models
of
the
human
arm.
Many
different
tasks,
it's
quite
simple.
E
The
way
it
works
is
that
you
apply
random
perturbations
to
the
activations
of
the
various
neurons,
and
then
you
apply
a
certain
plasticity
rule
a
reward,
modulated,
plasticity
rule
which
works
like
this.
That
is
the
weights
between
any
two
neurons.
I
and
j
is
modified
by
the
product
of
the
activity
of
the
input
neurons
times
the
perturbation
in
the
output
neuron
multiplied
by
the
reward.
E
E
So
this
is,
we
have
this
simple
reward,
modulated
learning
rule
and
an
additional
subtlety
in
our
model
is
that
the
change
in
the
connection
is
also
affected
by
the
evolutionary
plasticity
of
the
connection.
That
is
how
plastic
the
connection
is,
as
determined
by
evolution
again
again.
Connections
that
have
very
high
p
parameter
connections
that
are
very
plastic,
are
going
to
change
a
lot
as
an
effect
of
rewards
and
stimuli.
E
E
Now,
if
we
want
to
evolve
any
an
agent
to
be
able
to
learn
novel
tasks,
we
need
a
set
of
tasks
to
evolve
over.
So
how
are
we
going
to
find
a
sizeable
set
of
meta
learning
tasks
of
learning
to
learn
tasks?
For
that
we
turned
to
a
framework
from
computational
neuroscience,
the
framework
of
young
and
colleagues
who
implemented
a
lot
of
tasks
from
the
neuroscience
literature
from
actual
actual
tasks
from
animal
training,
literature
in
a
common
framework
where
all
the
tests
can
be
implemented
more
or
less
in
the
same
manner.
E
E
And
importantly,
one
of
those
tasks
is
going
to
be
withheld
from
the
training
sets
at
least
we're
going
to
keep
one
task
that
we're
never
going
to
show
to
the
agents
during
evolution
and
then
periodically
we're
going
to
test
the
agent
on
how
well
it
can
learn
this
task
that
it
has
never
seen
before
during
evolution.
Because
that's
what
we're
really
interested
in
not
just
the
ability
to
perform
certain
already
seen
tasks
with
the
ability
to
learn
novel
cognitive
tests
that
he
has
never
seen
before.
B
E
And
produces
a
response
and
receives
a
certain
reward
which
affects
the
neuromodulation,
as
explained
in
the
rule
that
I
talked
about
earlier,
and
so
some
neural
nation
is
applied.
Normally
plasticity
is
applied
to
the
network
as
explained
before,
and
you
repeat
that
400
times
for
with
400
trials
of
the
same
task
different
stimulated
time,
but
always
the
same
task,
and
so
the
agent
is
never
told
which
task
is
doing
exactly
it's
just
us
to
learn.
E
What's
what
it's
supposed
to
answer
for
any
part
stimuli
just
from
its
own
machinery
and
plasticity
now
so
far,
what
I'm
showing
here
is
exactly
the
same
structure
as
a
typical
metal
learning
experiment.
You
have
the
inner
loop
during
each
episode
and
then
you
have
an
outer
loop
where
between
episodes,
the
agent
is
optimized
to
perform
better
and
better,
except
that
the
agent
is
not
really
optimized.
It's
following
its
own
internal
machinery
to
modify
itself,
and
why
would
this
result
in
proper
learning?
E
That's
how
it's
going
to
work
again,
just
to
be
just
to
help
a
bit
seeing
things
we
have
three
different
groups:
the
inner
loop
during
each
episode,
the
task
loop
of
multiple
episodes,
during
which
you
learn
a
certain
task,
and
then
we
have
the
overall
outermost
loop.
The
evolutionary
group
during
which
you
learn
task
learning
ability
learning
the
ability
to
learn
any
kind
of
new
tasks.
That's
what
we're
really
interested
in.
That's
the
property
that
we
want
to
see
emerge
the
ability
to
learn,
novel
cognitive
tasks,
just
from
stimulating
rewards
alone.
E
E
E
So
what
do
we
get
as
results?
So
the
short
answer
is
that,
fortunately,
it
works
and
we're
quite
happy
with
that
here.
Each
data
point
represents
the
performance
of
the
agent
in
learning.
One
task
that
is
each
data
point
is
the
performance
over
the
400
trials
of
a
single
task
in
blue.
You
have
the
training
performance
that
is
the
performance
of
all
the
tasks
that
are
being
used
for
evolution.
E
That's
literally
the
evolutionary
curve
here,
so
it
lands
pretty
easily
and
pretty
well,
of
course,
because
those
are
the
tasks
from
the
evolutionary
training
sets.
What's
really
interesting
is
the
red
curve
here
the
actual
tests,
the
performance
on
this
test
that
it
has
never
seen
before
during
evolution
the
delayed
match
to
simple
task.
We
see
that
initially,
at
initially
when
the
network
has
been
randomly
initialized,
even
though
all
the
connections
are
plastic,
even
though
it
does
follow
this
remote
modulated,
persistent,
it
cannot
learn
the
task.
E
Of
course
it
can't
it
doesn't
have
the
structure
to
turn
this
blind
plasticity
into
efficient
learning,
but
as
time
goes
by
over
generations,
and
as
the
agent
is
exposed
to
more
different
tasks
and
goes
through
this
process
of
learning
and
evolutionary
optimization
slowly,
it
acquires
task
learning
abilities
until
eventually
near
the
end.
It
manages
to
learn
the
task
pretty
well,
even
though
again
it
has
never
seen
it
before
during
its
actual
evolution.
E
Another
way
to
see
the
performance
of
the
network
is
to
look
at
the
performance
during
each
episode,
that
is,
during
the
400
trials
of
the
task
which
I'm
going
to
show
now
now
here.
Each
of
those
graphs
represents
the
performance
of
the
agent
over
400
successive
trials
of
the
same
task.
Again,
both
of
those
are
for
the
test
task,
the
withheld
task
that
is
not
used
for
evolution
yeah
the
performance
is
average
over
eight
runs
now
at
generation
zero.
E
That
is
when
the
network
is
normally
initialized,
even
though
all
connections
have
plasticity
and
reward
modulation
and
neuromutation.
We
see
that
the
agent
is
completely
unable
to
learn
the
task,
but
it
goes
through
400
trials
of
the
task.
Every
time
it
receives
rewards
and
every
time
the
random,
the
numeration
and
the
plasticity
are
applied,
and
it
still
doesn't
learn
anything.
Of
course
it
doesn't
because
again
it's
not
a
trivial
task
and
this
structure
does
not
allow
it
to
convert
those
rewards
and
do
similar
into
proper
efficient
planning.
E
E
Here
I'm
showing
two
different
controls
to
make
sure
that's
two
safety
tests.
First,
I
try
what
happens
when
I
use
a
different
task
as
a
withheld
task
that
is
instead
of
using
delayed
match
to
sample.
I
use
the
logical
nouns
between
the
two
stimuli
and
we
see
that
it
works
even
better
because,
again,
as
I
said,
the
delay
master
sample
is
easily
the
most
challenging
task
of
the
whole
set.
E
Another
control
that
we
want
to
use
is:
do
we
actually
need
the
plasticity?
It's
not
a
silly
question,
because
there
are
many
studies
in
metal.
Learning
that
do
not
make
use
of
plasticity
and
still
agents
are
able
to
learn.
Reasonably
complex
task
here
is
very
unlikely
that
it
will
be
the
case
because
the
learning
will
have
to
extend
over
400
trials
each
of
a
whole
second
long,
just
by
storing
things
in
the
recurrent
activations,
which
is
extremely
unlikely,
but
just
to
be
sure
I
tried.
E
So
this
is
the
result
of
the
exact
same
thing
as
before,
but
without
the
plasticity
disabling,
the
reward,
modulated,
plasticity
and
just
having
evolution,
sculpting
the
innate
connection
of
the
network-
and
it
doesn't
learn
anything
at
all-
it
doesn't
acquire
successful
task.
Learning
abilities
that
is
after
500
generation
is
still
as
bad
as
the
beginning.
E
E
Now.
One
good
thing
about
this
system
is
that
what
I
showed
you
is
very
much
a
pretty
an
initial
project,
an
initial
version
of
the
project.
There
are
many
possible
ways
in
which
you
could
improve
it
and
make
it
both
more
powerful
and
more
interesting,
but
the
obvious
thing
it
would
be
to
have
more
complex
and
biologically
relevant
tasks,
which
is
just
implement
the
actual
original
test.
From
the
young
framework,
you
could
put
the
neural
relation
under
control
of
the
network
itself,
as
in
the
paper
that
may
was
talking
about
you
could
also.
E
This
is
maybe
a
bit
more
esoteric,
but
you
could
add
another
loop
between
those
three
groups
and
the
fourth
loop
in
the
system.
You
need
a
lifetime
loop
that
is
have
multiple
tasks
during
each
lifetime
and
see
how
well
it
does
continue
learning,
which
is
enough
time.
That
is
how
much
how
well
it
can
learn
new
tasks,
without
forgetting
the
old
ones
and,
if
possible,
by
having
some
forward
transfer
between
the
tasks.
E
B
B
E
B
Yeah,
that's
right,
it's
nice
to
think
about
them
in
a
very
structured
way,
and
also
you
know
thinking
about
the
previous
talks.
We
can
now
start
to
ask
you
know
how
do
the
different
neuromodulators
and
specific
properties
in
addition
to
what
you've
shown,
how
can
we
start
layering,
those
in
and
and
have
an
impact,
and
it
was
also
nice
that
you
broke
down
the
learning
rule
into
those
different
components
that
to
me
paralleled
a
lot
of
what
may
was
doing
with
several
different
components,
each
of
which
seems
to
be
required
in
here.
B
E
B
You
know
networks
learn
to
map
specific
inputs
to
specific
outputs
and
that's
about
it.
You
know,
as
you
showed,
that
acquiring
these
novel
tests
are
pretty
difficult.
I
mean
there
are
recurrent
networks,
but
they're
also
pretty
static.
But
then,
when
you
want
to
put
these
networks
into
real
applications
like
robotics
or
self-driving
cars,
then
you
have
to
write
a
lot
of
outer
loops
to
kind
of
manage
everything
properly,
because
essentially
the
network
itself
is
pretty
static
with
neural
modulators,
though
it
seems
like
we're
getting
a
very
different
picture.
B
You
know
you
know
very
broadly
speaking,
you
know,
neuromodulators
can
regulate
plasticity
in
a
number
of
different
ways.
You
can
regulate
overall
brain
state.
You
know
optimize
hyper
parameters
on
the
fly
optimize
for
novel
tasks.
You
know,
control
experimentation
and
foraging.
I
mean
you
kind
of
mentioned
that
a
little
bit
in
the
review
paper,
but
we
didn't
get
into
that
today.
So
it
seems
like
you
know,
and
also
the
impact
of
neuromodulators,
can
be
very
context.
Specific,
specific
to
cell
types
operate
at
different
time,
scales
and
spatial
scales.
B
So
is
it
you
know
the
picture
I'm
getting.
Neuromodulators
may
be
one
of
the
key
mechanisms
biology
biology
has
developed
that
sort
of
makes
humans
and
animals
really
dynamic,
flexible
functioning
autonomous
agents.
Do
you
guys
want
to
comment
on
that
is
this
is
a
very
sort
of
big
picture
way
of
thinking
about
it,
and
is
that
completely
out
of
whack?
Does
that
make
sense
to
you?
You
know
if
you
guys
can
comment
on
that.
That'd
be
great.
C
So
so
to
me.
Clearly,
yes,
these
neuromodulators
are
kind
of
the
the
the
the
cherry
on
the
cake.
If
I
may,
because
they're
really
exploiting
a
hardwired
circuit
diagram
to
carve
out
options
that
that
they
could
then
try
and
fill
the
cracks
and
and
and
get
the
whole
network
do
something,
depending
on
on
on
behavioral
and
and
environmental
contexts.
B
E
Yeah,
I
really
like
what
shrikhand
was
saying
about.
The
idea
of
reconfiguring
into
different
modes,
like
srikant,
has
shown
how
there
are
many
different
types
of
neural
modulators
and
different
sources
of
neuromodulation,
but
actually,
when
you
compare
the
number
and
source
of
your
modulators
to
the
immense
number
of
neurons
areas,
scots
columns,
it's
not
that
high
a
number
it's
relatively
low,
dimensional
like
dopamine,
for
example.
Recently
there
have
been
studies
which
have
shown
that
there
are
different
type
of
dopamine
neurons,
but
it's
not
that
large
number
many.
E
So,
to
speak,
the
enormous
diversity
and
complexity
of
of
the
whole
brain
and
to
guide
it
in
to
analyze
it
to
speak
into
a
certain
specific
predefined,
evolutionary
defined
set
of
behaviors
and
self-modification,
depending
on
the
circumstances
of
in
which
you're
finding
yourself
so
yeah.
I
really
like
the
sort
of
reconfiguration.
D
So
I
actually
have
a
couple
points
to
add,
and
the
first
thing
is
that,
like
the
studies
we
have
described,
especially
like
the
review
papers,
that
I
have
done,
this
artificial
modulation
in
your
networks,
the
thing
I
wanted
to
say
is
that
these
are
of,
of
course,
regardless
of,
if
that's
a
new
modularity
network
or
a
general
like
deep
neural
network,
we
can
see
that
there
are
really
small.
You
know
like
number
of
neurons
and
mostly
hundreds
of
them
and
actually
in
the
biological
brain.
D
The
modularity
is
a
bit
of
a
built-in,
for
example,
in
the
brainstem
or
in
the
spinal
cord.
There
are
different
classes
of
neurons
that
are
encoding
for
different
speed,
and
so
there's
actually
one
e-life
article
on
this
about
the
different.
You
know,
topological
organizations
of
brain
stem
neurons
that
are
encoding
different
speed
and
animal
can
just
switch
between
encoding
of
speed
by
by
having
a
different
excitability
of
these
classified
neurons.
D
So
these
are
building
modularities
in
the
biological
systems
and
of
course
we
have,
of
course,
these
neuromonitor
systems
that
are
integrating
or
innervating
different
populations,
and
this
is
built
in
modularity
to
me
and
of
course
like
even
you
know,
the
modern
network
of
modularity
is
a
way
that
has
been
suggested
to
address
a
continual
learning
problem.
Quite
well,
according
to
many
studies-
and
this
is-
I
think,
this
there's
also
a
really
nice
review
paper.
D
I
can
provide
a
link
to
that
as
well,
but
so
far
we
have
only
tested
a
modularity
in
a
kind
of
a
mechanic
way
and
we
have
the
same
neural
population
that
we
have
kind
of
a
manually
designed.
You
know
modular
network
and
it's
going
to
be
quite
interesting
to
see
how
this
can
be
learned
or
the
adaptive
way.
So
this
is
my
first
comment.
D
You
know
this
is
the
e
mother
did
a
lot
of
experiments
on
quotations,
neural
networks
and
how
the
new
modulators
play
a
role
regulating
them,
but
in
humans
and
primates
and
like
rodents,
we
have
kind
of
the
higher
order
functions,
for
example,
social,
cognition
and
execution
of
like
movements
of
high
dexterity
and
also
some
other
like
things
that
we
have
recently
evolved
as
human
beings.
I
think
it's
quite
interesting
to
understand
what
are
the
work
in
the
organization
of
neuromarketing
systems
in
different
species
across
time?
D
This
has
to
do
a
lot
with
the
evolution
of
evolutionary
perspective,
so,
for
example,
one
example
is
that
in
parkinson's
disease
or
cognitive
disease
these
are.
These
are
like
really
highly
modulated
by
neurons
in
striatum
and
the
basal
ganglia.
So
we
have
a
imbalance
in
in
dopaminergic
system
and
colonic
systems
in
certain
areas
inducing
a
lot
of
behavioral
problems.
So
this
is
something
that's
really
specifically
restricted
to
human.
D
I
think
like
apart
from
the
movement
perspective
like
by
movement
disorders
like
parkinson's,
so
we
maybe
it
might
be
interesting
to
see
if
we
can
kind
of
introduce
this
kind
of
different
evolution
and
directions.
You
know
not
only
towards
movement
and
navigation
or
pattern
or
soul
stage
about
towards
you,
know
social,
cognition
or
something
alike.
So
this
is
one
of
the
research
focus
of
actually
computational
psychiatry.
D
I
think
we
have
a
currently
interesting
that
so
I
think
that's
gonna
be
quite
interesting
to
maybe
like
already
to
talk
to
thomas
regarding,
like
we
can
kind
of
think
of
something
to
evolve
different
evolutionary
mechanisms
by
introducing
different
parameters
or
stuff
like
that,
and
it's
gonna
be
quite
an
interesting
story.
You
know
if
we
kind
of
think
of
the
average
trajectories
of
different
of
different
species,
yeah.
D
B
So
going
to
sort
of
the
first,
the
first
part
of
what
you
said
and
tying
also
to
what
thomas
and
chikan
said
about
this
modularity
and
having
a
relatively
low
number
of
neurons
that
are
involved
in
this,
and
I
think
this
relates
to
some
of
the
questions
in
the
q
a
as
well.
So
there
may
be
a
small
number
of
modules
or
a
relatively
low
number
of
cells,
which
is
good.
B
That
means
it's
more
tractable
in
some
sense,
at
least
from
a
coding
standpoint,
but
something
that
street
mentioned
is
that
it's
not
just
these
individual
things.
It's
it's!
The
interactions
between
them
are
incredibly
important,
so
maybe
you
guys
can
elaborate
on
that
and
I
think
the
experiments
also
kind
of
highlighted
that
and
the
questions
that
are
kind
of
there
in
the
in
the
q.
A
you
know.
Is
there
a
cascading
set
of
reactions
that
one
begets
another?
Is
there
balance
between
their
effects?
You
know.
B
Traditionally
people
have
thought
about
dopamine
and
isolation,
but
really
should
we
think
about
it
as
incorporation
and
opposition
with
other
neural
modulators.
So
there
are
a
number
of
questions
in
that
kind
of
vein.
In
the
linkage,
I
think
it's
a
pretty
important
area
to
think
through
here,
and
it
makes
that
actually
makes
it
a
little
more
complicated.
Unfortunately,.
C
Right,
so
so
indeed,
I
think
this,
this
cooperation
interaction,
even
competition,
a
fundamental
interaction
is,
is,
is
a
very
intriguing
organizing
principle
of
neuromodulatory
function.
Let's,
let's
just
go
back
a
step
to
this
whole
evolutionary,
a
point
that
was
raised
right
so
so
from
from
a
primitive
organism
like
c
elegans
to
mouse
to
monkey
to
man.
C
It
seems
that
a
lot
of
these
neuromodulatory
systems
are
actually
conserved
right,
so,
like
you'll,
find
dopaminergic
release
or
cholinergic
release
in
c
elegans,
as
you
do
in
mouse,
a
monkey
and
and
human
brains.
But
I
think,
as
as
you
you
climb
the
echelons
of
evolutionary
complexity,
I
mean
the
fact
that
there
are
a
lot
more
neurons
and
with
a
lot
more
neurons
and
and
increased
axons
and
dendrites.
C
This
seems
to
suggest
that
the
the
combinations
of
brain
states
that
these
neuromodulators
could
exploit
is
also
immensely
greater
right.
So
in
some
sense
the
the
function
of
these
neuromodulators
is
dictating
brain
state,
but,
on
the
other
hand,
to
get
to
a
certain
state.
The
brain
is
constantly
asking
neuromodulatory
systems
to
to
release
whatever
neuromodulator,
to
get
to
a
certain
state
right.
C
C
B
E
You
can
probably
look
it
up
online,
but
yeah.
So
it's
very,
very
general.
That
being
said,
of
course,
crickets
are
not
humans
or
rats
or
primates,
and
as
frequents
was
and
as
may
was
saying,
yeah.
The
different
evolutionary
pressures
on
different
types
of
animals
would
of
course,
produce
many
different
behaviors.
E
E
We're
still
not
quite
sure
what
exactly
dopamine
or
even
acetylcholine
do
to
a
natural
neuron
when,
when
they're
being
applied
to
it
in
vivo,
in
the
brain
like
in
actual
circumstances
where
they're
actually
working
so
yeah,
you
have
those
two
levels
at
which
really
there's
still
a
lot
to
learn,
and
I
think
computational
studies
do
have
a
role
to
play
on
both.
So
I
think
it's
important
to
remember
that.
B
B
D
Sorry,
oh
yeah
yeah,
exactly
exactly.
I
really
have
issues
with
japanese
names
of
stories,
so
so
they
recently
found
about
this
in
2014
after
the
development
of
of
optogenetic
tools.
So,
of
course,
in
the
future,
we
have
more
tools
to
really
differentiate
across
the
the
different
numerical
layers.
We'll
have
more
observations,
so
this
is
really.
D
You
know
something
that's
recently
occurring
a
bit
more
and
also
there
are
other
techniques
like
to
follow
imaging
engaging
in
parallel
with
that
and
also
a
lot
of
transgenic
transgenic
mouse
lines,
and
these
are
always
like
there
are
always
new
things
to
find
about
your
modulatory.
You
know
in
this
context
and
also,
I
think,
there's
a
paper
by
krishna
in
2013
or
something
they
found
a
region
in
the
brain.
D
That's
really
like
on
the
basilar's
floor
of
the
brain,
that's
innervated
by
all
five
big
five
neuromusculators,
but
they
all
know
like
there
are
five
new
models.
They
don't
know
what
exactly
each
one
is.
Each
one
is
doing
in
that
field.
So
it's
quite
interesting
and
also
you
know
when
it
comes
to
species.
There
are
some
preserved
neuromodulators
affecting
different
species,
for
example
for
the
mother
paper.
D
She
like
she
used
the
serotonin
and
and
doping,
but
for
crustaceans
there
are
like
other
neuromodulators
like
proctolin
and
there's
a
pi
pylocarbin
like
these
two
neuromodulators
that
are
not
so
much
found
in
other
mechanisms
and
then,
for
example,
primates
and
humans.
So
of
course
there's
a
generalizability
of
the
basic
neuromodulators
in
each
species,
but
of
course
we
have
a
bit
of
a
variance
across.
Maybe
we
can
find
you
know
like
functional
counterparts,
but
not
always
the
same.
So
I
think
this
is
something
that's
quite
interesting.
D
B
D
B
Thank
you
so
much
for
spending
two
hours
with
us
and
thanks
for
all
the
great
work
you're
doing
to
me,
it
seems
like
this
is
an
essential
component
of
ai.
It's
not
it's
not
an
optional
thing,
the
fact
that
it's
actually
preserved
across
this
wide
range
of
species.
I
didn't
really
realize
that
before.
But
you
know
it's
it's
a
loss
if
we
don't
consider
this
in
more
detail
in
ai,
in
my
opinion,
but
thank
you
again
for
participating.
B
D
Yeah,
of
course,
we
are
getting
an
intensive
studying
it
if
you
have
any
suggestions
or
comments
or
referring
coming
from
any
cognitive
fields,
we're
discussing
so
like
feel
free
to
reach
out
to
us,
because
we
are
always
looking
for
collaborators
from
different
fields.
We
really
need
to
kind
of
join
the
forces
to
make
it
work,
and
at
least
there
are
like
so
many
detailed.
You
know,
skills
we
wanted
to
address.
It's
definitely
helpful
to
really
talk
to
people
that
are
from
different
fields.
It'd
be
interesting.