►
Description
In this research meeting, guest presenters from the Neuromorphic AI Lab at Univ. Texas at San Antonio presented to the Numenta research team. Dr. Dhireesha Kudithipudi, Professor & Lab Director, and her team have been working on HTM related projects for nearly 7 years. Today, she and her student Abdullah M. Zyarah reviewed their recent paper, “Neuromorphic System for Spatial and Temporal Information Processing” published in IEEE Transactions on Computers 2019. The paper describes a complete memristor based implementation of HTM spatial pooling and temporal memory. Their system is continuously learning, and achieves impressive improvements in latency and power consumption.
Neuromorphic System for Spatial and Temporal Information Processing:
https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=9109629
A
A
B
A
C
Thank
you
so
much
super-tight
for
inviting
us
and
arranging
this
session.
As
you
said,
that
team
has
been
working
on
HTM
for
the
past
seven
years
or
six
years
now
and
we've
always
been
focused
more
on
the
HTM
hardware
and
I
think
it's
great
to
see
new
mentor
community
grow
significantly
over
the
past
few
years.
C
So
today's
work
is
primarily
going
to
focus
on
a
recent
research
article
in
I
Triple
E
transactions
on
computers,
which
is
Abdullah's
PhD
research
war,
but
before
he
delves
deeper
into
that,
I
wanted
to
give
you
a
little
bit
background
on
where
the
neuromorphic
systems
or
the
community
is
at
this
point,
because
you
mentioned
that
this
is
primarily
an
algorithms
or
neuroscience
community.
So
we
just
wanted
to
provide
that
short
debrief.
Oh
that'd.
C
C
There
are
several
applications
that
our
act
for
computer
memory
bound
deep
neural
networks,
and
there
are
several
scenarios
where
you
demand
low
latency
systems
for
inference,
but
also
there
is
a
new
class
of
interactive
real-world
applications
that
require
dynamic
learning
capabilities,
whether
it's
the
video
tracking.
This
is
a
fun
project.
Well,
that
was
done
by
the
students
for
a
class
assignment
in
my
lab.
You
can
see
that
for
a
lot
of
these
scenarios,
you
would
need
hardware
or
systems
that
can
process
this
kind
of
interactive
or
streaming
inputs
and
a
lot
of
times.
C
This
data
could
be
unstructured
and
it
is
rapidly
evolving.
So
we
need
for
these
type
of
scenarios.
There
is
a
need
to
identify
new
computer
era
dyes
that
can
support,
for
example,
in
the
SmartWatch
health
monitoring.
You
would
need
the
system
to
have
extremely
low
form
factor
and
energy
dissipation,
so
in
that
sense,
neuromorphic
systems
can
come
to
the
rescue.
C
So
if
we
look
at
the
original
definition
by
Carver
Mead,
he
envisioned
that
these
systems
are
making
the
neural
processing
in
silicon,
and
today
the
community
has
moved
to
a
much
broader
definition
per
se
within
this,
where
a
system
is
either
inspired
or
mimicking
the
neural
processing
in
silicon,
so
it
is
no
longer
pure
mimicking
of
a
neural
processing
in
silicon,
which
is
a
lot
more
challenging.
So
some
of
the
key
features
that
are
representative
of
these
systems
are
shown
in
this
B
plot.
C
You
can
see
sorry,
you
can
see
that
these
systems
have
are
robust.
They
can
process
more
efficiently,
streaming
inputs,
they
can
learn
in
situ
or
in
silicon,
and
a
lot
of
plasticity,
whether
it's
the
based
on
the
devices
we
are
using,
based
on
the
learning
we
have
with
these
algorithms.
So
there
is
a
lot
of
plasticity
at
different
levels
of
abstraction
and,
more
importantly,
the
driving
factor
or
the
thing
that
excites
the
community
about
these
systems
is
that
they
they
can
be
energy
efficient.
C
So
if
we
look
at
the
landscape
of
this
as
to
where
we
are
today
so
I
would
say,
we
are
really
in
this
first
generation
lot
of.
It
is
primarily
academic
and
research
phase,
ships
that
are
supporting
whether
it
is
spiking
version
or
non
spiking
operation
with
lot
of
event
based
representation
and
you
some
of
the
chips
that
are
getting
a
lot
of
attention
recently,
whether
it's
the
Intel,
slow
he
pokey,
Springs
or
IBM's.
True
north
and
few
other
startups
have
ventured
into
this.
C
You
know
morphic
systems
and
a
small
subset
of
them
can
support
true
learning
on
device
on
silicon.
So,
but
none
of
these
current
generation
ships
are
able
to
solve
complex,
real
world
problems.
If
you
think
of
that
DNN
accelerator
counterparts,
which
are
able
to
solve
complex
real-world
problems,
these
ships
are
not
there
step.
So
we
hope
that
in
the
second
generation
accelerators,
the
piece
can
address
some
of
these
challenges
and
hopefully
somewhere
down
the
line,
we
would
have
a
pure
standalone
systems.
C
So
what
is
the
challenge?
Then?
This
is
so
exciting.
Then
what
is
the
challenge
we
are
facing
in
a
community
right
now?
So
I
kind
of
divide
it
into
two
main
sets
of
challenges
in
here.
So
one
of
the
daunting
question
for
the
semiconductor
industry
or
in
general
for
the
hardware
community
is
as
scaling
today
in
CMOS
is
entering
an
uncharted
territory.
We
are
beyond
geometric
and
effective
scaling.
We
are
hitting
several
walls
there,
whether
it's
a
memory
wall,
thermal
wall
power
wall
and
can
be
even
shrink
these
device
sizes
further.
C
So
there
is
some
approaches
that
the
community
is
thinking
of
so
moving
away
from
the
geometric
and
effective
scaling.
Perhaps
we
think
about
this
in
a
hyper
scaling
era.
Beyond
twenty
twenty-five
sighs
aloud,
Dean
has
come
out
with
a
very
good
paper,
a
couple
of
years
back
in
there
on
this
front.
So
maybe
the
future
is
where
you
use
these
new
devices
and
lot
of
embedded
and
high-performance
memory
with
3d
solutions
that
could
offer
some
reprieve
in
the
challenges
we
are
facing
with
the
current
scaling
technologies.
C
But
then
there
is
a
specialized
set
of
problems
for
the
neuromorphic
accelerator
community
cells,
like
you
know,
customized
curveballs,
I
guess
so
we
we
have
a
challenge
in
one
of
the
critical
component
within
our
system,
the
synapse.
How
do
we
realize
these
efficiently
and
how
do
you
really
support
the
plasticity
on
device
can
be
support
through
plasticity
on
demise,
and
can
we
do
learning
in
silicon
in
silicon
for
complex
real-world
problems
and
and
several
others,
but
I'm
just
listing
the
relevant
ones?
For
today's
talk,
a
quick.
A
C
C
In
a
hardware
system,
I
mean
there
are
tricks,
and
how
do
you
do
it
right
now
there
are
hacks
and
how
you
can
do
it,
but
if
you
really
want
to
have
that
plasticity
at
multiple
levels,
whether
it's
structural
+20
in
different
types
of
that
which
I
like
how
do
we
really
achieve
that,
whereas
learning
inherently
is
like
can
be,
do
all
the
learning
on
device
dynamically
I
mean
in
a
compact
form
factor.
Can
we
do
the
learning
on
device
so.
A
C
So,
for
the
synapse
I
mean
there
was
a.
There
was
quite
some
progress
in
your
morphic
systems
in
early
90s
or
so
forth,
but
one
of
the
biggest
challenges
was:
how
do
we
realize
these
synapses
efficiently?
We
have
like
these
core
functionalities.
We
want
to
have
in
a
single
device.
We
want
to
provide
a
physical
interconnect.
C
You
want
to
modulate
the
signals
and
facilitate
some
type
of
adaptation
or
learning,
and
these
synapses
need
not
always
be
of
a
certain
bit
precision
when
we
are
thinking
of
Hardware
implementation,
they
need
to
be
able
to
support
this
dynamic
bit.
Precision
so
I
don't
have
to
tell
the
audience
here
about
the
total
number
of
synapses
we
see
are
significantly
large
in
a
biological
brain
and
some
of
the
silicon,
or
particularly
the
neuromorphic
chip.
So
you
see
support
anywhere
from
one
to
two,
fifty
six
million
synapses
per
Co.
C
So
what
is
the
challenge
is
like
you
want
to
design
this
unit
very
efficiently
when
you
have
such
a
large
number
of
these
synapses
on
your
course,
and
if
you
use
a
traditional
SRAM
in
CMOS
digital
hardware
implementation,
this
can
translate
to
significant
energy
and
area
cost.
So,
for
example,
some
of
these
are
not
even
non-volatile,
you
know,
so
you
lose
the
information,
so
that
is
not
an
efficient
solution.
C
So
it's
slightly
blurry
here,
so
you
might
have
heard
about
these
devices
from
my
earlier
talks,
but
one
synaptic
device
that
has
gained
lot
of
it
in
recent
times
is
the
memristor.
So
here
on,
the
cartoon
you
can
see
on
the
left
is
a
biological
synapse
and
on
the
right
is
a
memory
store
synapse
and
you
can
see
there
is
some
similarity
where
the
changes
in
the
resistance
state
is
based
on
the
flux
linkage
through
this
device.
C
So
if
I
apply
a
positive
right
voltage
to
this
device,
there
is
formation
of
this
conductive
filament,
depending
on
the
type
of
device
you
are
using.
It
could
be
different
types
of
materials
leading
the
device
to
a
low
resistance
state
and,
if
I
reverse
the
polarity
of
the
applied
voltage,
the
conductive
filament
gets
dissolved,
thus
leading
to
a
high
resistance,
and
you
can
do
this
both
for
inhibitory
and
excitatory.
Synapses
and
Abdullah
has
done
some
interesting
designs
around
it.
C
So
you
can
think
of
these
ions
participating
in
here
similar
to
how
the
neurotransmitters
are
released
at
the
onset
of
a
spike
and
so
forth.
So
there
is
what
are
the
similarities
and
there
are
different.
As
I
said,
there
are
different
materials
and
types
of
memory
stirs,
so
one
that
is
very
popular
and
one
that
we
have
been
using
on
in
our
lab
is
the
rear,
amp
device
and
the
right
energy
is
around
one
Pico
troll
and
the
cell
area
is
significantly
lower
than
that
of
the
CMOS
counterpart.
C
Where
F
here
is
the
feature
size
based
on
the
technology
node,
we
are
using
whether
it's
65
nanometer,
45,
nanometer
or
so
forth.
So
there
is
opportunity
to
save
area
and
energy
costs
by
using
these
type
of
devices.
Okay,
that's
really
good,
but
then
that
also
brings
us
into
a
second
set
of
challenges
that
you're
I
was
mentioning
earlier.
C
Can
we
do
this
use
these
devices?
That's
great.
We've
solved
the
problem
with
the
synapse
implementation,
but
can
we
learn
on
device?
Can
we
had
these
different
types
of
plasticity
rules
associated
with
the
neuromorphic
algorithms
on
device?
So,
and
can
they
do
this?
In
sorry,
can
can
we
do
the
learning
in
real
time
and
whether
these
systems
can
tolerate
noise
and
variability?
C
C
Individual
memory
stress
is,
what's
shown
in
this
here
right
here
and
then
this
one
is
in
the
in
the
context
of
a
network.
What
happens
if
there
is
noise
induced
in
several
of
these
devices?
How
does
the
performance
degrade
or
how
is
the
performance
affected
by
this
type
of
behavior?
So
in
some
cases
there
is
gradual
degradation
or
graceful
degradation,
but
in
a
lot
of
context
it
fails.
C
So
you
need
to
have
some
kind
of
symbiosis
between
the
algorithm
and
the
device
or
the
system
architecture
to
address
some
of
these
issues
and
Abdullah
is
going
to
talk
about
that.
So
the
idea
is:
can
our
understanding
of
neural
processing
lead
to
new
neuromorphic
algorithms
and
hardware
advances?
This
is
one
of
the
core
things
that
we
are
trying
to
address
in
our
lab
and
that's
where
HTM
really
the
design
effort
with
HTM
comes
into
play.
C
So
we
focus
initially
on
hardware
and
then
I
think
one
of
my
students
wanted
a
more
mathematical
grounded
framework
on
the
spatial
Pooler
james,
manatsu,
ganyan
and
since
then,
I
think
there
was
a
lot
of
progress
on
the
hardware
design.
So
with
that
I
would
like
abdullah
to
talk
about
the
HTM
design.
Would
you
like
to
share
your
screen
of
the
logo?
Yes,
so.
D
E
D
So,
as
dr.
kay
mention
earlier,
one
of
our
goals
is
actually
to
come
up
with
a
design
that
is
law
that
offer
low
latency
and
energy
efficient
for
our
little
temporal
memory.
So
the
so.
The
system
that
that
we
are
building
is
Politiken
inspired
because
it
is
already
based
on
article
temporal
memory
which
considered,
which
we
consider
as
a
colony
algorithm
for
our
model,
so
support
or
device
learning
and
real
time
processing.
It
is
fault,
tolerant
and
the
devices
that
we
are
targeting
as
like
end-user
is
going
to
be
the
eight
devices.
D
B
D
Do
so
here
we
have
the
designer
flow
for
our
project,
so
the
should
start
basically,
basically
by
extracting
the
silient
structure
on
the
algorithmic
properties
from
the
biological
neocortex
and
then
the
design
flow
progresses,
basically
by
mapping
the
HTML
amorphic
chip
and
it
ends
by
deploying
this
neuromorphic
chip
on
energy
concentrate
platforms
such
as
mobile
devices,
different
the
first
transition.
We
have
like
I,
believe
in
human
touch,
focusing
on
that.
So
in
my
presentation,
basically,
I
will
shed
light
only
on
this
transition.
Basically.
D
So
the
design
process
started
actually
back
in
2014
as
I
mentioned,
and
there
we
identify
with
all
the
bottlenecks
that
we
have
in
the
HTM
and
here
I
divided
into
levels.
So
on
the
first
of
all,
we
do
have
the
inhibition
and
the
reason
why
identify
identify
inhibition
operation
as
one
of
the
bottlenecks
operation
that
we
have
in
HTM,
because
this
operation
is
essential
because
it
forces
the
sparsity
in
the
network.
D
But
at
the
same
time
it
require
a
lot
of
data
movement,
because
many
columns
in
the
region
need
to
talk
with
each
other
to
determine
the
set
of
active
mini
column
that
represent
our
input
and
also
these
operations
who
are
to
provide.
So
one
of
our
priorities
were
actually
was
to
come
up
with
a
circuit
that
can
implement
inhibition
without
the
need
for
data
movement.
Also,
we
need
to
have
the
for
wisdom,
so
that
was
one
of
our
priorities
and
we
did
it
in
a
paper.
D
So
we
develop
a
winner-take-all
circuit
that
will
do
the
inhibition
and
that
by
2018.
So
the
other
challenge
which
which
we
solve
it
in
this
and
they're.
Our
new
paper
was
for
the
problem
of
dynamic
structure,
so,
as
all
of
you
know
that
HTML
Gotham
keeps
evolving
as
its
own
model
of
the
world
and
having
such
dynamical
structure,
it's
really
hard
to
implement
in
hardware.
So
there
is
a
you
can
follow.
D
For
example,
you
can
use
memory
unit
to
have
to
emulate
this
dynamical
structure,
but
using
memory
units
is
a
probably
expensive
in
terms
of
energy,
consumption
and
area.
So
for
one
of
so.
For
this
reason
we
could
try
to
carry
on
this
work.
We
came
up
with
a
communication
scheme
that
slightly
will
lie
on
the
on
the
memory
unit,
because
synthetic
synapses
representations,
so
it
had
to
know
about
the
dynamic
structure
but
table
you
rely
on
the
random
generators
rather
than
like
any
more
unit.
D
So
the
second
level
we
do
have
a
set
of
operation
that
require
a
lot
of
data
movement
on
compute,
and
here
we
identify
the
or
computing
overlap
computing
priminister
tuning
on
the
Sigma
evaluation.
So
with
this
operation,
the
problem
is
not
actually
with
the
compute
as
much
as
the
problem
with
the
data
over
so
data
movement
in
hardware
is
very
costly
process.
D
If
we
compare
to
the
computer-
and
here
I
will
give
you
an
example
so
reading
one
like
32
bit
from
neuron,
like
cost
four
to
five
times
order
of
magnitudes
if
we
compared
to
perform
and
add
operation
and
that
using
the
same
technology,
so
our
goal
is
not
as
like:
reducing
the
the
computer
as
much
as
reducing
the
data
movement
and
in
order
to
reduce
the
data
movement,
we
need
to
have
this
shift
in
the
technology
as
dr.
K
mentioned
earlier.
D
However,
the
process
like
not
straightforward,
because
by
then
we
need
a
member,
so
devices
that
will
capture
the
synaptic
behavior
like
an
STM
like
a
TM
synapse,
is
very
unique
to
the
binary
in
nature
and
has
multiple
hidden
estates,
and
we
need
basically
a
synapse
that
will
exhibit
this
behavior
and
I
believe
like
earlier
in
2000.
Only
in
2018
I
came
across
this.
D
So,
and
this
also,
we
stopped
working
on
the
synapse
formation
and
here
basically,
basically
mostly
on
the
communication
scheme.
So,
as
I
mentioned,
you
can
like
emulate
all
the
dynamic
and
the
network
by
using
like
memory
unit,
and
for
that
reason,
for
that
you
can
use
one
of
the
available
communication
schemes
such
as
then
has
address
events
representation,
but
for
the
architecture
that
we
bought
for
its
required
72
megabyte
just
to
emulate
the
synaptic
connections.
D
This
is
too
much
of
memory
and
most
of
these
communication
scheme
actually
are
designed
to
to
work
for
an
internship
communications
not
like
within
the
chip.
It's
all
so.
For
this
reason,
one
of
our
priorities
like
in
this
work
is
basically
to
come
up
with
a
communication
scheme
that
try
to
rely
on
the
memories
and
also
it
used
the
alcohol
like
random
generators,
with
all
the
linear
feedback
shift
register
to
formulate
the
synaptic
connections.
B
D
D
B
A
A
B
B
B
D
D
So
so
this
is
part
of
the
design
methodology
that
would
cover
this
slide
and
this
and
this
presentation
so
now
I'll
talk
about
the
system
design.
So
in
the
system
design
we
have
three
core
parts,
so
we
do
have
the
in
color,
which
basically
take
all
the
data
and
translate
it
into
high
dimensional
binary
vectors.
D
So
the
the
operation
of
the
circuit
works
as
follows.
So
you
present
you
a
bit
here
everything
getting
coded,
because
the
output
of
thinkers
is
going
to
be
like
high
dimensional
binary
vector,
so
we
cannot
transfer
them
all
like
at
the
same
time,
so
we
divide
them
into
packet
and
the
flu
interrupter
will
go
to
the
to
the
region
and
the
process
each
packet
separately.
D
B
D
B
B
D
D
So
now
we
talk
about
the
individual
processing
unit,
so
so
this
is
the
mini-com
on
the
circuit
that
we
use
for
the
mini
:.
So
here
we
basically
here
we
locally
formulate
all
the
proximal
connections.
It
is
not
memory
based,
I,
never
learned,
I,
love,
source
and
and
also
that
the
inhibition
I
told
you
earlier
like
it
is
one
of
the
bottlenecks.
So
here
the
we
will
do,
the
other
inhibitions
that's
happening.
D
Wouldn't
origin
can
happen
concurrently
and
thanks
for
the
winner-take-all
circuit
that
each
middle
column
have
has,
and
also
this,
because
a
lot
of
operations
like
happening
here
like
in
power
such
as
the
overlap
and
like
you
in
the
synaptic
connection,
usually
this
operation
take
too
much
time
at
here
is
happening
in
parallel,
and
this
will
give
us
a
lot
of
speed
up.
So
the
proximal
quick
in
there.
A
D
One
I
believe
you've
sorted
the
paper
that
one
we
use
to
inhibit
the
spell
within
the
same,
a
mini
column,
and
we
do
have
this
different
winner-take-all
circuit,
which
used
to
select.
There
are
many
columns,
because
here
the
one
we
use
here
because
in
for
the
mini
:,
you
need
multiple
winners,
so
you
need
to
use
K
winner-take-all
circuit
in
order
to
select,
let's
say,
40
percent
forty
many
columns
out
of
1000
or
2000,
but
in
the
cells.
In
the
case
of
the
series
pick
all
you
want,
so
you
need
one
winner-take-all
knot.
A
C
D
A
D
So
we
have
three
units
here.
We
have
the
profile
unit
and
here
basically
you
are
formally
forming
all
the
synaptic
pathways.
We
do
have
the
proximal
unit,
and
here
we
have
the
registers
and
basically
humor,
storing
the
status
of
the
individual
synaptic
connections
with
that
is
connected
to
it,
an
activator
or
was
connected
to
enact
a
bill
and
the
citrons
of
the
individuals.
Natick
connections
is
emulated
here
by
using
Mercer's.
So
the
overlap
here
is
computed
also
here,
and
it
is
same.
Why
don't
we
not
take
all
so?
D
This
is
going
to
take
all
exists
in
every
minute,
every
mini
column,
and
they
are
connected
together
through
this
dot
line,
and
here
on
the
side,
we
do
have
a
waveform,
that's
showing
the
process
of
computing,
the
overlap
and
how
there's
negative
changes
for
one
medical,
the
soap
opera
on
hardware
more
and
here
are
showing
only
to
proximal
connections.
Other
words
going
to
be
messy,
so
so
one
thing
you
can
observe
from
the
I
mean
there
are
some
I
think
you
can
observe
from
the
plot.
D
For
example,
the
changes
in
their
proximal
connections.
Permanence
is
nonlinear
in
the
hardwood
model,
but
it
is
linear
the
software.
Another
thing
you
can
observe
is
that
all
the
changes
that's
happening
in
the
synaptic
connections
as
no
does
it's
not
crossing
the
threshold
doesn't
have
any
impact
on
the
overlap.
It's
called
like
in
the
software
model
and
the
hardware
model.
This
is
not
the
case
like
either
like
many
small
changes
like
in
the
the
conductance
can
have
can
have
like
impact
and
Ovilus
overlaps
core,
but.
B
D
D
A
D
Okay,
now
so
for
the
HTM
cell,
so
so
I
did
so
that
this
so
the
C
on
HTM
is
fairly
complicated
because
it
let
us
compared
to
the
mini
:,
because
a
lot
of
operations
having
here
are
a
little
bit
different
and
also
more
complex.
But
here
we
take
advantage
of
the
sparsity
that
we
have
an
HDMI
capture.
That
is
like
somehow
relatively
simple.
D
B
B
D
D
So
we
have
here
like
three
units,
so
we
have
the
synaptogenesis
where
we
are
basically
forming
all
the
stock
connections
here
and
also
we
have
a
set
of
memory,
sir,
to
describe
the
strength
of
the
individual
connections,
and
here
we
have
a
set
of
current
comparators
on
summer,
basically
to
evaluate
the
activities
that
are
detected
at
the
Sigma,
because
I
mean,
as
I
mentioned
earlier.
Here
we
are
forming
all
the
synaptic
connection
probabilistically,
because
we
are
using
a
LOF
Asura.
D
So
there
is
always
a
likelihood
to
form
a
connection
and
not
to
form
a
connection
with
the
cell
that
were
active.
The
previous
time
I
said,
and
this
kind
of
luck
and
forest
won't
force
us
to
use
like
a
big
segment
in
order
to
always
have
a
high
likelihood
to
from
connection
with,
at
least
with
20%
of
the
cell,
that
were
active
in
the
previous
time
step
and
and
also
we
apply
the
Union
property
so
that
we
can
cut
down
the
number
of
like
the
segments
that
we
have
in
the
hardware.
D
So
that
was
pretty
much
about
the
processing
element
and
the
network,
so
regarding
the
arbiter.
So
as
I
mentioned
earlier
here,
we
sort
is
responsible
for
regulating
data
sharing
among
the
processing
element
that
we
have
in
the
cell.
We
didn't
use
random
arbiter.
We
set
your
priority
arbiter
so
that
we
can
have
hard
channel
capacity
when
we
are
transforming
our
data.
D
D
So,
regarding
the
results,
so
we
evaluated
our
design
with
a
different
data
set,
so
the
one
that
we
reported
in
the
port
and
the
paper
was
the
Hajin
data
set
and
exceeding
New
York
New
York,
City,
Taxi
demand
and
in
both
data
set.
So
our
goal
was
to
make
prediction
for
the
next
and
five
steps,
and
here
I
will
talk
about
the
Hajin
data
set.
So
our
goal
was
basically
to
predict
the
power
consumption
for
the
next
two
and
five
hours
and
the
metric
that
we
use
for
the
evaluation
was
the
mean
absolute
percentage
err.
D
D
So
so
initially
so
here
we
are
recording
the
mean
absolute
present,
your
we
are
measuring
after
presenting
250
every
time
we
are
presenting
250
samples,
so
initially
the
mean
absolute
percentage.
Error
was
really
high,
but
as
the
network
start
to
get
familiar
with
them
with
pattern,
so
start
to
kind
of
settle,
but
we
still
have
this
gap
between
the
software
and
the
hardware
model
I'm.
Also
these,
like
actually
attributed
to
their
own
ideologies
and
demersal
devices
and
the
way
pole
you
are
forming
the
connection
with
the
previous
cells,
which
is
twice
probabilistic
and.
D
D
We
did
change
it
and
write
it
back
and
each
each
read
or
write
cycle
at
least
and
the
best
and
the
best
in
our
you,
because
you
want
side
clock
cycle,
but
in
the
a
lot
like,
if
you
are
using
reversal
devices,
so
these
operations
can
take
almost
no
time,
for
example,
if
you're
computing
the
overlap.
So
what
you
need
apply
the
voltages
here
and
the
current
that
will
collected
here
is
gonna,
be
your
overlap,
score
either
so
for
tuning.
D
A
D
A
D
You
can
but,
but
there
are
a
lot
of
challenges
once
we
start
to
go
beyond
these
numbers,
but
this
is
problem
can
be
solved
by
doing
what
we
call
slicing
so
in
2018
I
believe
so
we
are
propose.
We
talk
about
the
slicing
in
HTM,
so
rather
than
having
like
a
huge
region
where
you
have
like
thousands
of
many
columns,
so
you
can
cluster
every,
for
example,
two
thousand
mini
column
and
it
is
associated
with
in
one
region.
I
have
a
like
another
region
next
to
it.
Okay,
and
this
can
interact
to
our
router
know,.
D
C
As
we
fabricated
the
learning
component
on
the
memristor
crossbar
at
SUNY
Albany
and
on
IBM
65
nanometer
technology
node,
and
it's
going
through
the
testing
phase
right
now
and
in
the
next
year
we
are
planning
to
fabricate
a
full-scale
learning
system,
not
necessarily
HTM,
but
we
are
trying
to
see
if
we
can
get
to
the
next
stages
of
fabricating.
Even
the
some
of
the
ideas
we
have
for
the
HTM
I'm.
C
So
we
are
at
65,
nanometer,
I,
guess
that
gives
us
some
bigger
room
and
so
all
those
Pvt
tests,
initial
tests,
the
design,
looks
good
right
now
and
I
think
one
of
the
things
we've
done
is
also
kind
of
having
this
built-in
robustness
in
the
architecture
design.
So
even
if
there
is
variability,
I've
shown
this
in
one
of
my
earlier
slides,
there
is
a
lot
of
variability
associated
with
these
devices
and
based
on
the
type
of
device
the
variability
could
be
large
or
so
that
the
bounds
can
be
quite
varying.
C
So
in
how
you
design
your
architecture
and
how
you're
tuning
the
learning
within
the
algorithm,
you
could
ensure
that
you
can
overcome
some
of
these
challenges
so
in
our
doulas
design.
I
think
that
is
one
of
the
interesting
components
where
he's
shown
that,
even
if
you
have
some
stuck
at
faults
or
some
of
them
not
working
during
runtime
you're
still
able
to
achieve
the
performance
with
very
with
some
graceful
degradation
and
the
65
nanometer
process,
I
think
it's
going
to
be
shelved
pretty
soon.
C
Maybe
it
will
be
there
for
a
year
or
so,
but
I
think
the
process
is
pretty
tuned
in
Albany,
so
they
were
able
to
do
this
rigorous
testing,
because
the
devices
where
went
through
several
generations,
so
I
would
say
they
are
a
pretty
decent
I,
wouldn't
say
they're
the
best
but
they're
decent
and
I.
Think
this
for
any
of
these
emerging
devices.
C
Lack
of
inherent
or
or
endurance
issues
in
some
scenarios,
so
how
do
you
kind
of
take
advantage
of
those
instead
of
fighting
against
them?
So
that's
the
trend,
that's
happening
in
general
in
the
semiconductor
industry
right
now,
even
if
you
go
with
your
CMOS
as
we
are
scaling
to
really
small
technology
nodes,
we
are
seeing
a
lot
of
these
issues
so,
instead
of
fighting
with
them,
how
do
we
embrace
it
in
the
design
process?
That's.
E
What
I
was
curious,
whether
you
had
a
process
for
I
mean
since
there
were
reserves
there,
their
ability
to
switch
so
to
speak
could
be
variable
whether
you
do
a
characterization
to
see
what
there
would
be
and
maybe
gang
them
together
and
pair.
You
know
low
highs
together
or
something
like
you
know,
it's
something
to
make
it
more
robust.
Some
kind
of
built-in
robustness.
C
So,
yes,
we
did
that
in
this
specific
one,
I,
don't
know
Abdullah
you're
going
to
show
any
of
the
techniques
you
used
to
enhance
the
robustness,
but
we
did
that
in
some
of
our
other
papers
and
how
we
are
pairing,
the
devices
how
we
are
introducing
redundancy
in
the
number
of
devices
we
are
using,
because
using
a
single
memristor
is
actually
or
sorry
the
area
associated
or
the
cost
associated
with
a
single
memory
store
is
very
low.
Lot
of
the
cost
associated
with
these
kind
of
devices
is
the
peripheral
circuitry.
C
So
so,
with
that
we
could
actually
increase
the
number
of
devices
you
are
using
to
represent
single
inhibitory
or
excitatory
operation.
So
in
some
of
our
earlier
papers
we
did
that
like
instead
of
using
a
single
device,
we
use
we
paired
up,
like
10
of
them,
for
inhibition,
intend
for
exciter
that
the
kögel
cost
doesn't
go
up
because
lot
of
it
is
associated
with
the
peripheral
circuitry
still,
even
if
we
super
optimize
the
crossbar
so
and
that
one
was
CBM
devices
which
are
highly
variable,
so
we
had
an
problem
which
have
probabilistic
switching.
C
So
we
had
to
use
that
approach
in
in.
In
that
scenario,
and
in
some
cases
we
tweaked
how
the
learning
itself
happens,
we
kind
of
adapt
slightly
with
different
learning
rules
when
we
are
deploying
them
on
hardware
or
for
these
crossbar
architectures.
That's
how
we
addressed
in
some
scenarios
and
in
other
cases,
by
the
nature
of
the
algorithm
itself,
I
think
it.
It
has
a
built
in
robustness,
so
I
think
it's
a
layer
approach
in
a
way
when
you're
trying
to
work
with
these
kind
of
devices.
C
E
C
Mostly
at
the
circuit
level,
beyond
on
device
researchers,
our
collaborators
are
the
ones
who
fabricate
right
and
okay,
these
are
all
available.
This
is
a
variability
you
have
with
this
device
and
all
the
characterization
data
comes
from
them,
but
then,
once
we
have
that
information,
how
are
we
going
to
work
with
this
system?
So
a
lot
of
circuit
level
techniques
yeah?
Thank
you.
D
D
So
here
we
did
analysis
about
the
estimating
the
lifespan
of
the
network,
so
so
the
Mercer
is
great.
Let's
come
to
the
projection,
I'd
like
to
energy
politician
and
adducing
the
latency
event.
Also,
it
has
a
lot
of
challenges
at
the
4k
mention
earlier
when
she
talked
about
the
variable
T.
So
the
endurance
is
another
challenge.
Yugi
these
devices
have
like
limited
life,
and
this
is
defined
by
then
doings
like
how
many
times
you
basically
can
write
to
the
to
the
memory
element
and
for
the
rerun
that
we
are
using
this.
D
What
is
between,
like
the
indoors,
usually
between
10
to
the
6,
go
up
10
to
the
12.
So
for
our
analysis,
we
use
10
to
the
9
like
redundant
model,
which
is
which
I
saw
it
like
in
most
of
the
paper.
Actually,
the
you
they've
been
doing
since
10
to
the
9,
and
my
goal
here
was
actually
to
estimate
the
lifespan
of
the
network
and
what
I
mean
by
the
life
is
fine.
D
What
is
the
point
at
which
the
network
will
stop
learning,
because
we
are
at
least
probably
lost
the
demurrer
devices
capability
for
changing
and
for
the
for
the
network
that
we
use,
which
has
961
mini
column.
So
so
the
life
of
the
network
can
go
up
to
like
8
years,
but
over
time
you
would
expect
to
you
to
lose
some
of
them
in
columns.
Oh
man,
some
of
the
MILCON
will
become
an
elastic,
so
they
lose
their
capability
basically
for
learning.
A
D
D
And
I
believe
this
number
also
can
be
improved
like
if
we
find
a
better
way
to
like
apply
the
boosting,
because
the
boosting
factor
will
basically
regulate
help.
We
are
activating
the
V
columns
like
so
regarding
the
power
analysis,
so
the
average
port
consumption
for
the
network
that
we
built
was
a
twenty
nine.
Forty
point,
three
eight
million,
and
it
was
like
sort
of
magnitude
saving
if
we
compare
to
the
digital
design.
So
here
we
have
upload
that
show
like
an
estimation
for
the
port
conception
as
we
move
like
at
the
time.
D
What's
once
we
start
to
compute
the
overlap
score,
so
we
have
this
upper
increase
in
the
power
consumption
and
because
here
all
the
mini
columns,
we
are
computing.
The
overlap.
Probably
we
need
to
find
out
another
way
to
reduce
this
like.
Well,
then,
like
computing,
the
over
level,
all
the
mini
come
at
the
same
time
like
different
sessions
or
another.
D
So
here,
because
the
training
is
limited
only
for
the
active
mini
:.
So
we
have
this
small
spike
in
the
poor
increase
and
then
once
the
shell
start
to
own
the
computation.
We
have
this
increase
in
the
power
consumption,
and
this
is
because
we
start
this
increase
happening
here.
It
is
because
of
the
use
of
the
distal
segment
here
on
the
side,
and
this
is
all
for
the
tickler
sixty-five
technology,
not
for
the
energy
delay
product.
D
D
So
here
we
are
showing
the
roadmap
that
we
follow
basically
xx.
The
latency
on
energy
efficiency
of
the
network,
so
I
see
em
as
algorithm
gave
us
really
a
lot
of
advantages
when
it
come
to
a
new
data,
movement
and
arithmetic
cooperation
and
complexity,
because
it
offer
like
local
learning.
So
we
produced
the
data
movement,
but
the
story
do
have
this
all
the
mini
column,
at
least
the
sparsity
in
neuronal
activities,
and
the
connections
also
helped
a
lot
and
seen
a
lot
of
memory,
and
and
also
we
do
have
much
lower
precision
arithmetic.
D
So
most
of
them
are
binary,
and
that
was
great,
but
not
enough.
Unfortunately,
to
build
the
system
that
we
can
use
it
on
a
devices
and
usually
these
devices
have
very
limited
or
tight
budget,
so
our
goals
tried
use
the
memory.
Access
within
the
souls
are
Minicon
themselves.
We
want
to
increase
the
a
memory
computing
and
also
we
because
we
have
like
many
sellers-
are
many
common.
Most
of
them
are
not
used
in
the
computations
or
we
are
using
part
of
them
every
time.
D
Well,
the
network
is
evolved
in
particular,
computation,
so
want
to
have
a
kind
of
depended
driven
a
unit,
and
this
is
what
we
capture
most
of
these.
We
capture
like
in
our
design,
so
we
have
them
memory
computing
when,
because
we
use
them
every
such
devices
and
also
we
use
the
data
44
time
amps
and
we
compared
to
digital
design,
also
where
I
would
like
to
shut
off
some
of
the
original.
D
Basically,
some
of
the
unused
unit
so
cut
down
the
bottom
top
Shannon
Marie,
so
I
present
general
overview
of
the
disturb
based
Mac
signal
designs,
mixed
signal,
HTM
architecture.
So
the
design
that
we
talked
about
offer
super
plasticity
mechanism
such
as
napped
or
jesse's
neurogenesis
on
synaptic
plasticity,
some
of
them
I
didn't
talk
about
it,
but
I
can
show
you
the
paper
that
talked
about
this
mechanism
and
how
we
went
over
implementing
them.
D
So
we
had
like
continuous,
in-situ
learning
and
also
we
demonstrated
our
architecture
on
a
spatial
and
temporal
information
processing
so
for
the
system
evaluation.
So
we
had
like
increase
in
the
latent
decrease
in
the
latency,
so
our
design
was
three
times
faster
compared
to
pure
CMOS
implementation
and-
and
if
we
are
running
then
both
are
the
same
speed.
Probably
this
number
will
be
doubled
like
several
times
for
the
energy
today.
D
A
B
C
A
D
The
best
of
our
knowledge,
or
at
least
my
knowledge,
so
this
is
the
first
one
that
capture
all
the
spec
that
we
have
in
the
HTM,
because
all
their
previous,
like
at
least
we
are
talking
about
the
mixed
signal
design,
because
all
the
previous
mixed
signal
design
implementation
of
the
temporal
memo
it's
based
on
them.
They
caught
the
magic
process,
so
they
have
like
they
store
like
they
have
a
command.
A
You
know
even
thinking
really
broadly
in
the
field
of
neuromorphic
computing.
You
know,
they're
a
show,
you
kind
of
started
out
saying
that
you
know
it's
been.
It's
been
hard
to
kind
of
cross.
That
chasm
over
to
where
the
GPUs
are
and,
and
so
on,
I
mean
this
seems
to
be
now.
You
have
a
system
which
could
potentially
do
that
is
there
yeah
and
it
could
have
significant
advantage
over
the
existing
solutions.
Yeah.
C
C
Not
a
continuous
learning
yeah
that
is
not
shown
yet
I
think,
but
there
was
some
demonstrations
of
how
it
can
learn:
fewer
samples
and
so
forth,
but
I
I
don't
think
we
have
like
a
full-scale
continuous
learning
algorithm
and
an
in
general
right.
The
research
in
continuous
learning,
algorithms
or
lifelong
learning,
algorithms
is
itself
in
an
evolutionary
phase
right.
There
are
a
lot
of
them
out
there
and
I
think
so.
I
think
taking
this
baseline
with
HTM
gives
us
an
advantage
in
that
sense
for
Abdullah's
research.
C
B
That's
obvious
I
mean
it
is
still
an
academic
exercise,
but
it's
it's.
A
very
important
I
can
exercise,
and
you
know
it
goes
back
all
the
time
back
when
Carver
Mead
started
this,
which
is
what
in
the
1870s
I,
don't
remember
that
was
but
I
think
you
know,
the
research
community
at
large
is
trying
to
would
like
to
know,
there's
going
to
be
an
end
in
sort
of
design
paradigm,
for
you
know
really
whether
it's
edge
or
not.
B
There's
going
to
be
M
paradigm
for
how
intelligent
machines
are
built
and
we're
not
even
close
to
that
today,
and
so
you
know,
I
think
you're
really
pushing
the
edge
here
on
many
many
fronts
and
it's
impressive,
even
though
it's
not
a
commercial
product
and
neither
was
the
algorithm
really
a
commercial
product,
but
it's
still
pretty
impressive.
I
haven't
I
have
a
specific
question
for
you
about
this.
It
may
be
obvious,
but
it's
it's
a
stupid
thing.
I
just
want
to
ask
it
as
you
go
through
the
designs.
You're
simulating
this
on
these
Hardware
simulators.
C
C
So
I
think
definitely
that
is
one
of
the
bigger
challenges,
as
we
are
scaling
up
these
systems
and
I.
Think
in
the
earlier
architecture
of
the
lower
showing
like
we
abused
some
h3
models
in
try
in
terms
of
kind
of
transferring
this
data
to
different
cells
or
different
units,
but
I
think
in
lot
of
academic
efforts-
and
this
was
I
think
goes
back
to
quite
a
few
years.
C
What
we
see
is
like
the
assumption
that
the
data
is
already
available
to
you
and
then
what
you're
doing
with
it
right
somehow
the
data
is
already
coming
in
and
then
how
are
you
going
to
move
it
around
so
that
there
are
two
challenges?
The
data
movement
cost
is
what's
happening
within
the
chip,
how
you're
moving
that?
You
don't
want
that
to
be
too
high
because
it's
expensive.
C
C
B
It
gets
much
worse
with
the
kind
of
dynamic
architectures
that
you
know.
Synaptogenesis
leads
to
number
of
connections.
You
know
real
neurons
have
thousands
of
connections.
Yes,
I
mean
this
is
really
puzzling
problems.
You
know,
and
one
of
my
criticisms
of
a
lot
of
people
work
on
camera
stirs
in
the
past,
but
they've
always
for
the
at
least
for
many
years.
I
don't
keep
up
with
your
field,
but
for
many
years
those
those
efforts
were
really
isolated.
Like
oh,
let's
just
get
the
image
to
look
like
a
bunch
of
sit.
C
Yes,
yes,
yes,
the
community
has
come
far
along
since
then,
but
again,
there
is
like
right
now,
hundreds
of
devices
you
can
choose
in
memristors
different
material
stacks
and
each
of
them
comes
with
its
own
set
of
challenges,
but
people
are
starting
to
build
small
scale
systems
with
it
now.
So
there
is
some
demonstrations,
but
still
there's
a
long
way
to
go.
D
Your
questions
great,
however,
this
is
I
mean
you
have
to
consider
the
routing
problem,
but
this
is
the
case
if
you
are
really
having
the
physical
connections,
but
in
our
design
we
don't
have
physical
connections,
so
everything
is
very
local
Nationals.
So
we
don't
have
to
worry
much
about
the
routing,
but.
B
D
We
do
have
I
mean
definitely,
there
are
like
a
lot
of
connections,
gonna
be
there.
For
this
reason,
what
I
asked
me
about
like
scaling?
This
I
told
him
rather
than
having
like
everything
cluster
in
one
region.
We
divide
it
into
slices,
so
every
subway
you
will
be
either
like
by
itself
or,
like
can
be.
We
can
have
it
on
a
separated
chip,
yeah.
A
D
The
wrong
thing
is
mainly:
this
is
something
to
worry
about
only
for
for
that,
for
basically
connecting
the
main
unit
that
we
have
in
the
network
was
not
like
for
forming
or
lateral
movement
connection,
because
all
these
actually
are
formed
like
synthetic
matter.
It's
not
like
there
is
a
real
physical
connection.
Yeah
I
understand.
D
D
Basic
when
we
received
a
packet
from
the
router
and
you
need
to
prod
cast
it,
so
things
would
go
from
them
from
the
main
contributor
and
go
to
all
the
cell
that
we
have
and
because
we
do
have
a
lot
of
plastics
and
like
in
the
water
like
resistance
capacitance.
So
we
would
expect
the
signal
to
be
to
be
affected.
However,
the
great
thing
about
HTM,
because
all
the
things
that
are
digital
digital,
like
that
everything
is
represented
digitally.
So
this
makes
the
impact
of
noise
really
low.
B
You
know
what
will
be
the
architectures
right
now,
there's
so
many
different
parameters,
so
many
different
ways
of
going
about
there's
so
many
different
ways
to
implement.
You
think
this
your
head
can
spend
a
time
please
from
the
outside.
Looking
in
occasionally
your
engineer.
Well,
that's
what
it
looks
like
to
me.
Maybe
it's
all
clear
to
you.
I,
don't
know.
B
And
on
the
algorithm
front,
we're
going
you
know,
work
what
you've
been
planning,
it's
just
the
tip
of
the
iceberg.
Yes,
I
mean
you
know
we're
now
working
on
these
full
cortical
column,
models
that
are
very
sophisticated
compared
to
what
you've
done
already
in
and
and
so
there
needs
to
be
sort
of
an
architectural
paradigm.
They
that
comes
out
of
the
kind
of
work
you're
doing,
which
can
scale
to
the
very
varied
different.
B
The
new
pieces
of
the
algorithms
that
they're
gonna
have
to
be
in
there
I
mean
a
cortical
column
is
a
very
sophisticated
thing.
We
and
following
a
work
and
now
understand
that
they
have
these
reference
frames
and
grid
cells
and
I
mean
all
kinds
of
crazy
stuff.
That's
going
on
in
them,
and
we're
gonna
understand
all
that
not-too-distant
future.
And
so
you
know
so
I
look
at
your
work.
It's
like
okay.
A
B
B
You
know
hundreds
of
thousands
of
those
neurons
in
the
cortical
column
or
a
hundred
thousand
tens
of
thousands,
and
they
have
slightly
different
algorithmic
relationships
to
each
other
to
implement
all
the
things
we
know
that
they
have
to
do
how
much
of
this
kind
of
architectures
can
be
preserved
in
that
future
design,
and
so
you
know,
whenever
I
look
at
any
in
your
morphic
computer
architecture.
I
always
ask
that
questions
like
not
what
is
it
doing
today,
but
I
wasn't
gonna
fit
fifty
years
from
now
and
you
know
I
don't
know.
C
B
C
I
think
keeping
it
modular,
keeping
it
flexible
enough
so
that
it
can
support
these
heterogeneous
units
or
learning
units
is
very
important
and
in
some
is
some
way
we
tried
our
best
to
do
that,
so
whether
it
can
and
as
super
they
rightly
capture,
the
core
components
are
there,
which
can
easily
be
used,
as
you
have
new
modules,
but
I
think
even
the
some
of
this
architectural
framework
as
well,
can
be
preserved
and
have
new
modules
added,
even
at
that
level
of
abstraction.
But
but
I'm
not
saying
it's
going
to
be
a
smooth
sail.
B
B
A
B
And
and
you
implement
them
it,
he
comes
so
we
didn't.
We
only
implemented
like
one
layer
of
cells
in
a
mini
come
and
a
true
mini
common
in
your
cortex
is
about
120,
neurons,
roughly
and
in
beginning
to
a
beginning,
develop
a
theory
about
exactly
what
why
what
the
mini-com
does.
What
is
its
functional
role?
B
But
you
know
I
720
cells
and
I'm
coming
to
believe
that
that's
going
to
be
the
the
computational
unit
of
these
something
does
the
equivalent
of
those
in
the
future
because
we're
starting
to
understand
what
what
they
do
and-
and
so
that's
encouraging,
because
it
says
that
the
thing
we
have
to
build
a
lot
of
isn't
10,000
neurons,
there's.
Actually
something
has
120
neurons.
B
It's
still
damn
complex,
it's
really
complex
with
that
thing
it's
doing,
but
that
that
becomes
your
basic
repetitive
unit,
and
then
you
have
to
figure
out
how
to
put
400
those
together
or
500
thousand
of
those
together
to
get
a
cortical
column
him.
You
know.
Basically,
you
know
you
know.
Hundreds
of
you
know
millions.
A
B
Them
together
to
get
you
get
a
brain
I'm,
just
passing
that
on
to
something
to
think
about
so
I.
There
isn't
there's
a
light
to
the
end
of
the
tunnel.
It's
not
going
to
become
more
complex,
I'm,
always
optimistic
and
I.
Don't
think
we're
I
think
we're
through
it
within
a
few
years
of
actually
really
understanding
this
thing
completely
and
then
you'll
have
a
better
target
to
shoot
for.