►
From YouTube: HTM Engine Tutorial: Traffic Anomalies
Description
A walkthrough of the HTM Engine, which allows users to create and run hundreds of anomaly detection NuPIC models simultaneously, with an example application of New York City live traffic speeds.
See the code for this tutorial at https://github.com/nupic-community/htmengine-traffic-tutorial.
A
So
this
is
kind
of
a
as
opposed
to
the
hot
gem
tutorial,
where
we
were
just
really
creating
one
model
and
doing
prediction:
we're
going
to
create
over
a
hundred
models
in
this
tutorial
and
I'm,
going
to
show
you
how
to
do
it
using
HTM
engine
and
we're
going
to
run
them
all
at
once
and
get
anomaly
indications
out
of
them.
So
without
further
ado,
first
I'm
going
to
introduce
you
to
the
HTM
engine
architecture.
If
you
look
here,
basically
it
can
htm'
engine
has
a
few
services.
It
must
have
my
sequel
and
RabbitMQ
installed.
A
These
are
used
internally
to
keep
track
of
all
of
the
different
models
that
it
creates
and
queueing
data
that
will
be
input
into
those
models,
but
we
don't
need
to
really
know
about
this.
We
can
take
a
look
if
you
would
like
at
the
Numenta
apps
repo
here.
This
is
where
this
code
base
exists
and
if
you
look
right
here
here
is
the
HTML
gen
repo.
So
that's
what
I
am
talking
about?
That's
what
we're
going
to
be
doing
today.
It's
really
sort
of
a
standalone
tool.
A
It
does
use
it
some
utilities
in
here
which
will
install
as
well,
but
we
will
get
to
that
a
minute.
But
that's
where
the
HTM
engine
architecture
looks
like
there's
a
really
nice
scaffolding
provided
by
my
friend
and
co-worker
Austin
Marshall
called
the
skeleton
HTM
engine
app.
This
is
what
I
use
to
create
this
application
and
it
makes
it
really
easy
to
create
an
app
because
HTM
engine,
all
it
really
needs
to
run,
is
a
bunch
of
configuration.
A
So
what
he
created
was
all
the
configuration
with
everything
you
know
a
lot
of
default
values
and
stuff
in
there
and
which
is
a
configuration
scaffold,
and
so
you
have
to
go
in
and
change
some
of
those
values
to
make
it
work
for
you
in
your
environment.
But
it's
really
easy
to
do,
and
this
skeleton
app
also
contains
this
nice
HTTP
API.
So
I
can
write
my
application
in
whatever
programming
environment
I
want
and
communicate
with
the
HTM
engine
over
HTTP.
A
So
this
is
nice
because
I
don't
have
to
write
my
app
and
python
and
import
all
the
HTM
engine
stuff
I!
Don't
have
to
go
to
my
sequel
or
RabbitMQ.
We
just
let
that
little
HTTP
interface
take
care
of
that.
So
that's
what
we're
gonna
do
in
this
application
and
let
me
give
you
a
quick
overview
of
the
full
architecture
of
what
I'm
going
to
show
you
today
so
over
on
the
left
here
in
the
green
Python
box,
you
can
see
basically
what
I
just
showed
you
HTM
engine
and
the
skeleton
stuff
around
it.
A
The
configuration
and
the
HTTP
API
and
what
I've
created
is
a
client
to
that
HTM
engine.
That's
all
written
in
JavaScript
that
gathers
traffic
information
from
a
service
on
the
internet
and
pushes
a
information
about
New,
York
City
traffic
in
real
time
as
we're
getting
it
from
the
New
York
deity
and
pushes
those
values
into
the
HTM
engine
so
that
it
can
generate
models
around
all
these
different
paths
and
traffic
speeds
in
New,
York,
City
and
you'll
see
exactly
what
I'm
talking
about
when
when
I
get
to
that.
A
But
for
now
let's
go
to
this
repository
when
I'm
talking
about
HTM
traffic
tutorial.
That's
what
I'm
going
to
be
walking
through
today
with
you,
and
there
is
a
readme
here
with
all
of
these
nice
installation
instructions.
So
there's
really
three
parts
to
this.
As
I've
already
explained,
there's
the
Python
server
that
runs
HTM
engine
runs
those
services
and
has
the
HTTP
API,
and
there
is
a
no
js'
client
application,
which
is
the
JavaScript
runtime
and
the
Riverview
data
service.
A
I'm
not
going
to
talk
too
much
about
this,
but
that's
this
red
diamond
here,
that's
where
the
data
is
coming
from.
This
is
just
raw
speed
data
for
a
hundred
and
fifty
three
different
traffic
paths
in
New,
York
City.
So,
like
I,
said,
that's
the
architecture!
If
you
want
to
build
your
own
application,
the
same
way
I
built
this
one,
do
it
off
of
that
scale
in
the
HTTP
application?
It's
a
that's
the
best
place
to
start.
It's
really
easy
to
get
going
there.
Okay,
so
requirements!
A
Let's
get
this
thing
installed
and
and
get
going
the
HTM
engine
we're
going
to
install
that
it
needs
NT,
a
utils
which
will
do
supervisor,
which
is
a
service
controller.
My
sequel
RabbitMQ,
which
is
a
message
queuing
system
and
our
client
is
going
to
need
nodejs.
You
also
need
a
Google
Maps
API
key.
If
you
want
to
do
all,
if
you
want
to
see
all
the
maps
and
stuff
it's
really,
it's
free
easy
to
get,
but
you
have
to
go
to
Google
to
get
it.
A
Ok,
so
starting
it
up.
First
of
all,
we
need
to
start
the
HTM
engine
and
install
it.
So
we're
going
to
move
over
to
that
readme,
which
is
in
the
Python
folder.
It's
got
its
own
readme
install
and
start
required
services,
so
I'm
gonna
need
my
sequel,
RabbitMQ
and
supervisor.
So
I'm
not
going
to
go
through
how
to
install
my
sequel
or
rabbitmq
or
supervisor,
because
it's
different
on
every
system.
You're
gonna
have
to
figure
that
out
yourself,
but
I
am
going
to
start
my
my
sequel
server.
A
A
Ok,
there
we
go
and
I
want
to
show
you
also
in
queue,
admin
that
if
we
list
all
the
queues,
there's
no
thinking
RabbitMQ
right
now,
it's
totally
empty.
So
there
we've
got
rabbitmq,
writing
and
supervisor
I
have
installed,
but
I'm
not
going
to
start
it
right
now,
but
you
will
have
to
install
supervisor
just
go
to
you.
Can
click
this
link
go
to
the
website
and
figure
out
how
to
get
it
running
on
your
system?
It's
just
a
nice
service
manager,
a
process
manager
type
of
software
you'll
see
it
in
action
in
a
minute.
A
Okay
number
two
install
HTM
engine
and
n
ta
utils.
So
this
assumes
well,
you
must
have
Numenta
apps
checked
out
somewhere
locally,
because
we're
going
to
be
running
off
of
that
code
base,
so
I
have
that
installed:
n
ta,
momenta
apps
and
we'll
go
into
HTM
engine
and
run
Python
setup
develop
user,
and
this
will
install
all
of
the
Python
modules
that
HTM
engine
needs
to
run
and
we'll
also
go
down
into
NTA
tills,
which
is
a
sibling
directory
and
ta
utils
and
run
the
exact
same
command.
Python
setup
develop
user.
A
If
you
like,
that's
up
to
you,
okay,
so
we
have
the
Python
modules
necessary
for
HTM
engine
and
the
utilities
to
run
now
installed
required
python
modules
for
each
image
and
traffic
tutorial
itself
on
its
dependencies,
so
we'll
be
going
back
to
where
I
have
HTM
traffic
tutorial
of
that
repository
checked
out,
and
it
says:
I
need
to
run
this
command.
Pip
install
are
and
those
requirements,
so
this
basically
just
installs
all
of
the
Python
modules
that
HTM
engine
traffic
tutorial
needs
and
that's
for
the
Python
server
part
of
it.
A
Okay,
we
also
need
to
export
an
environment
variable.
This
is
for
HTM
engine.
It
needs
to
know
where
the
application
configuration
is,
in
our
case,
it's
in
Python
engine
comp
and
we'll
be
talking
about
changing
some
of
the
values
and
the
files
in
a
moment,
but
for
right
now
we're
just
going
to
export
this
application
config
path
there
we
go
to
point
to
the
conf
directory
and
python
engine.
A
Alright,
so
next
step
create
a
my
sequel
database,
so
we're
gonna
create
one
called
traffic
should
be
able
to
just
do
this
from
the
command
line.
Now
we
should
have
five
sequel
database
I'll,
just
trust
that
it's
there
didn't
get.
Any
errors
apply
database
migrations.
Okay,
so
we
have
an
empty
database,
but
it
doesn't
have
a
schema
yet
so
running.
This
command
will
create
the
appropriate
schema
that
this
that
HTM
engine
needs
to
store
all
the
data
it
needs
to
store.
A
So,
let's
take
a
look
now
route
and
we
should
use
traffic
that
database
should
be
there
show
tables,
and
we
have
a
schema
I'm
not
going
to
go
over
all
this,
but
you
can
look
into
it.
If
you
want
to.
This
is
what
HTM
engine
uses
to
run,
but
we
don't
need
to
look
at
it
right
now.
We
have
applied
the
database
migrations
and
now
there
are
two
configuration
files
and
that
cough
directory
I
was
telling
you
about
that
need
to
be
changed.
A
Ok,
so
I
will
open
up
an
editor
here
and
we
will
go
into
those
files.
The
first
one
it
says,
is
in
Python
engine
comp
supervisor,
Python
engine
comp
supervisor
and
look
for
any
where
it
says:
users,
M,
Taylor,
NT
a
because
that's
me
look
for
that
and
there's
two
results:
there's
one
for
the
environment
and
that
points
to
a
cough
directory.
A
You
need
to
replace
this
with
where,
whatever
path
you
have
to
your
check
out
to
HTM
engine
traffic
tutorial
in
this
location
and
also
at
the
bottom
and
this
location
under
include
files,
so
update
those
two
values
in
supervisor
decomp
and
also
in
the
model
checkpoint
comm,
which
is
a
sibling
file.
There's
storage
root.
This
is
where
HTM
engine
saves
model
checkpoint
so
where
its
serializes
models
to
disk,
you
can
put
this
wherever
you
want.
A
Just
put
it
point
it
to
a
location
on
your
filesystem
and
if
it
doesn't
exist,
I
believe
it
will
go
ahead
and
create
it
so
update
that
those
are
the
only
two
places
in
your
config
files
that
you'll
need
to
change.
Far
as
I
know,
and
now
we
can
start
our
services
with
supervisor.
When
we
run
this,
this
is
going
to
start
those
services
that
we
saw
up
here,
the
anomaly
service,
the
metric
listener,
new
metric
store
and
the
model
scheduler.
So
let's
do
that.
A
We
want
to
be
in
Python
engine
make
logs
directory
case
that
we
don't
already
have
one
so
python
engine
there's
already
a
logs
directory
and
we're
gonna,
say:
supervisor
D
C
for
config
and
here's
the
path
to
the
config
file
that
you
should
have
just
updated
with
the
right
paths.
This
will
start
supervisor.
Now
you
can
look
a
supervisor
CTL
status
will
show
you
that
those
four
services
we
just
talked
about
are
running.
A
You
may
also
go
to
localhost
9001
to
see
the
web
view
of
these
services,
but
that's
unnecessary,
but
it's
an
easy
way
to
look
at
the
logs.
If
you
feel
like
looking
at
some
logs,
so
supervisor
is
running.
Htm
engine
is
running
right
now,
interesting
thing.
If
we
look
at
our
rabbit
queues,
we
have
a
few
queues
in
there.
So
when
HTM
engine
started
it
services,
it
created
some
queues
in
rabbit,
so
we
could
start
communicating
with
itself.
A
Okay,
so
we
have
services
started.
We've
checked
the
status,
everything
good
everything's
good.
We
can
stop
all
those
services
by
running
this
command.
Let's
go
back
to
the
readme
and
continue
from
where
we
left
off
now.
We've.
We
need
to
start
the
HTTP
server
all
right.
We
said
there
was
a
this
little
shim
HTTP
interface
that
needs
to
be
started
all
on
its
own,
so
we've
art
we've
taken
care
of
the
scaffold.
We've
started
all
the
HT
mention
services.
A
Let's
start
this
little
server,
it's
very
easy
to
start
we're
in
Python
engine,
so
we
just
say
fights
on
web
app
and
there
we
go
so
this
little
server
is
going
to
run
for
the
duration
of
the
demo
here
so
I'm
just
going
to
move
this
guy
over
put
them
over
to
the
side
here
and
we'll
start
seeing
at
doing
things
soon.
But
right
now,
all
it
is,
is
a
stateless
HTTP
server.
That's
just
passing
commands
back
and
from
from
my
client
application,
it's
not
running
yet
into
HTML.
A
Okay,
so
we've
got
that
started
that
runs
on
localhost
8080.
Now
we
need
to
go
to
the
JavaScript
side
and
start
the
HTM
client
so
that
stuff
back
to
the
traffic
tutorial
directory
is
in
this
node
client
directory,
so
go
into
node
client.
This
is
where
our
node
app
is
and
to
install
its
dependencies.
We
run
npm
install
dot,
which
tells
it
install
this
directory,
and
this
will
install
all
of
the
node
packages
that
need
to
be
there
for
it
to
run.
A
Similarly,
what
we
did
when
we
did
a
pip
install
requirements
in
the
Python
world,
okay,
so
all
that
stuff
is
now
installed.
We
should
be
able
to
say
npm
start
at
this
point
and
watch
stuff
happen.
Let's
do
it
NPM
start
and
there
we
go
so
what's
going
on
here,
you
can
see
our
HTTP
shim
is
plugging
away.
It's
getting
a
bunch
of
requests
right
now
from
the
client
application.
A
I
am
also
going
to
show
you
my
CPU
usage,
because
it's
gonna
spike
like
crazy
here
when,
as
this
thing
starts
up
there,
we
go
just
don't
we'll
just
put
this
over
here,
so
we
can
see
what's
going
on
with
our
CPU.
So
let
me
come
back
to
this
diagram
and
try
and
give
you
an
indication
of
what's
actually
going
on
here.
A
A
It
gets
all
of
the
different
IDs
for
those
traffic
paths
and
it
pushes
the
model
IDs
for
those
routes
into
through
the
HTTP
API
to
the
HTM
engine
and
says
create
a
model,
so
it
creates
one
model
for
every
single
one
of
these
traffic
routes
that
it
gets
from
the
data
service
should
probably
show
you
what
this
data
service
actually
looks
like
here.
So
let's
do
that.
The
data
is
coming
from
a
service,
we're
running
called
Riverview,
there's
NYC
traffic
here,
and
this
contains
really
into
at
least
10
minutes
intervals.
A
Data
for
all
these
different
traffic
paths.
So
each
one
of
these
rows
here
is
a
different
traffic
path
that
the
New
York
City
d-o-t
makes
available,
for
example,
here's
traffic
route
444.
If
we
look
at
the
metadata,
we
will
see
that
this
is
in
Manhattan.
It's
from
West
Street
South
battery
place
to
FDR
north
catherine's
lip
whatever
that
means
and
that
we
actually
have
coordinates
for
it
and
we'll
see
these
routes
plotted
on
the
map
soon
enough,
but
I
just
want
to
show
you
where
the
data
is
coming
from.
A
We
look
at
the
data
you
can
see.
We've
got
two
values
here:
speed
and
travel
time.
Our
app
is
only
going
to
use
speed
and
I
assume
that
over
the
entire
path
of
this
traffic
route,
this
is
the
average
speed
from
whatever
sensors
the
d-o-t
has
installed
there
for
the
past
10
minutes
or
so
so
that's
what
we're
going
to
use.
So
this
comes
in
a
HTML
form.
If
you
want
to
view
it,
but
you
can
also,
and
what
the
client
application
is
doing
is
getting
it
in
Jason
form.
A
A
Okay,
so
that's
the
traffic
service
that
we're
getting
our
data
from
back
to
here
and,
like
I,
said
for
each
one
of
those
153
routes,
a
model
is
created
and
then
it
goes
and
gets
all
the
traffic
history
speed
for
that
traffic
for
that
route
and
pushes
it
for
for
each
model
in
serial
every
different
value.
So
right
now
why
this?
A
Why
are
CPUs
are
totally
pegged
now
is
because
HTM
engine
has
153
new
pic
models
running
and
a
bunch
of
data
queued
for
each
one,
and
it's
pushing
a
piece
of
data
into
each
model
over
and
over
and
over
and
over,
and
it's
struggling
to
catch
up
with
the
amount
of
data
I'm
pushing
to
it.
So
if
we
look
at
the
output
for
a
client
application-
or
it's
saying
okay,
so
for
this
path,
I
am
posted.
A
I,
just
posted
data
to
HTM
engine
I've
got
139
more
models
to
go
130
or
127
more
miles
to
go,
so
it's
keeping
track
of
where
it
is
in
the
queue
and,
as
you
can
see,
there's
a
lot
of
data
points
being
pushed
to
this.
Guy's
got
almost
3,000.
This
guy's
got
200
or
755
etcetera
and
we're
just
gonna
have
to
sit
and
wait
while
this
catches
up-
and
you
can
see
all
the
data
posts
that
are
happening
over
here.
A
A
The
engine
is
still
got
a
whole
bunch
of
data
in
its
queue
and
its
queues
for
its
models
that
it's
got
to
still
have
to
process
so
that
it's
going
to
take
a
while
I
think
last
time.
I
did
it
took
me
about
half
an
hour,
but
the
nice
thing
is,
you
can't
kill
this
so
here
we
go.
I
just
killed
the
client
application.
You
can
see
from
my
CPU
usage
I'm
still
pegged.
My
CPS
are
still
running
because
HTM
engine
is
still
running.
A
The
HTTP
API
is
still
running
I
just
turned
my
client
off
I
basically
stopped
pushing
data
and
you
can
just
restart
it
and
what
it
will
do
is
go
to
the
HTM
engine.
Ask
how
much
was
the
last
data
point?
You
got
for
all
these
different
models
and
start
polling
or
start
pulling
more
data
from
the
traffic
service
from
that
time
stamp.
A
So
you
can
start
and
stop
this,
even
if
it's
not
through
posting
data
to
the
HTM
engine,
and
it
will
just
proceed
where
it
left
off,
and
here
we
go
we're
already
back
to
119
more
models
to
go,
and
at
this
point
it's
probably
a
good
place
to
mention
that
if
you
don't
want
to
sit
here
and
wait
for
half
an
hour
or
so
depending
on
your
system,
I've
got
8
cores.
You
might
have
4
it's
going
to
take
twice
as
long.
A
If
you
don't
want
to
sit
here
and
wait,
there's
a
way
you
can
speed
this
up.
So
let
me
show
you
real
quick:
how
to
speed
this
up
and
close
our
config
files
here
in
the
node
client
there's
a
conf
directory
and
there's
a
config
module
down
at
the
bottom.
It
says:
path,
whitelist.
Currently
it's
undefined.
That
means
it's
going
to
load
every
single
traffic
path,
but
you
can
specify
individuals,
so
if
I
just
put
4
for
to
that,
for
example,
this
is
a
data
path
for
42.
A
It's
only
going
to
load
this
one
traffic
path
and
not
all
153
of
them
that
are
that
that
are
available,
so
you
can
kind
of
handpick
in
here
which
ones
you
want
to
load
and
just
tell
it
specifically,
what
is
the
whitelist
I
could
put
442,
134
or
whatever
and
then
it'll
take
a
whole
lot
less
time
for
it
to
stand
up,
because
it's
only
got
one
path
or
a
handful
of
passes
that
153.
So
that's
just
in
case.
A
You
want
it
to
run
faster,
I'm,
going
to
sit
here
and
let
it
run
and
run
and
run
so
that
we
can
like
get
the
whole
data
experience.
So
I'm
going
to
pause
the
recording
right
now
and
I
will
be
back
as
soon
as
HTM
engine
is
caught
up.
All
the
data
has
been
posted
and
we'll
take
a
look
at
what
kind
of
traffic
anomalies
we
can
find.
A
A
What
was
the
last
time
you
saw
data
for
each
one
of
these
paths,
take
that
timestamp
and
go
to
the
data
service
and
ask
for
any
new
points
of
data
that
occurred
after
those
timestamps
and
push
those
new
points
into
HTM
edge
and
so
at
10-minute
intervals,
you're
going
to
see
your
CPU
usage
spiked
yet
again,
but
for
a
limited
period
of
time.
Well
it
just
processes
whatever
occurred
in
the
past
ten
minutes.
A
So
but
now
we
can
go
to
the
web
app
with
all
of
the
stuff
going
on
on
this
console.
You
might
have
missed
that
web
application
started
at
some
point
here,
while
it
was
creating
models
and
all
that
jazz
here
it
is
so
it's
a
it's
running
on
localhost
8080
three
and
here
is
the
HTM
edge
in
traffic
tutorial
there.
We
go
I.
Think
this
a
little
bit
easier,
see
here.
A
So
this
is
the
app
now
it's
got
all
of
the
data
within
it.
So
let
me
give
you
a
little
bit
of
walkthrough.
What's
what's
going
on
here,
let's
first
look
at
the
route
index.
Well,
let's
look
at.
Let's
look
at
the
map
first,
so
you
can
see
all
these
routes
mapped.
Here
are
the
routes
that
we
pulled
data,
for
they
all
have
these
little
icons
by
them.
That
show
you.
This
is
153
where
it's
at
and
then
links
to
some
charts
and
stuff.
A
But
let
me
take
the
route
markers
off.
You
can
see
I've
just
colored
them
randomly,
so
you
can
sort
of
see
where
they're
at
and
then
the
route
markers
are
the
first
point
in
the
route.
If
we
want
to
see
it
so
that
there's
where
all
the
routes
are,
and
you
can
see,
there's
lots
of
gaps.
This
is
I
mean
there's
so
much
traffic
in
this
area.
A
I
wish
I
could
have
gotten
a
lot
more
data,
but
we're
gonna,
try
and
dig
in
and
see
if
we
can
associate
some
traffic
anomalies
with
some
actual
traffic
accidents
and
we're
gonna
find
out
that
a
lot
of
accidents
occurred
so
far
away
from
any
of
these
routes.
There's
nothing
we
can
do
about
them.
Okay,
so
that's
that's!
That's
the
map
of
all
the
routes
that
we
have,
let's
also
look
at
the
route
index.
A
So
this
is
a
view
of
every
one
of
these
routes,
and
you
can
you
know
it
identifies
what
borough
I
think
you
can
sort
yeah,
so
you
can
sort
it
up
here
if
you
want
to
sort
on
address
or
ID
or
borough,
etc,
I'm
gonna,
sort
on
ID,
and
then
each
one
of
these
routes
has
a
chart.
I'm
gonna
go
to
442
because
I
know
that's
an
interesting
route
and
it
has
a
lot
of
data,
it's
at
the
very
tip
of
Manhattan
southern
tip.
So
let's
look
at
these
routes.
A
A
So
that's
just
an
idea
of
one
route.
When
you're
looking
at
one
traffic
route,
you've
got
some
options
over
here.
You
can
do
it
on
the
map.
You
can
view
the
daily
charts,
which
is
interesting.
So
if
we
go
to
daily
charts,
this
will
map
out
24-hour
periods.
So
the
first
one
is
a
24-hour
period,
starting
now
going
backwards,
24
hours
and
then
each
consecutive
chart,
underneath
it
is
the
previous
24-hour
period,
and
you
can
see
there's
big
gaps
in
data.
A
Okay,
so
let's
go
back
home.
I
also
want
to
show
you
you
can
you
can
look
at
the
boroughs
directly?
If
you
want
to
see
a
map
of
just
the
Manhattan
routes
there
they
are.
If
you
want
to
see
charts
of
just
the
Manhattan
charts
here,
they
are,
and
we
can
actually
once
these
charts
load,
we
can
actually
grab
one
chart
and
it
will
Reese
resize,
all
the
other
charts
rescale
them
to
be
that
time
period.
So
this
is
from
midnight
Sunday
to
right
now
and
then
I
can
go
through
and
click
this
hide
button.
A
This
is
not
working
there.
It
goes.
I
click
the
hide
button
to
hide
the
routes
that
don't
have
any
interesting
things
going
on.
This
is
this
is
typically
what
I
do
to
try
and
do
some
like
analysis
on
what
traffic
anomalies
are
happening
in
what
areas
so
I'm
just
gonna
I'm
just
going
through
and
hiding
all
these
and
it
added
a
hide
hide
hide.
A
This
is
this
could
be
made
better
because
it
keeps
popping
up
to
the
top,
but
I'm
not
too
worried
about
it
anyway.
There's
a
lot
of
routes
in
Manhattan
and,
as
you
can
scan
through
time
periods
and
see
okay,
ten
o'clock,
something
crazy
happened
on
this
route
route,
one
the
same
time
a
similar
time.
Something
interesting
was
happening
on
this
route
route
120.
A
A
Let's
close
this
so
I
can
show
you
this
full
screen.
Okay,
so
let's
say
I
want
to
get
all
traffic
anomalies
for
like
from
rush
hour,
it
said
starts,
maybe
422
up
until
now,
New
York
time,
and
we
want
any
anomalies
that
are
over
0.99
click.
This
button
wait
a
few
seconds
and
we'll
get
all
of
the
routes
through
in
all
of
the
boroughs
that
had
data
points
over
that
threshold
of
0.99
and
so
there's
two
options
here:
click
here
to
plot
them.
A
This
will
bring
up
a
new
screen
where
you
can
see
during
that
time
period.
What
each
of
these
routes
look
like
what
their
daily
scores
look
like.
So
this
is
a
good
way
to
try
and
identify
anomalies.
For
example,
it
looks
like
something
is
occurring
around
10
o'clock
here.
So
let's
hide
in
all
of
these
different
guys
that
didn't
have
like
a
bump
up
at
that
time
period.
Hide
you
hide!
You
hide
you
it's
a
little
bit
of
a
bobble
leave
him
hide!
You
hide!
You
know
now.
A
A
So
this
is
another
way
you
might
be
able
to
identify
something.
So
something
strange
is
happening
between
10:00
and
11:00
on
several
of
these
routes,
so
this
is
a
big
big.
This
is
why
this
anomaly
scores
high
cuz.
It
jumped
from
54
miles
per
hour
down
to
one
and
then
back
up
to
55
ten
minutes
later.
That
was
probably
just
a
bad
data,
but
definitely
anomalous
and
in
the
others.
So
maybe
you
don't
know
like
Route
122
weather.
This
is
like.
A
Maybe
we
want
to
see
the
history
of
this
so
I'm
gonna
click
on
the
daily
routes
for
Route
122,
and
then
we
can
see
so
I.
Don't
really
trust
the
this
anomaly,
because
we
have
seen
a
whole
lot
of
data
right.
There's
a
the
data
points
in
the
past
are
really
few
and
far
between,
and
it
just
got
enough.
So
the
anomaly
likelihood
started
giving
decent
values
here
so
I'm
not
really
going
to
trust
that
one.
A
Let's
look
at
another
one
3:15
in
Queens
this
one
similar,
it's
only
been
getting
anomaly
likelihoods
for
the
past
day.
Having
a
lot
of
data
is
definitely
good.
I've
got
four
weeks
of
data
and
for
some
routes
that's
just
excellent,
like,
for
example,
let's
look
at
the
route
442
I
always
go
to
that
one,
because
it's
at
the
tip
of
Manhattan
and
there's
a
ton
of
data
for
it.
So,
as
you
can
see,
I've
got
lots
and
lots
of
data
days
and
days
and
days
of
decent
data
for
this
route.
A
So
I
can
trust
the
anomaly
score
a
lot
more
in
this
case,
because
it
seems
so
many
days
of
traffic.
So
I
know
that,
like
right
here
when
we,
when
we
jumped
right
here
when
we
jumped
from
3
miles
per
hour
up
to
30
miles
per
hour,
and
that
was
consistent
over
you
know
an
hour.
Almost
that's
something!
That's
unnatural
for
that
time
period
and-
and
you
can
see
the
anomalous
work
fell
off
because
of
it.
A
Ok,
so
I'm
just
trying
to
show
you
like
ways
you
can
investigate
anomalies
here,
so
this
is
one
way
eat
and
there
is
yet
another
way
you
can
do
this,
which
is
to
browse
the
map
over
time.
This
takes
a
while
to
load,
but
what
it'll
do
is
it
will
plot
all
the
maps
and
get
all
of
the
data
for
all
the
anomaly
scores
for
the
routes
in
the
browser
and
give
you
a
little
time
slider,
so
you
can
move
it
back
and
forth,
and
the
colors
of
the
routes
will
change
over
time.
A
So
let
me
turn
the
route
markers
off
here
and
zoom
in
so
we
can
see
some
of
these
colors
change
if
I'm
looking
for
something
in
Manhattan
or
Queens
or
whatever.
Let's
see
so,
here's
my
slider
up
here
right
now.
It
is
three
hour
period
from
eleven
twenty
six
to
two
forty
six
in
the
afternoon,
I'm
going
to
just
scoot
it
over,
and
we
can
see
some
of
those
routes
changing
like
there's.
We
got
a
red
route
over
here,
we're
gonna
red
route
down
here.
A
That
means
that
the
during
that
three
hour
time
period,
those
routes
that
are
changing
color
had
high,
not
only
scores.
So
this
is
another
way.
There's
there's
some
more
down
around
the
tip
of
Manhattan
some
time
right
before
6:00
a.m.
something's
going
on
there.
The
thing
is:
there's
so
much
construction
around
Manhattan.
A
What
you
never
know
what's
going
on
in
some
of
these
routes.
So
what
I
like
to
try
and
do
is
is
correlate
actual
traffic
incidents.
So,
let's
say
I'm
gonna,
look
up,
New
York,
City
traffic
accidents
and
one
of
the
let's
see
NBC
in
New
York
has
like
a
live
traffic
map.
I'm
gonna,
look
at.
Let
me
turn
the
flows
off
and
just
turn.
A
Incidents
on
and
you're
gonna
see
a
ton
of
incidents
here
and,
let's
see
if
we
can
find
a
traffic
accident
that
is
close
to
somewhere,
we
have
data
that
I
mean
that's
the
important
thing.
The
track
up
traffic
accident
occurs
all
the
time,
but
you
know
I,
don't
have
any
routes
over
here,
but
here's
one
in
the
middle
of
Manhattan,
8th,
Avenue
and
42nd
Street.
So,
let's
take
a
look
at
8th,
Avenue
and
42nd
Street
72nd,
42nd
and
8th
Avenue
is
right
around
it
here.
A
So
we
probably
not
gonna
affect
any
of
routes
going
through
the
Lincoln
Tunnel.
This
is
happening
in
on
the
streets
of
Manhattan,
not
on
any
of
these
streets
that
we
have
traffic
incidents
for,
but
I
do
want
to
show
you
something
visit.
I
mean
it's
hard
to
find,
live,
live
traffic
accidents
and
correlate
them,
but
I
was
able
to
earlier
through
this
website
and
I
took
a
screenshot
of
it.
Let
me
pop
this
up
real
quick,
so
here's
what
I
found
earlier
this
happened
at
9
11
a.m.
A
this
morning
looks
like
there's
lane
block
due
to
a
stalled
truck
on
I
to
78
eastbound
at
2nd
Avenue.
So
that
was
let
me
go
back
to
my
view
here.
We
want
to.
This
was
at
nine,
what
did
I
say:
nine,
eleven
right,
yeah,
nine,
eleven
a.m.-
and
where
is
this?
This
is
down
here
right
yeah.
How
is
it
right
here?
A
Mm-Hmm
I
look
at
the
yeah
right
here,
which
is
right
around
in
here:
okay,
yeah
I,
78
eastbound
exit
23,
which
is
2nd
Avenue,
and
that
2nd
Avenue
exit
23
is
right
around
in
here
somewhere.
So
there
is
some
incident
that
was
slowing
down
traffic
here
at
nine.
Something
and
I
want
to
see
if
there's
any
correlation
with
the
right,
there's
a
there's,
a
speed
up,
there's
an
right
there.
This
is
eastbound.
So
what
is
this
guy
sure
this
route
marker?
A
We
don't
have
a
whole
lot
of
data
for
him,
but,
looking
at
today,
so
here's
midnight,
2,
4
6
8
9,
look
at
that
right
around
9/11.
So
this
is
where
you
can
definitely
see
the
traffic
speed
drop
from
47
miles
per
hour,
pretty
quickly
down
to
around
5
miles
per
hour,
and
that
was
enough
to
cause
an
anomaly.
You
go
up
through
that
full
time
period,
so
that
definitely
looks
like
a
correlation.
I
mean
if
this
is,
if,
if
eastbound
to
78,
got
blocked
around
here,
it
would
certainly
potentially
affect
this.
A
This
traffic
route,
the
traffic
flowing
eastbound
down
to
78
here
so
I
you
can
catch
incidents
like
this
pretty
easily.
There
there's
another
website
that
I
go
to
to
to
check
these
things.
Well,
there's
a
Twitter
feed
called
5-1-1
NYC
I
can
I
can
do
sometimes
do
some
correlations
on
these
incidents,
but
it's
interesting
to
to
do
to
find
this
out,
because
I'm
I
mean
a
lot
of
times.
A
If
you're
sitting
here
trying
to
trying
to
look
through
rush-hour
and
figure
out,
what's
going
on
here
and
you'll
see
you
know,
a
bunch
of
routes
will
light
up
in
one
area
and
if
you
dig
into
them
and
look
at
their
data,
it's
obvious
that
there's
like
major
traffic
slowdowns
and
several
of
those
routes.
So
something
is
occurring
in
that
geographic
area.
That's
affecting
the
traffic
speed
in
those
routes
and
nupoc
is
saying:
there's
anomalous
activity
in
these
routes,
but
I'm
not
able
to
associate
it
with
any
traffic
incidents
that
are
coming
from.
A
You
know
the
news
or
the
Twitter
feed
or
there's
email
alerts
that
I
get
from
NYC
VOT,
but
something
is
definitely
going
on.
I
mean
you
can
manually,
see
it
and
new
picked
certainly
sees
it.
So
at
this
point,
what
I
would
really
like
is
more
data
if
you're,
following
along
with
this
tutorial-
and
you
find
some
interesting
traffic
anomalies
that
you
can
associate
with
actual
traffic
incidents
that
occurred
in
the
New
York
City
area.
I
would
love
to
see
those
as
well
so
email.
A
The
mailing
list
take
a
couple
screenshots
and-
and
let
me
know
what
you're
finding
I
think
this
is
a
really
cool,
interesting
application
of
HTM
and
anomaly
detection
and
it
could
be
applied
to
a
lot
of
different
things.
I
mean
any
type
of
scalar
values
changing
over
time,
whether
it's
traffic
speed
energy
consumption.
A
You
know
water
levels,
weather
conditions,
there's
a
right
for
us
to
start
generating
anomaly
scores
for
so
before.
I
sign
off
here,
I
feel
like
I,
should
at
least
introduce
you
to
some
of
the
code,
especially
in
the
JavaScript
client
that
interfaces
with
HTM
engine.
So
let
me
show
you
that
real
quick
here
endowed
client
under
the
Lib
directory
I've
got
all
of
the
code
that
interfaces
with
HTM
engine
in
this
HTM
engine
client
class
and
all
it
really
is-
is
an
HTTP
interface,
because
that's
that's
what
the
Python
engine
gives
us.
So
it
has.
A
You
could
easily
like
pick
up
class
and
reuse
it
for
whatever
you
wish.
It
has
functions
for
posting
data
to
a
model,
creating
a
model
given
a
min
and
Max
value
and
a
unique
identifier
and
also
getting
data
from
a
model.
So
this
could
be
easily
reused.
The
last
one
is
gets
last
updated.
So
this
will
give
you
the
timestamp
of
the
the
last
updated
value
that
HTM
engine
has
seen
for
that
model,
which
is
really
useful
when
you're
deciding
what
future
data
to
send
it.
A
So
this
is
totally
reusable
as
a
client
for
HTM
engine,
the
other
classes-
and
here
most
of
this,
has
to
do
with
the
website
itself.
The
interesting
thing
is
probably
the
traffic
pusher.
It's
that
this
is
the
logic
that
decides
what
data
to
query
from
the
data
service
and
push
to
HTM
engine,
and
it's
just
handed
in
HTM
engine
client
object
and
a
data
client
object,
which
is
here
this
traffic
data
client.
It
is
the
client
for
the
River
View
data
service
and
it
knows
how
to
get
traffic
data
from
that
service.
A
So
the
traffic
butcher
just
gets
two
instances
of
those
those
objects
and
depends
on
them,
knowing
how
to
fetch
their
own
data.
The
the
primary
logic
and
this
traffic
pusher
is
in
a
fetch
method.
If
you
want
to
get
into
exactly
the
details
of
it,
but
if
you
want
to
reuse
any
of
this
feel
free.
The
HTM
engine
client
is
probably
the
most
useful
because
that's
where
you'll
interface
with
the
HTM
engine
but
yeah,
just
a
quick
overview,
I
won't
get
into
all
the
client
side
stuff.
A
A
Prs
are
welcome
for
this
code
base.
If
you
find
something,
that's
wrong
or
it
can
be
improved,
feel
free
to
create
a
PR
I'm,
happy
to
review
it
and
accept
it
I'm
going
to
keep
this
tutorial
out
there
and
our
new
community
codebase,
so
anybody
can
play
around
with
it
so
so
right,
I
I,
hope
you
have
enjoyed
this.
This
HTM
Engine
tutorial
emphasis
on
how
to
do
traffic
anomaly
investigations
with
live,
New,
York,
city
traffic,
best
wishes
and
I
will
talk
to
you
guys
all
on
the
mailing
lists.
Thank
you.