►
From YouTube: Detect Safety Zone Violation in Manufacturing with SAS Event Stream Processing and ONNX models
Description
This session will present an in-production solution that takes advantage of SAS Event Stream Processing and ONNX runtime to support the detection of safety zone violations using computer vision pre-trained ONNX Model and involving multiple cameras. This solution was deployed at the factory edge with an architecture that, using Kubernetes and Kafka, ensures a reliable and stable environment for productionized computer vision solutions complemented with a cloud-centralized infrastructure to monitor, manage and collect information from multiple factories
A
A
His
solution
was
achieved
thanks
to
one
of
the
sastool
called
sassy
and
stream
processing.
That's
an
interesting
processing
is
part
of
the
complex
event
processor
family.
So
it's
a
tool
able
to
analyze
in
real
time
a
million
of
even
per
second
and
so
by
using
a
published
subscriber
interface.
The
data
streaming
come
to
the
publisher.
Interface
are
elaborated
by
the
tool,
apply
some
algorithm.
A
This
tool
is
also
engineer
fragility
so
as
a
very
small
footprint
that
allows
him
to
run
the
smallest
devices
like
raspberry
or
Nvidia
Jets,
but
also
scar,
is
here
to
biggest
system
like
a
cloud
or
bigger
Edge
solution
like
in
the
scenario
this
tool
could
be
accessed
that
could
be
developed
using
online
graphical
Studio,
it's
rcsp
Studio,
that's
allow
you
to
draw
the
flow
of
the
data
so
and
apply
all
the
transformation
needed
to
to
fulfill
the
requirement
or
the
other.
You
need
to
run.
A
A
What
we
did
together
with
Microsoft
is
integrate.
Onyx
runtime
inside
domain
theme
processing
allowing
the
customer
to
access
a
the
capability
using
the
standard,
interface
and
and
the
other
facility
from
a
substance
processing
to
integrate
this.
We
allow
to
achieve
from
our
customer
to
get
the
generous
model
on
their
favorite
Frameworks.
That
could
be
also
Sasa.
We
also
have
a
framework
called
visual
data
mining
machine
learning
that
is
partly
knowledge,
but
also
from
machine
learning.
Frameworks.
A
This
only
smaller
than
its
processities
is
a
is
attached
to
an
ESP
Studio
model
that
allow
them
to
transform,
for
example,
to
modify
the
picture
that
is
coming
to
the
flow
in
a
format
or
suitable
for
the
model,
and
then
this
this,
the
model
along
with
the
project,
is
shifted
to
the
USB
server.
That's
thanks
to
a
plugin
with
developer
that
could
use
the
facility
to
the
Onyx
learn
time
and
then
also
leverage
the
execution
provider
to,
for
the
other
acceleration.
A
What
is
what
proposal
we
suggest
to
our
customers?
Since
we
have
this
requirement
of
to
be
a
resilient,
we
suggest
them
to
a
modular
solution
that
split
the
activity
needed
for
processes.
Imagine
in
several
steps
and
each
step
then,
is
using
kafka's
as
a
buffer
you
to
ensure
that
if
something
goes
wrong
in
one
of
the
processing,
the
full
flow,
the
full
pipeline
is
not
armored.
Just
we
could
just
keep
the
data
in
the
Kafka
and
then
the
when
the
one
of
the
process
restored.
We
are
able
to
continue
the
flow.
A
The
first
step,
of
course,
is
acquire.
Imagine
image
from
cameras.
I
will
see
that
we
have
one
step
for
each
camera
to
ensure
that
if
there
is
a
one
problem
not
all
camera
are
affected,
then
we
have
a
model,
that's
process,
the
computer
region
and
level
a
GPU,
the
user
as
cash.
So
we
condense
in
this
model
all
the
requirements
for
computer
visual
processing,
and
then
we
have
another
model
that
responsible
for
processing
two
cities
in
in
the
rear
deployment.
A
As
you
see
the
step,
one
is
the
ingest
dividend
and
we
have
one
for
the
inside
of
Google
and
this
for
each
camera
to
ensure
that
nothing
could
happen.
No,
the
camera
cannot
armor
the
other
camera.
No,
no,
there's
no
possibility
that
we
lost
more
than
the
camera
and
at
the
moment,
are
not
really
working.
A
All
this
in
just
videos
is
placed
on
a
Kafka
basa,
then
process
it
by
by
another
product.
That's
running
new
speed
as
processing
and
ready
made
from
Kafka
and
create
the
result
of
the
computer,
publish
the
result
of
computer
visual
model
this
this
pod.
The
number
three
is
complemented.
This
is
optional,
but
but
it's
usually
important
by
some
additional
sensor
data
from
the
customer
that
allow
us
to
understand
if
the
equipment
that
we
are
recording
in
the
in
the
video
of
the
the
particular
camera
are
active
or
not.
A
This,
of
course,
is
important
because
it's
usual
that
people
will
go
to
the
field
to
repair
machine
where
I
stop
it.
It
is
allowed
it's
not
an
alert,
but
of
course
this
is
completely
different
to
if
the
machine
is
running.
In
this
case,
we
need
to
enforce
the
rule
and
send
an
alert
in
case
that
people
enter
in
the
space.
A
Possible
to
check,
along
with
a
database
of
location
in
each
camera,
if
camera
could
have
multiple
locations
that
are
dangerous
and,
of
course
we
have
multiple
cameras,
so
we
have
a
database
that
register
all
the
all
the
dangerous
location
and
we
read
this
database
and
we
determine
if
the
coordinator
of
the
person
we
detected
on
the
step
three
are
inside
outside
the
desired
side.
We
we
write
out
an
alert
that
is
then
processed
by
the
steps
file
that
is
responsible
to
create
a
save.
A
The
image
to
solution
could
be
sure,
Point
another
report
in
Sydney,
and
then
we
have
the
step.
Six,
that's
sender
to
the
system.
I
learned
along
with
the
part
of
the
image
with
the
violation.
This
process
is
complemented
with
the
some
herbit
solution
to
inject
all
products
running
and
also
process
fault
solution
that
ensure
that
is
something
wrong.
A
For
example,
if
a
camera
doesn't
work,
we
automatically
inform
the
app
system
or
to
mention
that
this
architecture
is
currently
in
production
at
one
of
our
customer
facility,
and
we
are
currently
working
to
expand
this
to
more
than
20
plant
in
the
coming
months.
This
slide
highlight
the
key
benefit
of
the
solution
from
our
customer
perspective.
A
Youngstown
time,
integration
ensures
an
optimal
time
today,
thanks
to
public,
available
pre-trained
models
and
the
reuser
or
customer
asset.
This
permitted
has
to
reduce
overall
project
timing
to
less
than
a
month.
The
architectural
agency
was
another
key
benefit,
and
this
was
ensured
thanks
to
kubernetes
aircraft
performance
scalability,
especially,
are
other
fundamental
aspect
of
this
architecture,
having
centralized
the
usage
of
GPU
to
a
single
pipeline
modules,
Syrian
efficiency
in
other
acceleration,
and
also
enable
scalability
by
allowing
to
increase
the
number
of
GPU
when
needed.
A
Finally,
the
solution
allowed
to
increase
the
number
of
camera
easily
and
provide
support
for
different
kind
of
post-processing
analysis.
Beyond
defense
used
in
this
scenario
as
an
example,
we
could
introduce
a
logic
that
counts
the
number
of
people
in
an
area
avoid
crowds,
or
we
could
verify
that
a
Machinery
is
not
performing
dangerous
actions.