►
From YouTube: PHINUG Online Dev Sessions 2021.02
Description
We'd love to see you at our first PHINUG Online Dev Session for 2021 where we'll be talking about seeing things with .NET!
This session is run by the Philippine .NET Users Group user group (https://meetup.com/phinug) that hosts meetups in Manila, Philippines.
Meetup link: https://www.meetup.com/phinug/events/275856986/
==============
At this meetup our talk would be:
Intro to Computer Vision in .NET
---
Computer Vision is revolutionizing the world around us. It's ubiquitous, yet the entry points are opaque and can be a bit daunting to understand. This talk will break down these barriers as we cover some of the basics of Computer Vision in .NET - edge detection, facial detection, feature tracking, and some basic image projection using OpenCV in .NET.
==============
About our speaker:
Steve Lorello
---
Steve Lorello got his start building .NET software while working on the Global Positioning System and has been obsessed with .NET development ever since. Today, Steve works as the .NET Developer Advocate for Vonage by day and studies artificial intelligence and computer perception as a grad student at Georgia Tech by night.
A
Oh,
let
me
mute
myself
over
here,
daughter,
all
right
yeah.
So
let
me
introduce
myself,
I'm
john
lim
happ,
I'm
a
lead.
I
lead
the
philippine.net
users
group
and
the
community
lead
and
I'm
also
a
dotnet
developer.
I
work
for
a
company
called
pageup
as
a
principal
developer,
yeah
making
me
make
a
software
products
with
with
net,
and
tonight
we
have
a
very
special
guest.
He
has
from
florida
and
he's
actually
a
dot
net
developer
advocate
at
a
company
called
vonage.
A
So
without
further
ado,
I
am
giving
you
over
to
our
speaker
tonight,
steve
lorello,
he's
again
he's
from
he
is
from
florida,
it's
very
early
there.
So
so
we
thank
him
for
for
getting
up
here
and
he's
going
to
talk
about
an
intro
to
computer
vision
in
dot
net,
and,
along
with
that,
you
know
interesting
stuff,
like
machine
learning
and
and
so
on.
So
I'll
not
make
this
any
longer
and
I'll
give
the
stage
over
to
steve.
Thank
you
for
being
here.
B
B
C
B
B
B
So
what
is
a
digital
image?
An
image
when
you
break
it
down
is
really
just
a
function
that
function
denotes
a
quantized
sample
of
intensity
values
along
a
two
dimensional
grid,
those
intensity
values
otherwise
known
as
pixels
denote
light
intensity
along
an
arbitrary
range,
for
example,
a
byte
zero
to
255..
B
B
When
we're
looking
at
grayscale,
those
numbers
will
actually
be
on
the
range
of
negative
one
to
one
or
zero
to
one,
but
technically
these
values
as
long
as
they're
along
the
bounded
range,
the
math
will
still
hold,
and
actually,
when
you
start
getting
into
other
applications
of
image
processing,
for
example,
hdr,
the
ranges
of
values
are
exponentially,
larger
and
technically
infinite,
so
that
image
of
zero
that
we
saw
before
is
the
size
of
size
n
by
m
by
three.
So
n
is
the
height
of
this
image.
B
It's
the
number
of
pixels
that
comprise
the
height
m,
is
the
width
and
then
there's
three
channels.
Those
three
channels
are
just
a
light
intensity
for
a
particular
color.
In
that
instance,
blue,
green
and
red,
so
this
picture
here
is
a
picture
of
a
pepper
and
it's
actually
a
classic
example
for
looking
at
an
image
as
a
function
in
a
one-dimensional
space.
B
B
B
Now
that
insight,
where,
like
the
lighter
spots,
are
higher
intensities
and
the
darker
spots
or
lower
intensities,
actually
is
very
powerful
and
we'll
take
a
look
a
little
bit
later
in
this
talk
about
how
that's
applied.
But
I
just
want
you
to
think
briefly
about
where
the
edges
in
this
images
are:
where
are
they?
Where
is
the
color
changing
this
image
is
in
the
peaks
of
this
little
graph
on
the
right
here?
Is
it
in
the
valleys,
or
is
it
somewhere
in
between
them?
B
B
For
example,
there's
a
lot
of
they
have
a
lot
of
powerful
image
processing
functionality,
but
I
like
opencv
as
for
the
most
part,
it's
free
to
use
and
the
docs
are
pretty
easy
user
friendly
in
addition
to
explaining
how
to
do
something
they
as
a
bonus,
sort
of
explain
the
math
and
insights
that
behind
what
everything
does
mgu
cv,
is
a
net
opencv
wrapper
that
we'll
be
looking
at
here.
It's
widely
available
in
the
dot
net
framework.
It
does
enable
you
quite
a
lot.
B
B
So
now,
let's
look
at
just
a
little
bit
of
code
here,
mgu
cv
is
structured
around
a
bunch
of
static
functions
in
a
class
called
cv.
Invoke
here,
we're
using
three
of
them.
Now
I've
stored
an
image
in
the
resource
directory
of
this
project
called
xero.jpg,
and
that
file
is
just
copied
down
to
the
build
directory.
When
I
run
it,
so
I'm
just
assuming
it's
going
to
be
there
now.
B
This
first
line
here
actually
reads
the
image
of
zero
into
a
matrix,
so
this
var
here
is
becoming
a
matrix
as
you're
doing
this
there's
two
things
to
note.
First,
the
matrices
that
we're
dealing
with
are
by
default
read
into
the
bgr
space.
That's
blue,
green
red,
rather
than
the
rgb
space
that
you
might
be
used
to
it's
the
same
thing.
It's
just
the
channels
are
flipped.
B
The
second
line
creates
a
window
with
a
title
provided
by
the
first
argument
here
and
displays
the
image
from
the
second
argument
here,
and
so
you
call
cv
invoke
dot.
We
key
this
final
line
here,
that's
similar
to
waiting
for
a
console
input.
It
just
waits
for
you
to
press
the
key,
while
you're
focused
on
the
image
window.
Without
that
call,
the
image
will
just
disappear
and
you
won't
get
to
see
it
so
I'll
just
show
you
that
real
quick.
B
B
B
B
So
you
run
this
filter
or
kernel
over
the
whole
image,
and
you
produce
a
feature
image
now.
Typically,
like
51,
wouldn't
be
the
output
here,
you
would
actually
sum
up
this
whole
kernel
and
then
divide
it
from
the
feature
image.
So
that's
produced
four,
but
this
basic
operation
convolution
becomes
very
important
when
you're
dealing
with
image
processing.
B
Now
I'd
like
to
actually
illustrate
why
convolution
is
such
a
powerful
tool
when
we're
thinking
about
computer
vision,
we
could
do
a
really.
We
could
do
some
really
cool
stuff
with
it
and
to
accomplish
these
things,
the
only
thing
we
need
to
tune
is
the
kernel
that
we're
using
so
this
kernel
here,
this
filter
is
called
a
sobel
operator.
The
sobel
operator
enables
you
to
do
something
really
critical
edge
detection,
so
recall
that
image
of
the
pepper
that
we
looked
at
earlier.
B
B
B
Are
the
two
classic
sobel
operators
they
produce
the
gradients
in
x
and
the
gradients
in
y
and
after
combining
the
two,
you
actually
get
a
full
gradient
image.
So
it's
just
a
recap
here:
we're
essentially
going
to
be
looking
at
how
the
color
changes
between
every
pixel
in
the
in
an
image
and
that
those
rates
of
change
are
going
to
show
us
where
the
edges
are
in
the
image.
B
So
let's
illustrate
this,
here's
my
buddy
again
zero
in
his
mr
rogers
sweater.
That's
what
I
like
to
call
it
at
least
notice
all
the
edges
in
this
image
when
you
think
about
it,
those
edges
the
places
where
the
color
is
changing
really
rapidly.
That's
sort
of
where
all
the
information
is
the
flat
surface
is
like
this
wall.
In
the
background
here
or
the
tile
floor
over
here.
B
So,
after
writing,
a
convolution
with
a
sobel
operator
over
our
image
of
zero,
we're
left
with
the
grading
outline
of
all
the
edges
in
the
image.
Those
are
the
parts
that
are
very
bright
now.
The
operation
convolution
has
a
wide
array
of
applications,
particularly
around
feature
detection,
which
we
can
leverage
in
something
else.
You
might
have
heard
of
a
convolutional
neural
network,
which
is
a
topic
for
that.
We're
going
to
get
to
a
little
bit
later.
B
So
if
you
want
to
calculate
the
sobel
gradients
for
an
image,
use
the
process,
so
I'm
assuming
that
the
color,
the
images
that
I'm
reading
in
our
color
near
the
bgr
space,
so
the
first
thing
I
need
to
do
is
actually
convert
those
color
images
to
grayscale
images
and
that's
actually
what
this
first
line
is
doing
here.
It's
taking
our
input
image
and
restoring
in
this
gray
image
a
conversion
between
the
bgr
space,
the
blue,
green
red
space
and
the
gray
space.
B
The
next
line
here
this
gaussian
blur,
is
something
that
you're
going
to
see
quite
a
lot
of
when
you're
looking
at
image.
Processing
gaussian
blur
actually
blurs
the
image,
which
is
actually
a
good
thing,
and
it's
something
that
we
really
quite
like
to
do
and
I'll
explain
why
in
a
little
bit,
this
next
bit
here
is
actually
calculating
those
gradients
in
x
and
ny.
B
So
you
pass
in
the
grayscale
image
you
pass
in
the
output
image
that
you
want
to
store
it
in,
and
then
we
pass
in
this
data
type
here.
This
is
the
data
type
that
the
output
image
is
gonna,
be
stored
in
and
you'll
notice
that
it's
a
six
cv,
one
six
s
which
is
a
16
bit
short
and
the
reason
we
store
it
in
a
16
bit
short
as
opposed
to
a
8
bit.
B
Byte
is
because,
as
we're
performing
this
operation,
this
convolution
operation,
it's
actually
possible
for
the
pixels
to
overflow
and
spill
over
the
255
limit.
So
we
put
the
we
output
them
into
the
16-bit
short
temporarily,
and
then
we
pass
in
the
dimension.
We
want
to
calculate
next
and
then
y
and
the
kernel
size
that
we
want
to
use
after
that
we
actually
have
to
convert
the
the
16-bit
shorts
back
into
8-bit
bytes
and
then
finally
combine
the
two
together
to
actually
produce
our
final
images.
B
B
So
when
we're
looking
and
we're
scanning
left
to
right
in
here,
we
ski
we
see
the
lines
that
are
perpendicular
to
the
to
the
sole
operator
that
we're
using,
and
so
it
actually
detects
the
vertical
lines.
In
spite
of
the
fact
that
x
is
the
horizontal
bit
of
the
cartesian
plane
and
the
converse
is
true
for
y-
and
that's
actually
why
we
need
to
calculate
both
so
that
we
can
get
this
nice
final
result
here
and
you'll
see
after
I've
combined
the
two
together.
B
A
B
B
Cool,
so
probably
the
most
ubiquitously
used
kernel
is
the
gaussian
kernel.
Lots
of
operations
in
computer
vision
will
run
what's
called
a
gaussian
blur
over
an
image
prior
to
using
it
in
other
cv,
functions,
for
example,
the
silver
operator
we
just
saw
gaussian
might
sound
familiar
if
you've
ever
taken,
statistics
or
probability.
B
B
B
So,
to
illustrate
this
to
you,
let's
look
at
some
images
with
some
salt
and
pepper
noise
added
to
them.
Now
the
noise
off
the
digital
camera
sensor
isn't
necessarily
that
bad
this
bad.
But
this
is
that
image
of
the
pepper
we
saw
before-
and
this
is
an
image
of
einstein-
look
at
all
the
noise
in
here.
This
is
just
static.
B
So
here
we're
just
going
to
go
from
our
original
to
our
blurred.
Now,
like
I
said,
blurring
loses
detail
for
the
origin
for
us,
but
it
actually
gains
clarity
for
these
programs
now
on
the
left
here.
Another
thing
you
can
do
is
actually
sharpen
an
image.
So
if
you
take
this
blurred
image-
and
you
subtract
it
from
the
original
image-
you're
actually
left
with
this
detail
image
right
here
and
if
you
take
this
detail
image,
that's
the
detail
that
you
can
use
to
sharpen
it.
B
You
can
just
add
that
back
to
the
original
image
and
get
this
nice
sharpened
image,
now
it's
probably
easier
to
look
at
these
two
side
by
side.
So
look
at
this
original
image
of
zero
and
then
look
at
the
sharpen
images
zero
and
see
how
much
sharper
this
image
is,
how
much
more
precise
these
things
look
as
opposed
to
when
you're
blurring
and
you
actually
unweight
the
center
pixels
you're,
essentially
upweighting
the
center
pixels
for
each
of
these.
So
you
have
each
pixel
more
prominent
than
the
output
image.
B
So
an
artificial
neural
network
is
a
decision
system
inspired
by
a
biologic
neural
network.
It's
a
collection
of
nodes
modeled
into
by
modeled
on
biological
neurons
called
artificial
neurons.
These
artificial
neurons
are
arranged
into
layers,
the
input
layer,
some
set
of
hidden
layers,
some
set
of
hidden
layers
and
an
output
layer.
B
This
input
layer,
as
the
name
suggests
where
the
input
comes
into
the
hidden
layers,
are
proceeding,
functions
that
feed
forward
to
the
other
hidden
layers
in
the
output
layer
and
the
result
of
these
functions.
The
output
are
whether
this
is
active
or
not,
and
actually
speak
to
like
whether
it's
going
to
classify
something
or
not
so.
Here's
the
anatomy
of
an
artificial
neuron
you'll,
see
the
input
and
weight
links
on
the
left
coming
from
the
previous
layers
and
feeding
into
this
input
function.
B
This
activation
function
is
usually
some
nonlinear
function
like
a
integrated,
gaussian
or
something
now
neural
networks.
Allow
us
to
do
a
bunch
of
really
neat
things.
For
example,
classification,
though
there
are
some
important
limitations,
so
the
last
thing
we
need
to
talk
about,
which
is
regular,
neural
nets
is-
and
this
is
what
really
makes
them
very
powerful-
is
backward
propagation,
and
this
is
sort
of
how
they
learn.
B
When
you
seed
a
neural
network,
those
weight
links
that
we
spoke
about
earlier.
Those
are
the
weights
that
each
neuron
provides
all
of
its
inputs
and
when
it's
being
initialized,
it
doesn't
have
any
preconceived
notions
as
to
what
those
wages
are
supposed
to
be
and
the
way
that
it
learns
is
by
updating
those
weights.
B
B
Now
there
are
some
pretty
significant
limitations.
When
you're
talking
about
a
standard
neural
net
with
dealing
with
small
sample
sizes
and
linear
decision
boundaries,
neural
nets
can
be
quite
limited
in
what
they
can
do.
You
might
want
to
use
something
more
like
a
k,
n
or
something
like
that
k
nearest
neighbors.
B
On
the
other
hand,
when
you're
looking
when
you're
trying
to
do
classification
on
images,
neural
networks
on
their
own
are
quite
limiting
for
perceptrons,
the
layers
are
fully
connected.
So
if
we
look
back
at
this
thing,
fully
connected
means
that
each
neuron
is
connected
to
each
neuron
of
the
next
layer
and
vice
versa.
So
each
neuron
in
each
layer
is
connected
to
each
neuron
in
the
previous
layer.
B
Every
neuron
has
you
know
in
that
picture
of
zero
we
saw
before
is
1536
by
2048.,
so
you
have
close
to
three
million
or
million
different
connections
between
the
input
layer
up
between
the
input
and
each
layer
of
the
in
the
neural
network.
So,
even
if
you
decimated
that
down
to
1
8th
scale,
it's
still
like
49
links,
so
the
combinatorics
don't
really
favor
that
approach
and
that's
actually
where
we
bring
convolution
back
into
the
equation.
B
So
that
brings
us
to
convolutional
neural
networks,
which
are
neural
networks.
With
you
guessed
it
convolution
edited
so
convolutional
neural
network
is,
as
the
name
suggests,
essentially
a
combination
of
these
two
techniques,
convolution
and
neural
networks,
rather
than
feeding
each
individual
pixel
from
the
input
layers
into
the
hidden
layer
directly.
B
After
these
successive
layers
of
convolution
and
subsampling,
the
resulting
set
of
inputs
into
our
neural
network
are
actually
quite
small
and
limited
and
quite
useful
because
they
actually
just
you
know,
say
what
features
the
thing
has,
as
opposed
to
like
just
a
large
involved.
Massive
pixels,
so
you'll
see
you'll,
take
like
a
robot
and
maybe
you're
looking
for.
Maybe
one
of
your
features
that
you're
looking
for
is
the
hand
of
the
robot,
so
you'll
run
that
over
it
and
you'll
actually
get
this
spot.
Where
that
feet.
B
So
we
can
actually
use
this
in
net
and,
in
this
case,
we'll
be
building
a
chat
bot
which
will
detect
different
dog
breeds
since
over
social
media
channels,
for
example,
I'll
use
whatsapp
in
this
case
image
classification
we're
going
to
be
using
machine
learning.net.
This
is
a
machine
learning
library
available
to
c-sharp
and
f-sharp
programmers.
It
runs
in
this
and
it
runs
cleanly
in
the
clr
tensorflow,
specifically
the
ml.net
tensorflow
library
is
what
we'll
be
using
to
construct.
B
So,
for
the
sake
of
brevity,
I'm
not
going
to
review
how
to
set
up
the
entire
pipeline,
hence
the
ellipses,
but
it's
a
few
lines
of
code
that
will
determine
how
the
input
is
pre-processed.
So
what
size
the
input,
what
size
the
input
images
are
going
to
be
decimated
down
to
like
how
the
colors
are
going
to
be
extracted,
and
then
it
loads
up
a
pre-built
tensorflow
model
called
inception,
which
is
a
popular
model.
It's
just
so
you
don't
have
to
build
it
yourself.
B
You
don't
have
to
tune
the
whole
neural
network,
there's
actually
a
fairly
popular
data
set
out
there
that
I
I
use
for
the
sample
on
the
other
end,
we'll
be
we'll
fit.
Some
training
data
to
our
model
and
that
model
is
what
we're
going
to
be
using
for
the
classification,
but
at
the
end
of
this
pipeline
build
we
just
essentially
take
some
data
from.
B
B
B
I
this
is
going
to
be
in
a
regular
mvc
controller,
I'm
dependency,
injecting
a
config
that
has
an
app
id
and
a
private
key
path
that
I'm
going
to
use
for
actually
there's
some
user
secrets
that
phonics
needs
I'll
extract
those
credentials
and
that
enables
me
to
essentially
send
a
whatsapp
or
facebook
messenger,
fiber
message
or
sms
message
for
that
matter.
Using
the
advantage
messages,
api
next
I'll,
just
build
the
request
structure,
it's
going
to
be
a
text
with
a
dog
prediction.
B
You
see
it
so
I
actually
trained
it
with
a
specific
image
of
a
dachshund,
but
I'm
not
sure
if
anyone's
gonna
be
able
to
read
that.
So
I
apologize
because
I
don't
think
I
can
zoom
in
with
this
desktop
version
of
whatsapp.
But
it's
saying
I'm
about
90
sure
that
what
you
sent
me
is
a
dachshund
and
I
can
send
like
a
picture
of
my
greyhounds
honey
and,
let's
see
how
it
does
with
that
one.
Second,.
B
B
B
We
can
actually
run
face
detection
in
real
time
on
multiple
video
streams,
so
we're
going
to
be
discussing
a
technique
for
facial
detection,
developed
by
paul
viola
and
michael
jones,
which
is
both
really
rapid
and
robust.
We'll
take
we'll
take
these
little
masks
called
heart,
like
features
which
we'll
see
in
a
minute
are
just
very
simple.
B
These
far
light
features,
as
they're
called,
are
simple:
black
and
white
images
split
into
two,
sometimes
three,
sometimes
four
different
rectangles.
They
may
look
like
silly
little
shapes
but
they're
actually
extremely
useful
when
it
comes
to
running
all
sorts
of
detection.
Algorithms
in
particular,
here
we're
going
to
be
looking
at
using
them
in
facial
recognition,
so
one
or
two
of
them
by
themselves
aren't
particularly
useful,
but
hundreds.
C
B
Applied
at
varying
scales
to
the
same
image
against
the
well-tuned
model
produces
a
really
robust
and
probably
more
importantly,
very
fast,
and
I
mean
can
run
in
real
time
on
multiple
video
streams
with
low
overhead
fast,
but
wait
you're
asking
you're,
probably
asking
yourself
you're
running
hundreds
or
thousands
of
these.
At
the
same
time,
you
have
to
sum
up
all
the
pixels,
under
both
regions
of
the
image
of
this
highlight
feature
and
subtract
them
from
each
other
at
varying
scales.
So
how
could
this
possibly
work?
B
Performantly
and
the
answer
is
actually
at
the
core
of
the
viola
jones
algorithm,
so
an
integral
image,
and
we
thought
we
were
done
with
our
calculus
terms
for
the
day.
So,
if
you
might
recall
what
an
integral
is
an
integral
is
just
the
area
under
a
curve,
that's
the
sum
of
everything
up
to
a
certain
point
in
the
function
and
that's
exactly
what
these
are
remember.
I
said
that
an
image
at
the
end
of
the
day
is
just
a
function.
Well,
that's.
B
B
I
yeah.
I
didn't
check
this.
That's
fine,
but
the
the
point
is
that
these
every
pixel
in
here
is
supposed
to
add
up
to
every
pixel
above
and
to
the
left
of
it
in
the
in
the
integral
image.
And
what
does
that
do
for
us?
Well,
those
hard
light
features
we
were
looking
at
before
require
us
to
sum
the
white
area
in
the
black
area
and
find
the
difference
between
those
areas
and
that
summation
is
on
the
order
of
n
by
m,
where
n
is
the
height
and
m.
B
Is
the
width
of
that
feature,
and
now
that
might
be
acceptable
for
one
feature,
but
this
algorithm
requires
you
lots
of
them
to
robustly
detect
faces,
so
this
diagram
actually
illustrates
why
the
integral
image
makes
this
algorithm
so
much
faster.
So,
to
find
this
red
rectangle
here
we
need
to
take
the
bottom
right
point
subtract.
The
bottom
left
point
the
top
right
point
and
then
add
back
the
top
left
point,
and
how
does
that
work?
Well,
remember
d.
Is
this
entire
region
here?
B
So
that's
the
sum
of
all
the
pixels
in
this
region
c
is
the
sum
of
everything
over
here
b
is
the
sum
of
everything
up
here,
so
we
just
need
to
cancel
out
b
and
c,
and
then
we
need
to
on
negate
the
double
negated
portions,
that
of
the
overlap
between
b
and
c,
and
so
that
allows
us
to
in
an
order.
One
operation
calculate
this
whole
region
in
here,
using
just
that
integral
image.
B
B
So
how
does
this
all
work
on
our.net
program?
Well,
it's
actually
quite
a
bit
easier
than
what
I
just
described.
Opencv
has
a
large
collection
of
hard
light
features
pre-arranged
into
a
model
for
a
cascading
classifier.
The
model
is
available
in
an
xml
file
that
can
be
very
easily
adjusted
using
mqcb.
B
B
That's
this
bit
right
here,
where
I
just
pull
in
that
xml
file.
I
referred
to
earlier,
pull
in
the
image
and
then
run
this
detect
multiscale
on
that
image
and
I
actually
pass
the
min
size.
So
that's
the
minimum
size
of
a
face,
I'm
looking
for
in
this
image,
and
then
I
just
draw
a
rectangle
on
each
face
that
was
detected
in
the
image,
and
here
is
actually
the
result
of
that
little
program
run
on
an
image
of
me.
As
you
can
see
it
detected
my
face.
B
So
that's
it
for
how
you
do
basic
face
detection.
Now
this
method
is
robust
to
different
scales
of
an
image.
In
fact,
it's
almost
a
little
too
robust.
You
see
he's
still
up
there,
so
I
actually
ran
this
the
first
time.
The
reason
I
needed
this
300
by
300
mint
size
is
because
this
thing
is
so
sensitive.
B
B
So
now
we're
going
to
take
that
same
technique
and
apply
it
in
a
real-time
application
to
see
just
how
powerful
it
is
and
for
that
we're
going
to
be
using
the
bonnet
video
api.
So
we're
just
going
to
create
a
wpf
client
app.
Add
the
opentalk
client
sdk
to
it
and
add
a
new
class
implementing.
What's
called
an
eye.
Video
renderer
called
face.
Detection,
video
renderer,
which
is
extending
a
control,
and
I
video
render,
is
just
an
interface
that
the
opitalk
library
uses
to
publish
individual
frames
to
a
to
your
video
app.
B
I've
gone
ahead
and
I
populated
the
api
key
and
session
id
and
a
token.
The
api
key
is
tied
to
your
finance
api.
Your
violence,
video
api
account.
The
session
id
is
just
generated
using
the
bondage,
video
rest
api
and
the
token
is
jwt
that
your
server
generates
for
you,
so
that
all
can
be
created
dynamically
as
well.
I'm
just
hard
coding
them
for
the
sake
of
simplicity,
notice.
How
I'm
constructing
this
publisher
here
this
is.
This
is
where
your
videos,
your
output,
video,
is
going
to
go.
B
B
Now
this
function
is
what
we
are
going
to
register
the
stream
created
event
callback.
This
will
create
a
face:
detection,
video
renderer
for
all
inbound
streams.
Anybody
else
calling
to
the
video
call
and
add
that
renderer
to
the
layout
and
set
the
rendering
mode
to
the
current
rendering
mode
and
then,
of
course,
subscribe
to
the
stream.
B
So
now
that
we
have
the
layout
and
the
session
initialization
working,
we
need
to
manage
the
actual
real-time
phase,
detection
and
the
display
of
our
boxes.
So,
by
having
our
face
detection,
radio
renderer
implement
the
ivideo
renderer.
We
are
going
to
be
able
to
get
each
frame
that
comes
into
the
renderer.
B
B
B
So
getting
back
to
performance,
the
a
video
call
at
best
you're
going
to
be
getting
30
frames
per.
Second,
that's
like
at
the
absolute
like
peak
performance.
It's
not
like
a
it's,
not
a
video
game,
you're
not
going
to
get
like
60
or
120
fps,
but
at
30
fps.
This
thing
was
able
to
run
each
video
frame
disabled
process
in
five
to
ten
milliseconds.
B
B
B
A
B
Anyway,
so,
like
I
said
this
is
where
my
camera
is
supposed
to
go,
but
because
of
stream
yard,
I
guess
it.
I
guess
it's
just
holding
my
camera
open
for
some
reason
and
on
windows
you
can
only
only
one
application
can
hold
your
camera,
so
I'm
just
going
to
connect
to
this
session
and
that's
not
good.
B
C
B
B
C
B
C
C
B
B
B
B
But
if
I
like
turn
my
head
around
or
like
I
change
the
shading
of
my
face,
it
can't
really
detect
it,
but
if
I
like
went
back
the
scale,
the
scale
works
really
well
and,
like
I
said
this
is
working
in
five
to
ten
milliseconds
per
frame.
And
if
I
had
my
camera
on
here,
you
would
also
see
it
on
both
ends.
It
would
actually
operate
off
both
ends,
but
yeah.
That's
how
you
do
face
detection.
B
B
B
B
You
can
send
text
messages
with
us,
but
basically
any
communication
channel.
You
need
to
talk
to
your
customer,
whether
it's
sms
or
voice
or
whatsapp,
five
or
facebook,
messenger
or
videos
like
you
can
use
us
for
our
main
documentation
website
is
developer.expo.com
clarisse,
who
organized
who
set
this
up
for
us,
gave
us
a
coupon
code
for
10
euro
worth
of
free
credit
for
our
main
api,
our
main
apis
and
the
video
api
comes,
I
think,
with
a
ten
dollar
free
credit
too.
So
yeah,
I
mean
check
this
out
developer.nexpo.com.
B
So
this
so
here
are
some
resources.
Everything
I
showed
you
is
available
in
github,
github.com89
you'll
find
everything
you
need
some
other
resources.
I
put
the
mgu
cv
documents
up
here.
I
didn't
actually
find
them
that
useful.
They
they
show
you
sort
of
how
to
do
stuff,
but
I
think
the
open,
opencv
docs
are
better
and
for
the
stuff
I
did
with
the
bottas
video
api.
B
A
That's
fine
and
I
think
we
yeah
it's
if
it's
fine,
let's
leave
the
the
screen
here
so
that
we
could
give
some
of
the
participants
a
chance
to
to
yeah
get
take
look
up,
some
of
those
urls
and
and
and
yeah
check
them
out
for
themselves.
Yeah
really
awesome
talk.
I've
always
been
interested
in
this
and
I
finally
have
an
answer
for
when
for
for
when
my
kid
asks
me
what's
a
practical
use
for
calculus,
because
I
don't
think
there's
any.
B
A
A
Let's
take
this
opportunity
to
answer
some
questions
and
the
first
question
that
was
posted
in
the
channel
was
actually
from
from
from
from
louise
beltran
who
who's
who
is
from
the.net
foundation
joe.
So
so
let
me
take
this
opportunity
to
thank
the
dotnet
foundation
for
for
for
well
hosting
this
stream,
and
all
of
that
so
louise's
question
was
how
to
decide
whether
to
use
a
max
spooling
or
average
pooling,
and
then
this
was
yeah
the
earlier
the
edge
detection,
yeah.
B
B
Too
much
into
like
the
actual
tuning
of
these
algorithms,
especially
with
cnns,
but
those
tend
to
be
like
whatever
your
use
cases
you
need
to
decide,
you
need
to
sort
of
decide
for
yourself
like
what
you
find.
What
you
find
with
these.
A
lot
of
these
techniques
is
like
okay,
well,
here's
how
the
course
thing
works
for
this
thing
now
I
have
a
particular
application.
I
want
to
apply
it
to
so.
B
A
Okay,
I'll
I'll,
let
louise
say
if
that's
actually,
if
that
actually
answered
this
question
anyway,
we
could
we,
we
could
go
back
to
it.
If,
if
there's
further
questions,
yes,
luis
says
it's
all
good
all
right,
cool
yeah,
francis
libertavar
has
three
questions,
and
so
so
here's,
how
is
the
sample
image,
mapped
to
an
m
m
by
n
matrix?
Is
it
unrolled
into
intensity
values
per
channel?
So
you
end
up
with
an
m
times:
n
is
written.
B
I
like
this
question
so
that
that's
a
great
question
francis
so
like
I
said
so
like
I
said
when
you're
when
you're
thinking
about
these
images,
the
math
abstracts
out
to
any
number
of
channels.
I
use
the
the
two,
the
m
by
n
matrix
because
it's
just
simpler
to
think
about,
but
you
can
essentially
convert
between
the
different
channel
planes,
so
I'm
just
gonna
rewind
a
little
bit
in
my
presentation
in
my
presentation.
B
So
if
we
go
all
the
way
back,
it's
probably
just
easier.
If
I
popped
out
of
this
thing,
wouldn't
it
be?
Do
you
hear
to
the
so?
This
is
that
sobel
operation
thing
I
was
talking
about
before
you'll
see.
The
first
thing
I
do
here
is
I
actually
convert
between
the
different
color
planes
and
the
reason
you
want
to
convert
between
the
different
color
planes.
B
You
could
still
do
it,
it's
just
a
more
complicated
operation
and
I'm
not
a
hundred
percent
sure
that
opencv
always
supports
the
old
sports
at
operation
with
larger
dimension
matrices.
So
let
me
pop
back
over
to
the
that's
last
slide
there.
So
I
think
that
should
have
answered
this
question,
though,.
B
A
B
B
Techniques
like
the
first
thing
you
do
is
convert
it
to
grayscale
blur
the
image,
and
then
you
run
you
run
whatever
it
is
over
it.
Probably
in
most,
I
would
guess
in
most
cnns
most
convolutional
neural
networks.
The
first
thing
you
do
is
convert
it
to
convert
between
color
scales
and
what
is
it
gray
out
the
image
or
create
out
the
image
or
resize
it
and
stuff
like
that?
B
When
you
resize
an
image,
you
actually
do
lose
information
or
you
lose
information
in
both
directions.
Actually
I
didn't
get
into
it,
but
it
really
interesting
operation
is
resizing.
When
you
down
scale
an
image.
What
you
do
is
you
actually
delete
each?
You
delete
every
other
row
and
column
from
the
image
and
then
you
run
a
convolution
over
it
to
actually
pull
the
information
back
in
and
make
it
a
more
sensible
object
that
you
have
a
more
sensible
image
that
you
have
at
the
end
of
it.
A
A
B
I
I
mean
I
I
didn't
say
that
I
didn't
tune
anything
so
like
I
never
tuned
anything
in
that
inception
model
that
I
was
using
for
the
the
cnn.
That
was
just
like
an
example
of
like
how
to
do
something
but
yeah
you
have
to
tune
your
hyper
parameters,
usually
based
off
of
what
your
application
is
all
right.
B
Model
retrain,
if
I,
if
I
tell
it
to
if
I
tell
it
that
this
is
a
like.
This-
is
a
greyhound
yeah
it
just
retrained,
it
retrains
the
model,
so
it
it
preserves
the
pipeline,
and
you
know
what.
Since
I
mean,
we
have
actually
really
good
crowd
here.
So
I'll
just
show
you
the
pipeline
that
I
built
this
thing.
It's
very
simple
one!
It's
not
there's
nothing
too
complicated
going
on
here.
B
Here,
it's
just
basically
it
resize
it
loads,
the
images
resize
them,
it
pulls
the
pixels
out
of
them
and
then
it
loads
everything
into
this.
This
tensorflow
model,
that's
the
inception
model
that
I
spoke
about
earlier
and
then
it
loads
everything
into
the
the
second
a
lot,
the
last
hidden
layer
of
it
and
then
the
effectively
the
output
layers
become
the
predicted
label
value
for
the
output-
and
I
preserve
this
pipeline
here
and
actually
in
this
ad
training
image.
So
you
saw
earlier,
I
like
I
took
like
this
picture
of
my
dog
honey.
B
A
A
A
B
B
But
like
so
most
so
hold
on.
Let
me
just
take
a
step
back
there,
so,
like
most
of
the
biggest
use
case
in
the
vanish
video
api
by
far
is
online.
It's
online,
it's
like
in
a
browser
or
something
like
that,
so
like
chrome,
for
example,
and
chrome,
uses
its
own
encoders
that
we
build
on
top
of
and
those
encoders
might
or
might
not
use
hardware
acceleration.
I
think
they
mostly
do
if
it's
available.
B
B
A
A
For
example,
there's
people
are
starting
to
use
it
for
for
practical
applications
like
like
online
online
video
interviews,
something
like
that
and
and
one
of
the
one
of
the
the
thing
I
read
earlier
was
about
the
study
made
by
by
this
group
of
journalists
around
the
api
of
of
of
a
startup
in
germany.
That
was
using
just
using
machine
learning
to
do
some
behavioral
analysis
of
the
of
the
interviewees
and
they
they.
A
They
noted
that
even
just
a
change
in
attire,
a
change
in
the
background,
the
change
in
lighting
would
do
would
affect
this
course
of
stuff.
So
so
I
guess
I
guess
my
question
there.
What
do
you
think
of
that?
How
much
is
this
stuff,
for
you
know
something
as
advanced
as
as
analyzing
behavior?
I
mean
right
now,
right
now,
the
the
most
practical
use
of
of
of
machine.
A
You
know
machine
vision
that
I
know
is
when
I
go
to
a
local
mall
here
in
in
the
philippines.
There's
actually
a
camera
pointed
at
you
measuring
your
temperature
already
so
yeah
as
a
precaution
as
a
precaution
against
covered
and
and
that's
one,
that's
the
that's
one
application,
but
using
it
for
job
and
for
for
behavioral
analysis
and
job
interviews.
That's
that's
something!
That's
so
I
don't
know
mission
impossible.
A
B
B
A
A
Yeah
yeah,
it
definitely
is,
it
definitely
is,
and
you
know,
with
the
pandemic
and
everything
it's
even
becoming
more
relevant
as
we
become
more
more
dependent
on
things
like
computer
vision,
for
you
know
just
just
remotely
monitoring
things
yeah
well.
That
was
a
very
interesting
topic.
It
definitely
definitely
a
lot
more
to
be
learned.
There
yeah
you
displayed
earlier
a
a
a
was
that
a
coupon
code
yeah
for
for
for
the
vanage
api.
A
So
I'm
gonna
ask
that
from
you
and
and
we're
going
to
post
that
in
the
philippine.net
users
group,
both
in
the
facebook
group
and
in
the
in
the
the
facebook
group,
the
facebook
page
and
in
the
twitter
account
so
that
so
that
yeah,
those
those
who
had
watch
this
video
could
could
be
able
to
use
that
and
and
yeah
just
check
out
the
vanage
api
for
themselves.
All
right.
B
A
A
All
right,
thank
you,
steve,
and
that
is
steve
lorello
from
vanage
talking
to
us
about
machine
learning.net,
all
right
before
we
end
this.
Before
we
end
this
talk,
we
have
a
few
friends
over,
I
believe
from
from
cloud
staff
we'd
like
to
to
ask
yeah
to
ask
them
to
go
online,
but
but
while
we're
waiting
for
them,
I'd
like
to
take
this
opportunity
to
thank
thank
the.net
foundation
for
hosting
this
stream
for
us.
Thank
you
very
much.
A
So
so
I
I'm
aware
that
a
lot
of
people
have
have
been
looking
for
life
for
opportunities
in
this
pandemic,
for
especially
for
remote
working
ones
and-
and
we
have
been,
the
philippine
users
group
has
been
partnered
with
cloud
staff
for
for
for
some
time
now,
working
in
some
events
together,
and
that's
so
they'd
like
to
talk
about
about
some
of
some
of
their
their
offerings
for
the
community.
So
to
talk
to
us
from
cloud
staff,
I
turned
this
floor
over
to
marissa
cordero
so
good
evening.
A
C
C
All
right,
hopefully,
everyone
can
see
my
presentation,
I'm
marissa
cordero,
I'm
one
of
the
development
leads
at
cloud
staff,
philippines,
and
we
are
here
to
share
with
you
some
of
the
net
open
positions
in
our
company
and
a
brief
introduction
about
cloud
staff,
philippines
and
who
we
are
all
right.
So
why
do
we?
Why
work
at
cloud
stuff?
C
C
What
we
realized
is
that
work
from
home
is
here
to
stay
and
we
are
quite
flexible
in
that
area.
We
have
continuous
development,
we're
actually
building
cloud
staff
university
as
of
the
moment,
and
most
especially
you
have
your
success:
managers
for
employees
and
clients.
What
that
means
is
for
anyone
who
will
be
joining
cloud
staff
is
that
they
will
be
having
your
support
system
with
you
to
ensure
that
that
you
are
on
boarded
properly
into
the
company
into
your
client.
C
C
Lots
of
perks
actually
do
visit
us
at
www.cloudstuff.com.
If
you
would
like
to
learn
more
about
us
and
now
moving
on
to
the
open
roles
that
we
have
for
that
net,
we
have
here
in
clark
pampanga
it's
for
morning
shift.
This
is
in.
You
will
be
in
the
office,
it's
a
death
net
and
summer
in
a
developer
with
there's
a
minimum
experience
in
software
development
proficiency
in
that
net
core
seashell
and
submarine.
Of
course,
there
will
be
some
front-end
web
development
for
you,
that's
javascript
and
angular.
C
You
also
have
another
open
position
in
senior
as
a
senior.net
developer,
with
at
least
six
years
of
experience
in
development,
and
this
is
a
work
from
home,
so
anyone
who's,
even
if
you're
living
outside
of
pampanga
that's
fine.
This
is
for
a
mid-shift
position,
another
senior
that
net
developer
position-
and
this
is
again
in
clark.
This
is
gonna,
you're
gonna
be
in
the
office.
This
is
a
night
shift
position.
It's
for
front
end
development
you'll,
be
using
angular
in
here,
and
some
of
the
tech
staff
are
mentioned
here.
C
Another.Net
developer
position
again.
This
is
a
work
from
home
position.
It's
it's
good!
It's
a
morning
shift.
We
are
looking
for
at
least
five
years
experience
in
development
using
that
net.
With
this
one
again
take
a
look
at
the
technology
stock
in
the
screen.
If
this
might
interest
you
another
open
position
in
clark
morning
shift
it's
five
years
again
experience
in
software
development
and
nice
to
have.
We
have
at
least
if
you
have
experience
in
agile
or
scrum.
That
will
be.
C
That
will
be
an
advantage
in
the
front
end
web
development
as
well.
This
is
for
a
software
development
developer
position.
I
think
this
is
for
an
internal
position.
I
think
this
is
for
my
team.
The
net
team
internal.
This
is
a
work
from
home
at
least
two
years.
Experience
is
what
we
need
for
here
again,
that
net
angular
and
that
net.
This
is
more
of
a
front
end
position.
C
C
We
have
another
open
position.
This
is
another
work
from
home.
It's
morning
shift
csharp.net
developer.
This
is
for
at
least
you
need
to
have
experience
in
that
network
development,
web
services
for
soap
and
dress
and
the
strong
innovation,
object-oriented
programming,
mvc
design
patterns
and
solid
principles.
C
Lastly,
we
have
a
full
stop
that
net
developer
position
in
clark
pampanga
for
morning
shift
technologies
required
are
shown
on
the
screen
and
familiarity
with
different
tools
such
as
jira
gate,
visual
studio
will
be
an
advantage
for
anyone
who
might
be
interested,
and
I
think
we
will
be
sharing
more
of
this
with
you
on.
I
think
in
page
in
the
ug
page,
so
wait
for
wait
for
further
announcements,
so
do
register
and
apply
ensure
that
you
use
this
link.
It's
jobs.cloudstuff.com,
slash
register,
a
question.
Mark
ref
is
equal
one.
C
Four
five:
please
use
this
link
and
we
would
like
to
request
everyone
to
to
fill
out
your
profile
and
do
not
forget
to
upload
your
cv
and,
on
our
page,
make
sure
to
select.
Where
do
you
hear
about
cloud
stuff
to
select
on
the
drop
down
fin
ug,
if
I'm
saying
that
correctly
under
drop
down
again,
please
use
this
link
to
register
so
that
you
will
be
on
our
database.
C
Your
skill,
your
asking
salary
to
a
potential
client
all
right,
so
a
quick
look.
This
is
our
jobs
that
cloud
stuff.
If
you
use
that
url,
I
advise
everyone
to
register
by
you
know.
Mine
will
use
first
name
last
name
instead
of
registering
via
google
and
complete
your
profile
upload.
Your
cv
provide
us
your.
B
C
A
We'll
definitely
share
that
link
on
our
yuji
yugi,
page
ug
channels
and
for
for
everyone's
benefits,
so
yeah.
If,
if
you're
looking
for
you
know,
you
know
a
a
work
from
home
job,
something
that
is
that
yeah
keeps
us
safe
with
this
pandemic
right
yeah.
These
are
some
awesome
opportunity,
opportunities
and
yeah,
even
if
you're
not
looking
personally,
please
do
share
these
to
you
to
your
friends.
As
you
know,
we
can
help
each
other
in
in
these
times
all
right.
That
is
it.
C
A
Presentation,
okay,
so
I
think
that
brings
us
to
the
end
of
this
evening.
Again,
we
thank
our
friends
from
we
thank
our
friends
from
cloud
staff.
We
thank
our
friends
from
the.net
foundation
and
we
thank
our
our
speaker
from
vanage
for
for
for
that
really
great.
If
evening
of
of
of
learning,
and
thank
you
so
much
for,
thank
you
so
much
to
all
the
people
who
had
been
in
this
live
stream
for
for
for
the
evening,
so
I
hope
you
have
a
good
night.