►
From YouTube: Predicting on a Custom Vision ONNX Model with ML.NET
Description
How to use a model from the Custom Vision service in ML.NET to make predictions.
Code - https://github.com/jwood803/MLNetExamples/blob/master/MLNetExamples/CustomVisionOnnx/Program.cs
Netron - https://github.com/lutzroeder/netron
Custom Vision Sample - https://github.com/dotnet/machinelearning-samples/tree/master/samples/csharp/end-to-end-apps/ObjectDetection-Onnx
Custom Vision model video - https://www.youtube.com/watch?v=zr6M7Lzr48w&t=28s
ML.NET Playlist - https://www.youtube.com/watch?v=8gVhJKszzzI&list=PLl_upHIj19Zy3o09oICOutbNfXj332czx
Contact:
Twitter: https://twitter.com/JWood/
Blog: https://jonwood.co/
Gear used (affiliate links):
Mic - https://amzn.to/2YEXtxI
Mouse - https://amzn.to/2ZtASoQ
A
Hey
everyone,
so,
while
back
I
did
a
video
where
we
used
Microsoft
custom
vision
service
to
create
this
model
here,
where
it
not
only
tells
us
if
we
have
a
red
or
white
one
in
the
photo,
but
it's
actually
an
object
detection
model.
So
it
tells
us
some
information
on
where
in
the
photo
the
white
is.
But
this
is
all
own
custom
vision.
What,
if
I
want
to
use
this
model
in
mo
Annette?
A
Well,
we
can
do
that,
and
the
first
thing
here
is:
we
need
to
export
this
model,
so
we
go
to
up
to
performance
here
and
then
export
and
we
got
different
types
of
files
that
we
can
export
here.
Coromoto
tensorflow
darker
file
with
the
model
in
it.
But
what
I'm
going
to
use
is
that
onyx
file
here
so
I'll
click
on
this
and
export.
A
Alright,
then
I'll
click
download
once
it's
ready
and
it
downloads
the
model
file
and
some
other
stuff
here.
For
me
all
right,
so
this
download
finished:
let's
go
ahead
and
export
our
extract
out
the
files
here
and
I
see
what
we
have
is.
We
have
our
onyx
file
here
that
we'll
be
using,
and
we
also
have
the
labels
file
here.
That
tells
us
red
or
white,
which
is
the
labels
in
our
custom
vision
model.
Then
it
gives
us
some
sample
code
and
c-sharp
and
Python
that
we
won't
actually
be
be
needing
here.
A
So
let's
go
to
visual
studio
here
in
order
to
have
a
project
set
up,
have
a
model
folder
here,
a
test
folder
with
a
couple
of
test
files
that
I'll
be
using
one
for
red
one
and
a
little
for
a
white
one
then
have
a
this
bounding
box
class
and
if
you
notice
in
the
program
here,
I
have
some
some
variables
and
helper
items
already
set
up
that
we'll
be
using
later
before.
The
model
file
that
we
have
here.
A
All
we
need
to
do
is
take
the
Onix
file
and
the
labels
file,
and
we
move
those
over
to
the
model
folder
here
and
I'm
just
going
to
set
these
to
copy
over
now
that
we
have
our
model
exported.
Now
we
can
start
using
it
and
I'm
going
to
go
ahead
and
download
some
new
get
dependencies.
So
the
first
thing
we
need
is
ml.
A
Dinette
itself
use
the
1.4
version
here
since
we're
doing
images
I'm
going
to
get
the
image
analytics
NuGet
package,
and
since
we
have
a
model
file,
I'm
gonna
get
the
Onyx
transformer
package
as
well.
Alright,
with
that,
we
can
get
started
building
our
ml
net
a
pipeline
here.
The
first
thing
is:
we
need
the
new
in
little
context,
we're
just
using
the
onyx
model
here,
we're
not
using
any
input
data
I'm
going
to
create
some
empty
data.
A
A
A
So
the
first
thing
I'm
gonna
do
is:
do
a
transform
and
resize
the
images,
and
these
this
is
the
pipeline
when
you
give
it
an
image
to
run
against
the
model.
So
it's
going
to
first
resize
the
image
I'm
set
resizing
property
and
I'll
set
the
resizing
khand
to
fill
and
I'll
set
the
output
column
name
to
data
the
image
width
using
that
same
image
settings
the
image
height
to
the
image
settings
as
well.
The
input,
column,
name,
I'll,
use
the
name
of
keyword
and
now
use
the
wand
input
image
property.
A
So
that's
it
for
the
resize
images.
Next
I'll
append
to
it
another
transform
where
it
extracts
the
image
pixels
then
I'll
just
set
the
output
column
name
for
here
set
of
the
data
as
well
and
our
last
atom
in
the
pipeline.
We
will
call
the
transform
to
apply
our
next
model.
You
can
give
to
the
model
file
which
is
in
the
model
folder
called
modeled
onyx
give
the
output
call
name,
we
don't
get
to
actually
name
the
upper
column
name
whatever
we
want.
A
We
had
to
be
specific
to
whatever
the
output
column
name
and
this
onyx
file
here
and
there's
actually
way.
We
can
get
that
name
using
a
tool
called
Neutron.
So
let's
take
a
look
at
where
to
get
that
and
how
to
use
it
all
right,
I'm
at
the
github
repo
here
at
the
Metreon
project-
and
this
is
just
a
little
desktop
app
that
where
you
can
get
some
information
on
a
model
here
and
to
to
get
it
just
go
to
the
releases
and
then
download
the
release
for
your
platform.
We
started
up.
A
It's
gonna
be
just
kind
of
an
empty
screen.
Here
we
can
open
the
model
and
just
give
it
the
model
that
we
had
downloaded
from
custom
vision.
You
know,
and
it
gives
us
some
information
about
our
model
here
we
can
click
on
the
model
properties
and
it
gives
us
our
inputs
and
then
our
outputs.
In
fact,
we
gave
our
input
the
same
as
the
input
here.
A
A
A
It's
going
to
be
a
float,
float
array,
I'll
call
it
predicted
labels.
Let
me
give
it
an
attribute
here:
the
column
name
attribute
from
pillow
dinette
and
I'll
call
it
model
outputs
0,
make
it
match
what
we
saw
in
the
net
Ron
application
for
the
model
and
before
we
can
predict
on
it,
we
need
to
get
our
references
to
our
test
files
here.
Pretty
quick
I'll
just
do
a
variable
I'm,
a
string
array
with
the
names
of
those
test
files
that
we've
shown
and
then
I
can
loop
through
each
of
those
files.
A
So
we
have
the
image
name
and
then
I'm
going
to
use
a
stream
specifically
a
new
file
stream.
That
can
read
that
following
get
that
the
file
information
from
it
so
I'll
give
it
the
path
and
I'll
give
it
file
mode
open,
and
then
the
stream
I'm
going
to
set
a
test
image.
Variable
I'm
gonna,
do
image
that
from
stream
and
passing
the
stream
here,
let's
test
image,
I'll
set
up
here:
it's
a
bitmap!
A
You
get
an
error
here.
You
can't
convert
it,
so
I
can
do
an
explicit
cast
to
a
bitmap
from
that
from
stream.
Alright.
So
now
that
we
have
the
bitmap
of
our
test
images,
we
can
create
a
prediction
using
the
prediction.
Engine
dates,
give
it
a
new
one
input
and
just
set
the
image
to
the
test
image.
Now
here's
where
we're
going
to
be
starting
using
all
these,
these
extra
helper
methods
here
and
I'm
gonna,
be
honest.
I
didn't
come
up
with
all
of
these
in
more
than
that
sample
that
reads
from
a
custom
vision.
A
Our
next
model
has
these
so
so
I
use
those
as
well
and
honestly.
I
doubt
I
could
come
up
with
all
this.
If
you
look
at
all
these
helper
methods
here
and
what
these
pretty
much
all
do
is
allow
you
to
use
that
bounding
box
class
here
and
just
gets
the
bounding
box
information
from
the
model,
and
it
just
calculates
it
to
what
we
can
put
into
our
image
to
create
the
bounding
box.
That's
pretty
much
all
this
is
doing
so
we
can
get
our
bounding
boxes
using
the
parse
outputs
method.
A
A
Let
me
get
the
original
width
of
the
test
image.
Let
me
get
the
same
width,
the
height
and
so
I'll
do
a
check
here.
If
we
have
at
least
one
bounding
box,
I'll
get
the
max
confidence
that
it
gets
from
it
using
the
max
link
method
here,
and
it
has
that
confidence
properly
on
it
now
get
the
top
bounding
box.
With
that
Max
confidence.
A
Used
to
first
your
default
link
method,
it's
gonna
be
the
first
one.
It's
gonna
be
first
bounding
box,
where
the
confidence
equals
that
nice
confidence
now
I'm
gonna
clear
all
these
bounding
boxes
and
then
add
the
top
one.
So
I
only
have
one
bounding
box.
That's
gonna,
be
the
one
with
the
max
confidence,
because
I
can
give
you
multiple
bounding
boxes
for
each
of
the
predictions
that
it
gives
you.
A
So
it
can
have
multiple
predictions
for
where
in
the
image
the
one
is,
but
we're
just
gonna
take
in
the
top
one-
and
here
is
where
we're
gonna
be
using
a
bunch
of
that
a
bunch
of
those
helper
methods.
In
fact,
I'll
just
I'll
just
copy
these
over,
because
it's
a
good
bit
of
math
here
and
I
misspelled,
the
original
height
and
width
here,
but
basically
it's
just
getting
the
X
&,
Y
width
and
height
of
the
bounding
boxes.
A
So
we
know
where
to
draw
the
rectangle
onto
the
image
and
we're
using
the
graphics
API
here
to
draw
that
rectangle
and
we're
gonna
draw
the
string
of
the
description,
which
is
pretty
much
give
us
the
confidence
percentage
and
then
we're
gonna,
save
that
test
image
to
the
file
system
and
give
the
name
and
then
that's
what
we
can
use
to
look
at
the
file
that
has
the
information
from
our
model
all
right.
So
let's
run
this
and
see
what
we
get
here
or
so
that
ran.
A
So,
let's
go
to
our
bin
folder
and
our
test
I
guess
we
got
our
two
predicted
files:
here's
the
red
one
and
hopefully
y'all-
can
see
this
okay
I
get
our
a
nice
bounding
box
on
it,
and
it
says
it's
a
red
wine
and
a
76
percent,
confident
on
it.
White
wine
is
a
little
bit
different
at
volume,
box
didn't
get
the
entire
one,
but
it's
pretty
close
and
it
did
predicted
to
be
whites
in
this
fifty
nine
percent.
Confident,
alright,
everyone,
that's
just
how
you
can
use
your
custom
vision
model
within
in-law
dinette.
A
If
you
look
at
the
pipeline
is
actually
pretty
simple
just
dealing
with
the
images
the
resize
extract
them
and
then
we
just
apply
the
Onix
model,
and
then
you
have
to
use
that
metrion
tool
to
get
the
output
column
names
in
there,
noting
that
the
the
biggest
part
here
is
gonna
be
doing
this.
Hopefully,
then
it
gets
easier
in
the
future,
where
we
don't
have
to
do
all
this
to
get
our
bounding
boxes,
but
for
now
this
is
what
we
had
to
do
so
I
hope
you
enjoyed
the
video
and
thanks
for
watching.