►
From YouTube: Automating metadata tagging with AI using Power Automate
Description
In this 17-minute Citizen Developer-focused demo, Uday Adhikari presents a scenario and solution. Scan and tag 15,000 inconsistently formatted documents annually that are uploaded to a SharePoint document library. Presenter shares thinking behind 3 capable options (OneDrive, OCR, Computer Vision) and based on technology strengths/limitations, he choose Azure’s Cognitive Services - Computer Vision Read API that’s called from a Power Automate flow. The customer need only update a keyword mapping in a list rather than editing the Flow. This PnP Community demo is taken from the Microsoft Power Apps community call - March 2021, recorded on March 17, 2021.
Demo Presenter: Uday Adhikari | @udayadhikari
Supporting documents:
• Documentation - Document automation toolkit | https://docs.microsoft.com/en-us/ai-builder/doc-automation
• Documentation - Optical Character Recognition (OCR) | https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/concept-recognizing-text
Stay connected:
• Twitter https://twitter.com/microsoft365dev
• YouTube https://aka.ms/M365DevYouTube
• Blogs https://aka.ms/M365DevBlog
B
B
A
Back
in
december
january,
and
things
have
changed
a
little
bit
as
microsoft
has
also
introduced.
I
released
a
solution
where
you
can
actually
automate
the
document
tagging,
so
I'm
gonna
kind
of
start
talking
on
a
surface,
but
thank
you
thank
you
for
having
me.
I'm
super
excited
to
present
my
learnings,
my
sharing
right.
So
what
I've
learned
during
this
project?
What
about
me?
I'm
a
first
time
mvp
passionate
about
power
platform.
A
I
came
from
app
dev
sharepoint
for
many
years
and
then
power
platform
I
ran
sharepoint
user
group
still
run,
but
don't
tell
anyone,
but
I
have
more
love
towards
power
platform
now
so
but
yeah.
I
live
in
austin
texas,
very
close
to
reza
here
and
on
reza
for
a
long
time.
So
we
should
never
let
reza
go
first
because
see
I
I
we
were
given
15
minutes.
You
took,
I
think,
25
minutes,
so
listen
and
learn.
Reza
will
go
last,
but
fantastic,
wonderful
session
yeah.
A
I
definitely
learned
a
few
things
there
all
right,
so
that
was
a
little
bit
about
me:
lead
power,
power
platform,
practice
at
cyclotron
and
obviously
I
work
from
austin
texas.
So
anyone
in
the
call
interested
to
start
a
group
here,
power
platform,
user
group-
let
me
know,
let's
work
together
and
then
hopefully
things
will
settle
down
here
soon.
So
we
can
go
in
person.
Hang
out,
have
some
beer
and
chat
all
right.
A
So
the
project
was
about
the
problem
statement.
The
client
was
like:
we
received
thousands
of
documents
per
month
over
a
year.
It
could
be
around
15
to
20k
documents
and
then
or
like
how
about
templates.
My
first
thing
was:
is
it
just
one
template
we
can?
I
can
automate
it
using
ai
builder
right,
create
them
all
model,
and
then
you
input
and
you
boom
done
no
and
then
maybe
then
came
in
with
that
form.
Processing.
A
Now
you
can,
you
can
handle
up
to
100
templates,
meaning
you
can
create
in
one
model
you
can
have
100
different
form
templates
and
that
can
handle
in
one
model.
So
I
said
like
how
about
this.
You
may
only
have
100
no
more
than
100,
that's
a
lot
and
like
no.
No,
we
receive
these
warranty
security.
All
these
documents
from
thousands
of
companies
and
there
are
no
format,
there's
no
format.
It
could
be,
it
could
be
way
over
100
format
for
sure
I
was
like,
then
all
right.
So
what
do
we
do?
Then?
A
We
kind
of
started
looking
into
options
and
started
with
ai
builder.
You
even
looked
at
rpa,
robotic
processing,
automation
to
see
if
we
can
automate
using
ai
builder
and
finally,
where
we
went
was
and
then
the
the
the
last
requirement
was.
We
need
to
be
able
to
search
for
some
keywords
in
the
document
because
there
they
were
having.
I
think,
a
couple
of
people
full-time,
sorting
out
this
paper
and
then
categorizing
and
uploading
into
sharepoint,
and
so
what
keywords
they
were
looking
were.
A
Those
are
the
key
point
for
us
where
okay,
we'll
we'll
map,
we
create
a
list
with
all
these
keywords.
Mapping
right!
If
you
have
this
this
keyword,
then
this
is
the
the
the
the
mapping.
Is
this
right?
So
that's
the
design
we
went.
The
first
solution
was
using
ai
builder
and
then
went
over
to
azure
cognitive
services
and
then
computer
vision.
We
landed
a
computer
vision,
the
last
option
and
I'll
share.
Why
the
first
one
ai
builder,
I
already
told
you
why
we
could
not
do
it
because
of
various
templates.
A
The
model
was
we'd,
have
to
create
constantly
these
new
models
and
train
anytime.
New
documents
would
come,
and
so
because
of
the
variation
of
document
that
we're
receiving,
we
could
not
do
that
and
then
so,
as
we
were
kind
of
looking
into
that,
microsoft
came
in.
I
think
earlier
this
year,
where
this
document
automation
so
okay,
so
many
texts
in
here,
but
I
mean
kind
of
summarized-
requires
a
model.
Tribute
trend
that
I
just
now
mentioned
just
now.
A
Right
right
supports
only
up
to
100
templates
in
one
model,
and
then
the
document
automation
was
recently
released.
It
has
some
dependency
on.
You
need
a
dataverse,
you
need
a
app
license.
You
need
ai
model
builder
add-on
and
then
also
you
have
reviewing.
Whoever
are
the
reviewers,
they
need
licenses
and
then
came
in
onedrive
for
a
business,
so
onedrive
per
business
has
an
action
called
convert
file.
The
only
problem
was,
if
you
have,
you
can
fit
this
into
ocr
and
ocr
requires
optical
character.
Recognition
right
requires
optical.
A
A
I
have
all
of
these
references
towards
the
end
I'll
put
it
in,
but
I
will
show
you
my
other
screen
here,
and
this
is
just
a
high
level
flow.
But
I'll
show
you.
I
know
the
call
is
for
powerapps,
but
I'm
showing
you
guys
kind
of
sneaked
in
just
the
power
automate
part
of
it.
But
trust
me.
You
can
create
an
app
to
automate
just
for
the
time
being
here,
because
this
is
only
limited
time.
So
I
I
was
focusing
on
just
the
core
that
I
wanted
to
pass
all
right.
A
In
this
case,
we
were
getting
it
from
ftp
that
from
third
party
we
were
getting
the
files
and
then
we
the
way
we
did
was
one
time
upload
into
the
library
and
then
files
were
scanned
processed
because
you
have
to
extract
metadata
for
each
of
those.
So
I'm
showing
you
here
all
three
options
and
how
we
were
able
to
get
strings
right,
extracted
strings
and
then
obviously,
like
I
like
I
mentioned,
we
went
with
this
option
because
this
was
more
scalable
the
what
this
gave
us,
what
we
needed.
A
We
ran
into
issues
on
this
one
because
of
limitation.
We
were
again
this
one
was
ocr
the
on
one
drive,
because
not
all
documents
are
received
in
an
emails
if
they
are
received
in
pdf,
then
onedrive
for
business
connector
would
only
give
us
the
one
piece
I
did
kind
of
reach
out
to
microsoft.
I
haven't
heard
yet
and
like
hey:
when
are
you
gonna
have
this?
A
So
if
this
becomes
onedrive
for
business
can
convert
all
of
the
pdf
pages
into
image,
then
this
is
becomes
much
much
much
simpler,
whereas
right
now
we're
doing
a
http.
So
I
do
have
cognitive
services
api
that
I
have
to
create
the
service
and
azure
same
thing
with
a
computer
vision
service
in
azure
right.
So
you
definitely
have
to
create
those
endpoint
keys.
Then
only
you
can
use,
but
this
is
very
straightforward.
A
You
just
have
the
key
plug
it
in
establish
the
connection,
and
then
you
start
using
it
I'll
show
you
first,
the
the
first
thing
that
we
tried
here
basically
passed
in,
like
I
said
we
got
the
file
and
then
the
solution
also
that
we
implemented
was
I'm
going
to
show
you
here
settings,
and
so
if
the
status
is
new
or
rerun,
sometimes
it
may
have
failed.
So
they
want
to
rerun.
A
A
I
don't
know
from
one
or
two
users
user
licenses.
You
can
do
an
azure,
very,
very
minimum
from
from
your
comparing
dollars
to
pennies
right,
so
definitely
best
practices
use
that
one.
It
saved
us
a
lot
of
runs,
which
means
a
lot
of
money
for
the
customer
and
then
so,
when
you
convert
you
I
can.
I
could
only
use
for
one
page.
If
it's
a
one
pager,
then
the
convert
one
drive
worked,
so
I'm
extracting
the
results
and
then
basically
it
has
the
the
it
has.
A
A
Seriously,
oh,
there
you
go
and
then
that's
the
the
category
subcategory
and
then
the
keywords
that
we
wanted
to
search.
So
this
is,
you
may
think
this
is
still
manual,
but
at
least
it
gives
you
an
option
too,
and
so,
when
we
give
this
to
the
customer,
they
were
like
oh
yeah.
This
is
they
wanted
to
be
able
to
manage,
so
they
don't
have
to
go
to
flow.
To
say.
A
A
You
can
change
right
as
long
as
the
keywords,
the
the
columns
and
they
are
same,
you
can
use
the
same
flow
again
and
again
for
to
multiple
departments
as
well
all
right
and
then,
if
I
show
you
here,
I'm
not
watching
the
chat,
I'm
just
focusing
on
here,
but
I'll
get
back
to
any
questions.
If
there
are
so,
let
me
see
the
output
so
there's
the
output
right
from
that
file.
That
was
uploaded.
A
I
am
okay,
so
that
didn't
run
and
how
about
on
the
last
one
and
do
until
so.
If
I
come,
if
I
were
to
compare
so
similar
cont
like
see
the
text
were
extracted,
so
the
the
gist
of
this
demo
is
that
there
are
multiple
options.
Understand
your
scope
of
it.
You
have
you
hit
limitation
on
ai
builder,
out
of
the
box.
Connector
extract
text
from
image.
There's
a
limitation
ocr
same
thing
is
a
limitation
because
of
one
drive.
A
If
your
content,
if
your
source
file
is
all
images,
then
ocr
is
your
perfect
place
to
go
very
easy
to
configure.
There's
no
just
one
api
that
you
have
to
register
in
azure
and
then
after
that
you're
set.
But
then,
if
that
nothing
else
works
then
go
to
computers
with
visa
and
read
api,
and
that
solves
the
problem.
B
I
did
see
a
question
that
daniel
had
regarding
the
confidence
score.
So
when
you
do
the
ocr,
you
can
get
back
a
score.
That
tells
you
how
confident
it
was.
That's
right.
You
got
the
answer
right
right,
that's
right!
Yep!
Do
you
have
anything
built
into
your
process
that
if
it's
below
x
score
that
it
sends
an
email
to
somebody
to
look
at
it
or
rejects
it
automatically
or
anything
like
that.
A
So
there
is
the
the
api,
has
different
statuses
and
so
based
on
that
we
are
we're,
logging
everything
into
another
list
and
so
based
on
the
type
of
the
solution.
I
don't
have
it
here,
but
on
our
if
it
failed
like
five
times
in
a
row
within
the
50
like
30
minutes,
then
we
will
alert
sometime
they're,
not
important.
A
A
We
had
to
put
some
delay
because
the
rest
api
was
taking
time
to
process,
it
will
be
on
running,
and
so
there
are
some
logic
to
it.
I
think
on
here
somewhere
I
had
to
put
30
seconds
and
that
we
found
that
by
playing
that
we
were,
we
were
able
to
overcome
their
error
messages,
otherwise
would
just
fail
and
say
it's
still
running
and
then
the
the
next
step
will
start.
So
we
had
to
wait
for
30
seconds
for
that
to
finish
so
yeah.
There
are
we're.
B
A
B
A
So
we
did
not
test
the
volume
that
they
were
receiving
were.
Yes,
they
were
about
15,
to
12,
to
15
15,
to
20
somewhere
around
like
15k
files,
but
it
was
basically
spread
out
there.
They
were
so
we
have.
I
have
not
tested
with
like
hundreds
of
files
we
tested
about
100
couple
of
hundreds,
but
not
in
thousands,
and
since
they
were
spreading
out,
the
files
they
were
receiving
was
they're
receiving
from
different
vendors
and
they're
kind
of
spread
out.