►
From YouTube: June 11, 2018 Audit Committee
Description
Minneapolis Audit Committee Meeting
A
Good
morning,
welcome
to
our
regulation
regularly
scheduled
audit
committee
meeting
for
Monday
June
11th
2018.
My
name
is
Lenny
Palmisano
and
I
share
this
committee
with
me
here
on
the
dais,
our
Park
Commissioner
Chris,
Meyer,
citizen
member
David,
Fischer
citizen
member
Scott,
Neal
and
council
member
Jeremy
Schrader.
Please
let
the
record
reflect
that
we
have
a
quorum
and
we
are
authorized
to
do
the
city's
business
before
you
is
a
proposed
agenda
today.
A
First,
on
this
agenda,
it
is
the
adoption
of
the
agenda
for
people
that
that
are
new
and
it
does
anyone
have
any
questions
or
need
to
add
anything
to
today's
agenda.
With
that
I'll
move
this
agenda
forward,
all
those
in
favor
say
aye
aye
opposed
that
carries.
The
second
item
is
to
accept
the
minutes
of
April
9th
2018.
That
was
our
last
scheduled
meeting.
May
I
have
a
motion
to
approve.
A
Thank
you,
second,
moved
by
Commissioner,
Neal
or
sorry
move
by
member
Neal
seconded
by
member
Fisher,
all
those
in
favor,
please
say:
aye
aye
opposed
that
carries
the
first
item
of
new
business
before
us.
Today
is
the
appointment
of
the
director
of
internal
audit.
Let
me
give
you
a
little
bit
of
background.
It
is
fundamental
to
the
success
of
this
department
to
fill
this
role
with
an
individual
who
understands
the
city,
the
enterprise,
its
goals
and
its
personalities
that
exist
within
it.
A
Therefore,
it's
been
up
to
this
committee
to
recruit
a
leader
capable
of
developing
and
implementing
an
enterprise-wide
vision,
combined
with
a
collaborative
approach,
in
addition
to
the
requisite
audit
experience
here
to
present
a
brief
report
about
the
hiring
process
that
we've
been
conducting
since
January
of
this
year
is
mr.
John
fruit
from
counsel
staff.
Mr.
food.
B
Morning,
madam
chair
members
of
the
committee,
my
name
is
John
fruit
and
I'm.
The
senior
policy
aide
to
the
13th
floor
council
office
at
the
chairs
request
I'm
here,
to
give
a
brief
summary
of
the
designed
hiring
process
for
the
director
of
internal
audit
back
in
February,
a
job
announcement
was
posted
both
internally
on
the
city's
job
board
and
externally
on
industry
websites
detailing
the
qualifications
for
a
new
director.
B
The
human
resources
department,
processed
applications
and
forwarded
a
group
that
met
the
minimum
qualifications
to
the
hiring
team
sheriff
Palmisano
along
with
Susan
committee
members,
Scott
Neal
and
David
Fisher,
along
with
formal
director
of
internal
audit,
will
Tetzel
selected
candidates
to
participate
in
additional
interviews.
This
provided
an
opportunity
to
better
get
to
know
the
candidates
and
help
them
understand
the
audit
committee
and
the
audit
process.
Within
the
city.
We
conducted
phone
interviews
based
on
technical
expertise.
We
interviewed
candidates
in
person
we
brought
in
candidates
from
out
of
state.
B
It
was
during
this
lengthy
process
that
one
candidate
rose
to
the
top
and
through
which
miss
ginger,
Big
B
comes
before
you
today
before
I
conclude
my
remarks.
I
do
want
to
again
thank
Rebecca
Hart
Dell
from
Human
Resources
for
her
input,
advice
and
flexibility
through
this
process.
That
concludes
my
prepared
comments.
I'm
happy
to
stand
for
any
questions
are.
A
A
Travis
and
Carroll
and
Rebecca
hard
l
from
Human
Resources
is
here,
as
both
has
been
instrumental
in
this
process,
all
those
in
favor
of
appointing
ginger
Big
B
to
be
our
city's
next
internal
audit
director,
please
say:
aye
aye
opposed
the
eyes,
have
it
Miz,
Big
B
you're
appointed
to
the
position
of
Director
of
internal
audit
for
the
city
of
Minneapolis.
Congratulations
next
on
the
agenda
is
the
administrative
motion
that
is
to
authorize
the
chair
of
this
committee.
That's
myself
to
execute
documents
necessary
that
finalize
her
hire
all
those
in
favor
of
this
motion.
A
C
Thank
You
chair,
Palmisano,
Thank,
You,
Audit
Committee
members
for
this
opportunity
to
serve
as
the
city's
internal
auditor
I'm
very
excited
to
get
started
in
a
few
weeks
as
I
wind
down
my
current
responsibilities.
A
couple
of
the
first
things
that
I'd
like
to
address
are
looking
at
the
department
policies
and
procedures,
our
department's
policies
and
procedures,
to
see
where
we
can
enhance,
enhance
and
strengthen
our
processes,
perhaps
induce
introduce
a
quality
assurance
and
improvement
program
and,
most
importantly,
look
at
the
city's
risk
environments.
C
Make
sure
we
understand,
understand
the
control
environment
that
surrounds
all
of
the
city's
activities
and
services,
so
that
we
can
have
a
flexible
audit
plan
for
2019
and
going
forward
and
really
focus
our
efforts
on
the
areas
that
are
most
needed
to
ensure
that
the
the
City
Council
and
the
mayor
can
achieve
their
goals
and
objectives.
Also
I'd
like
a
better
understanding
of
the
open
audit
issues
that
we
have
right
now.
What
a
risk
exposure
lies
around
those
output
audit
issues.
C
So
while
the
department's
and
the
internal
audit
Department
are
working
hard
to
shore
up
those
gaps,
make
sure
that
we
know
which
ones
pose
the
highest
risk
to
the
cities,
so
that
at
any
given
time,
we
have
a
clear
understanding
of
our
risk
exposure
and
it's
a
pleasure
to
be
here.
I
really
thank
you
for
this
opportunity
of
may
I
also
say
that
I'd
like
to
thank
my
current
supervisor
and
colleagues
for
all
of
their
encouragement
and
support
as
I
moved
to
public
service.
A
D
A
E
E
This
audit
was
conducted
as
part
of
the
2017
audit
plan.
It
was
a
a
lengthy
process
in
part
due
to
our
commitment
unexpectedly
to
do
the
body,
camera
presentation
and
audit,
and
also
to
deal
with
some
staffing
changes,
which
is
why
it
has
taken
significantly
longer
than
expected
and
I
appreciate
the
patience
of
Minneapolis
and
the
convention
center
staff.
With
the
lengthy
process.
E
C
E
The
scope
of
the
audit
was
from
January
1st
2015
through
February
26th
of
2011,
so
we
covered
all
1516
due
to
the
timing
of
the
audit.
We
also
covered
part
of
2017,
to
the
extent
that
we
looked
for
continuation
of
certain
issues
that
we'd
otherwise
noted
so
30:26
is
that
date
of
the
2017
year-end
report
and
our
approach
for
this
audit
focused
very
heavily
on
interviews
with
various
staff
and
requesting
and
reviewing
supporting
documentation
for
the
various
reports
and
supporting
documentation
to
show
compliance
with
contract
provisions.
A
E
Our
audit
resulted
in
four
findings
and
one
area
of
improvement.
The
first
finding
groups
observations
about
a
number
of
different
performance
indicators
that
are
presented
in
the
quarterly
and
annual
reports,
so
there's
first
finding
it
has
seven
components.
The
second
finding
has
observations
about
Neapolis
conflict
of
interest
policy.
The
third
finding
looks
at
some
contract
required
reporting
elements
that
did
not
appear
in
the
report.
Written
reports
that
the
city
received
the
fourth
finding
comments
on
our
oversight
of
the
contract
and
then
the
there
was
an
area
of
improvement
for
record
retention,
observation.
E
So
the
seven
components
of
the
first
finding
finding
one
one
one
of
the
economic
one
of
the
performance
indicators
is
the
economic
impact
performance
indicator.
It
is
a
estimate
of
how
much
direct
financial
impact
events
that
meet
Minneapolis
has
scheduled
have
brought
to
the
Twin
Cities
metro
area.
E
This
metric
is
developed
using
an
industry
standard
tool
that
Mimi
Atlas
inputs
event
result
data
into
and
spits
out
a
dollar
figure
that
gets
aggregated
for
all
events
over
the
course
of
the
year
and
presented
in
the
reports.
We
observed
two
issues
here.
The
first
issue
was
the
way
certain
events
were
tracked
in
MeetMe.
Appaloosas
tracking
system
resulted
in
double
counting,
so
an
event
that
had
overflow
rooms
that
were
identified
after
the
initial
booking
would
need
to
get
a
new
event
type
created
so
that
hotels
could
be
found.
E
But
unless
the
individuals
creating
those
extra
events
updated
all
the
data.
A
lot
of
the
initial
event
data
would
just
Auto
populate
down,
and
that
would
result
in
a
overflow
room,
overflow
booking
for
a
hundred
rooms
to
have
the
same
attendance
as
the
original
event
of
ten
thousand
rooms
effectively
significantly
overstating
the
impact
of
certain
events
by
double
counting
them.
There's
another
similar
issue
where
individual
contracts
for
a
series
of
events
would
get
entered
in
once
and
then
duplicated
for
every
quarterly
or
monthly
meeting,
also
counting
certain
impacts.
Four
times
or
twelve
times.
E
We
observe
this
issue
in
our
sample
and
then
we
reviewed
the
total
data
set
to
estimate
the
economic
impact
adjustment
and
saw
that
in
2015
we
estimate
an
overstatement
of
a
hundred
and
twenty
million
versus
the
four
hundred
and
thirty
million
impact
reported
an
impact
of
25
million
in
2016
versus
the
four
hundred
and
sixty
million
reported
and
an
impact
of
50
million
overstatement
in
2017
versus
the
six
hundred
sixty
million
dollar
impact
reported.
So
the
double
counting
did
have
a
measurable
impact
on
the
reported
numbers.
E
The
secondary
issue
is
the
figures
used
to
estimate.
The
impact
was
were
based
on
pre
event,
attendance
for
certain
events,
the
Neapolis
Convention
Center
does
a
headcount
of
the
actual
attendees
and
that
results
in
a
more
accurate
estimate
of
attendance
than
you
would
have
two
or
three
months
prior
to
the
event.
E
That
meant
that,
when
calculating
economic
impact,
in
some
cases
much
more
accurate
event,
attendance
data
was
available
but
was
not
used.
The
overall
impact
was
own
was
fairly
minor
because
some
events
were
underestimated.
Some
were
overestimated,
but
individual
event
swings
could
be
very
high
and
considering
that
the
better
estimate
data
was
readily
available,
it
seems
like
an
easy
opportunity
to
reduce
the
risk
of
misstatement
by
using
the
most
up-to-date
attendance
figures
and
I.
Do
want
to
say
one
more
thing
here,
and
that
is
this:
isn't
one
of
the
key
performance
indicators.
E
So
this
is
not
one
of
the
four
measures
that
were
determined
by
the
city
and
Minneapolis
to
be
the
most
important,
but
it
is
probably
one
of
the
more
visible
figures
and
the
most
commonly
discussed
figures
for
destination
marketing
organizations.
So
making
sure
that
we
have
accurate,
clear
reporting
on
here
is
fairly
important.
E
Second
issue
identified
was
with
the
private
revenue
key
performance
indicator.
We
did
not
see
any
issues
with
the
calculation
methodology,
but
we,
when
discussing
this
indicator,
noted
that
the
goal
of
including
this
as
an
indicator
was
not
to
just
measure
how
much
private
revenue
was
raised,
but
to
measure
how
well
meet
Minneapolis
leveraged
City
provided
funds
with
the
idea
of
the
city
pays
in
one
dollar
rather
than
taking
that
in
hosting
an
event.
We
turn
it
into
more
than
one
dollar,
because
the
private
revenue
indicator
in
the
report
in
the
contract
was
just
a
pure.
D
E
Finding
1/3
is
on
the
future
leisure
room
nights.
Key
performance
indicator.
The
this
is
a
key
performance
indicator,
so
it's
tied
to
incentive
funding
at
the
end
of
the
year.
The
contract
required
that
a
third
party
audit
confirming
these
figures
be
conducted,
and
this
was
not
something
that
was
being
done.
E
Adjusting
the
contract
might
be
reasonable,
because
a
lot
of
these
figures
come
in
from
a
third
party.
Anyway,
they
come
in
from
event
planners
or
hotels
that
state
how
much
leisure
room
nights
were
booked
as
a
result
of
MeetMe
outlets
activities.
So
there's
already
some
third
party
assurance
there.
So
it's
more
of
a
contract
compliance
issue
with
one
of
these
key
indicators,
that's
tight
incentive
funding
that
we
observed
not
being
followed.
E
E
During
a
given
period,
we
observed
that
there
was
a
significant
revision
in
how,
in
some
of
the
Convention
Center
numbers
after
it,
provided
them
to
Minneapolis,
and
that
this
revision
wasn't
communicated
to
meet
me
appleís
until
we
started
requesting
information
for
the
audit
so
where
we're
seeing
numbers
that
are
inaccurate,
not
through
Minneapolis
fault
but
through
their
source
and
just
a
non
communication
of
the
end
result
and
of
the
issue
noted.
As
part
of
this,
we
also
didn't
see
any
specific
measures
saying
at
what
point.
E
Our
revision
should
trigger
a
restatement
of
a
prior
quarterly
report.
So
our
recommendations
here
are
to
put
in
place
some
sort
of
process
whereby,
if
the
Convention
Center
revises
figures
that
mimi
atlas
relies
on,
there
should
be
a
setup
process
to
communicate
those
revisions
and
there
should
be
a
set
expectation
of
if
the
performance
measure
reported
needs
to
be
adjusted
by
some
amount.
That
caused
a
reiations
of
the
report,
whether
that
be
5%,
10%
2%.
E
The
next
to
findings
are
similarly
related
ones
engaged
digital
users
indicator
and
the
other
is
visual,
visitor
requests
for
enter
information
indicator.
In
both
of
these
cases,
we
observed
that
there
is
a
change
to
how
a
measure
was
being
calculated
that
significantly
changed.
The
number
that
was
reported,
the
goals
being
set
for
meet
Minneapolis
did
seem
to
take
these
into
account.
The
issue
we
took
was
the
disclosure
of
the
change
in
methodology.
There
wasn't
anything
other
than
the
communication.
E
The
verbal
communications
between
me
apples
in
the
city
for
most
of
these
changes
and
the
actual
quarterly
reports
didn't
state.
We've
changed
how
this
is
being
presented
and
they
didn't
provide
a
comparable
adjustment
to
prior
figures
to
really
be
able
to
have
a
user
of
that
report
understand
why
there's
a
change
and
how
much
of
a
change
it
is
and
to
be
able
to
compare
year-to-year,
and
our
final
observations
on
the
performance
indicators
was
the
way
the
return
on
investment
performance
indicator
was
developed.
E
E
The
operating
budget
was
established
and
the
economic
impact
goal
was
established
independently
of
this
return
on
investment
indicator,
which
meant
that
to
achieve
the
return
on
investment
goal
that
was
set
before
them,
MeetMe
Neapolis
would
have
had
to
either
way
over
perform
their
economic
impact
goal
or
way
under
spend
their
budget
because,
while
they're
interrelated,
they
were
considered
separately.
So
the
way
this
goal
has
been
established
was
essentially
either
setting
up
meeting
apples
to
fail
or
requiring
them
to
over
perform
in
some
other
way,
and
that
seemed
incongruent.
E
Our
second
finding
for
this
report
is
on
the
conflict
of
interest
policy.
Meet
minneapolis
has
a
conflict
of
interest
policy
which
is
reviewed
by
their
board.
This
policy
includes
a
gift
limit
of
$50,
beyond
which
a
clear
conflict
of
interest
would
be
expected,
and
it
required
annual
conflict
of
interest.
Disclosures
from
board
members
officers
and
employees.
E
Meeting
Atlas
didn't
fully
implement
these
policies.
They
collected
annual
conflict
of
interest
disclosures
from
the
board
members,
but
relied
on
information
provided
by
employees
at
new
hire
orientation
and
relied
on
these
employees
to
self
disclose
any
changes,
rather
than
collecting
the
required
annual
disclosures.
E
The
other
in
congruence
was
the
employee
handbook
had
a
higher
gift
limit
of
$100,
beyond
which
conflict
of
interest
would
be
expected
from
certain
parties
versus
the
lower
$50
limit
of
the
conflict
of
interest
policy,
and
the
additional
comments
I'd
have
here
is:
there
is
not
necessarily
the
idea
that
$100
or
$50
is
is
appropriate
and,
of
course,
in
annual
conflict
of
interest.
Disclosure
is
only
as
good
as
the
person
willing
to
disclose
the
information,
but
this
speaks
to
corporate
compliance
culture
and
communicating
that
and
having.
E
D
Vasily
for
those
of
us
that
work
in
in
city
government
or
in
state
government,
local
government
in
general
of
$50
and
$100
gift
limits
are
quite
a
bit
different
than
what
we're
used
to
is
there.
Is
there
a
statutory
gift
limit
for
meet
Minneapolis
that
they
need
to
comply
with
to
the
best
of
your
knowledge?
Madam.
E
E
A
Just
to
reiterate
what
you
just
said,
because
there
are
city
council
members
that
sit
on
this
board,
what
you're
saying
is
that
the
City
Council
members
are
held
to
the
city's
gift
policy
and
limits
above
it.
They
go
well
I'm
well,
above
and
beyond
the
conflict
of
interest
policy
of
meet
Minneapolis
correct
that.
E
A
F
That
being
said,
I'd
have
to
review,
meet
Minneapolis
contract,
but
generally
contracts
have
within
them
when
the
city
issues
a
contract,
a
clause
that
requires
the
contractor
to
revise
by
the
terms
of
the
city's
ethics
code.
So
there
would
be
an
argument
made
that
meet
Minneapolis
to
the
extent
that
it's
being
funded
if
it
were
being
funded
entirely
by
City
dollars
that
the
city's
gift
ban
would
apply
if
it's
not
being
funded
entirely
by
City
dollars.
F
It
gets
a
little
more
complicated
than
that
and
then
and
many
of
our
other
vendors
have
gift
policies
that
are
different
than
the
city
policies,
but
for
most
definitely
all
of
the
city.
Employees
and
the
elected
officials
who
serve
on
the
meet
Minneapolis
board
would
need
to
abide
by
the
city's
gift
ban
as
set
forth
in
ordinance.
Fifteen
point
five:
zero.
E
The
third
finding
in
the
report
was
related
to
contract
required
reporting
the
contract
with
meet
Minneapolis
lists,
certain
elements
that
are
to
be
included
with
quarterly
reports
with
urine
reports
and
with
budge
plans.
We
didn't
observe
some
of
these
elements
when
we
were
provided
copies
of
these
plans
and
these
reports
specifically
the
budget
plan
worker
contract,
is
to
include
narratives
addressing
outcomes
provided
by
various
funding
sources.
The
budget
plans
we
received
didn't
have
narratives
the
contract
also
lists
that
quarterly
reports
are
to
include
narratives
of
prior
quarter
activities
and
in
highlighting
successes
and
challenges.
E
We
did
see
a
few
narratives,
but
they
seem
to
talk
about
very
specific
initiatives
and
did
not
provide
a
summary
of
quarterly
activities,
highlighting
successes
and
challenges.
And,
finally,
the
year-end
report
is
to
include
documentation
needed
to
explain
material
deviations
from
the
approved
budget
plan
and
sales
and
marketing
plan.
The
copies
of
quarterly
reports
we
receive
or
year-end
reports
apologies
received,
didn't
have
supporting
documentation
based
on
our
discussions.
These
elements
were
provided
verbally
other
than
supporting
documentation
which
is
difficult
to
provide
verbally
and
when
we
communicated
with
our
city
staff.
C
E
The
fourth
finding
lets
you
sitting:
Neapolis,
is
contract
oversight
and
specifically
with
the
key
performance
indicators
which
are
tied
to
incentive
funding.
The
contract
requires
that
the
city
review
supporting
documentation
for
the
completion
of
these
indicators
for
the
accuracy
and
applicability
of
that
documentation.
When
we
request
supporting
documentation,
we
were
informed
that
this
process
is
done
verbally.
So,
aside
from
the
annual
report,
provided
that
lists
the
performance
indicators
and
their
achievement,
the
city
was
not
doing
documentation
review
and
was
not
reviewing
the
documentation
for
accuracy
and
applicability
again.
E
A
So
I
just
want
to
point
out
here
as
you
change
this,
as
you
change
the
slide
to
go
on
that.
Well,
it
says
on
the
slide
that
meet
Minneapolis
agrees
with
the
finding.
There
has
been
some
more
back
and
forth
here
that
Carol
is
mentioning
about
how
they've
proactively
work
to
remediate
or
to
take
some
action
already.
So
that's
that's
good.
Go
ahead!
I'm.
E
The
final
area
to
discuss
here
is
record
retention
during
field
work.
A
portion
of
the
documents
we
requested
were
unavailable
because
Mimi
Apple
has
changed
a
vendor
and
did
not
retain
the
original
documentation
for
the
contract
required
six
years.
This
was
initially
listed
as
a
finding.
However,
as
part
of
the
draft
response
process,
Mimi
appleís
was
able
to
contact
the
vendor
and
get
a
commitment
to
obtain
this
documentation
that
was
not
available
to
us
during
the
course
of
our
review,
because
that
was
a
foundation
of
this
issue.
E
Was
me
appleís
not
having
access
to
these
records
or
the
records
of
time
period,
because
that
was
addressed?
It
didn't
seem
appropriate
to
continue
holding
that
finding.
However,
we
do.
We
did
still
recommend
that
there'd
be
an
internal
review
of
how
record
retention
is
managed,
especially
when
vendor
changes
to
ensure
that
this
doesn't
happen
again.
I.
E
Have
a
final
slide
here
to
discuss
follow-up?
This
was
a
third
party
audit,
so
our
purview
is
limited
to
whether
or
not
meeting
APIs
is
complying
with
the
contract
provisions.
As
a
result,
the
focus
of
our
follow-up
will
be
to
work
with
the
convention
center
to
determine
whether
or
not
these
issues
are
being
addressed.
If
the
convention
center
deems
that
it
is
appropriate
for
us
to
communicate
with
meet
Minneapolis
directly
on
issue,
we
would
do
that,
but
our
expectation
will
be
to
work
primarily
with
the
convention
center
rather
than
Minneapolis,
to
resolve
the
identified
issues.
E
I
also
do
want
to
note
that
the
current
contract
with
me
Minneapolis
is
up
for
renewal,
I
believe
in
2020.
So
for
some
issues
where
the
appropriate
resolution
might
be
adjusting
the
terms
of
the
contract,
it
is
likely
that
makes
more
sense
to
do
as
part
of
a
2020
conduct
revision,
rather
than
trying
to
put
together
an
amendment
now
which
will
be
getting
reworked
in
several
months
anyway.
A
Thank
you.
Questions
from
my
colleagues.
I
did
have
some
that
I
think
we
covered
in
agenda-setting.
I
can't
remember
them
right
now,
so
I
am
I
appreciate.
This
audit
I
think
that
those
here
presents
also
appreciate
this
audit
and
a
thorough
look
it's
to
how
we
can
work
better
with
our
Convention
Center
anything
else
from
anybody.
G
Thank
you,
I,
don't
sit
up
here
or
not.
My
my
reading
of
the
of
the
report
is
that
Nia
has
agreed
with
all
the
findings
that
you've
made
and
are
acting
on
them
as
we
speak,
and
that
there's
nothing
particularly
significant
than
the
findings
I
found
that
were
troubling.
It
seems
like
it's
cleaning
up
a
little
bit,
rather
than
actually
finding
anything
of
significance
of
that
be
an
accurate
statement.
G
E
Chair
mr.
Fisher
I
do
want
to
note
that
the
response
was
a
joint
response
from
the
Neapolis
Convention
Center
and
Mimi
appleís.
So
it
wasn't
just
meeting
me
apples
providing
response,
but
also
the
convention
center
I
think
that's
important
to
note,
because
they
are
an
integral
part
of
the
contract
oversight
process
and
I.
Think
that
your
statement
is
is
very
accurate.
E
The
issues
are
more
cleanup
and
an
adjustment
and
making
sure
that,
if
we're
putting
something
in
the
contract,
we
make
sure
it
happens
or
we
don't
put
it
in
the
contract
in
the
first
place
it's
in
there.
If
it's
important
enough
to
have
been
in
the
contract,
we
should
make
sure
that
we're
enforcing
those
provisions
there's
also
somewhat
of
a
learning
experience
with
this
contract.
Specifically,
this
was
the
first
one
with
Mimi
appleís,
where
specific
performance
indicators
and
numerical
goals
were
even
set
up
and
established,
I
think
from
prior
conversations.
E
It
might
be
one
of
the
first
types
of
contracts
like
this
for
a
destination
marketing
nation
anywhere.
So
that's
part
of
a
learning
process
of
figuring
out.
How
do
we
actually
measure
this?
Is
this
a
good
number
or
not?
How
do
we
approach
the
methodology
that
maybe,
if
we
document
in
the
contract
and
then
realize
it's
wrong,
we
can't
now
change
it
without
amending
a
contract.
There's
different
approaches
to
having
those
goal-setting
processes
that
are
better
documented,
so
I
think
with
any
contract
revision
in
2020.
E
A
Mr.
basilia
I
do
want
to
note
that,
on
page
seven
of
your
report
you
mentioned
this,
it
does
talk
about
the
estimated
overstatement
of
economic
impact
over
these
three
years
being
over
200
million
dollars.
I
think
that's
significant,
but
you
also
mention
that
this
is
a
figure
that
is
used
in
industry
and
less
important
to
the
actual
kind
of
day-to-day
work
or
work
between
the
city
and
the
convention
center.
E
Madam
chair,
so
this
number
is
a
common
measure
of
what
what
the
effectiveness
of
marketing
is,
because
it's
really
hard
to
think
of
that
you
hold
an
event
and
you
spend
a
hundred
hours
planning
it
and
you
host
it,
and
maybe
you
can
get
rent
and
some
other
items.
But
the
real
impact
of
the
city
is
not
just
that
event.
It's
also
people
coming
in
and
staying
at
our
rooms
eating
food,
doing
site
doing
tours.
E
So
that's
what
the
this
indicators
supposed
to
measure
there's
this
indicator,
which
is
direct
economic
impact,
but
there's
also
a
larger,
broader
indicator,
which
is
also
commonly
used
and
feels
less
reliable,
which
is
overall
in
economic
impact,
not
just
direct,
so
that
would
be
measuring
well,
if
it
weren't
for
this
person.
Staying
in
this
hotel,
the
hotel
staff
wouldn't
have
jobs
and
wouldn't
have
money
and
cascades
down
to
several
levels.
So
we're
not
measuring
that
which
is
even
harder
to
try
and
get
your
arms
around
and
the
city
in
developing
this
contract.
E
With
me,
me,
appleís
elected,
not
to
use
this
as
one
of
the
key
performance
indicators
I
think
in
part,
because
it's
it's
an
estimate
and
without
going
to
an
event
and
interviewing
everyone
and
finding
out
where
they
stayed
and
where
they
went
to
eat
and
are
they
from
out
of
town
or
in
town.
It's
really
hard
to
get
an
actual
measure,
which
makes
it
less
of
a
useful
measure
to
determine
incentive
funding
that
the
impact
overstatement
is
two
hundred
million
dollars.
E
Almost
the
number
isn't
necessarily
the
numbers
large,
which
makes
it
more
impactful,
but
it's
not
the
specific
number.
It's
that
the
reporting
was
inaccurate
and
miscounted
certain
events
in
a
systemic
way
over
the
course
of
several
years.
So
it's
more
of
a
changing
that
methodology
or
adding
an
additional
control
process
to
review
these
figures
before
they're
putting
in
put
into
a
report.
That's
really
important
one
more
thought
on
the
usefulness
of
this
indicator
while
we're
not
necessarily
using
it
to
decide
whether
or
not
there's
additional
incentive
funding
for
Mimi
appleís.
E
That
doesn't
mean
that
a
future
contract
wouldn't
decide.
This
is
more
important
and
I'd
also
like
to
note
for
one
other
contract,
an
estimated
economic
impact
was
used
as
part
of
discussing
the
Target
Center
curbing
of
renovations
and
an
economic
impact.
Analysis
study
over
the
course
of
the
life
of
the
renovation
showed
that
it
would
bring
in
enough
money
that
it
was
worth
for
the
city
to
invest
tens
of
millions
of
dollars
into
a
renovation.
E
So
while
here
it
might
not
be
the
most
valuable
thing
in
its
current
form,
it
has
been
used
in
other
places,
to
support
significant
investments
as
being
economically
viable,
so
accurate
reporting,
while
not
being
used
right
now,
could
be
being
used
at
a
later
point
in
time
to
justify
significant
expenditures.
That's.
A
A
A
E
Thank
you,
madam
chair.
So
while
this
is
our
regular
update,
we're
doing
it
in
a
more
specifically
called
out
way.
The
annual
audit
plan
is
established
every
year
and
approved
by
the
audit
committee.
It
can
be
amended
with
approval
of
the
audit
committee
which,
whenever
there's
been
changes
to
the
plan,
have
been
incorporated
in
part
of
their
updates,
but
it
seemed
appropriate
to
specifically
call
it
out
to
the
same
level
that
the
initial
plan
was
so
that
there
is
clear
community.
E
E
E
Then
we
initially
understood
expected
and
understood
them
to
be
doing,
and
because
there
are
a
lot
of
Records
and
doing
a
focused
review,
as
we
were
initially
planning
did
not
seem
appropriate
to
really
effect
the
chain,
the
approach
to
records
for
the
department,
it
might
identify
specific
issues,
but
we
don't
have
time
to
go
to
everyone's
office
and
for
the
important
result
here,
which
is
being
ready
to
move
into
a
completing
office.
Building.
E
Everyone
needs
to
have
their
records
clean.
So
it's
going
to
be
more
of
a
department,
level
processes
approach
and
making
sure
that
the
department
has
the
authority
and
the
process
to
take
those
recommendations
from
clerk
records
staff
which
are
really
the
experts
here
and
to
implement
them
so
that
when
they
are
relocating
to
the
new
building
in
2020,
we're
not
scrambling
and
shoving
things
into
expensive,
far
away
long-term
storage
that
we
need
every
day
and
I
did
expand
this
to
overlap.
E
E
The
other
item
that
was
presented
last
time,
but
is
here
called
out
more
clearly,
is
an
audit
of
the
contract
with
AEG,
and
the
objectives
will
be
to
very
similarly
to
meet
Minneapolis.
Look
at
how
well
a
DG
is
complying
with
specific
provisions
of
its
contract.
Look
at
the
accuracy
of
reporting
and
look
at
how
well
the
city's
oversight
for
this
contract
is
functioning.
E
E
E
Several
months
ago,
he
reached
out
to
saying
that
the
park
board
is
rolling
out
body-worn
cameras
for
its
police
officers
and
requested
that
we
review
that
rollout
process
look
at
their
policies,
procedures
and
look
at
their
compliance
with
the
state
statute.
We're
not
looking
at
it
we're
not
conducting
a
biannual
compliance
audit
for
that
statute.
E
What
we
are
going
to
look
at
what
has
been
put
in
place
and
use
our
knowledge
that
we
have
gleaned
from
working
with
the
city's
police
department
to
hopefully
identify
some
potential
areas
of
improvement
for
the
park
board
before
they
have
to
comply
with
that
biennial
state
audit
statute.
Think
about
a
year
and
a
half.
E
We
noted
last
time
that
the
police
records
management
system
based
on
the
state
it
is
in
and
how
far
along
implementation
was,
rather
than
consulting
on
developing
appropriate
security
policies
and
procedures
and
controls
during
implementation,
because
they
were
so
far
along.
We
are
waiting
a
short
period
of
time
until
the
application
is
actually
done
so
that
we
can
review
how
well
it
went
and
the
current
configurations
and
because
we're
reviewing
something
that
is
already
in
place
rather
than
assisting
with
developing
something.
For
later.
E
It
is
much
more
appropriate
to
conduct
an
audit
with
actionable
follow-up
recommendations
rather
than
consultation
with
suggestions
and
also,
as
previously,
the
third-party
risk
management
process.
Consultation
has
been
put
on
hold
because
there's
not
been
a
completed
project
for
us
to
look
at
to
see
whether
or
not
I
t's
implemented
their
new
process
and
until
they've
got
that,
there's
not
really
anything
to
do
for
that
project.
A
Thank
you.
One
clarification
mr.
vasilia
on
the
consult
with
NPR,
be
in
their
body,
worn
camera
program.
Is
it
just
about
compliance
with
state
statute,
or
does
it
also
involve
some
of
their
policies
and
procedures
and
how
that
might
be
more
consistent
with
Minneapolis,
Police,
Department's,
evolving
policies
and
procedures?
Madam.
E
E
Think
the
Park
Board
current
policy
is
heavily
based
on
the
prior
MPD
policy,
so
maybe
highlighting
why
some
of
the
changes
occur
to
the
policy
and
areas
for
consideration
will
definitely
be
on
the
table
and
as
we
meet
with
chief
AHA
toe
and
talk
about
any
specific
concerns
you
might
have,
we
can
see
if
we
can
address
those
within
the
plan.
Scope
of
the
review
here.
G
E
Madam
chair,
mr.
Fischer,
they
have
developed
policies
and
procedures
and
put
them
in
place.
However,
the
policies
and
procedures
are
here's
how
we
will
approach
a
third
party
law,
a
third
party
contract
from
risk
management's
perspective,
and
while
things
are
in
place,
the
goal
here
is
to
verify
it
is
actually
happening
and
advise
where
maybe
what's
been
put
in
place
could
be
improved
and
that's
what
we're
waiting
for.
So
everything
has
been
established,
but
there's
nothing
to
execute
that
new
process
on.
So
we
don't
necessarily
know
if
it
works
until
they're
trying
to
use
it.
A
Thank
you.
I
also
want
to
point
out
that
last
time
we
did
that
IT
audit,
that
involved
the
business
impact
analysis
for
disaster
recovery
purposes
and
from
this
committee
we
did
forward
on
to
another
committee
to
mr.
Bursch
Ami's,
councilmember,
Sammy's,
Ways
and
Means
Committee,
and
and
we
did,
we
did
do
a
staff
direction
from
Council
to
ask
them
to
to
put
out
an
RFP
in
fairly
short
order
for
that
business
impact
analysis,
so
that
is
moving
forward
and
is
moved
beyond
this
committee
from
prior
work
that
we
did
this
year
with
IT.
A
So
thank
you
for
this
I'm
gonna
make
a
motion
to
approve
this
amendment
amended
internal
audit
department,
annual
audit
plan,
all
those
in
favor,
please
say
aye
aye
opposed
that
carries
they
also
are
you
are
you
completed
with
this
update?
Is
that
there
you
go
I?
Will
you
have
this
presentation
upcoming?
Is
that
what
you're
saying
yes?
Okay?
So
with
that?
We
will
continue
on
to
the
auditors,
update
mr.
Vasily
EV.
E
Thank
you,
madam
chair
honor,
update
a
brief
summary
of
the
I
plan
in
progress
comments
on
prior
findings
and
a
couple
of
items
for
department
level
information.
So,
with
the
audit
plan
progress,
this
hasn't
been
updated
for
the
just
approved
amendments,
but
will
be
next
time,
but
for
the
audits
we
have
completed
the
third
party
out
of
Minneapolis
and
diversion
solutions
for
the
year
which
were
carriers
from
last
year,
we're
in
the
middle
of
field,
work
for
payroll
and
are
waiting
for
a
few
more
final
request
items
before
we
can
start
wrapping
that
up.
E
E
The
other
two
items
on
here
are
going
to
be
happening
once
we've
completed
some
of
these
other
areas
and
are
able
to
focus
on
additional
projects
for
the
consultations.
We
have
identified
that
review
area,
as
mentioned
before
I
think
I
do
want
to
note
that
for
Fleet
Services
there's
been
some
delay
in
communicating
with
them,
because
they
were
focusing
a
lot
on
establishing
their
budget
for
2019,
and
that
has
taken
a
lot
of
their
attention.
E
E
For
the
NCR
process
review,
we
do
have
three
points
of
feedback
that
we've
defined
with
NCR
for
the
consult
and
have
completed
one
of
them
and
are
targeting
the
next
point
of
feedback
later
in
the
summer,
early
fall
and
a
final
point
of
feedback
with
a
resulting
final
report
by
end
of
the
year
as
to
our
work
and
for
the
biscuity
disaster
recovery.
We're
doing
essentially
a
survey
with
gathering
specific
information
on
current
systems.
That
departments
know
about.
So
that
can
be
a
reference
point
both
for
any
business.
E
And
then
for
the
IT
work
to
progress,
we
did
not
have
a
completed
IT
audit
to
present
for
this
meeting,
but
the
consultants
do
believe
that
both
of
the
access
and
critical
content
systems,
continuity
and
access
for
the
critical
traffic
control
and
the
A+
water
systems
should
be
available
for
the
August
committee
meeting.
So
their
reviews
is
progressing
well
and
Public.
Works
has
been
communicative
about
providing
information
for
that.
E
Findings
in
response
to
a
staff
direction
from
this
committee,
we
check
in
with
individuals
responsible
for
addressing
state
auditors
findings
and
provide
information
as
to
whether
or
not
they
are
completed
to
this
committee.
We
are
still
in
progress
on
the
federal
ward
program,
procurement
policies
and
procedures
for
the
city.
Our
city's
internal
controls
manager
is
coordinating
with
the
state
auditor
to
make
sure
that
we
resolve
these
in
a
way
that
fixes
it
and
is
acceptable
to
the
external
auditors
and
the
federal
grants.
Cash
management
reporting
requirements
are
also
still
in
progress
at
the
board.
E
E
The
middle
column
here
represents
where
we
were
effectively
at
the
beginning
of
this
year
at
the
first
audit
committee
meeting,
and
we
where
we
are
now
along
with
target
completion,
dates
there
as-
and
we
haven't
done-
that
much
in
terms
of
updating
this
from
the
prior
committee
meeting,
but
we
have,
on
the
back
end,
been
able
to
go
through
and
significantly
progress.
How
many
of
the
completed
items
we
were
able
to
review
and
say?
Yes,
they
are
complete.
E
E
The
thing
that
had
been
on
here
before
but
is
no
longer
on
here-
is
a
vacant
director
position
which
has
been
filled,
which
is
good
news
and
will
help
us
get
a
lot
of
our
work
progressing
faster
and
then
we
did
also
with
a
lot
of
help
from
IT
finalize
a
SharePoint
site,
which
is
now
that
I
look
at
it
completely
illegible
on
here.
But
it
shows
our
work
in
progress,
the
individual
assignments
to
individual
staff
and
then
for
each
project.
A
Thank
you.
The
state
findings
look
like
look
quite
well
work
done
and
remediated
some
of
those
have
set
out
there
for
many
many
years
before
this
audit
team
was
coalesced
and
mr.
I
know
mr.
ruff
has
been
working
on
it
and
his
team
and
and
I
appreciate
your
tracking.
It
I
think
that
looks
like
it's
getting
into
good
shape.
A
It's
a
small
department
and
I
know
it
takes
all
of
you
and
thank
all
of
you
for
continuing
to
work
as
a
team
and
I
think
that
you've
really
kind
of
coalesced
and
there's
a
really
good
culture
going
on
right
now,
an
internal
audit
department,
so
I
appreciate
your
efforts
and
in
being
good
team
players
and
I,
especially
mr.
Vasilyev,
appreciate
all
the
work
that
you've
done.
Helped
this
and
I
know
that
you've
grown
a
lot
through
this
time
as
well.