►
From YouTube: What is an Election Audit?
Description
Learn more about election audits and the options available to state policymakers and officials to ensure accuracy and ease.
A
When
you
hear
the
word
audit,
does
your
heart
start
racing,
maybe
you're
thinking
of
a
tax
audit,
which
is
good
for
the
nation
but
scary
for
the
individual
taxpayer,
when
jennifer,
morrell
and
matt
masterson
hear
the
word
audit,
their
hearts
might
race
too,
but
with
excitement
and
that's
because
jennifer
and
matt
are
election
administration.
Experts
are
the
highest
caliber
and
one
of
the
most
exciting
things
in
their
world
is
post-election
audits.
A
So
fortunately
they're
with
us
today
on
this
webinar,
I'm
andy
zak,
a
policy
specialist
in
ncsl's
elections
and
redistricting
program,
and
our
goal
today
is
to
make
these
kinds
of
audits
exciting
for
you,
too,
or,
if
not
exciting,
to
at
least
demystify
audits
and
we're
going
to
do
that
by
taking
a
high
level
look
and
then
going
into
a
bit
of
detail,
and
we
want
your
questions
along
the
way.
So
you
can
go
ahead
and
put
those
in
the
zoom
chat
as
they
come
to
you
and
we'll
address
them
at
the
end.
A
Otherwise
we
ask
that
audience.
Members
keep
themselves
on
mute,
so
matt,
let's
get
started
with
you.
You've
had
a
number
of
roles
in
elections,
including
serving
as
a
republican
member
and
chairperson
of
the
u.s
election
assistance
commission,
as
the
state
election
director
in
ohio
and
as
a
senior
cyber
security
advisor
at
cisa
and
for
those
who
don't
know
their
acronyms.
That's
the
cyber
security
and
infrastructure
security
agency.
A
A
B
Yeah
mandy
thank
you
and
thank
you
to
ncsl
for
the
opportunity
to
talk
to
the
folks
on
this
webinar.
I
am
excited
to
talk
about
audits,
which
may
say
more
about
me
than
anything
else,
but
this
is
an
area
of
election
administration
that
that,
quite
frankly,
we've
seen
grow
and
improve
upon
at
rapid
rates,
perhaps
faster
than
any
other
area
of
election
administration
than
the
time
that
I've
been
in
elections.
When
I
started
in
elections,
which
was
in
the
2005-2006
range
audits,
were
done
in
some
jurisdictions.
B
Some
states
had
existing
requirements
around
them,
but
they
weren't
widely
used
and
there
wasn't
they
weren't,
efficient
or
largely
effective.
They
were
something
that
many
states
did.
Some
states
well,
but
most
didn't
understand
the
purpose
they
said
you
know
basically
well.
We
have
recounts
if
it's
close,
we'll
recount
the
election.
It's
fine,
but
in
fact
auditing
our
election
in
the
variety
of
different
types
of
audits
is
incredibly
important
to
the
elections.
Audits
can
serve
multiple
purposes
most
of
the
time
when
we
talk
about
election
audits.
B
What
we're
talking
about
is
post-election
audits.
The
purpose
of
a
post-election
audit
is
to
verify
the
accuracy
of
the
vote,
tally
result
of
the
election.
So
in
some
states
you
see
that
they'll
sample
a
certain
percentage
of
ballots,
two
percent
five
percent,
whatever
the
case
may
be,
and
in
other
states
they
have
varying
ways
of
doing
this.
B
The
purpose
is
to
assess
the
accuracy
of
the
the
voting
equipment
that
the
tally
was
correct,
that
the
aggregation
of
vote
totals
reached
an
accurate
result,
but
there's
many
different
kinds
of
audits
that
we
see
across
elections
beyond
just
the
post-election
audit.
You
can
have
process
audits
where
you
go
through
and
look
at
your
controls,
your
processes,
your
procedures
and
evaluate
both
the
effectiveness,
the
efficiency
in
areas
for
improvement.
B
You
can
have
legal
audits
where
you
apply
and
look
at
the
laws
in
place
to
make
sure
that
you're
implementing
laws
as
intended
in
that,
in
fact,
your
election
office
is
in
legal
compliance
with
all
applicable
statutes
and
laws.
You
can
have
access
audits,
where
states
counties
will
literally
go
out
to
every
polling
place
and
assess.
Do
we
meet
the
requirements
of
the
ada
state
law,
but
also
best
practice
to
ensure
enfranchisement
of
voters
with
disabilities
and
just
voters
generally
around
polling
places?
B
You
can
have
what's
called
configuration
audits.
We
conducted
one
of
those
in
ohio
where
we
literally
went
out
and
looked
at
every
voting
system
in
the
state
to
ensure
that
it
was
still
in
the
certified
configuration
that
is.
It
was
set
up
loaded
with
the
correct
software
firmware
and
hardware
for
the
certified
configuration
that
the
state
said
it
had
to
be
in
and
ensuring
the
accuracy
of
that
you
could
also
have
which
talked
about
pretty
often
right.
B
Now,
a
forensic
audit
and
a
forensic
audit
has
a
very
specific
definition.
Forensic
audit
means
looking
at
the
overall
evidence
and
information,
including
whether
it's
the
voting
systems,
voter
regis,
otherwise,
to
ensure
the
the
correctness
of
the
administration
of
the
election.
So
you
don't
just
evaluate
processes
or
procedures,
but
you
begin
to
look
at
each
piece
of
evidence.
Ballot,
manifest
chain
of
custody,
voting
system
results,
tapes
items
like
that
to
to
dig
into
the
evidence
to
ensure
accuracy.
B
All
of
these
audits,
however,
mean
something
or
the
word
audit
means
something.
It
means
first,
consistency
that
you
could
repeat
these
audits
in
multiple
places
or
at
multiple
times
consistently
using
published
and
understandable
policies
and
procedures.
Audits,
aren't
something
that
you
could
just
make
up
on
the
fly
and
wing,
but
in
fact
those
to
be
audited
understand.
B
What's
going
to
take
place,
those
to
conduct
the
auditing
understanding
can
document
each
step
of
the
audit
so
that
when
there
are
findings
and
results,
you
can
go
back
and
recreate
those
if
necessary,
the
the
the
folks
being
audited
in
the
auditors
can
go
back
through
and
say:
okay,
you
have
a
finding.
How
did
you
reach
that?
Finding
what
processes
procedures
systems
did
you
use
to
reach
that
finding?
And
how
do
we
understand
that
better
in
elections?
B
A
A
So
I
want
to
turn
to
jennifer
morrell
now
she's
been
a
local
election
official
in
both
utah
and
colorado
consultant
with
the
elections
group
and
is
the
nation's
foremost
expert
on
verifying
election
results,
and
that
includes
all
things
audits,
jennifer.
I
am
pretty
sure
that
all
states
verify
their
election
results.
I
mean,
after
counting,
all
states
have
some
kind
of
canvassing
or
a
certifying
process
right
absolutely
so
one
of
my
colleagues
will
put
ncsl's
resource
after
the
voting
ends
into
the
chat,
which
has
more
details
on
canvassing
and
certification.
A
But
of
course,
there's
more
that
can
be
done
and
that's
to
do
a
post-election
audit,
in
other
words,
to
look
after
the
fact
that
how
the
election
was
conducted
and
to
verify
that
the
outcome
was
the
right
one.
So
we
also
have
a
web
page
on
post-election
audits,
a
number
of
states.
The
majority
of
states
have
some
form
of
a
post-election
audit,
and
that
makes
it
seem
like
they're
commonplace,
but
they
aren't
actually
all
the
same.
In
fact,
this
can
be
kind
of
confusing
terrain,
so
jennifer.
C
I
would
love
to
it's
been
a
really
interesting
point
of
conversation.
Actually,
as
matt
said,
we've
been
talking
a
lot
about
audits
over
the
last
few
years,
and
quite
often
we
we
come
to
this
point
where
we
say
is:
is
this
one
of
those
areas
in
election
law
where
we
might
want
to
consider
having
some
sort
of
national
standard
or
having
some
sort
of
national
best
practice,
because
there
is
quite
a
bit
of
variety?
C
So
I'm
going
to
go
over
a
couple,
a
couple
of
different
states
that
I've
highlighted
here
just
to
sort
of
give
you
a
sense
of
the
flavor
for
how
different
post-election
audits
are
conducted.
So
we'll
start
with
two
they're,
actually
not
audits,
necessarily
but
they're,
a
post-election
logic
and
accuracy
test,
so
north
dakota
and
wyoming,
just
as
a
refresher
for
everybody.
C
All
election
offices
across
the
country
prior
to
running
their
election
are
required
to
do
a
pre-election
logic
and
accuracy
test,
and
essentially
what
they're
doing
is
they're
ensuring
that
the
voting
equipment
operated.
The
way
it's
been
programmed
that
it's
reading
ballots
correctly
for
each
and
every
candidate,
each
and
every
race,
north
dakota,
north
dakota
and
wyoming-
do
this
after
the
election
to
ensure
that
nothing
has
changed
they're.
Nobody
has
manipulated
that
system,
since
the
pre-election
testing
was
done.
So
it's
a
good
way
to
sort
of
go
back
afterwards
and
validate
that
that's
operating
correctly.
C
Then
we
have
this
little
bit
of
landscape
and
matt
sort
of
touched
on
it
between
sampling,
ballots
and
reviewing
those
and
doing
a
compliance
audit.
So
I
actually
start
with
arizona
and
new
mexico,
coincidentally
they're
next
to
each
other.
They
have
something
that
I
often
refer
to
as
like
a
hybrid
style
audit
sort
of
a
hybrid
between
a
traditional
post-election
audit,
where
we
take
a
fixed
percentage
of
ballots
or
fixed
number
of
ballots
and
review
those
for
accuracy
in
in
arizona
they
take.
C
Two
percent
of
the
precincts
or
two
percent
of
the
vote.
Centers
and
their
state
audit
board
designates
a
margin
for
errors,
so
assuming
once
they
they
do
that
review
if
they
are,
if
they
have
not
exceeded
that
margin
of
error
rate
they
and
it's
very,
very
tight.
There's
not
a
lot
of
tolerance
there.
It's
just
a
couple
of
ballots
that
can
be
off
the
audit's
good
and
it's
completed,
but
what's
really
cool
about
that
their
this
particular
type
of
audit
is
if
there
are
discrepancies.
C
The
audit
is
designed
then
to
to
escalate,
so
they
they
have
to
recount
the
back
the
ballots
that
were
selected
for
audit.
If
there
are
still
discrepancies,
they
have
to
sample
more
and
ultimately,
it
can
end
up
in
a
full
hand
recount
and
the
precincts
are
randomly
selected
just
which
is
a
best
practice
across
the
board.
C
It
is
done
again
across
all
counties
and
what's
interesting,
oh,
and
also
like
arizona,
it's
designed
to
expand.
So
if
there
are
discrepancies,
if
there
are
issues
the
audit
is
designed
to
continue
to
sample
until
they
either
feel
confident
or
they
they've
done
a
full
hand.
Recount.
What
I
like
about
new
mexico
and
we'll
talk
a
little
bit,
I
think,
later
on,
when
we
we
talk
about
best
practices,
we'll
talk
about
just
national
standards
across
the
country
for
certified
auditors
of
any
type
there.
C
There
is
a
professional
sort
of
auditing
organizations
across
the
country
and
new
mexico
actually
employs
a
cpa
firm
that
does
professional
auditing
and
adheres
to
those
standards
to
review
their
audit
practices.
So
that's
kind
of
interesting
matt
mentioned
compliance
audits.
So
we
kind
of
like
put
our
focus
on
michigan
here.
Michigan
looks
at
all
of
those
procedural
things.
It
does
happen
after
the
election.
Oh
and
sorry,
I
should
mention
arizona
and
new
mexico.
C
C
So
you
can
get
to
learn
more
but
michigan
when
they
go
in
and
do
their
these
procedural
audits
or
these
compliance
audits.
They
actually
look
at
processes
that
required
both
pre-election
during
the
election
and
post-election
making
sure
reconciliation
forms
were
completed
correctly.
Logic
and
accuracy.
Testing
was
done
correctly
and
sort
of
all
the
things
that
they've
outlined
in
their
manual
for
conducting
an
election
were
complied
with
or
were
done
according
to
the
law.
C
The
last
two
that
I'll
mention
new
jersey
is
kind
of
a
funny
state
where
they
had
a
law
put
on
the
books
years
ago.
Back
in
2008,
I
think
at
the
time
they
thought
it
was
a
risk
limiting
audit
law.
It's
not
quite
and
they've
had
they
have
not
been
able
to
get
that
changed
legislatively,
and
so
they
take
more
of
a
traditional
approach
where
they
examine
two
percent
of
election
districts.
C
What
I
love
about
new
jersey,
though,
is
that
the
attorney
general's
office
does
appoint
an
audit
board,
and
it's
required
that
at
least
two
members
of
that
audit
board
have
verifiable
sort
of
professional
experience
in
statistics
and
in
auditing.
So
that's
really
helpful.
The
last
date,
I'll
just
I
have
highlighted
here
in
yellow,
is
colorado.
A
B
B
It's
so
good,
but
it
gives
me
a
lot
of
optimism
because
it's
an
area
again
where
we've
seen
so
much
improvement,
where
we've
seen:
expansion
of
auditing
principles
and
techniques
beyond
just
the
post-election
environment,
into
pre-election
policies
and
procedures,
evaluations
of
of
voting
systems
and
otherwise
to
ensure
the
accuracy
of
the
election
and
an
area
that
I
think
we
can
continue
to
grow.
There
is
so
much
that
goes
into
running
an
election
from
start
to
finish
so
much.
B
You
know,
starting
with
voter
registration
and
voter
registration
list
maintenance
and
being
able
to
audit
that
all
the
way
through
to
tabulation
and
ensuring
the
accuracy
of
the
results.
We
see
election
officials,
embracing
both
the
transparency
and
and
value
of
audits,
that
I
I
think
it's
an
area.
I
know
it's
an
area
that
election
officials
will
continue
to
to
expand
and
kind
of
a
nerdy.
B
Part
of
this
is
going
to
be
that
more
and
more
elections,
data
is
available
and
can
be
used
effectively
and,
and
one
of
the
ways
that
that
could
be
used
is
to
conduct
these
audits.
So
as
more
elections,
data
becomes
available
becomes
usable
a
lot
of
times.
It
was
just
in
pdfs
and
unusable.
What
we're
seeing
is
election
officials
utilizing
that,
as
jennifer
said,
to
work
with
professional
auditors
to
professionalize
themselves
in
auditing
to
work
with
other
folks
in
order,
including
legislatures
to
understand?
A
C
Yeah,
so
we're
going
to
look
again
at
this
kind
of
patchwork
here
you
see
a
map
really
representing
sort
of
the
more
traditional
as
well
as
those
that
don't
do
an
audit.
I
will
talk
about
the
white
states
in
just
a
minute,
so
those
are
rla
states
and
we'll
get
to
those
one
thing,
though,
that
I
think
is
really
important
and
and
matt
hit
on
so
many
of
those
definitely
the
actual
doing
of
the
audit.
So
the
mechanics
of
that
how
we
count
the
ballots,
how
big
are
the
batches?
C
What
is
the
process?
Most
states
define
that
statutorily.
One
thing
that's
important,
though
that
is,
is
pretty
consistent
across
all
50
states
is
that
they
all
allow
for
observers.
They
all
allow
have
a
way
for
the
public
to
come
in
and
observe
the
audit.
Some
go
a
step
further.
They
actually
require
members
of
the
political
party
to
be
present
or
bipartisan
audit
teams
to
be
the
ones
actually
reviewing
the
ballots.
C
So
I
think
that's
really
an
important
component,
so
just
quickly
then
kind
of
going
back
to
this
patchwork
of
post-election
tabulation
audits
and
keep
in
mind
as
matt
mentioned.
There's
there
are
all
sorts
of
areas
in
election
administration
from
the
time
of
order
registers
until
we
seal
those
ballots
up
for
retention,
where
we
have
the
opportunity
to
conduct
a
test
or
an
audit
to
verify
that
the
work
was
done
correctly.
C
What
we're
talking
about
specifically
here,
at
least
over
the
next
few
slides,
is
the
post-election
tabulation
audit,
the
way
that
we
verify
that
the
vote-telling
equipment
operated
the
way
that
it
was
intended.
That's
kind
of
the
thing
that's
in
the
news
right
now,
so
we
thought
that
we
should
focus
on
that.
So
we
have
these
states
and
please
nobody
judge
me
on
the
colors
I've
used
here.
C
They
don't
mean
anything
so
don't
read
into
them
at
all,
they're,
simply
just
colors.
The
orangey
colored
states
represent
our
states
that
are
doing
that
traditional
or
fixed
percentage
on
it.
So
they're
required
by
statute
to
draw
a
certain
percentage
of
precincts
or
of
districts,
sometimes
of
machines.
C
So
that's
a
little
bit
interesting.
West
virginia
is
another
traditional,
fixed
percentage
audit
state
where
they
audit
three
percent
of
their
ballots
prior
to
certification.
If
there's
a
discrepancy,
they're
required
by
law
to
do
a
full
hand
recount,
they
have
laws
and
rules
and
regulations.
For
that
I
already
talked
about
our
hybrid
states.
We
also
have
the
states
in
green.
I
mentioned
already
north
dakota
and
wyoming
arkansas
does
a
type
of
procedural
at
it.
C
They
don't
actually,
and
they
have
a
pilot
now
actually
to
review
ballots
as
well
and
and
compare
those
to
the
voting
system
in
south
dakota.
They
look
at
some
electronic
data
and
then
we
have
our
speeds
in
yellow.
Where
currently
there
aren't,
there
is
no
requirement
to
do
any
type
of
post-election
audit.
You
have
a
little
star
by
nebraska,
so
their
law
actually
says
the
secretary
may
call
for
a
post-election
audit.
C
Now
I
want
to
kind
of
move
to
our
our
risk
limiting
audit
states-
and
this
is
the
area
where
I've
spent
most
of
my
time
and
I'm
most
familiar
over
the
last
few
years
and
I'll
we'll
talk
a
little
bit
about
mechanics
in
just
a
minute
just
to
give
you
an
idea
of
where
states
are
at
so
we
have
colorado
and
georgia
and
rhode
island
risk.
Limiting
audits
are
their
official
method
of
auditing.
It's
what's
required
in
law.
C
Their
statute
says
that
the
the
election
official
may
do
a
traditional
audit,
that's
defined
in
statute,
how
that's
done
or
they
may
choose
to
do-
a
risk
limiting
on
it
and
the
way
that
that's
conducted
is
also
defined
in
ruling
statute,
and
then
we
have
these
jurisdictions
in
yellow.
So
we've
got
indiana,
michigan,
nevada,
new
jersey,
pennsylvania,
virginia
all
sort
of
either
implementing
rlas
or
piloting
rlas,
and
that's
actually
been
really
cool.
To
be
part
of
that.
C
C
What
that
experience
has
been
like.
So,
let's
talk
just
a
little
bit
more
about
rlas,
just
to
remind
everybody
risk
limiting
audit,
as
I
mentioned,
we're
talking
about
post-election
tabulation
audits.
So
it
applies
just
to
that.
That's
where
auditing
the
voting
equipment,
we
examine
a
random
sample
of
voted
ballots
and
we're
looking
for
a
statistical
level
of
confidence
that
the
outcome
of
the
election
is
correct.
So
you
know
it
does
not
have
to
be
a
hundred
percent.
Everything
doesn't
have
to
be
a
hundred
percent
correct.
C
There
is
a
little
bit
of
opportunity
for
some
discrepancies,
but
not
that
would
change
the
outcome
of
the
election,
and
I
know
that
that's
an
area
where
sometimes
we
get
sort
of
caught
up
in
that
language
you're
confused
about
that
language.
I
know
my
fellow
election
administrators,
we
don't
like.
We
want
things
to
be
a
hundred
percent.
We
want
it
to
be
100
right,
black
white,
but
part
of
why
that's
important
part
of
why
we
don't
just
do
a
full
recount
every
time.
C
They
are
useful
for
validating
the
integrity
of
our
voting
equipment
and
they've
been
shown
to
do
that
in
states
that
have
adopted
those
they
do
an
excellent
job
at
verifying
the
accuracy
of
our
results,
and
the
audit
itself
is
designed
to
detect
and
correct
any
errors
that
would
change
the
outcome.
So,
just
like
I
mentioned
with
the
hybrid
audit
in
arizona
and
new
mexico
rlas
are
designed.
If,
if
we
do
have
discrepancies,
we're
required
to
sample
more,
we
want
to
keep
looking
until
either.
C
We've
met
that
measure
of
confidence
or
if
it
just
happens
that
there
there
are
so
many
issues
or
discrepancies
or
errors.
Whatever
the
case
may
be,
it
may
require
us
to
do
a
full
hand
count.
So
it's
an
efficient
way
of
auditing
by
starting
with
the
smallest
number
of
ballots.
We
need
to
reach
our
level
of
confidence,
but
also
having
it
a
mechanism
to
trigger
more
sampling,
more
review
if
it's
needed.
C
Often
in
a
traditional
audit
only
ballots,
the
only
ballots
that
are
reviewed
are
in
the
selected
precincts,
so
we
may
select
a
handful
of
precincts
or
districts
and
only
look
at
those
ballots
and,
as
I
just
mentioned
already,
an
rla
is
designed
to
escalate
to
additional
rounds
of
auditing
if
discrepancies
are
discovered
in
a
traditional
audit.
Some,
as
I
mentioned,
arizona
new
mexico
have
designed
into
their
statute
a
way
to
escalate
that
other
states
have
been
less
clear
about
that
or
are
silent
on
that.
C
If
the
audit
determines
that
there
was
an
issue,
as
I
mentioned
west
virginia
says,
we'll
go
to
a
hand
recount.
Others
are
quiet
on
that
and
there
is
no
mechanism
to
sort
of
do
anything
further
and
I
think
for
policy
makers.
That's
important
to
understand
the
last
thing
I'll
talk
about
is
just
a
little
touch
on
the
compliance
audits
and
sort
of
what
happens
there.
C
C
The
great
thing
is
often
it's
already
done,
and
it's
under
the
requirements
in
law
for
canvassing
or
certifying
so
most,
almost
all
the
things
I've
listed
here
across
all
50
states
there's
some
requirement
to
sort
of
review
one
or
more
of
these
as
part
of
the
certification
process.
So
that's
just
assuring
that
all
of
the
records
were
completed
accurately
and
completely.
C
C
This
is
the
one
that
again
all
50
states
a
requirement
for
certification,
we're
looking
at
that's
sort
of
how
many
people
voted,
how
many
people
checked
in
how
many
people
were
recorded
in
the
public
versus
how
many
ballots
were
actually
cast
and
some
jurisdictions
will
actually
require
you
to
look
at
the
number
of
eligible
voters
right
and
that's
just
to
verify.
We're
gonna
have
more
people
voting
than
we're
eligible
or
registered
to
vote.
So
sorry,
not
eligible
voters
registered
voters.
That
would
be
a
really
bad.
I
gotta
correct
that.
C
We
look
at
those
that
were
registered,
those
that
were
checked
in
and
then
ballots
that
were
actually
counted
and
we're
looking
for
a
reconciling
across
those
numbers,
and
then
we
can
look
at
other
things
like
duplication
logs,
so
I'm
going
to
just
stop
there.
I
feel
like
that,
was
kind
of
a
big,
broad
overview
of
what's
happening.
What
can
happen
and
I'm
gonna
turn
it
back
over
to
mandy.
A
D
There
are
three
methods
for
conducting
risk,
limiting
audits,
ballot,
polling,
batch
level,
comparison
and
ballot
level
comparison
to
conduct
a
ballot
polling
audit.
All
you
need
is
voter,
verifiable
paper
ballots.
You
know
who
reportedly
won
and
by
how
much
to
audit
the
contest
you
randomly
retrieve
a
ballot
and
manually,
examine
and
tally
the
votes
for
that
contest.
Then
you
retrieve
a
second
ballot,
then
a
third.
D
You
continue
to
tally
randomly
selected
ballots
until
you
have
statistical
confidence
that
the
correct
candidate
won.
In
other
words,
you
look
for
evidence
that
the
winner
won
by
a
similar
margin,
as
originally
reported,
like
the
other
two
methods
for
risk,
limiting
audits
with
a
ballot
polling
audit,
the
smaller
the
margin
of
victory,
the
more
ballots
you'll
usually
need
to
count.
If
you
discover
major
discrepancies,
you'll
end
up
doing
a
full
recount
valid
polling
audits
verify
that
the
outcome
of
the
election
is
correct.
D
For
a
batch
comparison
audit,
you
need
a
voting
system
that
can
provide
subtotal
reports
by
batches.
For
example,
each
precinct
could
be
a
batch
or
even
each
scanner
to
audit
you
randomly
select
a
batch.
Then
you
manually
tally
every
ballot
in
that
batch.
Then
you
compare
your
account
to
the
reported
subtotals.
D
This
process
repeats
until
you
have
statistical
confidence
that
the
correct
candidate
won
batch.
Comparison
audits
verify
that
the
outcome
of
the
election
is
correct
and
also
provide
some
insight
into
possible
tabulation
errors
to
conduct
a
ballot
comparison.
Audit.
You
need
the
ability
to
produce
a
cast
vote
record
that
can
be
matched
to
physical
ballots.
D
A
cast
vote
record
is
an
electronic
record
of
how
the
voting
equipment
interpreted
the
choices
made
on
each
and
every
ballot.
Physical
ballots
should
be
numbered,
labeled
or
organized,
so
they
can
be
matched
to
their
digital
records.
To
audit
you
randomly
select
ballots
and
compare
the
physical
ballot
to
its
digital
record.
This
method
is
more
efficient
than
the
other
two
methods.
Since
you
look
at
fewer
ballots
to
reach
the
same
level
of
statistical
confidence,
valid
comparison
audits
provide
the
deepest
level
of
insight
into
possible
tabulation
and
process
errors.
C
A
B
A
A
C
Yeah,
so
that's
actually
a
great
question
again,
depending
on
whether
a
jurisdiction
is
bringing
ballots
in,
for
instance,
mail
ballots.
We
know
that
states
have
different
time
frames
for
when
those
can
be
processed
some
start
prior
to
election
day.
Some
start
on
election
day.
We
have
early
voting
taking
place.
We
have
election
day
voting.
What
I
think
is
important,
though,
for
the
person
that
posted
the
question
is
jurisdictions.
It's
it's
not
just
tracking
those
or
auditing
on
election
night,
but
they
are
tracking
those
as
their
cast.
C
So,
for
instance,
mail
ballots
ideally
have
a
batch
tracking
sheet
that
accompanies
those
and
every
time
they
move
or
change
hands.
This
is
what
we
talk
about
when
we
what
we
mean
when
we
here
talking
about
chain
of
custody,
somebody
needs
to
document
or
verify.
I
started
with
a
hundred
ballots.
They
went
to
signature
verification,
maybe
we
or
some
whatever
verification
process
every
state
has
a
different
verification
process.
Maybe
we
removed
a
couple
because
there
was
an
issue,
something
was
omitted,
we're
gonna
document
that
now
we
have
98.
C
we're
going
to
move
those
to
ballot
opening
once
we
get
through
that
process.
Somebody
has
to
do
some
sort
of
manual
count
and
document
I
still
have
98
ballots
or
or
maybe
we
removed
one,
because
it
was
damaged
and
needed
to
go
to
duplication
and
then
it
gets
to
the
scanning
station
and
this
and
once
they're
scanned.
We
also
verify
we
still
have
98
ballots
in
our
custody.
C
When
we
get
ready
to
do
the
audit,
we're
going
to
look
at
that
again
and
we're
going
to
say
our
paperwork
shows.
You
know
that
we
and
I'm
just
simplifying
this.
We
have
a
hundred
ballots
in
our
possession
and
when
we
go
to
do
the
audit,
we're
going
to
do
some
verification
of
that
piece
count
as
well
to
ensure
to
the
posters
question.
We
haven't
added
or
lost
or
gained
ballots
somewhere
in
the
course
of
processing
them
or
through
some
security
breach,
or
something
like
that
and
I'll.
B
So
you
could
actually
see
the
life
of
an
individual
ballot
as
it
works
its
way
through
the
process
to
prevent
that.
On
the
security
side,
there
are
security,
I'll,
just
sort
of
push
service.
From
my
old,
my
old
agency,
the
cyber
security
and
infrastructure
security
agency
has
protective
security
advisors
who
live
and
work
in
every
state
who
can
conduct
and
have
conducted
thousands
of
physical
security
reviews
on
election
offices
and
it's
their
job
to
go
through
with
the
local
officials.
B
Sometimes
it's
with
other
folks
that
that
are
responsible
for
helping
secure
the
buildings
that
they
live
and
work.
In
that
you
know,
you
have
cameras
that
you
have
locks,
that
you
have
authentication
methods
all
of
those
things
to
ensure
the
custody
and
protection.
The
other
thing
I'll
add
is
that
many
states,
not
all
in
order
to
have
access
to
ballots,
particularly
once
they've
gone
through
and
are
either
stored
or
ready
for
auditing,
require
bipartisan
access.
No
one
singular
person
can
have
access.
They
typically
I'll.
Tell
you
my
home
state
of
ohio.
B
They
sit
behind
locked
door.
That
requires
two
two
keys:
a
democrat
republican
rep
in
order
to
access
and
that's
a
fairly
common
practice,
not
true
everywhere.
So
there's
lots
of
steps
you
can
put,
but
recognizing
that
locks
can
be,
you
know,
picked
or
accessed.
You
know
that
that
seals
can
be
bypassed.
You
know,
that's
why
you
have
cameras
typically
on
your
storage
vault,
for
your
ballots.
Things
like
that.
C
And
I
would
just
add
those
for
those
jurisdictions
and
states
that
do
the
compliance
audits
that
we
were
talking
about
a
review
of
those
logs.
That
matt
was
mentioning
to
sort
of
document
who
came
and
went
and
had
access
to
those
ballot
cages
or
ballot
storage
rooms
is
actually
part
of
that
compliance
process
or
can
be
part
of
that
compliance.
Audit.
That's
done.
A
C
Yeah,
so
I'm
I'm
actually
going
to
put
the
spotlight
on
that
and
let
him
go
first,
but
I'll
sort
of
lay
the
foundation
chain
of
custody
is
essential
right
and
it's
not
exclusive
to
oh,
I
said
I'd,
let
you
go
first,
I'm
going
to
talk
for
a
minute,
no
question:
it's
not
exclusive
to
election
administration
right.
This
is
a
requirement.
C
Anybody
that's
practiced
law
or
had
to
go
in
front
of
a
court
right.
We
know
that
evidence
is
a
good
example
of
something
that
requires
really
strict
chain
of
custody.
I
said
I
put
matt
on
the
spot
because
you
know
a
couple
of
years
ago.
Elections
was
designated
as
critical
infrastructure
in
2017
and
elections,
along
with
all
the
other
sectors
that
fall
under
that
designation,
have
requirements
for
chain
of
custody
of
equipment
and
data
and
other
things
that
are
really
sensitive.
C
So
it
is
important.
It
is
really
critical
thing
that
we
do
in
terms
of
policy
making.
I
think
I
think
there
is
probably
an
opportunity
to
look
a
little
bit
closer
of
that
in
light
of
the
critical
infrastructure
designation
and
and
ensure
we
it's
one
of
those
areas
where
sometimes
we
go
back
and
forth
like
do.
Are
we
explicit
in
policy
or
should
we
leave
some
of
those
things
up
to
administrative
rulemaking
and
where
is
sort
of
the
balance?
C
B
You
you
nailed
it.
I
mean
from
my
point
of
view
chain
of
custody.
It
is
everything
as
we
look
at
providing
evidence
to
the
voters
to
candidates
of
the
integrity
of
the
election
right.
You
you
have
to
be
able
to
not
just
talk
about
it,
but
show
it
and
chain
of
custody
is
one
of
the
core
pillars
of
being
able
to
do
that
and
a
lot
of
times.
What
I've
seen
in
states
is
either
through
administrative
rule
or
or
actual
law.
B
What
are
they
already
doing
where
and
how
are
they
tracking
that,
but
then
two
is,
is
setting
the
expectation
in
law,
the
chain
of
custody
access
controls
are
maintained
across
the
process,
but
then
allowing
them
through
administrative
rule
and
and
local
procedure
to
kind
of
establish
for
their
operation,
because
the
reality
is
both
physically
and
in
a
cyber
sense.
It's
going
to
be
unique
to
individual
counties,
towns
and
cities
because
of
their
facilities
because
of
the
systems
they
use.
A
Thank
you
both.
I
want
to
reiterate
to
the
audience
that
if
you
have
questions
you
can
put
them
in
the
chat.
If
you
want
to
ask
those
questions
anonymously,
you
can
send
them
to
me
wendy
or
sage
directly,
and
we
will
ask
them
without
mentioning
your
name,
and
I
have
a
couple
other
audience
questions
gathered,
but
I
think
we
are
going
to
briefly
go
into
what
we've
tentatively
called
a
lightning
round
of
questions
for
matt
and
jennifer.
No
need
to
be.
A
You
know
too
brief,
but
we'll
try
and
keep
them
quick
so
that
we
can
go
through
a
bunch
of
different
pieces
and
your
answers
to
this
last
question
actually
lead
very
nicely
into
my
first
question.
So
jennifer
I'll
start
with
you,
if
you
had
one
ask
for
legislators
as
as
a
former
local
election
official,
as
you
know,
expert
on
this
topic,
what
would
you
ask
legislators
to
do?
What
would
that
ask
be.
C
Yeah.
I
think
I
think
getting
everybody
to
this
place.
Where
we
do
we
do
a
post-election
tabulation
audit
prior
to
certification
is
really
important.
I
think
one
thing
we're
struggling
with
is
this
need
for
independence?
We
know
that
by
law,
the
election
official
is
the
legal
custodian
of
those
ballots
and
of
those
equipment.
They
are.
The
expert
are
the
professional.
C
They
understand
how
the
system
works
and
need
to
be
present
and
a
part
of
that,
but
we
want
that
transparency
and
so
thinking
thoughtfully
thinking
about
not
just
observers
are
allowed,
but
perhaps
appointing
citizen
audit
boards
or
or
some
sort
of
independent
body
to
observe
oversee
review.
I
think
is
really
important
when
we
think
about
using
audits
as
a
way
to
build
trust.
B
Yeah,
I
I'm
gonna
take
sort
of
the
the
one
that
legislators
hear
the
most
most
often
but
it,
but
it
really
is
key
and
that's
resourcing
in
order
to
meaningfully
audit
an
election
in
in
order
to
offer
the
level
of
transparency
necessary,
the
level
of
bipartisanship
professionalization,
you
have
to
invest
in
it
and
that
doesn't
have
to
just
mean
money.
It
does,
of
course
mean
money,
but
it
also
means
making
resources
available,
including
we
know
local
election
officials
who
will
partner
with
other
auditors,
if
they're
not
already
auditors.
B
In
some
states,
the
local
election
official
is
the
county
auditor,
and
so
there
are
already
professional
standards
for
them
around
auditing
and
approach
to
auditing,
but
making
those
resources
available.
Seeing
cross-county
collaboration,
even
cross-state.
Collaboration
on
on
providing
audit
resources
is
really
critical.
A
lot
of
times
you
can
make
up
the
the
financial
deficit.
C
And
mandy
sorry,
but
I
see,
like
somebody
posted
a
question
that
I
think
is
a
great
follow-up
to
what
matt
just
said
in
that
we
don't
have
to
create
this
from
scratch.
Right.
We
have
professional
organizations,
we
have
the
aicp,
whatever
the
the
american
association
of
professional
accountants,
with
ethical
guidelines
about
doing
no
harm,
doing
an
audit
where
you
don't
have
bias
where
you
are
remain
independent.
C
B
And
just
I
mean
because
jennifer
now,
mandy
we're
just
going
off
the
the
rails
now,
but
because
it's
a
great
question
from
the
audience
about
buy-in-
and
this
is
something
election
officials
quite
frankly,
right
now
are
obviously
talking
about
a
lot
and
that's
trust-
confidence
in
elections
right.
What?
What
can
we
do?
They've
been
been
handed
a
tough
situation
where
confidence
or
trust
in
elections
in
certain
areas
is
is
declining
or
low.
B
What
can
we
do
and-
and
part
of
that
is
offering
transparency
to
the
process
right
requiring
as
jennifer
mentioned,
having
citizen
audit
boards
certainly
having
meaningful
public
oversight
on
auditing
processes
and
procedures?
Second,
is
having
the
participation
of
the
political
parties
in
the
audit
process
in
some
way,
shape
or
form
really.
Does
people
want
to
know
that
their
people
are
in
the
room
and
participating
in
this
and
what
we
have
seen
happen
in
the
aftermath
of
this
election?
B
In
many
places
where
republicans
democrats,
other
political
parties
are
in
the
room
and
participating
is
those
people
that
participated
have
pushed
back
on
some
of
the
concerns
or
educated
their
party
or
their
folks
about
it,
because
they
were
in
the
room
and
part
of
it,
and
so
they
understand
what
took
place
and
how
it
happened,
and
so
that
the
benefit
is
not
just
that
you've
been
transparent.
You
can
talk
about
being
bipartisan.
It's
in
fact
those
people
go
back
and
talk
about
what
they
experienced,
what
they
saw
and
how
they
participated,
and
then
the
media.
B
This
is
a
fantastic
question
there.
There
are
necessary
accesses
for
media
that
need
to
be
part
of
this
as
well.
Certainly
election
officials,
when
it
comes
to
tabulation,
manage
that
when
it
comes
to
auditing,
need
to
manage
that,
but
one
of
the
best
things
that
folks
can
do
that.
I've
seen
work
in
counties
and
states
across
the
country.
Invite
the
media
invite
state
legislators,
invite
others
in
before
the
election
and
walk
them
through
how
the
process
works,
including
how
auditing's
gonna
work
when,
before
you
know
the
heat's
turned
up
before
you're.
B
In
the
midst
of
the
action
election
officials,
love
to
engage
and
show
off
the
work
that
they
do
and
so
bringing
in
state
legislators
bringing
in
members
of
the
media,
either
separately
or
together.
Sometimes
it's
more
fun
together
is
a
really
great
way
to
educate
and
run
a
mock
audit
for
them
show
them
how
you're
actually
gonna
physically
look
at
the
ballots
and
audit
them.
A
That
is
a
great
great
idea,
and
I
think
everyone
likes
to
take
a
tour
of
an
election
office,
at
least
if
you're
on
this
call,
you
probably
think
that's
interesting
jennifer
did
you
want
to
add
anything
to
that
about
about
media
okay,
good?
Well,
then,
I
was
going
to
pick
up
another
thread
that
matt
mentioned,
which
was
about
resources
and
ask
well
jennifer,
we'll
start
with
you
again
about
the
cost
of
risk,
limiting
audits.
A
C
You
know
it
really
is.
It
is
a
little
bit
of
a
loaded
question,
though,
because
you
know-
we've
talked
a
lot
here
today
about
what
happens
after
the
election.
Successful
rlas
really
rely
on
the
work.
That's
done,
pre-election
in
a
very
thoughtful
sort
of
organization
and
storage
of
ballots,
and
that
can
potentially
mean
some
additional
cost.
C
Often,
when
I
find
a
jurisdiction,
that's
struggling
with
that,
it's
one
of
two
things:
they
don't
have
the
space,
so
they
don't
have
adequate
space
to
store
and
organize
their
ballots
in
a
way
that
makes
them
easy
to
retrieve
for
an
audit
or
they
don't
have
enough
staff,
and
so
doing
that
you
know
filling
out
all
those
batch
forms
doing
that
reconciliation.
Doing
that,
counting
all
those
things
that,
when
done
well,
make
an
rla
go
smoothly.
C
That
takes
people
and
if
you
don't
have
enough
staff
to
do
that,
that
is
sometimes
one
of
the
first
things
that
gets
pushed
aside
for
more
important
tasks.
So
there's
that
the
actual
operation
of
the
audit
really
isn't
expensive
any
voting
system.
Now
that
has
a
paper
trail.
So
we
have
a
physical
piece
of
paper.
With
a
vote
recorded,
we
can
do
an
rla
on
depending
on
the
size
of
the
jurisdiction.
C
There
might
be
a
need
to
bring
in
some
extra
help
so
poll
workers
or
temps
or
citizen
bipartisan
citizen
audit
boards,
as
we
see
some
states
use
to
you,
know,
help
with
the
actual
retrieving
of
ballots
and
reviewing
those
and
and
entering
those
other
than
that,
the
only
other
cost
is
it
can
be
useful.
It
can
be
helpful
to
use
auditing
software.
A
Thank
you,
matt.
I'm
gonna
go
to
you
with
this
question,
which
goes
way
back
to
the
beginning
of
this
hour.
When
you
refer
to
forensic
audits,
so
you
said
there
was
a
standard
definition.
Would
you
be
able
to
explain
what
I
mean?
Is
there
a
citation
for
that?
Could
you
give
us
a
little
bit
more
yeah?
I
could
follow
definition
within
the
context
of
elections.
Yeah
sure.
B
And
I
can
follow
up
with
with
additional
detail,
but
often
so
in
in
cyber
security
or
forensic
audit
has
a
specific
definition
right
and
it's
looking
at
the
evidence.
Trails
on
systems,
including
things
like
logs,
even
digging
into
actual
chip,
sets
and
hardware
to
forensically
evidence-based,
understand
the
operation
or
or
miss
operation
in
many
cases
of
the
system
right.
B
So
when
cyber
security
teams
deploy
out
to
respond
to
an
incident
and
understand
what
took
place
in
that
incident,
they're
going
to
quote
unquote
forensically
review
the
systems
to
look
for
the
evidence
of
the
compromise
understand
where
the
compromise
came
from,
how
it
progressed
and
went
through
the
systems
in
elections.
We
could
think
about
it.
B
Similarly,
in
that
what
you're
looking
for
is
the
the
the
evidence
and
information
that
shows
the
integrity
of
the
process
right,
so
that's
not
always
what
a
digital
representation
a
lot
of
times
in
a
lot
of
election
offices.
That's
paper
tracking,
right
of
ballot,
manifest
of
chain
of
custody
sheets.
Again,
it's
the
access
control
logs
to
things
like
voting
systems.
B
It's
looking
at
voting
system
logs
and
and
again
the
the
purpose
of
the
forensic
audit
is
is
largely
if
for
when
you
experienced
a
problem
to
get
to
the
bottom
of
what
caused
that
given
problem
again,
I
could
follow
up
with
with
additional
information,
but
the
idea
is
is
to
use
auditing
standards
and
approaches,
right,
documented,
understood,
approaches
and
and
and
techniques
and
processes
and
procedures
to
understand
what
took
place
and
try
to.
You
know
in
some
cases
recreate
the
problem
from
the
evidence
available.
A
B
A
B
But
but
contrast
you
know
what
I
just
described:
there
is
something
in
in
the
cyber
world
known
as
what's
called
an
open-ended
vulnerability
assessment,
which
is
when
you
allow
skilled
individuals
to
crack
on
a
system
to
look
for
problems
in
that
system.
That
is
not
an
audit,
nor
is
that
a
forensic
audit
in
any
way
that
there's
a
reason
it's
referred
to
as
open-ended
there's,
not
a
lot
of
documented
procedures.
B
There
may
be
some,
but
it's
largely
allowing
really
you
know,
skilled
people
to
to
go
into
a
system
and
just
dig
around
looking
for
things
right
and
that's
in
stark
contrast
to
an
audit
where
you
have
controls
expected
results,
procedures.
Things
like
that.
So
just
to
kind
of
that's
a,
I
guess
a
cyber
description
of
something,
but
but
it's
important
to
understand
those
types
of
activities
exist,
but
but
that's
not
an
audit.
That's
not
what
that
is.
A
Really
helpful
to
have
that
clarified,
and
so
thank
you
sending
along
more
information
about
that.
That
would
be
wonderful,
and
that
reminds
me
if
both
of
you
are
either
of
you,
are
willing
to
share
your
contact
information
in
case
our
audience.
Members
have
questions
if
you
want
to
put
your
email
addresses
in
the
chat.
That
would
be
helpful
since
we
are
slowly
wrapping
up
like
I
said
I
don't
see
any
new
questions
in
the
chat.
A
I
have
one
more
that
I
don't
think
we've
really
touched
on,
which
is
shifting
gears
a
little
bit
here,
but
it's
about
the
relationship
between
audits
and
recounts
right,
so
risk
limiting
audit
can
escalate
to
a
recount
but
they're,
not
necessarily
the
same
thing
so
jennifer.
Do
you
want
to
talk
to
us
about
audits
and
recounts.
C
Sure
they
are
not
the
same
thing
and
you
know
matt
made
an
important
point
and
I'll
just
reiterate:
both
are
prescribed
in
law.
Both
have
laws
that
say
this
is
when
an
audit
will
take
place,
and
this
is
how
it
will
be
done.
This
is
when
a
recount
will
take
place
or
what
might
trigger
a
recount,
and
this
is
how
it
will
be
done.
C
I
think
the
the
important
couple
of
important
distinctions
on
a
recount
recounts
can
be
done
by
hand,
so
the
law
may
require
you
to
physically
examine
that
piece
of
paper
and
hand,
tally
that
typically,
that's
what
we
see
as
audits,
escalate
and
we're
sampling
we're
doing
that
by
hand.
Recounts
can
also
be
done
by
machine,
so
there
are
states
that
require
the
election
official
to
rescan
or
re-run
all
of
those
ballots
through
the
machine
and
ensure
that
they
got
the
same
results
again,
so
very
different
processes.
C
I
think
the
one
thing
that
I
hope
will
be
a
topic
of
discussion
at
a
future
ncsl
conference
or
something
is
as
states
adopt
these
laws
to
do
an
audit
that
has
the
potential
to
escalate
to
a
recount.
If
there
is
an
issue,
do
we
still
need
a
requirement
for
a
separate
recount
and
I'll
just
quickly
share
with
the
audience
the
first
time
I
did
an
rla.
I
think
we
did
it
exceptionally
well.
C
Matt
was
actually
the
chairman
of
the
eac
at
the
time
and
came
out
to
colorado
to
be
a
part
of
that.
We
we
executed
it
perfectly.
We
found
no
issues
that
would
have
changed
the
outcome
and
then
immediately
after
that
by
law,
I
was
required
to
do
five
recounts,
because
the
the
margin
between
the
winning
and
losing
canada
either
fell
within
that
threshold.
For
a
recount
that
was
automatically
triggered,
or
in
two
cases
we
had
candidates
willing
to
pay
the
fee
to
to
request
to
have
that
done.
A
Thank
you
to
everyone
who
decided
to
hop
on
to
yet
another
zoom
meeting,
zoom
webinar
and
learn
from
these
experts,
and
you
can
always
reach
out
to
us
at
ncsl
with
any
further
questions.
You
can
share
your
questions
directly
with
jennifer
and
matt,
and
I
hope
that
you
will
just
stay
in
touch.
So
thank
you
all
so
much
for
joining
us
and
I
hope
you
have
wonderful
days.