►
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
Welcome
everyone
to
evaluating
maths
with
tests
and
ensembles.
This
is
a
session
that
would
never
have
been
done
at
one
of
our
meetings
in
2011
or
2010.
and
in
fact
I
have
to
admit
that
ncsl
did
not
put
the
word
ensemble
in
the
redistricting
law
2020
book
and
we
published
that
just
one
year
ago.
So
I'm
going
to
say
that
was
an
oops.
I
already
have
a
file
for
changes
we
want
to
make
for
the
2030
edition,
and
that
is
entry
number
one
in
in
that
file.
A
So
the
reason
I
say
it's
an
oops
is
that
getting
familiar
with
tests
and
tools
that
are
based
on
algorithms
is
actually
mission,
critical
right
now
for
anybody
who's
going
to
be
involved
with
redistricting.
A
It
could
be
that
your
state
will
be
using
one
of
these
tools,
but
I
can
assure
you
that
people
outside
the
state
process
will
absolutely
be
using
these
tools
and
the
results
will
potentially
be
presented
in
any
court
cases
and,
as
we've
heard
before,
it's
not
a
question
of
whether
you'll
be
sued,
you
will
be
sued.
So
today
we
have
two
presenters
with
us
and
they
will
help.
A
You
know
what
you
need
to
know
on
this
new
topic,
but
before
I
introduce
them,
I
want
to
bring
to
your
mind
the
musical
oklahoma,
so
go
back
in
your
history
and
remember,
oklahoma
and
remember
the
song.
The
farmer
and
the
cowman
should
be
friends
well
that
I
think
is
appropriate
for
here,
because
we
have
moon
dutchen,
who
is
a
mathematician
and
patrick
lewis?
Who
is
an
attorney
you
put
those
two
together?
A
I
don't
know
which
is
the
cowman,
which
is
the
farmer,
I'm
not
really
sure,
but
surely
attorney
and
mathematician
are
the
modern
day
equivalent.
Fortunately,
these
two
people
have
proved
that
they
can
in
fact
be
friends
with
that
moon,
I'm
going
to
just
turn
it
over
to
you,
and
let
me
know
what
you
need.
B
Okay,
thank
you
so
much
best
introduction.
Ever
okay,
I'm
gonna
go
to
a
screen
share
and
give
you
a
couple
slides
so
right
now
I
aim
to
just
do
a
quick
introduction
to
what
it
is
that
wendy's,
describing
when
she
talks
about
ensemble
methods,
so
aka
the
millions
of
maps
approach.
So
there's
an
idea
out
there
that
computers
these
days
can
build.
So
many
alternative
redistricting
plans.
B
B
B
This
is
a
dream,
for
you
know,
at
least
since
the
1960s
that
computers
might
be
able
to
show
you
all
the
alternatives
and
maybe
even
to
automate
redistricting
and
do
the
redistricting
for
you.
So
the
idea
goes
something
like
this.
You
have
your
redistricting
criteria.
B
Well,
it
turns
out
that
you
can't
do
this.
The
universe
of
plans
is
just
much
too
big
and
wild.
So
what
you
can
do,
though,
is
sample
from
it
and
that's
what
I
want
to
talk
about.
There
have
been
a
lot
of
breakthroughs
in
the
last
five
years
or
so
in
how
you
can
draw
samples
of
districting
plans
in
a
way
that
gives
you
a
really
good
sense
of
the
landscape.
So
this
is
what
the
term
ensemble
means
here.
It
means
a
large.
B
You
know
diverse
collection
of
legally
valid
and
plausible
maps
and
I'll
tell
you
a
little
bit
about.
What's
going
on
in
this
area,
but
first
I
want
to
zoom
out
a
little
bit
so
patrick
and
I
will
be
talking
a
little
bit
about
the
methods
and
about
the
legal
context.
And
so
you
know
if
you,
if
you
look
back
in
the
last
couple
of
terms
of
the
supreme
court,
where
they
thought
about
partisan
gerrymandering,
in
particular,
here's
a
alito
from
oral
argument
in
in
rucho
in
the
north
carolina
partisan
gerrymandering
case.
B
So,
what's
alito
saying
he's
saying
if
you
make
a
list
of
the
so-called
neutral
criteria,
compactness
contiguity
protecting
incumbents
if
that's
really
neutral,
respecting
certain
natural
features
of
the
geography
and
you
have
a
computer
program
that
includes
all
of
those
awaits
them
all
and
let's
assume
all
that
is
neutral.
And
in
the
end
you
get
a
large
number
of
maps
that
satisfy
all
of
those
criteria
and
he
goes
on
to
say
he
really
wants
to
kind
of
think
about
this
collection
of
maps.
So
he
says
here:
you've
got
100
maps,
you
might
have
25.
B
I
think
you
probably
have
thousands
and
then
he's
going
to
say
once
you
have
them
all,
then
you
still
have
to
think
about
what
it
means
to
be
best.
You
have
to
choose
among
them,
and
the
legislature
typically
is,
is
the
one
who's
going
to
make
the
choice?
Okay,
so
first
thing
I
want
to
say,
is
alito
you're
on
the
right
track,
you're
off
by
a
few
orders
of
magnitude.
B
So
actually
there
are
not
thousands
but
trillions
upon
trillions
of
legally
valid
maps
in
in
most
states
and
under
most
under
every
set
of
rules
that
I've
encountered.
Example.
Minnesota
minnesota
is
a
nesting
state
where
the
state
house
districts
nest
inside
senate
districts,
and
I
can
tell
you
exactly
how
many
ways
there
are
to
make
senate
plans
in
minnesota
just
using
the
house
districts
as
the
building
blocks.
B
There
are
six
quintillion,
the
number
that
you
see
here
on
your
screen
and
now
just
try
to
imagine
if
you
use
census
blocks
as
the
building
blocks,
there's
just
an
unimaginable
number
of
ways
to
do
this.
So
you're
kind
of
out
of
luck.
If
you
want
to
understand
them
all,
but
that's
what
you
need
to
understand
if
you
want
to
evaluate
a
map?
Okay,
so
what
can
you
do?
B
You
can
sample
and
that's
what
I
want
to
talk
about
it
used
to
be
that
you
could
just
close
your
eyes
and
and
draw
a
bunch,
and
now,
in
the
last
few
years
we
have
much
better
mathematics
for
saying
that
you're
getting
a
representative
sample
of
all
the
possibilities.
B
B
What
you
can
see
here
is
that
you
get
kind
of
a
bell
curve
of
possible
outcomes,
I'm
showing
you
the
number
of
democratic
seats.
If
you
use
the
exact
way
that
people
voted
in
the
2016
senate
race,
you
see
there's
a
range
of
opportunities,
it's
pretty
hard
to
get
a
proportional
outcome
right
and
then
you
can
take
the
plans
that
are
actually
proposed,
and
these
are
some
of
the
plans
that
were
discussed
in
the
remedial
process
and
you
can
benchmark
them
against
the
world
of
alternatives.
B
So
it
shows
you,
you
know
what
kind
of
plan
might
give
you
this
many
seats.
What
kind
of
plan
might
give
you
that
many
seats
and
how
does
it
compare
to
all
the
different
millions
or
billions
that
a
computer
might
draw
according
to
the
rules
and
then,
on
the
other
hand,
here's
my
home
state
of
massachusetts
where,
for
the
last
20
years,
presidential
and
senate
races,
we
tend
to
have
about
a
third
of
voters,
preferring
republicans.
But
we
haven't
sent
a
republican
to
the
house
of
representatives
from
massachusetts
since
the
90s.
B
Well,
it
turns
out
that's
because
it's
essentially
impossible
so
because
the
distribution
of
where
people
live
in
the
state
and
where
those
votes
are
located,
if
you
do
an
ensemble
in
massachusetts,
you'll
find
every
single
plan
that
you
draw.
Has
a
9-0
democratic
delegation.
Okay.
So
this
is
the
kind
of
evidence
that
you
can
draw
once
you
use
these
ensemble
methods.
B
So
older
methods
methods
from
5
to
10
years
ago,
I
like
to
call
petri
dish
methods.
They
grew
districts
here
you
see
iowa's
counties
and
they're
growing
out
districts
like
cultures
in
a
petri
dish.
Until
you
fill
up
the
state,
that's
what
you
saw
again
five
to
ten
years
ago,
maybe
more
recently,
new
new
mathematical
methods
use
iterative
transformation,
something
called
markov
chains
and
so
here's
an
example.
B
I
just
want
to
show
you
recombination
on
pennsylvania,
so
you
get
a
sense
of
what
it
looks
like.
So
this
is
real
time.
Two
districts
are
being
merged
and
re-split
at
every
step,
and
you
can
see
that
after
just
a
short
amount
of
time,
you've
got
a
completely
different
way
of
redistricting
a
state.
So
this
is
a
like
a
glimpse
at
how
ensemble
methods
actually
work
to
build
lots
of
alternatives.
B
Okay,
there's
a
few
points
that
are
important.
This
is
not
optimization.
Okay.
This
is
this
is
representative
sampling
that
I'm
talking
about,
so
the
computer
can't
find
the
best
plan,
and
here
I
mean
can't
in
two
ways
like,
even
if
you
tried
to
program
it
to
optimize
something
the
space
is
too
big
and
wild
to
get
a
provable
optimum.
So
that's
one
point
of
view,
but
another
point
of
view
is
that
in
the
sphere
of
redistricting,
there's
not
a
simple
way
of
saying
what
it
even
is
to
be
best.
B
So
there
has
been
a
lot
of
progress
and
actually
later
today,
at
2
45,
sam
hirsch,
and
I
will
talk
about
some
new
work.
Assessing
voting
rights
act,
compliance
using
some
of
these
automated
and
algorithmic
techniques.
So
there's
been
a
lot
of
progress
thinking
about
how
to
quantify
criteria,
but
ultimately
a
computer
can't
tell
you
how
to
balance
those
against
each
other.
So
the
point
of
view
that
I
want
to
espouse
today
is
that
redistricting
is
still
for
people
still
for
you
all.
B
You're,
the
ones
who
are
going
to
be
drawing
the
lines
in
a
lot
of
cases,
but
these
methods
can
help
you
think
about
trade-offs.
So
again,
these
techniques
are
for
evaluation,
not
selection.
Sometimes
people
have
tried
to
use
computer
generated
maps
and
have
enacted
them,
and
I
think
maybe
patrick
will
tell
you
a
little
bit
more
about
some
of
those
instances
but
sampling,
ultimately,
in
the
method
that
I'm
describing
here
is
for
telling
you
kind
of
how
you
fit
into
the
world
of
possibilities.
B
So
I
want
to
emphasize
that
being
right
in
the
middle
of
a
bell
curve
doesn't
mean
you're
great.
You
have
different
competing
priorities
that
you
need
to
balance
against
each
other,
but
being
way
out
in
the
tails
being
an
outlier
against.
One
of
these
distributions
has
been
found
in
several
of
these
court
cases
to
be
a
red
flag.
B
Okay,
so
the
main
points
I
want
to
make
for
this
part
we're
making
big
strides
in
the
science
here
you
know
I
like
to
say
it's
not
rocket
science,
but
it
is
data
science.
So
you
know
this
isn't
going
to
be
something:
that's
ever
going
to
be
done
at
the
push
of
a
button
you're
going
to
need
people
who
are
thinking
about
the
methods
and
making
sure
that
everything's
working
in
your
individual
state
warning
just
using
a
computer
doesn't
make
it
fancy
right.
B
So,
if
you
don't
know,
you
know,
under
the
hood
of
what
I'm
describing
when
you
sample
you're
sampling
from
a
distribution
on
the
all
the
possible
plans
that
are
out
there,
if
you're
working
with
an
algorithm
that
can't
describe
its
sampling
distribution,
it
can
still
give
you
interesting
examples,
but
being
an
outlier
isn't
as
meaningful
like
for
those
petri
dish
methods
where
you
don't
know
why
they're
drawing
one
plan
and
instead
of
another
being
an
outlier,
there
is
probably
a
little
less
meaningful.
Okay.
B
So
that's
pretty
good
for
my
first
part
I'll,
stop
there
and
pass
it
to
patrick.
C
All
right
well,
thank
you
very
much,
so
I
will
start
with
the
start
with
the
you
know,
take
the
jumping
off
point
of
the
the
math
and
talk
a
little
bit
about
the
law
that
underlies
the
use
of
some
of
these
ensemble
techniques
and
sampling
techniques
in
court.
So
let
me
share
my
screen.
C
All
right,
so,
hopefully
everyone.
Hopefully
everyone
sees
that
so
sampling
techniques
have
developed
in
the
context
of
litigating,
partisan,
gerrymandering
cases
to
date,
they've
not
really
seen
these
techniques
being
used
in
cases
brought
under
the
voting
rights
act,
although
you
know
stay
tuned
because
we
you
know
they
may
be
coming
soon.
C
So
just
wanted
to
you
know,
provide
just
a
brief
history
of
how
partisan
gerrymandering
has
evolved
in
both
the
federal
and
the
state
court
systems
and
then
talk
a
little
bit
about
how
the
techniques
that
you've
just
heard
described
can
be
used
in
some
of
the
limitations
that
you
know
have
been
seen.
You
know
in
some
of
these
cases,
so
we'll
start
with
federal
courts.
They
are
you
know.
C
Federal
and
state,
of
course,
are
are
different,
and
this
has
been
you
know,
sort
of
described
as
sort
of
the
quest
for
you
know
answering
the
question.
You
know
how
much
politics
is
too
much
politics
in
the
under
article
one
section,
four
of
our
constitution.
We
entrusted
the
drawing
of
political
lines
to
our
political
bodies
of
government
and,
unsurprisingly,
perhaps
you
know
political
actors
in
a
political
boundary.
C
Faced
with
a
very
complex
problem
like
taking
a
very
large
amount
of
geography
and
dividing
it
into
districts,
you
know
there
are
you
know:
politics
have
come
into
play
to
one,
you
know
manner
or
another
for
since
the
beginning
of
time.
So
initially
there
were
k.
You
know
you
have
a
case
like
in
1946,
cold
grove,
v,
green.
Where
the
court
says
we
don't
want
to
enter
the
political
thicket.
The
court
holds
that
up,
essentially,
a
gerrymandering
claim
is
non-justiciable.
C
As
a
matter
of
the
federal
constitution.
The
court
did
not
want
to
venture
into
politics.
It
didn't
feel
competent
to
make
the
choices
between
you
know
when
a
map
is
fair
and
when
it's
not
in
the
1980s,
you
then
have
the
case
davis
v
bandamir,
which
was
looking
at
indiana's,
partisan,
gerrymandering
challenge
in
indiana
and
it
sort
of
cracks
the
door.
It
concludes
that
there
that
the
claim
was
non-justiciable,
but
it
it
then
opens
at
the
door
and
says
well.
The
claims
might
be
justiciable.
C
If
you
can
show
a
politically,
you
know,
managed
or
judicially
manageable
standard
to
decide
when
politics
is
to.
You
know
when
political
considerations
have
been
too
much
and
vandemir
suggested.
C
You
know,
essentially
the
rule
being
something
akin
to
you
know
when
the
minority
party
is
essentially
shut
out
of
the
process
it
was,
it
was
almost
a
process,
focused
analysis,
but
at
that
point
still
there
had
been
no
plan
ever
thrown
out
as
a
partisan
gerrymander
in
federal
court.
C
Almost
20
years
later,
you
have
veith,
which
was
a
case
that
looked
at
the
pennsylvania
plan
from
2001,
and
you
know
there
and
again
that
plan
was
upheld
in
the
supreme
court
and
justice
kennedy
and
a
concurrence
were,
you
know,
said:
look
the
the
the
real
close
to
our
question
here
is:
can
we
come
up
with
a
judicially,
manageable
standard
for
deciding
how
much
politics
is
too
much
really
framed?
The
question
very
narrowly
like
that
and
sort
of
set
loose.
C
You
know
over
a
decade
of
research
of
of
political
scientists
and
and
now
mathematicians
and
statisticians
and
and
and
computer
scientists
and
and
folks,
from
all
manners
of
disciplines,
to
try
to
answer
that
question.
C
There
have
been
then
some
lower
court
decisions
and
we've
all
everyone
on
this
zoom
has
probably
heard
about
all
of
them
by
now.
But
you
know,
essentially
it
comes
to
a
head
in
in
the
rucho
case
in
2019,
which
was
a
challenge
to
north
carolina's
congressional
plan,
and
there
the
court
essentially
concluded
as
a
matter
of
federal
constitutional
law.
There
was
there
was
no
manageable
way
to
answer
the
question:
how
much
politics
is
too
much?
It
was
just
beyond
the
competence
of
of
federal
courts.
C
C
If
you
have
population
deviation,
you
know
like,
for
example,
larios
v
cox
in
georgia,
where
you
had
all
the
republican
districts
were
overpopulated,
the
democratic
districts
were
under
populated,
and
that
was
that
deviation
was
seen
as
driving
a
politically
unfair
result.
That
type
of
claim
might
exist
might
still
exist
under
this
framework,
but
the
general
political
gerrymandering,
not
so
much
so
state
courts
and
the
state
constitution
now
come
into
play,
and-
and
this
came
in
pennsylvania
a
case
I
actually
worked
on,
and
you
know
there.
C
The
pennsylvania
supreme
court
had
previously
heard
challenges
to
partisan
gerrymandering
challenges
and
had
essentially
adhered
to
the
rule
of
vanderbear
and
found
that
for
such
a
claim
to
be
justiciable
under
pennsylvania
state
constitution,
you
had
to
again
make
the
showing
of
the
judicially
manageable
standard
minority
party
being
shut
out,
and
here
we
had
probably
the
first
use
that
I'm
aware
of
of
major
use
that
I'm
aware
of
of
ensemble
techniques
you
know
being
utilized
in
you
know
a
challenge
to
a
a
plan.
C
I'm
aware
that
some
had
been
used
in
in
the
wisconsin
litigation,
but
this
I
think
these
were
in
other.
In
other
cases,
but
there
were
some
techniques
that
were
used
that
were
coming
out
for
the
first
time
in
this
case
and
here
the
lower
court.
So
the
pennsylvania
supreme
court
ultimately
decides
the
case,
but
the
commonwealth
court,
which
is
an
intermediate
appellate
court,
had
tried
it
and
there,
the
commonwealth
wealth
court
judge
also
concluded
there.
B
C
Manageable
standard,
but
under
the
supreme
court,
applying
its
state
constitution
concluded
that
that
it
was
and
ultimately
struck
the
plan
down
and
the
court
framed
that
issue
as
being
about
whether
the
plan
subordinates
traditional
criteria
in
the
service
of
partisan
advantage
and
there's
sort
of
a
discussion
about
it,
and
you
know
essentially
looked
at
you
know,
took
the
bell
curve
that
that
moon
showed
us,
and
you
said
well,
your
plan
is
an
outlier
and
and
concluded
that
that
was
sufficient
evidence
to
strike
the
plan
down
as
a
partisan
gerrymander
on
the
heels
of
the
pennsylvania
case.
C
He
then
had
common
cause
versus
lewis
in
north
carolina.
You
know
which
applied
a
very
similar,
although
not
identical
analysis
and
and
there
the
court
found
the
legislature
acted
with
a
predominant
intent.
So
it
applied
a
predominant
standard,
intent
to
control
and
predetermine
the
outcome
of
legislative
elections.
C
And
again,
just
kind
of
following
along
those
those
lines.
So
how?
How
are
these
sampling
techniques
actually
used?
So
plaintiffs
in
the
prior
redistricting
cycle
used
these
techniques
to
show
that
a
plan
had
an
extreme
partisan
bias
that,
by
that
bias,
is
typically
measured
in
seat
share.
So
your
your
plan
has
you
know,
12.
C
You
have
16
seats
in
the
plan
12
or
are
one
party
for
the
other
party,
and
you
use
the
an
ensemble
approach
to
conclude
that
that's
outside
of
you
know
a
fair
range
or,
what's
deemed
to
be
a
fair
range
and
two
that
the
bias
is
due
to
partisan
intent
and
not
another
reason
that
second
part
is
is
part
of
the
power
of
ensemble
methods
as
determined
by
courts,
because
one
thing
that
that
parties
that
use
these
ensemble
methods
of
claim
is
that
they
allow
you
to
control
for
geography
that,
if
you
have
you
know
the
old,
you
know
the
commonly
accepted
norm
that
you
know.
C
C
Sometimes,
as
moon
suggested,
that
these
techniques
have
been
used
in
the
remedial
phase.
For
example,
in
north
carolina,
when
the
north
carolina
legislature
passed
a
new
plan
in
response
to
the
common
cause
litigation,
they
actually
used
simulated
plans
that
have
been
developed
by
one
of
plaintiff's
experts.
C
C
I
think
some
of
the
issues
that
are
still
going
to
need
to
be
explored
with
these
ensemble
methods
are
the
use
of
the
criteria
that
you
supply
to
the
to
the
redistricting
plan
or
to
the
ensemble
plans
that
you're,
creating
that
you
know
choices
about
how
to
balance
those
interest
choices
about
how
to
control
for,
for
example,
for
incumbents
or
the
level
of
population.
Deviation
that
you
allow,
or
you
know
how
to
control
for
voting
rights,
act.
Compliance
how
to
how
to
look
at
other
political
questions
like
communities
of
interest.
C
There
are
a
number
of
these
types
of
techniques
or
types
of
considerations
that
legislative
bodies
do
consider
that
can
affect
these
results.
So
it'll
be
it'll,
be
interesting
to
see
how
these
plan,
how
these
techniques
develop
in
future
cases,
and
I
for
one,
am
absolutely
looking
forward
to
to
seeing
that
development.
So
with
that,
I
will
turn
it
back
over
to.
A
I'm
gonna
just
hop
right
in
here
for
just
a
moment
and
say
that
we've
got
a
few
questions
and
moon.
It
might
make
sense
for
you
to
address
the
questions
from
senator
clark
sort
of
all
in
a
group
and
if
both
of
you
could
take
a
a
stab
at
the
question
about
independence
that
came
from
richard.
So
moon,
if
you
could
kick
us
off
here,
please
sure.
B
Absolutely
so
some
of
the
questions
are:
what's
the
distinction
between
why
use
ensembles
for
evaluation,
but
not
for
adoption.
What
is
the
distinction
between
the
methods
used
by
the
various
experts
and
what
about
sh
also
know
that
that's
a
question
from
super
deep,
so
we'll
get
to
that
in
a
moment.
So
clark's
third
question
is:
oh
yeah:
will
there
be
ensembles
ready
for
all
the
states
right
after
the
data
drop?
B
So
let
me
try
to
take
those
together
so
firstly,
so,
as
you
just
heard,
how
was
what
ended
up
happening
in
this
north
carolina
case
was
that,
because
of
the
compressed
timeline
in
the
remedial
phase
for
finding
new
plans,
it
would
have
been
hard
to
go
through
the
entire
process
in
the
normal
way,
which,
as
many
of
you
know,
takes
months
and
lots
of
different
deliberation
over
lots
of
different
priorities
and
how
to
combine
them.
B
And
yet
there
was
this
ensemble
of
maps
sitting
there
with
a
bunch
of
maps
that
score
well
in
various
ways.
So
it's
very
tempting
source,
but
but
what
you
heard,
the
distinction
that
you
heard
patrick
make,
I
think,
is
essential
here.
Do
you
use
the
map?
B
Do
you
pick
it
out
of
the
computer
and
adopt
it
or
do
you
use
it
as
a
starting
point,
and
that
I
think,
is
where
you
know
and
in
the
second
part
of
this
our
I
hope
to
talk
a
little
bit
more
about
using
ensembles
outside
of
the
litigation
context,
but
but
that
I
think,
gets
closer
to
the
approach
that
I
hope
that
a
lot
of
you
will
consider
don't
take
a
map
from
the
computer
and
try
to
enact
it
because
you
know
much
better
than
the
computer
does.
B
What
all
the
holistic
human
factors
are
that
go
into
a
good
plan,
but
the
the
ensembles
can
help.
You
find
interesting
starting
points.
If
you
didn't
think
it
was
possible
to
do
this
and
that
at
the
same
time
the
computer
might
be
able
to
show
you
a
way
to
do
that
and
then
you'll
look
at
that
map
and
be
like
no.
I
wouldn't
want
to
split
this
down
here
and
you'll
modify
it
right.
B
B
Will
there
be
ensembles
ready
in
all
50
states
within
two
weeks
of
the
census,
data
drop
yes,
and
actually
our
my
lab
has
software
out
where
you
can
already
start
drawing
maps
based
on
the
previous
census.
Geography
and
that'll
be
updated
as
soon
as
the
new
geography
is
available,
which
should
happen
late
this
month
and
into
next
month.
B
I
here
it
is.
B
Right
so
I
talked
about
these
sort
of
petri
dish
methods
or
what
you
might
call
agglomerative
methods,
and
so
in
in
a
lot
of
the
chen
testimony
in
the
past
in
the
last
cycle,
he
was
using
agglomerative
methods
like
this
and
then
on
this
slide.
I
showed
you
flip
chains
where
you
change
one
unit
at
a
time
versus
these
recombination
chains
which
make
bigger
steps.
B
So
a
lot
of
mattingly's
testimony
was
based
on
flip
chains,
though
the
work
that
my
lab
has
done
has
developed
these
recombination
chains,
but
the
differences
are
disappearing,
so
the
latest
work
from
both
chen
and
mattingly
has
switched
over
to
recombination.
I
think
this
is
probably
becoming
the
new
standard
right
and
actually,
if,
if
I
could
I'll,
just
use
that
opportunity
to
get
into
the
next
question
really
briefly,
which
is
what
about
district
shapes?
Why
is
it
that
the
recombination
plan
that
I
showed
you
in
pennsylvania
was
producing
compact
districts?
B
That's
one
of
the
things
that
makes
it
pretty
you
know.
Sort
of
well
suited
for
redistricting
analysis
is
that
the
algorithm
underlying
recombination
upweights
compact
plans
automatically,
not
because
you
tell
it
to
because
of
the
method
that
it
uses
to
split
districts.
It
promotes
compactness
internally,
whereas
the
agglomerative
and
the
flipped
chains
had
to
layer
in
compactness
metrics
on
top
so
yeah.
I
think
that's
one
of
the
reasons
that
recombination
is
starting
to
be
adopted
by
all
these
different
groups
that
are
doing
this
work.
B
Sure,
I'll
start,
just
by
saying
nothing
about
these
methods,
is
fundamentally
binary.
It's
the
voting
patterns
that
I
was
showing
you
that
are
largely
binary,
but
when
you
have
a
voting
pattern
in
which
there
are
large
numbers
of
votes
for
more
than
two
candidates,
these
these
ensemble
methods
pick
that
up
just
fine.
So
let
me
emphasize
that
the
ensemble
methods
show
you
how
to
divide
the
geography
into
pieces.
The
voting
pattern
is
whatever
it
is,
and
the
thing
that
you
measure
the
summary
statistic
that
you
measure
over
the
districts.
B
It
can
be
anything
you
want,
it
can
be
the
vote
share
or
as
we'll
get
to
in
the
second
part.
It
can
be
the
county
splitting
it
can
be
the
shape
you
can
measure
anything
you
want,
and
so
there's
nothing
fundamentally
democratic
and
republican
about
it.
It's
just
that's
the
way
it
got
applied
in
these
court
cases
in
the
last
cycle.
C
Right,
yeah,
let
me
just
yeah.
Let
me
just
briefly
add
to
that
point
by
saying
that
most
of
the
virtually
all
of
the
uses
that
I've
seen
litigation
and
that
in
the
literature
that
have
studied
that
have
applied
these
techniques
to
redistricting
problems,
they
look
at
actual
election
results.
They
don't
look
at
party
affiliate.
You
know
regis
voter
registration
data,
that's
that
is
not
seen
as
the
gold
standard
for
assessing
partisan
behavior.
C
You
have
people
that
maybe
they
don't
register
as
a
democrat,
but
they
vote
85
percent
of
the
time
as
democrats
and
and
whether
you're
studying
endogenous
races
or
you're
studying
the
other
technique
that
is
used
is
you
know:
you'll
create
this
partisan
index
where
you'll
look
at
how
units
of
geography
vote
across
maybe
six
or
seven
different
elections,
average
them
together.
You
create
a
score.
The
point
is
that
they're,
looking
at
actual
voting
behavior
of
the
voters
within
the
jurisdiction.
C
So
you
know
in
the
typical
environment
where
you
have
the
two
major
parties
are
accounting
for
95
or
more
of
the
vote
share,
and
then
maybe
you
have
a
minor
party
with
two
percent
or
something
like
that.
Sometimes
those
minor
parties
will
be
dropped
and
we'll
just
look
at
what's
called
the
two-party
vote
share,
but
the
differences
are
relatively
minor.
C
So
I
just
wanted
to
add
that
point.
Let's
see,
I
guess
I
can
go
on.
We
talk
about.
I
think,
there's
a
question
here
about
you
know,
can
a
let's
see
here,
you
know:
can
a
districting
ensemble,
based
upon
limited
neutral
criteria,
be
used
to
identify
the
existence
of
the
state
political
geography
that
would
lead
to
a
lack
of
proportionality
election
results?
Well,
there
are,
I
mean
I
think
you
can
look
at.
I
mean
frankly,
some
of
the
initial
work
by
doctors,
rodney
and
chen.
Does
precisely
this.
C
They
look
at
you
know:
they've
applied
their
techniques
to
his
earlier
techniques
anyway,
to
I
think
florida
and
several
other
states
and
concluded
that.
Yes,
indeed,
that
geographic
concentration
of
democratic
voters
does
lead
to
a
difference
between
proportional
representation-
and
you
know
the
the
actual
election
results.
C
Other
political
science
literature
has
discussed,
what's
called
a
winner's
bonus
where,
when
you
have
to
do
the
impact
of
a
single
member
district
system
where
you
can
have,
if
you
look
at
the
vote
share,
you
might
have
say,
party
x
gets
55
percent
of
the
vote
share.
They
might
very
well
end
up
with
60
or
more
of
the
seats
because
of
the
impact
of
the
single
member
district
system
of
election.
C
So
there's
there
are,
you
know,
bases
upon
which
you
can
show
a
deviation
from
strict
proportional
representation
on
a
map,
but
I'll
I'll
stop
talking
there
and
turn
that
one
over
to
moon
too.
I
think
she's,
probably
more
qualified
than
me
on
that.
B
Well,
you
know
just
I
I
just
want
to
agree
with
you
on
both
points.
You
just
made
one
it.
You
absolutely
have
to
look
if
you,
if
you
want
to
understand
anything,
partisan
or
anything
competitiveness
and
any
of
those
things.
You
really
need
to
look
at
the
votes,
not
not
try
to
identify
people
by
their
registration,
but
look
at
actual
voting
patterns.
I
think
that's
essential
for
this
kind
of
analysis
and
the
political
geography.
That's
exactly
the
strength
of
this
method.
B
Right,
whereas
10
20
years
ago,
people
were
very
oriented
to
that
winners.
Bonus
point
of
view.
That
said,
maybe,
if
you
get
this
much
vote
share,
you
should
get
this
much
seat
share.
The
the
whole
point
of
the
ensemble
method
is
to
suck
the
should
out
of
that
and
say
well,
look
what's
actually
possible,
given
where
people
live
and
given
where
the
votes
fall.
How
could
you
actually
divide
them
up,
not
just
kind
of
how
do
you
wish
you
could,
but
what's
actually
possible?
That's
the
strength
of
this
method.
A
B
All
right
part
two,
so
here
I
want
to
just
briefly
talk
about
so
the
focus
there,
because
we
were
looking
at
some
of
the
recent
litigation.
It's
partisan
gerrymandering
cases,
as
you
heard,
were
where
a
lot
of
these
ensemble
methods
got
traction,
but
right
now
we're
we're
thinking
about
drawing
new
maps.
So
how
does
how
does
ensemble
analysis
play
well
with
other
kinds
of
tests
and
metrics
and
how
might
it
actually
help
you
in
the
process
of
drawing
that's
what
I
want
to
say
a
few
things
about
now:
okay,
so.
B
They're
not
just
for
lawsuits.
I
I'd
like
to
talk
a
little
bit
about
how
they
can
measure
the
impact
of
different
roles.
I
wrote
here
in
the
reform
context,
but
also
just
in
the
line
drawing
context
as
you
as
you
sit
down
to
draw
fresh
lines.
How
can
ensembles
help?
So
I
want
to
say
something
about
how
they
can
help
you
address
questions
like.
Can
you
write
a
good
competitiveness
role
in
your
state,
one
that
won't
have
unintended
consequences,
one
that
will
promote
various
of
the
attributes
that
you
seek
voting
rights
opportunity?
B
Are
you
sacrificing
voting
rights
opportunity
on
the
altar
of
county
splits?
In
other
words,
does
pursuing
one
goal?
Cost
you
in
terms
of
another
and
and
the
question
that
we
just
talked
about
one
moment
ago.
How
does
the
human
geography
of
where
the
votes
actually
fall,
restrict
your
ability
to
do
various
things
that
you
might
seek
to
do,
such
as
producing
proportional
outcomes?
B
Okay,
so
you've
already
been
hearing
at
this
ncsl
in
a
previous
ncsl
sessions,
lots
of
talk
about
the
criteria
about
various
kinds
of
scores
and
tests
and
metrics,
so
without
rehearsing
them
all
I'll.
Just
remind
you,
even
just
when
it
comes
to
partisanship,
there
are
so
many
there's
proportionality.
B
It
has
its
ideal.
Maybe
the
vote
share
should
equal
the
seat
share
that
would
be
proportionality
or
there's
the
efficiency
gap.
Did
one
party
waste
a
lot
more
votes
than
the
other,
and
there
the
ideal
would
be
that
the
parties
waste
the
same
number
of
votes
so
called
there's
something
called
the
mean
median
gap.
B
B
One
of
the
problems,
though,
is
exactly
what
was
just
kind
of
referenced,
which
is
it's
not
always
possible
to
do
the
thing
that
you
might
think
is
ideal
right,
just
based
on
where
people
live.
You
might
not
always
get
there,
but
in
fact,
if
you
try
to
prioritize
other
things
like
community
integrity,
maybe
that
pulls
you
even
farther
away
from
the
putative
ideal.
So
that's
the
kind
of
thing
that
ensembles
can
help.
You
understand
here's
a
quick
example,
not
a
partisan
example,
but
county
splits
and
compactness.
B
So
as
part
of
the
multi-year
process
in
virginia
to
get
a
new
constitutional
amendment
in
place,
a
lot
of
questions
were
flying
around
about
how
different
criteria
might
interact.
That's
one
of
the
things
that
my
lab
took
up
to
study,
and
so
we
were
asked
to
look
at
the
question:
can
you
preserve
counties
in
virginia
without
giving
up
compactness,
or
are
these
two
priorities
at
odds
with
each
other?
B
B
So
I
can
build
a
collection
of
plans
in
ensemble
that
doesn't
pay
any
attention
to
county
lines
or
I
can
wait
it
towards
county
preservation
and
then
I
can
ask
what's
the
impact
on
compactness
of
ignoring
versus
paying
attention
to
county
lines
and
what
you
see
is
in
this
particular
study
that
we
did
preserving
counties
improved
the
compactness
score.
Measured
this
way,
that's
also
true
for
other
ways
of
measuring
it.
B
B
B
B
If
you
expected
a
bell
curve
centered
at
zero,
that's
just
not
what
it
looks
like
right,
and
so
in
fact,
the
landscape
of
possibility
in
pennsylvania,
given
current
voting
patterns
and
current
conditions
is
that
virtually
every
plan
you
can
find
is
going
to
have
what
is
labeled
a
republican,
favoring
efficiency
gap.
So
what
that
might
tell
you
to
do,
is
you
know?
Take
that
eg
equal
zero
ideal
and
it's
probably
not
the
right
standard
to
have
in
mind
but
as
to
what
is
the
right
standard
to
have
in
mind.
B
That's
a
question
for
deliberation
right:
okay,
another
example
partisan
symmetry
in
utah.
So
this
was
of
interest
to
us
in
my
group,
because
utah's
2018
reform,
voter
initiative
included
reference
to
partisan
symmetry
as
a
consideration
that
a
possible
new
commission
should
keep
in
mind.
So
we
took
a
look
at
what
would
happen
if
you
try
to
impose
partisan
symmetry
standards
in
utah.
B
B
This
is
in
a
circumstance
where
recent
election
results
are
something
like
70
30..
So
that's
that's
a
bit
of
a
surprise
that
to
look
fair
to
the
symmetry
scores,
you
need
to
lock
out
democrats.
So
this
is
what
I
would
call
an
unintended
consequence
and
it's
the
kind
of
thing
that
you
can
only
find
when
you
actually
do.
The
modeling
right,
if
you
just
sit
there
and
think
symmetry
sounds
good,
that's
one
thing
and
if
you
actually
look
at
what's
possible,
it
gives
you
a
different
view.
B
Okay,
so
the
main
points
that
I'm
trying
to
make
here
in
this
second
part
is:
what
do
these
neutral
ensembles
these
ensembles,
that
aren't
trying
to
gerrymander
one
way
or
another?
What
do
they
do
for
you?
Well
they're,
responsive
to
the
framework
of
roles
that
you
set
up,
so
you
get
to
say-
and
I
think
this
is
important-
these
are
modeling
techniques.
B
These
aren't
do
your
districting
for
you
techniques,
they
say
if
you
set
up
these
rules
and
goals,
here's
a
range
of
what's
possible
and
if
you
set
up
these
rules
and
goals,
here's
a
range
of
what's
possible
and
that
can
help
you
actually
articulate
your
criteria
and
your
prioritization
of
those.
So
they
can
help
you
explore
trade-offs.
They
can
help
you
discover
on
something
that
sounds
innocuous
and
might
backfire,
and
so
I
think
you
know
it's
it's
for
me.
B
It's
a
pleasure
to
get
to
talk
to
such
a
large
group
of
folks
who
are
going
to
get
to
actually
be
drawing
these
lines.
I'd
like
to
say,
ensemble
analysis
is
your
friend:
it's
not
just
for
litigation.
It
can
help
you
do
a
good
job,
making
the
rules
drawing
the
lines
and
making
sure
your
plans
live
up
to
your
own
rules.
C
C
All
right,
so
some
of
the
metrics
that
that
we
just
heard
about
have
been
with
us
for
a
very
long
time,
partisan
symmetry
was
was
once
viewed
and
they
still
be
viewed
as
sort
of
a
a
great
way
to
look
at
one
way
to
look
at
the
fairness
of
a
plan
you
know
sort
of.
C
If
you
put
republicans
in
democrat
shoes,
you
flip
the
you
you
flip
the
results
around
you
know,
is
it
more
or
less
fair
to
one
party
or
or
the
other,
those
techniques
date
back,
the
modern
techniques
date
back,
at
least
in
1992,
if
not
before
you
know,
we
saw
the
the
rise
of
the
efficiency
gap,
starting
in
2015
in
the
wisconsin
case,
but
we,
you
know
in
the
last
partisan
gerrymandering
case,
that
you
know
that
we
had
in
north
carolina.
C
See
the
efficiency
gap
make
an
appearance,
so
that
was-
and
you
know,
chief
justice
roberts
refer
to
some
of
these
techniques
as
sociological
gobbledygook.
During
the
oral
argument,
I
think
that's
you
know
it
may
impact
their
use
in
in
litigation.
There
was
actually
a
newer
metric
which
which
moon
didn't
talk
about
called
the
declination
method,
which
is
another
way
to
look
at.
B
C
In
her
presentation,
because
I
think
they're
actually
very
very
valuable,
you
know
one
of
the
the
weakness
of
the
ma
of
some
of
the
metric
techniques
that
have
been
pointed
out.
Some
of
the
in
many
of
these
cases
are
the
fact
that
they're
not
based
on
the
the
range
of
what's
possible.
C
They
don't
look
at
the
facts
on
the
ground,
and
so
one
of
the
the
use
of
some
of
these
sampling
techniques
that
you
were
described
as
it's
an
attempt
to
define
the
range
of
of
possibilities
and-
and
that
is
important
for
any
any
number
of
reasons
and-
and
certainly
you
know-
is-
is
helpful
at
its
core
redistricting
is
a
multimodal
optimization
problem
that
political
actors
or
policy
actors
are
trying
to
answer
you're,
always
weighing
different
and
sometimes
competing
and
sometimes
not
competing
depending
on
how
it
may
be
goals.
C
You
can
look
at
you
know
we
talk
about
preservation
of
communities
of
interest
that
can
take
any
number
of
forms.
You
know
we've
seen
fights
in
in
in
states
about
you
know,
for
example,
if
you
have
a
particular
county,
maybe
an
urban
county,
maybe
a
county
of
the
large
military
base.
Is
it
better
for
that
county
to
be
split
and
to
have
two
congressional
representatives
defending
its
interest
in
washington,
or
is
it
better
to
have
one
you
know?
Should
certain
neighborhoods
be
preserved
and
violence
should
neighborhoods
be
split?
C
You
know
if,
if
you
are
a
state
and
you're
faced
with
having
a
reduction
in
the
number
of
districts,
there's
some
states
some
folks
listing
and
today
will
be
in
that
position,
and
nobody
is
retiring
that
year
from
congress.
You
know
who
who
gets
paired.
You
know
what
what
reps
you
know,
what
members
of
congress
or
what
state
representatives
may
be
having
to
get
paired
together?
Where
does
the
seat
collapse
happen
or
where
does
the
seat
gain
happen
in
other
states?
C
C
But
you
know
one
can
also
see
where
those
sampling
techniques
can
also
be
used
to
evaluate
the
impact
of
some
of
those.
Some
of
those
judgment
calls.
I
know
there
was
some
discussion
about.
You
know
competitiveness
as
a
as
an
optimization
method.
There
is
some
disagreement
in
the
political
science
literature
over,
even
whether
that
is
you
know
whether
you
necessarily
want
to
draw
a
plan.
That's
optimized
for
maximizing
competitiveness.
C
Again,
that's
a
that's
a
political
science
and
a
political
consideration
that
some
of
these
modeling
techniques
you
know,
may
help
answer
and
will
certainly
be
a
an
issue
that
that
will
be
will
be
litigated
in
and
that's
something
that
we'll
all
have
to
kind
of
be
facing
in
the
next
cycle.
C
So
that
is
really
what
I've
got
on
on
on
this.
This
is
all
very.
These
are
very,
very
interesting
techniques.
I
think
the
redistricting
cycle
you
know
coming
up
that
that
all
of
us
are
about
to
embark
upon.
I
think
we
can
expect
to
see
these
techniques
being
used
and
and
public
during
public
comment
and
during
other
other
things
during
the
redraw
process
and
it'll
be.
It
should
lead
to
an
interesting,
an
interesting
process
and
one
we're
all
looking
forward
to
seeing.
A
Right
when
we
do
have
some
questions
and
I'm
going
to
ask
you
moon
to
start
with
the
question
about
differential
privacy
and
then
patrick,
if
you
want
to
share
on
that,
and
that's
because
ncsl
has
some
concerns
about
differential
privacy
and
the
quality
of
the
data.
We're
not
quite
sure
what
it's
going
to
mean.
If
you
can
offer
any
thoughts
on
that,
that
would
be
just
great.
B
Oh
absolutely
so,
actually
my
lab's
got
a
big
differential
privacy
project.
We've
been
working
on
for
a
year,
separate
topic
from
ensembles,
but
happy
to
talk
about
it
and
folks
feel
please
feel
free
to
get
in
touch
with
me.
If
you'd
like
to
hear
more
more
detail,
but
let
me
give
you
a
little
bit
of
an
overview.
So
what
differential
privacy
is
a
method
that
the
bureau
is
going
to
use
to
privatize.
B
Its
aggregate
aggregate
counts
right,
and
that
means
that
every
number
that
you
see
of
how
many
of
a
particular
kind
of
person
live
in
a
particular
geography,
is
going
to
have
some
noise
added
to
it,
and
that
definitely
sounds
scary,
but
first
of
all,
it's
necessary
to
protect
from
reconstruction
attacks
where
people
with
powerful
computers
can
learn
the
entire
person
by
person
database,
that's
the
risk,
and
secondly,
after
a
year
of
studying
it.
B
Actually,
I
think
we're
the
only
group
outside
the
bureau
who
can
run
the
bureau's
top
down
code
like
we
went
in
there
github
and
we
got
up
and
we
can
actually
run
their
code.
So
come
talk
to
us.
If
you
want
to
hear
a
little
more
about
how
dp
is
going
to
play
out,
but
it
has
nothing
to
do
with
it's
kind
of
orthogonal
to
this
ensemble
conversation
so-
and
I
think
this
also
speaks
to
a
few
more
of
the
questions
I
was
seeing
in
the
q.
B
It
just
looks
at
the
ways
to
divide
up
the
the
people
now,
there's
a
small
caveat
to
that,
maybe
large,
which
is
that
we're
used
to
the
idea
that
population
balancing
is
paramount
right,
that
a
lot
of
the
decisions
that
you
make
are
driven
by
this
one
person,
one
vote,
maybe
even
zero
balancing
for
congressional
districts.
So
it's
it's
true
that
if
you
shift
the
population
counts
on
the
units,
then
what
it
means
to
be
population
balanced
moves
a
little
bit,
but
typically
the
way
these
methods
work.
B
Is
you
you
build
lots
of
you
build
these
big
ensembles
that
have
maybe
one
percent
population
deviation
and
then,
if
you
want
to
tune
that
to
perfect
balance,
you
typically
do
that
with
the
smaller
geography.
That's
something
that
you
might
do
by
hands.
That's
not
something
that
we
seek
to
do
in
this
in
this
method.
So
so
the
differential
privacy
should
have
no
impact
at
all
on
how
these
how
these
methods
work
and
separately.
B
I
think
that
the
whole
redistricting
community
should
be
a
little
bit
less
afraid
of
differential
privacy
than
folks
have
been
because
a
lot
of
our
work
on
that
suggests
that
for
voting
rights,
act,
enforcement
and
many
of
the
other
things
that
exact
counts
are
used
for.
Differential
privacy
is
not
as
big
of
a
threat
to
the
sort
of
fidelity
of
the
signal
to
our
ability
to
pick
up
racially
polarized
voting
and
so
on.
B
It's
not
as
big
of
a
threat
as
it
may
have
seemed
so
more
on
that
another
time
I
see
a
few
questions
that
that
I
thought
might
be
good
to
address.
So
one
tony
fairfax
mentions
a
question
about
the
code
itself,
so
have
the
experts,
computer
source
codes
been
challenged
in
courts
to
have
errors
or
nefarious
bias?
So
that's
something
that
I
want
to
say.
I
I
just
think
is
so
important.
B
If
you're
doing
this
kind
of
algorithmic
redistricting,
you
really
want
your
algorithm
out
there
for
people
to
examine
and
that's
been
one
of
the
limitations.
Is
that
a
lot
of
the
court
cases
the
code
isn't
shared
and
only
gets
shared
sort
of
in
a
short
compressed
time
period,
with
the
experts
of
the
other
side
of
the
litigation.
It's
much
healthier.
B
If
we're
trying
to
do
good
science
for
us
to
put
our
code
out
there
for
vetting
and
for
contributions
and
for
sharing,
and
so
in
particular,
the
code
base
that
I
was
talking
about
is
totally
public
and
we
have
peer-reviewed
articles
showing
kind
of
that
it
samples
in
the
way
that
we
claim
that
we're
able
to
get
these
these
kinds
of
heuristics
that
tell
us
that
we're
sampling
from
a
target
distribution
so
yeah
just
to
agree
with
you.
B
I
think
that's
it's
really
important
to
take
the
science
and
not
just
have
the
science
be
debated
in
court,
but
to
make
sure
that
the
science
is
debated
in
kind
of
a
peer-reviewed
way
where
the
open
source
community
can
take
a
look
at
what's
going
on
in
in
the
code
and
in
the
algorithms.
All
right,
I
don't
want
to
hog
the
time.
So
let
me
pass
it
back
to
patreon
all
right.
C
Well,
thank
you
very
much
and
I
I
do
want
to
pick
up
on
the
point
about
the
source
code
in
in
court
cases
that
one
of
the
challenges
that
that
you
have
and
when
you're
litigating
these
cases
is
that
they're
oftentimes
brought
particularly
when
you're
on
the
defense
side,
which
the
state
legislatures
and
and
other
folks
on
this
on
this
zoom,
are,
are
going
to
be
in
that
position
as
there's
oftentimes
a
highly
compressed
time
frame,
and
you
know
to
evaluate
and
to
to
to
test
these
these
plans.
C
And
these
simulations,
and
in
many
cases
you
don't
have
the
source
code
available
or
you
get
it.
But
you've
only
got
two
weeks
to
look
at
it
or
three
weeks
to
look
at
it.
And
you
know
that
does
impose
certain
limitations,
especially
because
there
are
a
number.
The
choices
that
go
into
building
these
ensemble
models
can
oftentimes
matter
very
much
to
the
outcome,
and
so,
if,
in
an
ideal
world,
there
would
be
sufficient
time-
and
hopefully
courts
in
the
future,
will
recognize
this
and
will
allow
for
greater
time
for
these.
C
These
ensemble
methods
to
be
deployed
and
to
be
studied
more
thoroughly
because,
again
those
those
input
parameters
can
matter
a
great
deal
to
the
outcome
and
how
you
weight
things
and
what
factors
you
look
at,
what
you
control
for
and
all
these
things
can
have
a
tremendous
impact
so
that
that's
certainly
something
that
someone
that
litigates
these
plans
and
has
litigated
plans
is
something.
C
I
would
certainly
like
to
see
so
there's
a
question
that
says:
how
do
you
recommend
that
our
excision
can
ensemble
techniques
be
used
to
evaluate
you,
know
racial
balance
in
addition
to
partisan
balance
or
other
criteria
like
incumbency
production,
so
in
the
modeling
and
the
techniques
that
I've
encountered
so
far,
they've
not
been
applied
to
address
racially
polarized
voting
and
and
those
types
of
considerations.
C
Typically,
what
happens
is
if,
if
either
a
court
or
voting
jurisdiction
is
deemed
a
certain
district
to
be
majority
minority
or
that
district
exhibits,
those
criteria,
whether
it
was
deemed
that
way
or
not
or
in
the
case
of
north
carolina,
where
a
federal
court
had
imposed
a
remedial
plan
to
remediate
alleged
violations
of
the
voting
rights
act
and
drawn
districts.
Those
districts
were
frozen
or
those
districts
were
somewhat
limited.
We've
seen
in
other
cases
where
the
use
of
of
certain
voting
rights.
Analysis
of
you
know,
potential
comparative
districts.
C
You
know
the
computer
was
instead
instructed
to
draw
a
district
that
has
certain
floor
of
african
american
voting
age
population.
You
know
located
in
one
unit
of
geography,
so
there
are
different
ways
that
that
technique
has
been
used
as
far
as
incumbency
protection.
C
Look
that
can
be
looked
at
multiple
ways:
it's
not
just
mostly
it's
used
by
just
making
sure
that
you
don't
have
two
incumbents
in
the
same
district.
It's
called
double
bunking.
You
try
to
avoid
that,
but
many
people
argue
that
the
better
way
to
look
or
a
different
way
to
look
at
it
is
preserving
the
cores
of
districts.
It's
not
just
making
sure
you
have
two
incumbents
in
the
same
district,
it's
making
sure
that
that
incumbent,
you
know,
retains
a
certain
amount
of
the
voting
population
that
placed
that
incumbent
into
office.
B
Yeah,
that's
been
something
of
great
interest
to
to
me
and
some
of
my
collaborators
lately-
and
so
I
think
I
mentioned
this
earlier,
but
in
a
later
session
today
at
2,
45
hirsch,
and
I
will
be
talking
about
new
work,
how
that
shows
you
ways
to
take
electoral
history
and
effectiveness
for
minority
voters
into
account
when
you
build
ensembles
so
that
you
can
say
that
you're
building
an
ensemble
of
largely
vra
compliant
plans
and
weeding
out
most
of
the
vra
violative
ones.
B
B
Just
another
point
of
agreement
with
with
what
you're
hearing
you're
really
hearing
me
and
patrick
agree
on
a
lot
of
things,
two
weeks
to
review
source
code,
that's
not
commented
and
with
no
unit
tests
and
it's
just
not
enough
to
know,
what's
really
going
on
right,
and
so
I
think,
there's
an
instructive
comparison
here
for
those
of
you,
who've
been
doing
redistricting
for
for
decades.
In
many
cases
think
about
racially
polarized
voting.
There's
a
great
comparison.
B
There
there's
a
there's
a
time
several
decades
ago,
when
you'd
have,
when
you
had
your
experts,
talking
about
polarization
they'd,
be
using
competing
methods,
they'd
fight
about
methods
about
statistical
methods,
and
then
gary
king
and
collaborators
came
along
in
the
90s
and
the
early
2000s,
and
provided
not
only
a
different
statistical
framework
but
open
source
software
for
actually
carrying
it
out
and
what
followed
that
is
largely
and,
of
course,
experts
still
fight
about
racial
polarization,
but
they
no
longer
fight
about
the
techniques
and
they
by
and
large,
even
use
the
same
software
and
that's
really
like
progress
in
in
the
redistricting
world.
B
When
you
can
really
understand
and
use
the
same
assumptions
and
then
sort
of
move,
the
shift,
the
argument
to
whether
you're
looking
at
the
right
data
but
not
have
to
also
argue
about
methods.
So
I
think
that's,
that's
that's
quite
healthy.
A
question
that
I
saw
come
up
in
the
q.
A
was
about
city
level
redistricting
and
about
what
level
of
mathematical
or
technical
capacity
is
required
to
use
these
tools.
B
So
there
are
a
couple
different
kinds
of
tools.
One
is
the
tools
to
draw
your
own
districts
and
there
are
just
numerous
groups
that
have
put
out
open
source
public
software
for
district
drawing,
and
you
know
my
group,
for
instance,
we
have
something
called
districtor.org
that
you
could
check
out
and
draw
your
own
districts.
The
level
of
technical
capacity
needed
there
is
like
virtually
zero.
You
can
do
it
on
an
ipad
and
draw
with
your
finger
right
so
that
we
that's
engineered
for
maximum
accessibility
when
it
comes
to
building
ensembles.
B
This
is
I
want
to
emphasize
laptop
computing.
You
know
something
in
the
past
and
even
the
recent
past
people
thought
you
might
need
a
super
computer
to
do
a
good
job
with
ensembles,
but
we
know
now
that
that's
not
the
case.
My
my
little
macbook
air
here
can,
with
some
of
our
fastest
algorithms,
that
we
have
now.
B
I
can
build
a
million
plans
in
eight
seconds
that
was
our
last
benchmark
for
speed
on
a
on
a
you
know,
cheap
little
laptop,
so
the
the
hardware
that
you
need
isn't
isn't
an
obstacle
in
terms
of
technical
capacity.
We
built
our
code
in
python
because
it's
easier
to
understand.
What's
going
on
and
kind
of
easier
to
use,
and
so
I
would
say
if
you
want
to
try
using
our
software,
put
a
person
in
touch
with
us.
B
We
have
a
readme
and
we
can
walk
you
through
and
get
you
to
where
you
can
do
it
yourself.
So
the
the
hope
here
for
for
my
group
and
for
a
lot
of
groups
that
are
working
really
hard
in
the
redistricting
space
is
to
democratize
the
tools
a
little
bit
and
make
it
easier
for
more
different
folks
to
give
it
a
try
to
really
understand.
B
What's
going
on
to
try
different
things,
and
so
I
would
say
that
my
group
in
particular,
but
many
other
groups,
I'm
happy
to
put
you
in
touch
with
folks
in
your
state.
If
you're
interested
in
in
more
local
expertise,
there
really
is
a
distributed
kind
of
collection
of
people.
Who
can
do
this
well,
but
we'd
love
to
to
help
you
get
up
on
these
techniques
yourself.
So
yeah.
A
I
think
I'm
going
to
call
bring
us
to
a
close
at
this
point,
because
this
has
been
a
great
conversation
and
that
seemed
like
a
good
stopping
point.
I
want
to
say
a
couple
of
things
that
I
got
out
of
this,
and
one
is
that
I
think
you've
told
us
that
we
can
all
be
drawing
maps
right
this
very
minute
to
become
familiar
with
the
software,
so
it
would
behoove
us
perhaps
to
try
it
with
the
old
data
and
then
we'll
be
ready
when
the
new
data
comes.
A
I
like
that
you've
mentioned
that
the
audit,
the
class
is
the
session
that's
coming
up
later
on
today,
where
you'll
talk
about
what's
latest,
for
you
on
using
ensembles
for
vra
type
analysis
before
we
get
to
that,
though,
we
do
have
fundamentals
of
map
making
and
there
everybody
has
three
choices,
and
we
have
three
sets
of
of
software
and
drivers.
A
I
guess
I'd
call
it,
and
the
great
thing
about
this
is
that
if
you
watch
one
person
draw
maps
and
you
go
oh
my
gosh-
I
don't
really
understand
why
they
were
doing
it.
The
way
they
did,
you
can
go
back
and
you
can
watch
the
next
one
and
the
next
one.
So
you
actually
get
three
variations
on,
how
the
not
that's,
not
the
software
person,
but
the
lawyer
type
person
is
thinking
about
what
did
they
want
to
to
do?
And
now.
A
My
last
comment
was
that
I
think
I
heard
you
basically
say
that
your
system
can
be
used
to
measure
other
systems
that
the
ensembles
can
give
you
information
about,
proportionality
or
or
partisanship,
and
that
that's
kind
of
cool
that,
instead
of
if
I
I
had
thought
previously,
that
there
were
like
several
tools
in
the
box,
and
yours
was
one
of
them
now,
I'm
hearing
you
say
yours
is
like
an
uber
tool
that
can
be
used
to
look
at
those
others
well
just
to.
B
Be
careful
I'm
trying
to
separate
out
like
building
alternative
districting
plans
from
what
it
is
that
you
measure
about
them
right,
and
so
the
ensemble
generation
is
just
cutting
up
your
state,
your
city,
your
county,
lots
of
different
ways,
and
then,
once
you
have
all
those
alternatives,
then
you
can
lay
that
over
your
data
and
you
can
so
you
can
lay
that
over
your
population
data,
your
electoral
history,
anything
that
you
want
and
ask
questions
about
what
you
tend
to
see.
B
But
but
it's
it's
I
think,
valuable
to
understand,
and
this
is
why
it's
going
to
be
possible
to
create
lots
of
ensembles
before
we
even
have
detailed
numbers
that
we
might
get
later
in
the
summer
from
the
census.
But
if
you
just
have
those
alternative,
partitions,
different
ways
of
breaking
up
your
state,
you
can
lay
that
over
whatever
data
you
have
and
ask
whatever
questions
you're
curious
about,
so
I
wouldn't
call
it
an
uber
tool.