►
From YouTube: Evidence-Based Policymaking
Description
Kristine Goodwin, Director of NCSL’s Center on Evidence-Based Policymaking and Iris Hentze, Policy Specialist- NCSL share best practices and ideas related to evidence-based policymaking. June 26, 2020.
A
Thank
you
for
joining
us
today
for
our
very
last
virtual
learning
opportunity
in
this
covered
19
series
that
the
education
program
has
been
hosting.
My
name
is
Michelle
Eckstrom
and
I'm.
The
group
director
of
the
education
program
and
today
we're
going
to
end
with
a
really
important
opportunity
for
professional
development,
for
you
all
on
the
topic
of
evidenced
and
form
policy
making.
You
know
that
you
have
really
important
decisions
ahead
of
you
and
we
want
to
make
sure
that
you
have
the
tools
that
will
help.
A
You
be
good
consumers
of
information,
as
you
are
looking
for
research
data
and
other
guiding
information
to
make
those
important
decisions.
Today
we
have
joining
us
to
NCSL
staff
who
are
working
on
this
topic
in
NZSL,
so
I'm
delighted
to
have
them
join
us
before
we
get
started.
I
would
just
like
to
remind
everyone
about
the
protocols
for
today's
virtual
meeting.
First
of
all,
please
join
us
by
video
rather
than
by
phone
and
please
be
sure,
to
turn
on
your
camera.
A
It
makes
for
a
more
interactive
experience
and
we'd
love
to
see
your
face
and
be
sure
to
add
your
full
name
to
your
tile,
both
your
first
and
last
name
and
the
organization
that
you
are
from
either
the
agency
or
the
state
or
the
organization
that
you
represent.
Please
be
sure
to
keep
yourself
on
mute
and
less.
We
ask
you
to
unmute
your
minds
and
participate
in
discussion.
You
may
also
virtually
raise
your
hand
to
be
recognized
today
by
by
me,
or
by
any
of
our
speakers
too,
so
feel
free
to
use
that
tool.
A
You
can
type
your
questions
in
the
chat
box
and
we'll
be
monitoring
that
and
we'll
be
looking
for
information
too,
to
ask
our
speakers
so
be
sure
that
you
share
your
questions
there
or
any
information
that
you
would
like
to
share
from
your
state.
Please
do
not
share
your
screen
under
any
circumstances.
We
will
just
ask
that
I
share
my
screen
in
the
moderator
or
I'm.
A
So
let's
get
started
so
today
we
have
joining
us
two
speakers,
Christine
Goodwin,
who
is
a
program
director
here
at
NZSL
and
I,
was
tense.
He
was
a
policy
specialist
here
at
ncsl
and
they
are
going
to
talk
with
us
about
the
work
that
they've
been
doing
on
this
topic
and
share
some
tips
and
tools
and
and
make
this
as
relevant
as
possible
to
the
work
that
you're
doing
in
education
as
well.
So
I'm
gonna
stop
sharing
my
screen
and
Christine.
You
can
feel
free
to
share
yours
at
this
point
and
we
will
get
started.
B
All
right,
Thank,
You,
Michelle
I'm
pleased
to
be
here
and
have
an
opportunity.
Excuse
me
to
join
you
all
today
to
talk
about
evidence-informed
policymaking
and
to
share
with
you
some
of
the
exciting
work
that
we're
doing
at
NCSL
in
this
area.
So,
as
Michelle
said,
my
name
is
Christine
Goodwin
and
I'm
joined
today
by
my
colleague,
iris
NC,
who
is
going
to
walk
you
through
a
little
bit
later,
an
interactive
discussion
that
we
have
planned
to
help.
B
You
kind
of
think
through
some
of
the
questions
that
you
can
ask
as
a
legislator
or
legislative
staffer
when
someone
claims
that
a
program
or
a
policy
is
evidence-based.
So
we
know
we've
heard
from
many
folks
that
the
term
is
used
so
broadly
and
it's
so
ubiquitous
and
in
some
cases,
has
become
a
bit
of
a
buzzword
to
justify
someone's
position
or
someone's
support
for
a
specific
program.
B
So
our
goal
today
is
to
really
try
to
break
it
down
so
that
it
becomes
a
little
more
meaningful
for
you
to
explain
what
it
is,
how
you
can
access
evidence
and
then
how
you
can
use
it
to
inform
the
decisions
that
you're
making.
So
as
we
move
along
today,
I'd
ask
you
to
think
about
what
challenges
you're
facing
in
your
state
regarding
finding
the
right
type
of
evidence
and
how
NCSL
might
be
able
to
help
I'll
be
asking
you
a
bit
later
for
ideas.
B
So
if
you
kind
of
think
about
that
through
the
conversation
today,
that
would
be
very
helpful.
So
I
had
the
opportunity
to
sit
in
on
the
Tuesday
virtual
meeting
that
Michelle
and
her
team
put
together
around
state
budget
revenue
shortfalls
and
the
impacts
due
to
Kovan
19
on
education,
and,
if
you
haven't
had
an
opportunity
already
to
listen.
B
I
would
highly
recommend
that
you
access
that
archive,
and
we
know
that,
on
top
of
the
budget
challenges
that
were
really
highlighted
in
that
session,
we
know
that
you're
also
asking
the
tough
questions
about
how
education
funding
can
support
the
best
outcomes
for
students
during
this
really
difficult
and
unprecedented
time.
So
we've
heard
from
many
legislators
and
staff-
and
this
is
probably
not
a
surprise-
that
if
data-driven
decision-making
was
important
before
2020
hit
it
it's
even
more
important
now
as
you
navigate
through
and
out
of
this
crisis.
B
So
a
couple
of
the
themes
we've
heard
from
members
of
a
workgroup
that
we're
working
with
and
I'll
talk
about
that
in
a
bit
is
that
data
and
evidence
can
really
help
States
get
back
on
their
feet
faster.
So,
while
we're
not
really
focusing
on
Kovan
19
in
this
session,
that's
not
the
emphasis
of
this.
B
So
this
is
sort
of
a
roadmap
of
where
we're
going
today
will
out
by
talking
about
what
is
evidence-informed
policymaking.
Why
it
matters,
then
we're
going
to
share
some
fresh
new
findings
that
we'll
be
publishing
and
very
soon
related
to
the
seven
principles
of
evidence
and
foreign
policymaking,
provide
a
few
state
examples
to
kind
of
flesh
out.
What
does
that
look
like
in
states
and
then
I'll
turn
it
over
to
my
colleague
iris?
B
Who
will
walk
you
through
an
interactive
discussion
around
a
real-life
example
and
then
we'll
take
a
few
minutes
to
talk
about
where
NCSL
is
heading
in
this
space
and
then
I'll
turn
to
you
to
see
how
NCSL
might
be
able
to
help
you
so
before
we
start
I
wanted
to
hear.
Maybe
I
can
get
one
or
two
volunteers
to
talk
to
me
about
your
go-to
sources
of
information.
Where
do
you
tend
to
find
evidence
and
data
to
help
inform
the
decisions
that
you're
making
I've
provided
some
examples
of
information
here?
B
Okay,
well
so
what
we're
gonna
be
focusing
on
today
is
the
second
bullet
around
the
research
about
a
program
or
policies.
Effectiveness
feel
free.
If
you
just
didn't
have
time
to
talk,
you
can
also
put
it
in
the
chat
bar
and
we
can
talk
about
it
throughout
our
discussion
today.
So
what
is
evidence
and
foreign
policymaking?
So
we
tend
to
talk
about
evidence
and
foreign
policymaking
a
little
even
more
so
than
evidence-based
policymaking.
B
To
recognize
that
we
know
that
policymakers
make
decisions
based
on
a
variety
of
information
and
factors,
and
we
realize
that
using
data
and
evidence
is
one
important
part
of
that.
So
evidence-informed
policymaking
really
refers
to
using
the
best
available
evidence
and
data
to
inform
decisions
at
all
stages
of
the
policy
process.
You
can
see
what
those
look
like
in
this
schematic
at
the
right.
B
It
also
helps
refers
to
this
process
of
sort
of
building
information
about
what
we
know
and
building
our
knowledge
to
inform
future
decisions
and
then
finally,
evidence-informed
policymaking
offers
a
framework
and
a
set
of
tools
that
you
can
use
to
allocate
resources
to
achieve
the
results
that
matter
most
gosh.
This
slide
is
a
bit
of
a
bear,
I
realize
but
I'm
adding
it
here,
just
because
these
are
some
of
the
common
terms,
you'll
hear,
and
that
will
even
speak
about
today
and
realizing
that
this
is
simplified.
B
This
is
simplified
version
and
we
can,
if
you,
if
you're
looking
for
additional
information
we
can
provide.
We
can
drill
down
on
that
as
well,
but
I
think.
The
important
point
here
is
that
correlational
evidence
tells
you
whether
there's
a
relationship
between
a
policy
and
an
outcome,
but
not
that
the
policy
caused
the
outcome.
Causal
evidence.
On
the
other
hand,
you've
heard
of
it
can
tell
you
whether
an
intervention
like
a
program
or
policy
is
likely
to
produce
a
specific
outcome.
So
researchers
often
use
randomized
control
trials
or
RCTs
to
demonstrate
this
causality.
B
So
I
thought
it
might
be
helpful
to
sort
of
look
at
an
example
of
what
this
might
look
like.
So
imagine
that
you're
shown
a
study
that
looks
at
whether
smaller
class
sizes
improve
test
scores.
So
an
observational
study
might
look
at
differences
in
class
size
and
then
correlate
those
differences
to
the
test
scores.
Perhaps
concluding
that
lower
test
size
is
associated
with
better
student
outcomes
or
better
test
scores,
but
there's
a
problem
here.
B
The
problem
is
that
there
may
be
other
variables
or
confounding
factors
that
really
get
in
the
way
and
that
kind
of
come
into
play
with
this
relationship.
So
is
the
difference
in
test
scores
really
because
of
class
size,
or
are
there
other
factors
that
go
along
with
simply
just
being
in
a
smaller
class
site
in
a
smaller
classroom
or
a
whole
host
of
other
factors
that
might
come
into
play?
B
So,
in
contrast,
causal
evidence
is
the
strongest
type
of
evidence,
because
you
can
use
it
to
predict
a
policies
outcome
in
a
policies,
impact
and
the
people
it
serves
so
randomized
control
trials.
Our
CTS
are
a
research
method
that
addresses
the
problem
of
other
confounding
variables
by
randomly
assigning
people
to
receive
the
intervention
that
we're
interested.
In
so
back
to
that
class
size
example,
an
RCT
could
randomly
assign
student
students
to
a
classroom
with
say
thirty,
thirty
to
a
classroom
in
another
classroom
with
fifteen,
when
it's
done
right.
B
All
of
the
other
variables
that
could
influence
the
the
test.
Scores
should
be
evenly
distributed
between
the
two
groups.
So
this
allows
us
to
sort
of
isolate
to
better
isolate
what
the
effect
of
the
class
size
would
be
without
the
interference
of
those
other
variables
and
that's
why
randomized
control
trials
are
generally
thought
to
be
more
reliable
than
observational
studies
and
then
finally,
I
have
on
here
systematic
reviews.
They
draw
from
multiple
experimental
studies
and
not
just
one
study
so
because
we
see
these
results
are
playing
out
time
and
time
again.
B
B
So
many
ways
to
think
about
the
tears
of
evidence,
you'll
find
examples
in
state
and
federal
statutes.
So
this
slide
summarizes
how
the
every
student
succeeds
act
or
essa
defines
evidence
as
falling
within
four
tiers.
So
the
top
three
there's
one
two:
three
really
requiring
statistically
significant
effects
on
improving
student
outcomes
and
then
the
fourth
four,
those
models
that
hold
promise,
but
maybe
don't
yet
have
the
evidence
that
would
qualify
for
the
top
three
levels.
B
That's
a
tie,
certain
federal
funding
to
the
use
of
proven
approaches
so
that
that's
another
sort
of
framework
in
that
can
be
helpful
to
think
about
this.
So
where
can
you
go
to
understand
the
evidence
supporting
a
specific
program?
Clearinghouse
databases
can
really
help.
You
understand
what
the
research
is
and
I've
listed
a
few
of
them
here,
for
example,
one
of
them
the
results.
First
Clearing
House
database
has
information
on
over
3,000
programs
in
different
policy
areas,
including
education,
health,
criminal
justice
and
others.
B
What's
nice
about
this,
one
is
that
it
compiles
or
synthesizes
evidence
and
research
from
nine
national
clearinghouses,
and
it
assigns
a
color
coding
based
on
a
program's
effectiveness.
So,
for
example,
you
could
go
into
this
database
and
filter
click
on
education
and
then
I'll,
ask
for
all
of
those
programs
with
the
highest
evidence
of
effectiveness,
and
the
database
would
generate
a
list
of
programs
that
meet
that
meet
that
far.
B
So
what
does?
Why
does
it
matter?
Why
are
we
talking
so
much
about
classifications?
Well,
simply,
it's
important
because
it
can
tell
you
how
much
confidence
that
you
can
have.
If
you
replicated
a
program
or
a
policy,
it
would
achieve
a
similar
outcome
to
what
the
study
found
so
for
lower
quality
evidence.
We
don't
have
much
confidence
that
the
outcomes
will
be
replicated.
C
B
That's
a
great
question
and
I
think
that
that
is
one
of
the
complications.
It's
one
of
the
questions
that
legislators
and
legislative
staff
can
ask
when
they're
really
digging
into
the
evidence,
what
what
are?
What
are
the
sort
of
equity
dimensions?
And
so
that's
something
that
maybe
we
can
also
discuss
a
little
bit
later
when
we
get
into
those
questions.
But
it's
a
really
important
facet.
B
There
are
programs
and
policies
that
don't
have
a
rigorous
research
base
and
effect
that
affect
different
populations,
and
so
there's
sort
of
a
host
of
constraints
that
we
can
get
into
a
little
bit
later
too.
It's
a
great
question,
so
something
important
to
think
about
when
looking
at
these
sort
of
guest
frameworks
is
that
looking
at
evidence
in
this
way
doesn't
need
to
hinder
innovation
of
new
or
promising
programs.
We
know
that
not
every
program
or
policy
has
rigorous
evidence
of
effectiveness,
and
that's
okay.
You
don't
want
to
squelch.
B
You
know
innovation
if
there's
a
new
program
or
a
model
that
shows
promise
that
could
address
your
state
specific
needs,
but
there
are
ways
to
nudge
those
programs
along
so
that
they'd
start
to
develop.
That
research
capacity,
so
that's
why
I
stayed
so
I've,
given
an
example
of
Colorado,
look
at
evidence
on
a
continuum
and
something
lacking
strong
evidence
today
could
really
move
up
this
continuum
so
with
additional
supports
and
new
data
and
evaluation
findings
so
for
models
that
are
lower
on
the
continuum.
B
So,
to
sum
this
all
up,
why
it
matters
using
evidence
and
data
can
help
policymakers,
allocate
resources
to
effective
programs
to
promote
innovation
by
identifying
those
promising
new
and
innovative
models,
and
then,
by
providing
this
kind
of
framework,
this
clear
process
for
how
decisions
are
made.
It
really
offers
an
opportunity
to
promote
transparency
and
accountability
in
terms
of
how
decisions
are
made
and
then,
finally,
by
using
data
to
identify
bright
spots
and
areas
for
improvement.
B
So
now
that
I've
talked
about
what
evidence
is
and
why
it
matters
I'd
like
to
share
some
of
our
new
insights,
about
the
fundamentals
of
evidence
and
what
those
principles
are
in
States
and
what
can
what
principals
can
support
an
evidence-informed
approach?
So
we've
been
working
closely
with
through
the
generous
support
of
our
funder,
the
Pew
Charitable
Trusts
and
a
bipartisan
cross
branch
workgroup
to
do
a
couple
of
things.
B
One
of
the
one
of
the
things
this
workgroup
is
helping
us
do
is
to
define
what
the
principles
for
evidence-informed
policymaking
are
want
to
go
through
those
we're
calling
them
the
ABCs
of
evidence
and
foreign
policymaking.
The
the
group
is
also
advising
the
NCSL
on
a
new
on
the
launch
of
a
new
center
for
evidence
and
foreign
policymaking.
Gonna
talk
about
that
in
a
minute
as
well.
So
just
a
few
background
words
on
the
principles.
They
focus
on
broad
actions
that
states
have
taken
to
facilitate
or
sort
of
pave
the
way
for
an
evidence-informed
approach.
B
B
So
all
told
the
group
has
helped
us
identify
seven
principles
that
support
evidence-informed
policymaking
and
I'll
list
them
here
and
I'll
kind
of
highlight
a
couple
in
particular
and
then
in
a
moment,
I'll
provide
some
examples
to
flesh
these
out
a
little
more.
So.
The
first
example
is
around
agreeing
upon
the
standards
and
the
terms
used
to
describe
evidence.
B
So
you
know
that
there's
this
very
strong
idea
that
implementing
and
sustaining
evidence-informed
practices
over
time
really
requires
the
buy-in
and
the
collaboration
of
folks
across
branches
of
government.
So
legislators,
legislative
staff,
executive
branch,
staff
and
agency
leaders
and
staff
and
other
stakeholders
and
states
have
found
that
really
bringing
people
together
can
help
to
have
these
kind
of
big
conversations
about
setting
priorities
and
identifying
outcomes
that
states
want
want
to
set.
For.
B
The
fourth
principle
relates
to
directing
resources
to
those
programs
and
policies
that
have
that
evidence
base
that
are
backed
by
research
and
then
encouraging
promising
ones
to
build
to
build
their
base,
and
so
we
kind
of
talked
about
that
a
bit
ago
with
the
Colorado
continuum
example.
So
some
states
have
developed
tiered
grant
programs
this
evidence,
continuum
that
I
just
shared
or
other
frameworks
that
may
give
preference
to
programs
with
strong
evidence,
while
also
providing
an
opportunity
for
new
or
untested
programs
to
develop
research.
That
demonstrates
their
impact.
B
B
Clearly,
and
then
just
in
kind
of
some
one
of
the
things
that
we've
heard
is
that
one
of
the
keys
or
lessons
is
that
there
are
many
ways
of
going
about
this.
There's
not
one
way,
it's
not
all
or
nothing,
and
that
the
process
is
really
a
marathon
and
not
a
sprint.
That
change
takes
time,
and
so
the
the
idea
of
the
principles
will
be
that
you
can
kind
of
look
to
see
where
there
may
be
opportunities
to
integrate
data
and
evidence
into
your
own
decision-making.
B
So
I
pulled
out
a
few
examples,
you'll
see
when,
when
the
report
comes
out,
I
think
we
have
maybe
at
least
two
dozen
state
examples,
but
I've
just
pulled
out
three
very
recent
ones.
That
kind
of
animate
the
principles
that
I
talked
about.
Mississippi
is
a
great
example
of
a
state.
That's
updated
evidence
terms
in
statute
and
so
makes
clear
what
we
mean
when
we
talk
about
evidence-based,
research-based
promising
and
other
programs.
B
Alabama
is
a
great
example
of
a
state
that
recognized
the
importance
of
building
consensus
across
government
and
established
a
commission
on
the
evaluation
of
services.
Something
unique
about
this
commission
is
that
it
recognized
the
importance
of
engaging
the
executive
and
legislative
branch
and
both
are
represented
on
the
Commission
and
both
co-lead,
the
Commission.
So
that's
a
really
interesting
example,
and
then
finally,
New
Mexico
is
a
great
example
of
a
state.
That's
worked
to
direct
resources
to
evidence-based
programs
and
policies.
So
it's
there
Senate
bill.
B
So
this
is
their
it's
seven
elements
of
quality
program,
design
and
the
framework
what
I
like
about
it.
It
really
offers
questions
that
policymakers
can
ask
regarding
a
programs
research
base
and
a
state's
implementation.
Our
programs
implementation
plan
so
I'm
going
to
turn
it
over
in
a
moment
to
Iris.
B
But
these
are
some
of
the
types
of
questions
that
may
be
useful
as
a
policy
maker
to
ask
when,
when
you're,
given
up
a
specific
program
to
think
about
and
I
don't
see,
and
there
may
be
an
opportunity
to
sort
of
flesh
this
out
and
add
the
components
of
equity
onto
this
list
of
questions.
So
before
we
move
on
I'll.
Stop
here
to
ask
if
you
have
any
questions
or
if
you
have
any
reactions
to
these
ABCs.
If
anything
is
missing
or
if
you
have
examples
from
your
own
state
that
you
would
like
to
share.
B
Great
well
reach
out
to
me.
If
anything
comes
to
mind,
we
are
in
the
final
stages
of
putting
together
this.
This
report
and
I'd
be
delighted
to
learn,
learn
more
from
from
you
all
so
at
this
time,
I'm
going
to
turn
it
over
to
Iris
who's,
going
to
lead
you
through
an
interesting
conversation
about
questions
to
ask
iris.
C
Thanks
Christine
so
now
I'm
going
to
walk
everyone
through
an
exercise
that
will
hopefully
help
you
Bette
claims
that
you
may
hear
from
your
colleagues
from
policy
makers
or
from
others
in
the
policy
making
process
when
they
say
something
is
quote,
unquote,
evidence-based,
so
the
questions
we'll
be
asking
today
are:
how
is
evidence
being
defined?
Is
it
causal
or
correlational?
How
strong
is
the
evidence?
C
What's
the
goal
of
the
policy
and
how
will
we
know
the
policy
works
once
we
implement
it
and
I'll
be
asking
you
a
couple
of
questions
as
we
go
through
this
exercise
so
feel
free
to
unmute
your
mic
and
shout
out
any
answers
as
you
have
them,
or
you
can
type
them
into
the
chat
box
and
with
that
I
think
I'm
good
for
the
next
slide.
Christine.
C
And
you
all
have
been
such
a
talkative
group
so
far
today,
and
hopefully
this
isn't
too
painful
for
you.
So
let's
set
up
a
scenario
for
this
exercise.
Ever
since
schools
might
remote
last
March,
they
have
had
to
scramble
and
are
doing
the
best
they
can
to
make
sure
they
deliver
an
equitable
quality
education
for
their
remote
learners.
In
our
scenario,
let's
say
that
you
all
sit
on
an
Education
Committee
in
your
state's
legislature.
C
The
agency
wants
you
to
vote
on
their
proposal
for
funding
and
part
of
the
argument
they
offer
to.
You
includes
the
claim
that
evidence
shows
that
investing
more
funding
into
remote
learning
technology
will
improve
student
learning
outcomes
in
our
state,
so
right
away.
Are
there
any
questions
that
come
to
mind
when
you
hear
this
statement
as
a
member
of
your
safe
education
committee?
What
do
you
want
to
ask
the
agency
about
this
Lam.
D
I'll
go
ahead
and
jump
in
I
would
and
looking
back
on
as
I
found
that
slide
helpful
and
sorry
I'll
go
on
camera,
see
it's
all
hi.
My
name
is
Emily
LA
FC
in
New
Mexico.
This
is
my
third
week,
so
I'm
super
new
so
feel
free
to
feedback.
If
this
is
not
the
right
question,
a
good
question
that
comes
to
my
mind
is
what
evidence
exists
to
support
this
proposed
approach,
because
I
think,
with
everything
being
really
new
I
would
be
curious.
C
Yeah,
that's,
that
is
great.
We
definitely
need
to
know
not
only
kind
of
what
the
definition
of
evidence
is
that
they're
using,
but
also
how
do
they
intend
to
measure
learning
outcomes,
and
does
that
mean
test
scores,
or
does
that
mean
engagement?
What
does
that
mean
and
does
anyone
else
have
any
other
questions
that
they
want
to
ask
this
agency
off
the
top
of
their
head?
C
A
C
Great
questions,
yeah,
okay,
those
are
all
great
questions.
Thank
you.
So
much
for
sharing
those
and
letting
me
kind
of
draw
that
out
of
you
Christine.
Could
you
advance
to
the
next
slide
so
now,
let's
get
into
how
we
might
be
able
to
apply
some
of
the
five
questions
that
we
talked
about
earlier,
as
well
as
some
of
these
questions
that
you
all
have
shouted
out
that
the
evidence
shows
investing
more
funding
into
remote
learning
technology
will
improve
student
learning
outcomes.
C
You
know
their
best
friend
told
them
this
really
like
compelling
anecdote
and
that
that's
the
evidence
that
is
helping
them,
bring
this
legislation
forward,
or
maybe
they
have
lots
of
studies
and
they've
kind
of
gone
through
and
made
sure
that
those
studies
match
one
another
in
terms
of
their
their
outcomes.
So
it
could
definitely
span
a
bunch
of
different
types
of
things.
C
So
it's
important
to
to
have
the
same
definition
of
evidence
to
make
sure
everyone's
talking
about
the
same
thing,
and
it's
also
important
to
to
levels
that
kind
of
at
the
beginning
of
considering
a
piece
of
legislation
like
this.
The
second
question
we
might
want
to
ask
is:
is
the
evidence
causal
or
correlational?
C
So
with
our
example,
the
State
Education
Agency
is
telling
us
the
allocating
more
fun
to
remote
learning.
Technology
leads
us
to
improve
student
learning
outcomes.
We
want
to
ask
them.
Does
the
evidence
really
say
that?
Is
there
a
proven
causal
relationship
between
spending
more
on
remote
learning,
technology
and
learning
outcomes.
C
Thanks
Christine,
so
next
up,
the
third
question
we
might
want
to
ask
our
agency,
who
is
requesting
more
funding,
is
how
strong
is
the
evidence
that
exists.
If
the
evidence
really
does
show
that
investing
more
money
into
remote
learning
technology
is
going
to
help
students
in
your
state,
how
many
other
states,
or
even
countries,
have
tried
implementing
a
similar
policy
and
have
gotten
the
same
results?
The
general
rule
is
the
more
times
the
policy
has
been
tried
and
has
produced
the
same
results.
The
more
you
can
really
rely
on
the
evidence
to
say
hey.
C
B
C
A
C
Yeah
I
think
that's
one
of
the
biggest
things
we
really
come
up
against
with
evidence-informed
policymaking
is
there
can
be
examples
or
studies
of
something
working
in
another
state,
but
ultimately
it
concerns
about
the
specific
context.
I'll
always
come
into
play
for
sure
all
right.
Well,
the
next
question
is
another
one
that
seems
obvious
and
it
was
brought
up
before,
which
was
great,
but
again
it
has
everything
to
do
with
the
interpretation
of
certain
words
and
making
sure
that
everyone
is
on
the
same
page
in
terms
of
definitions.
C
So
just
like
how
we
had
to
ask
our
Education
Agency
earlier
about
their
definition
of
evidence.
Now
we
need
to
ask
them
about
their
definition
of
success.
They
claimed
that
putting
more
funding
towards
remote
learning
technology
would
have
learning
outcomes
for
students,
but
do
we
really
know
what
they
mean
by
improved
learning
outcomes
for
students,
and
do
you
all
have
any
thoughts
that
just
come
to
mind
on
what
could
be
considered
improved
learning
outcomes.
C
I'm
thinking
test
scores,
maybe
also
attendance,
maybe
graduation
rates
I'm,
definitely
not
an
education
policy
specialist,
but
there
are
just
so
many
different
ways.
They
could
be
measuring
those
improved
learning
outcomes.
You
can
see
that
it's
important
to
make
sure
that
that
definition
is
kind
of
sussed
out
as
you're.
Considering
a
policy
like
this.
C
So,
finally
and
I
guess
you
want
to
do
you
want
to
go
back
to
the
last
slide
again,
Christine,
sorry
I'm.
Just
on
the
last
question.
Finally,
you
might
want
to
ask
the
agency
that's
coming
to
you
with
the
funding
request
if
they
have
included
or
if
they've
thought
about
including
specific
measures
or
benchmarks
to
track
its
progress,
and
the
agency
is
claiming
that
this
policy
is
going
to
improve
student
learning
outcomes.
C
So
how
can
you,
as
a
legislator,
it's
sitting
on
the
Education
Committee
to
make
sure
that
if
you
vote
to
increase
funding
or
remote
learning
technology,
students
really
are
better
off
for
it?
Another
committee.
We
could
definitely
look
at
other
state
examples
to
see
what
measures
they
used,
but
are
there
any
other
ways
we
could
measure
the
success
of
the
policy
like
this.
If
we
implanted
it,
does
anyone
have
any
measure
ideas.
C
I'm
just
the
last
note
here,
concerns
about
equity
and
implementation
and
impacts
of
a
policy
on
different
diverse
groups
in
a
state
can
really
be
addressed
here
by
legislators
and
others
crafting
this
policy,
and
they
can
make
sure
that
measures
are
put
in
place
alongside
the
policy
to
keep
track
of
the
equitability
of
outcomes,
and
you
can
go
to
the
next
slide,
Christine
perfect,
so
just
to
wrap
up
this
exercise.
We
took
this
example
from
our
recent
meta-analysis
of
over
a
hundred
and
twenty
six
studies
on
education
technology.
C
The
body
of
evidence
that
exists
from
all
of
these
studies
suggest
that
supplying
computers
and
internet
alone
don't
necessarily
improve
student
learning
outcomes
in
the
k-12
setting.
While
expanding
access
to
technology
does
help
improve
computer
skills,
it
doesn't
necessarily
lead
to
improved
grades
or
improved
test
scores.
So
as
members
of
the
Education
Committee
in
our
legislature,
through
this
process,
we've
asked
the
agency
requesting
more
funding
a
number
of
key
questions
about
their
claim
and
we
now
have
a
better
picture
of
the
evidence
that
does
or
doesn't
exist
to
support
their
claim.
B
So
this
is
something
that
we've
been
working
with
this
bipartisan
work
group
over
the
last
several
months,
so
since
December
of
2019
to
kind
of
plan
for
to
identify
what
what
the
mission
and
sort
of
the
vision
for
the
center
is
and
we're
preparing
to
launch
the
center
later
this
summer.
In
conjunction
with
the
release
of
that
report,
that
I
told
you
about
so
we
haven't
named
the
center
yet,
but
we
are
exploring
ideas
and
I'll
just
say
a
few
words
about
what
you
can
expect.
B
So
ultimately,
the
goal
of
the
center
is
to
meet
States
where
they
are
to
improve
the
use
of
evidence
in
state
policy.
Decisions
will
be
working
so
that
the
center
is
is
general
in
terms
of
the
policy
focus
so
just
like
the
principal's
didn't
relate
specifically
to
education
or
health
or
criminal
justice,
but
will
be
working
across
the
organization
to
provide
the
best
assistance
to
States
and
really
to
sort
of
based
on
where
you
want
to
go
with
it.
B
So
I
asked
you
at
the
beginning
what
you
know:
what
challenges
you're
facing
and
how
NCSL
might
be
able
to
help
you're
familiar
I
know
with
the
many
ways
that
NCSL
typically
can
help
you
in
your
States,
including
through
technical
assistance
and
customized
research
and
trainings,
and
resources
and
publications
and
our
digital
offerings,
which
you're
all
familiar
with.
But
I
wanted
to
just
take
last
few
minutes
to
hear
from
you
about
whether
there
are
specific
topics
or
services
or
ideas
that
you
have.
B
That
would
really
help
you
in
your
state,
with
some
of
the
specific
challenges
that
you
are
facing.
So
I'll
open
it
up
to
see
if
you
have
ideas
or
if
there
are
specific
principles
that
you're
especially
interested
in
learning
more
about
we're,
definitely
open
and
we're
taking
notes
on
what
would
be
helpful.
So
are
there?
Is
there
any
anyone
want
to
provide
some
some
information
about
what
would
be
helpful.
B
One
of
the
as
you're
thinking,
one
of
the
things
that
we've
heard
from
our
workgroup
is
that
many
states
do
a
really
great
job
of
training,
but
that
training
is
really
important
and
so
and
that
it's
very
kind
of
customized
training
is
needed
for
there's
one
set
this
sort
of
101
and
then
there's
a
lot
of
you
see
and
recognize
a
lot
of
these
names.
Maybe
it's
the
201
301
so
really
providing
those
kind
of
customized
training
resources
to
help
States
kind
of
advance
along
their
own
evidence.
Continuum.
B
So
I'm
perfect,
so
I'll
give
you
my
contact
information
at
the
end.
If
you
think
of
ideas,
please
just
pick
up
the
phone
and
call
I'd
love
to
talk
to
you
to
learn
more
I'm,
including
some
the
resources
that
I
touched
on
today,
so
both
NCSL
resources
and
then
other
select
resources.
This
is
just
a
small.
You
know
this
is
barely
scratching
the
surface,
but
I
just
included
some
of
those
kind
of
recent,
very
helpful
resources
that
I
think
may
be
a
good
start.
So,
finally,
please
don't
hesitate
to
reach
out
to
iris
or
myself.
B
If
you
have
questions
you
want
to
track
down
any
additional
information
or
if
you
want
to
sort
of
discuss
how
the
center
might
be
useful
to
the
work
that
you're
doing
and
thanks
to
Michelle
and
the
education
team
for
carving
out
a
little
bit
of
time.
I
didn't
realize
this
was
your
last
virtual
meeting.
So
thank
you
for
giving
us
the
opportunity
to
share
the
work
that
we're
doing.
A
Well,
thank
you
for
joining
us.
It
was
such
a
delight
to
have
you
both
with
us
today.
I
just
think
this
is
such
an
important
topic
and
it's
really
key
to
the
work
that
we've
been
doing
at
ncsl.
So
it
is
very
much
a
conversation
we
want
to
continue
and
we
are
hoping
to
do
additional
work
more
in
depth
with
all
of
you
around
this
topic
with
Christine
and
iris
as
well.
So
thank
you
for
joining
us.
This
does
conclude
our
Virtual
Kovac
19
education
series.
We've
now
hosted
20
meetings
for
you.
A
They
all
are
archived
on
our
website
and
are
on
our
YouTube
channel,
so
feel
free
to
send
the
links
to
your
colleagues,
if
you
think
they
might
be
helpful,
revisit
them
when
you
have
information
requests
or
these
topics
come
up
in
your
chambers,
we'd
love
to
think.
Have
you
think
if
these
is
good
resources?
I
might
also
also
mention
that
at
this
point
now
we're
starting
to
pivot
to
more
in-depth
topic,
oriented
meetings.
A
So
we
will
continue
to
engage
you
in
this
virtual
platform
as
long
as
we
need
to
and
really
appreciate
your
time
and
want
to
be
here
as
a
resource
for
you.
So
please
reach
out.
Let
us
know
what
you
need
the
stuff
we
can
be
resource
and
thank
you
all
for
your
time
and
a
big
thank
you
to
Iris
and
Christine
have
a
good
afternoon.
Everyone
all.