►
From YouTube: Torus Community Meeting February 2022
Description
Monthly meeting to discuss Open Learning Initiative's next generation platform. This meeting was the second meeting about the functionality and features for Data, Analytics, and Reporting.
A
B
B
Okay,
it's
12
35
and
I'm
looking
at
the
participant
list,
and
most
of
us
are
here
to
to
hear
the
discussion
more
than
probably
participate,
but
maybe
we'll
get
some
additional
folks
in
as
we
move
along
today's
meeting
is
data
data
analytics
and
reporting
two.
This
is
the
second
meeting
to
dive
into
what
kind
of
analytics
visualizations
are
used
for
learning,
teaching
research
and
this
time
we're
we're
even
going
to
our
plan
to
talk
more
about
data
shop
and
learn
sphere
and
to
find
out
who
makes
use
of
those
tools.
B
B
B
Sorry,
folks,
I'm
just
trying
to
there,
we
go
okay,
so
any
suggestions,
anything
to
get
the
discussion
started,
maybe
talk
about.
Does
anybody
have
experiences
with
not
only
using
or
trying
to
use
data
shop
and
learn
sphere,
but
maybe
working
with
folks
who
needed
to
use
these
these
tools
to
do
some
deeper
analysis
and
what?
What
were
those?
What
were
people
trying
to
accomplish
with
them?.
B
So
we
can
talk
a
little
bit
about
models
at
first
because
norm
and
I
have
talked
a
lot
about
what
we
think
we
want
to
be
able
to
enable
in
the
new
platform
and
one
of
the
ideas
he
has
is
that
we
should
be
able
to
explore
and
use
different
types
of
models.
So
right
now,
as
you
may
know,
oli
has
learning
objectives
and
and
even
sub-objectives
or
knowledge
components
whatever
you
want
to
call
them.
We
call
them
skills
in
the
past
in
taurus,
they're
right
now
called
sub-objectives.
B
I
believe,
if
they're
called
anything,
I
think
they're
just
more
learning
objectives,
but
I've
been
told
that
there
might
be
some
other
models
like
maybe
someone
wants
to
explore
topic
real.
You
know
topic
model
models
instead
of
really
modeling
out
learning,
objectives
and
skills,
maybe
connecting
questions
to
just
topics
and
what
would
that
lose
or
or
provide,
but
that's
just
really
an
example
of
maybe
other
types
of
models
we
want
to
help
others
or
enable
others
to
be
able
to
explore.
C
I
know
that
the
way
that
we're
traditionally
using
the
data
set
is
we're
not
we're,
not
really
concerned
with
learning
curves.
You
need.
You
need
a
certain
number
of
questions.
You
need
a
high
enough
end
to
really
like
build
those
learning
curves
and
when
you
have,
when
you
have
the
kind
of
package
that
we
do,
which
is
which
is
just
a
few
core
questions
that
you
ask
students
with
sort
of
the
embedded
questions
and
then
the
quiz
questions.
This
there's
not
really
a
good
way
to
build
learning.
Curves.
A
C
Rather,
we're
more
concerned
with
tracking
you
know
performance
for
the
checkpoint
quizzes,
so
we're
interested
in
like
the
checkpoint
quiz
performance,
and
then
we
also
match
that
up
with
the
survey
outcomes.
So
we
you
know,
we
look
at
correlations
between
what
people
have
included
for
their
surveys
and
we
also
do
before
and
after
surveys
and
then,
as
you
know,
we
also
look
at
how
that
correlates
with
course
performance,
because
we
have
that
additional
data
set.
B
So
you
do
use
data
shop
data
but
you're
you're,
doing
it
as
a
comparison
of
particular
places
or
or
assets
that
students
interact
with
so
you're,
not
generating
learning
curves
but
you're
you,
you
are
you
saying
that
you
do
use
data
shop,
but
just
for
this
specific
use
case
or
you
get
the
data
like
the
data
shop
export,
but
you
don't
really
or
you
don't
use
data
shop.
You
just
look
at
that
data
in
different
ways.
C
B
B
B
C
Actually,
we
make
sure
that
our
learning
objectives
have
a
one-to-one
match-up
between
the
in
the,
in
course,
formative
piece
and
then
the
summative
piece
at
the
end.
But
then
yes,
I
actually
personally
have
found
the
fact
that
you
have
to
have
a
learning
objective
correspondence
to
be
somewhat
cumbersome
because
there
are.
C
So
the
reflective
questions
have
to
do
with
student
experience
and
those
really
shouldn't
be
matched
up
to
course,
learning
objectives
and
I've
had
to
create
what
I
call
phantom
learning
objectives
that
are
learning
objectives
only
in
the
system
that
students
never
see.
So
yes,
that
I
can
say
that
that
is
cumbersome
to
have
to
create
those
phantom
learning
objectives
that
students
are
not
aware
of,
but
I
need
them.
C
Potentially,
if
you
were
to
say,
like
you
could
say,
like
no
learning
objective
like
if
there's
no
learning
objective
option,
then
that
would
be
nice
like
a
drop
down
menu.
I
don't
know
I.
I
don't
really
know
how
you
envision,
having
that
in
taurus,
but
right
they're,
not
real
learning
objectives,
they're,
simply
placeholders
that
we
have
to
write
that
are
unique.
B
C
I
don't
put
them
at
the
top
of
the
page,
so
students
see
the
three
learning
objectives
for
each
module.
That
are
the
learning
objectives
that
are
at
the
heart
of
the
module
and
that
we
want
students
to
see.
But
I
call
them
phantom
learning
objectives,
because
the
students
never
are
asked
to
look
at
them.
They're.
A
B
C
B
B
B
C
Oh,
yes,
there's
two
different
types
that
we
gather
yeah.
So
the
types
of
sorry,
the
types
of
reflective
questionings
that
we
ask.
Some
of
those
are
intended
for
students
to
do
their
own
reflections
and
there
are
a
lot
of
those
that
we
have
students
do
their
own
reflections
and
articulate
for
themselves
how
they're
currently
using
study
strategies
and
those
are
embedded
at
the
end
of
every
module,
and
then
we
also
sorry.
We
also
have
some
that
are
targeted
at
gathering
feedback
about
kind
of
how
students
feel
about
the
platform
itself.
Yeah.
B
And
the
learning
experience
and
the
content
kind
of
thing
yeah
so
yeah,
it
sounds
like
for
those,
so
even
if
we
got
rid
of
even
if
we
allowed
you
to
attach
to
create
questions
that
weren't
attached
to
learning
objectives,
it
sounds
like
about
half
of
those
are
actually
more
than
halfway
like
three
quarters
of
those.
Whether
you're
gathering
feedback
could
actually
be
tagged
to
some
objective.
C
Maybe
it's
a
delivery
objective.
Well,
the
at
least
the
thing
about
gathering
information
for
iterative
improvement
is
definitely
a
delivery
objective,
but
then
there's
also
is
more
like
a
reflection
objective
for
the
for
us.
But
again,
that's
very.
That
could
be
very
niche
for
our
group
because
we're
doing
metacognitive,
coursework.
B
B
In
this
way?
Are
there
certain
experimental
things
that
they're
doing
that
they
we
just
need
to
like
get
their
thinking
about,
so
that
when
they
get
the
data
on
the
other
side,
they
can
actually
prove
or
disprove
whatever
hypotheses
they
had
going
into
it
into
the
into
the
delivery
of
that
content,
not
just
the
content
itself.
So
I
I
kind
of
like
this
idea
resonates
with
me.
B
Yeah,
so
it's
almost
like
we
were
offering
the
ability
in
our
assessments
now
to
have
those
curriculum
banks
where
you
can
use
tabs
on
those
questions
we
almost
kind
of
want.
Maybe
a
similar
tagging
on
learning
objectives
to
be
able
to
maybe
for
ourselves
in
the
back.
You
know
back
end,
not
making
it
visible
to
students,
but
so
that
we
know.
Oh
these.
This
group
of
learning
objectives
is
for
this
particular
serving
a
certain
purpose,
and
this
you
know
what
I
mean.
C
Yeah,
I
mean
one
one
other
thing
that
we
can
conceptualize
is
it's
not
really.
You
could
even
have
there
be
a
type
of
question.
That's
not
like
a
learn
by
doing
question
with
short
answer
response,
you
could
actually
have
a
different
label
called
just
like
reflective
questions,
a
reflective
response.
C
B
Well-
and
we
want
to
encourage
more
of
that,
because
we
know
it's-
you
know
it's
a
proven
method
that
helps
students
learn
better
going
forward
right.
So
it's
something
we
talk
about
and
we
try
to
encourage
but
you're
right
if
we
had
a
particular
label
or
question
type.
That
is
just
like
reflective
response.
B
C
B
B
Yeah,
I
wish
you
know
what's
funny
about
that.
Is
that's
actually
a
lot
of
the
stuff
I'm
trying
to
get
at
in
these
meetings.
How
are
people
using
the
system
and
what
we
make
available
in
ways
that
we
don't
even
know
right
and
as
a
developer
of
these
types
of
courses,
I've
done
it
a
million
times
like
when
I
had
access
to
the
xml.
B
I
could
use
those
xml
tags
in
various
ways
just
to
make
my
like
okay,
this
isn't
quite
an
example,
but
I'm
going
to
use
that
example
tag
and
to
make
it
look
a
certain
way
and
fit
on
the
page
with
the
other
material
a
certain
way.
You
know
things
like
that
where
we're
using
the
the
semantic
labels
available
in
the
system
really
for
for
styling
and
and
for
styling
purposes,.
C
B
C
I
feel
like
we
always
get
to
like
different
points
like
you
have
really
helpful
kind
of
guiding
questions,
because
I
definitely
don't
think
we
would
have
gotten
to
this
discussion
without
your
help,
and
I
didn't
even
think
about
the
fact
that
we
were
using
this
in
a
non-traditional
way.
I
was
just
like
oh
yeah,
we
use
those
learn
by
doings
and
I
was
like
oh
but
they're,
not
really
learned
by
doings.
You
don't
even
really
think
about
the
workarounds
that
you're
using
until
someone
articulates.
I
know.
B
I
know
in
fact
it's
you
know,
I'm
glad
you
said
that,
because
I'm
always
asking
for
what
are
people's
work-arounds
and
I
don't
think
you're
you're
exactly
right,
I'm
not
thinking
actively.
Oh
here's
the
list
of
work-arounds.
I
use
it's
more
like
what
else
do
you
use
the
learn
by
doing
label
to
do
you.
B
C
The
other,
the
other
thing
that
we've
kind
of
struggled
with
in
the
past
erin.
Is
this
really
like
hard,
this
hard
discrimination
between
activities
that
are
summative
and
formative
and
you
have
to
either
fit
in
one
box
or
the
other
I
feel
like
that
is
increasingly
becoming
kind
of
a
gray
area,
and
I
know
that
that's
something
that
is
very
important.
C
C
C
We
know
that
one
of
the
things
that's
happening
with
lms
platforms.
At
least
I've
noticed
this
change
in
rlms
is
that
students
are
no
longer
reading
through
modular
content,
they're
simply
going
by
the
activity
due
dates,
and
then
they
go
to
those
activities
and
people
are
trying
to
come
up
with
workaround
fixes
for
that
in
canvas
right
now
I
have.
C
So
being
mindful
of
this,
it's
not
super
great
for
me
that
the
grade
book
only
shows
these
checkpoint
quizzes
and
these
surveys.
I
want
the
gradebook
to
show
that
there's
these
formative
activities
that
students
should
be
doing
throughout
the
course.
I
would
also
love
it
if
students
couldn't
proceed
from
one
page
to
the
next,
without
going
through
the
previous
page
and
that's
something
we
talked
about
before
we
don't
really.
A
D
I
come
from
a
weird
perspective,
because
wgu
uses
a
course
like
this,
but
we
have
our
own
sort
of
external
stats
test,
so
everything
everything
anybody
ever
does
for
anything
is
merely
summative.
Sorry
merely
formative
right.
So
you
got.
You
got
to
really
weigh
what
you're
motivating
and
yes
aaron.
It
would
be
awesome
if
we
could
tell
the
gradebook
what
shows
and
percentages
right,
whether
it's
formative
or
summative
give
us
the
ability
to
turn
off
or
turn
on
percentages
for
the
students
to
see
my
perspective.
D
My
perspective
is
a
little
bit
contradictory
to
yours.
Lauren,
in
that
I
would
never
gate
progress
through
the
course,
because
maybe
they
know
a
bunch
of
stuff
already,
but
I
might
gateway
something
that's
going
to
account
for
their
grade,
so
I
might
have
an
external
quiz
or
maybe
even
aaron,
a
quiz
inside
this.
D
B
B
So
it
really
matters
how
those
instructors
are
particularly
using
the
course
materials
and
what
instructions
they're
giving
and
what
their
grading
versus,
not
so
lauren
you're,
really
beholden
to
that,
but
would
be
nice
is,
if
you
at
least
knew
that
context
and
what
practices
in
in
the
classroom
so
to
speak,
are
are
motivating
or
demotivating
the
students
to
complete
certain
work.
You
know
what
I
mean
so,
for
example,
a
one
of
the
use
cases.
We
we
say,
people
use
a
lot
which
I
think
still
is
the
case
is
instructors
assigning.
B
You
know,
gating
the
assessment,
the
summative
assessment
at
the
end
of
a
module,
but
you
know
making
sure
students
do
the
practice
before
they
open
up
the
exam,
for
example,
so
they're
really
assigning
that
practice
as
homework.
There
might
be
instructors,
though,
that
don't
do
that
and
are
just
you
know,
just
kind
of
encouraging
by
not
even
meaning
to
the
behavior,
for
students
to
just
go
to
the
assessment
and
see
what
kind
of
grade
they
get
first
before
they
do
the
practice
it
really
matters.
D
And
one
thing
this
always
reminds
me
of
is
that
over
my
17
years
doing
online
instruction,
I've
seen
students
sort
of
evolve
into
wanting
smaller
bites
of
knowledge
and
deeper
interactive
evaluation
of
it
right.
So,
even
though
your
materials
are
written
in
a
way
that
can
be
read,
sometimes
it
goes
on
for
four
five,
six
seven
pages
before
there
is
a
checkpoint
right.
D
I
think
the
modern
sensibilities
would
be
more
like
half
a
page
or
a
page
and
a
half
before
there
is
a
much
smaller
checkpoint
on
a
smaller
group
of
materials,
maybe
with
a
percentage
grade
or
you
put
that
under
our
control
and
then
you
can
also
have
markers
where
you
wrap
up
four
five.
Six
of
these
mini
checkpoints
into
a
major
checkpoint
where
they're
checking
a
whole
competency
or
whatever
your
language
is
whole
topic
and
then
even
more
major
ones
that
feel
more
like
unit
exams
and
then
maybe
a
final
one
right.
D
So
it's
it's.
This
never-ending
cycle
cyclical
nature
of
the
learning
that
I'm
starting
to
see.
Students
want
more
of
that
and
until
you
break
apart
the
checkpoints
into
the
books,
more
granularly
they're
still
probably
going
to
be
like
eight
pages
of
reading.
Let
me
just
go
to
the
checkpoints
and
try
and
then
read
backwards
to
fill
in
the
gaps,
which
is
painful
and
inefficient.
B
Yeah
I
see
what
you're
saying
is
if
we
delivered
more
checkpoints
more
often
in
smaller
chunks,
that
might
automatically
motivate
the
students
to
to
get
through
the
material
and
even
the
formative
assessments,
because
it's
a
more
digestible
practice
session
and
and
and
then
and
then
moving
on
to
the
checkpoints.
That's
interesting!
B
Actually,
as
you
talk
gideon,
I
just
keep
thinking
we
we
have
to
just.
We
have
to
develop
a
course
with
you
for
your
particular
use
case,
because
I
think
it'd
be
beneficial
to
a
lot
of
folks
and
it'd
be
interesting
to
see
what
kind
of
things
we
do
to
fit.
Your
particular
audience,
which
is
yeah
like
again,
I
think,
would
serve
others
well
as
well.
D
For
those
who
don't
know,
we
do
structured,
independent
studies,
so
we
have
the
wgu
assessment
and
we
provide
them
the
learning
resources
to
use
in
any
way
or
not
at
all.
We
have
some
grad
students
who
come
in
knowing
stuff,
so
they
just
take
it.
Take
the
final
exam
and
pass
others
know
nothing
and
they
go
through
every
line
of
our
learning
resources,
but
most
of
them
try
to
skip
a
lot
because
adults
are
busy
everyone's
busy
and
they
go
straight
for
the
assessment
and
that's
okay.
If
the
learning
cycle
is
small,
that's
okay!
D
B
D
B
Yeah
I
mean
wgu
is
kind
of
you
know
you
still
hold
synchronous
sessions
occasionally
and
invite
students
to
those.
If,
if
necessary,
I
mean
students
really
are
in
charge
of
their
own
learning,
meaning
they
could
even
just
not.
They
could
use
a
completely
different
set
of
learning
resources
as
long
as
they
still
supported
them.
In
passing
that
that
final
assessment.
D
B
Yeah
the
model
is
interesting,
I
think,
like
you
said,
I
think,
there's
some
things
we
could
do
for
them
for
those
types
of
students
that
would
still
resonate
beyond
that
so
yeah.
So
I
took
some
notes
here
about
you
know
capturing
that
context
when
delivering
a
course
which
is
actually
in
alignment
with
some
of
our
you
know.
Our
research
has
shown
particularly
lauren
hircus's
research
has
shown
that
you
know
instructors
need
support
and
just
adopting
materials.
B
B
And
so
I
think,
because
of
that,
it
would
be
nice
to
even
just
have
like
a
short
short
place
in
any
given
course,
where
we
kind
of
just
ask
you
know
what
do
you?
What
are
some
of
these
policies
used?
What
are
some
other
techniques
or
activities
that
you
perform
in
the
classroom
in
alignment
or
in
conjunction?
You
know
in
parallel
with
this
type
of
course,
material.
B
I
think
if
we
start
capturing
that
we'll
get
a
lot
more
ideas
about
how
to
how
to
structure
and
what
kind
of
controls
to
give
instructors
that
will
help
them
in
their
delivery
and
maybe
even
serve
that
data
back
to
people
like
lauren,
but
also
lauren
herkus,
but
also
you
lauren
and
I
mean.
Would
it
be
nice
to
have
a
kind
of
contextual
information
with
the
data
that
you
receive,
or
is
it
too
much.
C
So
I
mean
basically
what
we're
delivering
at
this
point
in
time
has
to
be
completely
contained
within
just
the
online
delivery
mode,
because
also
I
mean
we
don't
have
really
control
and
it
doesn't
really
whatever
context
is
happening.
It's
great
when
instructors
are
encouraging
students
to
work
their
way
through
the
toolbox
and
use
practice
questions,
but
we
also
have
you
know
very
little
control
over
what
is
happening
in
terms
of
the
larger
course
context.
C
How
instructors
are
telling
students
to
work
through
this
content?
We
try
to
be
as
contained
as
we
can
in
terms
of
having
all
of
the
instructions
within
our
course
itself.
My
observation
was
merely
that.
I
know
that
students
are
changing,
how
they
work
through
our
learning
management
system
and
only
looking
at
kind
of
the
gradebook,
and
this
is
this-
is
what's
due
at
this
time
point.
So
I
would
like
to
have
the
opportunity
in
the
gradebook
that
students
look
at
to
have
students
see
not
only.
C
This
is
a
checkpoint
quiz,
but
these
are
practice
activities.
I
just
would
love
it
if
the
practice
activities
could
show
up
in
the
grade
book,
so
that
students
are
aware
of
the
fact
that
there
are
embedded
learn
by
doing
activities
that
they
should
be
focusing
on
and
that
they
shouldn't
just
click
on
the
checkpoint
quiz.
To
do.
I
think
it
trains
students
to
look
for
those
learning
activities
that
are
practiced
and
not
just
look
for
the
final
quiz,
because
that
sort
of
behavior
doesn't
really
serve
students.
C
So
if
we
have
the
option
in
the
gradebook
to
say
like
practice
activity,
I
don't
care
about
percentage.
In
fact,
I
don't
even
care
if
the
students
see
their
percentage
correct
for
the
checkpoint
quiz.
I
know
that
that's
something
that's
embedded,
but
if
they
even
just
got
a
check
mark
for
completing
the
practice
activity
that
would
be
hugely
helpful
so
that
they
can
see
that
there
are
practice
activities
that
they
are
missing
if
they
don't
go
into
the
module.
I
don't
know
if
that
makes
sense.
B
It
does
so
you
just
want
to
be
able
to
change
the
behavior
of
the
students,
regardless
of
what
instructions
or
what
the
instructors
are
doing
with
the
thing.
What
I
was
suggesting
is
that
I
mean
I
mean
at
least
you
would
maybe
know
what
how
they're
doing
it
and
then,
if
we
capture
some
of
that
information,
but
I
see
what
you're
saying
too
yeah.
B
C
That's
precisely
you
just
you've
just
nailed
it
so
aaron
we
have
us.
We
have
some
students
that
do
the
whole
toolbox,
they're
going
through
each
page,
they're
doing
the
practice
activities
they're
doing
the
surveys,
they're
doing
the
checkpoint
quizzes,
and
then
there
is
a
larger.
I
mean,
there's
a
large,
not
larger,
there's
a
substantial
group
of
students
that
are
simply
going
to
the
gradebook
and
doing
the
checkpoint
quizzes,
which
is
not
what
we
intend
for
them
to
do.
A
C
B
But
there's
definitely,
I
think
more
robust
gating
features
built
into
taurus
than
there
was
in
in
legacy,
but
that's
another
future
topic
idea
topic
as
well
is
gating
in
general
and
how
that
works,
and
maybe
there's
ideas
for
for
how
else
it
might
be
done
so
great,
I'm
gonna
go
back
to
the
slide
to
see.
If
there's
anything
else,
I
can
dig
into
so.
We've
talked
about
models
types
of
data
we'd
like
to
collect,
I
guess
or
differently,
from
how
we
currently
do
it.
B
I
don't
hear
anyone
on
here
or
see
anyone
on
here.
That
probably
has
done
a
lot
in
terms
of
learning
curves,
so
performance
profiler.
I
think
what
this
is
and
tanvi.
You
can
keep
me
honest
here,
but
I
believe
what
that's
referring
to
it's
a
it's,
a
it's,
a
ability
and
data
shop
to
be
able
to
see
the
performance
on
specific
questions.
A
B
You
so
much
so
for
those,
so
gideon
lauren
are
those
types
of
analyses.
I
know
you
probably
use
some
of
this,
maybe
not
in
data
shop,
but
you're
analyzing
it
in
some
way
for
iterative
improvement
right.
This
is
this.
Is
we're
not
talking
like
real-time
analytics
now
we're
talking
like
after
the
fact
analysis,
so
how?
How
valuable
are
these
types
of
analyses
to
to
each
of
you.
A
C
B
C
A
C
Yeah,
I
have
no
idea,
I
will
ask
kathy
and
victor
they
also.
You
know
when
we
came
to
cmu
to
do
the
actual,
like
the
cmu
week-long
workshop.
I
did
the
course
creation.
One
and
kathy
did
the
data
shop
one,
so
I
didn't
do
the
data
shop
one.
I
have
like
no
formal
training,
so
I
will
ask
her
and
victor
if
they're,
using
the
kind
of
interactive
components.
B
Yeah
and
just
like
yeah,
let
me
know
at
a
high
level
if
they're,
like
you
know
it's
great,
we
use
it
all
the
time,
it's
very
helpful
or
if
they're,
like
oh
we're
kind
of
you
know
we're
kind
of
using
it.
But
in
these
specific
ways
or
you
know
what
we
just
take
the
data
and
look
at
it
on
our
own,
you
know
I'm
kind
of
trying
to
get
that
kind
of.
B
C
And
in
answer
to
your
question
cindy,
I
think
primarily,
you
know
the
exported
data.
We
then
merge
with
data
sets
that
I'm
creating
from
data
outputs
from
our
learning
management
system
on
individual
course
performance,
and
then
this
is
being
integrated
and
merged
in
spss.
So
pretty
much
all
of
our
analyses,
I
think,
are
being
done
in
spss.
B
B
That
allows
us
to
or
allows
anybody
to
up
upload
a
gradebook
and
basically
it's
going
to
analyze
for
you
what
activities,
what
formative
activities
contribute
most
to
better
performance
on
summative,
so
you
can
kind
of,
and
anyone
on
here
that
knows
better
than
me
can
can
correct
me,
but
I
believe
it's
more
of
like
okay.
Here's
of
all
these
things
that
you
grade
on
these
particular
activities
seem
to
be
contributing
the
to
the
best
performance
on
on
final
exam.
B
So
to
speak,
which
is,
I
think,
something
that
would
be
really
helpful
for
just
everyday
teachers,
but
I
don't
think
it's
widely
known
that
this
exists,
and
I
don't
know
how
widely
these
types
of
analyses
are
performed
by
everyday
teachers,
instructors
and
educators,
so
that
you
know
to
cindy's
point.
I
really
it
would
be
nice
to
to
hear
more
about
who's
using
what
what
tools
and
and
how,
essentially
so
you.
A
B
C
Okay,
so
for
for
me,
I
think
one
of
the
like,
like
the
biggest
obstacles
to
being
able
to
have
more
data,
is
simply,
as
I
said,
this
sort
of
like
a
la
carte
use
of
the
toolbox
like
this
student
just
went
in,
and
I
can
see
that
they
did
30
seconds
on
this,
this
this
checkpoint
quiz
and
then
they
went
in
and
did
this
other
checkpoint
quiz,
and
then
they
didn't
like
work
their
way
through
the
toolbox.
So
I
would
say
that
that's
like
the
largest
barrier
to
having
like
a
larger
end
for
us.
C
So
we
can.
We
can
see
that
basically,
we
can
see
those
students
based
upon
the
amount
of
time
that
we
see
their
check
in
and
check
out
time
on
the
different,
the
different
components,
but
really
what
would
be
more?
I
mean
it
would
be
helpful
if
there
was
an
analytic
that
was
basically
an
overall
like
if
there
could
be
an
overall
metric
of
this
is
completion
of
the
course
beyond
simply
well.
C
They
did
the
they
did
the
checkpoint
quizzes
right
so
like
we
kind
of
have
to
go
in
and
look
on
a
case
by
case
basis.
Oh
like
this,
this
student
has
a
check
in
checkout
time
of
this
quiz
and
this
quiz,
so
we
suspect
that
they
didn't
do
anything
like.
Wouldn't
it
be
helpful
for
us,
I
think,
to
know
okay,
the
students
spend
an
hour
in
the
course
great.
We
want
to
include
that
student
in
our
data
set.
C
They
actually
read
through
the
course
like,
I
feel
like
we
would
otherwise
have
to
parse
through
on
an
individual
by
individual
basis.
What
their,
whether
or
not
an
individual,
is
even
doing
the
course
versus
just
going
in
and
taking
quizzes.
So
that
would
be
really
helpful.
This
sort
of
com,
a
completion
metric
did
they
look
at
all
the
pages
in
the
course
do
they
actually
click
through
the
entire
course.
B
Yeah
I
mean
we
have
that
now
in
our
well.
We
have
that
in
the
grade
book,
but
it's
not
it's
not
it's
not
precise
right.
It's
it's
a
it's,
a
visual
bar
that
you
know
you
could
probably
interpret
a
different,
a
couple
different
ways,
but
you
had
a
couple
things
in
that.
In
that
description,
you
know
being
able
to
to
spot
trends
better
around
a
number
of
things
course.
Completion,
of
course,
is
your
big.
We
need
to
see
that
it's,
it's
really
something
that
comes
up.
B
B
C
A
C
Enough
for
us
to
keep
or
eliminate
a
student,
so
just
percentage
of
pages
that
were
viewed
so
did
a
student
view
at
least
70
of
the
pages
like
we
can
have
the
cut
off
but
column,
to
show
the
number
of
pages.
Viewed
because,
like
we
don't
need
to
look
at
all
the
click-through
data.
For
like
a
student
clicked
here
on
the
page
and
here
on
the
page,
we
want
to
be
able
to
make
decisions
about,
including
or
excluding
a
student
based
upon
whether
or
not
they
even
completed
the
course
not
in.
C
A
B
C
Nope,
but
that's
that
helps
us
eliminate
students
that
simply
clicked
on
the
checkpoint
quizzes.
So
if
we
can
just
quickly
eliminate
students
that
only
went
to
the
checkpoint
quizzes
and
didn't
view
any
content,
we
know
they
didn't
view
on
any
content,
because
it
shows
that
they've
only
viewed
seven
pages
out
of
70.
We
can
eliminate
those
students
like
if
you
only
clicked
on
the
checkpoints
you
viewed
10
of
the
course
content.
B
Right
right,
right,
yeah,
I
mean
even
having
having
some
of
that
data
is
better
than
having
nothing.
Even
if
you
have
to
make
inferences
from
it,
and
I
mean
that's
essentially
what
I
was
asking
like,
what
are
some
things
that
you
kind
of
like
you're
like
we're
guessing.
This
is
the
behavior
based
on
what
we're
seeing,
but
we
don't
know
because
we
don't
have
the
data
in
front
of
us
right.
C
B
C
B
And
what's
trying
to
be
saying
if
it
they
visited
the
page
but
just
used
it
for
going
to
the
next
page,
not
even
skim
through
it.
I
know
there's
not
a
good
way
to
really
know
that
they
actually
read
it.
But
I
guess
if
you
know
that
a
student
visited
every
every
page
and
then
did
well
on
the
checkpoint.
B
B
A
And
that
is
one
benefit
of
data
shop
in
the
sense
that
we're
tracking
student
actions
right-
it's
not
just
at
the
level
of
how
they
did
on
a
question
or
you
know
a
quiz
and
because
we're
tracking
student
actions
we
do
get,
did
they
click
on
this
page?
Do
they
click
and
then
you
it
affords
the
ability
to
then
take
further
analysis.
That
says:
yes,
they
clicked
on
all
these
pages,
but
we
know
from
the
very
next.
A
B
A
Good
question,
I
think,
that's
a
point.
That's
commonly
made
about
data
shop.
Is
that
there's,
and
that
was
the
idea
with
this.
You
know
the
great
book
analysis.
That's
sort
of
separate
from
learnsphere
even
was
that
you
know
to
bring
it
to
the
level
of
the
instructor,
because
data
shop
is,
I
don't
mean
to
be
insulting,
but
over
the
head
of
a
lot
of
people
and
not
at
all
what
a
lot
of
people
need
right.
A
B
B
You
should
be
testing
kind
of
extract
that
out
a
little
bit
for
the
people
who
don't
want
to
dig
in
and
and
and
play
with,
the
data
a
whole
lot
or
or
at
that
level
rate,
but
I
think
it's
super
useful
and
powerful,
and
and
for
anyone
who
hasn't
used
it,
I
would
encourage
even
just
going
in
and
looking
at
what's
there
because
they
think
that
even
just
generates
ideas
for
you
about.
What's
you
see
what's
possible
and
you
start
to
think?
Oh
now
I
can
put
this
with
that
and
but
yeah.
A
That's
kind
of
what
I
was
getting
at
with
the
comment
about
what
lauren's
colleagues
are
doing,
because
we
have
done,
and
we
want
to
continue
to
do,
take
things
that
other
people
are
doing
say:
they've
written
a
great
python
script
or
something
that
we
can
then
pull
into
learn,
sphere
and
make
available
to
other
people.
You
know-
and
I
you
know
even
other
people
being
taurus
right.
A
B
But
yeah
now
that
you
mentioned
that
I
didn't
capture
that
I
wanna
student
coming
to
box
folks
use
spss
and
I
think
that's
a
common
one
to
use
for
more
I'll
say,
lay
people
who
are
not
looking
to
dig
into
a
lot
of
data,
but
need
more
of
that
summarized:
data
spss,
if
I
remember
correctly,
it's
pretty
user
friendly
for
for
folks
who,
who
aren't
used
to
looking
at
data
all
day.
I.
A
C
B
C
Data
shop
has
a
tremendous
amount
of
information,
but
even
for
people
that
are
primarily
researchers
like
me
and
kathy
and
victor
are
researchers
that
click
data
is
not
translated
for
us
into
something.
That's
necessarily
usable
for
our
research,
because
what
we
need
from
that
click
data
is
for
it
to
be
combined
into
a
metric
for
something
that's
more
easily
utilizable
like
I
was
describing
before.
If
we
can,
just
if
we
can
just
know
this
person
clicked
on
every
page,
that
has
become
then
usable
for
us.
C
When
we,
when
we
have
6
000
students,
completing
something
in
a
semester
that
click
data
on
like
an
individual
page,
just
like
there's
this
many
clicks
on
this
page,
that's
like
not
that's
almost
like
unmanageable
and
I'm.
This
is
coming
from
a
place
from
someone
who
did
big
data
like
I
did
genomic
analysis.
C
C
I
think
that
that
is
meaningful
when
you
are
able
to
make
it
a
proxy
for
completion.
This
student
did
clicked
on
this
percentage
of
the
pages
that
then
becomes
completion,
data
that
then
becomes
utilizable,
but
simply
having
click
data
that
is
so
dense,
like
that's,
not
even
usable.
From
my
perspective,.
B
So
much
in
it
that's
granular
yeah
and
I
mean
to
cindy's
point
yeah.
You
could
put
that
together
and
know
and
know
even
better
about
what
students
are
doing
in
the
system,
but
yeah
not
having
a
visualization
or
like
we
need
kind
of
almost
a
little
porthole
that
like
allows
you
to
summarize
it
or
chunk
it
in
ways
that
you
can
make
some
some
draw
some
conclusions
from,
even
if
they
are
proxies
for
those
conclusions.
C
The
quizzes
happen,
and
if
I
see
that
a
student
did
quiz
one
two
seconds
later
did
quiz
two
a
minute
later
did
quiz
three.
I
know
that
they're
abusing
the
quiz
system,
but
so
I'm
saying
like
there
are
ways
to
tailor
this,
and
I
don't
think
I
mean
kathy
and
victor-
aren't,
writing
r
or
python
scripts
to
to
do
this.
But
if
someone
else
has
done
that,
I
love
what
you
said.
B
I
don't
know
that
thought
just
came
to
me.
It's
like
trying
to
get
more
people
to
use
what
already
exists
in
learn
sphere.
If
it's
already
been
done,
then
we
should
tout
it.
I
think
further
out,
so
that
folks,
like
lauren,
do
know.
Oh
this
thing
that
I
do
by
hand
every
day
is:
there's
a
there's,
a
workflow
and
learnsphere
I
could
just
plug
into.
B
A
A
A
Researcher,
not
even
the
level,
that's
not
even
the
right
word.
It's
our
researchers
doing
very
specific
things,
so
they're
reusable
for
them.
You
know,
like
we've
got
paulo
who
does
you
know,
runs
a
workflow
over
five
different
data
sets,
but
it's
the
same
workflow.
So
it's
reusable
by
him
and
we
recommend
it
because
it's
able
to
you
know
because
it's
linked
to
a
paper.
Yes,
but
I
think
we
could
definitely
still
benefit
from
having
some
that
are
more
at
the
quote.
Everyday
level.
B
Yeah
and
I'm
even
because
I'm
trying
to
think
of
projects
to
suggest
to
metal
students
for
an
internship
I'm
trying
to
to
put
in
place
starting
the
summer-
and
this
is
a
good.
Maybe
this
is
a
yeah.
This
is
a
great
one
yeah,
because
the
students,
the
medical
students,
have
been
really
involved
in
using
tourists
even
and
providing
lots
of
good
feedback
on
even
just
what
they
need
to
do
in
their
class
projects,
in
terms
of
analyses
and
what
what's
been
available
to
them
and
what
they
need.
A
B
C
B
C
And
if
you're,
not
an
expert
data
scientist
that
writes
your
own
code
or
you
know,
is
used
to
using
different
pipelines
and
plugging
data
in
where
I
just
it.
It's
it's
a
lot
harder
to
tap
into
you
know
getting
the
most
out
of
learn,
sphere
and
data
shop
and
the
people
that
are
really
good
at
that.
You
know
as
long
as
you
you
know,
notate
and
you
you
write
what
you
can
use
this
for
people
will
absolutely
use
that.
C
B
Yeah,
I
mean
that's
what
I've
been
been
trying
to
do.
I
mean
that's
been
my
job
for
the
last
few
years,
being
being
back
at
cmu
is
trying
to
bridge
the
gap
between
what's
in
data
shop
and
learn
sphere,
and
what
folks
need
everyday
folks
need
out
of
it
and
yeah
the
more
we
can
do
of
that,
the
better
for
sure
it's
making
more.
You
know
data
science,
more
accessible
right
to
a
larger
group
of
people
other
than
data
scientists.
A
A
A
C
C
B
A
A
A
C
A
C
B
Yeah,
it's
been
a
really
good
meeting.
I
was
actually
just
going
to
wrap
up,
so
you
won't
miss
anything
cindy.
Thank
you
so
much
for
being
here
so
yeah
I'm
gonna
wrap
up.
I
think
we've
taken
a
lot
of
good
good
notes
from
this
meeting
and
even
got
ideas
for
subsequent
topics
in
in
these
meetings.
So
I'm
just
pleased
and
thank
you
lauren
again
for
for
being
here
being
so
engaged
getting
in.
Thank
you.
I
always
love
to
have
multiple
perspectives.
B
In
at
least
you
know,
one
meeting
it's
great
to
have
different
use
cases
and
different
ideas
for
how
this
could
be.
So
thank
you
all
from
the
bottom
of
my
heart.
Please
continue
to
come,
share
the
link
or
share
the
registration
link
with
with
others,
because
the
more
the
merrier
always
and
I'm
really
looking
forward
to
this.
This
year's
worth
of
topics
and
the
continued
development
of
taurus
based
on
the
feedback,
I'm
collecting.
So
thank
you
so
much
and
enjoy
your
weekend.