►
From YouTube: Torus Community Meeting January 2022
Description
Monthly meeting to discuss Open Learning Initiative's next generation platform. This meeting was a discussion of the functionality and features for Data, Analytics, and Reporting. This was the first of several meetings about this topic.
A
A
A
A
B
A
A
C
A
A
A
Okay,
well,
I'm
gonna
get
started.
I
guess
we'll
have
a
small
group.
Today,
people
are
probably
still
on
break
or
I
hope
they
are
anyway
and
taking
some
time
for
themselves
but
happy
new
year.
Everyone
who's
here
and
I'm
so
glad
that
you
were
able
to
find
the
link
and
join.
So
I
went
to
kick
off
the
year.
I
knew
it
would
be
a
late
crowd,
probably
today,
so
I'm
gonna
make
this
topic
stretch
over
the
next
two.
A
So
excuse
me
this
one
and
the
next
one,
and
then
what
I
plan
is
whatever
we
gather.
Whatever
information
we
gather
in
this
particular
meeting
will
probably
set
the
stage
for
subsequent
meetings
like
so
right
now,
it's
all
about
data
analytics
and
reporting
all
kind
of
crammed
together,
but
I
think
knowing
what
you
all
care
about,
or
the
types
of
things
that
you
want.
A
Different
functionality
I'll
probably
be
breaking
these
up
into
separate
meetings,
going
forward
to
do
more
of
a
deeper
dive
and
collect
more
deeper
requirements
from
folks.
So
welcome
to
the
first
topic
of
the
year
and
we'll
have
these
once
a
month
for
anyone
new
to
this
meeting.
We
I
schedule
this
on
the
first
friday
of
every
month
and
at
this
link
let
me
show
you
now.
A
And
you
can
check
there
regularly
for
up
upcoming
meeting
topics.
Also,
you
can
get
the
recordings
from
past
meetings
there.
You
can
also
get
links
to
our
github
repository
where
we're
you
know
where
we
have
all
the
open
source,
content
and
code
for
tourists,
and
you
can
also
submit
ideas
and
requirements
that
you
that
might
be
important
to
just
to
you,
I'm
sure
to
others.
A
A
So
when
we
talk
about
data
and
analytics
and
reporting,
what
we're
really
thinking
about
are
the
data
that
any
one
of
us
uses
day-to-day
to
either.
You
know,
as
you're
teaching
a
class
an
oli
class
as
you're
looking
at
whether
you're
authoring
and
want
to
see
statistics
on
how
to
iteratively
improve
your
course,
but
also
research,
and
I
know,
there's
a
lot
of
researchers
that
use
our
our
data
anonymously
and
probably
has
some
requirements
about
how
to
see
that
or
what
to
see
a
little
bit
easier
then
we're.
A
A
What
data
do
you
use
now
and
then
think
about
how
you'd
like
to
see
it
better
or
how
you
would
like
to
or
what
else
you
might
need
in
addition
to
what
you
can
use
now
so
again,
what
do
you
wish
you
had?
What
do
you
think
would
make
it
easier
for
you
to
to
see
the
results
of
things
more
clearly
and
easily
and
again
just
how
do
you
feel
about
the
current
data
and
analytics
offered
for
each
of
these
areas?
A
So,
as
you
know,
or
maybe
don't
know,
we
have
in
for
delivery
purposes,
we
have
what's
called
a
learning
dashboard
which
tracks
which
tracks
student
progress
in
the
formatives
and
the
aggregate.
So
you
can
drill
in
to
individual
students,
but
our
learning
dashboard
right
now
shows
things
in
the
aggregate
and
then
you
can
like
drill
in
from
there.
A
A
Now
in
the
delivery
system,
you
have
to
kind
of
export
that
gradebook
data
and
import
it
into
your
lms.
But
right
now
it's
it's
push
button
updating
through
taurus,
so
those
are
just
kind
of
the
examples
that
we
have
already
so,
let's
dig
in
who
would
like
to
go
first
and
talk
about
data
that
they
would
like
to
see
or
get
access
to
maybe
start
with.
A
C
Recently,
we've
been
doing
this
through
how
like
we
make
a
request
to
get
analytics
data
in
the
form
of
those
compressed
cvs
files
that
has
basically
everything
that
we
want,
including
individual
responses,
repeat
responses
in
our
setup.
As
you
know,
we
have
a
survey,
a
pre
and
post
survey
on
what
students
believe
about
study
skills
that,
for
others
that's
what
our
course
is
about
teaching
study
skills.
C
C
Is
there
a
way
going
to
be
there
that
we
could
directly
get
those
data
rather
than
having
to
make
a
request
every
time?
And,
as
you
know
in
our
case,
because
we're
doing
so
much
developmental
work,
we're
always
asking
you
for
for
a
new
set
of
analytic
data
for
a
given
offering
and
and
now
that
we've
got
versions
of
this
course
for
physics
and
chemistry
and
biology
and
the
default
and
statistics.
A
A
So
one
thing
I
can
do
is
make
sure
that
we
start
with
your
request
and
build
that
in
to
the
system
as
one
way,
one
one
review
or
reporting
view
or
whatever.
But
you
also
gave
me
another
idea
of
something
that
we
don't
have
at
the
moment,
which
is
data
across
different
courses.
So
this
is
exactly
the
kind
of
thing
I'm
thinking
of
like
I'm
imagining
we
don't
have
a
lot
of
super
admin,
type
reporting
and
analytics.
So
me
as
even
an
admin
in
the
system,
can't
go
in
and
say.
A
Okay,
I
want
to
see
I
want
to
compare
or
see
the
analytics
for
this
course.
This
course,
and
this
course
kind
of
all
in
one
place,
that's
something
that
would
probably
be
really
valuable
to
you
to
be
able
to
just
have
like
a
almost.
I
don't
know
what
to
call
it
a
a
dashboard,
a
place
where
you
can
kind
of
click
into
the
all.
The
courses
that
you
own
and
see
them
across
different.
C
Yeah,
that
would
be
wonderful.
I
mean
right
like
so
right
now.
Here's
a
specific
example
of
where
what
you're
talking
about
would
be
really
useful.
We
we
deal
with
it
in
a
different
way
now,
and
that
is
that
when
we
first
started,
the
first
version
of
this
course
on
teaching
cognitively
supported,
study
skills,
so-called
deep
study
skills.
What
we
did
was
our
instructional
material
was
focused,
if
not
entirely
almost
entirely
on
promoting
those
deep
study
skills.
C
We
actually
began
to
build
in
instruction
about
the
shallow
skills,
and
so
here's
the
how
it
relates
to
you
is
what
we
do
is
we.
We
have
one
data
set
where
we
look
at
what
it
looked
like
initially
and
then,
as
we've
improved
it
across
semesters,
we
go
into
the
separate
data,
sets
and
look
at
the
pre
and
post,
but
what
you're
suggesting
is?
Maybe
if
it's
all
in
one
place,
analytically,
we
could
compare
those
different
versions
as
we've
gotten
better
at
instructing
all
in
one
place,
rather
than
the
way
we
do
this.
A
I
just
want
to
capture
that
too,
because
that
that
is
what
I
was
suggesting,
but
you
added
another
component
to
that
which
is
also
being
able
to
compare
subsequent
versions.
So
to
you
know
previous
ones,
so
you
can
actually
see.
Is
that
improvement
that
you've
done
to
the
course
actually
making
an
impact?
And
so
I
want
to
make
sure
I
capture
that
even
separately
to
make
sure
that
that's
part
of
that
okay.
Thank
you.
Excellent.
A
C
Let
me
just
say
one
more
point,
which
is
appreciation
and
praise.
That
is
the
analytic
data
that
we've
gotten
you
know
from
from
you,
for
our
course
has
been
not
only
wonderful.
We
couldn't
have
identified
what
kinds
of
iterative
improvements
we
needed
to
make
if
we
didn't
always
after
every
offering
of
that
course
go
in
and
get
those
analytics
because
our
our
team,
you
know
we
have
robust
meetings
over
a
period
of
time
after
each
offering
where
we
dig
into
those
data.
C
A
A
Why
I'm
waiting
for
this
I
mentioned
earlier-
maybe
some
of
you
weren't
on
yet
that
I
sent
out
the
announcement
for
this
with.
I
was
the
first
person
to
use
our
new
crm
or
new
customer
relationship
management
system
and
emailing
system,
and
so
I
was
on
like
a
help
desk
call
right
before
this,
where
they
were
kind
of
remote
desktoping,
and
I
think
that's
what
bogged
down
my
computer.
A
A
A
A
Uses
of
the
course
student
runs,
and
then
we
have
this
analytics
tab
where
you
can
open
up.
These
are
the
learning
objectives
that
are
in
this
particular
unit,
and
then
you
can
expand
these
to
see
the
skills,
the
questions
that
are
associated
with
those
skills
and
some
usage
statistics
here
and
what
these
represent
right
now
are
the
number
of
attempts,
how
many
of
those
attempts
how
what
percentage
of
students
got
it
right,
the
first
time
they
attempted
it.
How
many
eventually
get
it
correct?
A
And
then
this
number
here
is
a
relative
difficulty.
We
take
into
account
hints,
if
hints,
were
needed
as
well,
and
then
I
mean
this
this
course
actually
has
been.
It's
delivered
twice
a
year
to
basically
all
incoming
freshmen
at
carnegie
mellon,
and
so
the
person
who
owns
this
course
has
iterated
on
it
a
lot
over
the
years,
and
so
you
can
see
a
lot
of
these
numbers
are
good,
but
there's
always
places
for
improvement.
A
As
you
can
imagine-
and
I
know
already
know
some
places
where
the
analytics-
maybe
don't
look
so
good
or
aren't
available.
Oh
there,
we
go.
What
happens
is
these?
These
are
color-coded
to
show
red
and
yellow
if
those
numbers
aren't
what
we
think
they
should
be
again
apologize
for
my
slowness
in
my
computer,
but
here
we
go
here's
some
places
where
we
know
that
this
particular
question
students
obviously
get
it
wrong
most
of
the
time
right
from
the
get-go,
and
then
they
eventually
get
it
right.
A
So
this
is
a
place
where
possibly
the
person
who
owns
this
course
have
gone
in
and
looked
at
that
data
a
little
further
to
see.
Okay,
what's
going
on
with
the
question,
is
there
enough
supporting
material
around
it
to
help
students
answer
the
question,
but
sometimes
you
really
want
to
use
the
question
as
a
you
know,
as
a
learning
mechanism,
and
so
the
fact
that
they
get
it
wrong
the
first
time,
but
eventually
most
get
it
right
eventually
shows
initially
to
me
that
okay,
it
might
be
a
really
overly
difficult
question.
A
It
might
be
a
trick
question.
It
might
be
something
where
they
want
the
students
to
learn
from
the
hints
and
feedback
that
are
incorporated
into
the
question.
So
all
these
analytics
do
or
present
the
data,
but
it's
up
to
the
individual
developers
or
instructors
to
go
in
and
say.
Okay
is
this?
What
we
expect
is
this?
What
we
want
to
happen
that
sort
of
thing
and
make
improvements
based
on
that,
given
that
I
know
this
course
has
been
iterated
on
a
lot.
A
I'm
imagining
that
these
these
questions
that
have
the
four
percent
have
have
enough-
probably
hints
and
feedback
built
into
them.
That
guide
students
toward
the
correct
answer,
and
so
it's
okay,
that
they
don't
get
it
right.
The
first
time,
but
you
know,
if
you
saw,
I
can
imagine
when
you
first
built
a
course,
it
wouldn't
look.
It
would
not
look
the
screen.
A
I
can
guarantee
you
that,
but
so
this
is
some
of
the
the
analytics
we
have
that
we
can
deliver
right
in
context
where
people
are
iterating
and
authoring
their
courses
in
taurus.
We
did
build
this
out
similarly,
but
we
can
also
always
improve
it,
as
you
can
imagine,
so
let
me
get
to
taurus
and
again
I'm
just
kind
of
walking
through
these
things.
You
might
already
be
familiar
with
just
a
spark
ideas
for
oh
well.
That's
nice,
but
we'd,
really
like
this.
A
A
A
This
is
pretty
much
all
we
have
right
now.
It's
very
very
similar,
like
I
said,
but
you
can
do
some
different
views
and
toggles
and
and
we'll
just
keep
making
sure
that
this
gets
built
up
more.
I
think
we've
already
kind
of
encountered
through
some
of
our
piloting
courses,
that
being
able
to
see
this
and
know
when,
like
so
say,
I
go
in,
and
I
change
this
question
now,
because
I
find
that
the
analytics
aren't
very
good
on
it.
I
want
to
I'm
going
to
iterate
on
it.
A
C
Yeah
this
victory
again
this.
This
is
a
good
example,
so,
in
our
in
our
checkpoint
quizzes
we've
used
the
the
summary
statistics.
The
aggregate
statistics
that
you
know
the
success
rate
for
a
question
is
30
or
something
like
that,
which
then
prompts
us
to
go
in
and
look
at
the
question
just
like
you're
describing
here
and
you
know
almost
always
anyway.
We
we
something
pops
out
at
us
and
what
we'll
do
is
we'll
make
a
change
either
to
the
stem
or
to
the
the
troublesome
distractor
or
whatever.
C
To
see
I
mean,
as
you
say,
sometimes,
questions
are
just
hard
and
and
a
low
success
rate
doesn't
mean
it's
a
bad
question
necessarily
so,
but
we
we're
going
to
make
those
changes
and
again
so
what
we
then
do
is
we.
You
know
we
we
have
those
data
summarized
for
a
given
semester
or
offering,
of
course,
going
to
make
the
change.
What
we
then
do.
Is
we
physically
go
in
look
at
the
summary
analytics
for
the
checkpoint,
quizzes
and
go
oh
item.
C
13
went
from
33
percent
correct
hit
rate
to
you,
know
65
percent,
or
something
like
that.
We
think
we
address
that,
and
then
we
might
look
at
the
next
semester
so
being
able
to
do
that
in
one
place,
as
those
offerings
occur
across
offerings
would
be
a
good
feature.
I
mean
it
isn't
that
it
can't
be
done
the
way
it
is
it
can
the
other
just
go
back
to
the
the
analytics
data
that
we
request.
C
You
know
in
terms
of
these
cvs
files,
sometimes
we're
looking
at
the
aggregate
data
like
you're
showing
here.
That's
that's
all
we
need
to
do,
but
for
other
statistical
analysis,
mostly
for
related
to
our
research
questions.
We
want
to
see
how
an
individual
student
changes
say
across
the
course.
So
one
of
our
constructs
is
academic
self-efficacy,
and
we
assess
that
in
each
of
the
six
or
seven
modules
that
we
have
in
the
course.
A
You
don't
have
to
know
the
student
name
just
to
be
able
to
follow
that
data
point
through
the
whole
course,
and
that's
why
I
put
individual
student
changes
across
the
course
and
of
course
anonymized
is
fine,
but
with
maybe
you
know,
but
knowing
that
that
one
id
you
know
this
is
how
they
went
through
the
course
and
be
able
to
see
that
they
are
the
learning's
taking
place
for
any
individual
student
yeah.
We
don't
have
those
analytics
right
now
baked
into
our
views,
but
it's
been
requested.
A
One
of
the
projects
I
know
of
is
to
create
some
content,
put
it
in
front
of
folks
as
students
and
then
look
at
the
data
and
iterate
on
it,
and
they
have
been
the
most
active
in
terms
of
using
taurus
at
the
moment
and
giving
us
feedback
on
the
kind
of
data
and
what
the
analyses
they're
doing
on
that
real
short
timeline
and
what
kinds
of
things
would
would
they
would
like
to
have?
And
that's
that's
definitely
one
of
them.
So
I'm
glad
you've
mentioned
it,
because
I
don't
want
to
miss
that
thanks.
B
Yeah
erin,
obviously
I'm
from
the
same
group,
but
I
was
just
I
was
thinking
about
how
important
it
is
to
be
able
to
to
know
like
what
the
identifier
is,
because,
knowing
you
know
in
especially
if
we
had
the
capability
to
like
kind
of
choose
how
the
identifier
was
formatted.
That
might
even
be
useful
like
because,
obviously,
students
at
our
institution,
they
have
unique
student,
ids
and
so
be
nice
to
be
able
to
basically
match
up
whatever.
Your
identifier
is
with
the
identifier
at
our
institution.
A
Even
question
response
identifiers
would
probably
be
really
helpful
when
you're
looking
at
the
data
either
in
the
aggregate
or
drilling
in
so
you
you,
your
comments
sparked
another
set
of
requirements
that
I
don't
want
to
lose
out
on,
and
so
I
will
add
that
be
able
to
choose
identifiers
just
in
general.
But
I'm
going
to
say
here
for
students,
student
ids
questions
et
cetera,.
C
So
victor,
so
what
lauren
lauren's
point
for
what
we're
doing
now,
where
we're
collecting
data
internally
at
the
university
of
new
hampshire,
for
these
improvements
and
research
questions
and
so
on.
For
that
our
irb
permits
the
linking
of
student
background
data
with
the
the
data
we
get
from
cmu,
and
that's
where
you
know
we
have
some
kind
of
id
that
connects
unh
data
to
individual
student
data,
and
then
you
know
we
de-animize
everything
so
on.
That's!
That's
specifically
through
our
irb.
C
So
you
know
whether
that's
a
big
issue
about
student
data
for
the
open
oli
platform
is
maybe
a
different
question
than
if,
if
a
given
member
of
the
community
is
collecting
data
internally,
and
they
have
that
kind
of
irb
approval,
then
having
that
feature
is
well.
It's
critically
critical
for
us
anyway,.
A
C
No
well,
no,
I
mean,
I
think
it
it.
Actually,
it
is
because
we
have
these
unique
ids
from
the
cmu
data
which
we
match
up
to
unh,
so
I'm
just
it
isn't
that
it's
difficult
or
it
needs
to
be
improved
in
taurus.
It's
just
that.
Having
that
feature
for
members
of
the
community
that
do
have
the
ability
to
link
course
level
data
with
background
data
is
it
is,
I
would
imagine,
for
most
users
would
be
important.
It
sure
is
for
us.
A
B
C
A
Yeah,
but
I
have
to
say,
we
don't
have
like
an
easy
interface
for
it
now,
either,
though
we
you
guys,
probably
do
a
lot
of
munging
of
that
data
with
you
know
like
the
csv
files
and
things
like
that
to
have
an
interface
that
might
help
us
do
that
more
efficiently
and
effectively
would
be
great.
Yes,.
A
A
A
A
Okay,
so
what
I'm
doing
right
now
is
I'm
able
to
toggle
module
to
module
and
what
it's
showing
me
for
each
module
is
or
what
it
should
be
showing
me
if
I
are
the
learning
objectives
listed
for
that
module
and
again
I'm
going
to
go
back
out,
because
obviously
this
one
doesn't
have
data
in
it.
The
way
I
expected.
A
And
getting
to
correctness
with
the
formative
activity,
so
this
is
just
a
view
of
formatives
and
this
is
a
live
view.
So
the
idea
is
that
as
instructors
are
teaching
a
class
and
they
assign
just
one
use
case
assign
properties
of
matters
homework,
you
know
work
through
the
formatives
in
that
a
module.
A
The
idea
is
that
again,
just
one
use
case,
the
the
instructor
could
look
at
this
data,
maybe
before
going
into
a
lecture
on
properties
of
matter,
to
see
exactly
where
students
are
struggling
or
what
they
are
just
you
know
getting
and
again,
so
they
don't.
They
can
kind
of
tailor
that
this
flip
flip
classroom
model.
A
I'm
gonna
skip
you
yellow
for
a
second
red
is
for
those
folks
who
are
maybe
working
through
the
content,
the
the
practice
opportunities,
but
not
getting
to
correctness
or
skipping
around,
because
this,
the
algorithm
that
runs.
This
learning
dashboard
actually
takes
into
account
patterns
as
well,
and
so
these
folks
are,
they
could
be
actually
working
through
all
the
formatives,
but
not
getting
to
correctness
and
running
out
of
practice.
Essentially,
they
need
some
new
challenges
or
some
new
new
practice
opportunities.
A
As
a
kind
of
a
diagnostic
tool
to
see
okay,
if
students
are
struggling
with
this
learning
objective
well,
what
part
of
that
learning
objective
are
they're
struggling
with,
and
so
that
you're
not
just
speaking
to
the
whole
objective
as,
but
you
know,
describing
the
four
states
of
matter.
Maybe
there's
one
of
those
states
that
are
just
tripping
folks
up
more
than
the
others.
A
If
that's
how
it
it's
broken
down,
and
so
this
gives
a
really
neat
view
into
how
students
are
working
through
the
practice
opportunities
in
real
time,
and
you
can
even
drill
in
so
I
could
drill
in
and
see
which
students
are
in
that
are
in
that
category.
I
can
contact
them
from
there.
I
can
also
drill
into
the
skills
and
see
the
activities
associated
with
those
skills
and
how
many
students
are
are
doing
those
and
how
well
I
can
even
start
to
see
what
are
students
answering.
A
C
I
I
use
the
statistical
reasoning
course,
maybe
10
times
or
even
a
dozen
times,
not
teaching
it
anymore
over
a
number
of
years,
and
I
use
those
analytics
that
you
were
just
showing
there
the
those
that
that
dashboard
all
the
time
informative
way,
just
exactly
like
you're
talking
about,
is
that
because
I
I
always
had
an
assignment
that
was
due
before
a
class
session
which
focused
on
problem
solving
related
to
the
module
that
they
were
working
on
or
the
piece
of
the
module.
And
so
I
could
go
in
and
see.
C
You
know
those
gray
dots.
For
example,
you
know
the
people
that
didn't
do
anything.
That's
a
whole
other
set
of
issues
about
what's
going
on
there,
but
to
see
where
people
were
struggling,
and
that
was
very
useful,
so
I
sure
hoped
the
heck.
You
know
as
an
instructor
that
that
that
would
stay,
because
it
was
an
extremely
useful
tool.
A
Great
that's
good
to
know,
because
again
we
don't
want
to.
We
don't
want
to
change
anything
from
what
we
actually
have
already,
if
it's
useful
to
you
and
maybe
what
we'll
do
probably
in
subsequent
meetings
is
or
even
have
like
separate
meetings
with
folks
who
we
know,
use
the
dashboard
regularly
to
to
see
if
there's
any
kind
of
improvements
to
the
visualizations
or
additional
data
we'd
want
to
see
in
those.
But
I'm
glad
to
know
victor
that
it
was
that
you
found
it
very
useful
in
real
time.
C
A
I
don't
that's
a
good,
that's
a
good
question
that
I
would
want
to
know
the
answer
to
and
that
we'll
probably
find
out.
I
can
imagine
that
we're
going
to
be
sending
out
some
surveys
this
year
to
existing
instructor
groups
to
be
able
to
find
out
yeah
what
what
are
they
using
and
like.
I
said
when
we
held
that
that
session
not
too
long
ago
and
people
were
like
surprised
that
this
data
was
available
to
them.
A
A
So
that
leads
me
to
a
good
transition
to
reporting
in
general.
We
don't
have
a
lot
of
robust
ways
at
the
moment
to
be
able
to
generate
usage
usage
statistics
and
in
a
way
that
you
can
share
out
with
your
you
know,
your
provost
or
your
dean,
or
even
a
department
chair
kind
of
setting
where
it
would
be
kind
of
probably
nice
to
be
able
to
generate
reports
to
those
types
of
folks
as
well.
C
We
we
we
have
a
currently,
we
have
a
grant
that
is
supporting
the
development
of
this
course
and
so
twice
a
year.
We
do
a
report
to
the
funding
agency
that
you
know
we've
put
together
sort
of
the
general
kinds
of
statistics
and
graphics
and
so
on,
and
we
just
keep
updating
it.
So
we
have
that,
as
we
go
into
a
like
a
general
deployment
phase
at
the
university
and
which
is
going
to,
of
course
take
some
support
of
the
for
staff
in
the
teaching
and
learning
center.
A
So,
for
example,
we
have.
You
know
oli
course
use
where
we'll
have
a
statistics
department
somewhere
that,
where
you
know
four
of
the
teachers
use
zoli
stats
as
part
of
their
course.
We'd
probably
want
to
be
able
to
pull
that
information
together
somehow
across
those
different
courses
to
be
able
to
deliver
a
report.
To
that,
like
you
know
the
chair
of
that
department
to
try
and
get
a
sense
of,
you
know
how
effective
oli
courses
are
in
general
and
be
able
to
compare
them
to
maybe
some
other
other
types
of
course,
materials
online.
C
Yeah,
in
fact,
while
I
was
teaching
the
that
10
or
12
versions
of
statistical
reasoning,
I
was
also
trying
to
get
the
department
to
switch
over
to
that
course
completely
and
stop
having
the
students
buy
the
then
135
copy
of
jacquard's
introduction
to
statistics,
and
that
I
so
I
would
do
a
report
for
for
the
department
every
year,
updating
it.
So
that's
another
one.
I
could
send
you.
I
forgot
about
thanks
victor
by
the
way
they
didn't
they
didn't.
They
didn't
do
what
I
said
to
do.
A
B
I
I
had
a
couple
more
things:
aaron
that
were
peripherally
related
to
what
we've
been
talking
about.
One
of
the
things
that
I
think
you
and
I
had
had
a
conversation
about
was,
I
think,
it's
really
really
useful
when
you're
downloading,
like
big
data,
like
as
part
which,
which
what
is
what
what
you
have
what
you
have
when
you
download
the
information
or
we
may
make.
Our
formal
request
is
one
of
the
things
that
we
had
talked
about.
B
C
B
It's
like
the
csv
file
that
you
download
from
the
students
as
they're
completing
the
course
which
reflects
what
their
scores
are
actively
for
each
of
the
quizzes
that
they're
currently
doing
so.
This
is
just
a
reminder
that
we
do
need
to
have
the
ability
moving
forward
in
taurus
to
have
that
sort
of
snapshot
of
completion.
B
A
A
How
do
we
present
completion
and
progress,
given
the
kind
of
activities
and
things
that
we
provide
in
the
course,
so
something
that
gets
requested
often
is
being
able,
for
example,
to
not
just
see
completion
but
actually
use
it
as
a
as
a
grading
mechanism
and
are-
and
I'm
talking
about
informatives,
because
it's
easy
to
do.
It's
already
done
in
summatives,
where
you
can
actually
give
students
a
grade
based
on
either
their
performance
of
the
summatives
or
even
just
that
they
did
it
or
didn't.
A
But
we
don't
have
that
mechanism
for
the
formatives,
and
I
really
want
to
explore
that
a
little
bit
more
because
our
stance
on
that
is,
when
you
start
giving
grades
for
formative
practice,
it
actually
changes
the
behavior
of
the
students
and
their
notion
of
what
that
practice
is,
for
we
want
to
make
sure
that
they
understand
that
that's
a
safe
space
to
be
able
to
learn
in
and
not
be
penalized,
for.
You
know
not
getting
to
correctness,
for
example.
A
So
we
want
to.
We
want
to
push
on
that
a
little
bit
and
figure
out
a
good
strategy
for
how
we
allow
for
participation,
grades
and
completion
grades,
and-
and
that's
not
exactly
what
you
were
talking
about,
but
it
just
it
just
that's
what
where
it
took
me
next,
because
we
do
have,
we
could
provide
a
better
way
of
giving
completion
and
being
able
to
see
completion
in
the
summatives,
but
the
formatives
are
another
whole
can
of
worms.
B
One
thing
that
I
think
could
really
help
boost
with
that
aaron,
especially
if
you're
going
to
go
down
the
road
of
completion
grading
for
doing
the
formatives
is.
It
could
be
helpful
to
have
basically
like
a
toolbar
at
the
top
of
the
screen
for
the
students
that
they
don't
have
to
go
to
a
separate
page
to
see
you
know,
did
I
do
this,
or
did
I
do
that,
because
that
requires
an
additional
step
on
the
part
to
the
students
to
go
to
the
grade
book.
B
But
if
you
have
it
as
part
of
the
screen
that
they're,
seeing
as
they're
progressing
through
a
series
of
modules,
it
could
then
highlight
for
them
if
they're
missing
pieces.
So
let's
say
they
missed
doing
three
formatives.
What
if
that
showed
up
as
three
exclamation
marks
on
the
sort
of
progress
bar
at
the
top
of
the
screen?
That
would
be
a
way
for,
if
you're,
going
to
credit
for
formatives
the
real
danger
there
is
that
students
don't
do
it
and
they
don't
realize
that
they
didn't
do
it.
B
So
I
think
it
would
help
us
too,
for
all
of
our
students
to
know.
Okay,
I'm
progressing
through,
but
I'm
progressing
through
and
I'm
missing
pieces
and
they
could
be
different
types
of
colors
for
the
different
types
of
pieces,
they're
missing,
whether
it
be
a
survey
or
quiz
or
one
of
the
learn
by
doings.
But
anyway
I
I
would
love
it
if
it
was
part
of
a
progress
bar
on
the
screen,
so
that
students
don't
have
to
click
out
and
go
to
the
grade
book,
because
we
find
that
students.
Don't
really
do
that.
A
A
So
excellent
point
for
sure,
and
just
a
good
thing
to
probably
do
anyway,
because
I
think
it
would
help
students
with
their
medical
metacognitive
skills
to
be
able
to
just
putting
a
progress
bar
across
the
top.
Keeps
them
aware
that
you
know
this
is
something
you
know
it
was
almost
makes
it
it's
it's
a
very
slight
kind
of
gaming
mechanism
too.
A
B
The
other
possibility,
if
you
don't
add
a
progress
bar,
is
if
there's
like
an
easy
click
way
at
the
end
of
a
module
to
be
like.
You
know,
there's
an
easy,
quick
way
at
the
end
of
this
module
that
you
can
click
on
the
screen
and
it'll
show
you,
oh
did.
I
miss
anything
here
that
way,
it's
again
not
exiting
not
exiting
the
module
but
just
kind
of
cueing
them
into.
I
didn't
do
this,
or
I
did
do
that
because
we
do
occasionally
have
students
that
are
like.
B
A
Okay,
so
let
me
see
here
adding
the
progress
bar
signal
blue
students,
that
these
questions
are
being
graded.
That's
a
good
point
ton
v
and
I
don't
know
we
want
to
be
careful
about
how
we
communicate
grading
and
and
grading
versus
you
know.
Just
progress,
then
maybe
shows
just
completion
based
on
grades
yeah.
A
Those
are
good.
Those
are
good
suggestions
and
ideas,
but
what
tom
v's
question
leads
me
to
think
about?
Is
we
want
to
be?
You
know
careful
about
yeah
what
we
are
signaling
to
the
students
or
even
be
able
another
idea
that
that
just
sparked
for
me
is
giving
as
part
of
setting
up
a
course
section,
maybe
giving
the
instructor
a
place
where
they
can
communicate
how
they're,
using
the
course,
so
in
a
way
that
students
can
can
check
on
or
review
or
look
at
and
say,
okay,
wait.
A
So
I
feel,
like
I've
touched
on
just
about
every
aspect
that
I
wanted
to
touch
on
today,
so,
given
that
I've
collected
a
lot
so
far
and
we
and
then
I'll
be
waiting
for
some
examples.
Thank
you
guys
so
much
for
coming.
I'm
gonna,
probably,
and
I'm
just
gonna
end
the
meeting
a
little
early
and
I'm
glad
victor
and
lauren.
You
were
here
because
you
really
you
were
able
to
comment
on
just
about
every
aspect
of
these
different
pieces
of
analytics
and
reporting
that
give
me
a
good
sense
of
okay.
A
How
I
want
to
separate
out
those
topics
now
and
do
deep,
dives
and
collect
examples
from
you
all
you
all
do
probably
well.
I
know
personally
the
kind
of
analytics
analysis
that
you
all
do
and
it'd
be
nice
to
hear
from
to
reach
out
to
other
researchers
to
find
out
more
about
how
they
feel
about
this.
But
thank
you
guys
for
coming
and
being
so
participant.
B
Yeah
and
aaron,
if
you,
if
you
ever,
want
to
meet
with
me
and
victor,
you
know
you're
welcome
to
sometimes
I
think,
the
most
fantastic
way
to
think
about.
You
know
how
a
new
platform
is
being
developed
and
keeping
what
works
in
an
old
platform
is
to
really
kind
of
go
through
like
what
is
the.
What
is
the
process
that
happens
from,
like
course,
deployment
to
like
when
you're
done
with
the
data
and
all
the
steps
in
between.
B
B
Otherwise
I
I
miss
little
things
like
these.
So
all
of
your
little
queries
and
your
questions
are
very
helpful
to
me
to
prompt
me
to
think
about
well
this
aspect
of
the
process.
This
is
the
workaround
I'm
using
that
I
don't
even
think
is
a
workaround,
but
it
is
so
if
you
ever
want
to
just
kind
of
go
through
that
sort
of
deployment
process,
we're
always
happy
to
meet.
A
Oh,
we
will
we'll
be
reaching
out.
I
mean
I've
been
waiting.
You
know,
we've
been
developing
very
actively
to
get
the
kind
of
baseline
system
in
place
and
the
developers
have
been
doing
a
great
job
of
keeping
their
heads
down
and
really
getting
that
baseline
system
in
place.
So
now
this
year
is
really
going
to
be
focusing,
I
think,
a
lot
more
outreach,
a
lot
more
deep
dives
with
individual
users
and
so
yeah.
A
I
will
take
you
up
on
that
for
sure,
but
you've
also
given
me
an
idea
of
maybe
for
some
some
down
the
road
meetings
to
focus
more
on
just
not
just
individual
questions
but
a
workflow
right.
So
maybe
in
another
meeting
I
can
say:
okay,
here's
this
workflow
kind
of
show
how
the
usual,
how
usual
folks
use
that
workflow
or
perform
things
through
tasks
through
that
workflow
and
maybe
start
to
elicit
different
ideas
based
on
that,
instead
of
instead
of
feature
requests
or
like
workflow
ideas.
A
B
A
Well,
just
you
mentioning
it,
I
think
that's
what
I
was
assuming
people
were
doing
as
I
asked
for
features,
but
your
comment
was
really
helpful
to
help
me
rethink
like
okay.
How
can
I
prompt
more
and
elicit
more
in-depth
information
about
features
and
functionality,
and
that's
that's
the
way
to
do
it.
So
thank
you
great
suggestion.