►
From YouTube: Torus Community Meeting May 2022
Description
Monthly meeting to discuss Open Learning Initiative's next generation platform. In this meeting, activities and question types were discussed.
A
B
A
A
A
Okay,
well,
it's
12
35!
So
let's
get
started.
I
think
there's
a
lot
to
talk
about
today.
Hopefully
the
folks
on
this
call
can
help
give
some
insight
into
what
today's
topic
is
about
and
hopefully
generate
some
good
discussion.
So
I'm
aaron
cerwinski
welcome
to
the
taurus
community
meeting.
We
hold
this
monthly
to
gather
requirements
from
the
larger
community
for
our
for
development
in
our
new
platform
called
taurus
and
happy
to
see
you
all
here
today.
So
let's
get
started,
so
today's
topic
is
kind
of
an
exciting
one.
For
me.
A
If
you
don't
already
know
in
our
legacy
system
what
we
call
our
legacy
system
right
now,
which
is
oli,
delivery,
echo,
which
is
our
authoring
current
authoring
platform-
are
these
types
of
activities
or
questions
and
hi
lauren
thanks
for
joining
us,
so
I
tend
to
use
activity,
sometimes
synonymously
with
questions,
but
if
you
are
aware
of
how
we
develop
questions
and
activities
in
the
oli
platform,
activities
are
usually
groups
of
questions
or
they
can
be
different.
Other
types
of
interactives
that
that
give
students
practice
formative
practice.
A
A
We
also
have
the
ability,
of
course,
to
develop
distractors
as
well
as
write,
specific
targeted
feedback
to
each
of
those
distractors
and
that's
what
we
do
in
pretty
much
all
of
our
activity
types
except
for
one.
At
the
moment,
that's
the
submit
and
compare
and
legacy
they're
all
auto
graded
and
again,
except
for
the
dr
or
the
cement,
compare
and
and
feed,
except
for
the
saving,
conveyor
feed
our
data,
dashboards
and
underlying
data
structures.
A
Feedback
to
each
answer
option
but
also
have
general
feedback
that
could
be
delivered
to
any
of
the.
You
know
you
have
the
ability
to
say:
okay,
if
it's
not
right,
then
deliver
this
feedback,
but
you
also
have
the
ability
to
design
specific
targeted
feedback
for
each
answer.
Option
so
check
all
that
apply.
A
That's
when
a
student
can
check
more
than
one
answer
and
have
to
get
all
of
the
grouping
in
order
to
get
correctness
ordering
is
when
you
can
actually
design
a
question
so
that
students
have
to
drop
something
they
use
a
drag
and
drop
to
order.
The
options
multi-input
and
in
both
cases,
multi-input
in
legacy
and
taurus,
are
where
you
can
have
text
with
inputs
within
the
text,
so
you
could
have
multiple
ones
of
these
in
any.
A
Given
you
know
a
paragraph
you
could
have
two
or
three
of
these
sprinkled
in
a
paragraph
for
students
to
answer
fill
in
the
word
fill
in
the
blank
so
to
speak,
and
we
can
do
that
with
in
legacy.
We
can
use
this
multi-input
type
question.
As
the
answer
can
be
a
text
input
a
numeric
input
you
can
choose
to
have.
A
So
those
are
the
types
that
we
have
currently
available
in
our
legacy
system.
Now
in
taurus,
we
plan,
as
you
know,
the
plan
from
last
month's
meeting
that
we
plan
to
migrate
all
of
our
existing
content
into
the
new
system.
In
order
to
do
that,
we
have
to
have
all
the
same
activity
types
to
accommodate,
what's
been
already
developed
in
those
courses,
but
not
only
that,
but
we
have
some
like
new,
exciting
things
going
on
as
well.
So
multiple
choice,
of
course
check
all
that
apply
ordering
multi-input
in
this
case.
A
Multi-Input
is
a
similar
drop-down
text.
Number
single
response
now
is
the
new
type.
We
have
that
actually
can
be.
A
long
form
longer
form
answer
response
that
you
can
input
a
student
could
input
oops
as
a
sorry
as
a
number
short
text,
or
even
a
paragraph
that
can
actually
be
graded
and
and
checked
against.
A
A
Basically,
we're
going
to
give
folks
because
we're
developing
this
open
source
we're
going
to
give
folks
the
opportunity
and
the
tools
to
be
able
to
develop
multiple
their
own
activities,
but
this
oli
embedded
is
also
a
way
to
build
and
and
use
a
lot
of
the
custom
activity.
Development.
That's
done
in
some
of
our
courses,
so
that
it's
a
generalized
shim
to
play
those
custom
activities
in
taurus.
B
A
A
A
So,
as
you,
I
haven't
been
developing
in
taurus,
yet
I've
been
playing
around,
of
course,
but
I
haven't
actually
and
and
by
the
way,
anyone
who
on
here,
who
knows
better
than
I
about
the
single
response
you
know
feel
free
to
speak
up,
but
let
me
show
you
what
one
of
these
looks
like.
So
that's
a
multi-input.
Let
me.
A
A
This
is
a
single
response
and
you
can
see
here
it
can
be
short
text
number
paragraph
and
then,
if
you
select
answer
key,
you
can
decide
here
whether
you
want
it
to
be
instructor
manually,
graded
or
automatic,
and
if
it's
on
automatic,
you
hear
you
see
the
the
types
of
you
know.
You
can
say
what
the
answer
should
contain
and
give
feedback
targeted
feedback
to
certain
other
types
of
like
it
contains
or
doesn't
contain.
You
can
grab
grade
on
that.
A
I
haven't
played
around
with
this
a
lot
yet
so
I
haven't
seen
the
other
side
of
it
like
where
what
the
instructor
sees
when
the
students
input
this.
So
I
apologize
for
having
that
fully
baked
demo
for
you,
but
basically
we
have
the
ability
to
grade
longer
form,
auto
grade
or
instructor
grade,
longer,
form
responses,
and
of
course
this
is
new.
That's
another
reason
why
I'm
not
super
familiar
with
it,
so
we
can
investigate
use
of
it
even
further.
A
So
essentially,
some
other
things
I
want
to
talk
about
with
regard
to
activities
and
questions
we
now
have.
You
know,
like
I
just
mentioned,
and
I
didn't
show
completely,
but
we
do
have
computer
grading
of
text
more
long-form
text.
We
have
instructor
graded
questions
and
workflows
that
are
different
and
you
can
assign
those
on
any
given
single
question
like
you
can't
do
that
now
in
legacy
oops
we
have
the
ability
to
do
pooling,
but
actually
it's
even
more
powerful
than
pooling
so
question
banks
are
or
activity.
A
Banks
are
a
new
thing
where
you
can
design
questions,
and
let
me
just
show
you
this
real,
quick
I'll,
just
wait.
I
don't
want
to
do
that.
I'm
going
to
do
a
multiple
choice
where
you
can
have
questions,
and
you
can
actually
tag
these
questions
with
any
tags
that
you
want.
So
you
can
imagine
this
could
be
very
powerful.
You
could
tag
questions
with.
A
A
Let
me
show
you
this:
how
this
works.
My
internet
is
slow
2d.
Where
I
can
decide.
I
want
to
put
a
activity
bank
selection,
so
this
means
that
there's
a
question
here
that
will
be
pulled
from
the
activity
bank
based
on
my
selection.
So
how
many
from
the
bank
to
select
and
then
I
can
add
the
criteria
so
say
I
just
want
random
question
here
in
this
page.
For
my
pool
of
you
know
five
that
are
tagged
with
a
particular
tag.
A
I
can
say
you
know,
pull
pull
question
that
equals
this
tag
or
doesn't
equal
this
particular
tag.
So
I
think
that's
a
really
interesting
and
powerful
piece
of
functionality
that
we
didn't
have
before
that
again.
I
haven't
played
around
with
extensively,
but
I'm
already
imagining
the
ways
it
could
be
used.
A
Then
we
have
we're
trying
to
still
sort
out
this
idea
that
there
are
titles
on
questions
so
from
showing
echo
these
titles
and
what
they
actually
feed
and
show
right
now,
when
you
title
a
question,
it'll
show
up
in
the
insights
page
with
that
title
and
we're
still
tinkering
around
with
that
and
then,
of
course,
for
the
any
activities
that
we
have
or
activity
types.
There's
considerations
about
data,
how
you
see
the
data,
how
you
view
the
data?
A
What
unique
identifiers
are
part
of
that
data,
so
you
can
trace
things
back
to
specific
questions
and
outcomes
and
we're
building
more
pedagogical
type
tips
into
taurus
as
well,
so
that
when
someone
is
creating
generating
activities,
designing
activities
they
have,
I
just
want
to
go
back
to
that
again.
Show.
A
The
kind
of
tips
that
are
included
so
right
now
we
have
you
know,
suggested,
suggested
lead-in
content,
for
you
know
that's
geared
towards
that
type.
You
know
if
you're
gonna
pick
an
ordering
activity,
then
you're
gonna,
probably
I
might
put
that
in
there
actually,
but
the
pedagogical
guidance
I'm
talking
about
is
you
know,
feedback
for
correct
answer.
A
So
a
lot
more
flexibility,
functionality
and
kind
of
you
know
sky's
the
limit
ideas
for
activities
and
and
interactives
in
taurus,
so
for
discussion
today
I
thought
I
would
just
show
you
that
stuff
really
quick
quickly,
but
really
thinking
about
all
of
your
activity,
types
that
you
currently
used
think
about.
How
would
you
transfer
those
over?
Would
you
make
use
of
the
new
functionality?
A
A
Maybe
what's
been
on
your
wish
list
for
a
while,
again
kind
of
thinking
back
to
the
stuff
that
you've
developed
so
far
and
you
thought
wow?
I
could
really
do
this
cool
thing
if
I
had
these
types
or
if
the
the
types
of
questions
that
oh
all
I
had
worked
in
this
way,
so
I'm
really
open
to
hearing
any
of
your
ideas.
Any
of
your
pain
points
around
question
and
activity
development
to
really
start
to
get
ideas
of
what
else
we
might
need
or
how
we
might
make
the
ones
that
we
have
more
robust.
A
I'm
also
interested
in
any
kind
of
data
analysis
that
you
do
and
if
what
information
that
you
review
on
a
regular
basis,
either
regards
to
individual
students
or
aggregated
in
in
multiple
ways,
and
then
are
there
accessibility
and
dei
considerations
that
we
should
be
making.
Of
course,
accessibility.
We
want
everything
to
be
able
to
be
screen,
readable
tab
through
that
sort
of
thing,
but
are
there
other
things
that
you
can
think
of
that
might
help
with
help
people
with
disabilities?
A
B
B
A
A
A
Yes,
it
does,
and
it's
something
I
was
just
talking
someone
about
not
to
like
this
week.
I
can't
remember
who
exactly
or
what
the
context
was,
but
we
were
saying
yeah
we
would
need
you
know
some
sort
of
audio
voiceovers
for
for
visually
impaired
students
for
a
lot
of
things.
You
know
detailed
images
where,
instead
of
alt
text
alone,
the
video
or
the
the
voice
over
the
audio
would
be
more
helpful
in
some
of
those
situations
and
maybe
even
easier
to
produce
easier
to
consume.
A
But
then
yeah
have
it
in
a
way
where
you
know
the
seeing
students
couldn't
necessarily
use
that
as
a
as
a
way
to
answer
the
questions
either
it's
tricky,
but
I
think
it
helps
everybody
when
we
have
multiple
representations
of
the
content
anyway.
So
and
it's
formative
practice.
So,
if
they're,
even
if
they're
listening
to
it
and
getting
the
answers,
it's
it's
might
be
okay,
but
we're
also
considering
for
dragon,
drops,
there's
another
consideration
that
we
have
to
take,
and
that
is
for
motor
impaired
students.
A
Who
really
can't
even
do
that
dragging
motion.
What
do
we
do
for
those?
So
we're
really
going
to
have
to
carefully
think
of
and
probably
get
some
expertise
and
how
we
might
turn
some
of
our
existing
question
types
into
accessible
versions
or
or
do
things
with
that
type
that
easily
help
or
easily
can
be
easily
tabbed
through
or
and
make
it
very
easy
for
for
students
to
use
the
keyboard
or
even
in
some
cases,
students
who
might
be
using.
A
A
B
That
the
more
audio
voice
over
opportunities
that
we
have
the
better
I
agree,
I
think
that
drag
and
drops
are
inherently
not
not
accessible,
which
is
problematic.
I
you
know
when
you
think
about
you,
know
the
benefits
of
dual
coding.
So
dual
coding
is
fantastic,
but
it's
not
it's
not,
except
you
know,
dual
coding.
The
way
it's
traditionally
thought
of
is
not
accessible
for
visually
impaired
students.
A
Yeah
and
I'm
really
interested
in
exploring
all
of
that
one
of
the
the
things
I'm
trying
to
really
push
on
is
accessibility.
In
terms
of
you
know,
how
can
we
make
the
most
accessible
courses
available
out
there?
I've
had
some
experiences
this
year.
That
made
me
realize
how
how
difficult
it
is
for
students
who
have
any
disabilities
to
to
learn
it's
not
just
about
consuming
material
and
web
accessibility.
A
You
know
you
want
to
be
you
can't
you
don't
want
to
write
it
to
the
detail
where
you're
you're
providing
more
cognitive
load
for
a
student
than
is
necessary,
but
you
have
to
write
it
in
a
way
where
they're
getting
the
point
of
why
that
thing
is
there
is
and
part
of
the
learning
and
that
they
can
actually
recall
and
learn
it.
So
it's
it's
a
really
interesting,
fascinating
field
that
I'm
you
know.
I've
always
been
cognizant
of
always
as
an
instructional
designer
of
online
content.
A
B
A
It's
interesting,
I
never
thought
about
that.
I
think
we've
already
had
a
request,
especially
from
our
our
our
wonderful
languages
department,
to
have
audio
submission
as
a
as
an
input
as
a
as
an
activity
type.
Essentially,
I'm
trying
to
think
if
we
do
that
right
now,
I'm
actually
going
to
look
here.
A
I
need
a
short
answer
to
go
in
here
I
mean
I
can
add
an
audio
clip
that
I
produce
somewhere
else,
the
actual
that
was
for
the
question.
I
don't
know
for
an
answer
if
we
can
do
that
yet,
but
that
would
be
really
cool
if
we
could.
What
we
want
to
do
even
further,
though,
is
build
in
a
tool
where
they
can.
A
I
think
this
is
way
more
of
what
you're
thinking
of
where
they
don't
just
select
an
audio
clip
they
produce
somewhere
else,
but
they
can
produce
the
audio
clip
right
there
as
part
of
the
quest
answer
and
it
gets
submitted
to
the
system
right
now,
yeah
right
now,
I
can
imagine
you
know
baby
steps
for
that.
I
can
imagine
we
have
the
ability
to
input
audio,
but
then
have
the
player
right
there
to
input
audio
and
then
the
next
step
would
be
to
actually
like
turn
it
into
text
and
grade
it
right
now.
A
I
think
we'll
probably
build
I
can
imagine
we
would
build
something
where
you
can
input
audio
capture
it
and
then
have
it
graded
by
the
instructor
manually.
I
would
also
like
to
see
I
mean
for
talking
wish
list
here.
I
would
like
to
see
the
ability
for
that
to
go
into
maybe
a
peer
grading
system.
You
know
where
you
can
then
sign
other
students
to
kind
of
listen
and
grade
it,
and
I
you
know
I
can
imagine
all
kinds
of
cool
stuff
when
it
comes
to
activities.
B
A
And
natalie,
let
me
test
that
out.
I
was
going
to
test
it
and
then,
but
I
gotta
I
gotta
test
both
sides
of
it.
I
want
to
test
the
the
answering
of
the
question
and
then
the
grading
part
as
well.
What
I
was
what
yeah,
what
I
was
showing
right
now
is
just
the
ability
to
really
embed
an
audio
clip
anywhere.
I
asked
the
question
as
an
answer
option
I,
but
I
got
to
test
out
fully
if
you
can
do
it
as
as
a
response
to
a
question.
A
A
That's
a
really
exciting
part
of
it.
For
me,
is
you
know
a
lot
of
these
courses
so
far
have
been
developed.
I
don't
want
to
stay
in
complete
isolation,
but
they've
been
developed,
and
you
know
the
the
the
subject
matter.
Experts,
the
teachers
and
the
designers
who
were
part
of
that
course
could
imagine
lots
of
cool
stuff
and
develop
it,
especially
for
that
course,
but
it
wasn't
necessarily
shareable
then
across
to
other
courses,
and
course
developers.
So
I'm
excited
about
that
opening
up
a
lot
of
possibilities.
A
I'm
excited
to
play
around
with
the
the
single
response
too,
to
see
how
well
that
can
be
graded
automatically,
and
you
know
we'll
be
giving
a
tourist
demo
of
the
latest
functionality
and
everything
in
the
next
tourist
community
meeting,
which
is
why
I
didn't
quite
prepare
to
give
demos.
I
try
not
to
give
too
many
demos.
A
So,
even
though
I
know
you
probably
want
to
see,
what's
all
what
all
the
latest
and
greatest
is
but
we're
developing
so
fast,
I
need
time
to
kind
of
play
around
with
things
first
before
I
can
give
a
really
thorough
demo
and
answer
good
questions,
so
I
am
planning
for
that
in
the
next.
The
next
meeting
and
there'll
be
a
lot
of
cool
stuff
to
show
off.
B
A
Yeah,
that's
interesting.
I
actually
again
forgive
me
for
not
knowing
the
answer
that,
but
I
can
imagine
we
have
to
do
something
similar
because
we
have
a
lot
of
those
in
courses
migrated
already.
I
think
the
single
response
and
I'm
going
back
to
that
right
now
to
see
I
think
if
you
yeah,
that's
a
good
question,
I'm
going
to
I'm
going
to
put
that
on
the
list
to
make
sure
that
we.
A
I
mean
we
know
that,
because
it
will
need
maybe
needed
to
migrate
those
questions.
I
can't
imagine
we're
going
to
change
all
of
those
into
a
question
type
that
has
to
be
either
manually
graded
or
is
like,
because
none
of
those
know
that
feedback
was
written
to
be.
You
know,
character,
you
know,
character,
character,
recognition,
so
we're
going
to
have
to
have
some
way
of
doing
that
and
we
might
be
able
to
do
that
with
the
current
type
already
and
I'm
just
not
aware
of
how.
B
A
A
A
A
You
control
this
kind
of
stuff,
I
I
am
remembering,
and
maybe
it
just
didn't
work
out
where
we
did
have
the
ability
to
say.
Okay,
start
the
video
at
this
time
and
end
at
this
time,
so
that
you
could
hone
in
on
a
particular
few
seconds,
but
maybe
that
didn't
work
out
and
we
didn't
build
it
into
echo.
A
I
can't
forget
you
know
forgive
me
for
not
knowing
that,
but
I
agree
with
you
either
way
we
want
that
more
built
in
and
we
have
to
really
we're
going
to
have
to
open
up
and
expand
the
way
we
embed
video
because
we're
already
being
asked
by
some
of
our
partner
organizations
to
allow
for
things
other
than
youtube
or
vimeo.
So
this
is
something
that
we
are
exploring.
A
April
nice
to
see
you're
here
I
am
you
say,
you're
saying
you
need
to
be
able
to
export
an
aggregated
participant
by
module
for
all
modules
from
the
learning
dashboard
or
if
the
percent
complete
can
be
included
in
the
gradebook.
Yes,
this
is
something
we
get
we
get
asked
for.
I
swear
every
day.
Is
the
ability
to,
and
let
me
go
back
to
my
notes
here-
make
sure
I
capture.
A
And
this
I
lost
the
chat,
the
ability
to
know
basically
percent
completion
of
the
formatives
and
and
then
be
able
to
use
that
as
a
as
a
grading
mechanism
of
some
sort.
So
we've
been,
I
swear.
I
think
I
have
that
conversation
every
other
day
with
folks
who
really
want
that.
So
let
me
make
sure
I
capture
that
and
I'm
sure
we
already
have
some
user
stories
around
that.
So
we
are
carefully
thinking
about
not
just
replicating
the
the
learning
dashboard
as
it
is.
A
All
of
that,
so
we're
going
to
be
doing
some
pretty
cool
stuff
around
dashboard
and
really
expanding
the
ability.
I
know
in
norms.
Norm's
vision
is
to
also
be
able
to
have
other
folks,
maybe
build
visualizations
and
dashboards
and
see
how
they
work
with
our
with
our
content.
So
we're
really
thinking
deeply
about
all
of
that.
But
let
me
capture
this
just
to
make
sure
okay
need
to
see.
A
Where
are
we
at
with
pass
back
activities
from
canvas
to
tourists?
Oh,
my
goodness
april.
We
need
to
catch
up
because
there's
a
lot
going
on
we're
using
the
latest
lti,
so
grade
pass
back,
is
seamless,
essentially
and
and
and
controllable.
So
basically
anytime,
you
have
a
graded
page
in
taurus,
which
is.
A
Let
me
show
you
what
that
would
be,
because
when
you're
building
the
curriculum,
you
have
the
ability
to
do
a
practice
page
or
graded
page,
and
when
you
select
a
graded
page,
it's
just
like
any
other
page,
but
what
it
does
is
it
signals
to
the
lms
that
you're
using
to
put
a
column
in
the
gradebook
for
that
particular
page?
So
that's
already,
and
then
you
have
additional
controls
as
an
instructor
to
be
able
to
say.
Oh
no,
I'm
not
going
to
grade
this
page.
I
don't
want
it
here
or
oh.
A
A
So
what
she's,
referring
to
what
april's
referring
to,
is
essentially
the
old
way
we
used
to
look
at
data
activity
data.
It
essentially
is
replicated
here
with
this
insights,
page
and
and
echo
actually
I'll,
show
it
in
echo
first,
because
it's
pretty
cool
and
I'm
going
to
show
actually
your
course
april,
because
yours
is
the
is
my
demo
for
that.
A
Here
so
for
those
of
you
who
don't
know,
we
have
analytics
that
come
back
from
use
of
your
course
and
we've.
When
we
built
echo,
we
pulled
it
into
echo
in
such
a
way
that
you
can
generate
the
data
and
then
look
at
you
know
by
learning
objective
by
skill,
the
questions
and,
basically
the
performance
and
questions.
A
We
used
to
call
this
the
magic
spreadsheet,
because
we
used
to
actually
norm
a
long
time
ago,
written
scripts
to
be
able
to
pull
this
data
out
and
then
put
it
in
a
nice
little
spreadsheet,
where
you
can
view
this
and
it's
color
coded
and
everything,
and
so
what
this
shows
is
the
percentage
of
students
who
get
it.
That
question
rank
the
first
time,
those
who
get
it
eventually
the
percentage
gets
it
eventually
right.
A
A
difficulty
based
on
the
number
of
hints
students
might
have
needed
for
that
question,
and
then
this
is
just
the
number
of
attempts,
and
this
data,
as
you
can
imagine,
is
pretty
powerful
in
terms
of
iterative
improvement
efforts.
So
we
did
replicate
this
in
taurus,
but
now
we
have
even
better
control.
So
now
you
can
actually
show
this.
These
analytics,
by
page
by
activity
by
objective
and
review
that
same
type
of
information
this
course
I'm
showing
right
now,
has
like
no
data
in
it.
So
that's
why.
A
A
And
make
sure
it's
in
our
notes,
because
this
stuff
gets
turned
into
user
stories.
For
those
of
you
who
don't
know,
we
take
all
these
requirements.
We
talk
about
and
turn
them
into
user
stories
so
that,
when
the
engineering
team's
ready
to
tackle
that
piece
of
functionality,
we
have
a
lot
of
information
about
how
folks
would
like
to
see
that
play
out.
A
You
know
exactly
what
you
look
at
what
you
would
like
to
see
that
you
don't
have
access
to
how
you
would
like
to
see
it
displayed
and
that
sort
of
thing
right
now
we're
focused
mainly
on
migration,
but
we're
going
to
start
yeah.
I
will
definitely
be
reaching
out.
In
fact,
I
think
we've
been
we've
both
been
missing
each
other,
because
I've
been
wanting
to
do
this
for
a
while.
A
A
You
know
big
line
of
characters,
whereas
if
you
did
it
in
the
xml,
then
you
have
like
actual
names
that
you've
purposely
and
thoughtfully
coded
as
the
ids
of
those
questions,
so
yeah
we're
trying
to
figure
out
how
to
make
use
of
all
of
that
all
of
those
unique
identifiers.
Or
what
do
we
generate
so
that
when
you
see
the
data,
it
makes
sense.
A
I
volunteered
you
and
you
signed
up
so
great
lauren
same
with
you
I'll,
probably,
and
then
even
the
language
language
courses
have
a
whole
additional
layer
which
I
believe
you
and
lauren,
but
lauren
shares
as
well,
which
is
the
courses
are
used
by
other
institutions,
and
you
want
to
start
to
gather
data
about
the
use
in
specific
institutions
and
be
able
to
kind
of
have
an
overall
administration
of
all
of
your
customer
and
use
course.
Sections
used
by
the
customers
right.
B
Yeah
and
that's
it's
going
to
be
interesting
because
you
know
we
have
our
own
universities
use
and
the
data
that
we
collect
for
that
is
kind
of
inherently
different,
because
that's
administered
on
a
course
by
course,
basis
versus
the
data
that
we
will
be
getting
through
cmu's
platform.
So
yeah
there's
a
lot
of
considerations
there
and
I
think
we've
already
talked
about
my
wish
list
a
lot
for
what
I
you
know
what
I'd
love
for
data
exports.
B
A
previous
list-
I
didn't
see
it
in
your
demo,
so
I
don't
know
if
that
means
the
capability.
Isn't
there
yet
is
just
simply
conversion
conversion,
question
type
from
multiple
choice
to
multiple
answer
and
back
and
forth.
Just
since
it's
part
of
today's
presentation,
I
didn't
know
if
you
wanted
to
add
it
to
this
data
story.
So
if
we
have
a
question,
that's
currently
multiple
choice
and
we
just
need
to
convert
it
to
multiple
answer
or
vice
versa,
so
that
we
don't
have
to
remake
the
question.
B
B
Are
getting
an
like
more
students
are
choosing
wrong
than
we
think
that
they
should
for
a
particular
question,
we'll
consider
change.
We
change
the
question
type
often
we'll
say
like.
Would
this
be
clearer
if,
instead
of
being
a
multiple
choice
from
selecting
one
answer,
they're
selecting
actually
multiple
scenarios
and
there's
now
multiple
answers
correct
and
we
have
to
remake
the
entire
question.
So
that
would.
A
A
I
wouldn't
call
it
an
overhaul,
but
you
know
looking
at
the
data
looking,
you
know,
surveying
instructors,
we've
done
a
page
by
page
review,
with
subject
matter,
experts
to
identify
places
of
things
that
need
updated
and
changed,
and
now
we're
going
back
through
with
the
same
subject
matter,
experts
to
make
those
changes
to
identify.
Okay.
If
they
said
this
page
needs
to
be
updated,
multiple
ways:
okay,
giving
them
the
the
ability
to
see
all
of
that
information
and
make
comments
on
it
play
around
with
different
wording.
A
That
sort
of
thing
so
for
any
of
you
and
I'm
sure,
lauren
and
you
know
actually
all
of
you
have
probably
done
this
a
lot
where,
because
you
you
all
support
courses
that
have
been
in
oli
for
a
long
time,
and
this
is
the
first
time
I'm
doing
like
a
full
overhaul
like
that
and
the
workflows
that
we've
had
to
use
for
that
are
really
bad
and
hard
right.
You
know
we
identified
about.
You
know
800
different
things.
A
You
know
in
echo,
okay,
I'm
going
to
change
the
sentence
to
this
sentence,
but
actually
thinking
about
it
from
the
view
of
a
holistic
view
of
a
page
or
a
module
and
thinking.
Okay,
here's
the
learning
objective.
If
I
make
these
changes,
does
that
learning
objective
still
stand?
Do
the
activities
need
to
change?
Does
the
feedback
in
the
activities
need
to
change
you
know
and
seeing
all
of
that
in
one
place
to
be
able
to
think
about
it?
Cohesively?
A
Sorry,
I'm
like
right
in
the
middle
of
this,
so
it's
really
eating.
It's
like
on
my
mind,
all
the
time
like.
How
can
I
make
this
easier
and
so
caitlyn
and
alex-
and
I
have
been
talking
a
lot
about
that
and
figuring
out
ways
you
know
and
writing
user
stories
that
will
add
functionality
that
will
ease
iterative
improvement
efforts.
So
this
conversion
question
type
is
one
of
those
things
I
hadn't
thought
of.
A
B
What's
what
I
can
also
see
happening
is
us
deciding.
You
know
we're
not
really
understanding
from
this
that
we're
not
really
understanding
why
students
aren't
grasping
this
particular
question
and
the
best
way
to
get
at
that
is
to
have
them
type
in
an
answer.
So
that
could
also
be
one
of
those
question
types
where,
as
part
of
iterative
improvement,
we
say
like
we're
going
to
change
this
question
type
to
that,
to
garner
more
information
and
then
make
a
decision
about
how
to
alter
the
question
in
the
future.
B
And
yeah,
to
answer
your
question
to
caitlin,
I'm
always
happy
to
talk
about
the
iterative
improvement
process
that
we
engaged
in.
It's
very
intense
and
I
know
not.
Everybody
would
have
time
to
do
the
level
of
iteration
that
we
did,
but
that
doesn't
mean
that
we
haven't,
you
know,
learned
some
things
that
would
maybe
streamline
it
for
other
people
in
the
future.
A
A
A
The
question
here
are
the
distractors
that
I'm
writing
for
a
multiple
choice
question,
but
we
need
a
way
to
not
only
view
that
data
in
a
in
a
single
place,
but
also
then
be
able
to
turn
some
of
those
questions
around
and
put
them
in
the
course.
So
this
is
a
way
we're.
Thinking
of
I
mean,
there's
a
lot
of
reasons
to
do
more
learner
sourcing,
there's
dei
reasons,
but
this
one
of
the
reasons
is
to
be
able
to
help
folks
generate
lots
more
practice
right.
A
A
I've
been
talking
with
some
folks,
some
companies
that
offer
you
know,
question
banks
for
math
problems,
for
example,
and
it's
something
that's
that's
always
asked
about.
Is
you
know
more
and
more
practice?
How
do
we
get
more
and
more
practice
into
these
courses,
and
I
thought
that
might
be
one
way?
Would
anybody
here
be
interested
in
being
able
to
upload
and
use
activity
banks
from
other
sources.
A
B
A
B
B
To
think
about
you
know,
I
hate
to
think
you
know
this
way,
but
I'm
always
like
kind
of
that
like
well.
What
are
the
challenges
to
this
based
upon
how
students
use
existing
software?
But
you
know
we
can't
all
together
combat
that,
but
just
thinking
about
how
it
would
be
good
not
to
have
it
would
be
good
to
like
force
them
to
type
in
the
question.
Maybe
not
have
a
copy
paste,
just
an
idea
like
that
can
be
something
that
can
prevent.
You
know
just
like
the
fast.
A
But
what
I'm
thinking?
What
I
thought
of
that
I
hadn't
thought
of
before
is
the
fact
that
if
we
do
that,
then
we
have
to
be
very
careful
about
the
data
considerations
right
and
we
want
to
make
sure
that
if
we
are
going
to
have
a
participation
score-
and
we
add
all
this
extra
practice
in-
we
need
a
way
to
not
count
that
or
to
give
credit
for
the
extra
practice.
But
without
penalizing
students
who
didn't
do
it.
So
those
kind
of
have
to
be
kind
of
not
part
of
the
overall
participation
score.
A
A
Yeah,
a
couple
things
that
keep
trying
to
keep
forefront
of
everybody's
mind
with
any
new
functionality
is:
how
could
that
be
done
to
iteratively?
How
could
iterative
improvement
be
done?
What
are
the
data
considerations
in
terms
of
what
does
this
look
like
when
you
start
collecting
data
on
these
things?
A
I
was
calling
them
like
swim
lanes
and
we
started
thinking
about.
You
know
different
personas
that
need
to
take
advantage
of
the
same
functionality
but
use
it
in
different
ways.
So,
like
you
know,
what
would
researchers
do
with
this
this
functionality?
What
would
how
is
it
different
for
learners
versus
instructors
so
really
trying
to
keep
any
time?
We
talk
about
any
functionality
to
keep
okay?
Well,
how
would
this
impact
the
different
user
personas,
and
how
would
they
use
it,
maybe
slightly
differently
from
each
other.
A
B
Example:
activity
like
I
mean
that
would
I
mean
that's
very
selfish.
That
would
be
great
for
us,
but
I
don't
think
it
would
just
be
great.
For
us
I
mean
it's
perfect
for
quantitative
courses,
so
just
having
sort
of
the
multi-step
thing
be
part
of
it,
and
then
students
can
type
their
notes
on
the
side
it.
A
A
Okay,
we
need
one
of
these
on
every
page
start.
Here's
the
template
start,
you
know
doing
those
building
more
of
those
work
examples,
and
can
I
even
imagine
some
really
cool
things
in
terms
of
making
an
activity
type
where
students,
like
you
said
either
can
write,
notes
or
have
they
have
to
provide
the
next
step,
and
then
how
would
that
be
graded?
Oh
okay!
Yeah!
You
got
me
excited
about
that.
It
seems
like
such
an
important
thing
that
we
should
have
already.
B
Module
but
the
text
box
entry,
so
what
we
can
do
is
you
and
I
can
meet
about
that-
we'll
go
over
it
and
we'll
talk
about
how
to
translate
that
into
something
that's
more
workable
but
yeah.
That's.
I
also
like
that.
You
brought
up
fading
because
you
know
one
of
the
things
that
you
talked
about
earlier
is
adapt
adaptive
capabilities,
and
that
would
be
a
really
fancy
and
fun
adaptive
capability
to
have
the
fading
feature.
Work.
A
Yeah
that
gets
me
excited,
so
I
I
mean
you
guys
have
no
idea.
I
cannot
wait
until
we
have
all
of
the
the
courses
migrated
and
everything's
working
and
beautiful,
and
for
us
what
I
want
to
get
to
more
than
anything,
because
I
don't,
I
feel,
like
I've,
never
really
gotten
to
it
in
my
life,
but
I've
had
a
taste
of
it,
which
is
you
know,
envisioning
new
activity,
types
for
the
material
that
you're
working
on
you
know.
A
I've
worked
on
lots
of
different
courses
and
different
subject
matters,
and
I
can
already
you
know
every
time
I
can
imagine
a
new
type
of
activity
that
I
would
love.
You
know,
building
a
world
history
class
being
having
being
able
to
have
an
interactive
map
or
a
timeline
tool.
You
know
I've
done
a
little
bit
of
this
in
my
lifetime
and
especially
back
when
we
were
doing
the
xml
like
and
we
we
had
technologists
available
to
us
to
just
sit
with
and
say:
okay,
here's
what
I
envisioned
I'll
make
it.
A
A
B
Yeah-
and
the
great
thing
about
echo
is,
I
feel
like
you:
can
you
can
have
all
these
work?
Arounds
right
and
I
know
you're,
trying
to
like
work
towards
not
having
workarounds
and
that's
great,
but
at
least
echo
was
very
malleable
in
terms
of
allowing
us
to
do
all
the
things
that
we
need
to
do
so.
No
complaints
there.
A
Oh,
I
also
want
to
mention
right
now
and
probably
for
the
demo.
Oh,
I
can't
promise
it
for
the
demo
next
month,
but
I'm
hoping
I
mean
we're
we're,
building,
surveying
type
questions
and
ability
to
deliver
surveys
or
ungraded
questions
in
a
more
robust
way
and
make
it
easier
for
you
to
do
different
types
of
surveying.
A
We
get
more
and
more
requests
for
that
type
of
thing,
like
ungraded
question
types
right
where
you
don't
really
necessarily
want
to
give
a
score,
but
you
want
to
collect
some
sort
of
information
or
even
help
the
students
reflect
whether
you
don't
want
to
actually
have
to
say.
Oh,
you
were
right
or
wrong.
A
A
B
But
I
was,
I
was
attending
this
thing
about
new
quizzes
in
canvas
this
week
and
new
quizzes
in
canvas
now
has
basically
a
calculator
two
different
types
of
calculator
features
that
come
up
in
the
new
quizzes
platform,
so
the
calculator
feature
comes
up
for
the
students
which
I
think
is
fabulous,
but
I'm
wondering
if
you
had
that
for
oli,
then
what
you
could
also
do
is
get
data.
It's
like
click
data,
but
you
could
get.
You
could
get
pathway
data
on
what
students
are
actually
typing
in
with
the
calculator
feature.
A
A
So
we
do
have
the
ability,
in
some
other
words
like
chemistry,
to
be
able
to
it'll
pull
up.
Like
a
you
know,
an
equation
generator
that
will
then
serve
as
an
answer
to
a
question
again,
I'm
not
as
familiar
with
it,
but
it's
not
quite
the
same
thing.
So
that's
interesting
I'll
I'll
go
look
at
what
was
that
called
again.
B
It's
in
canvas
and
it's
in
the
new
quizzes
feature,
so
there's
like
old
quizzes
and
the
new
new
quizzes
has
just
kind
of
been
released
this
past
semester-
and
I
just
was
shown
this
like
amazing
calculator
feature
and
I
was
like
well
that's
fancy.
I
wonder
if
oli
is
collecting
click
data
on
like
entered
calculator
responses,
because
I
thought
that
was
pretty
cool.
A
A
A
So
well,
folks,
let's
see
is
there
any
discussion
points
that
we
haven't
touched
on
so
more
demos
later
for
sure,
like
I
said
earlier,
I'm
not
always
in
a
position
to
be
able
to
give
an
extensive
demo,
but
I
do
plan
for
those
I
think,
there's
a
couple
points
throughout
the
year
that
you
know
will
give
deep
dive
demos
into
what's.
New
we're
also
planning
on
is
around
this
sdk
for
developing
new
activities
and
integrations.
A
The
first
step
is,
of
course,
to
have
the
documentation
steps
after
that
are
to
actually
hold
some
webinars
to
walk
folks
through
the
documentation,
but
then
we
eventually
will
hope
to
have
some
like
really
hands-on
workshops
where
you
can
learn
to
use
the
sdk
to
develop
custom
activities.
So
we're
really
excited
about
that
in
the
future.
A
I
would
say,
please
go
back
to
any
notes
that
you
guys
have
had
about
you
know
or
old
emails
that
you
sent
oli
to
say:
can
we
have
this
type
of
activity
or
could
we
have
it?
Do
this
and
feel
free
to
just
bombard
me
with?
You?
Can
just
forward
me
stuff
that
you
used
to
that
you've
sent
before
or
you
know,
type
something
up
sent
to
me.
A
Okay,
well,
then,
I'm
gonna
call
it
a
meeting.
Thank
you
all
for
joining
in
and
being
so
and
sharing
all
the
great
information
about
your
experiences
and
what
you'd
like
to
see
and
how
you'd
like
to
see
things
play
out,
really
appreciate.
It
couldn't
be
doing
any
of
this
without
you
all
to
to
share
that
information
with
us
and
to
be
the
folks
to
kind
of
test
these
things
out.
I
can't
wait
to
get
many
of
you
into
taurus
actively
developing.
A
So,
if
you're
interested,
you
can
always
go
out,
create
an
account
and
start
to
play,
but
we're
still
working
on
the
migration
plan.
But
we
just
know
that
when
we
get
to
your
courses,
you
will
be
consulted.
We're
not
just
gonna,
you
know,
say:
oh
hey!
By
the
way
we
we've
changed
everything
in
taurus.
A
Yes,
thanks
aaron
thanks
guys,
and
I
will
be
talking
with
each
of
you
soon
and
join
us
for
the
next
one,
which
will
be
a
demo
of
of
taurus.
I
was
starting
to
try
to
list
out
all
the
new
stuff
that
you'll
get
to
see
in
there
and
there
was.
There
was
more
that
I
could
just
list,
so
I
just
stopped
at
products
and
sections,
but
lots
of
good
stuff
so
see
you
next
time
and
until
then
have
a
great
rest
of
your
month.