►
From YouTube: Torus Community Meeting October 2021
Description
Monthly meeting to discuss Open Learning Initiative's next generation platform. This meeting is about A/B testing functionality needed to support research.
B
B
Welcome
everyone!
I'm
glad
you're.
Finding
your
way
here,
even
without
a
reminder
sent
out
we're
in
the
process
of
changing
our
crm,
and
I
haven't
had
the
chance
to
to
set
it
up
to
send
out
reminders.
So
I'm
glad
you
guys
are
joining
regardless.
B
We'll
get
started
as
usual
about
12
35
eastern
time,
just
to
give
people
time
to
grab
lunch
or
whatever
find
their
way
over
to
the
meeting.
So
we'll
start
in
a
little
bit.
B
Welcome
folks,
who
are
joining
us
just
now
usually
get
started,
get
started
about
one
minute.
B
Okay,
I
think
we're
gonna
dive
right
in
so
welcome
everyone
to
the
tourist
community
meeting
today,
as
I
expected
it
to
be
late.
I
mentioned
earlier
that
the
mailer
that
I
usually
use
to
send
out
a
reminder,
we're
switching
our
client
and
didn't
get
a
chance
to
send
out
a
reminder,
but
we're
going
to
do
something
similar
next
month
as
well.
So
maybe
we'll
get
a
bigger
crowd
then,
but
this
is
fine,
I'm
so
glad
to
have
you
guys
here,
I'm
going
to
be
giving
a
demo
of
taurus.
B
Let's
see
here
today's
topic,
taurus
demo
show
you
what's
in
the
works
and
what's
been
going
on
with
taurus.
I
wanted
to
let
you
know
that,
as
I
do
the
demo
I'll
probably
point
out
things
that
are
different
from
our
legacy
system.
I'll
also,
hopefully
key
you
in
on
to
upcoming
plans
for
some
functionality
know
that
we're
started.
We
kicked
off
this
week,
actively
planning
migration.
B
So
what
we
mean
by
that
is
that
we
plan
to
have
every
course
that
is
currently
being
delivered
from
our
legacy
platform
migrated
over
to
taurus
and,
as
you
can
imagine,
that's
a
lot
of
planning
as
tourists
features
pair
up
with
the
courses
that
we
have
and
so
require
some
careful
planning
and
we're
gonna.
Do
some
piloting
see
some
lessons
learned
and
then
plan
out
really
hopefully
get
a
plan
in
place
for
the
new
year?
That
will
tell
everybody
exactly
when
we
hope
to
migrate.
Their
particular
course
so
know.
B
That's
in
the
works.
Don't
have
anything
more
at
the
moment
for
you
about
that.
But
as
soon
as
I
do,
I
will
be
sharing
that
out
and
I'd
like
to
have
some
conversation
not
only
to
get
your
reactions
from
the
demo,
but
also
to
talk
about
ideas
for
meetings.
I
want
to
plan
like
the
next
year
of
tourist
community
meetings
and
it'd
be
really
interesting
to
find
out
from
you
all.
What
kind
of
topics
you'd
like
to
see
me
delve
into
and
discuss
with
a
larger
community.
B
So
with
that,
let
me
go
over
to
the
demo.
B
So
I'm
going
to
be
doing
this
live,
and
so
I
don't
I'm
kind
of
sometimes
surprised
by
features
that
are
coming
online.
That
have
recently
come
on
so
bear
with
me,
as
I
fumble
through
a
demo,
but
pretty
it's
pretty
intuitive
I've
been
able
to
give
demos
pretty
frequently,
and
you
know
it's
I'm
always
impressed
when
I
come
in
here
that
the
dev
team
is
doing
a
fantastic
job
and
I
think
you'll
you'll
be
impressed
as
well.
B
On
here,
I'll
put
this
in
the
chat
in
case:
you
don't
have
it
and
sign
up
for
an
account,
I'm
an
admin
in
the
system,
so
I
have
a
list
of
like
every
possible
thing
in
here,
but
what
I'm
going
to
show
off
today
for
us
is,
if
I
can
manage
my
windows
here.
B
There
we
go
so
we
did.
We
actually
had
to
have
a
demo
of
statistics
in
here
and
it's
not
a
full
statistics
course.
I
had
folks
kind
of
copy
paste.
For
now
we
are
working
on
gesturing
tools
all
the
time,
so
that
we'll
be
able
to
ingest
content,
but
we
wanted
to
be
able
to
show
off
some
features
and
statistics,
so
I
only
have
a
portion
of
the
statistics
course
built
in
here,
not
the
whole
thing,
but
it's
enough
to
get
an
idea
of
what
we
can
do.
B
So
this
details
overview
of
any
particular
project
that
you're
in
has
you
know
title
course
description.
These
things
become
pretty
important
as
well
as
naming
conventions
throughout
you
know,
you'll
you'll
see
why
it's
always
been
important,
but
I
think
even
more
so
now
so
you
can
invite
folks
to
your
to
collaborate
on
your
project.
B
They
don't
have
to
already
have
an
account
in
proton
that
you
can
invite
them
just
by
their
email,
address
this
window
up
here
and
send
an
invite
and
then
they'd
be
invited
to
create
an
account
and
then
their
this.
Your
course
that
you
invited
them
to
would
be
in
their
project
list
when
they,
when
they
get
an
account,
so
really
easy
to
get
people
to
collaborate
on
your
courses
and
see
them
right
away.
B
Just
to
know,
we
do
have
the
ability
to
download
the
whole
course,
and
we
also
have
the
ability
to
download
the
data
for
in
a
data
shop,
format,
file,
format
to
be
imported
into
data
shop,
but
we
do
have
some
tools,
data
analysis
tools
built
in
that
I'll
show
a
little
bit
and,
by
the
way,
feel
free
to
to
stop
me
and
ask
questions
along
the
way
I
have.
You
know
this
is
always
an
informal
gathering,
so
I
have
no
problem
with
you
interrupting
me
and
saying:
hey.
B
Can
I
show
me
this?
So,
let's
see
in
taurus,
we
have
these
this
functionality
menu
on
the
left,
which
basically
this
is
where
you
get
into
your
course
through
your
well.
This
is
the
overview
page
we're
looking
at
now,
then
I'll
show
you
objectives
the
same
activity
bank,
which
is
a
new
feature
and
curriculum,
which
is
ends
up
being
your
units
modules
pages.
That
sort
of
thing
we
have
a
you
can
review
your
course
in
various
ways,
and
you
can
publish
from
here.
B
Insights
is
data
that
has
been
generated
on
the
course
and
then,
of
course,
I
have
this
admin
functionality,
but
let's
just
go
into
first
objectives.
So
this
is
where
you
enter
objectives
for
your
course
and
remember
that
objectives,
learning
objectives
should
be
measurable.
Student,
centered
and
they're
super
important,
like
probably
the
most
important
element
of
an
oli
course,
because
this
is
what
enables
you
to
tag
to
activities
and
generate
those
data
streams
per
objective
per
activity.
That
sort
of
thing
so
different
from
echo.
B
If
you're
familiar
with
echo
at
all,
you
could
enter
objectives
by
just
you
know
entering
the
information
here
and
hitting
create
you
also,
which
is
different
from
echo.
You
can
create
learning
objectives
on
the
fly,
meaning
that
as
you're
in
developing
pages
and
content
and
activities,
you
can
create
ac.
You
can
create
new
objectives
within
those
activities
that
will
populate
this
list.
B
So
that's
a
little
different
than
what
echo
echo
kind
of
forces
you
to
populate
your
objectives
in
the
objectives
section
and
then,
if
you
want
to
create
something
on
the
fly,
you
can't
you
have
to
come
back
to
this
page
and
create
them
that's
a
little
different
in
taurus.
So
far,
we
offer
the
ability
to
break
down
objectives
in
different
ways,
so
you
could
actually
now
create
a
hierarchy
where
you
have
what
we're
calling
sub-objectives,
but
you
can
imagine.
Everybody's
terminology
is
a
little
different.
Sub-Objectives
skills,
knowledge
components.
B
You
can
create
those
from
any
objective,
but
we
also
have
the
ability
to
create
an
overarching
objective
over
the
highest
level
objectives.
So
if
I
want
to
actually
take
this
and
make
it
a
skill
of
another
objective,
I
would
use
this
breakdown
tab
and
that's
what
it
would
do,
and
I
can
it
has
a
little
if
you're
seeing
this
pop
up.
It
has
a
little
description
of
what
happens
when
you
do
this.
B
When
use
this
breakdown,
so
just
a
little
a
lot
more
flexibility
and
objectives,
development,
skill
map
development,
as
well
as
more
tagging
flexibility
when
we
get
into
creating
activities.
A
B
A
How
many
levels
down
does
the
objectives
go.
B
I'm
going
to
jump
to
curriculum
bank
well
I'll,
show
you
activity
bank.
Unfortunately,
I
haven't
had
the
chance
to
really
play
around
with
this
feature
yet,
but
the
idea
is
this
will
enable
pooling,
but
will
also
enable
you
to
create
just
all
of
your
activities,
both
formative
and
summative
in
this
place,
and
then
add
them
to
pages
from
the
activity
bank,
either
directly
or
through
some.
You
know,
creation
of
parameters
to
add.
B
So
you
could
select
by
objective
by
activity,
content
by
item
type
or
tags,
we're
going
to
have
the
ability
to
tag
activities.
Questions.
However,
you
want
to
call
them.
I
kind
of
use
those
interchangeably
in
various
ways,
so
that
then
you
can
find
those
and
pull
those
in
various
ways
as
well
again,
not
functionality.
I've
used
yet
but
very
excited
about.
B
So
we're
gonna
dive
right
in
so
this
is
a
subset
of
the
statistics
material.
Just
like
any
authoring
tool.
You
have
the
ability
to
create
pages
or
containers
like
a
unit
when
you're
in
a
unit,
then
you
have
the
ability
to
create
modules,
and
I
love
this
breadcrumb
trail
up
here
too,
which
is
something
that
is
so
nice
to
have
so
when
you're
in
a
unit,
then
you
have
the
ability
to
create
modules,
so
those
are
just
the
containers
pages.
I'm
sorry,
my
zoom.
B
Pages
are
added
as
practice
pages
with
just
your
regular
pages.
Graded
assessment
is
a
type
of
page
now,
that's
just
the
way
you
create
quizzes
or
summary
of
assessments
in
taurus
right
now,
and
I
can
show
some
of
this
creation.
I
just
want
to
show
you
existing
material
in
here,
so
you
get
a
sense
of
how
something
that
we
already
have
kind
of
looks
in
here.
B
So
these
are
a
few
modules
that
we've
pulled
over
from
or
pages.
I
should
say,
we've
pulled
over
from
our
statistics
course
and
I'm
just
going
to
dig
in
you
can
see
from
this
view
you
can
get
a
view
of
this
is
of
the
learning
so
you're
seeing
the
list
of
formative,
activities
and
objectives
that
are
on
these
pages.
You
could
get
a
detailed
view
that
gives
you
information
about
updating,
information
or
the
basic
view
which
just
gives
you
the
titles
I'm
going
to
dig
in
here
to
this
page.
B
So
this
is
a
page
again
that
already
exists
right
now,
I'm
just
viewing
it
from
the
authoring
perspective.
I'll
show
you
all
in
a
minute
in
a
little
bit
how
it
looks
from
a
student
perspective,
but
again
all
the
same
things
that
a
regular
oli
course
has.
You
can
do
images,
tables,
headings,
text
and
activities,
multiple
choice
and
I'll
show
you
in
a
minute
what
all
we
have
here.
So
these
are
how
the
existing
activities
look
on
from
the
authoring
view.
B
What's
cool
and
different
about
this
is
now
we
have
the
ability
to
review
the
answer,
options
and
hints
with
these
tabs
right.
In
this
view,
if
you're
familiar
with
echo,
you
have
to
open
those
up
in
a
separate
window
to
edit
activities.
Activities
right
now
are
editable
editable
and
tourists
right
in
the
page.
B
If
I
can
get
to
a
place
where
I'll
show
you
how
to
add
new
content,
you
just
do
that
by
adding
you
know,
contact
or
activity.
This
little
pop-up
comes
up
right
now.
This
is
this:
is
a
bug
that's
being
fixed
where
sorry
for
all
the
scrolling
where,
when
you
add
it
it,
the
pop-up
is
comes
up
up
here,
at
least
that's
what
it
was
doing
that
see.
So
any
this
will.
B
This
will
change
so
this
pop-up
should
come
up
right
where
you
want
to
add
the
content,
and
you
can
add
content
anywhere
in
here.
So
you
have
these
little
boxes
of
content.
I
can
put
an
activity
right
here
if
I
want
and
again
pulls
up
this
pop-up
right
now.
These
are
the
things
that
are
available
to
create
in
a
taurus
page.
B
So
html
is
just
a
box
of
content
for
text
images
that
sort
of
thing
adaptive
activity,
it's
a
special
type
of
activity
that
usually
is
turned
off,
but
it
must
be
used
because
I'm
an
admin
I'm
seeing
it,
but
that
will
be
available
down
the
road
still
being
kind
of
worked
out
right
now.
We
have
check
all
that
apply,
multiple
choice,
ordering
and
single
response,
and
or
you
can
select
an
activity
bank
selection.
B
I'm
not
going
to
do
this
in
this
course
at
the
moment,
I'll
we'll
go
to
a
sandbox
course
of
mine
and
show
you
how
to
add
these
and
what
that
looks
like.
But
I
wanted
to
give
you
just
a
sense
of
how
can.
A
B
Singapore
response
actually
takes
care
of,
submit
and
compare.
You
can
actually
evaluate
now
longer
text
like
an
echo
now
and
an
oli.
You
can
only
have
a
literally
you
can
have
what
an
input,
a
text
input.
That
just
is
one
word.
This
can
now
evaluate
a
string
or
or
it's
used
also
for
numeric
input.
B
I
believe
it's
html,
just
like
it
was
in
an
echo
echoes
with
xml,
but
it's
a
mixture
of
html
for
the
yeah.
A
I
mean,
but
that's
hidden
from
the
author.
It
just
seems,
like
I
don't
know,
maybe
not
the
greatest
title
if
it's
just
if
it's
just
like
typing
in
a
wysiwyg
editor.
A
Yeah
yeah,
it's
yeah
greg,
sent
it
in
the
chat.
He
just
said
it's
basic
content.
A
B
Good
point
thanks,
michael
so
some
things
I
want
to
show
off
before
I
I
give
you
a
show
you
a
blank
slate,
I'm
just
wanting
to
show
you
show
some
cool
stuff
with
this
particular
course.
There's
already
lots
of
content
in
here,
and
so
what
I
wanted
to
show
you
show
off
here
is
the
review
and
publish
and
they
improve
so
our
review.
B
We
have
this
review,
and
here
from
here
you
can
run
a
review
which
will
populate
different
types
of
reviews
and
I'll
show
those
off
a
second,
where
you
can
preview
the
course
and
for
those
of
you
who
are
used
to
echo
preview
course
is
kind
of
cumbersome.
B
B
So
that's
cool
yeah.
These
audit
tools
are
always
in
the
works
right
and
right
now,
they're,
just
kind
of
some
of
them
are
prototypes
like
the
equity.
Some
of
them
are
are
are
good
or
you
know
we
want
to
add
to
these
essentially
to
catch
different
things
right
now,
if
you
are
select
to
look
at
the
accessibility
review,
you
run
the
review
and
it
will
bring
up
all
the
things
right
now.
B
It's
only
catching
any
images
that
don't
have
alt
text,
for
example,
but
you
can
imagine
we
can
add
a
lot
of
other
things
to
this
to
catch
other
accessibility
issues.
You
can
turn
these
toggles
on
and
off.
You
can
dismiss
them
so
say
I
want
okay,
this
one's
fine,
I
don't
want.
I
don't
want
it
to
have
all
text.
I
can
dismiss
it.
So
a
really
good
workflow
to
just
review
things
that
might
be
missing
from
your
course
and
get
to
them
very
easily
to
fix
those
issues.
The
pedagogy
go
ahead.
What.
A
B
Yeah
right
but
good
thing
to
note
and
then
pedagogical
review
right
now,
catches
when
you
have
learning
activities
that
aren't
tag
delivering
objectives.
Essentially,
so
you
can
go
through
and
you
either
dismiss
them
or
you
can
click
through
and
you
can
get
right
to
them
and
remedy
that
so
the
publishing
process.
So
I
want
to
show
you
some
publishing
stuff.
B
In
fact,
let
me
go
back
to
the
content,
because
I
want
to
actually
change
something
so
say:
I've
been
working
in
here,
I'm
going
to
edit.
This
welcome
page
right
now,
just
say
hey,
so
god.
B
Okay,
so
we'll
get
to
that
in
a
second,
when
I
go
to
publish
I'm
going
to
go
to
yeah
publish
you
can
see
right
here,
it's
telling
me
what's
changed.
It
keeps
track
of
all
of
the
changes
that
we
made
since
the
last
time
this
course
was
published,
and
if
I
want
to
push
this
out
to
existing
course
sections
I
can
do
that
and
then
it
forces
a
comment
here
which
is
good,
because
so
people
are
because
you'll
see
it
alerts.
B
B
We
have
the
ability
to
open
and
free
courses,
so
we
have
a
number
of
those
at
the
moment,
actually,
where
we're
publishing
things
open,
free,
which
then
provides
a
link
that
you
that
you
know
you
can
populate
in
a
number
of
places
or
send
out
to
students.
B
Let
me
go
back
and
we're
working
on
the
ability,
like
regular
oli,
delivery
mechanism
to
we'll
have
a
space
where
you
just
can
do
the
usual
have
a
log
in
for
your
students
and,
of
course,
key
that's
been
the
works
right
now
will
be
available.
Beginning
of
the
year,
we've
focused
on
lti
first
to
make
sure
that
was
going
to
be
really
smooth
and
that
we
built
in
clear
integration
with
with
grade
books,
and
things
like
that.
B
So
let
me
see
here
here
I
am
in
my
I'm
going
to
log
into
my
canvas
instance.
That's
where
I
have
this
course
published
I'll,
have
it
published
here
and
when
I
go
into
it,
it's
going
to
show
me
right
here
now
from
an
instructor
view.
This
isn't
what
students
see
so
when
instructors
see
who
are
using
this
course
and
set
up
a
course
section
based
on
that
package
that
particular
course
in
taurus.
It's
telling
me
there's
updates,
so
I
can
go,
and
I
can
look
at
that
and
it's
telling
me
what
the
updates
are.
B
B
And
then
I'm
going
to
go
back
and
show
you
that
it
updated
it
immediately.
So
this
is
one
of
them.
The
best
features
of
this
right
with,
with
both
authoring
and
delivery.
All
in
the
same
application,
we
can
publish
typos
instantly.
Basically,
if
you
want
to
fix
typos
and
publish
them
actually,
if
you
want
to
fix
anything
and
publish
it
out
to
your
students,
that's
the
mechanism,
and
it
can
happen
almost
instantly.
A
So
aaron
so
is
qa
and
production
are
those
going
to
be
a
thing
of
the
past.
B
All
production-
yes,
awesome,
yeah
very
exciting,
so
I
wanted
to
show
off
that
procedure
because
it's
very
exciting
and
then
I
want
to
show
off
the
other
thing,
which
is
data
aaron,.
A
Got
another
question
about
pushing
that
to
production
now
the
non-production
production?
B
Yeah
and
forgive
me
as
I'm
not
as
familiar
with
versioning,
but
I
know
in
the
overview
it
keeps
track
of
versions
and
then
I
know
in
the
publication
process
it
will.
It
did
have
some
versioning
information.
There
too,
I
think
they're
still
working
on
the
versioning
aspects
of
this.
B
B
You
also
have
some
control
right
now,
if
you
can
see
here
who
can
create
certain
course
sections,
so
you
can.
You
can
say
if
you
want
this
open
or
restricted.
B
B
But
again
I
just
want
to
show
you
they
are
building
in
this
ability
to
manage
versions
and
and
track
those
better
yeah.
B
Yeah,
in
fact,
let's
just
try
this
out
here,
because
I
have
how
do
I
get
back
out
to
courses
today.
B
B
A
Yeah,
so
so
as
as
it
stands,
the
system
decides
what
type
of
version
change
to
make.
You
can
see
that
it
has
selected
a
major
version.
A
It's
pumping
the
0.0
to
1.0,
because
there
are
structural
changes
if
it
was
just
typo
fixes
or
like
like
aaron's
first
edit
to
the
to
the
course
that
she
made
on
the
previous
course
that
was
a
minor
change,
because
she
just
added
some
text
to
it
an
existing
page
and
if
I
recall
previous
discussions,
the
end
user
instructors
will
be
given
the
option
of
accepting
and
or
rejecting
these
changes.
A
So
so,
right
now,
with
the
way
the
system
works
is
whenever
an
update
is
published
in
the
instructor
view,
you
will
see
a
badge
as
to
how
many
publish
it,
how
many
publications
are
available
to
basically
update,
so
you
can
either
choose
to
update
or
to
not
update,
there's
in
the
system
right
now,
there's
really
no
concept
of
rejecting,
because
it's
just
one
way
you
can.
A
You
can
just
choose
to
update
your
course
materials
or
not,
but
it'll
show
you
the
changes,
basically,
that
you
that
you're
kind
of
accepting
yeah
I'm
thinking
about
them,
reporting
back.
They
not
accept
it,
and
then
we
have
to
figure
out
which
version
of
the
course
and
say
well,
you're,
two
versions
behind.
A
Therefore
you
should
update
the
course
yeah.
That's
a
really
good
point.
I
I
made
a
note
of
that,
just
in
my
notes
to
add
that,
because
yeah
right
now,
the
the
only
way
that
instructors
can
see
what
version
they're
on
is,
if
there's
an
update
available,
but
we
should
definitely
be
showing
what
version
they're
on
as
well.
A
Yeah
not
something
completely
unrelated
when
you're
adding
a
question.
I
said
you
made
the
mistake
of
you'll,
see
url.
So
when
you're
adding
a
question
two
things,
one,
the
targeted
feedback
is
like
kind
of
clunky
to
add,
and
I
think
we
really
want
to
encourage
instructors
had
feedback
for
all
the
multiple
choice
entries
and
this
is
it
defaults
to
feedback
from
incorrect
answers,
which
is
you
know,
generic
and
then
anyway,
I
think
the
way
was
in
the
old
version
was
much
better.
A
It
was
encourage
individual
responses
and
the
second
thing
was,
I
checked
out
all
that
apply.
I
think
I
don't
know
if
there's
been
research
done
on
this,
but
it's
they're
very,
very
difficult.
It's
like
asking,
if
you
have
four
choices,
it's
asking
for
true
or
false
questions
all
at
once
and
it's
instead
of
which
is,
I
think
ken
has
had.
We've
eliminated
all
those
in
our
courses
and
ken's,
going
to
either
true
or
false
or
just
select
a
multiple
choice.
A
I
think
like
having
that
in
there
is
just
like
encouraging
people
to
do
that,
and
it's
not
maybe
that's
a
worth
the
conversation
with
the
eberly
center
about
using
that
even
having
that
type
in
there
type
of
question.
B
Right,
I
mean
we
can't
eliminate
a
type
that's
used.
However,
what
we
can
do
is
if,
when
we
control
the
design,
I
certainly
make
sure
that
folks
do
not
use
those
both
those
and
dragon
drops.
I
consider
catch-all
and
kind
of
in
some
cases,
instructors
do
tend
to
want
to
use
those
types
because
it
packs
a
lot
of
information
into
one
question
type.
I
mean
I
could
talk
to
you
for
an
hour
on
on
one
of
the
reasons
that
you
know.
B
I
encouraged
ken
to
change
those
because
he
was
complaining
about
the
data
and
how
to
review
the
data
on
those,
and
my
suggestion
was
to
eliminate
them,
but
they
are
useful
and
I'll.
Tell
you
why,
when
they're
useful,
when
I
think
they're
useful,
I
think
they're
useful
when
you
really
need
to
make
comparisons
against
categories
of
things
that
go
together.
B
So
I
haven't
been
able
to
prove
this
out
because,
unfortunately,
there's
not
a
lot
of
research
at
all
done
on
activity
types
there's
been
recent
rachel
van
kampen
out
did
a
study
in
the
acrobatic
fifa
study
that
did
start
to
look
at
activity
types
in
terms
of
engagement,
persistence
and
performance,
which
is
really
interesting
and
I'll
forward
that
to
you,
because
it
does
it's
one
of
the
only
studies
I
found
that
does
any
kind
of
research
around
different
activity
types
but
yeah,
that's
like
we
could
talk
about
that
philosophically
and
design
wise,
but
you
know
when
we're
building
a
system
for
a
lot
of
variety
of
folks,
not
all
who
might
develop
quality
courses
well.
A
B
B
Cool
thanks
yeah
I
I
could
like
I
said
I
could
talk
for
like
in
fact,
my
team
of
hearing
me
talk
about
multiple
select,
because
I
actually
really
really
don't
like
those
types.
A
Yeah,
okay,
so
we're
by
boring
the
other
people.
B
Yeah
yeah,
so
I
wanted
to
go
back
and
show
some
like
new
content.
You
know
to
to
michael's
point
of
what
this
looks
like
when
you're
adding
new
content,
so
text
block.
B
B
The
other
thing
you
can
do
is
add
a
webpage.
You
know
iframe
audio,
clip
youtube
video
image
code
table
so
again,
all
of
the
very
all
similar
stuff
and
you're
right,
maybe
are
these
in
these-
are
in
alphabetical
order.
That's
why.
B
Mm-Hmm,
so
I
want
to
add
a
multiple
choice,
so
I
can
attach
the
learning
objective
here
or
I
can
create
a
new
one.
Actually,
that
one
was
when
I
created
a
new
so
I'll
do
or
I
can
create
a
new
total.
A
B
B
B
What
michael
was
referring
to,
I
think
before,
is
that
there's
this
feedback
for
the
correct
answer,
but
we
added
this
feedback
it's
for
all
incorrect,
so
this
is
not
recommended,
of
course,
from
our
design
philosophy,
but
you
know
because
we
want
targeted
feedback
for
each
answer
of
choice,
but
we
have
had
the
request
to
give
generalized
feedback,
and
so
this
one
is
the
one
that,
like
any
answer
selected,
would
give
this
feedback.
So
this
would
be
feedback.
A
I
guess
the
same
same
thing
there
that
you
could
discourage
that
with
the
note
or
encourage
good
behavior.
With
the
note.
B
A
B
A
A
B
B
So
a
lot
of
options
is
the
is
the
thing
where
we're
adding
and
you
you
have
to
remember.
We
have.
We
have
customers
that
do
do
good
quality,
pedagogical
design
that
have
use
cases
for
these
things,
which
is
why
they're
building
here.
A
B
B
Yes
right,
that's
a
good,
that's
a
good
recommendation
and
then
we
have-
and
I
like
april's
suggestion
two
like
to
have
them
grouped
under
their
answer
auction-
would
make
it
much
easier
to
you
know
enter
it
all
in
so
thanks
to
april,
for
that
feedback
as
well,
and
then
we
have
the
ability
to
do
multiple
hints,
as
we
always
have
we're,
adding
some
guidance
here
about
what
kind
of
hints
you
can
add
and
you
can
add
all
kinds
of
hints
and
then
you
can
add
a
bottom.
B
What
we
call
bottom
out
him,
which
is
you
know
the
answer
for
like
a
numerical
numerical
input
question
or
even
if
it's
a
text
input,
it's
sometimes
nice
to
put
the
answer
in
the
last
hint
just
in
case
students
never
get
it
right.
B
B
Same
thing
answer
choice
in
same
way
like
you,
you
pick
what
how
what
combinations
you
want
to
give
feedback
for,
so
you
can
add
targeted
feedback
for
any
combination,
essentially,
and
really
you
should,
if
you
can't
think
of
unique
feedback.
This
is
again
my
little
commentary
on
multiple
select.
If
you
cannot
think
of
unique
targeted
feedback
for
every
answer
combination,
then
that's
the
wrong
option
for
a
question
type.
You
really
are
just
then
creating
a
series
of
multiple
choice.
You
don't
necessarily
need
to
check
for
all
that
apply.
B
B
I'm
going
to
show
you
this,
you
can
do
an
activity
bank
selection
even
as
a
formative.
So
if
I
want
again
this
isn't
this
isn't
functionality
I've
played
around
enough
with
yet
and
it's
completely
new,
but
the
idea
is
that
you'd
be
able
to
create
or
showcase
questions
from
the
activity
blank,
even
as
formatives,
based
on
selection
criteria
here
and
I
hate
to
put
eli
on
the
spot.
But
eli
do
you
know
if
it's
to
create
like
a
dynamic
set
of
questions,
even
as
formatives
or
would
it
be?
B
B
Did
I
lose
eli,
maybe
not
sure
yet
we'll
you
know
we'll
have
lots
of
guidance
on
this,
though
so
right
now,
I
can
make
a
selection
from
the
activity
bank
based
on
item
type
or
tags,
which
is
just
a
cool
feature
that
we
didn't
have
before.
B
So
then,
let
me
go
ahead
and
show.
B
Creating
a
new
assessment.
Well,
I
already
did
I
added
a
graded
assessment
page
essentially
here
and
what
it
is
is
just
you
know:
you're,
basically
building
a
summative
assessment
or
quiz
in
this
page
really
similar
to
what
you
know
already
about
how
we
generate
summative
assessments
in
echo
or
for
for
legacy
delivery.
B
It
shows
up
as
its
own
page.
It
also
populates
a
column
in
the
gradebook.
If
you're
using
an
lms
and
again,
you
can
add
expressions.
This
is
a
way
you
can
either
build
the
questions
directly
here,
just
like
any
other
page
or
you
can
use
this
activity.
Bank
selection
to
create
pooling,
essentially.
B
Is
insights
so
for
those
of
you
who
are
familiar
with
echo,
we
do
have
this
ability
and
echo
to
view
data.
That's
been
populated
already
by
you
know.
If
the
course
has
already
been
run
through
with
students,
we
have
the
ability
to
pull
that
data
into
echo
so
that
you
can
see
it
right
there
when
you're
ready
to
iteratively
improve
your
course
same
in
tourists,
only
it's
even
a
little
more
flexible.
B
B
How
many
get
it
first
correct
on
the
first:
try
how
many
eventually
get
it
correct,
and
then
this
relative
difficulty
takes
into
account
both
of
these,
but
also
hints
how
many
hints
were
needed-
and
you
can
view
these
now
by
page
or
by
objective
and
so
you'll,
see.
There's
those
objectives
aren't
a
lot
of
objectives
that
don't
have
questions,
because
this
is
only
a
portion
of
the
course,
but
it
has
all
the
objectives
in
it,
but
I
really
like
now.
This
is
new,
adding
the
actual
question
id
there.
It's
really
helpful.
A
B
It's
a
good
question
that
I
don't
know
the
answer
to
yet
so.
Basically,
the
question
is
like
how
to
how
to
segregate
data
into
the
various
course
sections
or
various
use
cases
right
like
you
can
imagine
you're
delivering
that
french
course
out
to
50
institutions
you'd,
want
to
maybe
see
the
data
by
institution
or
any
a
number
of
different
ways,
and
so
I'm
going
to
do
colleges
versus
four.
B
I
don't
think
that's
built
in
yet,
but
I'll
inquire
about
how
that's
coming
along.
If
that's
on
the
road
now,
so
I
think
that
concludes
everything
I
can
possibly
show
you.
You
know
quickly.
I
could
go
through
each.
B
You
know
more
activity
types
or
build
out
more
stuff,
but
I
think
you
guys
get
the
sense
of
what
what's
here,
how
it's
different
from
our
current
system,
and
you
know
some
of
the
things
that
we've
learned
over
the
years
that
we're
trying
to
apply
to
this
new
system,
make
it
more
streamlined,
more
flexible
for
various
users,
but
still
retain
a
lot
of
the
same
things
that
make
oli
unique.
B
There
any
other
questions
or
things
that
you
guys
want
to
see,
or
I
can
say
when,
when
they
might
be
coming
along.
B
Thank
you,
yeah.
I'm
really
impressed
with
with
how
they're
building
this
it's
clean.
It's
fresh
looking
it's
easy
to
use
easy
to
publish
we've.
Actually
we
have
folks
already
using
it
running
courses
using
taurus
we've
been
able
to
support
ken's
e-learning
students,
for
example,
to
rapidly
publish
and
iterate
on
their
course
courses
that
they're
doing
for
their
projects.
B
So
let
me
go
back
to
our
slides,
so
I'd
like
to
have
a
discussion
for
a
little
bit
on
upcoming
topics
for
meetings
that
you
know
things
you'd
like
to
see
functionality.
You
might
want
me
to
dig
into
discussions
we
want
to
have.
B
I
can
show
you
in
fact,
I
should
show
you
of
the
current
topic
list
that
was
in
existence
for
this
past
year,
so
we
did.
You
know
kind
of
an
introduction
to
these
this
community
and
this
this
consistent
approach
to
gathering
requirements
from
the
community.
B
Then
we
dug
into
and
discussed
some
lessons
learned
from
our
legacy
platform.
We
had
a
conversation
about
learning
dashboard
that
definitely
will
be
one
of
the
topics
we
revisit,
as
you
can
imagine
as
well.
As
you
know,
these
general
topics,
like
course,
design
math
support
tool
integration.
B
You
will
be
revisiting
some
of
these
topics,
but
I
want
to
make
sure
that
we,
like
a
b
testing,
will
be
another
one
right
now.
The
next
meeting
is,
is
meant
to
be
another
demo
and
discussion
about
next
steps
or
what's
on
the
road
map
coming
in
the
next
year,
but
also
what
the
probably
the
reveal
of
some
list
of
topics
in
the
schedule
for
next
year
as
well.
So,
given
all
of
that,
what
are
some
things
you'd
like
to
dive
into
and
suggest
that
we
discuss
for
some
secret
meetings?
B
Are
those
topics?
Are
there
topics
you
really
want
to
revisit,
because
you
feel,
like
you,
didn't
get
enough
from
the
last
one
last
discussion:
are
there
anything
we
haven't
touched
on
yet
that
you
want
to
talk
about
and
should
I
consider
other
formats
for
these
meetings?
I
don't
know
what
I
kind
of
meant
by
that,
except
you
know.
May
I
just
want
to
put
everything
out
there
like.
B
Is
there
a
way
that
you
think
I
can
make
these
meetings
more,
engaging
or
useful
for
you
and
then
just
a
suggestion
here
that
if
you
haven't
gotten
a
tourist
account
yet
you
I
highly
encourage
you
to
go
out
and
just
put
a
sandbox
course
out
there
and
start
playing
with
it.
B
But
let's
get
into
discussion:
what
are
your
ideas.
A
Oh,
I
I
have
one
that's
really
minor,
because
I
just
realized.
I
don't
think
you
showed
tags
on
the
questions
for
knowledge.
Components.
Oh
and
I
don't
see
them
there.
B
So,
like
here's
check
all
that
apply,
that's
in
here
already.
Here's
where
I
do
the
learning
objective.
B
Okay,
it
can
be
a
skill
as
well,
so
it's
whatever
it
can
be.
It
can
be
a
like
see.
It
says:
here's
a
sub-objective,
skill
or
knowledge
component.
If
I
go
to
my
objectives,
that's
actually
a
skill.
A
Yeah
I
got
yeah,
that's
that's
a
little
confusing
because
then
it
should
come
up
but
add
a
skill
under
this
learning
objective
and
then
you
should
be
adding
learning
objectives
to
to
assessments.
That's
just
wrong.
A
B
A
B
B
A
B
A
B
So
I
do
want
to
show
off
the
tagging
and
activity
bank
once
I
get
more
familiar
with
doing
that.
So
that's
a!
We
could
have
a
whole
discussion,
michael
if
you
would
like
on
what
we
call
things.
A
A
A
I
was
interviewed
by
npr
when
I
was
working
in
radio
and
they
said
that
I
missed
the
cut-off
score
by
one
question.
The
question
was
based
on
the
terminology
used
by
one
brand
of
audio
recorder
manufacturer.
I
use
the
terminology
for
another
brand
for
the
exact
same
function,
so
I
use
the
other
term.
B
A
B
B
B
Issues
with
that
ourselves,
so
yeah
so
april's
putting
in
the
chat
skills
mapping
is
hard
to
do
in
echo
and
at
least
iteration
on
skills.
B
You
know
we
had
a
lot
of
insight
views
into
the
system
by
being
able
to
go
to
the
xml,
but
that
was
also
because
the
authoring
platform
was
built
over
the
xml
structure
right,
so
we
kind
of
needed
that
that
window
and
view
into
the
underlying
original
creations
created
files.
This
is
done
a
little
differently,
but
we
still
like
I
get.
B
Is
there
functionality
that
we
haven't
even
thought
of
that?
You
all
have
had
some
experience
with
and
maybe
other
platforms
that
you'd
like
to
see
us
tackle
or
or
at
least
talk
through.
B
So
what
we
tend
to
do
at
least
what
I
tell
people
with
the
discussion
board.
We
have
the
ability
we
do
have
custom
activity,
types
that
allow
for
discussions
to
occur
between
students.
I
the
way
they're
done
in
ola
now
is
to
actually
have
an
actual.
You
know
you
have
to
design
an
activity
around
that
that's
attached
to
your
learning
objective,
and
that
way
you
can
actually
track
those
discussions
back
and
to
to
a
learning
objective.
B
Having
a
general
discussion
board
is
something
that
you
know.
Every
lms
has
so
there's
a
lot
of
debate
on
whether
we
want
to
do
it
that
way
or
we
want
to
actually
force
sound,
pedagogy
and
force
discussions
to
be
tied
to
learning
objectives.
A
Along
those
lines,
submitting
assignments
directly
in,
why
would
be
nice?
I
mean
a
lot
of
this
stuff
like
the
discussion
boards
and
rubrics
are
in
canvas,
but
you
know:
do
we
need
to
necessarily
go
through
canvas,
but
for
the
continuing
education
that
we're
developing
we're
not
going
through
canvas,
but
we
may
be
a
odd
case.
B
Well,
I
think
it's
a
case
we
want
to
consider,
because
you
know
you
might
be
an
odd
case,
but
you're
actually
an
increasingly
there's
more
of
these
cases.
These
types
of
cases
coming
up
learning
dashboard.
B
B
How
about
something
like
you
know,
support
and
that
could
be
in
creation
that
could
be
for
iteration.
B
I'm
just
thinking
of,
if
we're
starting,
to
build
more
kind,
more
guidance
into
the
system
in
the
form
of
tool,
tips
and-
and
you
know,
suggestions
for
how
you
use
these
features.
Maybe
something
also
showing
like
the
workflows
that
you
use
to
do
certain
things
might
be
interesting.
B
Format:
formatting
I'm
going
to
just
take
this
as
is,
and
put
it
in
this
list
standardizing
how
things
are
formatted
across.
B
And
I
could
show
off,
I
should
show
off
probably
in
another
meeting
how
what
those
integration
points
within
lms
are,
so
that
you
know
you
can
see.
I
can
show
off
how
how
gradebook
is
populated.
B
I
also
plan
I'm
working
on
right
now
some
ideas
about
how
to
how
to
do
iterative
improvement,
how
to
do
data
analysis
of
improvement,
and
I
can
imagine
once
we
have
a
nice
workflow
for
that.
You
know
building
in
some
guidance
around
that.
B
Admin
tools,
so
I
think
I
think,
mark
alert,
kind
of
was
headed
this
direction,
but
what
kinds
of
admin
tools
so
and
what
I,
what
I
mean
by
admin
tools
that
I
think
mark
was
kind
of
suggesting
is
how
do
you?
How
do
you
do
some
things
in
the
system
to
just
see
what
others
are
doing
or
to
manage
data
from
course
use
or,
like
you
know,
c
institutions.
B
Using
courses
any
of
the
other
kind
of
admin
tools,
you'd
like
to
see
built
in
like
right
now,
let's
see
what
can
I
do
now?
I
can
see
accounts
and
institutions.
B
This
activity
management
is
kind
of
like
what
activities
can
be
turned
on
and
off.
This
branding
area
is
probably
you
know,
new
ish.
B
So,
just
in
the
sense
of
like,
if
you're
trying
to
manage
your
course,
sections
that
are
in
use
at
other
institutions,
what
might
be
some
of
the
other
admin
tools.
You'd
like
to
see.
A
Hi
erin
it's
april.
The
comment
that
I
just
threw
in
the
chat
I
think
is
similar
to
looking
at
courses
across
institutions,
but
how
to
administratively
manage
cmup
and
cmuq
versions.
How
do
we
maintain
updates?
A
A
Export
course
pdf
and
then
comparing
two
pdfs
in
acrobat,
but
that
pdf
doesn't
give
us
any
of
our
assessment
content.
So
it
was
really
just
exposition,
but
it
was
something.
B
So
like
pdf,
printing
so
another,
that's
an
example
of
something
we've
taken
a
hard
stance
on
over
the
years
that
we
just
can't
anymore,
which
is
you
know.
We
have
always
said.
We
don't
offer
printing
printable
materials
because
we
really
wanna.
B
We
really
want
students
to
learn
by
doing,
and
so
we
didn't
wanna
offer
printable
versions
of
our
courses,
but
you
know
it's
just
something:
we
can't
take
a
hard
line
on
anymore.
People
want
printable
versions,
there's
accessibility,
reasons
to
have
them.
So
now
we're
really
trying
to
think
through.
What
does
that
mean
to
provide
a
printed
version
of
an
mli
course?
B
Do
we
generate
all
the
exposition
and
then
add
the
activities
in
a
way
that
can
be
worked
on
in
like
a
workbook,
you
know
one
of
the
hard
lines
we've
taken
in
the
past.
Is
we
don't
do
that?
Because
then
we
don't
collect
the
data,
but
again
it's
something
that
we
can't
just
not
do,
because
we
work
we're
philosophically
opposed
to
it.
It's
something
that's
expected
of
these
types
of
systems.
A
Well,
in
my
request,
surprisingly,
has
not
come
from
students.
Mine
is
more
for
authoring
and
editing
because,
with
a
lot
of
the
subject
matter,
experts-
I'm
not
giving
them,
you
know
final,
say
or
direct
access
to
edit
the
existing
materials.
A
B
In
echo,
that's
right
and
that's
great,
because
that's
something
I
do
all
the
time
too,
I'm
I'm,
I
use
student
workers
primarily
to
cut
and
paste
content
into
google
docs,
where
I
can
then
start
to
work
with
a
team
on
subject
matter.
Experts
saying:
oh,
maybe
we
should
change
this
content
suggesting
activities
and
then
me
cutting
it
back
and
forth
on
okay.
Well,
maybe
this
feedback
is
a
little
too
general.
Maybe
try
this.
B
You
need
a
space
to
do
that
and
so,
like
yeah,
getting
that
content
out
in
a
way
this
is,
I
actually
have
been
talking
with
daryn
about
this,
like
how
do
we
get
this
content
out
in
a
way
where
it
can
be
collaborated
on?
I
personally
don't
want
that
built
into
the
system.
I
I've
never
seen
it
done
in
a
clean
way
that
isn't
confusing
I'd
rather
just
use
google
docs
that
already
has
that
functionality
down.
B
I
think
at
acrobatic
we
tried
building
review
tools
into
the
authoring
platform
and
it
was
just
a
mess
and
then
never
used.
Even
those
a
lot
of
time
was
spent
on
that
on
that
functionality,
but
we
still
do
need
a
way
to
be
able
to
do
that
for
sure.
B
And
like
explore
the
tools
so
kind
of
a
long
line
of
being
able
to
get
the
content
out
in
the
form
in
a
different
format.
B
B
Thanks
so
much
michael,
thank
you
for
giving
me
some
ideas
for
things
that
we
need
to
probably
talk
amongst
ourselves
about
a
little
bit
more
before
we
go
forward,
and
I
know
we're
always
updating
and
changing
things.
You
know
there's
a
lot
of
times
where
the
tech
team
wants
to
just
get
functionality
in
place
and
we
can
always
revisit
what
things
are
are
called
and
what
the
you
know
the
ux
design
is
and
well
it's
always
helpful
to
have
your
opinions,
though,
for
sure
I
appreciate
it.
A
B
So
yeah
thanks
april
and
thanks
for
coming
thanks,
everyone
for
coming
and
please
look
for
the
the
reminder
for
the
next
meeting
and
we'll
do
something
similar
I'll
try
and
make
it
a
little
give
it
a
little
different
flair
for
next
time
so
that
you're
not
getting
the
same
demo.
I
probably
will
dig
in
a
little
more
to
like
some
of
the
stuff
that
I
I'm
not
familiar
with
for
this
demo,
but
I'll
get
familiar
with
for
the
next
time.
So
like
that
activity,
bank
usage
is
really
interesting
to
me.
B
I
can't
wait
to
play
a
lot
more
with
that
and
get
familiar
with
that,
for
example,
and
so
and
I'll
highlight
any
changes,
because
the
system
is
changing
all
the
time,
but
that
doesn't
mean
you
can't
use
it
now.
It's
in
use,
it's
delivering
courses
go
in
play
and
and
really
imagine
the
possibilities
and
bring
those
ideas
to
back
to
me
and
and
the
team.
So
thank
you
so
much
for
coming
and
I
hope
you
have
a
wonderful
rest
of
your
day
and
a
great
weekend.
A
B
B
It
was
probably
sasha
or
someone
because
they
for
some
reason
when
my
team
comes
in
to
this
zoom
they're
always
labeled
as
norm.
I
don't
know
why.