►
From YouTube: Torus Community Meeting May 2021
Description
Monthly meeting to discuss Open Learning Initiative's next generation platform. This meeting is about tools and functionality needed to develop and deliver math courses.
A
B
B
B
B
B
B
Everyone
is,
as
usual,
we'll
get
started
in
about
four
minutes.
B
A
B
B
B
B
Welcome
folks
we'll
get
started
usually
around
12
35
give
people
time
to
get.
B
A
B
Okay,
let's
get
started
hi
everyone
welcome
to
the
tourist
community
meeting
for
may
we
have
these
monthly
in
case
you're.
Just
now
joining
us,
you
can
join
us
every
month.
I
think
it's
the
first
friday
of
every
month.
Today's
meeting
will
be
about
math
support,
so
we're
really
excited
to
to
talk
to
those
folks
to
talk
to
you
all
who
develop
or
deliver
any
kind
of
math
in
your
courses
and
to
talk
with
us
about
what
you
need
and
what
has
been.
B
Maybe
some
pain
points
for
you
or
even
what
are
aspects
of
courses
that
teach
math
concepts?
What
do
they
need?
Maybe
there's
things
we
haven't
even
thought
of
before
so
right
now.
Just
a
few
of
these
are
just
a
few.
B
Pictures
of
some
activities
that
we
have
and
some
capabilities
we
have
in
our
courses
now,
but
what
those
are
essentially
are.
If
you
didn't
know,
we
have
the
ability
to
create
parameterized
questions,
meaning
that
you
can
create
a
question
that
has
variables
in
it.
That
then,
can
just
create
more
questions
from
it.
We
have
branching
questions.
B
We
have
different.
We
have
input
keyboards
that
pop
up,
particularly
for
latex,
for
either
authoring
math
content
and
questions,
or
even
for
students
to
input
equations
and
then
that's
kind
of
what
we
have
already.
But
let's
talk
about
anything
else
related
to
equations
equation:
authoring,
maybe
visualization
needs,
possibly.
Integration
needs,
meaning
external
content
that
you
might
want
to
integrate.
We
could
touch
on
that
a
little
bit
too,
but
I
think
I'll
I'll
open
it
up.
First
to
say
you
know
how
many
of
you
yeah
mathml.
B
We
do,
I
believe,
have
that
as
well
and
your
I'll
apologize
up
front.
I
I
am
not
very
versed
in
creating
content
for
math
courses,
so
I
did
invite
sandy
razor
and
I
see
that
davey
aaron
is
on
they're
used
to
doing
that
kind
of
stuff
with
chemistry
and
there's
some
other
folks
on
here.
I'm
sure
who's
had
to
deal
with
math
content
development.
A
C
C
I
don't
know
mathml
is
complicated,
but
once
you
get
the
hang
of
it,
it
works
fairly
readily.
I
have
no
idea
what
to
say.
B
That's
fine,
that's
fine!
I
actually
don't
know
the
types
of
things
that
need
to
be
discussed.
I
just
wanted
to.
I
wanted
to
focus
on
this
particular
kind
of
content
and
capabilities
again,
because
I'm
not
used
to
developing
these
courses-
and
I
want
to
kind
of
understand
if
the
needs
that
exist
already
are
good
enough.
If
there's
things
that
you've
experienced
outside
of
oli,
that
maybe
you
thought
what
why
why
can't
we
do
that
kind
of
thing.
It's
not
go
ahead.
Yeah.
D
Sure,
gideon
weinstein
I
work
at
wgu
and
since
we're
fully
asynchronous,
what
we
really
look
for
is
not
so
much
parameterized
questions
but
detailed
solution
keys
so
that
the
heavy
lifting
of
the
sort
of
the
sort
of
rote
and
practice
foundation
can
be
done
by
the
technology,
and
the
interaction
with
the
human
is
is
left
to
what
you
only
need:
a
human
for
pattern,
seeking
complete
confusion,
thematic
learning
tricks,
metacognitive
strategies,
specific
problem-solving
techniques
like
heart
of
mathematics,
talks
about
and
right.
So
what
we
found
so
far.
D
We
we
love
hate
pearson,
because
their
written
material
is
as
far
as
this
is
just
my
opinion.
I
don't
speak
for
my
institution.
D
D
That's
really
great
and
we
used
a
very
primitive
version
of
oli
statistics
and
one
of
the
main
problems
was
there's
only
two
versions
of
every
question
and
the
feedback
for
the
wrong
answers
was
useful,
but
but
very,
very
sparse.
D
So
a
deep
enrichment
of
feedback,
I
think
is,
is
what
what
any
oli
would
want
to
seek,
because
because
they're
often
going
to
be
used
online
and
and
even
if
they're
not
used
online,
don't
you
face-to-face
people
want
your
want
the
homework
you
assigned
to
be
maximally
effective
so
that
that's
my
main
deal.
We
also
we
had
a
college
geometry
course.
D
D
A
D
D
You
know
predicting
common
errors
and
we'd
write
the
solution
keys,
and
then
they
would
do
the
actual
programming
that
would
generate
the
numbers.
Do
the
algorithms
drop
them
into
the
problem
and
the
solution
manual?
So
the
student
got
that
interactivity
that
we
like,
but
we
did
all
I
mean
it
was
basically
we
could
have
just
learned
coding
done
it
ourselves.
D
B
Interesting
you've
hit
on
a
number
of
things
that
that
I
didn't
even
include
in
this
list,
but
I'm
glad
you
brought
them
up.
You
know,
solution
keys
is
something
we
struggle
with.
You
know
the
fact
that
we
pull
questions
and
the
way
our
pools
are
created.
B
You
know
makes
that
challenging,
but
it
also
because
we
have
to
you
know
we
have
to
create
all
of
those
questions
and
then
put
them
in
those
pools,
so
they're
fed
to
students
in
a
certain
way,
I'm
interested
in
knowing
more
about
the
authoring
capabilities
that
you
were
talking
about
with
pearson,
besides
coding
being
able
to
get
right
in
there,
I
mean
like
what
kind
of
interface
do
they
provide
for
that.
D
So
you
make
a
diagnostic,
we
call
them
diagnostics,
not
quizzes
or
tests,
or
anything,
because
we
have
our
own
internal
assessments,
of
course,
right
right,
so
we
create
a
diagnostic
it.
It's
rather
a
simple
interface.
You
simply
go
through
what
they've
provided.
What
they
provide
is
an
exact
mirror
of
what
they
give
in
the
various
subsections
of
their.
You
know
paper
and
paste
textbook
right.
You
know
the
kind
there
are
four
kinds
of
problems
or
eight
kinds
of
problems
in
that
120
item
problem
set.
D
What
they've
done
is
they've
sort
of
cloned
a
few
of
them
and
made
them
algorithmic
cloned
a
lot
of
them,
often
there's
a
choice
of
30,
50,
different
kinds
of
items,
and
each
kind
of
item
is
a
clone
of
either
that
type
of
problem
or
even
a
clone
of
a
specific
problem.
Where
what
happens
is
if
I
pick
it.
It'll
algorithmically
generate
the
variables
sometimes
it'll,
even
algorithmically
change
the
nouns
and
then
this,
and
it
will
often
algorithmically,
generate
answer
keys,
although
they
do
like
fill
in
the
blank.
A
lot.
B
D
Then,
when
the
student
reviews
their
diagnostic,
they
can
do
this
thing
called
review
results
and
what
that
does
is
when
they
were
wrong.
It
shows
them
how
to
do
it
right
and
the
best
thing
is
it's
step
by
step,
so
it
asks
for
it
does
a
step,
and
then
it
asks
the
student
you
know
what's
the
next
computation
and
then,
if
they
try
twice
and
fail
okay,
this
is
the
answer,
and
this
is
the
next
step.
D
Okay-
and
the
next
thing
to
do
is
this:
what
do
you
get
so
they
they
have
the
feeling
of
a
tutoring
experience
and
remember
all
these
numbers
were
algorithmically
generated
parametrized,
but
the
solution
keys,
pull
on
that
parameters.
That's
what
our
geometry
group
did.
They
pulled
on
the
parameters.
D
We
provided
the
full
text
of
the
solutions,
sometimes
even
step
by
step
and
they
put
in
the
parameters,
because
you
know
we
knew
how
to
use
them.
That
was
that
was
sort
of
the
the
big
deal
for
our
students.
When,
when
we
realized
that
we
could
do
that
and
and
then
of
course,
what
do
we
do,
we
customize
it
right,
we
use
pools.
Also,
oh,
we
don't
want
to
get
into
the
inverse
trig
derivatives,
but
let's
do
the
trig
derivatives.
D
B
E
Guess
one
thing
I'm
hearing
is
ken
cadinger
is
the
the
the
need
for
more
multi-step
kinds
of
problem
interactions
and
I
it
yeah.
E
Solution
key
thing
is,
it
seems
like
I'm
confused
by
that
because,
as
I
put
in
the
chat,
it
seems
like
more
than
eighty
percent
of
oli
questions.
Have
you
know
computer
gradable
solution
keys?
Certainly
in
my
e-learning
course
it's
probably
more
than
80,
I
mean
we
have
some
submit
and
compares
which
are
open-ended
text
fields
right
that
that
don't
have
solution
keys,
but
you
might
we.
We
don't
have
a
lot
of
multi-step
problem
sort
of
activities
that
get
get
feedback
in
the
way
yeah.
B
A
B
Yeah,
I
think,
in
the
past
what
we've
had
to
do
is
I'm
taking
some
notes
here.
It's
why
I'm
looking
down,
but
in
the
past
we've
we've
had
to
really
carefully
think
through
and
author.
Those
deliberately
not
have
that
done
kind
of
in
this
it's
kind
of
a
using
the
parameterized
question,
or
you
know
the
feel
the
ability
to
have
algorithmic
take
questions,
but
also
have
them
break
down
and
provide
solutions
at
every
step
and
yeah.
E
Yeah
well
one
one
version
of
that
is
like
the
assistant,
where
you
ask
a
one-step.
You
know
macro
question
and
if
the
student
gets
it
wrong,
then
it
expands
into
sub-questions
and
that
seems
technically
pretty
feasible,
but
it
would
mean
another
question
type
in
the
taurus
pool
of
question
types:
did
you.
F
See
what
it
was
exactly
and
that
in
the
chemistry
course
with
the
branching
questions,
so
basically.
E
F
Have
kind
of
the
the
macro
question:
that
is
a
problem
that
involves
multiple
steps
and
if
they
get
it
incorrect,
then
they
get
a
series
of
questions
that
kind
of
lead
them
to
be
able
then
to
go
back
and
and
get
the
correct
answer.
F
So
the
functionality
is
there.
I
I
guess
that
I'm
not
sure
if
that's
in
taurus
yet
or
not,
but
that
that
is
available
in
the
oli
system.
At
this
point.
B
Yeah
again,
it's
just
a
matter
of
you
know
knowing
it's
there
and
then
being
able
to
use
it,
and
it
still
requires
careful
design.
Like
you
know,
sandy
you've
had
to
really
think
through
what
all
those
steps
are
and
what
kind
of
scaffolding
would
be
needed
to
serve
up,
and
I
guess
what
I'm
hearing
from
this
other
system
is
that
it
does
it'll.
It
uses
those
algorithms
even
further
in
some
way,
which
is
interesting.
E
Well,
there's
no
substitute
for
good
design,
but
right
he's
not
gonna
magically.
Do
that
for
you,
but
I
guess
sandy's
question.
I
wasn't
sure
if
I
heard
is
that
type
in
taurus
at
this
point
it's.
B
Yeah,
I
don't
know
that
many
others
have
and
I've
seen
sandy
use
it,
and
but
I
haven't
seen
it
used
too
much
more
than
that.
But
again
it
could
be
a
matter
of
you
know,
folks
dipping
their
toe
into
authoring
and
not
getting.
I
mean
sandy's
been
iterating
on
this
course
for
for
many
years,
and
so
it
does
require
that
you
know
careful,
look
and
and
consideration
of
of
for
design,
and
then
you
know
looking
at
the
data
and
seeing
where
some
more
scaffolding
needs
needs
to
be
built
in.
B
A
There
now
that
it's
kind
of
like
you,
write
your
own
javascript
and
they
have
a
library
that
helps
right,
and
so
the
thing
that
that
the
earlier
speaker
was
referring
to
with
being
able
to
go
in
and
say:
oh
we'll,
just
do
these
functions
for
now
and
then
we'll
do,
you
know
add
in
layer
functions
arcsine
that
if
you,
the
current
system,
will
allow
that
in
the
sense
that
you
you
write
these
questions
once
you're,
given
the
ability
to
insert
the
javascript,
you
can
do
anything
you
want
and,
and
so
that
would
show
up
as
just
a
list
of
here's
the
things
you
can
select
for
this
random
variable.
A
B
D
Leads
to
an
interesting
follow-up,
but
I'm
brand
new
to
the
tourist
community.
So
if
I'm
covering
old
territory,
let's
say:
look.
D
Whatever
I'm
perfectly
happy
to
be
shut
down
and
shut
up,
I
like
david's
comments
about
you,
know
the
the
guts
of
it,
the
programming
of
it
right,
the
javascript
of
it,
but
as
I'll
pretend
I'm
a
typical
end
user
of
what
taurus
is
going
to
be
right.
I
don't
want
to
have
to
dig
into
that.
I
mean
I've
successfully
avoided
learning
latex
in
35
years
doing
mathematics
be
specifically
because
I
want
to
be
in
the
feet
of
a
of
an
end
user
who
was
very
busy
right.
A
college
low-level
institution.
A
E
B
Yeah
and
by
the
way,
it's
okay,
if
you're
not
familiar
with
with
these
things,
I
mean
I've
really
opened
up
this
community
so
that
we
can
get
more
folks
hear
more
from
other
folks,
maybe
who
maybe
aren't
familiar
with
our
tools
and
capabilities,
and
that
to
me
is
a
nice
balance
to
find
out
like
are
we
doing
things
you
know
are
the
things
we
have
helpful
and
and
what
we?
What
could
we
be
missing
that
we
haven't
thought
of
because
we're
seeing
it
just
from
our
perspective?
B
So
it's
okay,
if
you
aren't
familiar
with
some
of
our
stuff
right,
now
very
excited
to
have
this
good
discussion,
though
so
along
those
lines.
Let
me
let
me
check
the
chat
a
little
bit
here
and
see
if
I've
missed
any
good
points.
B
What
I'm
hearing,
though,
is
so
far,
is
that
this
idea
this
ability
to
do
these,
what
we're,
calling
parametrized
questions
or
or
algorithmic
type
question
pools
might
exist,
but
it
seems
like
there's
some
some
ux.
We
could
really
think
through
to
make
that
very
accessible
to
all
types
of
users.
A
They
may
not
have
the
functionality
they
need
to
add
to
make
it
easy
for
other
people
to
change
like
we
have
the
ability
to
change
the
list
of
parameters,
but
we
don't
have
the
ability
to
provide
a
user
interface
so
that
other
people
can
do
it
more
easily
than
we
can.
So
I
mean
like
that
part.
A
B
B
E
Well,
I
I
have
one
I
mean
related
to
this
discussion
for
the
20
or
so
submit
and
compare
questions
that
we
have
I'd.
Love
to
be
able
to
to
you
know,
do
do
the
kind
of
learner
sourcing
that
chu
wang
has
done
on
those
whereby
I
can
then
you
know
the
team
can
then
inspect
those
and
decide
you
know
which
ones
are
correct
and
which
are
near
misses
and
make
multiple
choice,
questions
from
those
that
can
be
graded
and
it
feels
like
yet
that
you
know
some
routine.
E
That
would
facilitate
that
kind
of
transformation.
Of
course
it
requires
somebody
to
go
in
and
encode
those
as
as
correct
dirt
and
near
miss,
but
you
know
you
only
need
sort
of
like
one
or
two
correct
and
and
three
to
six
near
misses
and
you've
got.
You
know
at
least
two
work
versions
of
that
question
at
that
point,
in
a
multiple
choice
form
and
where
the
you
know,
the
incorrect
choices
are
now
clearly
plausible
because
they
come
from
real
students.
B
Yeah,
that's
a
great
point,
ken
in
fact,
I'm
I'm
knee-deep
in
this
right
now,
tom
b
and
I
are
you
know,
we're
working
on
tom
v's,
managing
the
development
of
a
physics
course,
and
you
know
I
used
to
give
an
estimate
to
folks
who
are
developing
courses
of
any
type
that
you
know
per
question.
It's
going
to
take
you
a
good
30
minutes
to
write
a
good
question.
B
A
To
what
extent
do
we
have
a
decent
entry
of
equations?
Is
that
something
like
desmos,
the
graphing
calculator,
that
kind
of
functionality.
A
B
That's
a
good
question
that
I
actually
don't
know
the
answer
to.
I
believe
we
have.
We
have
an
input
keyboard
where
students
can
input
an
equation
as
an
answer,
I
don't
know
if
that's
in
taurus
yet
does
anybody
on
the
call
can
know
about
this
and
can
speak
to
it.
F
I've
not
used
it.
I
I've
thought
that,
like
dave,
there
there's
certainly
some
parts
of
the
chemistry
course
where
there's
potential
to
use
it.
But
aaron
do
you
know
if
anyone
has
used
it?
Do
you
have
any
examples
of
that
or
is
there
any
documentation
or
a
demo
or
anything,
because
I
I
believe
it
was
anders
that
was
working
on
it
and
when
he
first
started
working
on
it?
F
I
looked
at
it
and
gave
him
some
feedback,
but
I
haven't
looked
at
it
again
since
then,
and
just
a
little
bit
more
information
on
that
would
be
helpful.
B
Yeah
good
good
point
sandy,
I
don't
know
enough
about
it
to
know
if
it's
fully
done
and
if
it's
in
use
anywhere,
but
this
leads
me
to
you
know
an
idea
that
we
should
probably
hold
a
webinar
to
just
highlight
our
authoring
capabilities
as
they
exist
now,
because
even
folks,
I
think
who
do
this
on
a
regular
basis
like
yourself,
don't
even
aren't
even
aware
of
what
we
have
and
frankly
I
don't
really
either
so
good
idea
to
ask
that
question
and
we
will
remedy
that.
B
Use
the
the
equation,
input
keyboard.
C
I
would
think
something
like
that,
but
it
would
be
really
nice
to
have
a
keyboard
because
we,
this
has
actually
been
a
topic
of
discussion
recently,
and
we
have
a
little
guide
that
we
have
to
put
a
link
to
in
every
activity.
So
it
would
be
nice
to
have.
I
don't
like
reading
guides.
I
just
like
pushing
buttons
option.
A
B
B
Yeah
I'll
have
to.
I
will
definitely
look
into
that
more
and
see
what
examples
we
have
what
we
have
developed
so
far
and
see
where
they
are
with
it.
B
I
def
I
do
want
to
get
back
to
this
answer
key
thing,
because
ken
to
your
point,
yeah
a
lot
of
our
a
lot
of
our
questions
are
auto
created
right,
so
I
think
you're
thinking
of
it.
In
terms
of
that,
I
think
what
others
might
be
thinking
of
is
something
that
we've
done
for
statistics,
which
is
because,
like
the
quizzes,
the
end
of
module
quizzes
are
in
pools.
B
We
then
provide
for,
in
the
instructor
resources
section
all
the
possible
questions
that
a
student
could
have
with
their
with
their
answers
and
that's
more
support
instructors
who
are
using
these
courses
and
need
to
be
able
to
is.
B
Yeah
yeah,
I
think
how
they've
been
done.
I
know
we
have
it
in
statistics
and
I
had
to
actually
it
was
kim
larson
who
created
these,
and
I
had
to
actually
ask
her
how
she
did
it
when
I,
what
I
understand
is,
she
would
possibly
take
the
xml
of
the
whole
of
all
the
pools
and
dump
them
into
an
html
file.
C
Yeah,
that
sort
of
thing
in
general
is
something
that
we've
wanted
since
the
beginning
of
time.
As
far
as
I
know
just
a
way
for
instructors
to
see
all
possible
questions
on
their
quiz,
attempts
too,
like
that,
would.
B
So,
in
what
way
you
use,
those
then
is
it:
is
it
just
just
to
to
solve
disputes
of
the
grading
or
is
there?
Is
there
more
that
you
use?
This
would
use
this
for.
C
Well,
a
lot
of
the
time
we
get
new
instructors,
especially
asking
about
questions
drawn
from
pools,
and
they
don't
realize
it's
more
than
one
could
appear
in
that
location
and
so
there's
some
initial
confusion
with
well.
This
student
said
this
thing
about
question
six
and
this
other
student
said
this.
Other
thing
I
don't
know.
What's
going
on,
I
see
a
different
thing
entirely.
Yeah.
C
C
Not
all
students
will
see
the
same
question,
something
like
that
at
least
would
be
good
and
then,
whether
they're,
looking
at
their
own
attempt
or
even
at
the
student's
work
and
and
just
some
way
for
instructors
to
cycle
through
all
the
questions
that
could
appear
in
a
given
location.
A
A
Oh,
we
don't
do
that
specifically
for
instructors,
but
when
we
are
proofing
the
course
we
have
a
version
of
their
essentially
our
lesson
test
pools
where
every
single
question
is
there,
so
that
the
instructor
can
go
through
and
make
sure
that
they
are
done
correctly.
I'm
sure
there's
a
mechanism
to
make
it
so
that
it's
it's
instructor
only
page,
so
they
can
see
all
the
items
in
a
particular
pool.
D
But
isn't
isn't
a
different
version
of
the
exercise?
Okay,
just
so
long,
you
have
a
gradebook
that
the
instructor
can
go
into
and
see
the
exact
instance
of
what
the
student
did
just
right
so
recorded
in
the
grade
book
or
grade
book
whatever
right,
so
they
maybe
they
took
it
three
or
four
different
times,
and
they
want
to
talk
to
you
about
the
third
time
number.
Seven.
C
C
C
C
B
Yeah,
I
believe
what
you're
saying
is
is
possible
again,
I'm
just
not
sure
exactly
how
and
again
that's
why
I
was
saying
I
need
to
like
have
a
webinar
where
we
demonstrate
these
capabilities,
because
we
definitely
have
the
ability
to
do
these.
Do
something
that
you're
saying
I'm
just
not
quite
sure
how
and
I
don't
see
why
we
couldn't
create
it
and
then
duplicate
it,
because
we
have
the
ability
to
duplicate
questions
then
and
copy
them,
and
then
you
could
change
the
variables
if
you
want
from
that.
E
Once
one
strategy
that
can
be
used
to
do
this
is
to
do
rather
than
having
off
online
parameterized
question
generation,
it
can
be
done
offline.
E
Problem-Specific
features
get
put
into
a
column
in
a
table
and
then
you
can
add
columns
to
change
those
features
and
you
can
write
excel
formulas
that
change.
You
know,
change
those
features,
since
actually
you
can
do
parameterized
questions.
You
can
change
the
letters
or
the
numbers
and
make
as
many
columns
as
you
want
and
then
then
upload
that
table
to
get
a
whole
pool
of
related
items.
E
It
seems
like
that
idea
could
be
incorporated
into
taurus
in
a
way
that-
and
it
does
offer
some
nice
flexibility,
because
sometimes
when
you
randomly
generate
weird
things
happen-
and
here
you
know,
if
you
do
the
random
generation
offline,
you
can
then
check
for
those
weird
things
and
correct
it
in
the
table.
You
know.
B
So
that
capability
exists
in
c
chat.
I
know
I'm
not
again,
I'm
not
sure
exactly
how
it
works
in
an
echo
at
the
moment,
but
yeah
you
can
duplicate.
I
know
you
can
duplicate
questions.
So
if
you
set
one
off,
then
you
could
duplicate
it
and
change
what
you
need
to,
and
I
think
sandy's
saying
yeah
the
easiest
way
to
do
this.
We
duplicate
an
existing
parametrized
question
and
then
modify
it.
If.
B
E
To
the
go
ahead,
the
solution
yeah.
I
definitely
appreciate
your
point
aaron
about
the
quizzes,
because
now
that
I
think
about
it,
like
in
my
e-learning
design
course,
there
are
a
lot
of
quiz
questions
that
don't
have
sufficient
feedback
and
as
giddy
I
think
it
was
gideon
was
saying
especially
on
incorrect
responses,
but
this
seems
like
another
opportunity
if
we
were
to
do
some
learner
sourcing,
like
I'm
thinking
now,
I
should
put
on
every
quiz
questions
of
the
type.
E
Why
is
this
not
a
good
answer
to
this
question
right
in
an
open-ended
in
a
way
and
then
on
the
quiz?
You
know
we'll
have
to
grade
those,
but
if
it's
a
good
explanation
of
why
it's
not
the
right
answer
now
we
have
the
learner
source
explanation
to
add
in
next
year's
version
of
the
quiz.
If
you
see
what
I
mean
yeah
yeah.
F
Yeah
and
that's
similar
to
that,
can
I
I've
had
a
conversation
with
norm
and
darren
recently,
and
I
think
they're
gonna
try
and
work
this
into
taurus
four
quiz
questions
to
have
multiple
feedbacks,
so
you
know
on
maybe
the
first
time
they
take
it.
They
get
feedback.
That
is,
you
know,
just
helps
them
get
to
the
correct
answer
without
giving
them
a
full
solution.
F
But
at
some
point
you
may
want
to
give
them.
You
know
a
full
solution,
maybe
after
it's
already
past
due
or
whatever,
and
I
believe
they're
going
to
be
working
on
the
capability
to
be
able
to
do
that.
B
Yeah,
that's
a
good
point
sandy.
I
was
just
about
to
to
talk
about
how
the
feedback
has
generally
been
written
for
those
quiz
questions.
I
know
that
back
on
some
older
courses
we
were
developing.
You
know
we
weren't
developing
as
much
targeted
feedback
to
each
answer
option
in
the
quizzes,
but
more
directing
the
feedback
to
the
overall
question
and
then,
of
course,
never
really.
B
You
know
trying
really
hard,
never
to
direct
the
students
back
to
the
content,
but
really
give
them
enough
context
to
learn
something
from
that
feedback
right
then,
and
there-
and
you
know
different
folks
authored
differently,
and
maybe
some
of
the
feedback
on
some
of
the
quizzes
is
is
not
as
helpful
as
we'd
like.
B
F
Potentially
or
or
potentially,
you
know,
the
ability
for
the
instructor
to
be
able
to
release
the
full
solutions.
E
F
F
So
I
think
that's
where
we,
you
know
some
mechanism
for
the
instructor
to
be
able
to
release
those
full
solutions.
B
B
E
Probably
almost
always
I
I
was
just
thinking
about.
Like
one
thing,
I
tell
students
on
particularly
on
exams,
but
I
would
be
true
for
quizzes
too.
Is
you
know
if
you,
if
you're,
if
you
see
ambiguity
in
an
off
multiple
choice,
question,
and
you
want
to
explain
why
you
chose
your
answer?
E
E
Maybe
my
intended
answer
is
a
because
I
didn't
you
know
I
had
a
different
interpretation
of
the
question,
but
I'm
raising
this
because
that's
also
another
kind
of
learner
sourcing
opportunity
like
if
it
were
sort
of
routinely
possible
for
students
to
say,
for
example,
you
know
if
they
do
get
feedback
in
an
inline
and
they
want
to
say
wait
a
second,
I
think
my
answer
or
in
a
quiz.
I
think
my
answer
was
right
because
right
and
would
give
them
the
opportunity
to
enter
that.
D
D
A
D
Let
me
let
me
set
it
so,
the
first
three
times
they
get
light
review
right,
the
light
response
and
then,
after
that
they
get
the
deep
response
and
then
maybe
like
by
the
seventh
time.
They've
hit
an
item.
It
generates
an
email
to
me
because
now
I
know
that
person's
in
trouble
right,
that
kind
of
customization,
of
and
and
again
even
for
brick
and
mortar.
D
D
E
Certainly
part
of
that,
though,
is
already
possible,
like
in
my
class,
the
quizzes
happen
before
we
meet
and
they
can
take
them
as
many
times
as
they
want
until
they
get
the
score
that
they
want,
and
next
year
we're
gonna
incorporate
now.
Well,
I
don't
know
if
it'll
happen
in
time,
but
hopefully
next
year
we're
gonna
start
to
incorporate
this
knowledge
tracing
algorithms.
So,
instead
of
taking
a
full
quiz
each
time,
you
only
redo
items
for
learning
objectives
that
the
algorithm
has
determined.
You
haven't
mastered.
D
Right
the:
what
do
they
call
that
customizable
assessment,
differentiated
assessment,
adaptive.
A
D
A
F
There
is
currently
for
for
those
that
don't
know,
I'm
not
sure
if
it
exists
in
echo,
but
it
is
possible
within
the
system
and
it's
used
regularly
in
the
chemistry
course
that
you
can
layer
feedback
within
the
formative
questions.
So
on
an
initial
feedback.
Maybe
again
you
just
give
them.
You
know
some
information
to
to
guide
them,
and
then
you
know
the
second
or
third
feedback.
F
E
From
what
I've
seen
from
data
the
having
good
feedback
and
having
opportunities
for
repetition,
you
know
I
I
guess,
as
gideon
was
saying
like
at
least
seven
chances
kind
of
thing.
That
goes
a
long
way
and
and
often
we
don't
have
those
seven
chances.
So
I
I
would
and-
and
I
would
say
that
some
of
our
analyses
of
layers
of
hint
messages
have
been
kind
of
disappointing
in
terms
of
you
know,
do
the
vague
vegar
hints
do
much.
The
answer
seems
to
be
either
no
or
can't
tell
we.
E
E
B
And
it
seems
like
the
key
to
teaching
math
and
for
students
to
master
any
kind
of
math
concepts.
B
E
D
I
think
it's
all
about
automaticity.
I
hope
I'm
pronouncing
that
right
right
once
once
the
once
you've
automated
some
processes,
you
can
actually
do
more
thinking
right
like
when
you
learn
to
drive.
We
all
drive
a
lot
better
than
when
we
had
to
consciously
think
okay
pick
that
foot
up
off
the
accelerator.
There's
right
and
now
we
just
don't
think
and
get
it
done
at
least
in
mathematics.
That's
my
main
experience
that
necessity
for
automaticity
is,
is
very
large,
and
maybe
it's
just
the
bias
of
my
learning
and
teaching
experiences.
D
I'd
have
to
agree
with.
Can
I
think
I
think
it's
very
strong
in
mathematics
compared
to
even
you
know
any
other
topic.
Well,
I
mean
necessity
for
base
automaticity,
because
otherwise
you
can't
think
conceptually
you
can't
get
to
the
applications.
You
can't
like
you
can't
read
you
can't.
If
you
don't
know
your
letters,
you
can't
read.
If
you
don't
have
some
automaticity
things
go
too
slow
to
think.
B
Deeply
right,
but
then
even
sebastian
put
in
the
chat
here
languages
that
is
another
another
topic.
So
ken's
point.
It
really
is
just
about
any
any
topic.
You
can
say
that
about
it.
Just
feels
more
apparent
in
math,
and
I
guess
you
know
more
obvious
in
math
content
but
yeah.
B
D
D
D
That's
an
interesting
way
that
students
can
learn
by
hands-on
practice
and
observation
before
they
do
the
algebraic,
such
as
connect
the
midpoints
of
a
triangle
measure
that
little
triangle
measure
the
big
one
take
the
ratio,
it's
always
a
quarter,
but
you
can't
really
convince
a
student
until
you've
got
a
dynamic
diagram
that
they
can
move,
no
matter
what
triangle
they
make.
It's
still
a
one
quarter,
and
then
you
get
into
scale
factors
and
proofs
and
all
the
other.
D
You
know
all
the
other
deep
math
stuff
is,
that
is
that
kind
of
student,
controllable,
dynamic,
graphing
kind
of
things
either
embeddable
in
or
from
elsewhere,
like
desmos,
or
is
it
part
of
the
native
coding
system.
B
I
can
say
that
we
don't
have
that
as
part
of
the
native
coding
system.
We
have
the
ability
to
integrate
different
types
of
third-party
activities
and
labs,
and
things
like
that
that
exist,
but
usually
the
problem
with
that
is
getting
the
data
from
those
systems.
So
we
usually
have
to
follow
those
up
with
like
native
type
questions,
so
that
we
can
get
some
sort
of
data
on
how
effective
how
effective
those
are.
B
But
that's
really
interesting
and
that's
the
kind
of
thing
I
want
to
really
get
into
here,
because
we
don't
have
courses
in
geometry
or
or
calculus
right.
So
what
what
are
some
things
that
we
haven't
even
thought
of
yet
that
might
be
needed
in
those
types
of
courses
and
you've.
Given
a
good
example.
D
D
Well,
you
know
that's
because
we
all
have
advanced
educations
and
we're
theoretical
thinkers,
because
we're
interested
in
this
stuff.
That
would
be
a
great
little
thing
to
play
with
right,
just
from
the
make
it
balance
fifth
grade
sixth
grade
level,
and
then
you
lead
them
through
right.
So
if
I
could
embed
that
from
somebody
else
or
you
could
create
it,
I
would
you
know
the
sort
of
the
learning
by
doing
right.
The
series
of
questions
within
the
text
that
follow
up
on
it
and
help
the
student
develop.
Oh
wow,
that's
weighted
averages.
B
Yeah,
so
one
of
the
things
I'm
really
excited
about
in
terms
of
of
tourist
development
is
the
fact
that
it's
being
developed
open
source
with
this
idea
of
you
know
this.
This
attention
being
paid
to
interoperability
and
making
sure
that
you
know
we
can't
develop
all
these
crazy
cool
things
for
every
particular
course,
but
someone
else
might
be
able
to
and
then
be
able
to
easily
integrate
them
in
with,
in
with
our
content,
is
something
that's
key.
B
You
know
sandy
raised
a
point
in
the
chat
here
that
you
know
we,
they
embed
fat
simulations
into
chemistry.
We
have
some
of
those
in
physics
as
well
and
I'm
sure
other
places
where
you
know
they
are
manipulatable
labs,
as
you're
describing
and
but
the
issue
there
is,
I
mean
sandy.
We
can't
necessarily
get
get
the
data
from
those
right.
F
Yeah,
that's
correct.
I
the
way
that
they're
incorporating
incorporated
into
the
chemistry
course
is
that
they're,
given
some
instructions
to
manipulate
the
simulation
in
some
way
and
then
there's
follow-up
questions
within
oli.
You
know
based
on
what
they
did
in
the
simulation.
F
So
yeah,
you
really
can't
collect
data
directly
from
it,
but
they
they
embed
very
nicely.
They
look
like
they're
part
of
the
course,
but
yeah
you
can
kind
of
get
indirect
data
from.
D
Yeah
and-
and
so
that
sounds
awesome,
you
know
so
if
you
can
embed,
you
know
desmos
and
geogebra,
and
I
think
I
I
think
those
are
you
know
the
two
major
tools
that
come
to
my
mind.
Then
you've
got
a
lot
of
content
covered
if
they
can
be
quickly
embeddable
and
then
the
follow-up,
maybe
to
you
sandra,
maybe
to
anybody
right,
I'm
not
a
coder.
D
How
hard
is
it
going
to
be
for
me
to
write
the
first
low-level
and
then
deep
conceptual
questions,
some
that
I'll
make
multiple
choice,
others
that
will
be
open
feedback
right
is
that
do
I
have
to
you
know,
learn
the
second
layer,
the
first
layer
or
the
third
layer
of
of
that
ken
described
a
while
ago
to
to
create
it
right,
I'm
almost
thinking
about.
F
So
I'm
like
for
the
authoring
with
the
fat
simulations
that
that
I
did
in
chemistry,
you
know
there.
There
wasn't
any
heavy
coating
like
there's
a
an
iframe
right.
F
Stick
in
an
iframe
and
that
and
that
puts
the
phet
simulation
there
and
then
I
you
know
developed
a
short
set
of
instructions
for
something
specific
for
students
to
do
as
opposed
to
you
know.
Here's
this
cool
thing
try
to
learn
something
from
it.
You
know,
and
if
they,
if
they
go
through
that-
and
they
want
to,
you
know-
obviously
play
around
with
it
on
their
own.
F
Then
you
know,
then
that's
great
too,
but
you
know
some
specific
instructions
that
are
just
written
on
the
oli
page
and
then
follow-up
questions
that
go
with
it.
So
for
for
what
I
did
there,
there's
no
special
coding.
D
D
E
Interesting
design,
questions
and,
of
course,
I'm
around
how
to
write
you
know
good
and,
and
you
know,
the
cool
thing
I
think
is
sandy
and
dave
in
chemistry
have
a
lot
of
great
examples
because
they
have.
They
also
have
a
chemistry
lab
simulation,
and
you
know
why
is.
D
A
E
Of
water
in
bangladesh,
bad,
you
know,
go
and
do
titration
in
the
chem
lab
to
figure
it
out
and
then
come
back,
and
there
are
all
like
questions
about
it.
That
sort
of
thing
so
yeah.
B
D
I'd
love
to
catch
up
absolutely
and
again,
apologies!
If
I'm
asking
questions
that
are
way
basic,
but
yeah,
I
intend
to
stay
above
below
away
from
the
level
of
the
real
coding
on
underneath
right.
I
want
I
want
wizzywig,
so
I
can
throw
together
interesting
stuff
and
adapt
it
and
change
it
right,
as
I'm
training
people
for
their
licensure
exams
in
mathematics
that
changes
every
once
in
a
while
right,
you
give
me
my
wysiwyg,
I'm
a
happy
guy
yeah
right.
We
we
learn
that
we
need
to
do
a
little
more
in
stats
to
cover.
D
E
Stats
course
there
are
these
call
outs
to
use
r
or
minitab,
or
whatever
your
package
are,
and
maybe
not
in
the
version
you
saw,
but
like
yeah,
I've
been
wondering
we
definitely
have
the
technology
to
do
like
even
kind
of
like
regular
expression
matching
on.
You
know
right
on
our
command
to
to
to
to
do
this
task.
E
I
think
we
could.
We
could
do
more
in
that
regard,
and
I
I
don't
know
if
taurus
has
like
a
there's.
Various
versions
of
regular
expressions
or
stretchy
patterns
like
in
light
side,
uses
this
stretchy
pattern
idea
where
you
can
basically
say
well.
This
has
to
be
here,
and
this
has
to
be
there,
but
in
between
anything's
okay,
that
sort
of
thing
there's
definitely
code
in
ctec
for
doing
equation,
matching,
for
example,
that
probably
could
be
generalized
for
other
domains.
E
D
B
I
would
agree
with
that,
so
I'm
looking
through
the
chat
here,
oh
norm,
chinan,
said
vet
simulations
are
starting
to
be
developed
to
capture
data
from
them,
which
is
nice.
F
I
haven't
looked
into
it
but
they're
making
them
now
to
where
you
can
modify
them
somewhat,
but
I
haven't
haven't
dug
into
into
that.
But
I
know
that's
a
thing.
B
Yeah
I'll
have
to
look
into
that
some
more
norm
just
put
a
link.
B
So,
let's
see,
we've
talked
about
parameterized
questions
or
variable
type.
Questions
for
answering
questions
a
little
bit
import
input,
keyboards
like
equations
inputs.
B
E
B
A
About
significant
figures
and
accuracy
and
stuff
having
that
built
in
by
thinking
about
that
would
be
good
like
what
we
do
now.
I
think
there's
a
difference
between
the
accuracy
which
is
reported
like
that
they
give
it
to.
The
right
number
of
significant
figures
is
different
from
is
the
value
correct
and
it
can
just
be
a
real
hassle.
You
have
a
lot
of
the
different
places
to
get.
Are
the
response
wasn't
accepted
because
of
that
boring
little
thing
of
you
know.
A
You
were
five
point,
one
percent
off
instead
of
fight
that
kind
of
stuff,
not
sure
what
to
do
about
it,
but
it's
important
part
for
many
instructors.
B
Yes
great
and
thank
you
for
bringing
that
up
david.
I
think
there's
a
couple
things
about
that,
though
there's
we
have
the
ability
to
author
those
ranges
and
and
to
some
extent
the
accuracy
at
which
you
want
any
particular
question
to
be
answered.
But
I
think
what
you're
asking
for
is
kind
of
an
instructor
global
setting,
maybe
where
yeah.
A
Yep,
that's
part
of
it
too
yeah
different
instructors
care
to
different
amounts
like
I
really
don't
care,
usually,
and
so,
but
then
instructors
who
really
are
sticklers
for
it
having
them
being
able
to
say
yes,
my
students
have
to
get
it
right.
B
And
even
if
that's
available
for
even
just
the
the
high
stakes,
summative
assessments
is
probably
a
nice
nice
control
for
the
instructor
to
be
able
to
say
for
this
assessment,
I'm
going
to
grade
it
at
this
level
of
accuracy
or
significant
figures,
or
just
you
know
anything
like
that
interesting.
B
B
Well,
thanks
dave,
I
think
that's
a
great
suggestion
and
I
I
have
definitely
talked
to
different
folks,
and
I
am
sure
if
you
talk
to
hal
who
who
manages
our
help
desk,
he's
probably
had
that.
That
question
asked
a
bunch
like
how
to
toggle
that
or
how
to
how
to
manipulate
that
rate
ken
had
here-
and
this
is
kind
of
what
I
was
thinking
of
earlier,
too
kind
of
diagrams
as
emperor
output
or
even
just
the
ability
to
there's
a
lot
of
graph.
E
Yes,
for
sure
and
and
yeah
there's
a
lot
of
sort
of
conceptual
mapping,
kinds
of
questions
in
math
and
and
other
domains
like
to
number
lines
and
yeah
fraction
bars
and
that
sort
of
thing
and
yeah
so
he's
it's
a
jointly
supervised
with
someone
in
software
engineering,
where
they've
developed
this
kind
of
visual
programming
language
and
are
working
on
ways
to
use
it
to
generate
diagram
based
questions.
E
There's
definitely
a
lot
of
ideas
being
tossed
around
that
could
be
great
content
for
for
projects
in
the
all
eye
track
of
the
learn
lab
summer
school.
I
wonder
if
do
folks
know
about
that
opportunity.
F
You
one
thing
about
the
diagram
based
problems:
do
you
have
any
a
link
to
more
information
on
that,
not
sure
if
that
might
potentially
work
for
some
of
the
kind
of
molecule
drawing
in
in
chemistry,
I'm
not
sure
how.
F
E
F
E
Yeah,
I
I
can
connect
you
with
nemo
the
phd
student
and
and
see,
but
you
should
do
you
know
about
martina
rao's
work
with
in
chemistry
and
particularly
with
chemistry,
representation.
I'm
dave
definitely.
B
A
A
E
B
B
B
I
just
lost
my
my
multi
screens
here.
Oh
I
popped
in
the
the
link
to
learn
lab
summer
school.
It
is
an
amazing
experience
and
if
you
haven't
done
it
in
any
tracks
in
the
past,
then
I
highly
encourage
encourage
anyone
on
here
to
go
and
attend
and
do
it
and
you
don't
have
to
be
worried
that
you
have
to
do
something
really
or
that
I've
had
people.
Tell
me
like.
B
Oh,
is
it
really
for
me
and
ask
me
questions
about
that
and
I
just
have
to
say
just
just
jump
in
and
go
because
the
experience
is
amazing.
You
get
exposed
to
other
aspects
of
learning
engineering
that
you
might
not
even
know
exists
and
it's
a
really
great
energy
and
you
get
to
see
a
lot
of
different
things
going
on.
So
I
highly
encourage
you.
It's
a
great
experience.
B
It's
a
really
good
way
to
get
your
feet
wet
and
just
you
know,
dip
a
toe
in
even
and
get
the
exposure,
and
we
have
had
folks
who,
in
the
oli,
track,
develop
materials.
B
B
E
The
application
deadline
is
may
28th,
but
it
since
we'll
be
online.
We're
we're
much
more
open
to
to
anyone
coming,
but
we
use
the
applications
to
help
make
project
teams
and
and
each
team
gets
them.
Two
mentors
and
they're
different
different
tracks
do
that
slightly
differently,
but
that's
a
general
idea.
Is
that
really
it's
really
hands-on
as
they're
general
lectures,
but
there's
a
lot
of
hands-on
work
with
mentorship
along
the
way.
E
B
I
want
to
thank
everyone
for
coming
and
sharing
their
ideas
and
experiences,
and
I
will
certainly
follow
up
with
some
folks
who
and
by
the
way,
if
anyone
ever
needs
some
one-on-one,
you
know
demos
or
walking
through
some
functionality
that
we
that
we
have
available
already.
Please
don't
hesitate
to
reach
out
to
me.
I'm
always
happy
to
talk
and
show
things
and
dig
in
to
the
tools
and
and
talk
shock
talk,
design
and
yeah,
just
thanks
everyone
and
attend
next
next.
B
B
You
can
sign
up
for
more
of
these
meetings
or
even
there's
a
link
on
there
to
submit
ideas
for
taurus,
however,
know
that
we
take
very
detailed
notes
in
this
meeting
and
we
actually
produce
user
stories
out
of
the
conversations
that
we
have
so
just
so.
You
know
that
whatever
is
discussed
here
really
we
take
it
seriously
and
we
consider
it
as
part
of
what
we
need
to
build
into
taurus.
So
let
me
share
that
link
and
yeah
have
a
great
weekend.
Everyone
and
sorry.
A
Page,
the
the
student
cognition
toolbox
team
missed
the
first
session
and
I
watched
the
whole
session
this
morning
to
see
what
I
had
missed.
B
Lauren
I'll
reach
out
to
you,
let's
schedule
something:
let's
you
and
I
just
talk
and
and
talk
through
all
the
needs,
because
I
know
you
you
guys
use
the
full
functionality
of
echo
and
and
oli
so
yeah.
I
think
that's
worth
a
one
on
one
to
see
where
all
of
you
you
know
what
all
we
can
enhance
for
you.