►
From YouTube: Building CS50 with GitHub - GitHub Universe 2017
Description
Presented by David Malan, Harvard University.
CS50 is Harvard University's largest course and edX's largest MOOC. For years, the course relied on homegrown tools to manage students' homework submissions and teaching assistants' grading workflows. The course eventually migrated those tools to the cloud, leveraging IaaS but still reinventing too many wheels. The course has since embraced PaaS, with its server-side tools now Dockerized, and SaaS, with its client-side tools now implemented as abstractions atop GitHub. Via the latter, in particular, have we increased our efficiency and reduced our costs. We present in this session how and why we have leveraged GitHub to support CS50's thousands of students and hundreds of teachers.
About GitHub Universe:
GitHub Universe is a two-day conference dedicated to the creativity and curiosity of the largest software community in the world. Sessions cover topics from team culture to open source software across industries and technologies.
For more information on GitHub Universe, check the website:
https://githubuniverse.com
A
All
right,
thank
you
so
much
for
coming.
This
is
all
about
building
cs50
with
github.
My
name
is
David
Malan
and
I
teach
a
course
called
cs50
or
computer
science
50
introduction
to
computer
science,
one
at
Harvard
and
I
thought
I'd.
Give
you
a
sense
of
some
of
the
problems
we've
struggled
with
over
the
years,
some
of
the
solutions
we've
implemented
for
better
or
for
worse
and
ultimately
tell
a
story
about
how
many
now
off-the-shelf
tools
we
have
integrated
into
building.
What
is
ultimately,
this
introductory
course.
A
Here,
then,
is
the
picture
of
the
demographics
that
we
have,
and
this
paints
a
picture
perhaps
of
the
scale
on
which
we've
aspired
to
operate.
So
not
only
your
students
in
this
course
doing
their
work,
they're
submitting
the
work,
we're
reviewing
their
work
to
different
levels
of
engagement
based
on
the
type
of
student
that
we
have,
and
so
how
do
we
actually
go
about
doing
this?
Well,
we've
struggled
a
few
problems
in
the
past
and
thankfully
tools
like
github.
It
been
helping
us
solve
these
for
some
time.
A
The
course
itself
has
a
lot
of
web-based
applications
via
which
we
allow
students
to
interact
with
material
and
follow
along
and
so
of
course,
using
something
like
github
and
get
more
generally.
It's
actually
develop
these
apps,
not
just
myself,
but
our
team
of
teaching
assistants,
many
of
them
undergraduates
and
students
themselves,
have
certainly
been
a
common
workflow
deploying
apps
these
days
using
services
like
Travis,
so
that
we
actually
deploy
those
apps
that
we
do
develop
on
our
local
machines,
pushing
them
to
services
like
github.
A
That
then
triggers,
as
most
of
you
may
know,
post
receive
hook
of
sorts.
That
then
ensures
that
your
code
is
actually
in
some
cloud
provider.
We
happen
these
days
to
use
AWS
and
then
even
things
like
hosting
documents
for
the
first
seven
or
so
years
out
of
the
eleven
years,
I've
taught
this
course
I
was
using,
say
Microsoft
Word,
and
there
was
one
person
in
the
world
who
had
access
to
the
source
of
these
documents.
A
But
there
remains
some
problems
in
the
present
and
we've
spent
several
years
and
implementing
different
approaches
to
these,
a
few
of
which
I'll
share
here
today
and
to
ultimately
emphasize
just
how
consistently
we
I
think
made
mistakes
and
implementing
more
than
we
needed
to
and
home
growing
more
solutions
to
problems
than
we
needed
to,
and
these
I
daresay
are
the
three
problems
on
which
we
focused
over
the
past
year.
Students
doing
work
students
submitting
their
work
and
teaching
fellows
our
teaching
assistants
actually
reviewing
this
work.
A
We
have
generally
solved
with
a
homegrown
tool
students
submitting
their
work
to
the
class
for
review,
we've
generally
solved
with
a
homegrown
tool,
Teaching
Fellows,
reviewing
work,
we
generally
solved
with
a
homegrown
tool
and
I
daresay.
This
is
not
uncommon
with
in
higher
ed
I'm,
certainly
among
Cs
faculty,
probably
myself
included,
who
all
we
can
do
it
better
than
everyone
else,
and
that's
why
we
tend
to
work
perhaps
in
these
silos,
and
certainly
not
necessarily
sharing
as
many
solutions
as
we
might.
But
allow
me
to
propose
the
approach
we've
ultimately
took
ourselves
over
the
years.
A
A
So
for
the
context
here,
it
wasn't
just
that
every
year
we
decided
we
could
do
better
than
the
past
like
it
was
an
evolving
platform.
It
was
an
evolving
course
and
curriculum
and
involving
technologies
for
the
students.
So
back
in
the
day
in
the
late
90s,
when
mi
myself
took
the
class
and
then
in
the
mid
2000s,
when
I
took
over
the
class,
students
would
SSH
or
way
back
in
the
day
telnet
to
a
server
that
might
look
like
this.
A
They
all
had
a
home
directory
on
the
server
or
the
cluster
of
servers
using
NFS
shares
or
a
network
file
system
to
actually
share
their
home
directories
on
multiple
computers.
This
was
a
wonderful
topology,
because
then
everything
was
centrally
managed.
Everyone
was
on
effectively
the
on't
with
the
same
system,
but
this
was
managed
by
Harvard's
IT
folks,
and
there
was
tens
of.
There
was
tens
of
thousands
of
other
users
using
these
systems.
We
didn't
have
the
requisite
root
access
with
which
to
upgrade
software
configure
things
ourselves.
A
We
didn't
have
very
easy
access
to
students,
own
submissions,
because
we
ourselves
were
just
authorized
as
users
on
the
system.
So
a
few
years
later
came
out.
Nice
Harvard
edu
for
the
new
instructional
computing
environment,
which
fun
fact
of
23
years
later
is
still
operating
at
these
URLs.
If
you
SSH
to
or
if
these
host
names,
if
you
SSH
to
them-
and
it
was
little
different
other
than
a
oh.
A
So
that
was
step
phase
two
for
us
and
then
cloud
services
came
into
being
so
sort
of
in
the
late
2000s.
When
services
like
Amazon
were
coming
into
play,
we
actually
realized.
This
was
a
technological
opportunity
for
us
to
actually
migrate
the
same
kind
of
topology
that
had
worked
pretty
well
for
us
on
campus,
with
the
shared
cluster,
with
all
students
on
the
same
system,
but
one
that
we
could
actually
control
ourselves,
and
so
we
actually
had
a
load
balance
cluster
of
workstations,
running
some
variants
of
Linux.
A
We
used
NFS
to
load
balance,
as
this
picture
suggests
here,
so
that
all
students
instead
started
SSH
into
cloud
cs50.net
instead
of
to
Harvard's
own
cluster,
which
was
a
great
pedagogical
improvement
for
the
class.
The
price,
of
course
we
paid
there
was
that
now
we,
the
instructors,
have
become
our
own
sis
administrator,
a
system
administrator.
So
there
was
that
price
we
paid
a
few
years
later.
Did
we
transition
to
something
a
little
more
distributed?
A
A
long
story
short
the
course
is
long
supported,
open
courseware,
where
everything
is
made
freely
available
to
anyone
who
wants
to
tune
in
online,
but
for
many
years
it
was
very
passive
experience.
You
could
download
the
distribution
code.
The
problem
sets
you
couldn't
really
engage
with
the
course
or
do
the
problem
sets,
because
you
didn't
know
what
environment
variables
we
had
set.
You
didn't
know
what
versions
of
the
compiler
or
debugger
we
were
using.
A
Nine
wants
a
startup
now
one
of
Amazon's
web
services,
so
that
students
have
all
of
the
same
features
and
as
this
screenshot
suggests,
file
management,
tabbed
code
windows,
all
the
features
you
might
expect
in
a
lightweight
ID,
but
as
well
as
and
most
importantly,
a
terminal
window
in
which
they
have
a
full
root
access
over
a
container,
a
docker
container
underneath
the
hood.
So
what
are
the
processes
by
which
students
have
historically
submitted
back
in
the
day?
And
even
when
I
took
the
course?
A
There
was
a
set
UID
script
running
on
that
central
server,
which
meant
that
whenever
you
ran
the
command,
it
didn't
run
as
you
at
ran
as
routes,
which
meant
that
you,
the
script,
could
see
your
code
put
it
in
the
courses
account
and
actually
do
whatever
work.
It
needs
to
get
it
to
the
teaching
assistants
for
evaluation.
The
interface
was
pretty
arcane,
so
we
improved
upon
that
with
something
called
cs50
submit
there
after
essentially
wrapping
some
of
the
university's
own
code
to
make
it
a
little
more
user
friendly.
A
A
And
unfortunately
it
was
a
disaster
not
because
of
Fi,
but
really
because
of
the
demographic
we
had
at
the
time
and
because
of
the
stupid
little
things
that
was
sort
of
besides
the
point
git,
as
you
may
know,
tends
not
to
like
it.
If
you
accidentally
have
a
another
dot,
git
directory
is
a
sub
directory,
because
then
it
gets
sort
of
blacklisted
from
actually
pushing
it
to
the
server
and
that's
fine,
it's
fixable,
but
it's
not
very
conveniently
fixable.
A
You
would
get
conflicts,
and
so
those
two
were
sort
of
difficult
to
resolve.
So
all
solvable.
But
completely
beside
the
point.
When
in
this
class
we
were
trying
to
focus
on
implementing
solutions
to
software
problems
and
teaching
students
to
program
better
and
not
getting
hung
up
on
things
that
really
were
rather
intellectually
less
interesting,
and
so
we
transitioned
from
there
to
another
tool.
A
A
What's
that
I,
could
you
don't
think
all
right?
So
we'll
do
a
little
mr.
Rogers
here
all
right,
maybe
I
should
just
behind
the
podium
okay.
So
this
was
not
part
of
the
plan,
all
right
so
we'll
see
if
the
Sox
have
to
go
next.
So
in
terms
of
reviewing
work
back
in
the
day
truth,
this
has
been
an
evolution.
We
hashed
about
some
fate,
so
we
said
we
you.
Certainly
we
used
to
use
paper
back
in
the
late
90s
students
would
print
out
their
code
and
the
tapes
would
physically
mark
on
it.
A
We
eventually
learned
about
PDFs,
and
then
we
would
annotate
things
with
Acrobat
or
Bluebeam,
or
some
tool
like
that,
and
then
we
came
into
the
mix
that
same
tools
before
cs50
submit
which
actually
had
this
type
of
user
interface.
Long
story
short
at
this
phase.
In
our
development
we
were
especially
eager
for
the
TFS
to
be
able
to
provide
certainly
more
convenient
reviews
of
students
code,
but
also
start
to
Greta
collect
the
quantitative
data
that
had
long
been
lost
thanks
to
paper
and
PDFs.
A
Knowing
what
kind
of
comments
the
teaching
fellows
were
writing
and
how
frequently
they
were
providing
feedback
at
what
length
they
were
providing
feedback
to
students
which
students
were
actually
reading
their
feedback.
So
there's
a
lot
of
pedagogically
opportune
data
that
we
had
never
had
access
to
that.
A
Have
we
been
able
to
put
together
a
pretty
robust
solution
for
all
of
these
demographics,
so
why?
What
is
it
that
we
actually
need
to
solve
these?
Several
problems
of
students
doing
work,
submitting
work
and
the
Teaching
Fellows
reviewing
that
work.
So
we
need
a
client
by
which
students
can
submit
their
code,
whether
it's
command
line
or
web-based.
We
needed
a
server
actually
listening
so
as
to
receive
that
work
and
store
it
somewhere
on
disk.
A
We
needed
UI
for
code
review
much
like
the
one
we
ourselves
have
been
implemented
and
really-
and
this
is
the
anomaly
for
us-
we
needed
a
permissions
model
that
was
a
little
bit
flexible.
We
have
Teaching
Fellows
and
Cambridge
Teaching
Fellows
in
New
Haven.
We
have
instructors
in
both
cities,
we
have
students
in
both
cities
and
we
literally
have
tens
of
thousands
of
students
following
along
via
OpenCourseWare
online,
all
of
whom
should
have
federated
access
to
different
types
of
data
and
different
subsets
of
the
data,
but
wonderfully
too
and
something
like
github.
A
Do
you
have
the
notion
of
teams
and
contributors
and
user
permissions
so
that
plus
support
for
things
like
post
receive
or
post
commit
hooks
really
gave
us
so
that
we
could
trigger
Auto
grading?
Really.
This
was
the
menu
of
things
that
we
got
out
of
using
a
third-party
tool
like
github,
and
what
every
bullet
here
therefore
represents
is
literally
every
feature
we
ourselves
that
not
have
to
implement
or
wheels.
We
didn't
need
to
re-implement
so
reinvent.
A
So
how
do
we
go
about
actually
using
the
platform
and
how
do
we
actually
think
about
which
of
its
features
to
use
so
there's
a
few
options
when
it
comes
to
code
reviews
alone,
so
you
might
be
familiar
with
gist's,
where
you
can
throw
up
one
or
more
files
and
pretty
easily
annotate
on
it.
The
catch
with
gist,
though,
is
that,
even
if,
especially
if
the
file
is
long
or
if
you
have
multiple
files,
the
commenting
system
is
essentially
relegated
to
the
bottom
of
the
user
interface
and
so
certainly
for
newbies.
A
It's
not
ideal
to
be
talking
about
a
program.
That's
this
long
with
all
of
the
comments
relegated
here
so
they're,
just
sort
of
in
terms
of
UX
constantly
going
back
and
forth,
and
not
actually
seeing
proximal
to
what
they
did
well
or
not.
Well,
the
Teaching
Fellows
own
comments,
there's
also
pull
requests
and
pull
requests
are
also
used
if
you're
familiar
with
the
tool
called
github
classroom,
which
we
actually
seriously
considered,
we
thought
what
better
way
to
avoid
reinventing
the
wheels
than
to
build
on
top
of
our
use.
A
This
we
would
need
to
add
some
of
those
additional
features
for
our
various
demographics,
but
ultimately
a
model
that's
used
or
we
considered
this
pull
request
model
was
a
little
too
sophisticated
for
the
UX
that
we
wanted.
Our
students
have
in
the
pull
request
model.
The
idea
would
be
that
students
have
a
repo
for
the
homework
they're
working
on.
A
They
would
then
do
their
work,
push
it
to
their
own
repo
on
github
comm
and
then
submit
a
pull
request
from
their
from
their
repo
or
their
fork
of
a
main
repo
to
a
central
one
to
which
I
and
the
Teaching
Fellows
actually
have
access,
and
then,
by
the
typical
pull
request
you
I,
could
we
actually
comment
on
their
code
and
provide
feedback
iteratively?
The
downside
here,
though,
is
that
that
meant
now
students
had
to
understand
gits
and
they
had
to
understand
it
in
week.
A
Zero,
the
very
first
days
of
the
class
which,
for
an
introductory
course,
more
almost
all
of
whom
are
non
mayor,
90%
of
whom
are
non
majors
these
days.
It
simply
wasn't
the
sort
of
mental
hurdle
we
wanted
early
on
and
ultimately
to.
We
didn't
necessarily
want
students
having
both
their
own
repo
and
then
the
staffs
repo
and
trying
to
explain
to
them
conceptually
like
why
there
is
in
fact
this
dichotomy
because
they
were
the
only
contributors
ultimately
to
these
repositories.
And
so,
ultimately,
we
settled
on
this
approach.
A
As
the
student
is
working
from
start
to
finish,
we
really
for
the
courses
purposes,
only
care
about
the
end
result
and,
unfortunately,
if
we
now
have
to
go
through
n
different
commits-
and
you
eyes
like
this-
just
to
see
the
various
deltas
from
the
first
day
of
the
week
to
the
last
day
of
the
week
when
the
student
finally
submitted
that
to
wasn't
ideal,
we
wanted
to
actually
have
access
to
the
full
code
and
provide
an
entire
review
of
their
code.
Ultimately,
so
what
did
we
ultimately
go
about?
Doing?
A
We
saw
end
up
for
a
github
organization,
as
you
might
have
yourself.
We
then
went
to
education,
github
comm,
whereby
you
can
actually
flag
and
organize
is
educationally
oriented,
which
means
we
did
not
have
to
pay
for
the
private
repositories
and
there's
also
no
limit
and
I
dare
say.
One
of
the
challenges
for
us
is
finding
a
good
hosting
service.
A
They
can
tolerate
tens
of
thousands
of
repos
in
the
organization,
perhaps
not
a
common
problem
within
the
typical
org,
but
indeed
that
was
something
that
was
possible
here
and
then
we
had
to
decide
this
once
we
had
decided
that
we
wanted
students
to
have
their
own
repository
in
our
organization
so
that
we
had
full
control
over
at
API,
and
otherwise
there
became
this
question.
So
do
we
give
students
one
repo
per
project,
or
do
we
use?
A
One
branch
per
project
like
the
norm
would
absolutely
be
to
use
one
repository
per
project
or
per
homework
assignment
the
catch
here
is
that
we
have
nine
of
these.
During
the
course
of
a
semester,
the
students
are
seeing
git
and
github.
For
the
first
time
we
have
say
fifty
thousand
repos
right
now
in
the
organization.
A
So
even
though
it's
a
constant
factor
difference
the
difference
between
fifty
thousand
and
four
hundred
and
fifty
thousand
repos
is
kind
of
significant,
especially
when
these
projects
only
have
a
life
of
maybe
one
week
long
and
we're
not
contributing
to
them
over
time.
So
considering
one
branch
per
repo
was
an
option
instead
and
in
terms
of
what
the
UI's
would
have
been
for.
Students
is
if
they
went
to
our
organization's
page.
They
could
have
searched
further,
your
user
name
and
seeing
only
those
repositories
to
which
they
have
access.
A
But
we
would
have
needed
to
come
up
with
some
naming
scheme
to
avoid
collisions
so
that
every
students
could
have
some
unique
identifier
for
the
exact
same
problem
sets.
But
we
noticed
there
is
this,
as
you,
yourselves
might
know,
from
github
zone
UI,
there's
a
very
pretty
simple
drop-down
menu,
via
which
you
can
toggle
among
branches,
and
this
was
exactly
a
feature
in
our
own
coding,
review,
UI
and
cs50
submit
some
years
ago,
and
so
we
realized.
A
Maybe
we
could
start
to
leverage
even
this
piece
as
well,
and
so,
if
you'd
like
to
take
a
look
at
the
code,
ultimately
that
we
put
together
thus
was
born,
submit
50
the
command-line
tool
to
which
I
referred
to
earlier,
that
essentially
wraps
and
therefore
abstracts.
All
of
the
git
commands
that
students
themselves
would
have
had
to
write
or
execute
back
in
our
github
fi
days,
and
it
also
handles
a
lot
of
various
corner
cases.
For
instance,
we
use
a
separate
get
dur.
A
We
essentially
put
their
code
into
slushed
their
get
directory
into
slash
temp,
so
it
does
not
collide
with
anything
in
their
local
and
we
do
a
number
of
other
tricks
as
well
as
I'll
show.
So
this
is
the
user
experience
for
a
student
here
is
our
so
called
cs50
IDE
from
cloud
9.
This
is
a
terminal
window,
simply
maximized
in
the
screen,
and
the
student
is
currently
in
a
directory
called
workspace,
slash
below
where
he
or
she
might
have
a
file
like
hello,
dot,
C
or
any
other
files.
They'll
run,
submit
50,
hello.
A
We're
hello
in
this
case
simply
represents
a
unique
slug
or
technically
branch
name
to
which
the
code
will
be
submitted.
They
type
in
their
github
username,
their
github
password
and
then
they're
prompted
as
here
to
confirm
which
files
they're
actually
going
to
submit
using
another
wonderful
free
feature
of
git.
Ignore
files
by
storing
these
get
ignore
or
exclude
files.
Server-Side,
can
we
also
finally
start
black
listing
all
of
the
binary
files
or
core
dumps
that
we
actually
don't
want
white
listing
and
requiring
files
that
we
do
actually
want?
A
Students
are
prompted
whether
or
not
they
indeed
want
to
submit
these
files
and
if
they
confirm
as
much
they
have
submitted
in
this
case,
hello,
and
all
that
has
done
effectively-
is
the
requisite
get
commands
underneath
the
hood
again
at
all,
when
these
are
somewhat
simplified,
to
keep
keep
things
simple.
I'll
get
commits
with
a
message
that
we
defined
for
them,
and
then
it
get
pushed
to
specifically
the
branch
that
they
specified
at
the
command
line
and
if
they
resubmit
it
and
simple,
though
this
is
and
baked
in
to
get
though
this
is.
A
This
was
one
of
the
very
serious
challenges
some
years
ago,
when
students
were
moving
dot,
files
or
doctor
or
not
noticing,
when
they
move
files
around
that
there
was
a
documentary
or
others
simply
doing
a
trick
like
this,
whereby
we
do
a
bear
clone
of
the
repository.
So
even
if
they're
on
a
brand-new
machine
and
their
code
has
somehow
made
it
to
this
other
machine.
Do
we
clone
whatever
submission
history?
A
They
actually
already
have
we
update
our
reference
to
head
and
actually
then
push
the
latest
version
of
their
code
onto
it,
because
here
too,
we
ultimately
do
not
want,
in
this
case
to
just
see
dips
of
students
code.
Ultimately,
you
will
see
that
we
want
to
provide
those
same
comments
and
so
by
leveraging,
ultimately,
the
github
API,
specifically
the
repos
set
of
functions,
we're
able
to
achieve
a
user
experience
for
students
like
this.
A
So
this
is
our
submit
50
organization,
a
user
like
J
Harvard
for
John
Harvard,
and
this
is
an
example
where
the
user
has
submitted
his
hello
assignment.
You
can
see
a
little
drop
down
here
where
he
or
she
can
select
what
branch
it
is
they're
looking
at
and
therefore
what
homework
assignment
they've
submitted.
A
And
this
ultimately
is,
the
goal
is
a
simple
of
an
ideas
as
this
is.
The
ultimate
goal
was
to
provide
students
with
this
kind
of
experience
and
in
turn,
their
teaching
fellows
with
this
experience,
so
that
they
could
comment
on
each
and
every
one
of
those
lines
like
in
this
case
here
doing
it
in
line,
unlike
at
the
bottom
of
a
gist
or
by
some
other
UI.
So
how
do
we
actually
provide
access?
A
Well,
if
you've
used
github
permissions
before
you
might
know
that
you
can
give
a
particular
user
as
a
contributor
right
permissions,
as
J
Harvard
might
need
a
Teaching
Fellow
username
TF
here
read
permission
so
he
or
she
can
actually
review
the
code
and
certainly
any
number
of
other
teachers
that
might
need
it
via
web
hooks.
Did
we
install
in
the
submit
50
organization
a
hook
that
results
in
every
github
comm
payload
being
sent
to
one
of
our
servers
so
that
we
can
execute
certain
commands?
A
And
this,
for
instance,
is
the
magical
incantation
that
we
use
to
create
full
file
diff
again,
if
a
student
is
doing
multiple
commits
over
the
course
of
the
week
by
submitting
his
or
her
work
again
and
again
and
again,
you
would
get
one
commit
for
the
changes
they
made
up
here.
Another
commit
for
the
changes
they
made
down
here,
another
commit
for
the
changes
down
here
and
so
forth.
A
We
essentially
run
these
commands
in
the
context
of
a
few
others
where
we
clone
that
repo
specifically
checking
out
the
branch,
rather
the
tag
that
the
student
had
just
submitted
by
our
hook
and
then
run
this
command
check,
50,
which
is
independent
of
gif
itself.
But
it's
triggered
simply
as
a
result
of
that
hook.
But
a
challenge
we
had
years
ago
was
when
you've
got
students
code
on
your
servers,
and
now
you
actually
want
to
test
it,
and
you
want
to
test
it
without
human
eyes,
auditing
the
code
even
a
priori.
A
Where,
and
how
do
you
run
this,
especially
when
you
have
tens
of
thousands
of
humans
taking
a
class
who
you
don't
know
and
for
whom
it
would
be
a
delightful
challenge?
We
presume
to
try
to
take
down
our
services
or
be
more
adversaries
than
they
are
students.
So
we
needed
to
sandbox
students,
environments
in
some
way
and
so
years
ago,
in
the
first
version
of
this
back
in
2012.
So
we
kind
of
cobble
something
together
ourselves.
A
And
then
you
can
run
a
virtual
3
virtual
machines,
each
running
its
own
operating
system
with
its
own
software
installed
and
its
own
application
running
docker.
You
can
think
of
as
a
lighter-weight
version
of
this,
whereby
you
run
docker,
on
top
of
which
you
run
only
different
software
layers
that
give
you
slightly
different
versions
of
a
base
image
that
you
might
have.
A
So
they
all
share
some
common
kernel
or
some
core
of
the
operating
system,
but
it's
much
lighter
weight
and
that
you,
literally,
as
per
these
pictures
from
docker,
don't
actually
need
three
versions
of
Linux
or
Windows
or
some
other
OS
actually
installed
on
the
host.
So
via
docker
have
we
been
able
to
actually
automate
the
sandboxing
and
the
testing
of
students
code
so
that
after
they've
submitted
after
it's
been
cloned
after
check
when
check50
runs
it's
running
in
an
environment
quite
like
that
depicted
at
right
there,
and
where
do
we
do
this?
A
So
there's
a
number
of
cloud
providers
these
days,
docker
itself
is
becoming
fairly
commodity.
On
various
cloud
platforms,
we
happen
to
use
Amazon
ECS,
which
is
their
elastic
container
service,
which
just
means
that
it's
pretty
easy
these
days
for
us
to
start
and
stop
containers
as
we
need,
which
means
the
moment
of
students
code,
has
been
cloned.
We
started
container
mounting
their
directory
inside
of
that
container
and
run
it
essentially
in
isolation
of
everything
else.
A
So
the
high-level
description
of
where
we're
now
at
is
that
a
student
pushes
his
or
her
code
to
github
via
submit
50.
That
command-line,
a
post
receive
hook,
inform
cs50
of
that
submission
by
our
back-end
server.
That's
always
running.
We
in
turn
that
server
in
form
Amazon's
container
service
that
there
has
been
submission
so
that
ECS
itself
can
clone
that
repository,
auto
grading
things
inside
using
check50
and
then
ECS
by
another
callback.
A
Web
hook
essentially
tells
us
cs50
and
our
main
server
of
the
scores
so
that
ultimately
we're
able
to
put
into
our
own
database
any
of
those
final
results
and
so
I
sort
of
admitted
before
that.
Despite
this
wonderful
trajectory
of
homegrown
homegrown
homegrown,
not
homegrown,
we
kind
of
added
some
homegrown
at
the
very
end,
but
moral
of
our
story
ultimately
has
been.
A
Much
rather
see
a
menu
of
the
work
that
they
need
to
be
done,
and
so
this
is
a
screenshot
thanks
to
our
team
Brian,
especially
back
home,
whereby
this
is
what
our
teaching
fellows
now
see,
where
every
timestamp
in
that
drop-down
represents
a
git
branch
or
gift
tag,
each
of
which
has
been
time
stamped
per
that
work
earlier.
You
can
see
at
the
far
right
the
little
check
boxes
that
indicate
this
code
has
had
been
automatically
checked,
and
so
our
teaching
of
fellows
can
now
navigate
via
this
UI
and
then
ultimately,
even
input
their
grades.
A
So
this
is
the
one
thing
for
which
there's
probably
not
a
reasonable
feature:
request
of
github
to
support
sort
of
writing
arbitrary
data
to
repos
like
grades
in
this
way
akin
to
a
database,
and
so
we
store,
for
instance,
a
correctness,
score
see
a
design,
score,
D
a
style,
score
S,
and
that
simply
happens
to
be
in
our
own
sequel
database.
But
you
can
see
there
the
branch
names
that
map
directly
to
what
it
is
that
is
being
stored
on
github,
so
long
story
short.
A
If
you'd
like
to
take
a
look
at
or
contribute
to
or
fork
or
use
any
of
this
see
any
of
these
URLs
here,
I'm
happy
to
stick
around
for
questions.
But
again,
there's
any
takeaway
and
message
we'd
like
to
convey
is
to
not
do
what
we
did
for
years,
which
is
implement
homegrown
to
up
their
homegrown
tool
and
whether
it's
github
or
something
else
try
to
stitch
together
and
piece
together
via
hooks
and
all
of
these
modern
tool
chains.
Solutions
to
problems
that
you
yourself
might
have.
Thank
you
so
much
for
coming.