►
From YouTube: Keeping the Node.js Community Infrastructure Humming - an Update From the Build Workgroup
Description
Keeping the Node.js Community Infrastructure Humming - an Update From the Build Workgroup - Michael Dawson, IBM
"The build workgroup (https://github.com/nodejs/build/) has the
mission to provide Node.js Foundation projects with solid computing
infrastructure covering a wide range of platforms and different
software stacks. In this talk we will introduce the
general philosophy of how infrastructure is sourced, which has
a strong focus on community donations, the
overall picture of the substantial infrastructure we’ve manged
to build using this approach, some of the interesting
interactions and, just as important, how you might get involved.
Come learn about the infrastructure that powers
the delivery of Node.js !"
A
So
hi
I'm,
Michael
Dawson
and
today
I'm
going
to
talk
to
you
about
keeping
the
no
Jas
infrastructure
community
humming
a
little
bit
about
my
stuff
about
myself
before
I
get
started:
I'm
Michael,
Dawson
I'm,
the
tech
lead
for
the
team
within
the
IBM
runtime
technologies
group
that
builds
nodejs
for
consumption
and
use
in
our
products.
We
have
about
fifty
three
products
that
incorporate
it
as
well
as
for
delivery
to
our
customers
and
things
like
our
blue
max
cloud
platform.
Now
our
development
model
is
very
much
community
focused.
A
And
that's
you
know
why
I'm
here
today
talking
to
you
about
the
build
work
group,
so
I'm,
just
the
one
that
gets
to
be
here
and
have
the
privilege
to
talk
to
you
about
all
the
great
work
that
the
build
work
group
is
doing.
But
there's
a
there's,
a
great
group
of
people
working
on
this
and
keeping
the
infrastructure
running
in
particular,
I'd
like
to
call
out
Joanne
Bergstrom.
A
If
you
ever
been
involved
in
trying
to
get
something
up
and
running
in
the
infrastructure,
you'll
probably
run
into
him,
because
he
really
carries
a
large
portion
of
the
load
and
making
sure
that
things
get
done
and
that
the
infrastructure
overall
keeps
on
working
as
it
should,
as
well
as
what
bag?
Who
also
has
you
know,
has
made
it
quite
a
big
contribution,
particularly
around
you
know,
things
like
the
raspberry
PI's
that
you'll
see
later
later
on
in
talk,
but
overall,
it's
just
you
know
great
to
have
such
a
good
group
of
people
working
together.
A
As
always,
though,
if
you're
interested
in
this
area-
and
you
want
to
get
more
involved,
we
always
need
more
people
to
help
out
and
we
look
forward
to
having
you
join
us
so
just
to
start
out.
What's
the
purpose
of
the
build
work
group,
so
the
core
purpose
is
to
provide
the
computing
infrastructure
that
we
need
to
keep
all
the
things
running.
So
things
like
our
builds
our
website
and
so
forth.
A
We
also
focus
on
streaming
streamlining
our
builds
and
tests,
because
we
want
to
make
sure
we
can
get
those
three
through
quickly
enough
and
efficiently
enough,
that
it
doesn't
slow
down
the
overall
contribution
process
and
so
in
specifically,
we
end
up
doing
things
like
maintaining
the
J
infrastructure,
the
benchmarking,
graphs
and
generation
that
we've
seen
earlier
today.
The
website,
like
node.js
org
and
much
much
more,
which
I'll
talk
a
bit
about
as
we
go
through
the
presentation.
A
One
thing
that's
a
little
bit
unique
is
that
our
approach
is
very
much
donation
focused.
We
don't
necessarily
want
to
go
out
and
buy
all
the
infrastructure
that
we
want.
We
want
to
find
people
who
can
contribute
that
so
companies
or
individuals
who
can
help
contribute
the
infrastructure
that
we
need,
because
that's
really
in
line
with
nodes
philosophy
of
you
know,
contribution
driven
development,
and
you
know
where
we
can
find
people
that
are
interested
to
contribute.
A
The
infrastructure
probably
also
means
that
we
have
active
people
who
can
get
involved
in
in
node,
and
so
by
using
that
approach
we
get
and
encourage
more
involvement
in
the
node
community
itself,
which
is
what
we
want.
Most
of
the
infrastructure
is
cloud-based
and
it's
spread
across
one
or
more
providers
so
that
we
actually
have
some
redundancy
and
so
forth
in
terms
of
our
infrastructure
providers.
We
have
tier
1
and
tier
two
providers.
The
only
real
distinction
is
that
the
tier
1
providers
provide
larger
portion
of
the
overall
infrastructure.
A
In
addition
to
sort
of
corporate
participants
and
donors,
we
also
really,
you
know,
really
trying
to
encourage
community
donations
as
well,
and
a
good
example
is
you
know
in
terms
of
our
arm
testing,
we
needed
quite
a
few,
our
machines
and
we
wanted
to
cover
the
different
architectures
like
arm
six,
seven
and
eight.
So
at
one
point
we
just
really
didn't
have
enough
it's
harder
to
get
the
tests
to
run
quickly
on
arm.
So
we
didn't
have
enough.
We
made
a
community
call
out
and
said
you
know
anybody
who
could
donate
hardware
resources.
A
That
would
be
really
great.
We
had
a
really
good
response
and
you
can
go
and
see
a
number
of
the
the
donators
there's
even
more
now.
The
nice
thing
is,
if
you
do
give
some
hardware
it'll,
usually
end
up
showing
up
with
your
name.
So
if
you
can
go
into
the
CI
and
I
just
hope,
the
one
that
I
donated
isn't
always
down
and
people
curse
that
or
anything
like
that.
But
this
is
a
nice
picture
from
from
rod
vague
that
shows
part
of
our
Raspberry
Pi
Custer.
A
So
you
know
we
have
quite
a
few
of
them
and
there
again,
all
through
the
community
donations
with
is
what
made
this
possible
diving
down
into
the
infrastructure
itself.
So
we
use
Jenkins
is
our
continuous
integration
system.
We
have
two
separate
environments:
we
have
a
basically
a
build
test
environment
and
then
our
release
CI,
and
we
have
those
separated
out
so
that
you
can,
you
know,
restrict
the
number
of
people
who
have
access
to
the
release,
see
IM
machines.
So
we
can
have
really
really
really
really
good
confidence
that
you
know.
A
We
know
the
configurations
are,
as
we
expect.
Only
the
the
software
that
we
we
want
to
have
running
is
running
on
those
machines
and
you
know
have
good
trust
in
the
binaries
that
we
build
and
then
ship
from
there
in
terms
of
the
build
infrastructure
we
have
about.
116
machines
at
last
count
a
whole
bunch
of
different
architectures.
You
know
x86
32
and
64-bit
ARM,
various
flavors
powerpc,
big
endian,
little
endian,
as
well
as
SS
s390x,
and
a
bunch
of
operating
systems
like
linux,
windows,
AIX,
mac,
OS,
x
and
freebsd.
So
quite
a
good
variety.
A
The
release
infrastructure
is
a
little
bit
smaller.
It's
about
14
machines
when
I
took
count
still
has
you
know
a
wide
diversity
of
architectures
as
well,
as
you
know,
a
slightly
smaller
set
of
operating
systems,
but
as
we
bring
on
support
for
more
and
more
operating
systems,
you
know
we
should
see
that
expand
to
pretty
much
the
same
set
that
we
have
in
our
build
infrastructure.
I.
A
So
this,
if
you
go
to
the
CI
and
everybody
can
go
and
take
a
look
at
the
CI
jobs.
This
is
the
first
page
that
you'll
be
presented
most
thing
most
times
builds
start
out
with
the
node
test,
pull
requests,
so
somebody
says:
hey:
have
a
pull
request.
I
want
to
test
before
that
gets
committed.
You
know
it's
basically
a
rule
that
that
needs
to
be
tested.
So
somebody
kicks
off
the
node
test,
pull
request,
job,
which
is
the
one
down
here
at
the
bottom.
A
It
then
kicks
off
the
one
that's
higher
up,
which
is
the
node
test,
commit
and
then
no
test
commit
farms
that
you
know
sort
of
fans
it
out
across
all
the
different
platforms.
So
if
you
have
a
pull
request,
it's
going
to
be
tested
across
all
those
different
architectures
across
all
those
different
operating
systems.
A
When
you
launch
your
build,
which
is
which
is
a
good
thing
in
terms
of
the
job
so
now
looking,
you
know
delving
into
the
job
the
the
node
test
commit
and
there's
generally
a
you
know,
one
that's
specific
to
each
of
the
platforms.
But
this
is
an
example.
There's
not
a
lot!
That's
really
interesting
in
the
job
itself.
We
use
Jenkins,
but
we
don't
necessarily
have
a
lot
of
the
logic
built
into
the
Jenkins
job.
Instead,
it
you
know
is
built
into
them
to
make
file.
It's
a
good
thing
this.
This
is
a
picture
of.
A
You
know
what
you
see
if
you
actually
open
up
the
jaw,
it's
a
bash
like
syntax.
So
if
you're
familiar
with
with
writing
bash
shell
scripts,
you
you'll
feel
fairly
at
home,
but
really
it
just
sets
up
some
environment
variables
and
then
calls
the
build,
see
I
tap
target
to
actually
start
off
the
whole
process.
A
The
good
thing
is,
though,
is
that,
although
most
people
don't
have
access
to
go
and
look
in
at
the
jobs
themselves,
everybody
has
access
to
the
make
files
which
are
in
github
and
if
you're
interested,
you
know
check
that
file
out
the
make
file
out
and
you
can
look
at
all
the
different
jobs,
their
dependencies
and
all
that
kind
of
stuff.
So
you
can
easily
delve
into
the
details
of
exactly
what
to
make
makes
up
those
build
jobs.
A
Looking
at
the
machines
that
are
used
when
you
launch
those
builds,
so
most
of
them
are
virtual.
There
are
a
few
key
exceptions,
so
the
first
one
is
armed.
It's
not
easy
to
go
out
and
get
armed
two
machines.
So
you
know,
through
the
donations
in
the
community,
we've
managed
to
build
quite
a
big,
raspberry,
pi
cluster.
You
can
see
that
the
grouping
there
again
a
nice
picture
from
the
rod
sent
me
and
so
arm
is,
is
or
physical
machines
located
at
node
source
that
we
used
to
build.
The
other
inception
is
our
benchmarking
machine.
A
So
we
have
a
machine
from
softlayer.
That's
in
software.
You
can
actually
get
physical
machines
and
you're
the
only
one
using
it,
and
we
use
that
for
benchmarking
so
that
when
we
have
multi
you
know
benchmarking
runs
on
different
days.
We
can
actually
compare
the
results,
because
we
know
that
nothing
else
is
running
on
that
machine.
The
machine
hasn't
changed
and
you
know
we
get
comparable
results.
So
that's
important,
so
that's
the
other
other
than
that.
It's
pretty
much
virtual
machines
basically
distributed
across
the
world.
A
If
you
know,
I
gave
you
the
list
of
tier
1
tier
two
providers,
and
so
those
150
machines
or
so
or
pretty
much
spread
out
across
all
those
different
providers.
The
thing
you'll
start
to
realize
is
that
is
a
lot
of
machines.
How
do
we
possibly
manage
all
those
machines?
There
are
also
cloud
machines
which
there's
not
really
any
guarantee
that
a
machine
you've
provision
today
is
going
to
be
there
tomorrow.
A
You
know
most
of
the
time
they
stick
around,
but
the
providers
don't
guarantee
that,
and
so
you
may
be
faced
with
having
to
reprovision
those
machines
at
a
pretty
regular
basis.
Our
answer
to
that
is
ansible
and
I'll.
Tell
you
about
that,
because
it's
quite
a
neat
tool
and
it's
something
that
you
know
really
helps
us
in
the
bill
group
manage
all
those
machines.
A
First,
nice
thing
about
ansibles:
it's
agentless,
you
don't
have
to
go
and
install
anything
on
the
machines
themselves.
It
connects
over
ssh
and
does
all
the
work
that
it
needs
to
do
just
by
using
ssh
itself,
there's
three
key
files:
the
ansible
inventory,
the
ansible
playbook
and
the
ansible
bars
and
I'll
delve
into
each
of
those
individual
files
as
we
look
at
it.
A
A
The
PlayBook
is
your
list
of
steps
that
you're
going
to
have
to
configure
the
particular
machine
and
something
like
the
list
of
packages
you
need
to
install
you.
Don't
necessarily
want
to
have
that
hard
coded
into
that
overall
set
of
instructions.
You
just
want
to
pull
that
out
and
say
you
know,
here's
my
list
of
modules
and,
if
I
need
to
add
an
extra
module,
I
can
easily
locate
the
section
to
be
able
to
add
that
in
so
that's
where
ansible
VARs
comes
in
a
sort
of
your
configurable
pieces
and
I
know.
A
This
is
hard
to
read,
but
it's
basically
a
list
of
steps,
so
ansible
will
go
through
and
say:
okay,
I'm,
going
to
start
out
by
first
updating
you
to
the
latest.
So
if
you're
on
a
boo
too
it'll
say,
let's
do
an
apt
get
update
and
if
you're
14
it'll
leave
you
at
14,
but
let's
apply
all
the
patches
for
that
particular
level.
It
then
says:
okay,
let's
do
things
like
well
I'm,
going
to
need
a
particular
user
I'm
going
to
have
to
install
we
use
a
user
called
I
ogs.
A
So
let's
create
that
user
on
the
machine
it'll
go
through
installing
all
the
packages
it
does
things
like
pulling
over
the
Jenkins
jar
that
that's
Jenkins's
is
a
java
application.
So
we
need
a
jar
file
which
will
run
the
agent
that
connects
to
the
overall
infrastructure,
so
it
installs
that
jar
file
installs
a
startup
script
so
that
you
know
when
your
machine
reboots
it
starts
the
Jenkins
agent.
So
people
very
nicely
go
through
and
runs
all
these
steps
and
generally
it
knows
about
the
different
ways
to
do
that
on
different
platforms.
A
So,
for
example,
on
a
boot
to
it
knows
maybe
use
apt-get
and
on
RedHat
you
use
yum,
and
so
you've
got
your
list
of
packages
and
it
can,
you
can
go,
do
through
go
through
and
do
all
those
things
for
it.
So
it's
a
really
great
tool
to
say
I'm,
starting
with
a
bear
machine
point
ansible
at
it,
and
you
know
some
amount
of
time
later.
1020
minutes
you
end
up
with
the
machine
which
is
fully
configured
and
can
connect
into
the
the
framework
and
really
it
saved
me
a
lot
of
time.
A
On
the
PowerPC
side
we
had
to
move
from.
You
know
our
initial
set
of
testament
initial
set
of
machines
to
sort
of
a
new
test
infrastructure
they
wanted
to
validate
to
the
final
production
system
that
you
know
where
the
the
you
know,
our
new
machines
are
kind
of
come
out
of
so
I've
had
to.
You
know,
configure
another
five
machines
and
you
know
less
than
a
couple
weeks.
So
it's
something
that's
made
that
relatively
painless
versus
actually
having
to
go
on
and
actually
manage
each
of
the
machines
themselves.
A
So
that's
how
we
managed
to
deal
with
all
large
number
of
machines.
The
next
thing
is,
we
definitely
have
to
be
nimble,
because
we
have
to
make
sure
that
we
can
get
through
the
bills
quickly.
It
seems
like
in
the
community.
The
tolerance
level
is
maybe
around
30
minutes.
So
if
we
keep
it
under
that,
we
can
actually
make
sure
that
everything
gets
tested
across
all
the
platforms
for
every
poll
request.
If
we
start
to
spend
further
than
that,
it's
going
to
be
hard
to
keep
that
up.
A
So
it's
important
that
we
can
do
things
quickly
enough
to
actually
meet
that
a
couple
things
that
we
do.
We
make
sure
that
we
have
multiple
machines,
so
at
least
two
machines
for
each
of
the
platforms
is
going
to
be
tested,
because
often
you
know
there's
more
than
one
build
going
on
and
that
lets
us
make
sure
that
you
know
you're
not
constantly
waiting
if
there's
multiple
things
that
are
trying
to
have
at
the
same
time,
the
next
one
is
see
cash,
that's
a
tool
that
basically
says
if
I've
already
compiled
it.
A
Let's
not
compile
it
again,
because
when
you
look
at
the
build
jobs,
a
good
portion
of
your
time
is
spent
actually
doing
the
compilation,
and
if
you
can
basically
figure
out
I
haven't
changed
very
much.
You
can
really
cut
that
down.
So
I
think
that
shaves
off
10,
maybe
15
minutes
from
some
of
the
platforms
so
see
cash
is
an
important
tool.
A
There
are
some
platforms
where
we
just
can't
cut
it
down
to
be
fast
enough,
so
things
like
arm,
we
just
can't
get
the
full
set
of
arm
tests
to
run
fast
enough
to
sort
of
fit
in
that
envelope.
In
that
case,
we
use
what
are
called
fan
jobs
where
we
basically
split
out
the
jobs
and
run
a
portion
of
the
tests
on
each
of
the
on
basically
a
different
machine
in
parallel
and
at
the
end,
bring
all
the
results
back
to
get
your
results.
A
So
that's
another
key
technique
that
we
use
I
think
mostly
forearm
and
windows
today
to
be
able
to
get
down.
You
know
get
all
the
tests
executed
in
time
and
then,
finally,
not
just
when
running
in
in
the
infrastructure,
but
also
when
people
are
running
it
on
their
desktop.
We
want
to
make
sure
that
we
can
run
as
fast
as
possible
and
take
advantage
of
all
the
hardware.
That's
there.
A
So
the
communities
work
really
hard
to
basically
separate
the
test
site
into
like
tests
that
can
be
run
in
parallel
versus
those
that
have
to
be
run
sequentially
because
they
use
resources.
You
can't
you
know,
share,
and
so
definitely
in
the
jobs
in
the
the
CEO
jobs.
We
take
as
much
advantage
as
we
can
setting
the
parallelism
to
Mac
to
match
how
many
CPUs
we
are
to
get
through
those
tests,
and
you
know,
put
all
together.
It
lets
us
get
through
well
under
that
30
minutes,
which
is
a
great
thing.
A
Another
interesting
thing
we
have
to
deal
with
in
the
build
workgroup
is
trust
delegation.
We
have
a
bunch
of
different
environments
and
we
want
to
give
different
people
collaborators
people
who
are
on
the
bill
group
access
to
the
various
piece
of
the
infrastructure.
But
it's
also,
you
know
very
important,
that
we
make
sure
that
we
sort
of
limit
who
can
access
that,
because
we
want
to
have
confidence
that
the
infrastructure
is
going
to
continue
to
operate
as
we
expect,
and
so
we
have
this
this
concept
of
trust
delegation.
A
We
have
three
different
levels:
infra
test
and
release,
and
we
manage
key
so
there's
keys
that
will
give
you
ssh
keys.
That
will
give
you
access
to
those
particular
sets
of
machines,
and
then
we
use
gb
g
PG,
along
with
a
github
repo,
where
we
have
the
key
stored
to
be
able
to
easily
give
people
access
to
the
right
areas
and
basically
give
them
the
access
to
the
trust
that
they
need.
A
So
I
don't
want
to
leave
you
with
the
feeling
that
it's
only
about
builds
the
and
you
know
generating
those
binaries
which
is
important,
but
the
bill
group
also
does
other
things.
So
we
maintain
the
website.
Obviously,
very
importantly,
people
the
place
the
place.
People
go
to
get
information
downloads,
all
that
kind
of
stuff,
the
benchmarking
website,
so
maintaining
the
jobs
that
run
the
benchmarks,
publishing
it
to
the
website,
benchmarking,
nodejs
org.
A
We
have
tools
like
machine
notifications.
Again,
we
got
a
lot
of
machines.
If
something
goes
down,
we
need
tools
to
help
notify
people,
so
we
can
go
fix
that
we
have
a
team
looking
at
build
bots.
Those
are
BOTS
and
github
to
help
us
automate
routine
tasks
so
that
you
know,
hopefully
we
can
be
more
efficient
if
a
PR
is
opened
and
it's
got
tests
in
the
title
automatically
as
a
test
tag
and
basically
tools
to
sort
of
make
our
lives
easier
and
take
the
routine
action
so
that
we
don't
need
to
do
them
ourselves.
A
We
have
tools
to
provide
our
sink
mirror
so
that
people
can
get
downloads
through
our
sink
as
well.
As
you
know,
we're
even
thinking
of
supporting
other
projects.
You
know
we
have
quite
a
nice
infrastructure
across
a
lot
of
platforms
and
we're
figuring
out.
You
know
how
might
we
support
other
projects
like
node,
serial
port,
where
we,
you
know
it's
important
to
the
ecosystem,
to
have
support
across
all
those
different
platforms
as
well
in
terms
of
machine
access.
We
understand
that
not
everybody
has
access
to
all
the
different
operating
systems
and
platforms
that
we
support.
A
So
there
is
a
process
for
giving
people
access
to
those
machines,
so
you've
just
submitted
your
PR.
One
of
the
collaborators
runs
the
test
jobs
and
you
find
it
passes
on
everything
except
for
Windows,
but
you
don't
have
any
windows,
ball
machines
or
you're
running
on
Mac.
You
don't
have
any
Linux
machines.
There
is
a
potential
way
to
get
you
access
to
some
of
the
build
machines,
so
you
can
investigate
and
help
fix
that
as
well.
So,
if
you
want
to
go,
you
can
go.
I've
got
the
link
there,
you
can
go
read
up
on.
A
You
know
the
process
for
making
that
happen
in
terms
of
current
challenges.
Os
X
is
always
a
bit
of
it
is
not
always,
but
is.
It
is
a
bit
of
a
challenge
for
us
in
that
you
know
sort
of
two
ways:
it's
harder
to
find
a
provider
who
provides
OSX
virtual
machines,
so
we're
still
working
that
through
that's
the
one
area.
You
know
we
don't
have
a
provider
or
enough
providers
to
have
good
coverage.
The
other
thing
is
that
OS
X,
you
know
apple,
doesn't
make
it
easy
to
find
older
versions.
A
We
still
support,
I,
think
back
to
10,
8,
and
so
actually
getting
you
know,
versions
of
the
OS
from
10-8
forward
is
not
very
easy.
So
even
if
somebody
gave
us
hardware,
it
would
actually
be
hope
to
continue
and
continue
to
do
that.
So
that's
one
area
we're
still
working
through
to
come
up
with
something
that
we're
really
happy
with
the
other
one
is
Jenkins.
Jenkins
is
a
large
java
application.
A
Our
community
is
really
JavaScript
focus,
so
we
don't
have
as
many
people
as
we'd,
like
necessarily
who
are
really
interested
in
working
with
the
java
application,
and
you
know
configuring
Jenkins,
making
everything
work.
Well,
so
it's
it
works
well
for
us,
but
that's
one
area
where
we
sometimes
have
a
little
bit
of
friction.
A
So,
just
before
I
close
out,
you
know
I
want
to
make
the
call
to
say
get
involved.
I
think
the
build
workgroup
is
a
great
way,
a
great
place
to
learn
about
some
things.
You
wouldn't
necessarily
have
to
be
able
to
do
otherwise.
You
know
if
you
want
to
at
some
point
in
the
future,
build
your
own
infrastructure
to
support
a
good
sized
project
business
or
whatever
it's
going
to
give
you
the
opportunity
to
learn
about
a
lot
of
interesting
things
like
the
hardware.
The
operating
systems.
Ansible
is
a
tool.
A
You
know,
how
do
you
configure
your
machines
I,
even
things
like
OpenStack,
which
is
what's
often
used
to
configure
a
lot
of
the
the
virtual
machines
in
those
providers
that
we
have?
If
you
can't
don't
want
to
get
involved
in
time,
you
can
also
consider
just
you
know,
donations.
You
know,
make
a
small
donation.
If
you
already
have
a
Raspberry
Pi
that
you
bought
and
in
your
basement
never
use
it,
you
can
actually
ship
it
to
rod
bag.
You'll.
A
Add
it
to
that
night
that
that
nice
pie
cluster
that
we
saw
before
if
you
do
want
to
get
involved.
Most
of
the
work
is
done
through
github,
so
you
can
go
through.
Look
at
the
issue
sees,
would
see
what's
going
on
and
there's
also
an
IRC
channel
that
you
can
join
and
you
know
sort
of
participate
if
you
want
as
well.
So
that's
everything
I
had
to
they
want
to
tell
you
today
and
I
think
I'm
just
about
on
time.
So
thank
you
very
much
for
listening.