►
From YouTube: Infrastructure Orchestration from the Ground Up
Description
Slides from the workshop: https://docs.google.com/presentation/d/1VJ09Bz4NQ6ud3XIH_0RMWrC3THqt1A-iZhoKgjo-FN8/edit#slide=id.p
The Graph is an indexing protocol for querying networks like Ethereum and IPFS. Anyone can build and publish open APIs, called subgraphs, making data easily accessible.
Follow The Graph on social media
Twitter: https://twitter.com/graphprotocol?s=20
Instagram: https://instagram.com/graphprotocol
LinkedIn: https://www.linkedin.com/company/thegraph/
GitHub: https://github.com/graphprotocol
Website: https://thegraph.com
A
A
A
So
on
tuesday
we
have
talked
about
indexer,
best
practices
and
most
of
those
work
to
do
with
software
configuration
and
specifics
of
the
application
stack.
So
in
today's
workshop
we'll
try
to
cover
all
aspects
of
setting
up
your
indexer
from
picking
hardware
to
spin
it
spinning
it
to
picking
storage
all
the
way
to
day-to-day
operations.
A
So
the
the
structure
of
what
you
should
think
about
when
navigating
a
sea
of
options
in
a
very
complex
environment
will
be
our
agenda,
so
we
will
start
with
a
very
quick
review
of
the
indexer
software
stack.
This
is
something
you've
seen
before.
We'll
just
remind
you
so
that
you
know
what
we're
talking
about,
then
we
will
talk
a
bit
about
hardware
and
the
decisions
to
make
and
how
to
pick
that
hardware,
then
we'll
talk
about
understanding
the
different
types
of
blockchain
nodes.
A
They
will
cover
the
importance
of
storage
when
we're
talking
about
indexing
and
then
we'll
move
on
to
orchestra
orchestration.
So
you
have
picked
your
hardware
and
your
storage,
and
now
you
need
to
make
things
happen.
So
here
we
will
talk
about
declarative
infrastructure
versus
imperative
infrastructure.
A
The
index
node
would
then
store
index
data
into
a
postgres
sql
subgraph
database.
At
this
point,
the
subgraph
data
is
indexed
and
available
for
querying.
So
so
that's
the
detail
and
in
the
inductor
infrastructure
block
of
this
diagram,
you
can
also
see
the
different
subs
software
components
that
you
would
be
expected
to
run.
So
now,
if
we
go
to
the
next
diagram
to
the
next
slide,
please
this
slide
shows
which
of
those
software
components
are
stateful,
so
require
persistence,
and
this
would
be
the
blockchain
node,
the
postclass
sql.
A
A
A
Then,
in
order
to
scale
to
be
able
to
maintain
all
of
those
child
chains,
you
would
need
the
stateless
pieces
of
of
your
application
so
graph
node
for
the
index,
node
graph
node
for
the
query,
node
and
the
indexer
service.
You
need
to
scale
those
as
well
to
sustain
the
capacity
of
multiple
chains,
and
you
would
ideally
scale
them
horizontally
next
slide.
Please.
A
A
A
Okay!
So
here
we
have
a
diagram
of
the
hardware
with
initial
versus
ongoing
cost.
So
if
you
decide
to
pick
a
self-owned
bare
metal
servers,
those
have
a
really
high
capex,
because
you
buy
them
upfront,
they
will
cost
you
a
certain
amount
of
money
and
then,
every
month
from
there
on
your
cost
will
only
be
things
like
electricity
and
rent.
If
you
pay
rent
where
you
are
and
stuff
like
that,
so
very
high
capex
initial
cost
and
very
low
opex,
so
ongoing
monthly
cost.
A
A
I
think
someone
was
pointing
out
to
vulture
or
something
like
that
just
yesterday,
so
there
are
many
options
out
there.
So
for
that
option
for
bare
metal
cloud
where
you
are
just
granting
servers
on
a
monthly
basis,
you
have
no
capex
to
begin
with,
but
your
opex
is
will
be
fairly
high,
a
lot
a
lot
higher
than
than
the
opex
for
cellphone
bare
metal,
and
we
specifically
didn't
put
any
numbers
here,
because
the
cost
variables
depend
on
provider
depend
on
the
location.
A
A
So,
let's
talk
about
managed
public
cloud,
so
this
is
your
cla.
Your
gcloud
is
your
aws
and
so
on.
So
some
of
the
pros
of
managed
public
cloud
is
that
you're
not
exposed
to
hardware
failure.
You
don't
have
to
worry.
If
your
server
fails,
they
usually
are
set
up
in
high
of
highly
available
ways
so
that,
even
if
there
is
a
failure,
you
don't
have
to
fix
it.
Someone
else
is
fixing
it
for
you
and
it
resolves
fairly
fast,
then
usually
managed
public
clouds
have
amazing
seamless
user
experience.
A
A
A
Some
of
the
cops
some
of
the
cons
are
limited
capabilities
and
flexibilities
because
they
are
managed
servers.
You
don't
have
the
same
level
of
customizability
as
you
would
with
say
a
self
hosted
or
a
rented
server
in
another
provider,
and
then
they
also
have
very
high
opex,
so
they
tend
to
cost
quite
a
bit
more
than
than
the
the
other
option,
which
is
the
the
renting
servers.
A
A
A
Again
they
have
no
initial,
no
capex
and
low
low
opex.
The
the
pro
the
opex
depends
from
provider
to
provider.
They
have
remote
control
over
the
physical
hardware.
So
while
you
cannot
go
and
turn
on
and
off
the
server,
you
do
have
remote
control
over
it
via
ssh
or
other
ways.
They
are
fairly
easy
to
provision
not
as
easy
to
provision
as
the
public
cloud,
but.
A
And
better
value
for
money
and
in
terms
of
some
of
the
cons,
they
have
a
higher
maintenance
than
public
cloud
you're
in
charge
of
the
security,
and
you
are
exposed
to
risks
of
hardware
failure.
Let's
go
to
the
next
slide.
Please,
okay,
and
the
last
option
here
in
terms
of
picking
hardware,
is
self-hosted
servers,
so
the
self-hosted
servers
are
highly
customizable.
A
B
A
C
No,
no,
I
think
you've
covered
it.
Well,
sorry
for
distracting
you
with
all
the
chat
messages.
I
I
just
mentioned
that
there
is
one
more
option
which
is
co-location,
which
is
kind
of
like
a
combination
between
this
slide
self-hosted
servers
and
bare
metal
cloud
where
you
basically
buy
the
server,
but
then
you
just
rent
space
in
some
third-party
data
center
to
host
them,
so
you
buy
the
server
but
rent
everything
else.
The
power,
the
space,
the
networking
everything.
A
A
Okay,
so
now
that
we
spoke
a
bit
about
hardware,
let's
talk
about
understanding
the
different
blockchain
nodes,
because
there
can
be
some
confusion
there
and
yeah
it's
worth
quickly.
Talking
about
that,
so
keep
in
mind
this
diagram.
This
is
where
the
the
blockchain
node
is
so
the
border
blockchain,
node
and
next
slide.
Please.
A
A
Then
we
have
a
full
node
which
keeps
all
log
history
but
prunes
everything
else
that
this
historical
state
and
other
data,
so
you
can
use
a
full
node
to
send
transaction
and
to
sub
graph
and
to
index
subgraphs
that
only
consume
events
and
don't
need
to
access
historical
data.
Then
we
have
archival
nodes,
the
archival
nodes.
Let
you
send
transaction,
let
you
sell
index
sub
graphs
that
only
consume
events,
but
also
index
sub
graphs
that
query
chain
state,
so
they
keep
everything
a
full
node
would
keep
plus
historical
state.
D
A
C
Yeah
I
mean
the
bare
minimum
to
be
able
to
index
a
subgraph
will
be
a
full
node,
but
if
you
would
like
to
kind
of
seriously
compete,
you
probably
want
to
be
able
to
index.
You
know
whatever
sub
graphs.
Have
lots
of
curation
or
you
know,
are
receiving
many
queries
and
in
that
instance,
you'll
want
an
archival
trace,
node
to
be
able
to
index
everything.
A
A
A
With
the
caveat
of
you,
you
still
need
to
run
the
other
components,
so
you're
buying
a
host
rpc
service
to
replace
your
blockchain
node.
In
that
diagram
that
we
showed
earlier,
you
don't
have
to
worry
about
the
security
of
the
blockchain
node,
another
pro
in
terms
of
cons,
very,
very
high
opex,
so
the
monthly
operating
sys
of
operating
expense
is
high.
A
A
C
C
C
C
Similarly,
when
subgraphs
are
indexing,
you
know
they're,
constantly
writing
to
the
database
and,
at
the
same
time,
you're
issuing
queries
against
the
database.
You
know,
as
as
queries
come
in
from
the
gateway,
and
so
again
this
is
a
an
area
and-
and
just
to
recap,
I'm
talking
now
about
the
postgres
db
storage
that
actually
holds
subgraph
data.
C
This
is
an
area
where
high
performance
storage
has
has
a
high
impact,
so
it
allows
you
to
both
index
subgraphs
faster,
but
also
answer
queries
faster.
So
that's
that's
the
physical
media.
I
know
I
see.
Xfs
is
mentioned
in
the
chat.
It's
a
file
system
that
I've
personally
never
used.
Please
check
it
out
and
actually
yeah,
I'm
hesitant
to
try
saying
your
name
for
fear
of
butchering
it.
C
But
if
you
are
open
to
saying
a
few
words
about
xfs,
please
do
I'll
just
get
to
the
end
of
the
next
section
so
other
than
physical
media.
The
the
other
thing
is
like
how
you
actually
use
these
devices,
particularly
for
some
of
these
larger
chains.
So
I
think
gnosis
chain,
for
example,
a
full
archival
trace
node.
C
Lvm
and
zfs
both
offer
quality
of
life
features
like
snapshotting.
That
are
definitely
worthwhile,
and
you
know,
particularly
when
it
comes
to
just
the
operational
tasks
associated
with
your
indexer,
like
taking
backups
or
moving
nodes
between
machines
or
things
like
that.
Having
those
features
really
does
make
a
difference,
and
yes,
we
do.
C
A
C
B
B
Well,
you
know
the
one
thing
I
think
that's
interesting
about
zfs.
Is
that
all
the
features
that
come
with
it
right?
I
think
you
mentioned
that
chris,
but
it
really
can't
be
understated
how
how
often
the
features
of
gfs
like
snapshotting
and
the
fact
that
it's
a
copy
on
right
operating
system.
So
if
things
go
really
wrong
and
you
get
like
a
a
sig
interest
or
some
kind
of
really
awful
thing
happen
on
your
server,
then
zfs
generally
has
your
back
because
it's
a
copy
on
right
operating
system.
B
So
if
you
were
to
lose
power
immediately,
then
the
likelihood
of
you
actually
corrupting
something
in
the
office
and
it
not
auto
correcting,
is
super
super
low
and
then,
of
course,
snapshotting
and
compression
compression
is
free.
Space
and
free
performance
on
zfs,
just
enabling
compression
can
improve
even
with
uncompressible
data
can
improve
your.
You
know,
performance
biops
quite
considerably,
but
as
with
all
things
I
mean,
zfs
is
not
magical
right.
B
B
B
How
much
of
your
hardware,
your
very
expensive
micron!
You
know
nvme
u.3
ssds,
that
you
paid
huge
amounts
of
money
for
or
huge
amounts
of
to,
rent.
How
many
of
them
do
you
want
to
sacrifice
in
the
you
know,
for
the
sake
of
making
sure
that
you
have
a
very
resilient
zfs
pool,
you
know,
do
you
want
to
do?
Do
you
want
to
mirror
your
drive,
so
you
have
an
exact
copy
or
multiple
exact
copies?
B
Do
you
want
to
do
raid
z2,
where
maybe
you
can
afford
to
have
two
drives
in
in
part
of
the
pool
fail?
Or
do
you
want
to
just
you
know,
go
balls
to
the
wall
and
forget
about
all
of
that
and
just
go
for
pure
performance,
and
you
know,
put
loads
of
drives
in
parallel
and
really
to
you
know,
take
a
real
serious
hit
in
your
wallet.
B
Don't
don't
go
straight
into
production
with
what
you
think
is
going
to
work
because
zfs
has
got
you
know
you
can
you
can
organize
things
in
a
near
infinite?
Number
of
ways
same
with
any
any
sort
of
low
level
file
system,
but
don't
go
don't
go
in
thinking
that
your
theory
or
your
fio
is
the
tool
we
use
often
for
for,
for
benchmarking.
Don't
don't
just
believe
what
the
tools
telling
you
do.
C
Yeah,
thank
you.
Thank
you.
So
much
for
sharing
jim
some
great
stuff
in
there
a
couple
of
questions
in
the
chat.
At
least
some
of
them
have
been
answered
in
the
chat
but
christopher
from
lemonade
said,
wouldn't
compression
decrease,
throughput
and
thus
negatively
affect
performance.
It's
really
cool
to
see
how
zfs
impacts,
workloads
and
zfs
with
postgres
in
particular.
C
If
your
queries
are
are
reading.
Large
amounts
of
data
can
actually
significantly
increase
your
query
performance
because,
even
though
you
know
you're,
obviously
compression
is
not
free.
You,
you
have
some
cpu
trade-off
that
you're
making
there,
but
if
you're
pulling
large
amounts
of
data
off
of
the
disk,
you
can
actually
get
more
of
a
performance
benefit
from
compression,
allowing
you
to
read
and
write
less
data
to
the
disk
than
the
trade-off
you
make
with
cpu
so
yeah.
C
There
are
many
configurations
and,
as
as
is
being
said
in
the
chat
chat,
often
with
zfs,
you
kind
of
want
to
tune
it
for
each
workload.
If
you,
you
know,
if
you
kind
of
want
to
eke
out
maximum
performance,
one
last
thing
I'll
say
is
that
you
know
people
make
a
career
out
of
managing
zfs
and
indeed
many
of
the
components
in
this
stack.
You
don't
need
to
run
some
crazy,
optimized
zfs
setup
in
order
to
be
a
successful
indexer.
C
There
are
many
indexers
that
operate
well,
just
with
a
bug,
standard,
ext4
file
system.
So
yeah
I
mean
one
of
the
things.
That's
fantastic
about
this
ecosystem
is
there's
just
so
much
room
for,
like
technical
growth,
don't
feel
that
you
need
to
go
all
in.
You
know
in
every
layer
from
day
one
it's
something
that
you
can
kind
of
level
up
on,
as
as
you
get
more
comfortable
with
the
stack
okay,
if
anyone
has
anything
else
to
say
about
storage,
please
say
it
now,
and
otherwise
I
will
hand
back
down.
A
G
C
It's
it's
it's
hard
to
answer
precisely.
I
think
there
are.
There
are
aspects
of
the
workload
that
are
consistent
and
then
there
are
aspects
of
the
workload
that
are
highly
variable.
If
you're
indexing
a
large
number
of
subgraphs,
then
you
know
with
each
block
that
comes
in
your
graph.
Node
is
consuming
and
transforming
data
and
saving
it
into
the
database.
So
there's
going
to
be
some
consistent,
write
workload
that
scales,
you
know
roughly
linearly
to
the
number
of
subgraphs
that
you
index
the
side.
C
That
is
less
easy
to
answer
is
the
query
side,
and
you
know
that
is
not
just
a
a
matter
of
like
what
demand
exists
in
the
market
for
different
types
of
data,
but
even
right
you
can
express
different
preferences
to
the
market,
about
which
queries
you
serve
using
cost
models.
So,
for
example,
if
you
wanted
to
really
optimize
for
serving
queries
that
are
otherwise
you
know
massive
and
super
expensive
for
other
indexers
to
run,
maybe
they
have
to
scan
over
very
large
amounts
of
sequential
data.
C
A
A
A
So
here
we
talk
about
infrastructures
code,
I'm
sure
many
of
you
have
heard
this
term,
but
basically,
what
it
means
is
instead
of
going
around
and
clicking
or
running
one
command
ad
hoc
somewhere
to
set
something
up.
You
have
everything
written
in
code,
whether
that's
yamo,
using
something
like
ansible
or
whether
that's
terraform
using
hcl,
is
what
they
use
or
even
bash
scripts
it.
It
would
still
be
infrastructures
code
as
long
as
you
have
a
repeatable
way
of
recreating
the
same
thing.
A
A
A
As
chris
said,
we
strongly
recommend
the
approaches
of
version
control,
declarative
infrastructure
as
code.
Examples
of
tools
that
are
declarative
infrastructure
is
code
terraform,
which
maintains
a
state
and
is
very
widely
used.
I
think
it's
the
preferred
the
preferred
tool
for
many
sres
when
it
comes
to
provisioning
infrastructure,
there's
cloud
formation,
which
is
an
aws
product.
A
As
chris
said,
launchpad,
which
is
our
kubernetes
toolkit
for
indexers,
is
built
on
ansible,
kubernetes,
helm
and
help
file,
which
are
all
declarative
infrastructures
code
and
in
case
you're
wondering
why
ansible
and
not
something
else
is
because
we're
only
using
the
ansible
part
for
installing
the
different
for
for
installing
kubernetes
between
your
so
between
your
different
nodes
and
a
few
other
initial
setup.
Once
you've
installed
the
operating
system,
there
will
be
an
a
link
in
the
next
slide
for
launchpad
in
the
next
few
slides.
A
Okay,
next
slide,
please.
So
I
wanted
to
quickly
address
the
whole
docker
versus
kubernetes
debate
and
there's
many
ways
to
address
this.
So
first
docker
is
a
suite
of
software
development
tools
for
creating
sharing
and
running
individual
containers
where
containers
are
standardized
packages
for
microservices
applications
with
all
their
needed
dependencies
inside,
whereas
kubernetes
is
the
system
for
operating
containerized
applications
at
scale
where
an
application
consists
of
many
containers
and
to
orchestrate
them
in
a
production
environment
to
orchestrate
them
over
many
servers,
many
data
centers
you'd
use
kubernetes.
Now.
A
What
I
want
to
specify
here
is
that
docker
has
been
used
recently
has
become
the
thing
when
you
talk
about
containers.
So
if
you
think
in
our
google
example,
we
don't
say
I'm
going
to
do
internet
search,
you
say
I'm
going
to
google
this.
That
is
the
equivalent
of
what
people
mean
when
they
talk
about
docker.
They
just
mean
containers,
but
the
reality
is
that
there's
many
tools
for
docker,
so
there's
docker
for
mac,
there's
docker
compose,
there's
docker
swarm,
there's
docker
as
a
container
runtime
and
so
on.
A
So
going
back
to
docker
versus
kubernetes,
I
would
say
that
a
fair
comparison
would
be
docker
swarm
versus
kubernetes,
where
docker
is
also
docker.
Swarm
is
also
a
system
for
operating
containerized
application
at
scale,
as
chris
is
saying,
is
key
difference
is
that
docker
doesn't
provide
any
host
abstraction
clustering,
but
kubernetes
dance,
and
here
yeah
docker
storm
also
does
provide
host.
A
Host
abstraction,
but
it
has
been
deprecated
and
there
is
also
docker
compose
that
also
doesn't
provide
clustering.
Okay
next
slide
so,
and
maybe
this
would
have
been
a
better
way
if
I
would
have
switched
the
slides
around.
But
here
we
are
so
we
talked
about
containers,
but
let's
quickly
touch
on
self
compile,
so
your
options
are
running
containers
which
run
based
on
an
image
which
is
a
specification
of
what
you're
going
to
install
and
how,
in
that
container
or
self-compiling
applications.
A
A
A
However,
the
limitations
of
the
docker
compose
setup
are
that
you
can't
scale
that
beyond
beyond
one
host
one
server.
So
when
it
comes
to
supporting
multiple
chains,
you
can
only
get
so
far
then
soon
we'll
be
revealing
the
kubernetes
launchpad
from
graphops,
which
there's
a
lot
of
work
that
has
gone
into
that.
The
purpose
of
it
is
to
take
you
from
you,
give
it
an
ip
address
and
the
way
of
connecting
to
your
servers
and-
and
it
takes
those
servers,
creates
clusters
for
you.
A
A
A
C
E
Sure
thing
yeah,
I
I
I
I'd
say
we
should
keep
the
recording
in
the
faq.
So
like
I've
I
mean
I
think
people
are
feel
free
to
drop
off.
If
you,
if
you're
busy
but
yeah,
I
think
it's
it's
valuable,
so
we'll
keep
the
recording
the
faq,
so
yeah
so
keep
going,
but
obviously
try
not
to
keep
it
for
too
long.
D
C
F
A
F
A
E
C
I'll
grab
that
hey
dan
hoping
your
name
is
dan,
it's
not
a
huge
stretch
right.
So
the
question
is
as
a
newbie,
I'm
confused
as
to
whether
the
intention
is
for
missed
participants
to
run
through
every
chain
that
is
specified
in
the
program
or
if
you
think
you
can
only
support
one
chain.
You
should
just
pick
one
and
forget
the
rest,
so
you
can
participate
in
whichever
chain
phases
you
would
like
to.
C
As
far
as
I
understand
so
I'm
sure
this
won't
be
new
information,
but
through
the
mips
program
there
will
be
a
number
of
phases,
and
you
know
each
chain
you
can
kind
of
think
of.
Has
its
own
phase
and
a
phase
has
a
cycle
that
starts
at
test
net
and
so
mips
participants
are
expected
to.
You
know,
participate
in
test
net
and
meet
mission
and
qos
objectives
in
testnet,
then
one
once
testnet
qos
gets
to
qos's
quality
of
service
for
anyone
unsure
once
testnet
qos
gets
to
a
certain
threshold.
C
C
Follow-Up
questions
from
dan,
so
I
might
not
be
ready
for
nurses
chain,
but
I
can
start
getting
things
ready
in
order
to
jump
on
a
future
phase.
Yes,
that's
correct!
So
long
as
you,
you
know
applied
before
the
deadline
and
submitted
all
your
kyc,
but
yeah
you.
You
can
pick
the
chains
that
you
would
like
to
participate
in.
A
C
Maybe
just
to
clarify
the
node
client
is
nethermind,
but
you
can
run
nethermind
in
those
nodes
in
those
modes
that
were
covered
so
as
a
light
client
as
a
full
client
as
an
archive
etc.
So
you
want
to
run
ansible
for
nurses
chain
and
if
you
would
like
to
support
any
subgraph,
you
need
to
run
it
in
archival
trace
mode
which
yeah
guess.
I'm
sure
there
is
documentation
on
that
and
we
will
actually
cover
this
in
our
launchpad
workshop.
A
B
Someone
the
other
day
said
that
they
were
uploading
a
snapshot.
I
think
it
was
sun
tzu.
I
don't
know
if
he's
on
the
call,
but
maybe
one
of
the
community
members
might
be
providing
a
snapshot
we'll
see.
There
was
also
some
talk
about,
I
think
hypnosis
providing
them,
but
whether
that
actually
came
to
to
be
or
not,
I'm
not
sure.
C
Yeah,
the
there
are
a
few
efforts
to
organize
snapshots.
Honestly,
one
of
the
biggest
challenges
with
snaps
snapshots
isn't
so
much
making
them
it's
distributing
them
because,
yes,
as
state
machines,
said
a
1.7
terabyte
snapshots,
just
a
lot
of
bandwidth
to
pay
for,
if
you
know,
100
150
500
people
are
downloading.
It.
C
A
C
C
Also,
parts
of
of
asia
also
generate
a
lot
of
demand,
so
you
can
kind
of
just
think
about
it.
From
that
perspective,
it's
also
worth
saying
that
graphops,
in
collaboration
with
edge
and
node,
is
working
on
a
gateway
quality
of
service
oracle,
and
this
will,
you
know,
give
us
a
bit
more
quantitative
data.
All
of
us
to
understand.
A
A
B
E
I
think
so
too,
so
I
think
it
might
be
a
good
opportunity
to
wrap
up.
Thank
you,
everyone
for
all
of
your
participation.
This
has
certainly
been
a
very
educational
and
informative
conversation,
so
yeah.
Thank
you
anna.
Thank
you,
chris.
Thank
you
to
everyone.
This
has
been
an
absolute
pleasure
and
we
look
forward
to
seeing
you
all
on
tuesday
for
the
next
ioh
slash
mip
workshop.
Thank
you
all.
Take
care.