►
From YouTube: HybridCluster
Description
No description was provided for this meeting.
If this is YOUR meeting, an easy way to fix this is to add a description to your video, wherever mtngs.io found it (probably YouTube).
A
Luke
I
was
going
to
introduce
now
luke
marsden.
He.
B
B
Cool,
so
I
guess
this
is
just
a
sort
of
30
000
foot
view
of
some
of
the
features
in
zfs
that
we
use,
and
a
little
bit
about
the
sort
of
architecture
that
that
we've
been
working
on
for
for
some
time
is,
is
kind
of
different
to
the
sort
of
big
shared
storage
box
approach.
B
B
Cool
okay
and
it
works
good
cool,
yeah,
so
yeah,
I
guess
just
before
I
dive
into
how
specifically
we
use
the
features
of
zfs
a
little
bit
on
actually
what
the
sort
of
shape
of
the
architecture
that
that
we've
been
building
looks
like,
as
I
was
saying,
it's
a
little
bit
different
to
the
sort
of
approach
of
a
a
big
shared
storage
box
that
exports
data
over
nfs
or
iscsi.
It's
it's
more,
an
approach
of
unifying
the
compute
and
the
data
nodes
into
the
same
place.
B
So
if
these
are
two
nodes
in
a
cluster
for
example,
then
each
node
has
its
own
built-in
storage
pool,
and
then
we've
got
a
container-based
virtualization
layer
that
sits
on
top
of
the
storage
pool
and
we've
got
a
proxy
layer
that
sits
on
top
of
that
handling
any
incoming
requests,
and
this
can
all
run
on
sort
of
commodity
hardware.
It
can
run
on
top
of
cloud
infrastructure
and
it
sort
of
opens
up
some
some
interesting
possibilities,
but
so
yeah
I
mean
it's.
B
The
approach
is
is
really
to
having
a
multi-container
system
that
can
be
split
across
multiple
nodes
running
in
multiple
data
centers,
and
one
of
the
most
important
use
cases
that
that
we
use
so
we
lean
on
features
of
zfs4
is
is
effectively
backup
and
what
I
mean
by
that
is
snapshot,
and
then
I
use
zfs
destroyed
to
refer
to
pruning
of
snapshots.
B
So
what
our
system
does
is
it's
so
just
to
give
a
little
bit
more
context?
First,
actually
so
each
one
of
these
containers
that
might
be
running
some
web
application
or
database,
for
example,
will
be
backed
onto
its
own
independence,
lfs
file
system,
and
that
file
system
will
then
be
independently
snapshotted
and
replicated
out
between
the
different
nodes
in
the
cluster.
B
So
from
a
sort
of
implementing
backup
perspective,
we
are
using
continuous
snapshots.
So
whenever
a
piece
of
data
changes
on
a
user's
file
system,
we're
taking
a
new
snapshot
very
quickly
and
having
those
snapshots
available
is
a
useful
way
of
allowing
customers
to
roll
back
to
previous
versions,
and-
and
so
that's
a
typical
backup
use
case.
We
also
prune
those
snapshots
so
we're
automatically
taking
the
snapshots
and
we're
automatically
deleting
old
ones.
So
we
keep,
for
example,
the
last
hour's
worth
of
snapshots
down
to
30
second
resolution.
B
Before
that
we
keep
the
last
day's
worth
of
snapshots
on
an
hourly
resolution
and
before
that
daily
resolution,
and
so
on.
We
also
do
replication
across
multiple
machines,
and
this
is
where
zfs
send
and
receive
pipelines
come
in
really
really
handy.
So
whenever
one
of
those
snapshots
gets
taken,
the
system
will
automatically
it
automatically
sort
of
ensures
that
there
are
slaves
allocated
for
each
container,
that's
in
a
master
mode
on
a
on
a
server,
and
so
it
will.
B
Whenever
it
takes
a
snapshot,
it
will
replicate
that
snapshot
to
whatever
the
configured
number
of
other
slaves
is
that
could
be
in
the
same
data
center
or
in
a
different
data
center.
But
matt's
invention
of
zfs
send
and
receive,
is
absolutely
critical
to
us
being
able
to
do
this,
and
it
allows
us
to
do
near
real-time
replication
across
across
data
centers
and
that's
really
cool
and
then
the
last
feature
is
I
sort
of
already.
B
This
is
sort
of
tied
into
the
the
backup
piece,
but
actually
giving
users
the
ability
to
do
a
rollback
themselves
exposes
this
sort
of
cool
time
machine
don't
tell
apple.
I
said
that
feature
that
lets
you
if
your
website
gets
hacked.
For
example,
we
often
deploy
this
in
a
sort
of
web
hosting
scenario.
So
if
your
website
gets
hacked,
then
you
can
roll
back
to
before
it
got
hacked
and
apply
your
patches
or,
if
you
accidentally
drop
a
table
in
your
database.
B
You
can
just
roll
back
to
30
seconds
before
that
and
then
there's
some
other
sort
of
neat
things
that
we've
built
on
top
of
that
the
ability,
if
seeing
as
this
data,
is
being
continuously
snapshotted,
actually
just
sort
of
on
this
replication
piece.
B
What
I
think
is
really
quite
neat
about
the
approach
of
replicating
sets
of
snapshots
around
between
different
machines
is
that,
rather
than
a
classical
backup
architecture
which,
where
you,
where
you're
sending
your
backups
off
to
some
backup
server
and
then,
if
you
want
to
recover
your
data
from
that
backup
server,
you
have
to
drag
the
data
back.
B
The
architecture
that
that
we've
got
here
instead
allows
every
replica
to
both
be
a
user-facing
backup
and
a
hot
spare
at
the
same
time,
and
I
think
that's,
I
just
think
that
having
replicated
snapshot
trees
is
is
really
the
way
forwards
for
for
all
sorts
of
use
cases.
B
So
then
yeah,
because
we're
doing
this
replication
of
data
around
between
different
nodes.
We
can
also
fail
over
fail
over
to
a
very
recent
backup,
so
you
can
specify
a
threshold
to
say
if
I
lose
a
node,
I
want
to
recover
my
to
the
latest
backup
within
a
certain
threshold
or
that
failover
can
be
initiated
manually,
and
we
can
also-
and
that's
useful,
so
you,
if
a
server
fails,
you
can
recover
all
the
applications
that
were
on
it
to
another
server
in
the
same
dc
or
if
a
data
center
fails,
then.
B
Similarly,
you
can
fail
over
all
the
applications
that
are
on
that
server
to
another
data
center.
Then
the
live
migration
is
sort
of
fun
and
I'll
talk
about
this
in
more
detail.
In
the
other
talk
I'm
doing,
but
it's
basically
the
ability
to
with
our
proxy
layer
seamlessly
migrate
applications
around
between
different
servers
and
different
storage
pools
and
different
clouds.
So
that's
it
really
in
terms
of
the
platforms
that
we
depend
on
from
from
an
open,
zfs
perspective,
we've
built
our
platform.
B
The
initial
version
of
our
product
is
built
on
freebsd
and
it's
built
as
a
of
classical
web
hosting
stack.
But
with
some
of
these
neat
distributed
systems,
features
built
in
freebsd
is
an
excellent
base.
B
It's
been
very
stable
and
we've
done
quite
a
bit
of
work
to
improve
sort
of
festivity
on
freebsd,
we're
now
looking
at
moving
to
linux
because
of
market
forces
and
because
of
an
interest
in
containerization
and
importing
some
of
the
sort
of
storage
stuff
we're
doing
to
to
that
context,
and
hence
the
interest
now
in
zfs
on
linux,
and
we
hope
to
sort
of
reproduce
some
of
the
success
we've
had
making
zfs
on
freebsd
more
stable
with
linux
as
well
and
yeah.
So
I
just
a
general
point.
B
I
think
that
opens
edfs
has
huge
potential,
especially
in
the
sort
of
upcoming
cloud
infrastructure
and
containerization
world,
and
I'm
really
excited
to
to
see
such
good
forward
progress
on
it.
So
thanks
any
questions.
A
B
Yeah,
it's
great
question.
I
mean
the
the
short
answer.
Is
it's
configurable
and
it's
a
trade-off
between
like
disc
io
and
network
late
network
versus
your
replication
latency,
but
in
typical
configurations
we
see
a
replication
latency
around
30
seconds
and
obviously
it
depends
on
your
application
as
to
whether
that's
acceptable
for
automatic
failover
or
if
you
just
want
to
do
manual
failover
in
that
sort
of
scenario.
B
I
have
this
ongoing
curiosity
with
the
I
with
the
concept
of
doing
synchronous
at
fs
replication,
but
that's
not
something.
We
currently
have
resources
to
do.
C
In
your
practical
experience,
how
how
do
application
failover,
how
the
applications
behave
when
you
fail
over
and
there's
there
are
pending
connections
that
will
be
routed
by
the
proxy.
Do
they
behave
well?
How
about
they
have?
They
might
have
stale
data
30
seconds
ago,
and
the
connections
might
be
from
an
already
established
communication
and
on
the
other
side,
you
don't
have
the
data
that's
been
passed.
B
Yeah,
so
it's
a
good
question
I
mean
so
generally.
Most
applications
will
perform
pretty
well
out
of
the
box
when
you
take
a
consistent,
zfs
snapshot
of
an
application
and
its
file
system,
well
an
applications
file
system
and
then
recover
it
somewhere
else.
It's
just
like
that.
Server
lost
power
and
came
back
up.
B
So
if
you've
got
a
decent
database,
then
it
will
recover
automatically
pretty
quickly
if
you're
running
my
isom,
then
maybe
you're,
not
in
luck,
which
is
why
we
also
have
hooks
into
my
sequel
in
particular
to
do
a
flush
tables
with
redlock
operation
for
my
isom
tables,
but
yeah.
The
other
part
of
the
question
was
sorry.
What
was
the
other
part
of
the
question
around.
B
Yeah,
you
are
asking
also
about
connections
and
routing
as
well
and
yeah
I
mean
so.
The
application
behavior
is
is
generally
fine,
like
I
say
it's
as
if
it
came
back
after
power
failure.
The
the
question
of
sort
of
routing
connections
around.
B
The
approach
that
we're
taking
is
not
to
try
and
get
like
completely
100
seamless
failover,
as
you
can
tell
it's
more
of
a
sort
of
people
use.
The
word
cloudy
in
a
derogatory
sense
to
say
things
that,
like
aren't
like
really
really
hardcore
up
to
the
millisecond,
and
that
certainly
like
limits
some
of
the
applicability
of
this
approach.
But
I
think
that
there's
still
a
large
class
of
applications
for
which
it's
acceptable.
B
But
we
will
like
we'll
lose
in-flight
connections
when
a
server
fails,
for
example,
and
we'll
have
a
brief
period
of
downtime,
because
there's
a
timeout
before
which
we
initiate
a
failover.
So
if
someone's
doing
a
long-running
download,
then
they'll
have
to
restart
it
or
user
might
need
to
reload
the
page
or
something.
C
B
B
A
proxy
layer
will
also
publish
dns
records
and
it'll
publish
multiple,
a
records
for
each
thing,
so
you
can
even
have
like
multiple
a
records
in
different
data
centers
and
then,
if
one
of
the
nodes
fails
and
just
completely
goes
offline,
then
the
browser
will
retry
the
other
ip
address
and
most
well-behaved
clients
will
will
retry
another
ip.
B
So
in
response
to
like
what
what's
the
purpose
of
the
proxy
there's,
also,
the
proxy's
also
used
in
live
migration
and
I'll
go
over
that
in
the
next
deep
dive.
B
B
Cool,
so
thanks
very
much
luke
no
worries
I
was
gonna
get
back
to
then
just
talking
about
another.