►
From YouTube: Data Without Borders - Using Rook Storage Orchestration at a Global Scale - Jared Watts, Upbound
Description
Join us for Kubernetes Forums Seoul, Sydney, Bengaluru and Delhi - learn more at kubecon.io
Don't miss KubeCon + CloudNativeCon 2020 events in Amsterdam March 30 - April 2, Shanghai July 28-30 and Boston November 17-20! Learn more at kubecon.io. The conference features presentations from developers and end users of Kubernetes, Prometheus, Envoy, and all of the other CNCF-hosted projects
Data Without Borders - Using Rook Storage Orchestration at a Global Scale - Jared Watts, Upbound
The Rook project has established robust and reliable patterns for orchestrating storage systems running on-top of Kubernetes clusters. However, storage systems often need to extend beyond a single cluster to provide data to end users in many different locations. How can these globally distributed storage systems best be orchestrated and managed to meet the needs of their users on a global scale? In this talk, we will examine the key concepts of storage orchestration for cloud native applications in multicloud environments, as well as the major benefits of these types of deployments. We will learn about the challenges that arise and best practices to address them, such as replication, mirroring, snapshots, and disaster recovery. All of these lessons will be in the context of the Rook project, including a live demonstration of storage orchestration on a global scale.
https://sched.co/MPfq
A
Alright,
let's
go
ahead
and
get
started,
so
this
talk
is
called
data
without
borders.
It's
about
using
rook
storage
orchestration
at
a
global
scale,
so
my
name
is
Jarrod
watts,
one
of
the
founders
of
the
rook
project
and
a
senior
maintainer
on
it
now
and
I'm.
Also,
a
founding
engineer
at
a
Seattle
based
startup
called
up
bounds
and
we
are
hiring
so
let's
just
do
go
quickly
through
some
level
setting
here.
So
why
might
you
want
to
deploy
your
storage
systems
globally
and
you
can
also
think
about
when
you're
deploying
globally?
A
You
may
also
be
in
a
multi
cloud
scenario
or
configuration
as
well,
and
so
some
reasons
you
might
want
to
do
that.
There's
a
number
of
them,
but
one
of
the
big
ones,
definitely
is
reliability.
So
when
you
spread
your
data
out
across
a
number
of
data,
centers
regions
providers
all
that
sort
of
stuff,
you're
kind
of
hedging,
your
bets
against
failures,
so
you
make
sure
more
resilience
towards
any
one
particular
provider
or
region
having
a
catastrophe.
Performance
can
be
another
reason
as
well
when
you
globally
distributed
your
data.
A
If
you're
aware
of
where
your
users
are
that
are
consuming
the
data,
you
can
put
that
data
closer
to
them
and
decrease
the
amount
of
time
that
it
takes
for
them
to
access
that
data
increase
their
performance.
You
know,
cost
is
obviously
a
huge
one
as
well,
where
you
know,
if
you're,
if
you're
distributing
yourself
across
the
globe,
you
can
make
choices
about
where
you're
running
and
choosing
you
know
optimizing
on
cost.
A
Innovation
is
interesting
because
all
the
different
cloud
providers
they
have
some
really
good
engineers
and
they're,
consistently
cranking
out
new
features,
something
Amazon's
I,
don't
know
how
many
hundreds
of
services
they
have
now.
But
you
know,
when
you're
able
to
run
across
all
these
different
environments
and
providers,
you
can
start
taking
advantage
of
new
features
that
come
from
them.
There's
a
number
of
other
different
reasons
as
well.
But
you
know
compliance
is
important
to
like.
A
Perhaps
you
need
to
keep
your
data
within
one
particular
country,
or
one
particular
region
like
the
EU
and
some
other
ones
as
well,
but
there's
basically
there's
some
reasons
that
you
might
want
to
run
your
your
services
or
your
storage
globally
or
in
multi
cloud
scenarios.
We
can
get
through
this
pretty
quickly,
I
think
because
I'm
pretty
sure
everyone
here
is
fairly
familiar
with
a
lot
of
these
concepts.
A
But
one
thing
it's
important
to
note
here
couple
things:
they
difference
between
availability
and
durability,
so
availability
is
just
for
just
resistance
to
a
temporary
downtime
like
a
data
center
goes
offline.
If
you
are
highly
available,
then
you've
got
that
data
available
somewhere
else
and
your
clients,
your
consumers
requests,
can
be
fulfilled
from
another
data
center,
so
you
didn't
lose
any
of
the
availability
that
you
have
to
service
your
customers.
A
They
had
disaster
recovery
plans
in
place,
so
they
were
able
to
recover
from
that.
So
his
clusters
went
boom
and
they
had
an
answer
for
now
what
they
recovered.
It
was.
That
was
a
pretty
impressive
talk.
Definitely
all
right!
So
I'm
most
people
are
probably
familiar
with
the
Brooke
Project.
But
let's
get
through
this
quickly
here
of
what
is
Brooke.
A
So
Brooke
is
a
cloud
native
storage
Orchestrator,
and
what
that
really
means
is
that
it
extends
kubernetes
with
a
bunch
of
custom
types
and
custom
controllers
that
do
a
lot
of
basically
software
automation
for
your
storage
systems.
So
all
these
different
tasks
like
bootstrapping,
that
your
storage
system,
configuring
it
managing
it,
keeping
it
healthy
monitoring
it
all
those
different
tasks.
You
know
we
provide
a
lot
of
software
automation
around
those.
A
These
operational
tasks
such
that
normally
humans
do
those,
but
the
real
project
is
built,
a
number
of
orchestrators
or
operators
that
codify
that
domain
expertise
about
how
to
run
a
distributed,
complex
software.
This
is
the
storage
system
and
put
it
into
code
to
do
it
automatically
for
you.
Additionally,
it's
got
this
framework
that
can
be
used
to
build
or
some
or
multiple
storage
systems.
A
So
we
started
with
a
single
storage
solution
in
the
brook
project
and
now
I
think
we
host
or
support
like
six
or
seven
with
this
framework
that
we've
written
it
is
a
CNC
F
project.
We
donated
it
to
the
cloud
native
computing
foundation
in
January
of
2018
and
then
in
September
of
last
year
we
were
moved
to
the
incubation
phase
and
that's
why
we
get
to
have
a
little
flag
in
our
logo
all
over
the
place
here,
I'm
pretty
happy
about
that.
A
So,
let's
make
let's
just
make
a
distinction
here
between
control
planes
and
data
planes,
so
rook
as
a
storage
Orchestrator,
it's
a
control
plane.
It
is
not
the
data
plane
when
you
want
to
actually
read
or
write
bytes
to
your
persistent
storage
you're,
going
through
a
data
plane
you're
going
through
the
underlying
storage
provider
and
contrast:
rook
is
more
the
control
plane.
So
it
does
those
operational
tasks
of
deploying
and
bootstrapping,
and
configuring
and
managing
the
data
plane.
But
rook
itself
does
not
it's
not
on
the
data
path.
It
is
not
the
data
plane.
A
Therefore,
that
means
an
implication
of
that
is
that
rook
can
go
offline
for
minutes
at
a
time
and
not
affect
your
availability
of
your
data,
because
it
is
the
control
plane
which
is
separate
from
the
data
plane.
So
there's
multiple
data
planes
that
I
was
speaking
about
that
rook
performs
these
orchestration
services
for
and
we're
gonna
get
into
some
deep
dives
on
some
of
those
there.
A
But
if
an
analogy
here,
if
you're
familiar
with
sto
and
envoy,
is
that
that's
the
exact
same
relationship
where
sto
is
a
control
plane
that
manages
and
configures
and
sets
up
the
Envoy
proxies
and
then
envoy?
Just
all
it
does
is
worry
about
streaming
bits
around
and
routing
traffic
and
handling
the
data,
so
sto
control,
plane,
envoys
out
of
plane,
rook
control,
plane
and
you
know
edge
of
SF,
Cassandra,
cockroach,
DB,
etc.
Our
data
planes.
A
So
that's
an
important
distinction
to
make,
because
you
could
have
different
approaches
to
global
scalability
and
your
data
plane
and
in
your
control
plane.
So,
let's
first
take
a
look
at
the
data
plane
approaches
so
in
our
data
planes
at
a
high
level,
there's
basically
two
tight
different
types
of
architectures
here
one
is
a
storage
system
that
is
natively
sort
of
designed
and
Arkan
tected
to
work
at
a
global
scale
from
the
very
beginning.
That's
just
part
of
its
core
architecture,
so
edge
FS
in
cockroach.
A
Db
or
examples
of
that,
and
then
you
have
a
different
type
of
architecture.
You
have
a
storage
system
that
works
more
in
a
local
scale,
that
design
and
architecture
is
optimized
for
a
single
cluster
and
then
it
in
order
to
do
global,
scalability
or
global
scenarios.
It
works
more
like
a
federation
type
of
scenario
where
it's
it's
managing
all
its
data
locally
and
then
it'll
Fedder,
a
twith
other
clusters
to
replicate
and
mirror
its
data
outwards.
A
So
all
the
examples
that
I'm
about
to
give
you
here
are
from
data
plains,
storage
systems,
that
rook
does
orchestration
control
playing
services,
for
so
the
first
one
is
SEF.
So,
as
I
mentioned,
SEF
was
the
very
first
storage
system
that
Brooke
started
doing
orchestration
for
it's
the
most
stable
it
has
the
most
contributors
in
the
roof
project.
We
declared
it
stable
in
production,
ready
in
our
out
of
Nova,
0.9
or
1.0,
but
version
1.0
is
out
as
of
a
few
weeks
ago
and
so
stuff
is
production
ready
and
declared
stable.
A
So
the
the
key
part
here
obsessed
our
architecture
is
that
it's,
you
know,
as
I
mentioned
it's
for
a
single
local
cluster,
specifically
it's
not
designed
to
be
natively
a
global
storage
system,
so
something
that's
really
good
about
SEF
is
that
it
has
a
very
strongly
consistent
storage
model.
So
when
you
are
architected
to
work
within
a
single
cluster-
and
you
have
you
know
very
close
locality
to
the
other
nodes
in
the
cluster,
you
have
very
small
latency
times
sub
5
millisecond
latency
times
for
your
for
the
nodes
there.
A
You
can
start
doing
things
that
enable
high
performance
and
strongly
consistent
data.
So
when
you
write
a
bite
to
the
Ceph
storage
system,
it's
needs
to
replicate
it
copy
it
out
to
other
nodes
and
replicas
in
the
system,
and
it
does
that
synchronously.
It
will
do
the
writes
to
the
other
copies
of
the
the
replicas
in
the
system
and
only
until
all
of
them
have
acknowledged
that
right
will
it
commit
and
return
back
to
the
client
saying
that
your
data
has
been
written.
A
It's
it's
committed,
it's
done,
so
we
can
do
this
because
it's
designed
for
a
single
cluster
with
low
latency
among
all
the
notes,
but
another
really
cool
thing
about
the
Ceph
architecture-
is
that
it's
incredibly
highly
scalable
because
it
decentralizes
the
choices
about
where
to
place
data
in
the
cluster
independently.
Each
client
can
make
a
decision
about
where
the
data
is
supposed
to
be
located
by
following
a
consistent
algorithm.
So
you
don't
have
to
go
to
a
centralized
bottleneck
to
say:
where
should
I
put
this
fragment?
Where
should
I
put
this
fragment?
A
There's
not
this
central
store,
that's
authoritative
for
the
data
goes.
Anybody
could
do
it.
It's
an
independent
algorithm
that
each
client
can
do
on
their
own,
so
it
gets
a
global
scale,
though
now
really.
What
the
point
of
this
talk
is
about
is
that
this
single
cluster
we
have
here,
we
can
set
up
these
Federation
scenarios
where
we're
doing
asynchronous
replication
or
in
other
words
you
describe.
A
It-
is
mirroring
where
we're
taking
all
the
blocks
or
all
the
objects
that
are
stored
in
the
Ceph
filesystem
and
we're
a
synchronous,
asynchronous
lis
replicating
those
out
to
other
clusters.
We're
not
doing
rights
from
the
client
perspective
across
all
these
different
clusters.
We
are
writing
it
to
the
local
cluster
and
then
we're
mirroring
it
out
in
a
federated
type
of
scenario.
So
that
really,
you
know,
enables
two
scenarios
of
like
disaster
recovery,
high
availability
and
reliability
through
that
federated
type
of
architecture
now
HFS.
A
This
is
another
storage
system
that
rook
provides
control,
playing,
orchestration
services
for-
and
now
this,
in
contrast
to,
SEF,
is
a
natively
designed
system
to
be
globally
available.
So
edge'
fesses
architecture
is
based
on
immutable
blocks
kind
of
similar
to
get
and
when
I
say
get
I
don't
mean
a
blockchain
I
mean
that
blocks
of
data
that
make
up
files
are
immutable
and
they
have
so
it
might.
You
make
a
modification
to
a
block
and
it's
globally
unique
in
its
version,
then
that
modification
results
in
a
whole
brand
new
identity
of
the
block.
A
A
So
the
terminology
used
in
edgy
fest
is
it's
segmented
storage,
so
each
site,
you
know
like
let's
say
one
in
Asia,
one
in
Europe,
one
in
North
America,
those
can
all
be
thought
of
as
segments
and
then
they're
stitched
together
into
a
single
geo
namespace
and
the
component
inside
a
veggie
fest
that
does
that
is
called
an
interstate
way.
It's
connecting
all
these
segments
together
into
a
single
namespace,
so
one
of
the
when
you
have
a
globally
distributed
fallacies,
be
a
storage
system.
A
Some
you
need
to
make
some
optimizations
to
be
able
to
have
clients
have
good
performance
still.
So
one
of
these
optimizations,
that's
particularly
useful
here-
is
a
metadata
only
mode,
and
so
let's
say
we
have
one
segment
here
in
Asia
and
one
segment
here
in
North
America,
and
so
you
can
enable
this
metadata
only
mode
where
what
happens
initially.
A
Is
that
only
the
metadata
like
the
file
names,
the
file
system,
structure,
time
stamps
file,
sizes
information
like
that
gets
sent
across
to
the
other
segments
and
the
Geo
cluster
there,
and
so
that
enables
clients
in
other
parts
of
the
world
to
immediately
see
what
files
are
there?
What
files
they
can
read
to
write
to
make
decisions
about
how
they
went
in
and
want
to
interact
with
the
storage
system
and
then,
if
they
make
a
choice,
since
they
know
what
files
are
there,
they
want
to
actually
do
a
read
or
write
operation.
A
The
data
that
backs
up
those
files
that
we
have
metadata
for
can
be
fetched
lazily
or
on-demand,
and
so
this
is
absolutely
critical
for
enabling
scenarios
where
remote
clients
distributed
around
the
globe
can
start
consuming
the
data
as
fast
as
possible.
They
don't
have
to
wait
for
the
data
be
transferred.
They
have
all
the
information,
they
need
to
start
making
decisions
about
what
data
to
interact
with
and
then
they
can
lazily
populate.
A
The
data
chunks
edge
of
s
also
does
a
global
deduplication
as
well,
so
saying
that
all
the
blocks
have
unique
identities
and
when
you
modify
a
block,
it
gets
a
whole
brand
new
identity.
So
this
enables
you
to
do
deduplication
of.
If
you
have
identical
data
chunks,
you
can
you
can
take
multiple
of
them
and
only
just
store
it
once
or
you
know.
However,
many
times
you
want
to,
according
to
replication
that
you
want
to
achieve,
but
you
don't
have
to
continue
having
you
know,
tons
and
tons
of
copies
of
highly
popular
data.
A
You
can
constrain
that
down,
have
less
copies
of
them
and
do
less
work.
Disaster
recovery
is
kind
of
a
built-in
philosophy
of
edges
globally
distributed
data
system.
Where,
if
you
lose
a
data
chunk
on
one
particular
cluster,
you
can
just
repopulate
it
from
another
another
particular
segment
there.
So
your
local
caches,
your
local
segments,
your
sites,
will
always
be
populated
with
the
data
that
may
have
been
lost
locally,
so
cockroach
DB
rook
does
control
plant
orchestration
services
for
cockroach
DB
as
well,
and
cockroach
also
happens
to
be
natively
designed
for
global
storage.
A
So
one
interesting
aspect
of
cockroaches
architecture,
though,
is
it:
has
this
built-in
locality
awareness,
so
you
know
with
cockroach
you're,
not
necessarily
bound
to
the
data
centers
of
a
single
cloud
provider.
It's
an
absolute
design
goal
from
the
very
beginning
in
cockroach
DB,
to
two
steps
here.
One
is
to
minimize
latency
so
to
be
able
to.
When
you're.
A
Do
your
locality
aware
you
can
make
decisions
about
where
to
put
the
data
to
be
closer
to
the
users
that
are
actually
going
to
use
that
data
and
care
about
that
data,
but
you
can
also
as
a
secondary
or
a
complimentary
design
goal
you
can
avoid
sacrificing
availability.
Let's
say
your
users
are
going
to
be
accessing
to
that
in
New,
York,
let's
say,
and
so
to
really
minimize
the
latency
there.
You
could
put
it
all
in
New
York,
but
that
would
be
sacrificing
availability.
A
If
New
York
happens
to
have
a
disaster,
then
you've
lost
all
you
can
put
all
your
eggs
in
that
basket
and
you've
lost
them
right.
So
you
want
to
minimize
latency,
but
also
don't
sacrifice,
availability.
You
want
to
have
a
high
number
of
copies
somewhere
else,
though
in
case
you
lose
some
in
the
primary
zone.
A
Cockroach
TB
does
uses
two
different
distributed:
algorithms
to
do
its
magic
across
its
globally
distributed
storage
system.
There's
the
gossip
protocol
and
the
raft
consensus
algorithm
so
for
the
gossip
protocol.
That's
what
the
nodes
used
to
self-organize,
to
talk
to
each
other
find
out
where
they
are,
how
much
storage
capacity
they
have.
A
You
know
where
the
data
on
them
resides
all
that
sort
of
stuff
and
then
raft
consensus
is
basically
a
means
that
they
use
to
all
agree
upon
what
replicas
exist
on
them
and
what
state
they
should
be
in
so
Etsy
D,
which
is
the
backing
persistent
storage
for
the
kubernetes
api
server.
Does
the
exact
same
thing?
A
So
the
whole
goal
here
of
having
this
locality
information
is
to
increase.
Replica
diversity
is
what
it's
called
and
replica
diversity
means
that
you,
the
copies
and
the
data
you
have.
You
want
them
to
be
spread
out
across
the
machines
that
are
in
different
localities,
so
you
could
have
a
replica
kicking
out,
let's
say
three
or
five
on
three
or
five
different
machines,
but
if
they're
all
in
the
same
area,
they
have
the
same
locality.
That's
not
you're,
not
really
doing
anything
good
for
for
durability
and
availability
of
your
system.
A
So
increasing
replica
diversity
is
a
very
important
goal
of
cockroaches
locality
awareness.
When
there
is
a
failure,
though
cockroach
can
consent
temporarily
sacrifice.
Replica
diversity
in
favor
of
replica
counts,
so
say
that
we
have.
You
know
a
number
of
different
copies
of
the
data
spread
across
a
certain
number
of
regions,
and
then
we
lose
one.
You
know
instead
of
just
saying:
okay,
I
can't
get
the
replica
diversity
I
want,
so
I
won't
do
anything
it's
better
to
make
multiple
copies
within.
You
know
a
smaller
amount
of
regions.
A
It's
you
could
be
whole
question
to
the
ending.
If
you
mind
okay,
thank
you
very
much,
I'm
sorry,
so
the
one
do
I
think
was
something
that's
really
interesting
about.
Roach.
Is
that
this
replication
constraints?
You
said
it
gives
you
the
ability
to
to
put
this
this
locality
awareness
on
at
different
levels
of
the
system.
So
we
can
have
a
you
know
at
the
database
level,
you
can
have
it
re
the
cluster
level
database
level
table
level
even
down
to
the
row.
A
You
can
input,
you
know,
constraints
and
drive
the
data
in
that
system
to
be
put
into
or
forced
into
particular
regions.
So
you
can
kind
of
have
very
much
tight
control
over
how
you
want
their
global
data
layout
or
the
cockroach
system
down
to
individual
rows.
The
rows
one
I
believe
is
an
enterprise
only
feature,
but
I
still
think
it's
a
very
impressive
part
of
their
architecture.
Ok,
so
let's
move
on
to
control,
plane
approaches.
So
all
that
time
we
were
talking
about
the
data
planes
right.
A
So
now
we
want
to
talk
about
the
control
plane,
the
system
that
organizes
and
orchestrates
and
manages
of
the
data
plane,
instances
and
nodes.
So
that's
a
yeah
if
we
remember
that
rook
is
a
storage
Orchestrator
right
and
so
it
provides
orchestration
and
control
plane
services
at
the
level
of
just
a
single
cluster.
So
if
we
want
to
do
things
at
a
global
scale,
then
we're
gonna
need
something:
that's
more
of
a
global
Orchestrator
that
spans
across
clusters
across
clouds
and
enable
actually
global
scenarios
right
and
automated
scenarios,
at
least
so
this
this
global
Orchestrator.
A
A
So
you
know
a
global
control
plane
would
be.
Would
it
would
be
the
control
plane
for
global
storage
systems
and
it
would
be
a
to
do
even
more
than
just
setting
up
and
having
awareness
of
locality
and
making
these
decisions
about
where
the
data
should
live?
It
could
actually
do
even
more
interesting
stuff
that
you'd
want
an
operator
to
do
it.
Could
you
know
orchestrate
rebalancing
of
data
across
the
global
scale?
It
could
automate
some
disaster
recovery
scenarios
or
failover
scenarios.
A
You
know
you
could
do
all
these
sort
of
smart
things
when
you
have
this
set
up
here,
to
be
able
to
span
all
these
clouds
and
create
these
globally
distributed
data
planes.
So
a
critical
piece
of
infrastructure
that
this
type
of
global
storage,
our
construct,
Orchestrator
or
control
plane
would
need,
is
a
multi
cluster
or
multi
cloud
control,
plane,
and
so
there's
a
couple
examples
of
that.
A
One
of
them
here
is
cube
bed,
which
is
I,
believe
the
new
name
for
the
Federation
v2
effort,
and
so
you
know
this
is
a
control,
plane
and
kubernetes
that
spans
across
multiple
kubernetes
clusters
and
so
there's
three
key
parts
of
its
architecture.
Here
the
templates,
placement
and
overrides
so
templates
is
how
you
represent
a
set
of
resources,
they're
going
to
be
common
across
all
of
the
clusters
that
you
have
in
this
federated
scenario.
Placement
is
information
you
give
about.
Where
do
you
want
this
resource
to
be
placed
across
these
clusters?
A
Do
you
want
it
in
cluster
one
cluster,
two,
you
know
decisions
about
where
the
resources
will
get
put
to,
and
then
you
have
a
set
of
overrides,
which
is
kind
of
like
you
know.
You
have
rules
about
putting
this
particular
deployment
in
cluster
1
and
cluster
2,
but
you
can
override
that
with
specific
configuration.
For
you
know,
oh
in
this
cluster
do
something
a
little
bit
different,
so
you
put
all
these
components,
so
these
concepts
together
and
you
can
start
to
be
able
to
deploy
and
run
services
across
many
kubernetes
clusters.
A
Propagation
refers
to
how
from
this
single
control
plane,
do
we
make
decisions
and
then
make
those
resources
actually
show
up
in
the
other
clusters.
We've
got
to
propagate
these
placement
and
scheduling
decisions
to
other
clusters
so
that
our
target
clusters
are
actually
running
the
services
that
we
want
now
another.
You
know
global
Orchestrator,
a
multi
cloud
Orchestrator.
Is
they
cross
a
project
called
cross
plane?
That's
we
started
an
open-source
just
before
cube
con
in
Seattle
so
about
five
months
ago
and
so
crossplane
is.
A
But
then
you
know
instead
of
just
kubernetes
resources,
it
would
be
able
to
also
deploy
infrastructure,
our
applications,
our
platform
services
etc,
and
make
smart
decisions
about
how
to
globally
optimize
these
placements,
how
we
could
actually
put
our
services
and
distribute
them
and
propagate
them
across
all
of
our
different
clouds
and
clusters
that
we
are
spanning,
and
in
addition
to
that,
you
know
a
portable
set
of
resource
abstractions.
So
you
know
the
concept
of
a
Postgres
database
or
a
my
sequel
database.
A
Okay,
so
let's
get
here
towards
a
demo
here-
and
this
is
a
bit
of
a
busy
slide
and
there's
a
picture
on
the
next
slide
that
that
it
gives
you
a
visualization
for
this,
so
it'll
kind
of
make
that
a
little
bit
easier.
So
what
we're
going
to
do
here
is
we're
going
to
show
you
know:
Brooke
using
the
edge
FS
so
Brooke
as
a
control
plane
using
edge
of
s
as
the
data
plane
to
enable
a
globally
scale
the
global
globally
distributed
scenario.
A
This
is
the
connection
or
the
link
that
we
need
between
our
different
global
artists
to
be
our
different
local
segments
to
make
a
global
segment
and
a
quick
note
to
that
that
we're
doing
this
with
an
Amazon
scenario,
but
also
Google
Cloud,
is
actually
also
supported
by
Edgefest
and
Azure.
Work
is
in
progress,
so
at
the
end
of
that
you'd
be
able
to
stitch
all
of
the
cloud
providers
together
into
a
scene,
geographic
namespace,
so
here's
the
picture
as
promised
I'm,
not
an
artist,
but
it
does
definitely
represent
when
I'm
talking
about
better
than
that.
A
So
what
we're
gonna
see
here
is
on
the
left
side
of
the
diagram
and
we're
gonna.
Take
a
file
or
an
object,
we're
gonna
do
a
put
to
an
s3
bucket
and
then
I
have
that
connected
to
lambda
to
subscribe
to
bucket
events
that
will
then
just
propagate
or
copy
that
over
to
our
one
of
our
edge
FS
clusters
that
exposes
an
s3
interface
and
then
we
have
another
edge,
FS
cluster,
one
in
the
east
region
and
one
in
the
West
region
that
is
connected
by
this
intersegmental.
A
Esters
together
to
be
able
to
stitch
together,
two
segments
into
a
globally
distributed
namespace.
So
this
is
the
pretty
picture
here
or
picture:
let's
not
use
the
pretty
word,
and
now,
let's
do
a
demo.
So
before
I
start,
this
Tim
I
wanted
to
give
a
thanks
to
Dimitri
and
Ilya
from
the
IGF
s
team.
They
definitely
gave
me
a
lot
of
help
to
get
this
set
up
and
understand
all
the
concepts
so
I
wanted
to
make
sure
that
they
were
included
in
this
alright.
A
A
This
is
just
my
cheat
sheet
here,
because
I'm
really
bad
at
typing
and
talking
at
the
same
time,
alright
and
I
know
that
the
lights
at
the
top
of
the
screen
or
a
little
bit
bright
and
we
can't
dim
those.
But
basically,
what
I'm
showing
here
is
this
on
my
laptop,
my
little
MacBook
here
I
have
a
couple
of
different
kubernetes
contexts
set
up
so
that
I'm
pointing
at
two
entirely
different
kubernetes
clusters,
one
in
the
West
region
and
one
in
the
East
region
and
they're
both
running
edge
effects.
A
So,
okay,
so
we've
got
two
different
Gruber
Nettie's
clusters
set
up
and
let's
first
look
at
the
East
one
and
so
I'm
using
food
control
to
connect
to
the
east
cluster
and
I'm,
looking
at
what's
running
there
in
the
edge
FS
namespaces.
So
we
have
an
edge
FS
operator
running
the
brook
edge
FS
operator
running
and
we've
got
a
number
of
services
like
the
s3
service,
the
is
GW
service,
the
edge'
fest
manager.
A
A
So
let's
take
a
look
here
at
s3,
and
so
now
we're
going
to
go
ahead
and
upload
a
file
to
s3
we're
gonna
pick
this
little
pixel
picture
here
and
I
would
accept
all
the
defaults
and
upload
that
file
and
good
conference
Wi-Fi
is
working.
Thank
goodness
and
so
I
have
uploaded
a
file
into
the
s3
bucket.
And
so,
if
we
remember.
A
When
I
put
that
file
into
the
s3
bucket,
a
lambda
function
is
executed
in
response
to
the
bucket,
create
and
bucket
remove
events
that
then
propagates
that
object
over
to
the
s3
service
that
edge
FS
is
running
inside
of
a
cluster
and
then
the
is
GW
link
connects
the
two
disparate
geographically
distributed
clusters
to
copy
the
data
over.
So
what
we
should
see
now,
after
all
that
kicked
in,
is
this
on
both
the
on
the
east
cluster.
A
We
should
eventually
see
that
the
file
on
the
west
cluster
as
well
I
was
talking
to
Dmitri
in
Ilya
about
the
architecture
of
edge
of
s
and
there's
a
the
by
default.
One
of
the
optimizations
the
optimizations
did
they
do,
is
to
have
a
delay
in
the
propagation
in
order
to
be
able
to
batch
up
opera
files
and
objects
that
you
want
to
copy
across.
So,
let's
see
if
it's
going
through
now
cool,
so
it
is
on
the
west
side,
now,
god
yeah
and
so
I
think
that
that's
a
configurable
tunable
parameter.
A
You
can
turn
that
delay
off
completely,
but
I
left
the
defaults
on
there,
so
it
tries
to
get
as
many
fouls
as
they
can
and
with
a
delay
and
then
batch
them
all
up
and
send
them
over
the
wire
in
one
big
bundle.
Instead
of
opening
and
closing
connections
over
and
over.
So
that's
a
by
design
and
I,
don't
even
on
even
lying
about
that.
That's
the
best
part
about
it.
A
Ok,
so
then
I'm
just
doing
a
CD
over
to
my
desktop
and
then
I'm
gonna
use
the
s3
CMD
command
to
do
a
get
operation
on
from
the
West
cluster
to
go
ahead
and
download
the
little
file
that
we
first
started
in
s3
and
then
went
to
our
one
of
our
edifice
clusters
and
then
over
to
our
other
edge
of
s
cluster.
So
on
my
desktop,
we
should
see
that
the
file
is
back
and
we
it's
a
little
pixel
file
of
this
mug
here.
A
So
it's
not
a
very,
very
important
file
or
anything
like
that,
but
it
is
gone.
The
full
round-trip
from
all
my
laptop
uploaded
to
s3
through
lambda
to
edge
of
s
cluster
one
in
the
east
region
and
then
across
with,
is
GW
to
the
edge
of
s
cluster,
orchestrated
by
rook
in
the
West
region
and
then
downloaded
back
to
my
laptop.
A
On
slack
active
on
github,
happy
to
have
new
users
file
issues
test
out
the
services,
even
contributors
as
well.
The
Brooke
project
has
more
than
150
contributors
now,
so
you
know
that's,
let's
keep
growing
it
and
keep
building
cool
stuff
like
this,
and
the
last
thing
is
that
there
are
more
sessions
about
rook
here
at
coupon
I'm,
not
in
presentation
mode.
How
can
you
look
at
slides
if
it's
not
full
screen,
so
we
have
some
more
Brooke
sessions
here.
A
It's
sort
of
like
a
meet
the
maintainer
session,
where
the
rook
maintainer
project
maintainers
will
be
there
to
be
able
to
answer
questions
and
talk
about
more
stuff.
So
why
we
don't
have
much
time
for
questions
right
now.
There
will
be
the
opportunity
to
go
check
out
the
deep
dive
right
now
and
to
also
meet
the
rest
of
the
maintainer
team
at
the
CN
CF
answer
bar
and
later
on.
There's
more
local
talks
today
too.
So
that's
four
things
about
rook.
If
you
didn't
get
rook
enough
right
now
for
more
so.
Thank
you
very
much.