►
Description
Stu Berman of PiKNiK & ESPA speaks on what sorts of data ingestion rates are we building towards and what sort of capacity are we trying to hit, both in markets and in sealing workers.
Watch the recap of the Storage Provider meetup in Austin, hosted by the Filecoin Foundation!
Meetups are an opportunity to meet the amazing people and teams participating in the Filecoin community's decentralized storage network.
Learn more at https://sp.filecoin.io/
A
I
just
checked
from
what
I
can
see
right
now.
There's
105
pebbites
of
real
data
on
the
network
today
and
I'm
not
sure
what
everyone
else
is
seeing
here,
but
we're
seeing
a
huge
interest
in
onboarding
a
lot
of
data,
a
lot
more
data,
we've
got
potential
customers
that
are
looking
at,
providing
like
up
to
an
exabyte
of
data,
real
data.
A
So
take
aside
the
whole
issue
of
verified
deals,
fill
plus
and
all
that
stuff.
What
my
concern
was
is
this
topic
of
data
ingestion
rates,
because
traditionally
we
really
had
a
storage
revised.
We
had
two
things
that
really
mattered
right.
One
was
how
much
can
I
store,
which
is
a
function
primarily
of
how
much
filling
get
my
hands
on
an
affordable
rate
plus
some
capex
in
terms
of
a
bunch
of
jbods?
A
But
the
other
thing
was:
how
quick
can
I
store?
How
quick
can
I
seal
stuff,
and
now
I
think
it's
become
apparent
in
terms
of
how
do
I
get
the
data
on
the
network?
What
are
the
factors
that
influence?
How
quickly
I
conceal
one
of
the
things
I
you've
probably
seen
me
talk
about
on
on
slack
was
the
market
node,
which
has
been
being
replaced
by
boost
and
boost
is
phenomenal.
It's
it's
a
great.
A
It's
got
a
great
system
for
much
faster
data
transfer
rates,
but
it's
still
at
this
time
linked
up
to
one
market
or
boost
node
per
miner,
so
you
can
scale
out
your
ceiling
operation.
You
can
make
it
your
ceiling
cluster
as
big
as
you
want,
but
you're
still
limited
to
one
boost
or
market
node.
So
that
begs
the
question:
well,
how
much
are
you
going
to
seal
before
you
have
to
stand
up
another
minor
and
stand
up
another
minor,
and
is
that
a
problem
for
you
today?
Are
you
saying?
A
Well,
I
just
stand
up
additional
minor
is
not
a
big
deal
or
is
it
like
hey?
This
is
really
cost
inefficient
to
have
to
have
so
many
minors
just
to
ingest
as
much
data
as
I
want
to
so
I
just
threw
a
couple
questions
up
there
for
you
and
it's
not
necessarily
just
the
whole
market.
Node
scaling
issue,
it
could
be,
you
know,
fill
as
a
you
know
as
a
limiting
factor,
your
network
bandwidth.
I
know
we
talked
to
some
folks
here
who
are
from
australia.
A
So
really
it's
it's
meant
to
be
discussion
in
terms
of
what's
your
what's
your
target
and
what
are
those
things
that
are
really
causing
you
pain
today
in
terms
of
reaching
those
targets,
even
if
you're
not
feeling
it
today,
you
maybe
you've,
calculated
it
and
asked
yourself
gee.
What
am
I
going
to
do
to
reach
the
target?
I
want
to
reach.
A
C
D
I
mean
transfers
is
always
you
know
an
issue,
it's
probably
the
number
one
question
you
get
from
clients
is.
How
do
I
get
my
data
to
you
and
what
does
that
cost?
And
you
know,
and
then
that's
variable
right,
so
every
client's
different
their
infrastructure
is
all
different
and
then
yeah
I
mean
from
actual
ingestion
again
depends
on
your
your
system
and
your
setup.
F
Yeah,
stefan
I
want
to
piggyback
so
my
wife,
my
pregnant
wife,
gave
me
the
one
thing.
I
can't
speak
through
just
piggybacking
on
that
and
deeps
dodged
the
question
by
not
being
here
today,
but
I
know
we've
been
talking
about
backbone.
We've
been
talking
about
you
know,
zx
was
talking
about
five
petabytes
a
day.
We
physically
can't
move
that
around
the
globe
right
now,.
C
F
So
whether
it's
all
of
us
are
storage
providers
coming
together,
whether
it's
pl,
we
need
the
capability
to
move
petabytes
of
data
through
the
super
highways.
Otherwise
we
just
can't
achieve
our
goals.
However
fast
you
can
see,
oh
you
know,
we've
got
a
ton
of
ceiling
capacity,
but
we
can't
ingest
a
petabyte
a
day.
F
Receiving
say
from
clients
say
from
client:
you
know
you
guys
from
theirs
from
in
theory
we
can
receive,
but
depending
where
it's
coming
from.
If
it's
coming
from
a
client
storage
provider
that
doesn't
have
the
capabilities
same
thing
right,
you
you
went
through
the
journey
with
would
show
of
like
moving
it
physically.
Then
here
then
there.
So
you
know
that
as
an
ecosystem
is
a
huge
challenge
that
we
need
to
solve
globally.
C
So
basically,
two
problems.
One
is
like
you
mentioned,
and
you
mentioned
the
same
thing
moving
it
from
the
client
to
the
network
and
then
distributing
it
in
the
network
right
and
there's
an
initiative
ongoing
right
now
for
us
to
look
at.
How
can
we
take
certain
edge
points
in
the
network,
meaning
sps
that
also
can
act
as
sort
of
an
on-ramp
into
the
network
for
other
sps,
right
and
so
yeah?
C
Please
connect
with
us
if
you're
interested
in
collaborating
with
that,
because
in
the
end,
like
you
said
earlier,
zx
said:
hey,
our
goal
is
five
pip
of
ingest
today
now
we're
doing
around
the
pip
a
day.
So
we
need
to
multiply
that
by
five
to
get
to
400
pip
by.
I
think
it
was
october,
which
should
be
very
feasible,
but
we
have
to
have
this
builder
built
out
yeah.
F
And
you've
got
the
build
on
that.
You
know
talking
about
clients
that
100
petabytes.
You
know
large
large
amounts
that
you
then
say
again
times
how
many
copies
then
you've
got
to
get
it
to
other
storage
providers
and
the
build
up
on
that
is.
You
know
we
we're
relatively
large
sure
provided
with
a
lot
of
a
lot
of
compute
a
lot
of
a
lot
of
space.
F
Even
then,
when
we
start
dealing
with
a
lot
of
data,
you
say:
oh
what
about
network,
and
so
you
there's
a
lot
more
investment
that
then
goes
into
network
internally,
and
this
isn't
stuff.
So
you
know
inviting
new
storage
providers
on
and
getting
people
up
to
speed
trying
to
help
to
out.
You
know
work
on
the
outlay
of
costs
for
scale
and
what
you're
going
to
need
to
do
as
every
time
we
hit
a
new
kind
of
milestone
in
the
business.
F
A
Yeah,
certainly
there's
you
know
picnic
we've.
You
know,
we've
got
a
data
ingestion
engine
that
we're
looking
at
how
we
can
share
that
out
as
well
as
s3
connectors.
So
there's
the
technical
like.
How
do
I
connect,
but
then
there's
the
these
questions
about
the
the
raw
speed
and
I
think,
with
phil
plus
the
real
question
is
you
know
that
becomes
a
multiplying
factor
right,
maybe
times
five
times
ten
in
some
cases
in
terms
of
replicas,
so
now
you've
got
you've
got
to
worry
about
those
bottlenecks.
G
I
just
want
to
do
also
a
shameless
plug
for
the
very
thing
we're
talking
about
in
terms
of
the
infrastructure
issues
that
we're
facing.
We
are
very
much
thinking
about
a
hybrid
approach
in
terms
of
having
a
sort
of
backbone,
high
bandwidth,
sort
of
connection
across
certain
nodes
that
are
regional
and
then
complementing
that
with
a
more
scalable
and
operationalized
offline
data
solution
for
onboarding
and
first
mile
I
will
say
you
know
anybody
who
is
interested
in
participating
in
some
pilots
that
I'm
going
to
run
in
the
next
few
weeks.
Please
reach
out.
G
F
So,
yes,
just
to
add
one
more
thing:
sorry
and
I'm
monopolizing
this
a
little
bit,
but
you
know
stew
and
picnic
great
example-
and
I
know
vine
and
in
in
the
netherlands
as
well,
where
they
have
created
technology
that
they
put
a
lot
of
time,
effort
and
money
into
creating
that
are
going
to
help
with
the
process.
As
a
group
again
can
we
can
we
create
ways
to
ensure
they
can
share
that
and
be
compensated
for
it
right
as
an
roi?
A
D
D
C
So
I
guess
my
takeaway
is:
we
definitely
have
to
follow
up
on
that
program.
That
deep,
is
is
building
now.
So
we'll
give
you
an
update
on
the
next
storage
meetup
or
we
have
an
earlier
update,
we'll
do
it
online,
but
the
reality
is.
We
are
working.
We
are
working
on
it,
so
we're
open
to
ideas.
If
you
want
to
participate.
C
Is
it
ipfs
or
is
it
something
else
that
could
help
like
with
you
know,
becoming
that
staging
server
to
actually
ingest
it,
and
then
what
drives
working
on
is
even
also
is
cool
too,
because
that's
really
like
the
data
movement
from
shipping
it
from
the
client
or
maybe
from
an
sp
to
another
sp
as
a
physical
device.
Anyone
else,
oh.
H
Yeah,
so
it
seems
like
the
the
cost
of
actually
sealing
is
relatively
irrelevant
compared
to
the
cost
of
networking
in
transportation.
Is
that
a
fair
enough
statement
like
you?
Don't
even
really
barely
factors
into
the
conversation?
You've
already
invested
the
money
in
the
ceiling
equipment,
you
don't
use
outsourced
and
it's
just
focusing
on
the
network
side.
I'm.
I
So
that's
so
andrew
stango
here
finance
team
picnic
as
I
get
put
on
the
spot,
but
I
guess
I
was
thinking
about
that.
Can
I
think
about
that
in
reverse
for
a
moment
to
give
myself
a
little
headroom,
as
I
try
to
look
more
now,
I
understand
why
politicians
like
delay
by
the
way
because,
like
I,
was
trying
one
of
my
like
little
sub
games
or
questions
I
was
going
to
ask
is
like
just
as
a
thought
experiment,
because
we're
all
storage
providers
is
yeah.
I
How
much
would
you
charge
if
you
had
access
to
this
or
like?
How
much
would
you
pay?
Because
I
don't
know
because,
like
that's
a
it's,
a
huge
investment
right
like
it's
kind
of
wide
open,
you
don't
want
to
do
it,
there's
a
ton
of
risk,
but,
like
I
don't
it's
something
I
came.
I
actually
came
into
the
week
trying
to
figure
out
myself
if,
if
that
makes
sense,
because
I
don't
it's
definitely
not-
I
don't
know
just
like
in
reverse
like
if
you
say:
oh,
it's,
free
storage
or
free
ceiling.
I
G
G
J
Yeah
I
mean
without
going
to
specific
costs
and
financing,
I
mean
the
well
theoretically,
the
mining
rewards
will
can
fund
your
future
storage
capex.
So
you
know
balancing
that
upfront
capex
between
processing
capacity
and
storage
capacity.
You
know,
arguably,
we
probably
went
a
little
too
light
on
processing
up
front
and
and
sort
of
did
half
and
half,
but
in
hindsight
I
might
have
probably
would
have
invested
more
in
processing
up
front
and
used
my
my
block
rewards
to
fund
future
storage
capacity.
I
C
Yeah,
and
not
too
common
to
that
as
well
like
we
are
seeing
sps
that
are
here
in
the
room
looking
at
offering
ceiling
as
a
service,
because
there
are
sps
that
either
didn't
invest
enough
in
compute
or
yeah.
Don't
necessarily
have
the
need
for
a
compute
for
a
very
long
time,
because
you
know,
then
you
may
be
buying
computing
only
needed
20
of
the
time.
C
So
you
know
there
are
some
people
here
in
the
room
that
are
now
thinking
of
offering
ceiling
as
a
service
where
you
can
actually
do
the
numbers,
and
you
can
see
that
if
you
can
see
it
within
a
certain
time
frame,
let's
say
within
a
month
versus
six
months
and
yes,
it's
more
like
a
one-time
pay,
but
you
know
it
still
pays
off
because
you're
going
to
get
faster
access
to
block
rewards
any
other
comments
to
that.
No
anyone.
E
E
C
Yeah
all
right:
well,
we've
got
a
lot
of
u.s
sps
here,
so
good
collaboration
opportunities.
I
would
say
anyone
else
wants
to
contribute
or
any
comments,
suggestions.
Actions
for
us
literally
like
is
there
anything
that
we
should
be
thinking
about
because
ceiling
as
a
service
is
something
we're
actively
pursuing
right
as
well,
and
how
can
we
optimize
the
code
to
do
to
allow
the
sps
to
provide
better
ceiling
as
a
service
offerings
as
well?
C
The
other
initiative
that
we
are
driving,
which
we
haven't
really
talked
about,
is
the
improving
the
aggregate
ceiling
rate
as
well
with
you
know,
some
of
the
top
vendors
we're
actually
collaborating
on
a
regular
basis
to
see
how
can
we
optimize
the
performance,
so
you
can
get
more
throughput
out
of
your
existing
hardware.
Of
course
that's
a
little
bit
of
a
longer
term
process,
and
but
it
is
an
initiative,
that's
ongoing.
C
K
K
H
K
K
L
L
If
you
want
to
be
a
storage
provider
long
term
the
market
right
now,
I
suggest
you
to
buy
your
own
selling
machine,
because
if
the
monthly
fee
is
like
a
monthly
payment
of
actually
buying
the
seating
machine-
and
it's
it's
way
more
expensive
than
you
just
buy
a
selling
machine,
you
yourself
to
use
it
instead
of
like
paying
a
couple
months
because
eventually
you'll
need
your
own
selling
machine
and
the
best
way
is
just
to
find
the
the
cheapest
hardware
to
steer
your
data.
And
there
is
some
like
physical
restrictions
you
can.
L
B
Depending
on
I
mean,
I
agree
with
you
to
some
extent,
though
I
think
given
sort
of
the
price
action
on
filecoin.
As
of
late,
it's
hard
to
get
folks
to
make
that
investment
in
gpus
and
high
core.
You
know
epic
or
intel
processors,
so
I
think
there
is
a
sort
of
a
viable
means
by
which
to
to
do
ceiling
as
a
service
it.
Just
I
guess
we
just.
We
just
need
to
figure
out
what
the
mechanics
of
it
are.
B
I
think
s-r-I-o-v,
sorry
and
then
giving
you
direct
access
to
the
gpus,
so
we
can
seal
somewhere
to
the
tune
of
like
a
10
pips
a
day
for
folks,
and
we
don't
really
know
what
the
where
the
market
is
with
respect
to
costs,
but
we're
happy
to
engage
anyone.
That's
looking
to
do
anything
like
that
as
well.