►
From YouTube: RETMKT Builders - Textile Miner Index
Description
Andrew updates us on the textile miner index at the Retrieval Market Builders Mini-Summit in April 2021.
A
Hi
cool
so
yeah.
I
was
asked
to
talk
about
the
our
work
on
the
minor
index.
Today,
though,
I'm
I'm
feeling
like.
I
really
want
to
talk
about
our
new
project,
because
there's
so
many
cool,
conver
sort
of
side,
conversations
about
pieces
that
we're
interested
in
tackling,
and
so
I'm
looking
forward
to
doing
this
again.
A
Hopefully,
if
we,
if
one
of
the
followings
on
this
is
another
meetup
but
yeah
so
the
minor
index,
let
me
so
the
motivation
of
the
minor
index,
if
you're
not
familiar,
we
we
build
powergate,
and
so
powergate
is
an
ex
is
sort
of
a
library
that
runs
on
top
of
lotus.
Allows
you
to
has
some
opinions
about
how
you
can
manage
data
moving
back
and
forth
between
file
coin
and
ipfs.
A
It
also
has
some
helpers
to
help
you
make
deals,
and
one
of
those
helpers
is.
You
can
create
arrays
of
of
trusted
miners
and
you
can
set
parameters
on
creating
deals
with
a
certain
amount
of
replication
and
then
powergate
will
go
to
work
to
store
that
number
of
replicas
across
all
your
trusted
miners,
and
so,
if
any
of
the
miners
turn
down
your
deal,
it
will
move
on
to
the
next
miners.
A
And
that's
where
the
minor
index
came
from,
and
so
the
minor
index
is
actually
in
its
courts
built
into
powergate.
So
anybody
that
runs
powergate
can
be
collecting
a
lot
of
information
about
the
miners
on
the
network,
and
so
it
will
use
both
on-chain
data
and
attach
a
lot
of
additional
metadata
on
top
of
every
miner
from
the
view
of
the
powergate,
and
then
we
also
run
this
on
the
textile
hub
and
expose
it
on
as
a
public
api
and
so
telemetry.
Well,
we'll
get
into
all
of
that.
So
so
right.
A
So
we
exposed
this
as
a
as
an
api.
You
can.
I
can
share
the
url
over
the
zoom
call
in
a
minute
and
ignacio
is,
I
think,
on
the
call,
still,
although
it's
kind
of
late
for
him,
but
he's
the
mastermind
behind
a
lot
of
these
pieces
and
so
he's
here
to
answer
questions
on
chat
as
well.
I
hope,
but
right
so
the
the
minor
index.
A
Here's
open
api,
specs
swagger
specs,
so
you
can
go
to
it
and
you'll
find
that
it
has
basically
every
minor
on
the
network
and
you
can
look
at
the
their
sort
of
history
of
creating
deals.
But
this
is
really
from
the
textile
perspective.
There's
some
on-chain
information
and
it
will
collect
information
about
their
their
storage
prices
and
all
those
and
you
can
filter
through
that.
But
then
it
also
has
the
the
perspective
of
the
textile
power
gates
history
with
these
miners.
A
So
you
can
use
it
through
the
api,
this
open
api
or
you
can
actually
download
a
cli
and
use
it
in
on
your
com.
You
know
on
your
desktop
or
in
your
any
any
tooling.
You
can
actually
script
that
and
build
into
other
things,
and
so
you
can
see
here.
Here's
a
nice
sample
of
the
data
that
comes
back
when
I
get
the
information
out
of
the
index
for
a
single
miner,
and
you
can
see
that
it
has.
A
You
know
their
advertised
costs,
but
then
down
below
you
can
actually
see
this
a
sort
of
hint
of
the
deal-based
metadata
that
will
come
back,
and
so
the
original
intention
here
was
we
knew
there
was
a
constraint
around
our
perspective
of
the
network
versus
what
another
client's
perspective
of
the
network,
which
is
a
challenge
I'll
bring
up
later.
But
you
can
see
in
the
index
it's
actually
exposing
region-based
perspectives
and
our
thinking
was.
We
would
always
run
multiple
of
these
nodes
that
could
collect
the
telemetry
about
miners
from
multiple
different
regions.
A
So
you
could
find
the
region
closest
to
you
and
get
telemetry
information
that
was
most
relevant
for
for
selecting
your
miners
and
you
you
can
query
it
and
you
can.
You
can
sort
by
a
lot
of
different
information,
and
so
actually,
if
I
just
jump
over
in
some
collaborations
with
jimmy.
Actually
I
put
together
this
little
observable
that
just
shows
how
to
use
the
api.
Quite
quite
simple,
I'll
share
this
link
as
well,
but
pretty
nice,
like
obviously
open
api.
A
So
then
I
just
do
a
bunch
of
json
filtering.
I
find
just
the
miners
that
are
advertising
free
storage
for
verified
clients.
I
find
miners
that
are
accepting
less
than
a
certain
threshold
of
data
size
and
then
I
I
list
out
the
top
eight,
and
so
that
way
you
know
over
here
I
have
a
function
just
to
get
the
top
one
at
any
moment
in
time.
The
default
sorting
in
the
minor
index
is
actually
based
on
our
last
successful
deals
with
the
miner,
and
so
that's
helped.
A
A
You
know
other
users
of
our
system
to
find
good,
good
miners
for
their
deals
and
then
another
thing
in
there
that
we
thought
was
very
useful
that
we
had
intended
to
build
into
some
nice
user
interfaces,
as
well
as
just
simple
deal
calculators,
but
specifically
deal
calculators.
That
would
facilitate
deal
creation
so
right.
A
So
I
can
put
in
the
parameters
of
a
deal
that
I
want
to
make
with,
and
I
can
list
a
a
set
of
minors
that
I
found
on
the
index
that
that
I
want
to
make
deals
with,
and
then
I
can
get
all
the
calculated
costs,
but
also
I
can
get
these
outputs
for
some
of
the
tools
for
how
I
can
quickly
turn
that
calculator
output
into
a
real
request
for
storing
the
data
with
that
miner
right.
So
then,
finally,
I
can
actually
just
copy
paste
it.
A
In
this
example-
and
here
you
can
see,
I'm
using
the
the
the
tooling
on
the
textile
hub
to
move
directories
to
filecoin
and-
and
here
I'm
just
doing
an
on
the
fly
update
of
my
storage
parameters,
to
set
it
to
these
three.
These
three
miners
that
I
found
with
fast
fast
retrieval,
flagged
yeah.
So
here's
just
kind
of
a
zoom
in
on
on
the
way
that
works,
but
we
don't
actually
have
to
get
into
that.
A
This
is
a
great.
This
is
a
great
case
where
this
is
true,
just
because
a
minor
makes
a
deal
with
us
and
we
measure
them
as
being
high
quality
or
high
speed
doesn't
mean
that
the
another
miner
or
another
another
client
is
even
necessarily
necessarily
going
to
get
a
deal
with
them
at
all.
And
then
the
telemetry
is
is
very
affected
by
temporal
load
on
clients,
and
this
is
something
our
latest
work
is
actually
trying
to
improve.
Quite
a
lot.
A
So
there's
a
problem
right
is
if,
if
a
lot
of
people
start
using
our
index
and
all
continuously
have
good
experiences
with
the
same
miner,
this
can
actually
be
bad
for
the
miner,
because
they're
going
to
not
be
able
to
have
much
control
over
their
load,
so
they're
gonna
have
to
either
turn
down
deals
or
fail
to
to
get
deals
on
chin,
quick
enough,
and
so
there's
a
there's
a
funny.
There's
a
funny
mismatch
here
between
motivations,
yeah
and
then
things
like
property
properties.
Like
the
minor
location.
A
We
actually
use
the
max
mine
database
to
do
ip
matching
of
all
the
miners
on
the
network
and
create
locations,
but
this
is
a
pretty.
This
is
a
pretty
brittle
field
that
seems
probably
gameable,
and
so
those
those
kind
of
things
would
be
very
nice
to
figure
out
better
ways
of
collecting
them
and
yeah
we
haven't
got.
We
haven't
really
focused
on
retrievals
and
getting
that
data
in
here.
So
that
would
be
a
next
step
if
we're
gonna
take
this.
A
A
The
second
set
of
challenges
are
really
around
the
disconnect
between
reputation.
What
you
really
want
to
get
out
of
an
index
like
this
and
the
economics
of
filecoin,
that
there's
kind
of
a
mismatch
here.
That
is,
that
is
only
loosely
connected.
So
just
some
examples.
Details
within
there,
like
the
prices
that
are
advertised
by
miners,
that
they're
accepting
deals
that
aren't
necessarily
going
to
reflect
reflect
what
you're
going
to
get
what
you're
going
to
get
on
your
final
deal.
A
They
can
have
a
lot
of
reasons,
but
filecoin
itself
is
a
very
volatile
market,
and
so
that
can
be
really
tricky
and
miners
are
going
to
be
a
lot
slower
at
adjusting
what
their
costs
are
to
the
market
rates
of
file
coin,
and
so
you
have
to
account
for
that.
When
you're,
looking
at
an
index
like
this,
that's
somehow
supposed
to
be
a
source
of
truth.
A
So
that
can
be
pretty
tricky
so
like
knowing
the
true
and
then
also
that
leads
into
how
you
convert
this
back
to
your
understanding
of
like
us
dollars.
How
much
am
I
really
paying
for
the
price
and
there's
a
lot
of
other,
like
kind
of
more
meta
issues
there
to
solve
that?
But
but
that's
there
and
then
yeah,
and
so
this
actually
gets
back
to
the
point
of
if
this
index
we're
being
used
by
a
lot
of
clients
and
putting
a
lot
of
pressure
on
the
best
miners.
A
Potentially.
Is
that
a
bad
thing
for
the
filecoin
network?
I'm
not,
I'm
not
sure.
I
guess
there
could
be
arguments
either
way,
but
something
to
think
about
with
these
kind
of
consolidated
views.
The
third
challenge
is
again
around
the
economic
discontinuity,
but
the
reputations
aren't
driven
by
the
economic
incentives.
Yet
so
like
there's
like
no,
no,
no
miner
really
mind
so
much
about
this.
A
They
really
are
focused
on
how
how
they're
performing
on
their
the
things
they
have
to
do
on
chain,
and
so
that's
something
to
consider
is
how
we
can
we
can
bridge
reputation
to
to
the
on-chain
activities
more
closely
and
then,
like.
I,
keep
wondering
this
about
many
different
competing
reputations
is.
How
do
we
rank
the
reputation
makers?
There's
no
reason.
Somebody
has
to
trust
textile
as
being
honest
here
and
that's
a
really
tricky
thing
to
figure
out,
and
then
we
didn't
put
a
reputation
on
on
our
system.
A
It's
really
an
index,
because
the
fair,
like
scoring
system
doesn't
seem
obvious
to
me,
and
at
least
yet-
and
I
I
think
that
there's
there's
obviously
some
good
proposals
out
there,
but
making
minors
feel
like
they're,
being
measured
in
a
fair
way
is,
is
pretty
important,
I
think
they're
all.
I
think
many
of
them
are
working
really
hard
to
make
the
network
better.
A
A
We
could
layer
in
tricks
to
try
to
make
the
reputation
the
reputation
system
here,
but
those
tricks
are
going
to
often
be
matched
by
the
tricks
of
the
miners
themselves,
because
if
they,
if
they
end
up,
then
having
the
incentive
to
be
showing
up
what
as
high
quality
in
the
index,
that
just
creates
a
game.
So
those
were
kind
of
the
sample
of
challenges
that
we
kind
of
debrief
on
this
project
and
and
started
thinking.
How
do
we
get
to
the
next
level
here
and
think
through?
A
How
do
we
really
organize
high
quality
minors
to
accept
our
deals
in
a
like
high
success
rate
sort
of
way?
And
that's
what
I
want
to
talk
about
next
time
so
who
will
get
there?
And
then?
The
last
point
is
just
that
this
that
there's
this
additional
problem
that
our
our
index,
for
example,
only
knows
the
miners.
A
I
think
that
some
of
these
other
projects
are
going
to
leapfrog
where
we
got
to
so,
hopefully
we'll
all
learn
something
from
them,
but
yeah
that
was
our
minor
index
project.
It's
still
running.
Definitely
we're
still
collecting
data
lots
of
good
stuff
there
until
so
until
people
leapfrog
us.
I
think
that
there's
a
lot
of
cool
stuff
to
build
on
here,
and
ideally
this
will
migrate
into
the
next
group
of
standards
wherever
we
go
next,
so
hopefully
that's.