Filecoin Filecoin Liftoff - the Main Event, 22 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Filecoin Mining Hardware and Architecture

Description

Ready to start mining Filecoin but need guidance on your setup? Join our top Space Race miners as they take you through their setups and architecture and discuss some of the things to consider and avoid when constructing your rigs.

Join us for Filecoin Liftoff Week, an action-packed series of talks, workshops, and panels curated by the web3 community to celebrate the Filecoin mainnet launch and chart the network’s future.
https://liftoff.filecoin.io/

Events take place all week, October 19-23, 2020. #FilecoinLiftoff

For more information on Filecoin
- visit the project website: https://filecoin.io/
- or follow Filecoin on Twitter: https://twitter.com/Filecoin

Get Filecoin community news and announcements in your inbox, monthly: http://eepurl.com/gbfn1n

Chapters
00:00 IPFSMain & Neo Ge Introduction
00:47 Filecoin Mining Architecture
02:35 Network Topology
09:05 Hardware Information
15:55 StorageHive & Jacques Swanpoel Introduction
16:50 Learning from mistakes
17:59 PC1
22:55 PC2 / C2
25:32 PoST

A

Hi everyone,, my name is neo ge.. I co-founded ipfsmain back in 2018.

A

And we are the service provider instead of the filecoin ecosystem., we've been committed to the filecoin project for almost three years now., and it was a historical moment.

A

For the filecoin mainnet launched last week, and I'm really happy to present today.

A

The talk today will be very similar to the minor show and tell I gave last month.

A

I'm going to go through our filecoin mining solutions,.

A

Which includes mining architecture and network topology.

A

Then I'm going to talk about our hardware information as well.

A

Let's get started.

A

Over the past two years,, we have had multiple architecture designs for large scale. Filecoin mining.

A

after 1000s of hours of testing, we've decided to use this filecoin mining architecture.

A

As you can see,, when we talk about the architecture,, we don't mean the macro architecture..

A

We believe filecoin mining architecture is the design of communications network.

A

Includes the physical and logical layout of the network.

A

It also includes the framework of specifications.

A

Of equipments, services, protocols,.

A

And functions of filecoin mining.

A

in this current mining solution,, we are dividing major hardware into three different pieces,: the storage rigs, sealing rigs and snarks rigs.

A

And by separating the storage and computational part,.

A

This architecture gives us really good scalability.

A

And improves the hardware utilization efficiency.

A

On tremendous skill.

A

in this design,, we want storage rigs to complete and the first step of post generate candidates on their own.. At the same time, we use sealing rate cluster for seal_pre_commit_phase1, aka, pc1,, pc2 and c1,.

A

And snark rig cluster for c2 and window post.

A

We modified and rewrote part of the code, and made some redundancy changes to (coughs). Sorry.

A

Improve the post success rate.

A

by doing all that, with this mining, architecture, we've significantly improved the mining efficiency.

A

Especially for p1 and c2.

A

for example,, we could manage to finish c2 under 10 minutes.

A

At the start of spacerace one and we've done some other further improvements.

A

Recently, as well.

A

We believe a robust network architecture.

A

Should facilitate system level functionality,.

A

As well as robustness and scalability.

A

It's especially in the face of changes in software and hardware components,.

A

Application functions and so on.

A

in our network topology,, we are using two different internet service providers.

A

For each wan to increase the redundancy.

A

The communication core switch uses, multiple switches.

A

With 10-gigabit optical ports for stacking,, which is mainly connected to lotus_node and lotus_miner.,.

A

The data core switch uses dual switches with 10-gigabit.

A

Optical ports for stacking,, which is connected to lotus_miner,.

A

Storage, seal_worker &, snark cluster.

A

And the storage access switches have four 10-gigabit optical ports, plus 24 gigabit electrical ports,, duo-switch stacking of course.. The 10-gigabit optical ports are connected to data aggregation, switches,.

A

And gigabit electrical ports are connected to storage.

A

Each storage is connected with two network cables.

A

Respectively to the two switches.- and we did the dual network port binding to improve the storage network stability as well.

A

When you are mining filecoin at large scale,.

A

Especially like something over hundreds of petabytes.

A

And a well built operation and maintenance platform.

A

Is a must to keep everything in order.

A

So next I would like to highlight our one stop.

A

O&Amp;M platform called fil manager here as part of our filecoin mining solution..

A

The fil manager includes cmdb ci/cd monitoring system.

A

And work order system., I'm going to break it down into details with a screenshot for each system.

A

The configuration management database, aka cmdb.

A

Is an itil database used to store information.

A

About hardware and software assets.

A

It is useful to break down configuration items.

A

Into logical, errors. and cmdb helps us understand the relationship.

A

Between the components of our filecoin mining system,.

A

And to to track their configurations.

A

It can also be used for things like impact analysis,.

A

Root cause analysis and change, management.

A

So in the fil manager, we use it to help us.

A

Automatically discover the host, get the host hardware information,, identify host failures and identify which cluster the host is working for in real time.

A

And see ci/cd means continuous integration.

A

Continuous delivery., it bridges the gaps between development.

A

And operation activities and teams by enforcing automation,.

A

In building testing and deployment of applications.

A

Our ci/cd module can help us perform fast.

A

And accurate deployment whenever there's an update in the filecoin network.

A

The component monitoring system provides comprehensive monitoring of the health status.

A

Of the components like lotus, miner,.

A

Seal, storage, snark, window post.

A

Like this, one lotus monitor, helps us node, synchronization.

A

And monitoring the amanbo as well.

A

As in case, our mess is being stuck in the mempool and the minor monitor helps us on order. Status,.

A

Minor status order, management.

A

Like practicing different pricing strategy,.

A

And sector lifecycle, tracking.

A

The seal monitor focuses on the working status.

A

Of seal workers like what is the sector id.

A

And what stage that sector is currently in.

A

It is able to identify various errors.

A

That occurred during the sealing process.

A

And do with fast automatically, it will start receiving new tasks until the fault is repaired..

A

The storage monitor helps us on the sector, storage, status.,.

A

It will find the storage failures actively and perform data migration in real time.

A

And make the migration history traceable.

A

Snark monitor focuses on the working status of snark.

A

And the resource demanded.

A

We know post monitor, is responsible for post working status.

A

And history of all miners.- and these monitors as mentioned, are all part of the component monitoring system.

A

After that,, we have the host monitoring interface.

A

All the monitoring system can work together.

A

To identify and classify faults.

A

In hardware services and networks.

A

And notify the corresponding staff to deal with the meantime,. The work order system manages the workflow.

A

And can track various operations and maintenance events.

A

All thoughts identified through the monitoring system.

A

Can automatically generate work, orders.

A

And dispatch them to the corresponding person in charge.

A

This is our exception: handling system., the system catches all the exceptions.

A

Through locks and notified o&m personnel to deal with it.

A

If there are any advanced exceptions,, like synchronization exception, window post exception,.

A

The system will call the corresponding o&m personnel.

A

Through the phone interface and ask them to deal with the exception.

A

Hardware information,, just like I presented in the miner, show and tell.

A

I believe designing a good fiber mining solution.

A

Is like building a wooden bucket., no matter how high a bucket is,, the capacity of the bucket depends on the shortest plant.

A

And filecoin mining is different., it is different than any other computational crypto mining.

A

Like bitcoin or ethereum.

A

fiber mining is complex,, especially when you are mining at a meaningful scale.

A

If you are sitting too fast, for example, or too slow,,.

A

So that other components have to wait and stay idle,.

A

Then you are probably wasting the hardware performance.

A

So the key is to is to find a balance.

A

With cost effective hardware is so obvious that cost efficiency is another metric.

A

Of filecoin mining solution.

A

and as a filecoin miner,, you just need to find the right hardware.

A

Instead of the best ones., I made an analogy that smile,, it's like running a uber using lamborghini versus a honda civic.

A

The hardware requirements for filecoin mining are.

A

Are highly related to the computational resources needed to,.

A

To see what sector and generating regular proof of space time for every ceo to sector.

A

and these operations are computationally expensive.

A

And things like algorithm, implementation,.

A

Sector size would affect the hardware requirements.

A

So, let's focus on the current main net, with 32 gig sector.

A

First,, as a storage miner,, you need a cpu.

A

With at least 8 cores.

A

Also, the cpu has to support an intel, sha extension.

A

If you are choosing in vcpus,, you will need at least a sin. Mecca, architecture or later.

A

And if you choosing intel cpus, you will need,.

A

I believe is ice lake, processor or later.

A

Secondly,, you would need at least 128 gig of ram.

A

And as indicated in the icon box, there.

A

Should be, this should be complemented with a 256 gigabytes.

A

Of swap on fast nvme, ssd storage, medium,.

A

And a powerful gpu could significantly speed up.

A

The snark computations.

A

Effective team has provided a list of supported, gpus.

A

And then, but if you want to choose to use other gpus,.

A

You can definitely do that., but you have to manually enable them.

A

And remembered, please don't make the amd gpus.

A

And nvidia gpus, together as it will cause issues with open cl.

A

And the disk selection is is really important, as well.

A

The fact that network parameters are over 100 gigabytes.

A

And needs to be read and verify during miner start.

A

So,, you will need at least one terabyte.

A

In the ebs data disk space for cache, storage.

A

And this disk will be used to store the data during the sealing process to to cache files and parameter.

A

And to to serve as general temporal storage, location.

A

And on top of that,, you will need additional hard drives.

A

For the final stage, sort of final storage of seal sectors.

A

And lotus chain and etc.

A

please know that the above stack, I was talking about our general requirements, assuming all the mining operations are running by the same machine,, which is different than our solution and requirements.

A

Showing in the current slide.

A

And, like I mentioned earlier,, we've managed to separate everything into three different parts.. Another thing I like to highlight is that we are ready for the algorithm algorithm upgrade from sdr to nsc.

A

And that's actually one of the reasons why we chose current architecture and hardware designs.

A

They're built for algorithm upgrade.

A

and I believe, to make filecoin successful project.

A

We need to to to move most of our parties,.

A

And most of the parties in the psychological system closer to users., the current storage service provider could only go down to the idc level.

A

and I believe, one of the most important milestone.

A

Of filecoin project would be the time that we lower the hardware requirements.

A

To a level where everyone could mine at any scale.

A

And, as a storage miner,, we want that milestone to come sooner than later.

A

It may sound crazy when I say this, because we are one of the largest filecoin miner in the world..

A

Reducing the hardware requirements would lead to more miners, then increase the cost per filecoin. We mine.

A

But the key is to find is key to treat a filecoin mining as a long term. Thing.

A

To be successful, miner of filecoin, the filecoin network needs to be a successful network. First.

A

And that's why we are supportive even to most of the miner, our friendly parameters.

A

In the filecoin crypto economic model.,.

A

It's super exciting to imagine the future of distributed web.

A

I believe filecoin would become a big part of it, where we could, or where we will see loss of storage, miners,.

A

Retrieval miners and repair miners.

A

Hardware requirements won't be a barrier anymore.

A

With repair mentors help, people could probably mine filecoin.

A

Whatever hardware they desire. and whenever they want with flexible time, framing,.

A

Instead of 24/7,.

A

People can easily transfer petabytes of data with extremely cheap bandwidth.

A

I think this is the end of my talk.. Thank you so much for watching.

A

I think, yeah, two minutes over the time.

A

Sorry about that. next,, I think I'm going to turn it over to the next speaker.

A

From storagehive.

B

Yeah,, so my dogs going to be,, I think I was in showing dell space race for,.

B

It's going to be a similar kind of information. We will cover,, but definitely a quite a different approach.

B

To what neo just talked about.

B

So on my site,, I kind of looked at,. You know,.

B

As a small miner, solo miner,, I do actually have two other people in my company as well, that's,. That's helping out., so we're not quite solo,, but certainly on the small side, for now,.

B

You are quite keen and interested on the on the filecoin platform,.

B

I want to understand the economics of the platform and how we can scale it up., so really kind of focus on you know, how to build a platform as cost effective as possible,.

B

And at the same, time, maintain liability.

B

So kind of a lot of people talk that,, you know,.

B

What are they doing, and what worked, I actually want to step back a little bit,, as my wording says there, you can kind of learn from my mistakes got scratched out, you know, experimentation., so that kind of thing really key for all people that might be new..

B

I know, the process has been around for three years.

B

but now, of course,, as it's going to get more popular maintenance has launched, a lot of the crypto news,.

B

News sites are talking about it., so more and more people are going to come into the platform and and want to participate. so hopefully,. Some of the information I will share will help people speed up their process of getting into the platform.

B

it. Certainly it needs to be reiterated. For.

B

This is not like you know, bitcoin or ethereum, you don't just buy a system, plug it into the network, and it's up and running.

B

You know, it's it's a,, it's a complex platform,.

B

There are software, involved,, there's hardware, storage,, networking, internet,,.

B

All those things have to work really well together,.

B

To just work in the first place. and then once you kind of get to that level,, then the next level is to be able to actually do it cost effectively..

B

So it's not for the faint of heart to get into this.

B

However,, if you do is successful, it's really kind of exciting. Once you get to that point.

B

So let me kind of jump into talk. What I tried to focus on is not on the high end,.

B

But kind of on the more on the you know, single person, small end miner, what kind of like, you know,? The bare minimum requirement.

B

That you, at least for now,, I know the systems are changing,. Things are being optimized, as new. Also mentioned,, you know, he's,, you know, highly recommending that requirements do come down., but at the moment, the requirements are what it is..

B

So as of, you know, today, october 20th, 2020.

B

This is kind of like the minimum requirements that you need to get a miner up and running.

B

There are like three major steps to the sealing process,.

B

There's pre-commit one, pre-commit two.

B

commit one is there,, but it's like it runs so fast, you don't really notice it,. Nobody talks about it, and then commit two..

B

So it's really kind of the three major steps are pre-commit one, pre-commit two and commit two.

B

And so on, pre-commit one,, this step for now on. I know there are actually some changes that are being worked on it's in test at the moment.

B

That will allow to use multiple cores.

B

So that's will be coming down the line pretty soon, but as of right now,. It's still at this point in time,.

B

Heavily based on just cpu.

B

And note no gpu at the moment for pc1 work.

B

And so kind of on the bad side.

B

yeah,, like I said you need to isolate or newer intel cpu.

B

If you want to go that route., but yeah, in general,, most industry cpus, don't really work well,, especially on the on the lower end systems.

B

So I'd recommend that you have an amd based cpu, especially rising fives, are really cost effective.

B

At the moment to run those as the multi threaded option,.

B

Of pc one that's going to become available,, then the rise in sevens and rising nines will probably become more popular. for for pc1,, but for right, now, rising. Three and five is like really cost effective.

B

To get your pc1 rig up and running.

B

But on a gypsy side,, what I found is nothing like you have to kind of be careful.

B

On the bottom end, machines, a lot of them will only support like 32 gigabytes,, sometimes only 64 gigabytes..

B

If you do get a motherboard with that, are two chipsets,.

B

I know the b 55 is one,. I think the 450 as well.

B

But the b 50 definitely does support 128 gigabytes.

B

On the motherboard, because you need at least 64 gigs.

B

Available for every pc one process that you want to run.

B

So with having 128 gigabytes, available,.

B

You will be able to run at least two pc1 processes on that one system.

B

Soft face from what I've learned is. You do actually need a little bit more than what the official documentation says. I've noticed that the way the software does the calculation for. What's available,, you know,, there's overhead with,, you know, other processes and things that's already running on the machine., and you need to make sure that you have space for that plus thing for filecoin to run..

B

So it's always wise,. You want to make sure I've done it. 16 weeks,, I don't see gigabytes available. swap space, again, should be on an mv storage to make sure you can swap really fast.

B

The space. yeah, this is the part where it's doing all the work.

B

and you certainly want to be on ssds or nmv..

B

For the for the work, that's being done and kind of human being and doing your deals and different steps coming in.

B

I know one process from a 1632 gigabyte, sector,.

B

It will actually grow to about 500 gigabytes of workspace.

B

As is doing its sealing process.

B

and yes,, you need probably like bare minimum one. Terabyte, two terabyte is certainly more comfortable., but I even seen on some workers,.

B

I use four terabytes because, depending on you know,, if there are, you know, some tasks that are waiting for deals,.

B

Or it's kind of waiting for a process or task to get transferred to another worker,, as all those waiting has happened, it's good. If you have extra space, so that the system in the meantime can still actually start sealing additional additional sector.

B

Networking initially started with one gig,: don't recommend it,. I mean, those 500 gigabyte workspace, when a pc1 gets transferred to a pc2 worker,, it's 500 gigabytes that needs to be moved around on the on the network., and that takes like almost an hour and a half on a one gig link.. So you certainly want to look at 10 gig and an algorithm in the prices have come down quite a bit on 10 gig,, so it is becoming reasonably affordable..

B

And if you only have a main miner, and a few workers,, you can probably even just directly cable them,.

B

You don't necessarily have to worry about having a switch just got to you, know, figure out your outing and stuff so that all the machines can reach each other.

B

And yeah, and on the pc1, this at the moment, no gpu needed..

B

So that's kind of like the pc1 process, as kind of the bad side and good side.

B

That you want to use., then pc2/c2 in my environment,.

B

I actually run them on the same set of hardware, because at the moment, they do kind of need similar requirements.

B

Both processes do need more threads and reasonably moderate amount of memory and and gpu.

B

That's kind of the main driver difference between having different set of hardware for for pc1.

B

And different hardware for pc2/c2.

B

So again, on the bad side, yeah,, you don't want to use any cpu, that's older than five years. memory,. You got to be over 120 gigabytes or more..

B

Yeah,, if you again, in general, anything we while you sealing data,.

B

Spinning hard drives as specific as a single hard drive.

B

Is not going to get you far very fast., it has to be honest, as these may be. One comment.

B

Also notice these ssds,, you got to be careful with like really low end consumer level. Is these ssds.

B

on one of my workers,? I did kind of go really low, end.

B

And like shortly after spacerace, that node that actually die because the enemy, like literally got burned, up.

B

Because it can only take so many rights during its lifespan.

B

So you do need ssds and you are going to need you know,.

B

Higher level like enterprise level ssds that can maintain and handle quite a lot of, writes.

B

And rewrites on on the disk.

B

Gpu wise as well.

B

You want to make sure if you are going to go a little bit lower end gpu,.

B

It should have at least six gigabytes or more.

B

Memory per processor on the on the gpu card.

B

Because the filecoin process allocates around four and a half than a white four and a half we had number, but it does allocate like four and a half gigabytes of memory on on the screen. Card.

B

so do get a card with at least six gigabytes.

B

Allocated on it. actually,, but I've seen some times it does..

B

You do get some errors on. It sometimes stolen your location..

B

It is still better to have like a thin api or something that has at least I think it has 10 gigabytes.

B

Of memory on that card., so that's kind of like the sweet spot on the, you know,.

B

Entry level card would be the thin api.

B

I mean, in general, on the awesome side, you do need more fit to do the work of pc2/c2.

B

So arise in five, or seven and nines work really well,.

B

For these, in general, against ops, base,.

B

It's got to be a little bit higher than then the memory that you have in the system.

B

(Coughs gently), then you're less so on the post,, just kind of for people starting out scratch I've seen like over and over and over on the message boards,.

B

This is kind of like the main thing that they tripped over.

B

Once they got,, you know the pc1 and pc2 working.

B

That's not too hard to kind of get accomplished.

B

But probably the most difficult thing is to get you're the window post to complete successfully.

B

In a timeframe.- and there is with it's these lotus miner.

B

Proving deadlines command that you run,.

B

It has 38 48 sections as basically a 24 hour window.

B

Of 30 minutes sections,, where every 30 minutes, it does your proof. Some of the work that's already been sealed.

B

the challenge that it sends,. You need to be able to do all the work,, retrieve the data to the hash calculation and send the results, put it on the message, pool, and then send it back to the chain.

B

and all of that needs to happen in in 30 minutes., and if you don't have adequate hardware,.

B

That's why you're going to have so when you see faulty sectors,.

B

Or you lost all your power on the network,.

B

It's usually because your your posts are just not completing in time., I initially did start with a ryzen 5+.

B

As my my main kind of system during the post.- and it was just not fast enough.- I did upgrade to the to the amd threadripper,, which works really awesome..

B

But I would say now that I've learned a bit more,, it's probably overkill., so you can probably get away with.

B

If you have a dedicated machine for the post, a ryzen 9.

B

Should do the job as well. and again, good gpus, at least at navi or better.

B

And the memory and display requirements that you suddenly be able to get you through your through your posts. (coughs gently) and just to kind of quick reference,. I mean this is kind of how I monitor my environment got a whole bunch of buddy stations up and running and seeing what all the systems are doing. Is it's quite feeling good to see that everything is actually working and and seeing some data.