Filecoin Filecoin Liftoff - the Main Event, 22 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Filecoin for Solo Storage Providers - Part 1 with Benjamin Hoesjbo

Description

You’re ready to mine Filecoin but need help getting started. Join Filecoin miner Benjamin Hoesjbo as he shows us the optimal setup and steps to becoming a solo miner.

Join us for Filecoin Liftoff Week, an action-packed series of talks, workshops, and panels curated by the web3 community to celebrate the Filecoin mainnet launch and chart the network’s future.
https://liftoff.filecoin.io/

Events take place all week, October 19-23, 2020. #FilecoinLiftoff

For more information on Filecoin
- visit the project website: https://filecoin.io/
- or follow Filecoin on Twitter: https://twitter.com/Filecoin

Get Filecoin community news and announcements in your inbox, monthly: http://eepurl.com/gbfn1n

Chapters
00:00 Introduction to the talk
01:45 How do you balance the mining setup?
04:00 Filecoin Mining for Solo Miners–Experimental setups
06:00 Discussing Performance & Metrics
09:15 Minimal setup–3 servers
11:00 Alternate setup options
14:10 Running a server on the network
14:30 Larger Setup
15:30 What does mining with a smaller setup look like?
19:00 PC1–Worker Specific Pointers
22:07 Smells like burnt NVME
24:00 PC2 –Worker Specific Pointers
30:14 Miner Specific Pointers
37:50 Space Race 2/Mainnet Set Up
42:20 SR1 Intel Mining Rig–Old Hardware & Results

A

Yeah,, so my name is benjamin and I've been participating, now,.

A

And yeah,, since we did the calibration network.

A

Been a lot of experimenting with the mining setups.

A

And now I also kind of presented it already,.

A

During the spacerace., so we figured out that, you know,. Why not spend some more time on actually going into this kind of solo mining, setups?.

A

So one of the things that I also said last time was that.

A

I'm using an unmodified lotus version,, meaning that I generally don't go in and modify or optimize or do any hacks.

A

To lotus, so I kind of have to work with the set of,.

A

The software, how it is.

A

so how I do that, it's more like trying to make the hardware fit and trying to, to make that work as good as possible.

A

A

I showed some pictures of some of the equipment.

A

During spacerace one,, which was basically just taking.

A

A lot of hardware and trying to stack it up and to see how much storage we could get on there.

A

I'm going to show you some old pictures of what it looks like today, because obviously it's not like this., but the general idea for what we're going to talk about today is the sizing.

A

So now you see the rig here with some different computers.

A

And the general problem, when coming to being a solo miner.

A

Is that when you have quite small mining, setups,.

A

You really need to have a focus on. How do you balance.

A

The mining setup., so that's really important. now,. I also did some experiences during the spacerace one.

A

And doing some more experiments during spacerace two.

A

But basically it comes down to now that, that I kind of try to lay out what are the different ways.

A

Of doing this, with the quite small setups.

A

So this is so far what I've been working, on.

A

Basically, what you see here is a list of different setups.

A

Coming from having a single rig.

A

And into having a kind of,, you can't really call these big mining setups, because these are not data center. Size.

A

so,, you know,, no matter what, they're, not really that big.

A

But anyway, this is kind of trying to say.

A

What could it look like when you are dealing with a smaller size, computer customers?,.

A

So I grayed out some of them and that's basically because I have not tried them by myself..

A

So the community here on slack, is very strong.

A

You can get a lot help on how to set this up in all different kinds of ways..

A

I've only personally experienced with some of these, and it's not everything, that, you know,.

A

It's really feasible to actually do, although that it might, might work.

A

So what I try to do is a basic just put it up here.

A

Saying that, you know, for a single raid, we will only have a single computer, which is a miner.

A

Then you can add on workers to that.

A

and what roles would you have those workers to do?.

A

So basically, you start from just having a single computer.

A

And then to having the biggest setup here,, I had nine when this happened.

A

So from that, I also tried to do something down here.

A

In the bottom, where we have some performance, measures.

A

So one measure that I use is, in general, how many,.

A

So I guess most of us are running a 32 gigabyte blocks.

A

At the, at the moment., so that's what I measure everything in., but basically you have an idea. About,, you know,.

A

How many blocks would you be able to do every day?.

A

So that's one thing.. The other thing that I use very much is what I called.

A

My balancing metric.

A

and that's basically how I calculate everything.

A

and I would really like, you know, if we could get out of this session today,.

A

And you also had this idea about this kind of metric,.

A

Which is basically how long time does it take between? I can get a new sector.

A

So for these systems you see down here in the bottom,, we have a lot of them sitting around two hours., so how I would interpret that is like.

A

So every second hour I would be able to pitch a new sector.

A

Or,, you know, pack, a sector with fields in it,.

A

That doesn't really matter.. It is much more saying,. You know, how much capacity is there in here?.

A

So the reason why this gate is important about how many hours between, or how many minutes between I compare to new sector is because you can actually use that metric.

A

For each of the workers., so for each of the jobs that they have to do., so basically, if I need a rig like this,, I can do a new sector. Every two hours.

A

Then I would need a pc1 worker which can produce a new sector every second hour.

A

But it would be the same for my pc2 and my, for my,, a c1 c2 worker..

A

So basically I can apply this to all of them and coming up here in the end,, which is the biggest setup that I did,.

A

Which I was using here, on, on spacerace two,.

A

I got it down to about pitching a sector. Every 24, minutes.

A

And basically, for me to figure out,, you know,.

A

What is this number? well, one thing is: I can figure out how many pc1 processes can I run in parallel,.

A

So how much can I get through there?? So that's one thing,, but then I also need to make sure that I got enough. Pc2 workers.

A

To actually handle that workload.

A

and then also the c2 workers to actually finalize.

A

And get it, you know, out of the way, because otherwise I will get congestion somewhere in here.

A

So it just taking back to what, what this is, is, that you can have a single miner doing everything on the box..

A

You can have a split where you say that you have one being the miner, and then you can have a worker actually doing all the sealing.. That would be the basics of having,. You know, a separate sealing worker and then having your miner just doing the operations.

A

Of approving the storage and so on., and so that's also a way to do it, but I haven't tried any of these solutions. and I did only, did not try with anything simple.

A

I just went for the cluster directly., so the minimal setup that I have running right now.

A

Is that I am only using three, three service now.

A

So I have my miner,, which runs the daemon.

A

And the end miner process., and it actually has a pretty good gpu,.

A

But that was just the intention of making sure that I would be able to win new blocks and not having a miner that slow.

A

That it would actually not be able to mine.

A

So that's one thing., then I have a fairly little pc1 worker.

A

Which is nothing special., it does have some memory, and it has an epyc cpu,.

A

But apart from that,, nothing really special about it.

A

The reason why I'm holding a little bit back is also because we all know that there are all optimizations.

A

Going on., meaning that,, you know,. If we get a new version.

A

Of lotus, where we now have, you know,.

A

The new features and they optimize the pc1,.

A

Then we might have double speed on pc1..

A

So now I might end up having a pc1 worker which is actually producing the double amount of section's, power, or sorry,.

A

Time between every sector,, the half.

A

So then I need to balance the rest of it.

A

and then it might be next month that we also have optimizations going on for pc2 and for c2.

A

And that's why it's really important to understand these metrics, because every time something like that happens,, so something gets a double speed,, meaning that you can pledge now the double, well,, then the rest of it needs to rebalance to actually make it work..

A

So I'm not sure that we're going to get to see that the improvements is going to be for a pc1,.

A

Pc2 and c2 at the exact same time., so that's to take into consideration.

A

But this is actually the setup that I'm running now with the three servers.

A

Then we have another setup where you could actually,.

A

You could split it in another way., so right now I have a separate pc1 and then I have another worker with just do the pc2 and c2, which basically, in this set up.

A

Just needs to spend about, if it could do that within two hours,, then it's all good.

A

So both processes within two hours, because my pc1 worker would only spit out a new sector every second hour.

A

so,, you know,. That would work.. But another way to do it is actually to say that,, you know,, you might have your pc1 and pc2 on the same machine.

A

And you can actually do that with the newer lotus.

A

But the problem could also be that, if you do this, then your pc1 gets really slow, because pc2 is taking up some of the cpu resources.

A

but, in general,. This could be a good idea because you would use the pc2, which does not use much memory, and you would use the gpu or the pc2 and then generally leaving pc1 on its own.

A

So you could kind of share the same server for these two things., but currently I don't think we are, it might work,.

A

It might not,, but at at least, it is a way to do. It.

A

But not the way that I have done so far.

A

And then, for this set up here, you would have,.

A

You would have a separate c2 worker.

A

Then the other clusters that I've been working on so far has been the smaller set up here, where you have a dedicated worker for each role.

A

That's how I started., and that turned out to be a really good way of doing it.

A

In the early lotus, because a lot of problems, were,.

A

We were facing a lot of problems when we started spacerace.. Basically, the scheduler having some bugs,.

A

Which means that, you know, for some,.

A

It didn't really work at all.

A

I did not see much of that. and I believe that one of the reasons why I was not so,.

A

So harmed by these kind, of,, you know, bugs and things that need to be tweaked in lotus- was because keeping each of the jobs. On a separate note,.

A

Mean that I had got,, I got really good control over how the scheduler would assign jobs, because basically one job could only go to one worker.

A

And when it was done,, it would be ready for the next one.

A

and not having multiple jobs running.

A

And having to fight about the resources.

A

So I'm sure this has improved a lot in the later lotuses.

A

It's just, you know, for the early days,.

A

This worked really good, having just dedicated roles and also having this flexibility that you could basically scale up and down for each of the resources., and I guess this makes more sense, the larger you go,.

A

And if you go into data center size,, then obviously.

A

You would have, you know, much more flexibility.

A

In how much resources you put at each of these workers.

A

But for, for my minimal setup,, this was actually what I was going with. and I was almost hitting about a gigabyte.

A

Of the storage every day., so I got a bit over 10 terabytes in the spacerace one.

A

But I actually did a couple of different miners.

A

Just to try things out., but I'll just come back to that.. So then we had the bigger set up here.

A

And basically, what it did was take,.

A

I actually had a big server,, which I split in half.

A

And having two different workers running, one on each cpu.

A

So it had one terabyte of memory and it has two cpu's.

A

And basically, I just split it in half., but that was a pretty powerful set up for actually running also on spacerace.

A

And then to match that,.

A

Actually, the pc2 is not that problematic. If you have,.

A

If you have this, this kind of setup here,.

A

Because it is quite fast., it can actually,. I think it can do a new sector.

A

Every 25 minutes., so that would actually be all that you need., but then on the c2 side,. That gets a little bit heavy again, because each one takes about an hour. Each.

A

And then for the rest of the pc2,, I did like a big rig and actually it's a little bit oversized on some of the things. But it turned out to be a good idea, because some of the congestion that happens when you have to shut off the miner and start it up again and so on that actually takes some time..

A

So just to get back to that,.

A

I also showed this the last time.

A

How to actually try to visualize what is actually going on.

A

Now that we are mining in a smaller setup.

A

So what I tried to do here is show: what does it look like.

A

If you have a miner that can, well,, you have a pc1 worker.

A

That can do six sectors in parallel.

A

So that would be one of these workers that I just shared with the 512 gigabyte memory.

A

So, you know,, it's not small,. It's quite big.

A

But then again, you know,. This is just a setup to show what is the dynamics.

A

Of this kind of a setup.

A

and basically the pc1 is running six,.

A

Six sectors in parallel,, but they are all pledged with that,.

A

You know, that hour between them., so I wouldn't sit over a,. You know,, just not up to six pledge sectors at once.

A

Because what happens is that in the stock.

A

You underestimate how much nvme storage you actually need.

A

And then it, in the end,, it might be too much..

A

Another thing is with the memory.. It might also be if you do that, that actually doesn't figure out that you actually.

A

So what I do instead is, I actually have a pretty slow startup,.

A

A

You know, catch one sector at a time.

A

Right,, so what happens is when you do it like this.

A

Then you have in this set up a new sector every hour coming out of pc1,.

A

That's basically what happens. a sector comes in and a sector comes out. Every hour,.

A

Meaning that I would have one hour to complete my pc2,.

A

Otherwise, I would get congestion..

A

I would then have it sitting in waitseed.

A

And waitseed is fine..

A

I don't really care about that. Much about my time.

A

For packing and my time per waitseed,, because what happens is that the data is sitting on my nvme drives,, which is fine,.

A

As long as I have the capacity,, but I'm not spending any compute resources., so I don't have to take them into account. For, you know,.

A

Sizing the compute power I need., but in this set up, basically,, I have an hour to do every sector., so I have an hour to do the pc2..

A

I don't care about waitseed, but then I have an hour again to do the c2.

A

and as long as I can do that within an hour,, I won't have congestion in this setup.

A

and basically I had this running just like this,.

A

Like this set up right, here, the double set up of this, and it run very, very nicely.

A

As long as we're not updating lotus or doing any of this stuff,, the brand outtake and all these kinds of things that will screw up everything, as long as we are just in a normal,, steady state,, this is very, very stable..

A

And actually what happens? Is it's so stable that if I just, you know, set it to automatically.

A

Just set the sector and I come back the day, after,.

A

It is still going. and so the variance between how long time it does take.

A

To choose to process and seal each sector at any stage here.

A

Is pretty predictable. and I haven't seen that much variance., so that's pretty cool because it means that if you get into this kind of scale,, then it can actually be run. Pretty,.

A

Pretty quick without that much maintenance.

A

Unless something breaks, of course.

A

right., so what I wanted to go into is basically just go through some of the kind of like we call them like archetypes of workers.

A

Because the thing is that you can't just say that one worker would be the perfect set up.

A

Because it really depends on how,, you know,.

A

How deep your pockets are,? How much you want to invest.

A

And also some people have heartburn lying around.

A

Meaning that that might size some of it and then you need to be make the best fit.

A

And then, you know, in general,.

A

This could just look different depending on how you want it to look.

A

But I've seen these kind of different setups here.

A

I haven't tried the high one, so that's why it's a grayed out again,, but that would be like a pretty strong single cpu.

A

Pc1 worker. and again,. This is thought of the idea of having them individuals,, so not combined., which you of course could do,. But then you need to kind of,.

A

You know, also combine these numbers here.

A

But the thing is that this is thought out of a setup.

A

Where you are using a faster network.

A

So you have a 10 gig network between the nodes.

A

So you would have dedicated worker for doing pc1.

A

And what is stated here is that you can have this small worker,, which basically is what I have right now running.

A

Just he was starting my main net.

A

And basically, what you want is you want some kind of cpu,.

A

Which is definitely an a and b for this one.

A

So far we will have intel joining the party.

A

But it will be with the ice lake,. I think it's called,.

A

And so far it's only mobile processes.

A

and I think we will have to wait a bit for the ice lake sp,, which is the xeon version..

A

But sometime we will be here with the gen four.

A

Pc express as well., but it is pretty slow,, but that specific version of intel.

A

Cn cpus will also have the,.

A

A

The sar instructions,, which is specifically needed.

A

For this process, as it is today.

A

because another point that I want to make is that, although that I might come up with these archetypes now,, it is really important that you try to keep it flexible.

A

And not, you know, make each worker so individual.

A

And so customized that it will never be able to reconfigure.

A

Because we don't know in a couple of months, if we will be using these gpu processes.

A

Could be. so basically, you know,? It would be pretty stupid to make something that you where you need to go and buy new hardware when things change..

A

So that's one thing to take into consideration., we'll just get back to that., but that was just to say that you can have different sizes..

A

Another point that I want to make about this is: I have been doing most of spacerace.

A

And also starting maintenance here on consumer nvme, drives.

A

And I just had this funny calculation, where I actually figured out, so how much data is flowing, because I can see that in my network traffic, but I can also just, you know, use those snowball metrics for how big are the unsealed blocks that need to move between.

A

Pc1 and pc2., and I figured out that my poor.

A

Normal consumer nvme drives were doing just.

A

Like two and a half.

A

Drive rides per day, two and a half., so they are technically,.

A

You need to be able to handle about half.

A

A drive a ride., so basically, what it means is, if you have a two terabyte nvme,, it is designed to ride about one,.

A

One terabyte a day.

A

and I was using it a bit more, more like five terabytes a day., so you could argue that I might be burning these disks. Down.

A

And I,: that's why we make these jokes about that it smells like burned, nvme and someone,.

A

So far I haven't broken anything,, but I'm pretty sure that they are in pretty bad shape compared to when I started., and the point I want to make is basically, if you want to make a miner that needs to be ongoing seeding for years,. I think these consumer nvmes,.

A

They will burn in the end., so what you need is basically,, you know,. You have to pay.

A

Because enterprise nvmes.

A

Are way more expensive, but also they do provide this kind of set up that you need.

A

Where they can actually handle this workload without dying on you in the next couple of months.

A

Or half year at least.

A

yeah,, so that was just for the pc1.

A

We kind of have the same on the pc2..

A

What is different here is that if you have a dedicated pc2 worker,, you actually don't need that much memory, because it in itself is not really a memory-consuming process..

A

You can also do with more on this course.

A

Where on pc1,, you need to go with a high clock, frequency.

A

I don't think it's that needed here in pc2, because pc2 is in general, pretty hard on the gpu.

A

And you will see that by, if you have a dedicated pc2 worker. That only does that.

A

And it doesn't all the time,, you have a pretty hot gpu because it is hammered all the time., but that's pretty good, because it means that you,, your computer might not have to be that,. You know,.

A

The rest of it doesn't have to be that hot., but what does need to be there is. It is the same with the nvmes because, as you have nvmes in the pc1,.

A

Which need to create these kind of large si sectors,.

A

They need to be transferred to pc2 before they can be processed, there.

A

and that,. For that you need the network,.

A

You can't do one gig,, you need the 10 gig for that,.

A

But you also need an nvme in the other end that can actually, you know,.

A

Get that data in. yeah,, but the thing is that you also see here that these kinds of set up using a 2070 super, or a 2080ti, they can actually handle quite a lot.

A

They are pretty fast., so most of these, you know,, can actually handle.

A

A pretty big workload., but we've also seen that if you do go into some of the,.

A

The older, like 1080s,.

A

Not ti,, but the eight gigabyte versions,.

A

It can start to slow a lot., so I had one older machine with a 1080 that was small,.

A

Like one hour and 15 minutes to do a pc2., so these cards are really good at it.

A

Another point I could make here is that we have some very lucky people.

A

Experiencing with the 3000 series.- and I think the problem there is that basically.

A

When you do the bellman,.

A

That operation,, it needs to be optimized for the gpu.

A

And now the 3000 series comes in with a new code account.

A

Which is awesomely high and a new count of memory.

A

And there is this calculation: when you give it, how many protocols that you have, you know,, then it tries to,, you know, figure out what is the best set up.

A

but currently, you know,, I think that's a lot more juice in those dpus that is currently not being used.

A

Because you probably that needs to be also optimized again.

A

So it seems like right, now,, the 10,, the 2080ti.

A

Is probably the most optimized card for doing these processes., so it's still a good choice and the other ones will probably be better in the future,, but right now we don't really see it that much.

A

So on the c2, basically,, you also like a lot of cores here.

A

Because some of the c2 processes, also,.

A

A lot of the cpu intense actually.

A

So,, so that's good., a thing that I didn't say with a pc2 and the c2 is that you can actually use intel cpu's on this.

A

And it works fine., so I have an intel worker right now doing. Pc2.

A

And it does it, the pc2 and also these 20 to 25 minutes.

A

So no difference at all., but then again, you know,, if you are talking about high, call counts, and you will be talking about xeons,.

A

That's really nice if you already have them, but I experienced that it works better on the gold,.

A

On the gold, scalable versions., I also have some older, like a 2680 b4s., and although that is a b4,, you should think that.

A

It's like fairly new,, but it seems to be pretty lame.

A

And also it's using a lot of power., so it's kind of running on all the cores and spending all the power it can do, but it's still slow. So I don't think it's really,.

A

You know,, if you're going to buy new hardware,, then buy probably new instead of buying that kind of old stuff, hardware.

A

But it is important to say that the cns does the job.

A

Just as fine on these workers,, not on pc1.

A

Yeah,, so that's pretty much it.. The differences I just pointed out here is that.

A

While I was able to do a pretty good time on the 2070 super.

A

On pc2, then, actually here, the difference between these gpu's is pretty significant in c2.

A

So I would say for c2, specifically,.

A

It's a pretty good idea to have a strong,, a strong, cpu, or gpu, sorry.

A

Another thing that we also seeing is that a pretty big, epyc cpu connection..

A

They also do the whole c2.

A

In one hour and 45 minutes,, I think it was.

A

So if you're looking at this over here with the 2070 super.

A

Actually, it's about just as slow as if you just had the epyc.

A

Cpu actually doing the c2..

A

So that's interesting if,, if you're not going into having a,.

A

You know, expensive, gpus and doing it really fast..

A

So that's another thing,, but you see over here, when going back to pc2 that the difference between the.

A

These two gpu's, when doing.

A

A pc2, it's actually not that different. and probably both could work for having a pretty big set. Up.

A

So what I did was basically I had one of each., so one of each of these, and in general, they are in average about a half an hour.

A

If you combine them. so I would,, I was able to reach about a sector.

A

Every 15 minutes on this set up here combining these two.

A

Yeah,, so that was basically what I wanted to talk. About,.

A

What is interesting on these parts.

A

I also have some things here on the miner.

A

now that the miner in itself is much more difficult.

A

Because the workers, you know, doesn't need to be.

A

Very secure, high available and all that kind of stuff.

A

Because the worst thing that can happen if a worker breaks down all the (mumbles) burn and so on,.

A

Is that you lose the sectors that it's currently working on,, which is like not a big deal.

A

and probably, if you even have more workers, then you can reorganize them into keep on their sealing.

A

So it's not the biggest problem, but the miner.

A

Is a much bigger issue. and the thing is that,. You know,, I'm not going to say.

A

That you should do it in one way or the other, because in the end, you know,, if you miner breaks and you lose your,, I don't know,.

A

Kind of your meta data and so on and can't restore your sectors,.

A

Then you are like super screwed., so you know how to set this up in the best way is a,. It depends on how much money you got and how much risk you're willing to take,. I guess.

A

So having a single rig to do everything.

A

And not having any raid or anything that is maybe the more.

A

Like a cowboy style and as long almost it's making money.

A

And you get some of it out and it's good,, but you know if you lose it all,, then that's just it.

A

But for these setups, I basically just try to say that,, you know,, you can put it into three different types.

A

You have the ones which use a few like I do., so basically you, you rely on having a pretty strong gpu.

A

To do the window.

A

Post, which would be the proving sectors and making sure.

A

That you keep your, your power,, your storage, power.

A

And also the winning post,, which would be, you know,.

A

When you get enough power to mine blocks that you're actually fast enough to provide that winning post to actually be.

A

Eligible for a block reward., so that's also an important part., but what I learned, not by my own experience,, but what I learned from here from the last a month.

A

Is that you also have the opportunity,.

A

If you have a pretty large cpu, rig, you're, actually able to run this without a gpu.

A

And you're also able to mine blocks.

A

But the general thing here is that it probably needs to have.

A

A lot of cores, because if you are getting,, you know,.

A

A lot of the utilization by the daemon and so on,.

A

Then you might have issues on actually,, you know,.

A

Making your, your window and your waiting post in time.

A

So that's really important..

A

Another thing that I put in here is that, in general, you know, that your os data, and so on, probably should be.

A

On a raid1., you probably also need to do the backup,, which is also a new feature that you can do.

A

Of your data store. so just to keep that safe, keep your one safe and so on.

A

But at least you need to raid1, so you can continue working., but then again,, you know,. If you have a small rig,, it could actually be okay to just have backup.

A

And then, if it breaks, down, just rebuild it.

A

So we have seen workers are like, sorry,.

A

We have seen a small miners actually moving their setup.

A

From one computer to another., so it is totally doable.

A

So, basically, you could pack up things and then, if it crashes, hard then,, you know, rebuild it and get on it. Again.

A

But in general, you know,, I think that would be a good idea to have a raid1 to keep that up.

A

If one drive breaks down.

A

and then a general on the, on the data disk, I have so far been using raid6.

A

And I haven't seen much problems with it.

A

So I would generally say that if you have like small setups.

A

And once you make this work without too much trouble.

A

Then it might be a good idea to not make those raids too big..

A

So one of the things that we see is that if you have like using the new 18, terabyte disks,.

A

Well,, do you know how many hours it takes to actually initialize an 18 terabyte disk.

A

When it can only read and write with the, with 200 megabytes per second?

A

well,, that's one and a half days.. So if you need to rebuild a raid6 that that has a broken disk is one and a half days.

A

So, you know, it,. That can be a problem for you.

A

If you get too large., another thing is that don't put too many disks in., and so my raid system generally 12 disks each.

A

But I would rather have many raids.

A

and then, you know,. If one breaks totally to crap, then you know,, I still have another power., but this area I might not be able to recover.

A

I might need to do a backup of some data.

A

But that's a,, that's in general there.

A

so one thing that happened, another pointer here is that.

A

In general, I've been using the addpiece., so when you pitch a new sector,, I have only had that running.

A

On my pc1 worker.

A

in the start,- and I don't think it's like that now, but in the start, if I also had that enabled like addpiece.

A

Was true on my other workers,, then you could get an a addpiece job that went to a c2 worker that already had two c2 jobs.

A

So basically, it wouldn't pitch or start that process.

A

Of a pc1 piece so after like two hours., so that would be pretty stupid.. I think that changed now, but still, you know,.

A

I normally just use my pc1 worker, but to do the addpiece.

A

And it takes a bit longer, but again,. I don't care about that time because it doesn't really,. You know, go into the bigger resource, calculation.

A

And the other thing that I will generally stress here is that,. You know,, when you look into this size,.

A

And you might go back and see them. Afterwards,, remember that,, although that they are each specified.

A

A little bit different,, then you might be able to use the same computer for each of these. so think about maybe having the same, the same motherboard or the same server as base.

A

And then, you know, some of them.

A

You don't put that much memory in, some of them. You put a little bit stronger, cpu in,.

A

And some of them has a weaker one,, but in general, if you keep the platform, the same,.

A

Then, you know, when we go a couple of months from now.

A

And from the, the lotus team come up with something totally cool.. We changed the algorithm,, something like that., you know,. You are still able to rebalance and fix the system and not locked into that,.

A

You know,, I just built this on a ryzen,.

A

Which can have only 128 gigabytes of memory.

A

And now I have no process that I can run on there.

A

So that would be my recommendation. and it's not saying that you should get rid of it.

A

If you already have it, just,, if you find new stuff,, then my thing having,, you know,, a bigger set,.

A

Just having this idea about that,, you could,, you know, change everything if you,, if you'd, like to.

A

Yeah, so that's, that's about it.

A

So from here on, I have some pictures of what it looks like today in my basement..

A

I know that I got a lot of comments on this last time.

A

So that was a lot of fun., but the general idea is that,. You know,, it's not enough for me to be a little bit idealistic about filecoin..

A

I also think that this shouldn't,, you know,, not be a total waste of energy..

A

So what I do is basically have my workers and my miner.

A

Hooked up to my flow heating., so what I do is basically what you see here.

A

Is the back of these two rooms.? So that's basically, where I have my miner and my worker and my storage.

A

And what I do is basically just have everything water-cooled,, which is like a basic computer cooling.

A

But the trick I do is, then I send it up to the boxes. Up here,, which is heat, exchangers, and heat exchangers, basically goes on to my floor heating,, which is kind of dirty water., but I kind of keep the, keep it separate.

A

So you have that really cool green fluid that you can see.

A

But anyway, it is clean.

A

and you heat exchange, the heat from all the gpu, cpu,.

A

And so on, up into my flow heating.

A

And the good thing about my flow heating is that it's kind of a endless way of getting the energy,.

A

You know, out, because I haven't been able to pull it hotter.

A

Higher than 40 degrees celsius so far., so it's like,, you know, how much power I put in it, basically,, you know,!

A

There is so much that needs to be heated for the motor to be 40 degrees.. So that works very well.

A

So far., so I think I have a little bit over a 100 terabyte.

A

Of accessible data so far.

A

But I also made it really operatable..

A

So what you see is,, I don't know if you can see it,, but actually I have like.

A

A quarter of my actually drive base.

A

Populated at the moment., so there's enough space in here..

A

Another thing that is pretty funny in this setup is: I have these custom made., so I wanted something that, you know, would not fill up like a full big rack, because this is in my basement at least., but so what I did was I had these custom made,.

A

These disk arrays,, so they basically only can contain.

A

The front part and they only like a 25 centimeters long,.

A

About 10 inches, and then nothing after that.

A

Because what I do is basically, I just take the power cables from my,.

A

From my power supply on the miner and just drag them.

A

Into powering the array., so I don't have additional power supplies.

A

And also the access cables is just coming directly from the controller.. It's just using like a one meter test, cables.

A

And what that means is, I can basically run this whole array on my single miner.

A

And I'm not using redundant power supplies for this.

A

But I'm also not using that because,, you know, yes,, it's nice and we use that on all the server hardware I work with,, but for this purpose, it, you know,.

A

It doesn't really matter that much.

A

Of course,, I don't like data corruption,, but then again, you know,. It is unlikely that it will be everything broken.

A

Probably, if it's writing a sector when it happens,, then it's really bad., but apart from that,, I think this is a pretty good way.

A

And the cool thing about it is that,. You know, running the miner. Without any workers on,, I only have one power supply..

A

So another thing that I set up since spacerace one was that I actually downgraded from the 56 gigabit switches that we knew,. We normally use that.

A

On my half performance clusters- and I had something in spare., so I just used that for all the spacerace one.

A

But basically now I went back to the regular 10 gig.

A

And just use that. and in general, you know, that performs just as well and it's not really a bottleneck.. But I have this kind of combination of using a normal.

A

Rj45 for some of it. and then I also use a fiber for others.

A

So basically, all this is set up with fiber.

A

but, you know, whatever,. It works pretty good.

A

And I haven't seen much problem with the networking..

A

You just need to make sure that you're using double frames for that one.. So that's a mtu9000.

A

otherwise,. Basically you don't get the throughput..

A

So I see the throughput about a gigabyte per second.

A

That's pretty much around that.

A

Yeah and another comment that I was just making with this one is that when we did spacerace one,, I had this intel mining setup.

A

Where I'm trying to see how much can I actually do on intel, and I ended up with having everything else.

A

On pc1 working on intel, machines.

A

So the only caveat about this is that on this rig.

A

I did not get enough power to actually be able to,.

A

To win blocks., so I'm actually not sure if my intel xeon here.

A

Would be able to do that., but then again, you know,. It should also be a tpu job,.

A

Partly,, so it might just work at least for synchronization of the chain.

A

And for doing all the window posts and so on,.

A

No problems at all., so basically this is just to say that you can actually run this on pretty old hardware.

A

And, as you see here,, I also got some pretty old stuff going on over here. and you see the memory, it's not even fast and so on,.

A

But still, you know,, this is actually producing sectors.

A

So, you know,. If you have stuff like this lying around, it might be usable..

A

The only thing is probably you'll just see, that,.

A

You know, the power consumption of these old, big,. Slow setups might not be that efficient.

A

But I am actually still using the intel set up down here.

A

With the really small,, it's only a four core cpu.

A

And I've actually also been using that for c2,.

A

Which really stresses it out,, but it does the job.

A

So it's like,, you can still use that., so it really depends on what you got already in stock.

A

You can use a lot of different things,, but it really comes back to the beginning. With, you know,.

A

Having these kinds of calculations.

A

Where you're actually able to calculate so, how fast can I do pc1??

A

So how fast do I need to produce my pc2 and c2.

A

To make sure that I'm not doing any congestion, and also that I'm not oversizing my pc2.

A

And c2.. And if you go to the start of my presentation,.

A

You will actually see that the big setup I had for spacerace two was actually quite oversized.

A

On the c2 and the pc2., and that was actually just part of,, you know, figuring out who, or what of these workers was,.

A

You know, performing good and who wasn't.

A

So that was too much resources in there.

A

but,. You know, from the example that I gave with that timeline,. That's a pretty,, a pretty solid.

A

And pretty strict setup of how to run it right, now.

A

And it also comes with that thing about that, you know,. If we did this optimization with pc1.

A

In a couple of weeks or earlier,, then we might have a pc1 worker which now has doubled performance..

A

So now, basically, that's why I keep my pc1 worker.

A

Pretty small at the moment and do not have large amounts of memory is because,. You know,, if that goes to double performance,.

A

Then basically, the rest of my setup now needs to be doubled.

A

Until data has also been optimized.

A

But these are kind of the rules as long as we go with these smaller setups.

A

So I guess we are kind of coming to the end here and I think we also almost out of time.

A

This one is just saying the setup that I also showed in the spacerace one., so you can actually do it fairly cheap., but again, you know,. This does not have the enterprise nvmes., but that's pretty much everything for me.