Filecoin Filecoin Storage Providers, 28 Aug 2020

Previous Meeting

⏯

youtube image

►

From YouTube: Filecoin Miner Show & Tell - #1

Description

Join our top miners and learn about their architecture, hardware, and teams.

A

Hi everyone thanks so much for joining us this evening, um we're just going to wait a minute or so here um and let a couple of more people join, um please let us know where you're tuning in from this evening. um I am based in scotland um in the uk. It's about two o'clock in the morning here so bear with me if I'm a little bit slow um but we're just over the moon to have you join us, our very first uh minor show and tell for file coin.

A

Okay, I think that's everyone.

A

Great brilliant, okay, let's get started so like I said. Thank you so much for joining us for the first ever my falcone, minor show-and-tell um you're all here because you're part of a team, um that's mining, falcoin and space race, our incentivized test net competition. By doing so you're creating the building blocks for a strong, resilient and secure network.

A

Tonight's event, isn't the only one that will be running throughout space race, we're running a series of events and you can find out more on space, race, dot, falcon dot. Io we've got amas, we've got workshops with engineers from valcoin, we've got the minor show and tells, and we've got a weekly space race report where you can get up to speed on all of the action that's taking place throughout the week, a reminder that to be eligible for rewards during space race. You must register your miner on spacerace.filecoin.io.

A

This applies to all miners, including those who have already started mining. So please make sure you register your miner so that you can be eligible for those rewards. Okay, we have a great lineup of speakers for you today. Let's get started, our first speaker is neo g neo g neil is the founder of ipfs main over to you. Neo.

B

Okay, let me share the screen number one.

B

A

B

Perfect, perfect, hello, everyone, my name is neil g, I co-founded ipfs may back in 2018 and we are the service provider inside of the uh fico ecosystem. I'm very happy to be here uh also thank you for having us. uh We've been committed to the ficon project for almost three years now, and uh it's been such an incredible journey for all of us in ipfs maine and you know, working with the protocol. Labs team has always been such a pleasure and it's really exciting to present today.

B

So the presentation today basically going to go through our fico mining solution, which includes mining architecture and the network topology, then I'm going to talk a little bit about our hardware information and our team as well. I only have 10 minutes so, let's get started so over the past two years we have had multiple architectural designs for large scale. Phycomine and after thousands of hours of testing, we've decided to use this fico mining architecture.

B

In this current mining solution, we are dividing major hardware into three different parts: the storage rigs ceiling, rigs and snarks rigs. By separating the storage and computational part, this architecture gives us really good, scalability and improves the hardware utilization efficiency on a tremendous scale.

B

So in this design we want storage rigs to complete posts on their own, at least the first step of ust to generate candidates and, at the same time, we use ceiling rig cluster for zero pre-commit phase, one aka, pc1 and pc2, and pc and and c1 and snarks rig cluster for c2 and window post.

B

We modified and we wrote part of the code and made some redundancy changes to improve the pos t success rate and by doing all that, with this mining architecture, we've significantly improved the mining efficiency, especially for p1 and c2.

B

For example, we we could manage to finish c2 under, say, 10 minutes, and there is still a lot of improvements we could do in the next few weeks.

B

I mean in the network topology we are using two different isps for each one: to increase the redundancy. The communication core switch uses, multiple switches with, say, 10, gig optical ports for stacking, which is mainly connected to lotus node and the lotus miner. The data core switch uses door switches with 10 gig optical ports for stacking, which is primarily connected to lotus, miner storage, steelworker and smart cluster, and the storage access switches have four 10 gig optical ports, plus 24 gig electrical ports, doors that door switch stacking.

B

Of course, the 10 gauge optical ports are connected to data, aggregation switches and gate. Electrical ports are connected to the storage, so each storage is connected with two network cables respectively to the two switches, and we did the door network port binding to improve the storage network stability.

B

I would like to highlight our o m platform called file manager here as part of the fico mining solution.

B

The file manager includes cndb, ci, cd and lots of monitoring system and work order system. I think I have screenshots for each system.

B

uh Yes, the cmdb can help us automatically discover the host, get the host hardware information and identify host failures. The cicd module can help us perform fast and accurate deployment whenever there is an update.

B

The business monitoring system provides comprehensive monitoring of the health status of the business from different perspectives of lotus, miner, co, storage, snark window, posd and so on, like lotus, monitor, helps us node, synchronization and mining. Monitor, helps us on order status, minor status and sector life cycle tracking.

B

The seal monitor focuses on the working status of seal workers like what is the sector id and what stage that sector is currently in. It is able to identify various errors that occur during the sealing process and deal with foss automatically. It will stop receiving new tasks until the fault is repaired.

B

The storage monitor helps us on the sector story status. A will, find storage failures actively and perform data migration in real time and make the migration history traceable as well.

B

Snark monitor focuses on the working status of snark and the resource demand of it.

B

Window posd monitor is responsible for window post working status and all the history of all miners- and these monitors I've mentioned- are all part of the business monitoring system and business monitoring, log monitoring and host monitoring. And yes, this is the host monitoring interface. All the monitoring systems can work together to identify and and classify thoughts in hardware services and networks and notify the corresponding staff to to deal with deal with them in time.

B

The work order system manages the workflow and we can track various operations and maintenance events. All thoughts identified through the monitoring system can automatically generate work, orders and dispatch them to the corresponding person in charge.

B

The exception handling system. It can catch it. It can catch all the exceptions through logs and notifies the o m personnel to deal with it. If there are any advanced exceptions, like a synchronization exception, um windowpost exception, the the system will call the corresponding o m personnel through the phone interface and ask them nicely to deal with the exception.

B

Pretty nice huh hardware information, um so I believe, designing a good mining solution is like a it's like building a wooden bucket, no matter how high the bucket is, the capacity of the bucket depends on the shortest plank, so factor mining is it's not like any other traditional computation, crypto minings, like bitcoin or ethereum fico mining is complex, especially when you are mining at a meaningful scale.

B

If you are sitting too fast or too slow so that other processes have to wait and stay idle, then you are probably wasting the hardware performance, so the key is to find a balance with cost effective hardware, apparently cost effectiveness or cost efficiency.

B

It's another metric of fico mining solution where you could get the most beefy machines, regardless of cost or find the red hardware for best cost efficiency, which also leads to a best return on investment ratio. It's like running an uber using lamborghini versus, say honda, civic.

B

Another thing I'd like to highlight is that we are ready for algorithm upgrade from sdr to nse. That's actually one of the reasons why we chose current architecture and hardware designs. They are built for algorithm upgrades. I believe, in order to make ficon a successful project, is to invite everyone in the world to do the fico mining and to make it happen. We need to lower the hardware requirements so that everyone could mine at any scale.

B

At the end of this presentation, I would like to take this opportunity to express thanks to our team for all the great effort in order to make this happen.

C

We are four days.

B

Into the space race, and uh even though we are getting attacked from time to time, our team has been you know, taking appropriate actions and conducting different types of defenses.

B

I think one of our node minor id is t01235.

B

Right now is ranking number five on the board and we have multiple nodes running up for space race and with all these miners combined it's safe to say that we are ranking top of the board not only in the power category but also in the mining efficiency.

B

Now we are having over 80 people hard working every day, not just for this space race, but also for the coming minute launch.

B

So this is the end of my talk. Thank you so much for watching, and uh I think I tamped out for 30 seconds um next. I'm gonna turn it over to the uh to the next speaker v.

B

So angie, are you doing the transition.

A

Between yeah I'm here, thank you so much neil, um you said that you got involved with file coin in 2013. Is that right.

B

um No uh 20 uh like three years ago, oh.

A

Three years ago, okay, yes,.

B

And I founded ipfs main in 2018.

A

ah Right, okay, okay and tell me like how did you come across filecoin like when? Did you kind of first um hear about the projects or um yeah? Where did you come across that I'm just curious.

B

No problem, um the first time I've heard of the falcon project is actually in another community. It's called the scicoin. I used to be a host of cycon and also as a miner at saigon as well.

B

So I was looking for like a really you know revolutionary, uh like a cryptocurrency or blockchain project, especially in the storage field. So um I guess that's just the way I am. I have a passion about uh blockchain, especially in the storage.

A

That's great, that's wonderful! Well, listen! We're delighted to have you as part of the community. It's really exciting to hear about the progress that ipfs maine our making on space race, and I wish you continued good luck. um Okay! So, let's move on to our next and also um you know if anyone wants to chat with neil neil, I know that you're active in our slack channels. um So yeah, I'm sure. If anyone has any follow-up questions, they can find you on there.

B

Okay, brilliant.

A

Thank you so much thank.

B

A

Okay, moving on to our next speaker, we have uh leigh white, who is team, lead and co-developer from six block uh over to you, lee.

C

Okay, thank you guys, yeah. Let me show my slides okay. Can you see that.

C

Okay, uh let me start yeah, I'm lee from the sixblock.com yeah, the sexpot sexbrook.com is a mining solution provider and we start to enter the falcon mining from this chinese spring this year, yeah. So it's maybe four months or five months before yeah. We are very late to be a part of the fire coin uh community, but uh we uh when the falcon launches test net too, and we feel that it's very interesting and there's a so. We started the algorithm of fire coin, like the poip, like the ziki snack part yeah.

C

We because we are so familiar with algorithm, especially as a blockchain algorithm. So we think we can do something really good in the file coin mining, and so we decided to enter the file coin mining area in this chinese screen festival, okay, yeah, the sixblog.com, it's a project, it's a enterprise which is focusing on the unicorn projects in the blockchain area. They are providing solutions for cryptocurrency mining, including high efficiency, mining programs, money imports, hardware, cloud money and the wallets yeah.

C

This sentence is written by me yeah, I think our team and we should focus on something really good. The good. We call it a unicorn project. That is, I mean the project.

C

There are too many projects in the blockchain area, and some of them are just a scam, but some of them that contains high technology that I think we should focusing on those projects which have which is good in a technology part and as well as the economic part. So that's our aiming of the sixth block as.

D

C

Know six block mean also means uh the bitcoin. You know the transaction or the bitcoin will be confirmed after six blocks. So that's why we decided to use the uh hostname six block. Okay. uh Next, I will show our solution for the file coin: mining for the space race yeah. We have two types of workers and independent storage.

C

The first type is f3. The f3 worker is more efficient for small scale deployment. It's based on the amd 39 and 60x cpu.

C

As you know, the this pu is for home is for home usage, and so it only can support 25, 56 gigabytes of ram and the gpu we used, for it is uh two zero and zero ti uh this uh as it's widely used in mining area and the ssd we use is uh http, so you can see that the f7 f3 worker is much more cheaper than the md than the f7 worker. The f7 worker is more expensive.

C

It's based on the amd 7542, which is for enterprise usage, and we also use the gpu2080ti and the ram is one tetabyte and the ssd is uh six is 16 tb. So you can see that the price of this f7 workload is much more higher, but it's more efficient for larger scale deployment.

C

We, since we are money, solution providers, so our customers uh will buy these machines from us and we make everything public the hardware, the details of the hardware public. So this hardware, the hardware specification, is shown on our website: six dot, co, six block dot com, so everyone can see it and the storage we use is simply some inter inter service. It's based on intel, e5 and the ram is very low.

C

It's 16 gigabytes and we have a lot of storage a lot of hdds in the servers it's based on 14 tb uh and there are 24 or 36 hdds on that service. So you can see that the storage is only for the storage service. Only for storage is it do not have any ability to use for computing.

C

That's our hardware solutions and.

C

When we enter this area, we find that the hardware of our providers, our of other service providers, is not transparent, it's not public, so we make everything public and, as I know, there are many miners, many guys who just use the hardware we choose, especially the amd 3960x. I think it's very efficient okay, so we can. You can see that we use in some probably widely used hardware in our mining pool, and now I can show you guys our network yeah.

C

You can see that it's very different from the ipfs main we choose to choose to build the uh our mining pool uh the main our machines are divided into many uh small clusters uh for each cluster. It only contains uh maybe less than 100 machines yeah it's built. It contains one master node uh there is lotus and the miner and the worker is used is linked to the miner through a hub, and the storage is also use a hub to link it to the miner.

C

Now for each cluster we only have four hubs: yeah one two, three four four hubs and two of them are computing machines and the two of them are in story. Machines, so for each cluster of us it only contains maybe 100 machines, it's very little and the reality is. We have many minor ids uh on the minor list yeah here I can show you our many minor accounts. You know we haven't made it public before this is our minor accounts for the space risk.

C

We haven't made it public before because, as you know, there are many attacks on the space station network and you can see that at least there are. I can show you. There are one two, three four six, maybe 14 14 minor accounts and each of them will have maybe 200 or more than 100 tb storage power yeah. So another reason is that the customers of us would like to have their own clusters and their own minor accounts.

C

So we would like to separate our machines into different classes and they are, they are making different accounts yeah. So you cannot see our accounts on the top 10 of the minor list, but they are from the top 20.. They are in the top 20. yeah. So we have so much clusters and minor accounts yeah, and we have a problem that so how we name these clusters. There are too many.

C

We cannot uh distinguish them, so we have a problem that we, how how we name that yeah we first we want to name it after english characters, but I think the english campus is too few. They only have between 26 characters. So then I I'm watching a tv series which is breaking bad. There are many chemi chemistry inside, so I decided to use.

C

Can chemical elements to name our clusters, so you can see that our clusters have each characters is represented by a element in the chemistry like h-h-e-I yeah, and we name those clusters of three nine six zero uh for the car for the cam for an element that is more active like uh air I like be, and for the clusters build off the uh seven uh seven md7 series uh you, we use the cam elements that is more inactive yeah. So that's our minor accounts yeah. We have an issue all of the accounts.

C

Now there are three servers, because the screen is that big and I cannot show off all of them. Maybe we can show it tomorrow to we can we will register on the website and you can see all of our clusters?

C

Okay, and we have do a lot of work before all the software on the mining software uh one one part, is we improve that we do something on the engineering part yeah, so we will write the task scheduler here. You know there are many different types of machines in the file point mining. You cannot use only one, only one single scheduler for all of that machines, because machines, the their configurations, are different, so we rewrite them to make it suitable for our machines yeah.

C

So we write this task scheduler part and then for the algorithm part we improve the frequency one and the committed two.

C

Besides, we will write the code of the pre-committee to using opencl because, as you know, nowadays the pre-command to the greek number two is written by something like not a c plus plus, but that the languages is something we are not familiar with.

C

So we will write it using opencl yeah, because opencl is more widely used in the blockchain mining area and that's what we do to the software and our future plan is very simple, because we are noticed before the space race we haven't, do anything to the dear success rate yeah we so before only one or two days before uh before the space race. We are focusing on the network network stability and the storage stability, and we also choose to uh have some dear priority to the schedule module.

C

So we do a lot of a lot of things. We plan to do a lot of things and do more things to the dear success rate to improve our dear success rate. Besides, we also, as you know, we also develop a bar coin. Blockchain explorer. It's named fieldfox dot info yeah. I think.

C

As you know, the explorer firefox is very widely used all over the world and I think, for the far coin. The name fieldfox is similar to the firefox and they is used easily to be remembered. So I registered this hostname and have our uh and launch our explorer on that website.

C

I think it's we do a lot of work on the spoiler and I think it's really easy to be used. Okay, here is our all of our work on the space race and the falcon mining yeah. If you can have, if you have any problems or any questions you can get, you can send email to me. uh My email is on the website. Sixbook.Com yeah happy to have a good rankings uh of on the space race yeah. Thank you guys. Thank.

A

You thank you so much lee that was really really interesting. um I can speak for the file coin team when I say we're a huge fan of phil fox. um It's just a really great way to visualize, what's going on in the network, um so thank you so much for that. I also love that you've named with chemical elements. That's so clever for your miners and- and I think um your presentation and neo's presentation together goes to show that there are many different architectural approaches um to mining on filecoin. So thank you.

A

That was really really interesting um for all of us. Okay, on to our final speaker for this evening. uh Next up we have david shin uh david is the cto for data line over to you, david.

D

So you can't handle it, so you said it's just returning to the next page right. Can I handle it yeah, great okay, so our office uh star, uh you know, separated. You know you know located in seoul. Firstly, it is a headquarter, uh is about 10 members are here and another office is located in hangzhou in china and another technical team is in the data center next to the hangzhou office. Just so, can you turn next page?

D

Okay, great okay. So the first uh you know the left top is the our data center. Is you know? Actually you know the uh today uh we have another neil. You know um this uh same name is you know you know first presenter. Actually you know they are neo, uh he's a political manager. Actually uh he was supposed to introduce himself and into what he does uh with his team.

D

uh Actually, his team is, uh you know, uh uh working very hard at the moment, because we uh normally we just gather top uh uh in the first paging. You know ranking yeah I mean, but actually uh now so we got some problems and you know uh very uh you know it's just a.

C

D

uh We are trying to solve it at the moment, so so he unfortunately he didn't come out here. I'm sorry to uh for that. So I can uh present. You know I um uh briefly uh in terms of technical area, but um rather than that, so just I'd like to introduce you know about. You know business area and you know a ficon ecosystem as well, so we are doing you know globally at the moment. So uh yes, that's good! Thank you emily.

D

Yes, um we have roles, you know, uh according to the place, we just you know their uh headquarters in seoul, which is you know the uh the capital city in south korea. uh We performing uh you know some uh uh from end to end uh management. You know uh from funding to uh you know, maintaining everything. So you know procuring some some further. You know computer devices or some uh r d as well. So, but mainly we are proceeding in hangzhou office as well.

D

As you know, the hangzhou technical team so leading by you, know the uh apple or she is our leader uh in their hands office. Another one just uh you know neil, uh is he leo and david? They are leading um in the you know, technical team. So next page please thank you, so uh I can explain you know briefly uh in about our uh certain architecture.

D

So uh yes, uh lotus is, you know designed uh as a three part of you know the you know goals. You know, as you know, already, some uh minor and a worker in the storage. So actually we have, you know more than ten one thousand uh workers uh as well. As you know stories uh I say actually so um we have you know. Protocol uh for the stories is, you know so, which can uh add on you know unlimited.

D

uh It is just staff. Safe is a great uh protocol at the moment. It is, you know improving at the moment. So that's great okay. Can you turn this page? Please. Thank you, emily, okay. So actually we ranked like this, you know uh initially we, um it was great, but uh at the moment we are facing, you know some problems, so uh we try to solve it so yeah. uh You know there are a lot of uh you know issues because uh you know the space race is aiming to uh filtering some.

D

You know stress tests uh throughout stress tests, you know uh so uh we are. I hope we all get. Some, you know benefit to our experience in the space race at the moment. So at the next space, please, okay, I yeah. This is the yeah. You know the um solution of the you know: file calling uh ecosystem uh for the next steps. You know. um uh Let's say uh uh netflix is going to develop a local business in south korea. Then they I got a some. You know uh project to develop.

D

You know, uh step by step, so first step uh they're gonna uh develop their application next step. Is uh they identify stories? You know a lot of big stories, so we can be uh a big kind candidate for them. So that's why we need to provide you know some, uh like you know, sdk or library, uh which can uh run or provide. You know uh uh great functions. You know, uh yeah they can develop is easy, I mean so they can help uh the library can help the uh netflix.

D

You know some uh because you know netflix cannot.

D

Yes, uh absolutely they will have some problem uh in terms of you know, finding some stories or building some you know uh places you know to support uh to have some substance for korean people, because uh it is it's very expensive.

D

uh So, but if we have some stories so we can make a deal with them, so some sort of that. So that's why just so we can provide some library like that. So we have to provide. You know uh peer id, prip and cid as well. Then they can have it in the session pool so they can access uh yeah. Let's say uh I want to watch your movie tonight so uh to you know netflix, so uh so what the netflix can give us. You know quickly with the session pool yeah from our storage.

D

That's why uh we are planning to build this solution like that. So next page, please thank you so uh the business uh uh areas that can be like that you know. First of all, you know cdn contains uh delivery network uh partnership. uh Likewise, you know big global companies. You know google as well, can be candidate for us anyway and localization, uh meaning the second one. It's just. We are actually um dealing with the you know, uh governors and local governors. You know our ceo is trying to make a deal with them.

D

So, in terms of you know, um local government are trying to. uh You know, find some future technology uh for the local people. So that's why we are trying to make a deal at the moment. So uh we are. We are planning um on building uh our data centers throughout this nation, so this country. So all you know big provinces. We are planning on building. So that's why it is big project at the moment and and third one is just contains.

D

You know, university and schools are just locked down at the moment because of the um uh pandemic, so the uh covered uh by the copy, the 1980. Actually so it is unexpected. You know it's very you know situation at the moment we are facing at the moment. So now so uh we are uh preparing for the uh you know a great you know, services. If you have some, you know environmental infrastructure like that, so university, any schools are uh needed. They need uh to prepare some.

D

You know online content, you know video, which is big uh files. You know involving big files, so that's why uh we can provide it. So that's good idea, I think, uh for you know the games. um You know uh some people uh are uh well known. uh There are a lot of you know: game stations in south korea.

D

You know even koreatown in la, I think you know some uh sort of that, so uh the game station needs to update every day. You know about you, know uh more than terabytes, you know, but same station as uh game stations are full of in this country. More than thousands of you know shops you know here, so that's why I just it is another big. You know market some sort of. Actually, yes, my um partner just said to me to introduce you know some context, so I I'll finish it shortly.

D

Okay, south korea uh uh uh deadline company headquarters in south korea, south south korea is in um 2000. Currently it has operation um standards in japan, china and the united states, indonesia and australia, the philippines and veda.

D

The technical team comes from a core experts in finance system, development, security, algorithm, distributed storage and blockchain and yeah some kind of them. It was actually you know, with the korean government. Last year, actually, you know the current government is chasing uh future technology to provide. You know, uh you know uh further, you know, um uh and next industry uh revolution you know. So now it's just the um we are. We are just yeah.

D

It's just uh one of the labs of south korea uh actually uh already has been developing for three years in terms of blockchain. You know, that's why I just we got a same project. We were involved in the project last year, so we have experienced strong experience with you know the developing with you know the uh secret sharing. You know, charmers secretary.

D

Actually, uh at the moment we are using. You know um two five, six uh bit uh algorithm based on two five six bit but net, or we just developed the based on five one two bit, so it is really in a couple of that. uh You know very long address. uh We got that, so uh it is really uh good. Actually we are facing. You know, then some ai ages, so, and so we need to prepare uh blockchain and improve uh this technology personally and and globally, as well. So yeah, um sorry about that.

D

Just uh uh I was poor to prepare uh this presentation, but emily helped me and all steps of protocol labs and very smartly, and it explored me so smoothly. I'm very appreciative for all your sports. Thank you.

A

Thank you so much david, no, that was that was awesome and and really really interesting as well and yet another example of a really interesting approach to file coin mining. So thank you so much for joining us and sharing your expertise.

A

Well, everyone I'm sad to say that that's it. For this evening. um I want to close our session by saying a huge thank you to our wonderful presenters, neil from ipfs maine lee from six block and david from dataline for sharing their fail coin mining, expertise and experience with us and all of you tuning in thanks. So much if you'd like to present at one of our upcoming show and tells then please use the link in the chat or apply at space, race, dot, filecoin, dot, io, slash events.

A

You can also catch up on all the events during space race by subscribing to our youtube channel, which is linked in the chat, join us again next week for another session of show and tells from filecoin's top global miners and just a reminder: adjuster reminder there: you go I've just been saying miner so much! I can't stop saying it uh just a reminder that if you haven't registered your miner, yet you need to do so asap.

A

You can do that on spacerace.filecoin.io thanks so much again for joining us and see you again next week, bye.