Hyperledger Foundation Hyperledger Global Forum 2021, 14 Jun 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Performance and Scale Working Group - Haris Javaid, Xilinx

Description

Performance and Scale Working Group - Haris Javaid, Xilinx

To learn more about Hyperledger, the open source community focused on developing a suite of stable frameworks, tools and libraries for enterprise-grade blockchain deployments. Which serves as a neutral home for various distributed ledger frameworks including Hyperledger Fabric, Sawtooth, Indy, as well as tools like Hyperledger Caliper and libraries like Hyperledger Ursa click here: https://www.hyperledger.org/

Session Details & Slides (if available)- https://sched.co/jnPi
Access All Sessions: https://events.linuxfoundation.org/hyperledger-global-forum/program/schedule/

----
Hyperledger – Open Source Blockchain Technologies

Hyperledger is an open source community focused on developing a suite of stable frameworks, tools and libraries for enterprise-grade blockchain deployments.
It serves as a neutral home for various distributed ledger frameworks including Hyperledger Besu, Fabric, Sawtooth, Indy, as well as tools like Hyperledger Avalon, Cactus and libraries like Hyperledger Aries, Ursa. Learn more about Hyperledger projects: https://www.hyperledger.org/use

Case Studies: https://www.hyperledger.org/learn/case-studies
Training & Certification: https://www.hyperledger.org/learn/training
Tutorials: https://www.hyperledger.org/use/tutorials
Webinars: https://www.hyperledger.org/learn/webinars
Events: https://www.hyperledger.org/events
Vendor Directory: https://www.hyperledger.org/use/vendor-directory

Subscribe to the Hyperledger Newsletter: https://www.hyperledger.org/newsletter

Follow-us on Twitter @Hyperledger

Learn about Hyperledger Membership: https://www.hyperledger.org/about/join

#Hyperledger #HyperledgerGlobalForum2021 #Blockchain

A

um Welcome everyone, and thanks for attending this session, I am haris and I work as a researcher at zarlinx. uh Some of you may not know uh about zarling, so I will quickly introduce my company zarling says um the inventor of fp, which goes along with other hardware technologies such as general purpose, cpus, gpus and a6. Typically, fpgas are used for uh education, performance and energy efficiency.

A

I have been involved with the different hyperledger projects for the last two years and more specifically, I have been working on fabric and looking into different scalability issues, so I will give an introduction of the performance and skill work group at hyperledger, and then I will introduce some of our work and how we are trying to tackle some of these stability plugins.

A

So if you have any questions along the way uh you can post it um in the q, a tab on the right hand, side, and I will leave a few minutes at the end to answer those questions.

A

So pswg is basically the performance and scalability work group, a forum at hyperledger to look into these performance and scalability issues.

A

Projects is not tied to any one particular uh project and it's supposed to look into all the different projects under hyperledger, but uh most of the work has been done on fabric, because fabric is the most popular uh hypolizer project.

A

So, in terms of the goals the group started with two main focuses: one was on performance evaluation, uh for example, the idea there was to look into different type of metrics that can be used uh to compare um different blocks and platforms and different projects under hyperledger and early on, the pswg group published a white paper on metrics.

A

In the same context, pswg was also involved with some of the early work on hyperledger caliper, which is the tool now mostly used for benchmarking. Different hyperledger projects.

A

Focus group was on uh performance improvements and the scalability in blockchain deployments.

A

um The goal here was to gather ideas uh from different groups to see what kind of performance bottlenecks exist in these hyperlazer projects and how they can be scaled, and typically, the scaling involves either a scaling up, which is vertical scaling where you cannot add more resources in a single load of a network or you can scale out. uh Another term is called horizontal scaling where you can add more nodes into the network to improve its uh performance.

A

So uh in this context I had been attending pswg meetings on and off, but recently I got actively involved in the group and now I'm leading this uh new effort to revive the group to revamp the group and have more focus on scalability issues, especially from people who are using these type of their projects and deploying them in real world scenarios.

A

So I would like to talk a little bit more about scalability in blockchains. So when we look at a proof of concept project- and we want to move it into a real-world deployment, we would like to improve upon a few different performance.

A

For example, in a proof of concept I might have two organizations, but in a real-world deployment I might want to add five organizations.

A

What that means is that we would have more transactions generated in the network and hence the nodes in the network should be capable of handling that much workload and should have better or increased transactions per second throughput.

A

Likewise, in a production environment, we might want to gather much more data compared to a proof of concept, which means the transactions could have much larger size, which means that the blocks would be larger and again, the network should be capable of efficiently transferring larger blocks around the nodes and process them efficiently.

A

So um if we look at these uh four different performance metrics- and we want to improve them- we have to look at both uh individual nodes of a network as well as scale out the network by adding more nodes uh in the network and looking at the network traffic and how that could be handled in a better way. So at xilinx.

A

We believe that uh if you want to improve this metrics and go beyond what is available by software, only implementations which are available nowadays, we need to bring in newer hardware technology not only give better computer resources, but also give better network resource uh going a little bit more because of this. In what kind of work we have been doing at scalability.

A

So, coming back to pswg, the group had been dormant for a while and then we recently started to revamp it and in the past two months we are have tried to uh initiate two different threads in the group. The first thread is um around the performance, evaluate and thanks to the technical working group in china, they have done a bit of work on quite a bit of work on a hyperledger fabric, performance, evaluation and exploration, and they have produced two different tools: called tape and probe.

A

The tape tool is very much like hyperlink caliper, but it's lightweight and can generate much higher transactions and rates and can better stress test a network. The probe tool, on the other hand, is an exploration tool where you can kind of explore how the performance of a network behaves uh based on different parameters, for example different block sizes or creation of a block, etc.

A

Then the second thread we have been working on from myself and my colleagues at xilinx, so we have been looking into how we can bring in a newer hardware acceleration technology and remove some of these. Some of the performance bottlenecks in permission blockchains and how we can use hardware accelerators to achieve performance and scalability beyond what is currently available uh from software. Only solutions.

A

So let me go into a little bit of more details of our work um so ago. We started a project, um the blockchain machine at starlinks, and the aim was to look into permission blockchains and see what kind of bottlenecks appear in information blockchains. um If you look at um public blockchains like bitcoin and ethereum, we can see that they're already using specialized hardware, for example, uh bitcoin is dominated by asics uh ethereum is dominated by gpus and they use these s6 and gpus to have much higher rates.

A

This is necessary for faster proof of work consensus mechanism, but when we look at permission, blockchains, the consensus mechanism does not use proof of concept. uh Proof of work, which means that the bottleneck now shifts from consensus to the validation nodes and the validation nodes share. Quite a lot of core operations, for example, cryptographic operations and accesses from they holds the global data network and these operations become uh bottleneck. So where can we on hyperbolic fabric because fabric was the most popular formation blockchain under hyperledger umbrella?

A

uh We looked at a lot of different operations which are done inside the validation, node or a validator peer network, and what you can see here on the graph on right hand side is that the ecdsa signature verifications are the most time consuming operation and there are operations, for example, computation of hashes based on sha 256 and accesses from database, which becomes a big bottleneck in hyperledger fabric validation process. So what we was that, can we scale up?

A

Validator appear in hyperledger fabric using specialized hardware, which is implemented on fpgas, and then we can accelerate or execute these bottleneck operations directly on the hardware, and with that in mind, we came up with this modified validator appear in hyperledger fabric.

A

Are shown in this figure on the right hand side, so we split the operator in two parts. So the part on the right hand, side with green boxes, is all the operations which involve some kind of verification of the block or validation of the transactions or the endorsements, and then reading from database, applying some concurrency checks etc. So all of these operations are actually executed on a hardware accelerator and then the software, the cpus.

A

They still run the standard fabric software, but they skip all these verification and validation operations and wait for the results of the validation from the hardware and as soon as the validation results from the hardware are available.

A

I'm based with the blog and then the ledger is the block is committed to that, just like any other fabric here. So what this helps us to do is that we can actually accelerate and uh with much better performance the validation operations which are the bottlenecks, while we keep the non-bottleneck operations still on the cpu cpu side.

A

So now, let's look at the performance which achieve from the setup.

A

If we exclude the ledger commit part which is basically run on the software side, either with the hardware accelerator or without the hardware accelerator, we were able to see, observe a 17 times, improvement in block validation, latency and we were able to actually reach up to 95 000 transactions per second, as the commit of the validator peer. And what I would like to mention here is that if you look at visa payment system, uh these systems are supposed to be able to process some transactions per second or peak right now.

A

So what we are really trying to show here is that if we bring in hardware acceleration hardware specialized hardware technology into permission, blockchains, we can break the software only barrier and then we can actually have a much better scalability information blockchains.

A

So if you're interested in more details about this work, we recently put out a paper on archive. The link is embedded here. You can, you can go to that link and read the paper and the details. We also just started a new hyperledger labs project called machine.

A

This is not readily usable at the moment, but we will be putting out a lot of a lot more content into this repo in the coming weeks. So you can actually um add yourself to the watch list of the repo, so you can get all the updates and when it's ready to use you can actually uh use it.

A

So, coming back to, we have a couple of uh uh plans for for the next few months, so so the first definitely organized some deep dive talks around the tools from technical working group, china to talk more about performance, evaluation and exploration for different blockchains, uh and then we're also going to have some deep dive talks on blockchain machine from settings, which is what I just uh briefly described to you.

A

But, more importantly, what we are really looking for is to collaborate with other uh groups under hyperledger or other uh industrial partners who have actually deployed these hyperlogic projects in real world scenarios, and we would like to really know from them what kind of scalability issues they are facing and whether some of this newer hardware, acceleration technology, can be used to address those issues which could not be addressed in this software. Only solutions, so please join us.

A

We have monthly meetings that I have, uh but in the link here, uh meetings as well, and if you want to know more about our work and you want to reach out to me directly- I have also provided my contact details here as well. So with that, I will stop here and see if you guys have any.

A

A

Okay, so any questions in the q a tab, but if you do have any questions you can post them, even though that the session is supposed to end now. But if you don't leave the session, I can still answer your questions here in the chat.

A

So I'll wait another minute and if there are no questions, then I will end this session.

A

Okay, so I guess there are no questions, so uh thank you, everyone um uh for your time and uh coming to this session and again, as I said, if you're really interested in our work, please join us at psg talk to me directly. Thank you very much.