Hyperledger Foundation Hyperledger Global Forum 2020, 9 Mar 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Cross-organizational Secure Data Exchange with Access Control using Blockchain - Qiong Zhang

Description

Cross-organizational Secure Data Exchange with Access Control using Blockchain - Qiong Zhang, Fujitsu Labs of America

With the ongoing digital transformation, there are large amounts of data being generated by individuals, IoT, and organizations. Due to security and privacy concerns, currently these data are siloed, even though there is a strong motivation to utilize data across organizations to obtain valuable data-driven insights. To enable such data utilization, we have developed a new system for secure data exchange between organizations in a consortium with access control using Blockchain. In this system, the data providers can register metadata along with access control policy associated with the actual data. This metadata and access control is recorded on the distributed ledger, while the actual data remains off-chain and stored locally. The data consumers can discover the different data available in the consortium based on the metadata. When a consumer requests for data, the system uses smart contract to verify against the data’s access control policy, and if allowed, seamlessly establishes secure overlay connection on the Internet to exchange data from the provider. We will present the system architecture implemented using Hyperledger Fabric, demonstrate its features and detail a recent use case of a consortium of multiple organizations, such as real estate, mobile operator, in Tokyo area to securely share data based on access control policy and gain insights for city planning. Also, to overcome limitations of the node scalability in Fabric, we have implemented a multi-cluster exchange system with metadata shared across clusters. Lastly, we will show how to extend this system to realize privacy-preserving decentralized machine learning by training local AI models with distributed on-premises data at each organization and combining these models to achieve better performance.

----
Hyperledger – Open Source Blockchain Technologies

Hyperledger is an open source community focused on developing a suite of stable frameworks, tools and libraries for enterprise-grade blockchain deployments.
It serves as a neutral home for various distributed ledger frameworks including Hyperledger Besu, Fabric, Sawtooth, Indy, as well as tools like Hyperledger Avalon, Cactus and libraries like Hyperledger Aries, Ursa. Learn more about Hyperledger projects: https://www.hyperledger.org/use

Case Studies: https://www.hyperledger.org/learn/case-studies
Training & Certification: https://www.hyperledger.org/learn/training
Tutorials: https://www.hyperledger.org/use/tutorials
Webinars: https://www.hyperledger.org/learn/webinars
Events: https://www.hyperledger.org/events
Vendor Directory: https://www.hyperledger.org/use/vendor-directory

Subscribe to the Hyperledger Newsletter: https://www.hyperledger.org/newsletter

Follow-us on Twitter @Hyperledger

Learn about Hyperledger Membership: https://www.hyperledger.org/about/join

#Hyperledger #HyperledgerGlobalForum2020 #Blockchain

A

Good morning, everyone now, let's talk about cross-organizational, secure data exchange with access control using blockchain. My name is Chun Cheung and I'm from Fuji slaps America. This project is mainly done by my colleagues from Fujitsu labs limited in Japan.

A

Virus data are generated by IOT systems as sensors monitors and IT systems. Proper usage of massive data is the key for digital transformation, and data can be transformed to intelligence by AI or machine learning. Algorithms in many areas, such as predictive maintenance, anomaly, detection operator, support and clinical judgment, support and etc.

A

Data is the oil of 21st century. It is expected that in the year of 2025 the amount of data generated will reach 175 zettabyte.

A

The current data flow in rio and the cyber world is that many companies are independently collecting and utilizing data a.

A

New date, flow in the near future is to have secure and appropriate data exchange among companies and organizations.

A

However, their concerns on how to use data and data exchange, Jisoo, global digital transformations away is carried out in February 2019, and we ask the 900 business leaders in nine countries, among them 72 percent concerned that organizations explored personal data without permission and 59 percent of them concerned that data they use may have been, and the 70 percent of them found it difficult to judge. If online information is correct and trustworthy.

A

In the meantime, there are challenges on data exchange. First, in data utilization, there are diversity of data formats and models. It is challenging to convert different data models and the date models for different data users legand in data discovery space. It is a challenging to find the desired data because there is no information about the data and the location of the data can be unknown. In the data providing phase, there is a resistance for data sharing and data, providing due to the security and privacy concerns and unclear cost-effectiveness.

A

In this presentation, we'll focus on these two challenges.

A

Most current data exchanges systems are centralized.

A

Data from different sources are transferred to a centralized the cloud, and then the management and the security of data depend on cloud vendors and there is a more trustworthy data exchange which is decentralized. The data exchange in such decentralized, the data exchange system each and every owner manages his own data and can set the data access control policies for his own data, and the data owners and data users are connected with each other.

A

There are requirements for such decentralized the data exchange system in data discovery. We need to find out what are the available data and where they are so a data. A metadata directory is a must metadata. Directory, basically is a distributed. Collection of metadata and metadata is the description of data set, and it can include the data ID of the day, the attributes of the data and the access policies of this data.

A

After data was found from the metadata directory. In the data providing phase, we need to have data access control for the data provider, and we also need have a secure peer-to-peer data transfer to transfer the data from the data provider to the data user.

A

Or better satisfy these requirements, we can take advantage of the functions available in blocking, for example, the data access control can be more secure by using smart contract, because smart contract is executed at more than one nose verifying the data, access control policies and the peer-to-peer data transfer control can also use smart contract to act as a network controller to forward the data transfer requests and then data trace management basically is to track the data exchange transactions and the metadata directory again is a decentralized collection of metadata, and these two requirements can take advantage of the distributed leather ledger to have decentralized management.

A

Fujitsu has developed, heard decentralized. The data exchange system, VPX VPX, stands for virtual private digital exchange system. Now, let's talk about a more detail about it.

A

Bpx technology is a peer-to-peer communication technology. It has four major functions as we described previously. One is data, access, control, one is metadata directory and one is peer-to-peer data transfer, control and then data trace management. Vpx technology is also a blockchain technology.

A

There are different types of blockchains: we can have public type consortium type and private type. Here we px focuses on data exchange among several companies, so we chose to have consortium type of blockchain where there are multiple administrators and we use permissioned blockchain with cross validation for consensus algorithm, and we chose to developer VPX based on hyper ledger fabric.

A

Here we show the VPX architecture, we have data provider and data user data provider and the users enjoying the consortium buy locally. Installing VPX gateways data provider can publish and share metadata with data access control policies on the hyper, lighter fabric and theta stay locally and can be found based on the data ID by the reverse proxy, and then data and transferred from provider to user through VPX gateways and not through the hyper later fabric.

A

Now, let's discuss some major api's for VPX users.

A

First date provider can set access policies for his own data, as well as the metadata of his data and those access policies can be stored in the metadata.

A

There is a meta data published API for data providers to publish metadata on the distributed ledger and there is a metadata gate API for data users to retrieve a list of accessible metadata from the distributed ledger, and then the data user can discover what are the available data and defined the desired data. After that there the day to get API, you transfer the data from data provider to the data user securely.

A

And that there is a trails, get API to retrieve the trace information of data exchange transactions from the trace. We can know that how many data exchanges occurred and the details of data exchange, such as who is the dead provider? Who is the date user? What is the metadata? Those details must follow the set and configured access policy.

A

We can see that those api's are very easy to use for VPX users. It hides the complexity of the underlying procedures using smart contract.

A

Now, let's look at those complicated VFX procedures using smart contract that one is for date, provider to publish metadata here. Theta provider is on the left side and the data user is on the right side. It corresponds to the metadata published API for this step. First, data provider register metadata its VPX Gateway locally installed, and here is how a metadata look like it has. Data ID can be hash number of certain values, it has attributes of the data and it has destination address, which is the address of the VPS VPX gateway.

A

It is encrypted and it has access policy, so here is accessible to user X.

A

After receiving this metadata, wretched registration, the VX gateway, verifies the data provider and then distribute this metadata own to that distributed a leisure and then once each BPI gateway receives this metadata, it will create it, create a forwarding table and address translation table.

A

This table will be used in the data transfer step later for route data transfer, requests.

A

The second step is for the data user to retrieve the set of metadata and find the desired data, and it corresponds to the metadata GATT API. So in this step, the data user send a request for metadata to its VPX gateway and then the VPX Gateway verify state user and it's access right of the metadata on the ledger and then return the list of metadata that user X can access.

A

Then the third step is transfer of the actual data.

A

It corresponds to the data get API so here the date users send the request to the desired data who its own VPX gateway and then the we TX gateway verifies the date user and it's accessed right of the data, and then it acts as a reverse proxy, and here basically the VX gateway checks. This forwarding an address translation table and find out that an address for the data transfer requests to send it to next. So here, in this case, address C is the address to send the data transfer request the to so.

A

It's a sender to the address see here basically, is the address of the VPX gateway at the data provider site and similarly, in this VPX gateway, it will verify the data user and is access right of the data and then again it also act as reverse proxy.

A

It checks this table and to find out the next address to forward the data transfer request, the to which is address a which is the real data provider and then, once the request to reach the real data provide, the dead provider starts to sending data to the data user through the VPX gateway you're. In that this, we PS gateways we'll check the hash number of the getting data to make sure the integrity of the data and make sure that it is not phosphide.

A

We can see that all requests and data go through VPX gateways, thus hiding the location of the actual data from VPX participants for better security.

A

Here we show the VPX traces on the leisure. The log of each VPX, API call is stored on the leisure participants can trace the data exchange transactions. Here is the trace information. We are trails, get API and at the block number 20. This is the log of data get API and at the block 6. This is the log of method data published API.

A

Now we show a demo for v3 AXA network. Here is the overall VPX network view. This is the trace information and we have a date provider, which is the department store and the data user a and data user B. First, the data provider set the data access policy accessible to both user, a and user B, and this metadata is published and it is shown at user a and a user B.

A

And the next, the date provider would like to change a data access policy to be accessible to only user a so it also published this metadata and we can see that the metadata on user B disappeared and the user a can still access the metadata and then user a starts to purchase the data.

A

First, it will guess the API documents of the data about how to use the data.

A

And then use a a gets, the data and all those transactions on data exchange is recorded on the blockchain and.

A

We can see that using those using this decentralized, the data exchange or system data provider is very easy to change it access policy for his own data, so that it makes them more confident and more willing to share data in.

A

Fujitsu, intelligent data service called votura DX was commercially launched in Japan on May 14th 2018, and this votura DX commercialized the service adopted the vp x technology and we also had a data extended trial using voter, ID x and v PX for urban designing. The purpose of this trial is to make central tokyo attractive place.

A

The participating companies include a real estate company and a mobile company participating companies provide and exchange the data related to central tokyo, where vp x and the analyzed the data collected by vp x, because this is a new service and people are very interesting and curious about it. So they're willing to try it in different ways. So we have seen that more data were exchanged among participating companies and the cross analysis were conducted using variety of data.

A

Now here we have a little summary and vp x has several benefits. It has easy-to-use api's. It hides the complexity for developing, secure and distributed flat platforms. It provides data exchange of flexibility in terms of sizes or massive data. We can directly transfer from the data provider to the data user and for small data. We can publish it on the distributed ledger through metadata and we also had several VPX extensions.

A

One is multi class. The VP acts for improve the scalability. One is trained data lineage for better trusts. One is a federated learning on VPX for better data privacy. Now we will talk about each of them in more in high level.

A

To improve the scalability multi class, the VP X divides a larger vp x network into multiple clusters.

A

Within a cluster, it is the same as a regular vp x network. We can publish metadata, we can get metadata among with VX gateway gateways, but between clusters, metadata and the data are exchanged via representative nodes.

A

Main data lineage this technology is to integrate data processing, history and the data exchanger history. You provide end-to-end data traceability for a new level of trust, so here data processing history is at each individual company. Usually it's in their data lineage system.

A

It shows how data are converted and processed, and the data exchange history is recorded on blockchain VPX. Then we integrated both using hash values and a former chain which is called chain data lineage. By doing that, the data or region can be traceable and more trustworthy, and we recently produced a video on dated own chain data lineage. At the end of the presentation, I would like to show you the recent video.

A

Federated learning only px is our third extension on bpx. Our experiments have shown that bpx can seamlessly provide a secure, distributed a platform for federal learning tasks. You know about the federal learning, raise your hand, you know great, so federal learning is a distributed. Ai and machine learning approach, it trains, the AI and machine learning, models locally and clients. Our own promise word. Local data are stored.

A

Battery learning usually involves a group of federal learning clients, they usually store local data and a further learning server during federal learning process, AI and machine models are exchanged between clients and the server. Not real or actual data are exchanged here. Basically, the local data remain local, so it has better data privacy.

A

Better learning may have hundreds or thousands of training rounds in order to achieve or endured in order to converge to certain accuracy of the training model. So here we show three basic steps: in a single training round in federal learning the first step, a federal earning server provides a global AI, a machine learning model to a group of clients. So here we have two for doing that.

A

We can use metadata, publish API, to publish the global model metadata on to the distributed ledger and the second step lines, get the global model and train it locally with the local data and then provide the new local model back to the server for doing this step.

A

We can call each client can call metadata, get API to get the global model metadata and called the data get to get a global model and then a to train the local model, and then it can call metadata, publish API to publish the local model metadata on to the distributed ledger they own the third step. The server gets the local model and aggregates them to a new global model, and for doing that again, we can call metadata, get API and data get API to get a local model at the federal learning server.

A

So once the new local, once the new global model is obtained, the federal learning server can further use it to proceed. The next training round and repeating step one two: three.

A

We can take advantage of the consensus made on the leisure to synchronize the global model updates among this better learning clients.

A

So here we can see that the AI and machine learning models is actually the VP acts, data to extreme and the AI and machine learning model metadata actually is a VP accent metadata. So there is no real data exchange here and then the federal learning server and the federal learning clients. They are both data providers and data users, and we can see that, with this basic VPX api's were able to perform better learning tasks in a very stable manner and in distributed settings.

A

Okay, now I would like to show you the video Oh train data lineage, as I promised our.

B

World is more connected and more globally integrated than ever before. Massive quantities of data are being mined and shared to create new values and services. This leaves us with unanswered questions.

B

What is the source of the data? Do we have permission to use it? Who will use our data and how we don't know. Fujitsu knows that creating trading and using data means it must be trustworthy. This is why we created a new blockchain data. History management technology, Jayne data lineage chain data, lineage lets organizations exchange data that is securely verified.

B

Let's see how this works, data is collected and processed.

B

The data is shared among other organizations that process and use it. The data has a data exchange history that is shared. The data processing history is managed by each company and can also be shared when needed.

B

If we look at a single item of data.

B

All related history is picked up and can be viewed in graphical form. The history includes data exchange, history, data processing, history, the original form of the data and other information chain. Data lineage is a block chain data, history, management technology; it unifies data, processing, history and data exchange history that until now was scattered and distributed. It does this by using hash values of these histories, which are linked together and managed and shared unblocked chain chain. Data lineage manages the hash values of the exchange, history and processing history by blockchain, which avoids falsification and fabrication chain.

B

Data lineage goes further by adding consent to use information to the data exchange. History for supporting personal data exchange.

B

Data users can check that they can use the data safely by verifying its source and history. Data providers can also verify who is using the data they provide and half many organizations and industries can benefit from this technology. It has a wide range of applications, such as automobile insurance, new drug development and logistics.

B

The secret of trustworthiness is verification with chain data lineage, the power of blockchain takes trust to a new level with a verified history of critical data commodities.

B

Fujitsu shaping tomorrow with you thank.

A

A

Know we have time for questions.

A

A

That's a very good question and we did not use that feature and our metadata actually is majorly for access control for the large data stored locally, so yeah we did not use that feature.

A

A

Or the metadata we have also access control policies. Not only the data have access control policy, the metadata also have access control policies, so it's only exposed to the one who has the access right.

A

It is a actually on smart contract because when we publish the metadata and the access policy on the metadata, whenever people retrieve the metadata will be checked and it will be verified among multiple nodes.

A

It's not a plectrum format, because it's a massive data, it's harder to put on the blockchain, so we the control of the transfer. Basically looking for the route, the data transfer request is done by black by smart contract on the blockchain, but when we transfer the large data it's out of the blockchain and it's using HTTP overlay, Network.