Hyperledger Foundation Hyperledger Global Forum 2021, 21 Jun 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Building the Next Generation of Aries Agents - Andrew Whitehead

Description

Building the Next Generation of Aries Agents - Andrew Whitehead, Representing Govt of British Colombia

For the past year, the team at BC Gov and contributors to the Aries CloudAgent Python project have been working to extend the capabilities of the agent software while maintaining flexibility in the face of evolving standards and use cases. Historically, the project has depended heavily on the Hyperledger Indy SDK, but moving forward this dependency is shifting onto a trio of new Hyperledger projects. Indy-VDR (ledger transactions), Indy-Credx (Indy verifiable credentials), and Aries-Askar (secure storage) have all been contributed in the past year and will continue to evolve independently. Supporting multiple backends as well as a family of Aries communication protocols raises interesting design questions for ACA-Py. Andrew will provide a guided tour of its architecture as well as the possibilities made available by this new model.

To learn more about Hyperledger, the open source community focused on developing a suite of stable frameworks, tools and libraries for enterprise-grade blockchain deployments. Which serves as a neutral home for various distributed ledger frameworks including Hyperledger Fabric, Sawtooth, Indy, as well as tools like Hyperledger Caliper and libraries like Hyperledger Ursa click here: https://www.hyperledger.org/

Session Details & Slides (if available)- https://sched.co/j3cq
Access All Sessions: https://events.linuxfoundation.org/hyperledger-global-forum/program/schedule/

----
Hyperledger – Open Source Blockchain Technologies

Hyperledger is an open source community focused on developing a suite of stable frameworks, tools and libraries for enterprise-grade blockchain deployments.
It serves as a neutral home for various distributed ledger frameworks including Hyperledger Besu, Fabric, Sawtooth, Indy, as well as tools like Hyperledger Avalon, Cactus and libraries like Hyperledger Aries, Ursa. Learn more about Hyperledger projects: https://www.hyperledger.org/use

Case Studies: https://www.hyperledger.org/learn/case-studies
Training & Certification: https://www.hyperledger.org/learn/training
Tutorials: https://www.hyperledger.org/use/tutorials
Webinars: https://www.hyperledger.org/learn/webinars
Events: https://www.hyperledger.org/events
Vendor Directory: https://www.hyperledger.org/use/vendor-directory

Subscribe to the Hyperledger Newsletter: https://www.hyperledger.org/newsletter

Follow-us on Twitter @Hyperledger

Learn about Hyperledger Membership: https://www.hyperledger.org/about/join

#Hyperledger #HyperledgerGlobalForum2021 #Blockchain

A

I'm andrew whitehead an independent developer, working with government of british columbia in digital trust services.

A

um This is going to be a session on building the next generation of various agents. So hello, thanks for coming out.

A

um So a quick introduction to the work we've been doing at bc, gov aries cloud agent python- is something we've been working on for, I guess two to three years now. It is a sort of agent and agent framework written in python for developing aries agents.

A

Lets you hook up a controller written in your preferred language to communicate with the agent over arrest protocol.

A

We use it ourselves to issue credentials to the org book which acts as a holder of credentials.

A

Those credentials are business registrations for the province of bc, as well as related credentials having to do with certifications. We can put credentials in there, such as whether business has been registered as a green organization.

A

um A lot of what we do has been based on hyperledger indie, especially the non-creds credential supports, which relies on an indie node ledger an identity ledger which is permissioned and doesn't involve any mining.

A

We don't store any credentials on the ledger. They're only communicated peer-to-peer, we encourage basically single use, did com connections for that.

A

And the non-creds credentials have support for revocation where the registry is essentially published to the ledger, which allows it's one of many measures taken to avoid exposing any details about the holder of a credential unintentionally.

A

So occupy you see, we've been around a lot, it's quite popular um in terms of contributors and implementations. A lot of people are taking this framework and using it to build their own applications, and um we've started to run into some limitations of india sdk for our own use.

A

Our particular deployment runs on openshift and we like to have several agent pods running all issuing from a shared wallet or in the case of the org book, receiving and storing to a shared wallet and because of the architecture of the sdk, where everything is bundled together. You've got your ledger: clients, you've got the wallet, handling and you've got the credential operations.

A

um Sometimes we were finding that our performance was limited by that, but also the the flexibility of those components was a bit more limited.

A

So we've been on about a year project, I think of uh separating some of this functionality into what we call the shared components and I'll introduce some of those components and how we plan to use them and make them more available. More convenient for others to use as well.

A

So our hybrid repositories, apart from occupy itself, there's the indie shared rs repository, which contains indie credits into you, tills, there's ndwql in there. These are breast crates, so fairly low level um for most implementers, but we take these components and add language wrappers on them. So at the moment, indiecredx is available for python as well.

A

Indivdr is the ledger. Clients, library and aresascar is our sort of generic storage backend, which is not very opinionated about what you store in there, but uh can be used to develop specialized wallets.

A

There's a few links there below. If you need to explore the projects, otherwise I think you can just hit up hyperledger on github and and search for us. All of these are tagged as at least aries.

A

um Part of our motivation comes from the set of standards behind aip 2.0. This is the the aries interop profiles. At the moment, the ip 1.0 is in effect.

A

There are, um I don't know how many implementers, I think at least a dozen now that are conformed with aip 1.0 to different levels, which means they can interrupt with each other.

A

The first requirement there is generally peer connection establishment for aip 2.0. The um did exchange protocol has been introduced. This replaces the earlier connections protocol.

A

um The out-of-band protocol also kind of spun out of that effort. So connection invitations are now represented as out-of-band messages, as well as certain other kinds of requests. So a connectionless presentation, request or connectionless credential offer as possible using that protocol, and we have newer revisions of the protocols used to exchange credentials either to issue new credentials to a holder or to present.

A

Indie credentials to a verifier, as well as the new bbs plus credentials, which are being introduced now.

A

um And one point that isn't listed here is the did com v2 envelope formats. So that's going to be a big part of supporting aip 2.0 as well as the envelope format has changed. It's become a little more standard in terms of.

A

Being a jwe with with more widely implemented encryption.

A

Algorithms, so the first project contributed to hyperledger was indie vdr. I mean as part of this effort, the separation of the the shared components.

A

Indie vdr is directly adapted from india, ck code, so just just pulling out the ledger: client code, which uses 0mq to connect to the verifier nodes of uh uh sk ledger, be it uh sovereign, testnet, mainnet or a local ledger instance.

A

We use von network mostly for that to boot up a quick development ledger on your local machine.

A

So in dvd-r lets you read and write to the ledger um written in rust, part of the code. Adaptation meant uh moving to async in there and it has language. Wrappers written for python go and node.js python is most complete. There go is, I think, pretty close. It might be missing. One or two request types and node.js is more under development.

A

There's also an ffi. A foreign function wrapper, so you can call it directly from c if desired and ndvdr includes a basic rest proxy. It's called a dvdr proxy and you can just point that at a indie, node genesis file run the executable and it provides a rest interface to read and write from that ledger.

A

We are also using ndvdr directly in indie node, monitor and in details server now.

A

As opposed to using ndsdk there, because all they really needed to do was connect to a ledger under dsdk you'd have to create a wallet before you could load the genesis file, set up the pool and start interacting with the ledger.

A

So it gives us a little more freedom there and if any additional features are added, you can upgrade that component independently or not depending on what your needs are.

A

Indie credx was the second fork of this development. This was also extracted basically directly from india sdk um some adjustments to the api.

A

This provides the non-cred's credential handling. It's everything you need really to be an issuer holder or verify verifier of non-credits credentials.

A

These use cl signatures and they're inherently tied really to indie node ledger instances, because that's where you will find the schemas and credef's replication registry definitions and entries all published, as well as the.

A

Key information about the issuer of credentials.

A

Indiecredx also has a python wrapper and foreign function interface.

A

The major difference here is that the the methods, as called, will directly take um the credential cryptographic inputs so you're not giving it a pointer to something in the wallet to go and look up. You are going to your wallet, fetching the object that it needs and passing them into the inaudible's methods.

A

um That obviously takes some careful operation to avoid leaking those objects, but we trust you to do that on your own in general, you want to use a framework for these things anyway, so occupy. Does it in a way where the buffers are generally passed around with zero copying into memory managed by python itself, so the buffers are always zeroed. Once they're released.

A

um Indie credits doesn't do any threading itself internally because it tends to be cpu intensive, uh especially things like generating signatures or creating a new revocation registry.

A

So it leaves it up to the caller how many threads they they'd like to run to perform these operations um when we use it in occupy, for instance, we're running in an async framework. So we tend to use a thread pool for all of these operations.

A

And the most recent components that we've introduced is aries ascar, that is sort of lightly based on the design of of the indie wallet, but it doesn't really share any genetic information. The uh 0.2 release of asgar is coming out shortly. There there's been several rc's there and that introduces the new ascar crypto crate, that does it offers um generation and representation of all the key types that we need at the moment for an aries agent.

A

So there's everything from cha-cha encryption, various aes dm key wrapping, um ed25519, of course, x25519 and there's basic bls key creation that will be updated soon, as well as um the the addition of bbs uh signatures.

A

At the moment we use the matter global ffi, bbs signatures, uh python wrapper for for the bbs signature generation, but that will be moving into nascar as well.

A

um Ask our crypto itself supports uh the ecdhes and ecdh1pu.

A

Jwe encryption algorithms up to the latest drafts, as well as a generic encrypted key value storage. Internally. That looks a lot like how the indie wallet stores records in that there's just a large items table um if you're using normal database tools to browse the database, you can't really discern anything. All of the records are encrypted there.

A

um It's uh it allows searching for for these records by encrypted or unencrypted tags as well, which is the primary way of finding information once you've you've stored it in the encrypted storage, there are sql lights and postgres storage back ends much like india sdk at the moment uh both are built in so it's not a separate plugin for postgres likely be adding more storage back-ends there as we go along and at the moment there's only a python wrapper and foreign function.

A

Interface um love to have additional language wrappers for asgard here and, as I mentioned later, on, nascar crypto is designed in particular to be used in isolation.

A

So, if you don't need the storage functionality, you can still use all of the key handling and encryption operations. There.

A

So a primary goal of this project was reusability.

A

We wanted each of these components to be useful on their own um and for implementers that didn't need all the functionality they could just pull in one or two of these components.

A

As we go along, we we may or may not be focusing on the non-creds credentials. For instance, we may be doing more bbs plus credentials, which uh could, albeit the need for indie credits. There.

A

um So both individual and indie credits allow you to perform these operations without a wallet which was a primary restriction of india sdk and we're aiming to support multiple language wrappers. So that's the same tools can be used, whether you're building a framework framework in go or javascript or python, as we are housecover crypto. The the core functionality in aries ascar is no standard compatible at the moment, so should be able to target embedded hardware, although nobody's tried that yet and it should be compilable for asm as well.

A

Although again that's a an experiment that hasn't been tried and that uh crates on its own gets you a long way to supporting dead gong did com v2, it doesn't do the jwe construction and deconstruction, but it gives you all of the encryption algorithms needed.

A

um You can also generate your bls keys for doing bbs, plus signatures and shortly that will be integrated more directly.

A

So now that we have these three components, the next step is re-architecting aries cloud agent python a bit so that users of the framework can switch from india sdk to the new components.

A

uh There is a shared components: branch if you'd like to check it out that will be merged.

A

In preparation for a 0.7 release of occupy, where you can try out the new back end experimentally, so each of these components has a python wrapper and they're installed from pi pi um as wheel packages, because the core of each of these is is a shared library written in rest.

A

Occupy uses dependency injection, which means that in general, the protocol handlers are highly reusable. They don't need to know whether your targ, whether you're running india, sdk behind the scenes or whether you're running the new shared components.

A

So in general it hasn't been that difficult to add support for the new components. We just change, which implementations get injected for the various interfaces that the protocol and service handlers request.

A

So um using the new back end, you request the indie ledger base interface and you may get back in ndvdr ledger implementation as opposed to an indie sdk ledger implementation.

A

There are indie credits, issuer holder and verifier implementations, there's the ascar storage, implementation of base storage and the oscar wallet implementation of base wallet as well as um for 0.6. We introduced the concept of profiles, which is um roughly roughly corresponds to an identity which may or may not correspond to an entire wallet.

A

Profiles also have sessions which introduce some basic transaction management.

A

We as we go on we'll, be introducing more support for transactions in protocol handlers to make sure that logically, grouped operations, all proceeds within a transaction and um a large part of the reason we were able to switch these back ends as well is the the high rate of unit test coverage in occupy. So every pr goes through, I think about 1200 unit tests.

A

At the moment we have over 98 percent code coverage and we don't intend to change that with the new back-ends, but, um for instance, just just taking all of the existing wallet and storage tests and running them against the new back ends goes a long way to knowing that we can just plug those in and they will work equivalently to india sdk there.

A

We also require a fairly stringent um code, formatting of any contributed code, so that our diffs aren't full of uh changes just due to ides reformatting things so current status, all these packages are published currently for x, I should say it's x, 86 64..

A

um Additional platforms would be another um important area of development. We just need to work on the ci which all runs on github. At the moment to publish those additional platforms, you can check out shared components very soon. You can check out occupy 0.7, where all you need to add is wallet, type house, car and well and make sure you've installed the requirements.

A

um There's a separate requirements file for that or you can install occupy with the ascar feature flag.

A

It should support all the same protocols that it does now and very soon, you'll be able to take your nd sdk wallet and run a script to migrate it to a arizona.

A

Wallet, I um we'll stop for a second there to see. If anybody had questions see, we've only got five more.

A

Minutes um nobody's posted anything. So oh.

A

Yes, uh so yakub kochi was asking, um does uh base storage represents the non-secrets part of india, sdk.

A

Yeah, essentially, that is the parallel um nascar storage- is just a generic interface for fetching and retrieving records that are identified by a category and a name and are stored encrypted. The ascar wallet interface provides more key management.

A

um Things like signing with a key and verifying with a key and management of your dids in the wallet um philip ross was asking. Can I use only some of the new components or do I need to switch completely so within occupy? It's uh there's only the one flag, if you're using a nascar wallet, then you're using ndbdr to connect to ledger and you're using the credx uh for your credential management.

A

um That's because we can't really take in the sdk and just use it for credentials. You kind of need to use it for everything, but the components themselves are independent at the moment python packages um and there's nothing stopping you from using them in isolation.

A

That I know of.

A

I should mention on this slide that documentation is still lacking. I need to publish documentation for the the python apis.

A

The the rust crates generally are fully documented. I believe all the structures and methods have at least some description of what they do, but uh we need proper, um fleshed out, readmes, quickstart guides and documentation of the wrappers as well.

A

um Then yamamoto is asking if I could elaborate on bbs plus support and the canonicalization listed here. um I threw that in actually based on some comments by brent sindel at evernom that they're looking at a message, canonicalization format based on microsoft, spartan, I believe- and I'm still looking for more information on that. Actually, um the 2021 signature suite is, as far as I know, currently a set of notes by mike lotter, so we're following that and looking to contribute as well.

A

We're we've been doing a lot of work on bbs plus credential, revocation at the moment, which seemed like a big gap. As far as our usage is concerned, and I think we have a good solution there, which we'll be published publicizing shortly um as well as our other future plans. We'd love to see uptake of these components in existing frameworks did com rs.

A

For instance, I've seen um that's under the uh diff banner um I'm going to re-implement some of the things that are in ascar crypto and it could easily use the ecdh1pu implementation there uh von network, as well we'll be switching to ndvdr, actually had a branch that had switched over to that. For quite some time, but it hasn't become the main branch. Yet I think we've only got about a minute left here.

A

A

So if there aren't any more questions, I'll wrap it up there, I think thank you very much.

A

um Of course you can look us up on github if, if you'd like to contribute watch for the new releases of occupy, if you'd like to test the code out yourself and uh we're thankful for any contributions, everything we do is open source, so we'll uh rise all boats together right thanks very much.