DFFML Rolling Alice: Engineering Logs: Volume 0: Architecting Alice, 9 May 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Architecting Alice: Volume 0: Context: Part 15: Doctests to Download and Run SSI Service

Description

https://github.com/intel/dffml/discussions/1369#discussioncomment-2707445

A

All right we're back.

A

Working twitter back to the thread.

A

Back to the thread, oops.

A

Back to the thread here we are, this is right now, okay, so um where were we so yeah? I figured out a lot of stuff today. It's all written down, uh we're not gonna, go into it, just because um yeah okay, so um I'm gonna put everything on the chain. um You know where we're we've. We've figured out that we can do the thoughts as nfts, um so we're gonna try to encode a thought to an nft um and we want to you, know, put it on top of one of these did base change.

A

Also, we are going to um then do some stuff with cb band tool, which I can't remember if I talked about that already um yeah that'll be later, but let's just go put this thing on on on a chain so yeah, so we wanted to put it on transact.

A

What was that we just closed? Okay,.

A

Was the transact platform?

A

A

A

A

Okay, we wanted to see if we could point this nft storage at we might just have to clone the repo.

A

Is this all one font like what is this? I really don't understand. What even is this.

A

Where's, the code.

A

Is this all just.

A

A

What is this um so we're in nft storage? We're looking at this packages, create nft storage, create nft storage. Maybe this is like oh yeah. This is like a generator for the rest of these and for tools website. Api api, src index okay. So this is probably our main entry point for the nft start storage thing.

A

So internal block subscribe api pins, pins, api check, api cid, delete.

A

A

News to me thought there was a no delete situation with blockchains all right. um I think it's just what's in their database, but bad joke all right, so check has accounts restrictions, upload, okay, so this upload route is what we care about here.

A

These are just some middleware on this looks like they've got some permissions. I like that middleware maintenance. That's nice post course.

A

Without nft upload, so this is probably the thing we need. It is in roots.

A

Ft upload it looks like this is active, so, um okay, upload car, oh wait, upload car.

A

Okay, wait: request blob: okay, there we go. We grabbed the body is car application, car okay, so that seems to be like some kind of multiple thing: car file. Okay, so we're not going to deal with the car file format right now, um two nft response: if his car packed a blob, okay car equals patch.car.

A

Okay, so then they create this blob and they have a car associated with a single blob. So pack to blob is what we need.

A

So, where is pack to blob okay, so pack to blob, is this? um Is this thing that basically just shoots it into ipfs, so.

A

um Two nft response, so this is going to take a source cid, so they create this nft.

A

They return. This nft object.

A

Size: okay, but where does it add it to the chain.

A

Transform db response into nft response; okay, so it inserted into ipfs, and then it returns. This blob, which I'm assuming is what we're going to put on the chain.

A

But not seeing where that happens.

A

Maybe it's nft ad.

A

Tokens create.

A

Ft check nfp stored.

A

Let's go look at their api docs again.

A

A

A

How to custom metadata.

A

If your com, if your metadata, conforms to these standards, oh yeah, that's right that doesn't work anymore, defunct, link.

A

So if you want to customize metadata, you'll need you'll, be using the same off chain assets and many nfts. You can store your files separately and get their ipfs urls.

A

You can then use these ipf's uris and put them into a json file. However, you'd like to create your metadata, this metadata can be stored using resulting in an ipfs uri that then you can use that you can then store in your tokens on chain record. So basically, what it's saying is that go ahead. Put your token on the chain or put put your put your um str put your data in ipfs.

A

Put your metadata onto the chain as an nft.

A

Submitted to the convention yeah yeah you're, resulting in an ipfs uri, then you can store in your tokens on chain records, so the ipfs uris must be somehow unique.

A

Javascript, client, library, storing asset files, storage, blob, okay, so the upload okay. So this is the one we just found upload.

A

So this uh says the thing about car files that you looked at: okay, single file: this is the response. We got multiple files, so basically it tells us that we can shoot stuff at this nft storage and it will upload it to ipfs for us, okay, great cool, um so.

A

We can use this so or we can do it directly, so preparing your metadata. So once you have all your assets updated, you can update your metadata using the ipfs urls in these images, others that are part of your for each file, preparing fpfs some cases. You may wanna, okay, but basically try to put that yeah.

A

We read this last time so basically try to put the stuff in ipfs and then access over gateway to that same thing, but we're going to use, manifests and data flows to be our generic method of access need to use cidv0 your two version of ipfs contacts. Okay, so we don't care about. Ipfs maintain your nft. You can now mint tokens using the blockchain platform of your choice. We wanted to illustrate the minting process here.

A

Avoid strain okay, so this shows you building a flow nft.

A

This tutorial will teach you how to build a simple nrt market using the on the flow block. Chances flow next blockchain.

A

A

A

A

Interesting, okay, so.

A

Coinbase ventures interesting. So what is this.

A

This code flow, that's cool.

A

A

Flow applications.

A

Get event event: blogs collection.

A

This looks pretty cool.

A

Does this use d ids.

A

Because we care about the format, it would be nice if this did because this looks like pretty pretty easy to work with pretty good ecosystem. So um because then we could just spin this up. I'm not understanding how we get nft storage to interact with. Let's go look at firefly.

A

So, basically, I think nft storage is now. We don't even really care we're just going to put our we're just going to put the metadata on the chain. Okay, we're just going to spin up firefly.

A

A

A

A

So it must be something interesting sure.

A

Verifiable organizations, network team.

A

There is interoperability.

A

Credential exchange proof proposals, the ids public didst.

A

Okay, these are like they're testing, yeah.

A

Their matrix of tests, just like we're talking about sort of the commits this is a good example of like how we're going to map the forks to commits.

A

A

I don't understand where the main aries project is.

A

Okay, sound decentralized, there's a protocol, different tools, protocol source.

A

Agents, dids protocols.

A

A

A

A screw not a working indie, specific person, you want to build decentralized, identity capabilities, okay, so you should use india and I think we found that we should be using firefly versus indy.

A

Blockchain, okay, hyperelectric cactus.

A

Okay, yeah, we saw that I think we've determined firefly is the right thing.

A

Okay, so fireplace import, so it'll take takes inputs and outputs um and firefly um we'll.

A

Use the api d: what was it called.

A

A

Gateway interface at first then swap.

A

This operation for.

A

Okay, we can implement a cleanup operation which turns, or we can have our imp.

A

Enter dump uh firefly jump chain to cold storage. We can have later an alternate implementation of this, where we um uh load fro or where we just serialize to did after we know format in cold storage. Okay. So basically, what do we say here? So we're going to spin up firefly we're going to hit the gateway interface, we're going to start adding?

A

um We can we're going to start adding adding uh inputs to the chain right, we're just going to link this to to everything um or we're just going to link a couple inputs into here. Put it in the chain, um then uh we're going to we're just going to add their data indiscriminately um and then we're going to you know, dump it out to the chain we're going to have it. uh You know: do that.

A

A

Breakpoint we're going to do that here. All right, so try yield fire effects. uh Finally,.

A

To do uh stop firefly, okay, so just like we do with our docker my sql server spin up docker spin down docker when you're done, um and in this case uh we'll also add a to-do dump or output chain.

A

To file right so, let's create a flat file or files or or a bunch of files, that's appropriate that we can uh oops. I keep hitting the server.

A

Stop firefly I'll push into files and then we're gonna say you know to do to do implement.

A

Implement load from files on start, if given right and so then we can take those load parameters from the um you.

A

Know argument another argument: uh you know so another path left up half argument after cash right just in case, I'm totally forget what I'm doing later. Okay, that's exactly what we're doing okay! So let's go ahead, we'll download firefly! um Let's see, do we have these fields configured correctly now, okay,.

A

Cash downloads, unexpected indented block.

A

Okay, so we're gonna download the firefly cli and firefly input import. So this one is just gonna run, uh let's see so what did we wanna do? Yo? We grabbed we were coming down here and we wanted to say if self.parent.inputs equals depot inputs, otherwise, okay, so we wanted to basically grab.

A

We wanted to define an operation that whose run method uses the custom inputs, and then we wanted to.

A

You know call it so. Oh yeah and I was renaming things into odap. Wasn't I.

A

Okay, so this is firefly.import um the inputs.

A

Are blank and the outputs are blank?

A

The config class is firefly config.

A

We need to make all this easier does no expand and printer great. So this looks good to me. um This looks like it'll download that and then throw us a break.

A

um Let's look at the body of the implementation here: let's go ahead and remove all this stuff.

A

And so this is what I was talking about. Basically, we can just subclass directly from the operation implementation context and then wrap with op so that we support the creation of the operation implementation and we can still pass amp enter.

A

So then all we have to do is define imprinter, which we did so this one, I believe, was the one where we. This is the example where we'd redefine the inputs. So we're going to do that. So, basically,.

A

uh We're just going to leave it sort of you know open to open to the caller to define. um So this is, let's see, let's drive, let's, uh let's just work off of this right, so we're just gonna say um you know. Instead of run dataflow.

A

It's going to be.

A

Firefly import.

A

A

Okay, so firefly import and then so what are we going to do here? This is firefly input, so.

A

So this is our import.

A

So this is our import when we.

A

Okay, let me just finish rearranging.

A

A

So we've got our cash firefly cli we've got our firefly config, which takes our values right. So these are our default values for this operation. Whenever it's instantiated, uh here's the setup for the operation, here's the operation itself, let's give it a name firefly.import.

A

A

A

Okay and then we have this alternate implementation down here right, and this is just uh the same.

A

So this is firefly uh import uh gateway.

A

A

Import gateway.

A

And this is uh firefly.importpeerdid.

A

Right so we're going to output a pdid directly, there's no imprinter! There's! uh You know the output is going to be.

A

A

And the name is peri pure did so. This is the definition. The name is a pure did. The primitive is an object right, because this is a um a live object right, so we could serialize it, but it's not serialized right now. So it's an object.

A

Let us see so when this thing runs.

A

Right and it's got its inputs.

A

We just so create did.

A

After we know current formatted all right, so this is what we're going to do right, we're going to use this function here and we're going to create a did after we know the cold storage and and how we're going to find out the cold storage uh we're going to do this, firefly import gateway!

A

um So, let's see so what what kind of nodemon command do we have over here python dash? U okey, dokey.

A

A

Firefly import, giving.

A

So if we ran unit test.

A

Let's make sure we import the fml.

A

We're going to import that from dffmo.

A

And so do we have to import cash download of packed our forecast? So there's probably going to be a few more of these.

A

Okay from typing.

A

A

Gotta import some types.

A

Throw a few on there, it doesn't look like we're using a list we'll take that off all right. Great no tess ran so now we're going to add a handy dandy little trick here.

A

So if we were to go to the doc test library and we would go to the united test library or well, we already wanted to test we're running unit tests via the cli. So we are just going to go down.

A

All right yeah, so this shows us that and if we're doing unit tests then.

A

um What is called load tests.

A

Okay- so we can put this in here and then.

A

And we can use this to reference, this module.

A

And then it will run hey.

A

A

This is module's name.

A

I just did this the other day. I know it works.

A

Maybe we have to say specifically.

A

A

Wow that didn't work.

A

It just worked the other day. I wonder what I'm doing wrong.

A

A

A

What happens if we put this here.

A

A

So it doesn't run it if it's in a class is that the deal.

A

Didn't run it now didn't like it.

A

It's not the decorator.

A

um It's because we're redefining the name.

A

A

Yeah, it's just this gateway here.

A

It's always the dumbest ones,.

A

Okay, so firefly import gateway.

A

So we'll build this data flow.

A

So this was a data flow that did run data flow, so we're going to do last path, firefly import, so we're just going to add last path to that.

A

Firefly gateway, import.

A

uh Firefly gateway.

A

A

Okay, so there we go.

A

All right there we go now we got firework, firefly gateway, import, import gateway, dammit.

A

Okay, there we go all right so now we can import it. So we can say config. So we don't need to config here. So we can say: outputs, okay, so inputs so we'll say: inputs equals.

A

So we'll say last run this lastpass last last path.

A

Append these inputs.

A

And we don't need the subflow anymore, so we're going to run this last path, we're going to run this firefly import gateway and.

A

Okay, we don't need to worry about the output operation.

A

All we need to do is add that value um so.

A

A

Data flow is not defined.

A

Firefly import is not defined.

A

A

No dot didn't find and replace okay, so download firefly cli machine, two required positional objects, so default is set on both those. So why is this not coming from here because url hash value, caster?

A

Okay, so for now no we're gonna spell it out, because that's terrible. I was just about to be terrible, terrible, terrible.

A

See see how easy that was. Okay, I didn't have to expand. I was gonna totally just expand it and that would have been bad. This is more clear.

A

Face config has no attribute download url, so why is it using base config so does it not have it set.

A

um I think we're on an old we're on an old branch yeah. So there is a we need to commit.

A

Okay, where is that it's on the manifest branch so base config.

A

Okay, there we go yeah, so there was an issue, and this is you know something that like alice, would identify if you know so. This happens to be me- and I remember, um but uh you know if this was somebody else who would run into the same issue right, they would say, attribute error base. Config object has no attribute download url. Now this is something where alice would have. You know seen this exception.

A

She would have seen this happen almost like stack overflow for every single thing. You run right and you see why I think this is because, obviously, every time I save a file, it runs everything. So if we could only do this across multiple computers with lots of people or lots of computers, we get a lot done so right see I I was just able to cherry pick right.

A

So this is where you know: you're gonna, say: okay, what what am I working on now right so watching watching developer, watching developer, watching developer, right, okay, um developer hits issue known issue right, traverse, tree, look at input, object, determine feature, do feature extraction on input, object right or you just do see a direct lookup. Have I seen this thing before what happened? You know what what you know does does it relate? How does it relate to my current system context? My current top level system context.

A

How do the solutions that I can find you know relate to the assets that I have available to me within my top level system? Context, for example, like a developer's environment, we're going to do this whole thing with vnc it's going to be sweet um okay, so now we've got that, let's rerun it's gonna, yeah, okay, great see, look at that.

A

Caster looks like spelled caster without the r there we go all right now. Why am I running it this way? Because why did I not just run it.

A

Why did I now run it some other way um well or via the cli or something because we have.

A

A

Firefly cli weird it downloaded with extra dots, okay, because this here should not be like this.

A

All right so we'll remove that.

A

I think we should download with logging now, maybe it doesn't. Maybe it goes into the dock test. Environment. Oh yeah. I think it goes into the dock test environment. It doesn't preserve logging. I think we have that integration set up.

A

There are no regular stuff, but not within. This is just me running a one-off. um So where are we here? I wish there's. I think there might be a way to show the text around you a little bit more, but uh here look at this check this out locals import pp print pprint.ppprint locals now. That is what I call fun debugging. I love python.

A

Okay, so tree.tools file name looks good, firefly, cli download. Look at that ff all right there. It is looks good, looks good. Okay, breakpoint yield, firefly, cli, okay. So now, when we get into here, we should see I'll return, print inputs, self print print or well self.parent, dot, firefly,.

A

Okay, so yeah, so basically, this is this is and you're seeing doc test run here, and it's saying I was expecting expected last film and empath okay last element. This is where that is right. That's where that coming from so if we were alice, we would say um you know: we've identified this string here. It says that it expected this string right. So there was an issue. We know we're running doc test. We know previous executions of doctest yield.

A

This example we see this is the failure uh you know this is the line failed in dock test format. We can write strategic plans that tell us these things explicitly for this format, and we can say you know this is the expected line. Is the next line, and then we can say alice can say: okay! Well, this is you know the output like? Is it this that's wrong or is it this that's wrong right? um We should be able to tell things like that over time.

A

I'm not quite sure that one, yet you know when you have something where okay. This is wrong with. This is the source of truth right. The source code right well, okay, we're in the middle of changing the source code. So maybe you know how do we? How do we tell um I'm not quite sure anybody got any ideas posted.

A

All right, so we got that looks like it's a binary. uh We validated the sha. Let's go execute it, so um I think we may be lacking.

A

A

Okay, so we recently introduced this thing. A few of you have interacted with run command.

A

So this basically a little interface around running commands, yields, lines and events interesting. It was interesting to write this like a little slimmed down version of our event. Loop.

A

I like it, um let me know what you think, I'm curious, so basically yeah. We can use this to that's weird. I didn't understand that code block.

A

So we're gonna run we're gonna use this.

A

Until subprocess.

A

A

So delete all that delete all that yield. Firefly cli we're not going to do that. Yet here, okay, yeah. uh What are we going to do so logger equals, so these have a logger, so we can say self.logger. So then I'll just log all the output of the command and we can say sys.executable, okay, so we'll say um so: firefly download.

A

Ff okay, so we're going to run this firefly command. And what do the docs say that we should run here.

A

Come on where'd, you go.

A

Ff init, okay, so ff init all right. So what what? What was that? What was that that we just saw there in util? So this looks like it's about to prompt us for some inputs. Oh, uh oh, how are we going to script? This? uh Oh okay! Well, that's a fake o! Unless you can't hear my excitement um because we know exactly how we're going to script this.

A

um So what was that that we just saw here um I'm just very pleased with this clearly um exec sub process. So what does this do so, let's actually just copy this whole thing.

A

We're actually going to just copy the body yeah.

A

So this is what we had. This is run command and this is the body of run command. So.

A

A

A

Where's, our command.

A

There's your command all right, lost kw args here.

A

Do we have keto here? No, we don't have kwrx, okay, so.

A

And we'll say: logger equals self.logger.

A

So they're calling our args we're just going to throw them around here, so subprocess.creator said diff dot. I think it will expose these automatically. We have some automatic exposure of uh all variables within the code base from the init. So all right. What do we got here so looks like same stuff.

A

um I don't think we're seeing the logging on this yeah we're not seeing the debug logging. That's right. I think the doctor already said the doctor's thing gets rid of that. So um what are we gonna do here, so basically uh subprocess created. So let's just set a breakpoint.

A

Break point break point: okay or maybe we should say print event, results. Okay, um put.

A

And we'll put input or maybe no yeah, we'll say print event, result or point.

A

A

Okay event: result: subprocess, created, created, okay, continue print. If error, we can just say event.

A

Result completed was it.

A

A

Does it not have standard input shouldn't stand, oh because we hijack stanford standard input, okay, oops?

A

Okay, let's see what happens at the break point.

A

Did we hijack standard and put it away from it? We might have hijacked it standard input and then it tried to read and it was like. I can't read all right: let's just try down or running that real quick tools.

A

Oh because we just ran it: okay, okay, oops, all right. I forgot to add this: okay yeah, let's say init.

A

Okay, what does it want us to do? Init, okay,.

A

All right, that's the started event now it should prompt us for something unless our standard input has been hijacked which it looks like it has.

A

Oh, no, it didn't read a whole line. Let's see, can we switch the reedline mode.

A

A

Can we say read what happens if we just say read.

A

A

A

Goddamn surprises so what's happening, so nothing is hanging.

A

Okay, so that was bad.

A

Another trucker: what does that do.

A

All right, let's see if we can just do some kind of unguided thing.

A

Guest besu fabric corda blockchain provider, we want to make sure we're using one. That's.

A

It says it's taking on.

A

Multi-Party system enterprise database, purposeful system d transfer, centralized.

A

A

A

Yeah yeah, okay, maintains identity.

A

Search for runtimes.

A

Okay, I just want you to spin up a did based chain.

A

What is hyperledger grid.

A

We wanted to go into.

A

Grid, so you can use splinter.

A

Saber is the neighbor.

A

A

A

Running grid on saber.

A

Splinter privacy focused.

A

Sawtooth, okay, we know sawtooth.

A

Sawtooth would be a good way to go for the provenance information that we want to be collecting.

A

Do you consume the ids.

A

Every ledger grid.

A

Let's see where it wasn't, smart contacts.

A

Unsaturated, oh my god, can we just how do we get from some from dids? How do we get from d jesus odap help? How do we get from dids to sawtooth.

A

How do we do this great stuff.

A

So what one should we go with, though grid or splinter? I don't understand, why would we go with one over the other? Does it even matter? Should we just go with this did stuff? I don't understand what did stuff works with other stuff? I feel like I'm getting confused again.

A

I think if we can do the did stuff, we can get it somehow here into this. So the question is which one of these things does this firefly support, so bessu fabric quota.

A

Fabric did decentralized identifier.

A

A

That's you was it that's here.

A

A

Passive address, especially badges library and tattoo, to use answer, but it's fully managed to search list and wrap them in a did document. This is proposed to essentially okay, so, but this is old.

A

Did get yeah that's what I was thinking is this? Isn't that.

A

Chain: okay, damn's get that's what I thought.

A

There's just one more quarter.

A

A

What is this thing back to twitter.

A

A

Daniel daniel burke, okay, what is this thing.

A

What is this thing.

A

There was this thing: it's in our thread, too, wait. What is it this? This t something.

A

Okay, this looks cool, so where was this tb dex? So this tb dex? This is a blockchain based on.

A

Decentralized web node cell sovereign identity, service, 3xy.

A

A

It is not sourceforge, it's important.

A

We really hope it's just not published. These projects are open. Okay, identify early issues; okay, decentralized web node; okay, developing a reference implementation of the web node; okay,.

A

It's a protocol.

A

There's this repository okay, so this is where we can use their okay decentralized standards based primitives for using decentralized identifier, verifiable credentials, life cycle protocol, okay. So this looks like what we want to look at okay. So it looks like that. Firefly thing, after all, turns out, didn't really do what we wanted. It didn't spit out post a blockchain. It can't split blockchain. Okay, this is the problem with the approach.

A

Sometimes it didn't uh it doesn't it doesn't actually get us there, so we we should just we should we we we learned we learned, but it turns out. We actually want to just hang out in this tb dex spec, because this is a much better place to hang out than it seems like, because we don't really want to deal with running stuff right now. We just want to make sure that we are serializing to this protocol.

A

So this is our new best friend.

A

This is our new best friend this repo.

A

Which will help us encode and decode to these pdids? Okay, thank god wow! This is so much better alice.

A

It's purpose focus on fleshing out. Okay, I need to get involved. uh Who else wants to get involved with me? What types of messages do allison pfacing each other? No idea who pfi is it's an interesting name um just kidding all right? Let's message: storage, processors, oh I'm so excited! I just knocked over my water.

A

Alice look you're going to be talking to participating financial institutions.

A

Riveting evaluate payment risk.

A

And what do we want to use risk for off chain execution.

A

What do these messages contain? What does state machine look like for a thread of messages? Did library, protocol library, great.

A

Tb text protocol message, format, instructure message type. ah Yes, yes, yes, yes, yes, yes, this is just exactly what we need.

A

A

A

These people are our new best friends.

A

Helen do and.

A

Mr mo we salute you, helen, do and mr moe.

A

A

Tvd logo, that's hilarious.

A

A

Where did it go.

A

This is great. This is exactly what we outlined. Okay, I'm going to take a break.