Permissionless Software Foundation Technical Steering Committee, 17 Mar 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: PSF TSC Meeting - 3/17/21

Description

Agenda for this meeting:
https://github.com/Permissionless-Software-Foundation/TSC/issues/6

Main topics discussed:
- bch-api rate limits were recently refactored. We go into the 'what' and 'why'.
- bcp-js from Stoyan formats OP_RETURN data for better NFT handling

A

Okay, it says we're live and for those watching on youtube, hang out for a minute. We're gonna do a little housekeeping post, a link somewhere and then we'll get this meeting going.

B

Yeah, it looks like we're live, got the.

B

Link that there I'm going to update the agenda with the.

B

Link and update the agenda with a link to your bcp.

B

B

Okay, the agent has been updated.

A

Okay. Well uh thanks for attending the technical steering committee guys, it looks like it's just the three of us: let's just quickly go around and introduce ourselves before we get started, um I'm chris trotner. I founded the permissionless software foundation and fullstack.cache, which is a blockchain as a service for both the bitcoin cash and ecash blockchains and uh yeah. We're going to talk about some of the software updates today that has been going on around fullstack.cache uh stoian. Once you introduce yourself.

C

My name is storyan jakov and I'm helping with some of the javascript libraries like bcpgs for the pcf.

D

Yes, uh hey I'm joey king, I'm a javascript developer, currently working for bitcoin abc uh I'm working on the cash tab, family of web apps.

A

Nice and uh encourage everybody to check out cash tab uh as it's pretty cool app and browser extension.

A

Okay. So let's go to the agenda.

A

Pretty short one today, uh the main thing I wanted to cover was some major refactors that landed on bch api with regard to the rate limiting, and this is going to affect users of full stack, dot, cash and, and um it added some features that users requested for bch api.

A

um I'm going to also introduce some uh changes that have not been made yet, but will be made soon and could potentially be disruptive to a minimal slp wallet, which is the wallet engine behind wallet.fullstack.cache and some of the other wallet apps under the fullstack.cache umbrella, um and then I uh named this incorrectly stoian but uh stoian's going to introduce us to bcp.js after that.

A

So let me just dive in here. This is the repository for bch api, and um uh this is a fundamental part of what has been we've been calling the cash stack. So it's the rest, api component that talks to all the lower level infrastructure like the full node, the slp db indexer and the fulcrum indexer.

A

So this is sort of a rough diagram here and um the way full stack dot cash works is there's these rate limits and there's actually in in in talking with users uh and looking at the business of full stack, dot, cash and and and talking to some of the companies that also use the full stack dot cash um infrastructure that run their own isolated infrastructure, separate from the tiered services. There's really four use cases uh that that we needed to um target uh the first is users who uh who well.

A

This is actually a corner case. uh One I'm building up to is users who want to build, buy a job token for access for a short period of time like 24 hours. This is not a use case, it's really being used right now, but it's something that I want to target going forward.

A

Also, the I've gotten requests from users. Previously, I only had the 100 rate per minute or request per minute tier and they requested higher rpm tiers. So now we have a 100, a 250 and a 600 rpm tier for people who need higher rate limits and the main reason well I'll, go and I'll go into the reason why we need more. The third major use case, we're targeting is basic authentication which should not have any rate limits applied.

A

This is this is for this is actually probably the major use case uh for for clients of fullstack.cash who want to run isolated infrastructure for their company. They want the system to run as fast as possible. They don't want to be exposed to rate limits, but they still need some sort of authentication so that their servers are not publicly available, so that they're, the only ones using their servers. So that's basic authentication.

A

um The fourth use case is local installations that do not use uh uh either any sort of basic authentication or any sort of rate limits. So for those for that kit use case for, like the hobbyists and developers uh working locally uh bch api now has this uh do not use rate limit flag. It's an environment variable. So if you set that uh to you know to anything other like to anything um really.

A

If it's just exists, then it will uh skip this block and it will not implement either rate limits or basic authentication and then for the the basic off that hasn't really changed for those for those, um we did more thorough testing so that the rate limits are truly not being applied, uh and we added unit tests to make sure that the system's actually working the way we expect it to.

A

um Let me talk for a minute about these higher rate limits, so one of the one of the trade-offs I've been bumping up against and joey you- and I talked about this a while ago- is this sort of um these? This concept of heavy api calls uh there's there's very simple. Api calls like get uh transaction information, which is just one call to a full node and then there's these heavy calls like hydrate utxos or get public key or utxo.get. These are some of these newer, uh higher level.

A

Abstraction calls that actually set off they're one api call, but they set off a chain reaction of apis and so they're heavy in that they they are much more cpu intensive to the server and so the the trade-off. I've been wrestling with is how do we make it fair? How do we make um you know the people who are using the heavy apis? How do we charge them fairly for the people who are using the lower level apis, and so the this is the big change that happened with this rate limit refactor?

A

Is that heavy like, if you call hydrate utxos, which is just one api, call it's going to all the other api calls that it that sets off that chain reaction, that's going to count against the user's um rate limits, and so it makes it makes the heavy api calls. uh You know it's fair like uh and it lets us add, let's fullstack.cash, add more and more higher level.

A

Abstraction calls we've got like one call we want to add is uh a transaction uh call where you get the addresses of both the inputs and the outputs, as well as any slp token amounts for inputs and outputs. That's that it requires quite a bit of computation to get all that information together, and so that's going to that's going to be another high or heavy api type call call. We want to add another: one is um the the balance of an address over time so that you?

A

Basically it's a transaction history, but uh every uh every entry in that transaction history like has a uh an accumulated balance, so you can see the balance of an address over time. So these are. These are like feature requests that people have been have been submitting and we want to provide them, but they're they're, very heavy api api, heavy calls- and so this refactors going to.

A

Let us push forward by adding these these new desired features, but keep everything fair, but that's also why we needed a higher rpm tiers, because the 100 rpm tier is not going to go as far as it did in the past and um and as a result, we lowered the price. So I full stack that cash, the 100 rpm tiers 10 a month, the 250 rpm tiers 20 a month in the 600 rpm tier is 30 a month uh and then from there you can start to jump into isolated infrastructure.

A

So just trying to to offer more options uh to our users is was the main motivation. So that's the main change has happened to bch api um and now that that's been done, uh there's this this utxos get uh call api call, which is a very handy api call and you give it an address and it retrieves all the utxos and then this new little bit that I've never seen anybody do as far as I'm aware we're the only ones who do this. It does this very nice sorting of the utxos um so that anything.

A

So you get this nice bch utxos array and those are verified that you can spend them and they are not tokens and you can safely spend them.

A

uh There's all the if it's an slp utxo it'll get categorized, whether it's token type one or an nft uh minting baton or or other things like that, uh each utxo will get categorized and dropped into an array for its category and then anything because we live in a messy world and and things get messy and we hit rate limits, anything that can't be verified and that one call will get thrown into this null utxo sort of bucket array, um and that way a wallet app can safely just ignore those utxos or or try to hydrate them on a second call, but basically its exception handling is, and so this this function's been around for a little bit, um but it's very different from the way that minimal slp wallet works now, which again, is the the sort of core wallet engine that the the web wallet uses and so we're gonna the next over the next week or two.

A

The focus is going to be on doing a heavy refactor of minimal slp wallet to make use of this call uh yeah. So that's that's basically, what's what's coming down the pike?

A

uh Do you guys have any questions about that.

C

No, we was talking about this from several weeks, so yeah, it's not the big news, just yeah. We are happy with the progress mate.

A

Yeah yeah yeah no surprises uh for those who've been paying attention. Well, uh that's all that was on the agenda and stoian. I'm really keen to hear about this bcpjs. uh Why don't you tell us about it.

C

Can I share my screen yeah turn.

A

That on okay, you should be able to share now.

C

Okay, so I need to desktop.

C

Which one this one: let's try this one, oh sure.

B

Can you see.

C

A

C

Okay, so I have a little proposal for the protocol: name, blockchain, payloads or blockchain pointers, whatever will be so like vcp, and what is all this about in the moment? There's uh already a lot of different like uh type of nft tokens.

C

Like we type, I mean the way they connect the information on the blockchain, the nft record itself and the media, which is connected to this nft, like images audios.

C

All of them are doing this differently, so for the wallet developers and for exchange developers, it's pretty difficult. They need to implement all of these uh tokens differently to show, for example, the images.

C

So this proposal is mostly for the way how to describe uh how to create a little pointer to different resources like, for example, ipfs, hashes or uh or usually other transactions on the blockchain.

C

So I used for a model like we were talking about this media sharing protocol by you, chris and also, but it's based on memo dot cash, so I was wanted to go even lower. So I also read this swap protocol by winner money.

C

So physically, this protocol is is in fact just a simple op return transaction. uh Yes, op return transaction. It have its own low cut in only three parts which the first one is the message type of this uh data. It's very convenient because you can from here get only for audio files, for example, if you want to do the only audio like stuff or only video files, or in the moment this, this wi-fi type, you can just get this one cool and yeah.

C

The second part is so: where is this data like located and it can be inside the transaction itself like it can be a simple text encoded in this op return, so it will become like same like memo cache, maybe or it can be in it can be another transaction which will be represented like transaction id, or it can be url, which is possible not very good but yeah. It can point to some warrior resource and the best one is like it can be. Some ipfs hash.

A

That's great it's great to have those options.

C

Yeah it it can be. I also added here uh uh like bitcoin file protocol, but it's not implemented still, but it's possible option right and the last part is uh the the final. Like uh the real data need to point to this resource. It will be different for every source for ipfs. It will be the hash for uh url. It will be the warrior itself for the transaction type, it will be transaction id, and here is some example. Transaction like op return, this one okay.

A

I'm curious what uh what was your thinking there with the with using locati locat ids versus like the memo dot cash ids.

C

C

I wanted to be a separate protocol because this one can be used by itself not connected to nft. Even at all, it's pointers to data you can just how to see safe on the blockchain pointers to what you want. We was talking about the pointers to sip this sip call. So when you you can get this and make a phone call to somebody something like this one.

C

So I don't think it's text. In all cases it can be memo cash, but I want it to be even lower.

C

So, yes, I implemented. So this is the proposal. It's pretty simple and I also implemented the javascript library for this one.

C

It's ready it can be used even now nice, so it depends only from big big number, no other dependencies.

C

I used this sop media maker very low level, one for a model and sop parser, but this one depends only from bit number and it's.

D

Very nice, you use it.

C

You can include it in your browser or you can just install it. It have like uh some convenient method like you can create this op returns with any type you like in any source, or you can use some wrappers around this one to be even more easier like if you want to create a pointer to to audio file. You just create audio you put here. Where is this ipfs, for example?

C

And here the hash.

A

Nice, nice, so this library is just for generating the op return. It's.

C

Only for app returns.

A

C

So the next step, and when you generate this kind of uh op returns inside the nfts, you just put the transaction id of this guy on the bsp.

C

So nft connects to bcp and bcp connects to the real data. Very cool and yeah, and here is how the magic working the real working example like, for example, you can search for audio files.

C

So I created several.

B

C

Pointing to this kind of stuff, so inside the ticker you put some special like bcp audio and you can see. I have two bcp audio nfts.

A

And so so the pointer is going in the op return, but that's also used by the slp data.

A

C

Optional, but if you have this inside the nft ticker, you don't need to go to get the bcp and then see the type you can see it immediately here in the ticker.

A

Okay, so, okay, so the so the op return data that your bcp library is generating.

A

Does that does that go into a separate transaction? And then you put the transaction hash in the document. Hash of the token is that how.

C

A

Will show it, though,.

C

Okay, so there is the two tokens in the moment: the real one existing already on the blockchain. This one is pointing to the audio with the rain drops. Another one is sos morse signal, so yeah, let's see this one.

C

If you go there, you can see its usual nft child, which is originating from some group and uh yeah. This one is the ticker so from here I figured that it's an audio, it's optional step, but it's very convenient and the magic is here. The document url now is pointing to something which is a transaction on the blockchain with this bcp.

C

So if you go there.

C

And see this transaction.

A

B

A

Interesting this is this: I hadn't thought about putting a transaction hash into the url, so if.

C

You see this transaction, it's just one op return, and if you see this op return, you can see it's uh not this low cut and it's like uh audio type and it's ipfs type, and because of this you can get the ipfs cache.

C

C

C

You're just going to ipfs.

C

And yeah it's slow, but it will eventually return the real one.

A

Yeah, while we wait for that to come up.

C

A

I, like that you used low cat id I haven't played around. I think the main advantage um to doing that is that uh bitdb can index low cat ids, it's already built for that. So if you want to build an indexer to track these types of transactions on the blockchain, that would probably be the lowest hanging fruit, yeah.

C

But I I don't think it's uh how to see. I was thinking also, but you don't need to implement indexers, especially for these guys, because they will be uh nft's payloads.

C

So all of the stuff will be done on the nft level.

C

B

C

Can be interested to only index these guys but yeah, maybe in the future.

A

Well, that's a really good point: stoin yeah, um because yeah when you're talking about so this is very similar to yeah there. It is there's the media raindrops uh yeah. This is interesting because, where yeah, where you really need the indexing is, is a marketplace like an order book, um but that's not this. This is just a payload for an nft, so that's an interesting thought and then um there's something else. I thought was really interesting that you said um that is escaping me at the moment.

C

A

C

With this thing, I can have now several uh like uh nfts, pointing to the same context like in case of the uh like this game stuff. You can have five, for example, tokens uh pointing to the same uh context. You just have the same warriors inside it's interesting.

B

C

Yeah, it's like adding this uh middle level of bcps will increase a lot of nft usage. I think.

A

Yeah yeah, and also like you, could extend this protocol um to point to mutable data. um I've got another spec, uh that's similar to this in the uh in the psf specifications repo. So if that transaction id, if instead pointing to a transaction with an op return, if instead that operation pointed back to an address, then a wall, and that this is essentially what that immutable data pointer, spec is all about is in then a wallet can look at the transaction history of that address and and that can change over time.

A

That's mutable um get the get the latest pointer, um which is the latest transaction at that address. So that's how that's how in this spec you you can take a an immutable token and point it to mutable data. So, like a video game character whose stats change over time, you could represent that character as an nft, but the state of that character can change over time.

C

Maybe it can point to another stuff or maybe what I can add, the the address type uh like uh pointing to address directly.

C

So, for example, if you see id air it will be the bitcoin address or sop address.

A

Yeah, that would be a great yeah. That would be a great addition, um because that that would open the door to this whole mutable data thing yeah, so yeah, it's very easy. It's just. The parser is like pretty easy to be extended. Yeah.

C

A

I love that approach that you took like this library is just for generating op return. That's the only thing we're going to worry about that way.

C

You don't have to have any for generating and for parsing you can parse this uh already op returns data. You can parse them, it will return you a javascript object with the type the source and the data everything it's.

B

C

Both way like from uh from javascript object to op return data and from op return data back to the stuff, so you can get from the blockchain. This op return parse it. It will give you the object. So you can.

A

Very cool man wow very cool. This is great. Like you just blew my mind, I'm like trying to think of like all the different ways we could use this library there's a lot of them, wow cool cool. I can't wait to play with this library. This is really interesting and, like the whole, like idea that, like you, don't need an indexer, that's kind of blowing my mind.

C

Yeah and if this will be indexed because yeah, they already have everything implemented, no need to do this again,.

A

Yeah wow wow, very cool, very cool, great great work story, wow.

C

I hope somebody how to see of the word developers and the exchange developers also agree, so people will start using this one.

A

Yeah, well I mean you're you're on the forefront and everybody's kind of figuring it out right now. So that's I! Hopefully our work can converge here soon. Once we get the utxo get call implemented into minimal slp wallet, that'll, let wallet.fullstack.cache that'll that'll. Let us start building a plug-in for for nfts uh and for different types of use. Cases around nfts, because they'll be able to just tap right into the that those categorized utxos.

A

And then yeah, and then this this. This is really interesting like that that mutable that mutable, um pointing to mutable data spec, a lot of businesses have been interested in that, um but not so much that they've wanted to fund. You know development of it, uh and so if we build a some nft plugins that make use of this bcp library uh yeah I mean.

A

Then then that opens several doors like it opens, like you know a generic nft use case, but then this whole mutable data thing and then um the fact that you support different media types and different file formats makes it really flexible.

C

About this new hub c address source, you can have it even tomorrow. It's very simple change, so you can start playing with it. Cool.

A

Cool yeah yeah this is this- is beautiful. I love. I love how all these different things are coming.

A

Together right on okay, well, I think we can probably wrap it up with that uh joey. You got anything, no okay, cool all right! Well, um yeah I'll, see you guys in the permissionless software foundation, telegram chat, room uh yeah, it's diane! I can't wait to play with your software, and uh the ts will have a com com meeting at this time next week and then um I'm probably going to postpone uh the the technical steering committee meeting in uh two weeks from now uh I'll have a medical appointment.

A

I need to attend, so I won't be able to host that um but yeah. I guess I will see you guys online.

A

D

Sounds good thanks for.