Bisq Developer, 22 Aug 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Bisq Developer Session 22.08.2019 - Testing Roadmap

Description

See written agenda and minutes here:
https://github.com/bisq-network/events/issues/28

A

Yes, so welcome to the testing roadmap. Deaf core. Today's agenda is well, let's say all about testing I'd like to start with a general discussion on on the testing proposal and and the testing framework proposal.

A

Then Christophe will show us a work in progress on converting or doing a doing a test class in with the Kotlin language, so that we can discuss and maybe decide to use, Kotlin or not for testing at least then.

A

The next point is I I proposed a long time at long term strategy and a short term strategy in the in the testing proposal, so I'd like to discuss first, the long term strategy and how we can get that up and running and then to discuss the short term strategy and where we draw the line between testing where an addition works or if our feature set works and at the end we'll do a quick summer summing up and that will it be for today.

A

So I believe you guys made yourself familiar with their proposals. At least some of you I did.

B

A

Do you have any, do you have any notes, thoughts, questions just to start with, on the on the proposal.

C

The testing proposal, not the testing framework proposal, right testing.

A

Proposal, yes,.

D

C

Think we should. We should get to the point as soon as possible. There. We can at least read some some tests for every new purified threat.

A

Yes, that is, that is the plan.

A

Well: okay! We, if the, if that's all the questions you have generate questions, do you have them I, believe we recheck this one and I'd like to hand over to Christophe. Maybe you can give us some info on how Kotlin works for testing, even if it's not yet ready for production?

A

Okay, can you take over the screen? Yeah.

C

You have to stop sharing okay,.

A

Start a stop, stop share.

C

Okay, so this is like all of the most complex tests that we have and it's also what I'm currently working on the account signing. So basically, this is already copying code, but it's basically just Java code, converted from Cortland and and the Travis Travis file. It looked exactly the same.

C

It has a lot of variables because there's like the three accounts involved in designing tests and and it's it was really complex to read so I thought it would be good to have one cast at Howard's or the data for form for one entity of designing courses so I just so here we have a data class.

C

Basically that that's that's how you would in Lombok you would in Java you would just use mumbo and you can just see a beta class with all the fields and you don't have to set anything. It's and and so string and compute instructor and everything is automatically generated for you and that's I, think really useful. So so basically this test- it doesn't work right now, but I.

C

So the design is true and science race is still in the old style and sign it. With this one I converted to how I want to hold the test, look like so basically I, just I just created, like this sign, big mistake house was with the with deciding key.

C

With this hash and and and and basically or all the fields that the separate.

C

Yeah and and then and then I can just instead of always creating it like this. I can just omit a method quartet, witness that just course the decider. It was constructor with mister parameters from be stable, cancel. Okay, so, and the interesting part is at some point, the the first test or or.

C

Very secret, just the second test, tries to test what happens when a signature is not valid and- and that's possible right now, because because I create the signature in the data class because I dublicate it every up but I can just I can just move this here, then it's that is just a parameter which is default value and then in the next test, I can just I can just say: make assigned witness with account one and copy all fields except signature change.

C

The signature to a battery, that's invalid, so I think that the name parameters and the copy constructors make cop little much more readable than the Java code. That I think that it's especially important for unit tests or testing in general.

A

Okay, you said we could get rid of the long box stuff yeah.

C

I mean I mean this. This take a class here, it's basically something I mean we don't use Lombok in the unit test right now. So basically we use we could Lombok. If we used coupling for production code, I mean, for example, if we take okay, maybe that's not the best example, but we we do have a lot of. We do have a lot of cal systems are basically just beans, and then then we could get rid of number goodnight I mean people say that is complicated.

C

I think that that Lombok is also really complicated and I mean it's also pretty exotic, in my opinion, but anyway, right now we're talking about tests.

E

May a champion here, I mean there's one single we wish with Lombok, which is I, think it's wrong reason why we should try to get rid of it long term, because security I mean it's using suspect. As far as I know, aspect-oriented programming to get all this done and I think that's probably more risky like like what we want to have nobody.

E

When most people don't really know. What's going on behind the scenes with Lombok and it's yeah, we should be more conservative with I, wouldn't have had it any more after I knew that how Christine beam's told me once that it's using AOL and so, and that was kind of bad the risky decision at the end, a.

C

Good point I think, for that point view proteins, that's probably when we start using, if, if we ever start using carbon in products right but I, think for using for unit tests now is maybe a good idea, because I mean the unit tests that we have right now they are not really very readable, so it's I think most of in in a pretty short time frame. Most of the unit.

C

If we are successful, most of the unit tests will be new unit tests or reading unit tests and and so I think the right thing getting Kotlin. It's just I think a unit tests for me latest is like it's like documentation also. So if it's just bit easier to read with this boilerplate and that's for me, it's a good reason for me to use Kotlin yeah.

A

Of course, if you, if you remove boilerplate code, it's it's more readable, it doesn't matter what language you have yeah.

C

Yeah I mean that's basically, the.

C

The tested on right now and I think that once once, I have I mean in the end, all this private pass stuff will be gone and there will be just there will be just three accounts and if I want to change something, I can just use the copy constructor, and it's just I mean know all the way. How to write this test in in this readable way in in Java. It's just possible. Okay,.

A

So what you're saying is you could reduce the number of the lines of codes necessary for the very same tests by let's say, 50 percent or thirty. Seventy I mean.

C

I mean it's hard to say because rewriting the distance intuitive it needs to be, even if you rewrite it in in Java, but so it can be simplified, also each other, but not this basically without the copy constructor which of the named parameters. I can just say: okay, give me the same thing, but with this and it's something that you need very often in encoding and it's in test, and it's also not not about number flying to me. It's about how how this reads because I mean in the old test.

C

You need it all in separate barrel rebels, because, because you have to in test you always you always use, almost the same structure is just on difference and and this copy constructor basically tells me it's exactly like like in the like account one. But with this, and that's that's a really nice way to write it and.

A

You know it's very well, yeah, you have less variables, you have less lines of code, you have to be familiar with, and and so you can see the arrow or the feature fast, even.

C

With the variables, if you look at this, if you look at the lines below you just don't know you have to okay, this is account to signature to sign the truth. It's like you have to. If you change like this to 3 I mean nobody notice. The threat. Okay,.

A

So what you're saying is in this very example, the lion 128 is, is the Kotlin way to do it, and the Lions below is the the traditional Java way, and you just.

C

Basically automatic translation from Java home and then and then I wanted to change the style and basically introduced the data class. That holds all the data for one almost and.

C

Gasps I don't know, but I would not have the copy. Constructor and I would not sorry if, if, if I use the copy constructor that long book, if we can probably also copy this but I, don't have a name parameter because Java doesn't have that yeah and I also. And that also can say normally, please calculate the signature automatically, but you can also override it because I mean because I moved it from from a field to default, parameter, I.

C

A

C

Mean it's very simple: it features and I think that features that we can start using, invest and then people like it and grow more familiar with it. We can use more complex features, I mean all the all the Java 8 stream API stuff. As much and I say equivalence in caution. It's so basically, it's just I think I. Think using it in tests is a good way to get off it, and then we can see.

A

Let me just a backtrack to their question. I asked you: what what do you think is it's possible to save on in plain lights of code and we already discussed? If you have less lines of code, you can creek her get the whole picture and answer.

E

A

Complexity, what do you think we can? We can save with with with Kotlin versus Java I? Think.

C

It always about 50%.

A

50% is the number and.

C

There was also the got the LED status trota to read and and like I said, I mean if I just change this to a 3 yeah.

D

C

Will notice it some because you have to you have to when you scan the java code is history. You have to say okay, but what's exactly the difference between line 114 and 115 I mean it's very hard to read because of the repetition, and so you I think that and all the and all the things that you read is really relevant. Then it's relaxing to read. But if you have some lines where just boilerplate I think the boilerplate lines, they're really bad for the concentration and.

A

Then the boilerplate I insist is the stuff where you overlook errors and stuff. Yes, of course,.

C

Every line is relevant, yeah good understanding, not not to be running cook. Understanding of the test, yeah.

B

B

Like the same readability as you've got in line 128 in two lines in Java, okay, yeah so but maybe I'll show it a bit later when I will present the integration test. If we, if we get to that part.

A

Yeah, okay, yeah sounds great. Thank you, Christophe I believe I believe we got a picture on what the difference is between Java and Scotland and how we can maybe get better codes and more efficient code out of it more readable code out of it.

A

The strategy I believe we can go. Maybe we can agree on is that we can use Kotlin, at least for some tests and then see how I did how it goes and maybe decide. Then, if we want code lean more prominently in disk or not yeah, if you are there every are there any questions or thoughts on this from from the audience.

E

If there a way to convert back from Kotlin to java as well and ID, know.

C

Cokely to bytecode and from bad code to travel, but I think you don't want to do that for tests because they serve a very specific nature. Yeah but I mean you know, I think this should be really small, so I think once you have figured out how to test something that that's that's the most that's the hardest part of it so rewriting and also anytime, you rewrite a test. You can make it a bit better. So if it's just, you can probably convert it to travel from cop living yeah.

E

I should because most of it, it just I think which have to be aware, even if we start now, just with a few classes and it becomes 20, 30 50 hundred classes. Are it's it's a decision. What we cannot revert easily later, when we want to revert it, it will have big price or the other. So we have to be sure if we don't really want this and one when we want to do this. Yeah.

A

But the other way around is the same thing. So if we create classes in Java and then want to use coding for for whatever reason you can't convert them. Yes, but you didn't have Java in Kotlin version, but it's not coddling code, its Java code, converted to console and so yeah.

E

But I mean if a standard what we have no idea. Yes,.

A

E

Yes, changes yeah.

A

The the thing we are suggesting is I believe maybe I can find a few lines that societies more prominently. If we we try to use coding for tests. If we feel that we can be more productive and get more readable tests by using Kotlin, then we have succeeded if we do not feel that way. We stay with Java and get rid of cotton eventually, and and that's it.

A

But if we, if we feel that cotton is maybe the way to go, then we shouldn't miss it because, as I personally have gotten to know, cuddling is that you can write more more code, more functional code in less time, with a lot less boilerplate code, and that is alone a thing that I would give a chance at least a chance. So well, that is that that's the just a reason: I believe we should. We should try calling for tests I.

B

Just want to say that to write something efficiently in any language, you need to know that language. Well, so I, don't I, don't think I could write efficient language efficient, cutting tasks without knowing cutting, which I don't right now and I. Don't know how many of you guys know it, but if like there is only one or two people with no Colleen I, don't see other developers writing system coupling.

C

No developers writing tests at all, so I mean when you write the test. Usually you get you look at the most similar test. You look at the test to test something similar and if the test is in Java, then you use the power code.

C

Scotland and basically I mean you can just write your tests in Java nobody's forcing you to write the test copy exactly and I mean.

E

I saw yeah. We should evaluate how many developers are familiar with Scotland, also who could already right now Copland code and who, how many would to basically need to learn it to be effective, even if it's easy to run. But it takes some effort on some time and because I mean the review process will be different. Like I myself, when I saw the code link code, I clicked over I would need to learn Catalan first to meaningful meaningfully a review, and that will be other developers as well.

E

So when would turn out that you and floran are the only guys who are really familiar with Scotland, basically, either everybody is forced to learn it to be able to review, because it shouldn't be that only two people are cross, reviewing each other or yeah yeah, but with other good situations.

C

But the if you take a look at this sudden sent witness service test I think if you only know, travel and not caught him, then the coupling version is still easy to read the old term abortion, because yeah.

E

I said for me at non Kotlin or no means the method was copy, and then the this extra overwritten parameter was not completely.

C

E

I, do you think it's a competition I mean sure you can get it's not hard. I just say I think to meaningfully refuse things and there will be more yeah, some way advanced language features, and so it will take some time so and I tried it myself. Mr. Knightley a library once there was some issue and I try to look myself into it, because it is some nobody had time and I thought.

E

No I cannot do this, because I would need a few days at least to learn how to interest, and that's not a big problem. I just want to point out that we should be clear at the current developer community what we have- and maybe everybody which is here- maybe he can add some input.

E

If he's familiar with Scotland or I'd love, we said I think he's familiar with Scotland, even if he didn't saw that it's a good moment to to edit now and Christa fonescu or maybe a kingdom if they, if you're Willie, would be interesting to hear their statements as well.

A

E

A

Believe a super leaf way.

A

We can create at least a few cutting tests and then we can evaluate let's, let's, let's let people look at it and say well, do you understand what the test means and if it's okay, then it's okay and we can decide that there will be a proposal coming up, I, don't know in a month or so whether we should use cotton or not, and there will be more discussion about it, of course, but we believed it for for the for upping the testing coverage, we can easily create some examples of how to use cotton.

A

Maybe just maybe people will see well, that's maybe the better choice and the more efficient choice. Once you had your one or two or three days going to learn Kotlin but then every class you write takes half of your time. So maybe maybe it pays off. Maybe it does not, and that is the reason I'm.

E

Pretty sure that it's it's about the language and probably pays off, as just would like to see a little bit, how is the current from our developers coverage basically and and what they think are like I mean Devin is here for maybe Devon and wits.

E

You can add your comments, what you think about it and if you're familiar with Scotland and I, put a Kristoff and excuse well, I'm, not sure if dick and he crystal cannot join and the students on one holiday but I think it's important to get the voice of every other developers as well. On such important topic, I.

C

Mean let me let me say something about this, because, even even if you know Java, maybe you don't know how to write efficient, give a test in Java. So even then you would need to learn something new. Even then, you would probably need to take a look at a similar test and say: okay. Why is it done this way? So you have yes, I'm mocking library. I mean testing is hard so that that's, if you want good testing, there's a lot to learn for everybody and and to doing it in Kaplan, actually makes it easier.

A

Yeah I believe we can leave the discussion as it is now. I have noted that we will do a kind of a Perl and see how people are familiar with and figure out how we proceed on the current meta offline or in a fugitive car, but I believe crystal review.

A

If you stay on the topic on on creating tests in Cortland, we have something to look at and we can decide on this basis and I believe that Christopher alone is not able to to do hundreds of testing classes within the month two years or or do you.

A

Cotton is very effective yeah, so we have a couple of classes sent and we can take them and evaluate if we want cotton or not yeah, okay. So that's on the topic of Kotlin. You will not hear about here about Kathleen anymore in his deaf car I believe.

A

Are there any other thoughts on the.

A

Proposals and on the goals we have defined for I have sketched for further further proposal. If there are no questions for you, I want to go on to the to the long term strategy and discuss how we get that started and if there is any anything unsettling about it, are there any questions regarding the general discussion.

B

A

Okay then check that off so for dinner.

A

Well, I actually I have to just quickly.

A

I have created some approaches, approach, proposals for short-term and long-term. The short-term proposal is about getting getting in effect in the rather short term, so that we can.

A

Have testing on our site and the testing helps us getting things done. I propose to create an automated test framework that spins up a small piece could be to be a network, and so with anything you need. There is an alice there's, a Bob there's an arbitrator, it's eat now the Bitcoin, node and so on.

A

If we and and every time our Travis is going to do a test run, it fires up the whole small P to be a network of bisque and sees if things still work.

A

There is an obvious pro advantage to this approach, of course, because we can automatically see if things do still work and maybe we can reduce manual release testing release in in parentheses.

A

If we have this, if we can save time on manual release testing, we can use the resources we have in other ways, and another advantage is that we can somehow streamline the testing procedures innovated.

A

Firstly, a reviewer is not has not to do the whole manual release, testing thing for every single pull request that is out there and it's very time-consuming. Secondly, a developer sees well if this whole big integration test thing says well, my pull request is fine.

A

The developer can submit it with a clean slate to the to be a pull request to be merged and, of course, if you have a very new developer, that is just starting out and maybe sees in the issue list some some minor issue with with the GUI, where there is some I, don't know pop up loop or something like this.

A

It does not have to do the whole setup of the of the physical development framework just to fix this very small little thing in order to get started just the whole stuff, and now it's the time, I asked you for for your thoughts on on this short term strategy. What do you think about having this integration test that simulates a whole p2p network and and tries to do some some trades and some arbitration cases, and so on?

A

Would this be? Would this be good for bisque to do it in an automatic way, or should we spend our resources elsewhere?.

B

Well, I, don't see any tools for integration testing like end-to-end testing with the Java client, so to click through that we.

B

If we would have API, then we testing on the API level. So we could that's the trade protocol, which could be very useful but still I, don't think we could test the GUI itself in the end-to-end manner.

A

Yes agreed, but not as you have your created tapi, so there is some some some bisque API there. It is in the core package: I, don't know where you, where you can we win and and use it to do or risk you can do without a buoy. Isn't there.

B

It's not merged yet. So it's like a big thing that is under underway, but with that API doing the HTTP request, you could do the whole trade set up the account the payment accounts post and offer least offers take an offer mark of paid mark that the fans has been received and then withdraw to to the wallet and check if the balance in the wallets is okay. So so this trade level stuff that which is I think like the most important that's this.

B

This thing should be automated and it must work bisque, so my API would be really good match for those tests and actually I do have tests like that already and on the branch. But.

B

That the interface for those tests would be the HTTP API and we would still have to have manual tests of the UI.

C

Okay, I mean that's one possibility, but but that only works pretty good. Then when we have a lot of code between API and the rest of the app, and if we have to I mean the API will probably not call any any test of trust. So I think the API should just be a small wrapper around around the core model.

C

And yes, and if we have that, then we can, then we can just use things. We can't use Java cause to the koi API without doing any HTTP. It's just. We can just spin up the test and then do kava course to the to the and and I think that the that API tests I mean not HTTP API, but really the Java API tests for the core module that that's something.

C

That's really important and also I think we should not only have the integration test, part of it, but we should also have some kind of functional test, the don't that doesn't really do much of tcp/ip and tour. But it's really fast. So we have.

C

We have assimilated like that and the p2p that, like running a like, like in in bomb in bomb virtual machine spin up five instances to talk about the peer to peer network and and use that for functional tests or unit tests of the of the core module, and maybe with the same API, have a different mode. How to run the test in the real to environment, with different via.

A

Now this you try to review HTTP API into the into into bisque, somehow reverse the interface between these your API. Is it the core module somewhere.

B

Mm well, the HTTP API tries to be a rapper or for like the core elements like trade manager or open offer. Managers like that so I'm trying to do as little on the API level.

A

B

Course, the currently in current state, the the way that the code is written, is very much tied to the UI and a lot of logic happens in the UI, which should be refactor, so I have to mimic some of that code, that is in the UI and and I keep it in the eye level.

B

But I would like to ideally move refactor the core, so the sudden API doesn't have to do like anything. Apart from just passing. The restaurant translates the HTTP call into the Java code to specific method on some manager and just and translate the results so.

A

I understand that you, you see a way that we can use the core API, so the trade manager and the manager and stuff like this to to get this to get some integration tests running up and running that very you can create an offer and acceptance of an see if, if everything works,.

B

That's what I do right now: I'm spinning separated docker containers, so they work like separate computers and they talk to each other using.

B

Our peer-to-peer network- and this is very good for like real end-to-end tests and because there was like no way for now to beam up programmatically instances and stop them, because I think this is what Christophe is proposing. So what we from the test, we programmatically on the same machine just spin up two instances of disk and programmatically drive them so I, don't think there is now any way any easy way yeah.

B

Ideally, we could do that, but too much tangle for now and I think what the API is, the easiest way to achieve: programmatic access, the.

C

Api Testament they're really slow. If, if one test takes like 50 seconds and we have five hundred hundred hundred tests, then it's like when it runs for an hour yeah.

A

That's not visible, but we can. We do not use tor and then it's much faster I think.

C

My proposal was maybe- and maybe that's in addition to the other thing, but my proposal was to to spin up 92 instance in one Java Virtual Machine, luckily in like here.

A

C

A

C

Mean yeah, but.

B

You will still have peer-to-peer communication between seed, node and and two instances like Bob and Alice, and it this is that this stuff that takes time, it's not that docker is slow and starting the instance is slow because it's separate container. No, no, it's.

E

B

Biscuits office now.

C

Yeah I mean we will see because.

C

We can do it in like on virtual machine not like JVM, then we can decide what part of the communication we're going to mock and- and we can also make that programmatically and say. Ok now, this one is from the the very fastest version of it, because I'm still developing and I want to get feedback on my changes and then on CI rerun, the diversion that goes but steeper in its round-trip I, don't know. But it's it if it's all Java code, it's very configurable.

A

A

So pan out, what you're saying is that you for your HTTP API to be used for such a level of integration, high level of integration test? You had to create recreate some logic that originally has been in the in the in the GUI. To get the whole thing up and running. Is this correct.

B

hmm Yes, example: some validations of the you are of the offer a load happen in the UI, but you know that's like current state of this initial API.

B

Development and I think that, ideally, we should refactor the core so that API doesn't have to do any logic. Apart from just translating the data structures do.

A

You see a way to.

A

Spin up tests only by using the Java core, API now short term, without much hassle, that we can at least get some green lights on if the. If the code still works or not,.

B

I, don't think so, like internet I, don't see, I, don't see an easy way because stuff is like really complex, the whole startup process. This is really complicated and for me it was really the only feasible way to just have the machine started as a as it starts. Right now and and then just have some programmatic access to it and the the API was.

B

The king bet bet aloud that, but I don't I really would wouldn't do it that way having what Christoph is proposing. That would be ideal, but I don't see it to feasible.

A

Okay, so one way is to use your API to do to get fast and short term test up and running the test. The whole thing into end, because you already put some effort into creating you HTTP ABI and the stuff that is necessary to get just the thing up and running. Well,.

B

With the API the hardest part, is this I think the first, the first pull request that I have created because it contains all the dependencies. All the dependencies. I relayed are related to the rest framework. The jax-rs- and this poses like the greatest threats and concerns that maintainer have right now and once that's done then like subsequent.

B

Changes are small and.

B

There would be some refactoring for the trade manager and offer manager required, but I think that this shouldn't take too much time.

A

Open up would you would you care showing us what you which we already did testing wise with your API sure.

A

So please go and show your screen.

A

Okay, so there is to sum this up: there is no easy way to test agree for now or without much effort, and there is no easy way fast way to get. The API emerged into the project because there are, of course, security concerns.

A

Yet it is not possible to test the core Java API to use the core Java API, to spin up such tests, because some of the logic is in the GUI or in the HTTP HTTP API. So in would it be? Would we maybe would it be feasible to two weeks to do some refactoring first, so that we can remove logic from the API or the GUI, or something like this to get the whole thing up and running yeah.

B

I think so I think that the the three factoring should be the part of accepting like subsequent pr's. Can you see my screen guys I.

A

Do not you have you Internet screen, it's black for me.

D

Yeah, it's just a black screen. Oh.

B

Just a second, maybe I'll, try to join from because I'm on the different.

D

B

Maybe I'll try it from the black browser is having some issues sometimes so. Just a second I'll try to join from.

A

In the meantime, kristef, do you think it's can we can we? Can we find that the missing parts can we refactor the missing parts for such an integration testing into the Java API, the core API? What means this be a good, a good way to start things up. You.

C

Know, basically, will be always. We always have to use the core API for our tests, because, even if you, even if we have an HTTP API some some data that some code- that's it's needed it doesn't it doesn't it tests the HTTP API test, because the only part you know it can only test part of the best of efforts she had between HTTP API and in desktop. So just just using normal JVM tests for the core.

C

Api is for me still the the best way, but but but I know, and it's it's a bit hot, because we have a lot of of shared global state. So if you just give me m to run my two instances, I mean that's, then you have to get rid of a lot of the static data that we have look but I think that's the main problem. That's.

A

The what you go and it's.

C

B

Way to go, then you see my screen guys. Yes,.

A

B

Okay, so, as you can see in this changes history, there is this. My IP.

B

Security frame or API over tor and then I've got subsequent like comments which, right now each is on separate, separate branch and I would like to create separate PRS once this minimal is accepted.

B

So this come, it contains changes with the payment account only another with arbitrator management. That's required most people for the tests creating and taking over, and then there is other one for the like completing the trade. So I think that with each of those, each of those commits and separate pull requests apart from getting the exist. Adding wrapper I would also include the refactoring of existing core modules and drive being, but some stuff already is being modified.

A

Panade, would you please just focus on on your test cases so that we can see how you two did and somewhat end-to-end tests yeah.

B

Okay, okay I just wanted to just mention how I would see how we could progress with the getting functionalities into API and together with refactoring of some code. So.

A

I will schedule a staff curve for your API in the future. Sir, please show us that stuff.

B

B

First of all, we have to build the.

B

Containers for the 40 API in headless mode, so there is docker image descriptor for the docker container.

B

Here with docker compose, you can start Bob, Ali's, signaled, Bitcoin, note and arbitrator. If you want to do like manual test manual, interaction and.

B

Once you built the containers, you can run the test and this is sample test, but mm spins up here you got the configuration of the containers, so this is decorate. If you just declare, this is going to be.

B

Alice- and here is my wrapper- for basically, you use our Klien API to define what container name it should have from what image each, what docker image it should use. What comments should be used to start a dance and pour the bindings waiting strategies stuff like that, and then you just you, can annotate your tests, but they should, for example, happen in sequence. This is good for it's not good for unit tests, but those are end-to-end tests actually so to save time.

B

I execute them in certain order, for example, I need to I can test do the test. Let's try to do some. Some calls that should not change any state of the network for any internal state in the applications.

B

So that's why I ran them first and only afterwards. I do test that modified some state. It's cool, okay! Let's look! How how this test create, offer no arbitrator accepted. It should return status, 422.

A

Sorry, what you are doing is you use your starting up. Whole disk network, a small piece network with believe I, saw Ellison pop and an arbitrator and and see no to.

D

Something is the.

A

Bit cannot so you have have the whole network simulated and then you select either Elias or pop, or something like this and say to them. Well start a trade for example, and then you say pop well, except the trade that Ellis just published and and you can see if it works.

B

Yeah, so here I try to create offer and this. So this is what I'm trying to do. First part in the name of the test. Second part of the name is like the conditions or a scenario. So, for example, no arbitrator has been selected by by this client I think that here I'm using.

B

Create offer template, and here.

B

Yeah so I create offers against I, Alice client, so I'm using Alice fourth of the office API to communicate with others.

A

Okay, so so that we don't get lost in too many details, you are doing end to end integration testing with your API. Yes,.

B

Exactly so, I'm sending a request to Ally's to create an offer and well this test actually should not change any state of the network or Alice. It should just return me information, but something is not set up properly. For example, I have not selected arbitrator, so he.

A

B

Appropriate status code but later on.

B

Create offer happy path. There should be something like this: a.

B

Month, hi, okay, okay, so this test and tries to create offer with a valid payload and has sufficient funds, and so it should return me the offer so.

B

Yeah, so here is the part, but does the the initial request to create the offer it should read now it makes assertions on the response and yeah and then like there is like separate, separate tests, but checks if, if this offer is available on the offer list, so here I'm using the same note, I'm using the Alice port but I could use also the.

B

Bob's so another container, so another disc client to see if the offer has been propagated and I think this is done in the tests. Yeah with the take offer yeah and the take offer actually is using always the other guy. So it is going to use the Bob port because the I do exposes like on one port and Bob exposes the API to on another court.

A

So that if there is you you're, basically simulating a whole trade between Alice and Bob, and you would check if there is something wrong in. Let's say the coral appear to be a network because it just doesn't simple: it doesn't work. You get an error.

B

One of those tests at some point not even not won at least one would fail most probably if you would have problem creating offered see they're like really many tests, so those are create offer and then there is, there are many tests or taking the offer. So most probably all of them would fail, but that's fine. You just want to have everything in green with the end to end and like the whole flow must work.

B

B

Any questions from anyone guys I.

B

Think that it would be wonderful if we could a sort of divide parts of bisques, for example, to have like one module for just the peer-to-peer communication and it doesn't matter what kind of messages are being sent if it's a trade messages or some completely custom messages, just some a model just for communication, and then just do integration tests that are just sending sending those messages.

B

That would be wonderful, but I do I, don't think that that is feasible in any foreseeable future to bring that kind of loose coupling into and I think that we should first have tests that give test end to end and then try going into details.

B

If we would start a new project, then probably following the pyramid of the test would be cool to have like tons of unit tests, then a bit less, but still many integration tests, but have some sort of isolation and then like very few tests. But but there's like at least the happy path of end to end. But in this position, where we have already worked product I think.

B

Starting with end-to-end tests, since it is visible that would be awesome and then maybe drilling down with integration tests that have some stage of isolation down to unit tests. That would be good but I, think with starting with end to end, would be optional.

B

A

Okay Thank You panel, but quite impressive, yeah. Okay, so.

A

I'm well I believe we have to do some offline discussion on how to get enter and integration testing up and running. It's it's it's not as easy as as yeah. Well, we knew it is not easy, but it seems there is a lot more to it than meets the eye. At first glance, I would suggest we talk about how we can get this end-to-end testing live offline and see how we can we can create the tools we need for that.

A

Well, I like to close to the short-term strategy, stuff I think we agreed on that. It would be great to have the green light. Weather pull request or release is at least there is no major fault in it, and everything works as it as it did before.

A

Although we might get some errors- and there is made say well, I, don't know what happened, but it just doesn't work and for short-term strategy. I, think that is good enough, so that we can see well, there is an issue and we do not merge some. Some 40 pull requests into the internet master branch.

A

Yes, I like to proceed now with the long term strategy and, as you can see here, on the right hand, side. This is rather sketchy I'd like to hear I'd like to discuss a bit about how how how fine-grained we should do unit testing I'm. Not quite sure, is there some some reference. What the unit testing is and what integration testing is, and maybe we should somehow establish a common ground on what the words mean, but I'd like to discuss what what approach we should?

A

We should do a sketch it here in three categories for all T dressed test-driven development with this feature test raises testing. If we had two and two, if it is three or four in India in the in because in the state we are currently, if we, if we enforce that, every pull request has to test its changes might be difficult, and there are three reasons: maybe the pull request introduces or fixes something that is desperately needed to be fixed.

A

Yet there are no unit tests or any tests for that matter in place before what should we do? Should we postpone the the fix we need so desperately until there is a test coverage there?

A

Second, second thing is: if we, if we change something in the in delay- let's say heart of bisque: let's say something in the peer-to-peer network, optimization in communications and so on. If we wanted to test this without there being any test in place before it's it's it's, basically, you have to create tests to test everything. There is no slow start if you have to touch the heart of bisque and, of course, the last thing.

A

What what about corner cases should we test them extensively, or should we focus on a good testing courage, but not 100% testing courage, because the last 20% always takes 80% of the time? So what do you think? Yes, I? Think.

C

The qualities are the most important to test.

A

C

I mean I think that so, but, but usually what rule of thumb has always been for me, the the person who writes the code writes as many tests as he needs to feel safe about it and maybe also to the reviewer. Like you can say: okay, we need more tests to get this approved and then for everything that ever breaks in production. You have to write the test, so any n is anything that can do that. The Dino did go wrong at one point tend to be a test for that.

C

So so, basically you don't meet tests for every feature, but you need to test for every park.

A

Okay, that sounds interesting. It's quite a quite a nice idea. I think.

B

Yes, yeah I agree, but you must understand that this contradicts with this, mmm which one was it yeah. So the the PR requires fixes some critical issue right and we desperately need it.

B

So we know that there is an issue, so we found the bug already we're doing a quick fix. So should we now create the test.

C

I mean it was just just just somethings about it to really be sure that the fountain real back. You first need to reproduce the back by writing saline unit tests or feeling integration tests.

E

When I jump in I think that's not wrong, at least I, as you know, I'm, not the big test and I worked five years without this, and it's working productive application without any major tests. After an immature back so I'm, just like I did smaller fixes over the last week's I, just around application reproduce the stuff in a way that I can see. Okay, here's the problem and then I'll fix it and it's a different style of testing, maybe in a way but and I, don't want to be against this testing.

E

Of course we need to get there. It's just that, especially the difficult stuff is very difficult to test out those which are related to weird circumstances. From from timing, especially, you don't get there the response from the peer-to-peer network, because you got a timeout because something else was consuming too much resources or whatever how to test this I mean that's, it can be. Everything can be done it just when the test takes, then one months to write and normal fix takes one hour, I'm, not sure.

E

What's the trade-off, that's a little bit the difficult thing and we must not forget I mean we have a much much the biggest problem in business when we don't get the new trait or the call done in the next half year. We are high risk that they shut us down. Basically, so we have. We must not forget that this is the biggest danger and not that are exciting a pass because it works actually pretty stable.

E

I mean there are parts of course, but there are no major parts and no major problems at the end, I mean, of course, it's not say that we need to improve everything just want to keep the.

E

The view a little bit from from further away that it doesn't help you, when we are now getting super strong and getting stuff tested which actually works in production, which is just have not test coverage and, in the meantime, there. Besides stopping bisque when they're attacking the arbitrator, so yeah.

C

I mean method if it doesn't break that me doesn't be tests, but I have to test if it breaks the production. It's a test and and basically yeah.

E

But I said I I mean there was one issue which was just merged her, which was about why startup was taking very long and often Newton didn't cutter, psq or blocks and so on, and the reason was there when you were not connected since awhile. You've got too many trade statistic, objects and the way how it was protest was not optimized, so that could take her. It was not a user thread, so that could take a lot of resources for a minute.

E

In the mean time, you lost and connections, and that's why everything was slow and in stable how to test this I mean that's not. The fix was not hard. It was basically optimization from this code. Then it was instead of one minute. 15 milliseconds and the problem is gone about how to test it.

A

The real issue with testing such issues, as you just described, is to recreate the network status so that the issue yeah.

E

And not only the peer-to-peer network, the Bitcoin network entered out because all three to play together and all three are heavy and are influencing each other and those actually the critical stuff. These problems with this instability at startup we have since months nobody works on this. Nobody fixed it. Every new user get annoyed because of you need to restart, and you don't get connected at all and as a toes are really hurting things.

E

Work need to be fixed, other small vendors, whatever yeah I mean it should be fixed, but those are really the critical stuff, because all the easy stuff got fixed any all the time I mean when I was when it was a I thought and could fix it quickly. It was easy to fix.

E

Another open part, for instance, is that sometimes trade message has doesn't arrive or that that I don't know what's the reason, but sometimes people have the the publishing after or the creation of the after trade transaction of the deposit transaction doesn't work and I looked in this I said: that's it that's a difficult problem, but they're very important, because we got arbitration cases because of this and too many at the end, not terrible bad. But those are those they're, really big problems, the really big box.

E

What we have in bisque owl super super difficult just to to fix and second to test.

D

So I know I'm the new guy, but I just wanted to give my two cents. So many years ago, I was employed at opera software, which is a more Weejun company, making a web browser, at least until they were bought out by the Chinese.

D

But anyway, um I was actually employed as a QA engineer for testing the web browsers when I first started my career there and I think the biggest thing I took away from my testing career with the software is that obviously a web browser is probably the most complicated software application ever you know just ever seconds, maybe only the operating system itself, but the most productive use of testing in general was having a very scripted test, suite for an actual human to kind of run through, like step 1 open the app step 2, you know go to this website step, 3 click here and you should have an expected result and if the of course, if the actual result does not match the expected result, you file an issue and developers look into it.

D

But how do we have anyone who actually does like a full release test before we release a new version? Yeah.

E

That mean I, don't know if you can talk or, but if you can talk, maybe you jump in otherwise I will give it a little bit of you, but just interrupt me when you're available therein is a professional test. So it's a job sync now either way. Maybe you take over table 7 yeah.

D

Sure we do have a manual tests, we find we use a tool called test pad, so we do. We do have a a wide selection of different tests that test various features. There- they're not very detailed at the moment, but we we do not perform a full test, every release, and mainly it's it's. It's Christoph that is doing any final testing before publishing a release.

E

A

E

Everywhere we would need more support, because the basic idea was that every developer helps at their release testing and we had to just to spread the workload at the end. That was mostly Devin and Christopher doing the biggest chunk of the work. So it would be great if anybody could help for the next release, because especially our next release, which contains the current master, will require full tests of everything. Basically because we touched so many different areas in the code and yeah.

D

I think when before when, when the next release build is ready before we release it, I'd really love to sit down with Devin and kind of go through the test suite and maybe try and improve it with them to make it more. You know robust, but yeah. Let's do the full release test fit together. That sounds so very productive. Actually, he absolutely sounds good. Okay,.

D

I'll go back to muting my microphone and just watching the presentation.

A

Well, okay, so basically we we settled on. We have different opinions. The first Christophe said who enters the back. We should recreate the back with a test and then fix the back, which is one way to up the test coverage. The second point of Manfred has been well. If there is something really wrong, it often is because the environment is special, let's say, and it's hard to to recreate the environment, to actually reproduce the bug content so that we can fix the back end.

A

So I have a test that prevents the back from happening again, and the third step has been a third point has been from from with and and Darren.

A

You have this human interaction where the script says well, open the app and if you see well, they're just open, but there is nothing in it and you say well open issue and say I, just open the app and there's nothing in it, and it's not trivial to reproduce to create automatic test. That does exactly that.

A

So how do we? How do we get started? Business yeah as.

E

I didn't want to to reject the idea, of course, it's great when the reset button should be done. A test to cover it. I would just want to say that it's not always possible or the costs, for it are sometimes so high that that it's not feasible, because we don't have unlimited resources.

E

We have actually very low resources and basic I think we should just try everything and keep the goal in in our mind that we want the solid software and we also need enough resources to implement the missing features like the new trade protocol, and we must not lose this out of sight that this is actually the most important of all the facts. What we have that we have a big vulnerability with the current rate for the color I need to fix this. So.

A

Maybe maybe there is, there definitely is some refactoring to you. It has been for quite a while, and maybe we can agree on once any of us takes up the task of refactoring. Any small piece of characters are collected. Then it's it's it's necessary to create at leasts feature functional tests not on the 2+2 equals 3 lever, but I put this into the test and there should be some result, and this would be no exception of something like this, at least before we attempt the refactoring stuff. Is this a way to go? You think everyone.

B

Well, I, don't think we have. We won't have like the golden golden path. None is good, I, think we, or at least we can't afford some that could like Christoph, suggested that I think it would be really wonderful if we could I agree with Manfred with that we it's not easy and cheap to reproduce every single bug. It would be wonderful if we could test or write tests for as many bugs as we can find, but we don't have resources. So I think that we could.

B

We have to prioritize on something if we write some new functionality, if it's feasible to write good unit tests, because I think that the unit tests are doable, integration tests will be hard because the code is tangled because it testing is about writing testable code.

B

If the code is not testable, if it's not written with testing in mind, then it's like already too late or the cost of refactoring might be too hard too high, but we and for sure we will run into such situations where to have wonderful integration tests, we would have to change really core functionalities mmm that would spawn like I know. 50 files so I, don't we should decide on individual cases, probably yeah.

E

I think we shouldn't be too too dogmatic here, but rather pragmatic. We have all the same goal to improve the code, quality and more get better test coverage, and we just need to see case by case here. It makes sense here, it's maybe too expensive, or here it's even nearly impossible and just to not lose the broader picture at the end, I mean enter into reality that we are not when we are 20 people and we can put 5 people just for testing and building test environments and whatever stuff but yeah.

E

We are not there and it's the same Bitcoin when Bitcoin started. There was basically no test as far as I know, and it took them many many years and many many developers and now it's super well tested with all kind of tools around for simulating all kind of stuff, like the network and so on, but it took some years and many developers, so we are not. We are still not there and we cannot try to achieve this and forget the other important stuff. What we need to get done like the new trade protocol.

A

So we can agree on if there is some new feature and introduced an empiric rest and we, it is feasible to create at least some tests for this very feature. We should enforce that unit tests there at least some tests are there for the new feature. First, second, point is: if we, if we attempt some refactoring and testing, is possible, we can create some integration test so that we can stage create it's it's a well.

A

We create some tests that we can make sure, at least to a certain degree that the codes as it has been before is the same. The functionality of the code as it has been for the refactoring works the same way as of the refactoring and, of course, if there is some some buck showing up and we can reproduce the environment to to so that the back occurs, we can, of course, if there is not too much resources needed for this.

A

We can create a test and then fix the back and leave the test in the in the code base to prevent it back to resurface again, so we can at least stock up the testing coverage, and but we, of course, will be never on the under golden side, where everything is tested, of course, but I think we can agree on making it on making it happen, but not let it slow down the development of bisco there yeah.

D

Yeah and in terms of testing, if, if it's too difficult or infeasible to to automate the test- and we could potentially just throw in a manual test that we need to run as regression.

B

Yeah I think that it is very important that we start practicing the tests even but start getting this habit.

B

Because otherwise, we just can cancel the whole idea about testing and just go on with implementing new features.

A

Yeah and if we, if we are honest, if we are honest with ourselves whenever we wrote some some code and it hasn't been a very good feeling that we did not know if it actually worked in every case, we can imagine, because, on the last command the last commit that someone commented on the PO requests and said well, this line would be better otherwise or something like this.

A

You do the change and you don't even think about doing the whole testing and thinking stuff again and then bad things happen, and that is something we can at least reduce. If we, if you have the tests in place, yes, okay times up.

A

To summarize the whole thing we talked about the short term strategy and getting a green light on end-to-end testing it. It showed itself the situation. The situation is that it is it. We knew that it is difficult, but it might not be a short term strategy at all, because we don't have the tools, we cannot get it up and running. However, we will discussed it offline and see if we can get some green or red light for integration testing created for there for the long term strategy.

A

We settled on, let's say doing it, what what can be done and just trying where it cannot be done, and eventually we will get the test coverage up and running so that the risk code is gets more structural. It's maybe some refactorings come with testing and everything goes my fluid and more stable and at least the the core. Let's say that the basic functionality where there is no influence of the environment, of Bitcoin, of of the p2p network and so on.

A

We can rely on it so that there is no back in there and yes and last but not least, as Bernard said, we have to practice. We have to do the testing. Actually we have to do it and we will see where it gets us and that's basically it and that's what I wanted to hear so time is up. Are there any other questions regarding this topic.

A

Okay, so I think you're four for joining the car. Let's do some testing and not just talking about it and as usual, I will fill in the agenda. With the topics we discussed and with the suggestions we got, there will be a recording available on YouTube in a couple of hours days- I don't know, but there will be and please feel free to.

A

Let me see array suggestions, what with further topics we have. We should discussed in in dev course and in the topic collector.

A

That's it and thank you for joining, see you next time and.

E

Maybe I may add something as we have another the gross coal I think in roughly one hour, and it's think it's mainly about attracting new developers and making a little bit of noise, that we need more developers, and it would be great this other developers can join as well and also especially how we how we handle the onboarding of new developers when you develop is joining slack.

E

There should be people who are getting engaged with them and trying to get some direct conversation or whatever to get along with all of you, because yeah we are lacking on documentation and on written stuff that we can just point a new developer to so anybody who has time is very welcome to join us think it's at 8:15 in Central, European Time. This is correct. Yes, yeah, okay, plus it sector. Okay,.

D

E

D

By thank your work, you.