Jakarta EE TCK, 9 Oct 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Jakarta EE TCK Community Outreach Call Oct 9, 2020

Description

This call welcomes everyone interested in join and contributes to Jakarta EE TCK.
The call provided two live code demos showing how to run a set of tests and also how standalone TCKs already migrated the TCK architecture from JT Harness into arquillian and JUnit.

► Project website with links to the repository and the jakartaee-tck-dev mailing list: https://projects.eclipse.org/projects/ee4j.jakartaee-tck/developer

Find out more about Jakarta EE and follow:

Website: https://jakarta.ee
Twitter: https://twitter.com/JakartaEE
Facebook: https://www.facebook.com/JakartaEE
Linkedin: https://www.linkedin.com/groups/13597511/

A

Okay, so welcome to the jakarta e tck community call for october 9th. um Today we have.

A

We have plenty of items in our agenda, so, in sake of uh you know, keeping the timing of the call um we welcome to everybody who is joining uh and feel free to yeah. To to add your your name in the attendees- and we have uh is andy here on the call terrorists.

B

Yeah not yet so, let's make something um different this time. Matsim do you wanna, introduce yourself and say hello and oh hold on people, so go ahead.

C

Hello, I am maxim maximnessen from oracle, and part of my job is to do tck, tck testing and to create and run tck jobs for eclipse, and I can basically start with my this example.

C

So if you don't mind, I would like to show today how to run the tck test from command line, basically from localhost. So if you want to test something locally to check something before doing this his job or any any need, and but for this I would like to show my screen.

B

um Go for it assassin can we stop sharing a screen and thank you so much for transferring the knowledge and caring enough to show up and invest this hour.

C

Okay, so my screen is yes here and I would like to like start from the beginning so to run tck. You have to accomplish some prerequisites and I have a list of required stuff here. So the first thing is jdk 1.8, because gck gcks are now run on only on this jdk, so it's not possible to run it's only 11 or something and you can download it from oracle or from opengdk. I don't know any anyone.

C

Any version is suitable and it's required, because ant is essential for this recently we had version 1109, which is the newest from from 10th versions and furthermore, later older versions are not available. So if someone uses 1108, for example, it shall be corrected to 1.99 and glass fish 6. The last 56 can be downloaded from eclipse downloads.

C

It can be nightly or or weekly, but not current milestone one because current milestone, one is very old for now. So if you run tck for jakarta 9., you had to use glassfish version like nightly version or weekly version, but not published milestone.

C

And finally, you have to choose some tck bundle which is available here, and I will open this link.

C

So there is a list of tck bundles here and I would like to show how to use it on local environment. On my computer personally, and I will show how to do this, you have to run you have to open shell it, but for those who use who uses windows, I think it will be some unix-like shell, but I use directly so for me it's like native, and we will create tests for json b.

C

C

Here I will put some script and this script is not invented by me. It's available publicly. It's publicly available on.

C

Jakarta, etc, project. Basically, you can download from github the whole project, but for our needs I will use only one file. So sorry, so this file is hey, I bet. Actually I shall find it.

C

Yeah this file is located here and it's primarily dedicated for running standalone tck tests. So you can just use this one json btck as r sh, sorry and copy its context inside your script, but it shall be. It shall be actually.

C

C

So here I'm copying it and the important part it's this.

C

Is intended to be run a standalone job or task or just locally, but it's not yet decartified. So here you can see some passes which are java x and it's not correct. For now it shall be jakarta and you can you can compare this one json btck with json btck, two levels above here. It's it's more complicated script, but this script is is for for like jobs on jenkins and it's not intended to be used like locally, because it's more complicated it has parameters and it has some predefined or values from external external scripts, etc.

C

So, basically, it's not possible to use it. Just as is script. I've chosen is, can can be used as this, but with those adjustments for path, because this path is essential, it shall be copied.

C

C

C

And now you see that it's better, it has jakarta apis. It has some another variables which will be adjusted as well. So I have to define class fish top level there, because it's new stuff here, but it can be defined like normally by scripting.

D

Just a quick question, so this is this a contribution opportunity for somebody.

C

Actually, yes, if someone is interested it, this person can submit pull requests and it can be fixed because in my opinion, it shall be fixed, but the cts team doesn't have enough time for sure to parse and fix all stuff and those are not important for them, because they are only for some standalone local jobs which can be done by some other team or some other person. So it's for sure it's can be fixed and requests can be submitted. It's not it's like it's very preferable.

C

Actually, I was going to do this, but I don't have time. I don't have time to do this so as well, so uh here we can yeah and for for faster, faster performance.

C

I can just copy this from my.

C

From my previous attempts, because I will adjust this one.

C

When I was preparing to this meeting- and here we have required variables so.

C

So I put this here.

C

Yeah, I am using this tck bundle here, so this is bundled from from the page I was showing at the beginning. It's gck bundle for jakarta jason b, and now we are almost ready to write.

C

The the only thing is is to check this one.

C

C

C

Seems to be fine and final stuff is to to set this.

C

One yeah, and as soon as you're doing this locally, you can basically do some. uh Some hacks like not because I was used to do- is to do this like job in jenkins and in jenkins. It's not possible to unzip it manually or download something manually. You have to put everything unscript and sure for local usage. You can just put it inside script and enjoy.

C

C

C

C

C

So the script is basically done.

C

So we'll try to run it.

C

The longing is in progress.

C

I am, I bet for sure. I forgot to mention that you have to copy glass fish, because without glass fish you can, you cannot perform any tck test and glass. Fish is.

B

Hold on a second, I have a question maxine, so you're telling that we have more than 35 I'm about 46, well 35 for jakarta, e9 um specifications and to run the tck against all of them. We will have to run the tck for glass, fish.

C

For all tcks you can use glass, fish or tomcat, but for for standalone jobs, it's used or for steadily scripts. Basically, glass fish is used because glassfish contains all required apis and implementations, and you don't have to do basically anything you just download glassfish with all current implementations and apis inside so it's built in they are. I will show you they are here.

E

C

Modules, so here you see all all available apis implementations and everything you you want to actually test.

B

So so we're talking about you're talking about a bigger picture, not a specific api, because what I'm thinking about is the.

C

B

C

Testing, what are your testing is strictly strictly limited to tck bundle when you download one bundle you choose which exact, which exact api or implementation you will test and in our case yeah. In our case, we we had yeah by the way this guy is already running here, but it's failing because we had no glass fish.

C

And I will clean it and start again, probably so here we have glass, fish and json btck and json btck says that we will test only api for json b and implementation for json b. That's it. We cannot test anymore, for this bundle and and glassfish actually provides those api and implementation, and so this is conjunction.

C

You have tck bundle and you have glass fish with those api and implementations to be tested. I.

B

Think not maxine.

B

This is for another conversation and it needs to be hopefully in the specification committee, but we have a problem because when we release jakarta e8- and we saw the activity because the the glassfest runs and is makes everything else, dependable, all the other apis get hidden behind it. In all the contributions we either we need to remove. We need to put glass fish, and everyone knows this outside of what is jakarta, because that's.

D

B

It's unrelated what I'm saying here is what you're showing us you're showing us something. That is um what we need to do is say it's not that you need to contribute to glass fish. It is already done. It is easy, you just choose it run it you don't have to worry about. Finding bags. Is that correct, maxine.

C

uh Fighting boxware.

B

And the tck, like you know, sometimes when you run something and you find tax tests within the test, you need to go and fix that, and we are not doing that. We're just focusing on this is to enable you to run the tck and not have to worry about anything else. But what you, what the code, the code that you want to test right. This will tell.

D

You how to run the tck on glassfish to run on another server you you will typically have a different process.

B

A

That that was going to be a part of the feedback forwards. Last question because um uh maxine, as you explained here, you are, you are using this ctc file that contains implementation, because that that that comes with glasses right. um So, if you have another implementation, you will need to provide that.

C

Right, but that is another process right. My point is then, when you are implementing something to be used here, I'm showing just a simple example: when you have it bundled bundled inside glass, fish and it's already inside it, and it's integrated.

F

C

And the issue is that, usually, you would like to provide some new uh integration, a new implementation for for tck to be tested. So you just want to test your changes inside your implementation and if this situation occurs, uh the point is that here in glass fish, you have those models, uh those modules like listed, and you have to replace this one jar, this one jar of your choice.

C

You have to replace it with your implementation and run tck in the same way, I'm showing you so when it will be replaced. It will test changes in your implementation or the whole implementation, and this is the stuff you're asking, I suppose, because it's not required to have glass fish as whole.

C

It's just required to have it like to provide some application environment, but if you for sure you would like to test something more complicated and you just put inside inside those modules, you put your implementation or even api, and you test it like locally and you check errors which are which are shown as a result of those tck testing.

D

But since this is being recorded, we should let maxim finish so. People who want to run tests on glassfish know how to do that and aren't confused at this point. So.

C

So I'm I'm coming back to this tck test and for now we will run.

C

It and here you can see the process of the of the tck itself, so you just if you had tweaked your required implementation.

C

You are seeing results for your implementation. It's just showing you the progress. So basically the whole script is ready to be used here. You see it for now. It's not trick this pure glass fish without any hex.

C

But if you run it, let's say on tomcat you had to provide all your api implementation establish everything is required without proper application environment and only after you can run your dck, and in that case you will be sure that you are using the only yours api and only earth implementation.

C

In this case you are using. We are using glass vision, environment.

C

So for my presentation, I think it's finished, because script is running and it will run for about 25 minutes. We shall not wait for this.

E

Maxim um could you just maybe open up another uh tab here and um show people where to get the domain file the the domain log? Just so they? If, because there's the script output and there's also the um the glass fish domain log that I think.

C

Actually, you have, you have to store output yourself, because it's not the last switch domain. We are in this tck, we don't run. Even we don't start class fish itself is environment.

E

So so is the um the log file that you.

D

C

When you would like to stir the output here, we see the script inside yes, and the last line actually is run of tck test. This is final line and to store output. Just yes, sorry, just use something like.

C

C

This this comment will will store all the output in the run log file. So afterwards you will be able to parse that file and to read from it or to use it somehow, as a report.

D

There are also log files under the tck work directory.

C

Right actually, yes,.

D

So maybe you want to show people how to see the output for a specific test.

C

Here, usually, it creates cts logs here, and there are some tck report folder and tck work, folder, which I used to store output for the.

D

Maybe you could show us one output file.

C

Unfortunately, trans so I'm not sure I can.

D

Use it, you can.

C

D

You can just kill it and then pick to pick one of the most recent tests and show us how to find the output for our test.

C

um Okay, just just.

G

D

Well, right up there we had a past right, so it said you know the top of your screen. There's a there's, a test that says pass. Maybe you can find that.

G

Yeah sometimes we'll find all your yeah. You just pick the first one actually.

C

It doesn't yeah yeah here, so we have it. But no sorry, it's not output. It's just it's just some binary files. I, for some reason I don't have output here. I'm not sure why you did see.

G

The output come into the screen, that was, you, know, part of it, and there were no errors yet so they wouldn't it just to be like the test name and those kinds of particulars in the jtr which are.

C

Sure, but for some reason it doesn't found it didn't find just.

C

Maybe it's not properly configured for now it doesn't.

D

Well, maybe the test didn't run. Did that test run.

G

Multiple tests that there should be jtr files in the um in the work in the work, uh folder, calm under com- I think so yeah.

C

G

Going yep yep keep going, keep going if you yeah, you'll you're, really close you'll find one really really soon there. You.

C

Go yeah, okay. This is.

B

C

D

There's the output.

C

Yeah yeah sure.

D

Yeah, so so for people who don't know if you search for the star.jtr you'll find the individual test results um they're printed to the you know. So so you can go and just see if there is any yeah.

C

For particular tests, it's possible to browse to particular results the results yeah.

G

On this screen is probably like the test folder, which is interesting, so I don't see it yet, but um um it's just you know when you're looking at a failure in a jtf file, it's it it's good to just you know to note the um oh there. It is work out it's in work. I guess um it's. You know noting the path of where the test failure was, uh or you know, I guess the test equals um in this case. um It's interesting. So it's just.

G

If you go back to that source, folder directly and run from there, it's only going to run the test under the noted uh folder. So if we do hit an error you're trying to fix it, maybe you only run that particular test folder. When you subsequently run it again, so you don't have to run for 15 minutes just to see if your code change, you know is addressing it.

A

Yeah what I found, for example, I was testing a little bit of java mail um that sometimes the scenarios are prerequisites are are sequential, they are not independent. I don't know if that is common in all the tcks on the standalone tck case, but at least what I have found on on any particular issue I was seeing on java ml. Is that if you don't execute test a b, you cannot uh expect a good result from test c, for example.

A

So I don't know if that is that I think that's that's helping the question if that is general to all the other tcks or not.

D

Encountered that before, but I haven't spent a lot of time with the java mail dck.

A

F

Thank you. I would just consider that poor testing practice in general, you want to write tests that are individually runnable for exactly that reason, so that you can just run the test you care about. So if there's ordering dependencies across multiple test cases, that's typically an anti-pattern.

B

Hey andy, buenos diaz.

F

C

So basically my presentation is over, so I'm stopping sharing. I I stopped sharing my screen, so thank you.

B

Thank you, martin.

A

B

Okay, let me take another shot. Let me before we start, I think the next one that I want to share is andy iben, I believe, or what is on the agenda hold on everyone that wants to be shown two times before their faces. One come on everyone, one two three go ahead.

E

F

Right, I'm gonna fire up a screen share here I might have to install new stuff, so bear with me.

F

Okay, can you see my eclipse right now.

B

F

Yes, we can okay, um so uh dimitri invited me to give a quick demo here um so I'll just start off with the discussion. I proposed this idea a couple months back on the tc.

F

I think it was either the java ee dev mailing list or the maybe it was a tck mailing list, um but but essentially the idea that I was tossing around was to break out the tck tests from the kind of the monolithic tck repo that maxine was just showing and make these things more standardized and more easily runnable and consumable for for new folks.

F

So for there there's quite a few people on the call that are familiar with micro profile, and this is basically just adopting the same exact pattern that micro profile uses for their tcks, uh but just applying that pattern to some of the the jakarta ee specs.

F

So if you're not familiar with micro profile, uh the way micro playful does their tcks I'll just run through it very quickly. So I've done I've kind of taken json b as a as a test subject here. So I'll show all this in in the context of json b, but it could be applied to any of the specs really so right now we're looking at the json b api repo and we've got. You know the api, um you know interfaces here, we've got the specs and the part that I've added is actually a tck folder.

F

So here the json b api repo owns its own tck tests, and these are, I actually copied the source code from the json b tck tests in in the repo that maxine was just showing and I've copied them into here. So the the thing I like about this style is that um it keeps the tcks and the api in one spot, so that david, I don't know if you're trying to talk on this call or a different call, but okay, he must be trying to talk on a different call.

F

um So the thing I like about having the tcks here is that if we make a new spec enhancement, you know so adding a new feature. We can add the we can have the api changes, the spec doc changes and the new tck tests. All all in one single pr, so they can all be reviewed as a cohesive unit and also, I think, it's really helpful to view the new tck tests, along with a new spec change, because it really helps show like a working example of what that new function is going to do so.

F

The idea with the tcks here is it's just the java classes um and they will actually be published on maven central as an artifact themselves, so they can be published as a binary, and the cool thing here is that any implementation can then consume the tck test as a standard maven dependency. So here I'm this tab. I'm in now is yasen. So yasen is formerly the reference implementation for for json b, um so yasen doesn't care about any and most of the jakarta ee spec implementations uh can run standalone.

F

They don't necessarily have a particular allegiance to a specific app server. They can be run inside an app server, but they can also be run standalone. So with yas and that's the case. You can run yes and stand alone in just a regular javasc program or you can, you know, consume it into glass, fish or open liberty or whatever, and use it in a similar way, um but it for yasen and developers of yasen. We want to make sure that the code stays you know compliant with the tck.

F

So what I've done here is I've added a yas and tck subfolder and all we need to do in order to run the tck with the acid is just add a pom file here and it's just pulling in the json b tck as a maven dependency uh and then also I'm pulling in the arkilian weld embedded container because yes, and has a couple integrations with cdi.

F

So this is just running essentially json, b and cdi in a standalone javasc container. So, oh and I'll also mention that we have for yassin. We have a travis ci pipeline, where every single change that we put push through the yasen um goes through this this pipeline. So we have, you know, check style and copyright checks up front, and then we actually compile yasin and run all of our unit tests.

F

But then, additionally, after this, we actually run the json btck against these new, this new copy of tests and that only takes 49 seconds to run the entire json btck. uh And uh so it's it's very fast. I think what, um with the with the jakarta ee tck, mono repo, it takes, like, I think max team said like 25 minutes or something to run the json b tests. So it's considerably faster.

F

So now I'll switch over to eclipse here and just give you guys a look at the involved code. So the json btck tests are the same thing. That's in the jakarta eetck repo except I've, translated them to use junit and arcelian instead of the custom java test, harness that the tck typically uses. So like I've, just added you know at test annotations and then I've just changed some of the assertions to like the junit assertions or or fails, and things like that.

F

So then, if I switch over to yes and like yes and has our normal source and our normal unit test, but also in this in this yes and tck subfolder, we have that same palm that I was showing. So if I want to run all of the json btck uh with yassin, what I can do here is, I just go. I just all I've done is I've cloned the repo I have maven installed?

F

I have a java installed, which is very, is very standard for a typical java, open source developer to have installed already, um and I can just run maven verify and then that will initiate a maven build and it will run all of the yasen um tests as standalone, and this is just a this is just a basic. uh I think it's maven uh failsafe, maybe it's sure fire. I always get those two mixed up, but it's a stan.

F

It's the standard, maven way of running uh tests so and I think in general, open source developers are very familiar with this, so they they know how to do. Maven test may even verify and say I want to run just this json b config test. I can the standard way to do that with maven is. I can do dash d test, equals json b config test, and then it will only run that particular test class and then it will complete.

F

So there's just the just the 22 tests in that class run and then, if I want to run just a specific test method inside of that class, I can drill down further and do pound sign test get as- and this will just run the single the single test case and then, when we do this, so there just is ran one test case.

F

There are standard junit reports available for this, so in the target folder we just get a junit report that gives us. You know the overall test, the success rate and then I can click into any of these and get a more detailed breakdown of what the test results look like, and if there were any failures, then I believe it also captures the you know, standard out in standard error, streams um and and none I've. I didn't invent any of this tech this um stuff.

F

This is all just standard maven, which I think is good, because people will be more familiar with it out of the box.

F

So that's pretty much the gist of the the proposal. um So right now there are actually two copies of the json btck floating around. There's the official one in the the jakarta ee, tck, monorepo and then I've kind of mirrored all of those tck tests in the jsonb api repo itself, and my hope is that, eventually you know maybe in jakarta e10 we can, you know, only have one copy of the json b tck tests and then you know, sort of carve those out and just have them be owned by the json b.

F

Api repo, for the reasons I mentioned earlier and then eventually other specs could potentially follow suit if, if they wanted to so I'll, also mention that uh the cdi specs and the bean validation specs are already doing their tcks exactly like this, they already use. You know these binary, tck artifacts that are run with maven and our kilian.

F

So it's actually already a practice being used in in the realm of jakarta, ee.

B

I mean, when you say a few other apis are using this. Why is that that it has not come to the tck forum so that maybe those that are not yet there and have missed this could join the effort because simplifying the test time and also keeping everything under one area, it's just a huge win. Are we doing enough to?

B

Are we doing enough to help others in the same position, and is there a way for likely include the jakarta e marketing so that we can push this over media and blog about it or do something that enable us to do better right like at the moment? It seems to me that we're very uh isolated.

B

What do you think about this? I.

F

Think yeah, I I think the decision needs to be. It needs to it's a decision that needs to be made on a per spec basis. So I guess once json, b kind of gets permission to sort of blaze. This trail of transitioning from the monorepo to standalone tcks. Then at least that will set the pattern up for other specs that choose to follow suit.

F

But by no means do all specs have to move to this pattern, and I think it would actually be a bad idea to expect all specs to follow this pattern, because if you look at the more feature complete, more stable apis, such as like say, servlet, ejb and even jpa, those tcks are massive and the amount of work it would take to translate them to this new style would probably not be worth the benefit when we consider how often their tcks need to be run and how often those tcks will need to be.

F

You know added to and worked on in the future, but for relatively younger specs, like jsonb, that I think still have a lot of uh features ahead of them. Then it would be. It would be wise for those specs to to go through the effort and that's what I've done as a as an expert group member on the json b, spec, just to kind of make my life easier, I've tried to do this.

A

Great, I I have a. I have a question about uh what you just mentioned. Right is my voice.

F

You're super quiet, scott uh go ahead.

B

He got a new computer, I had started talking.

G

Before the amelia started it, and I was worried- okay, no one can hear me again. No, no.

D

Yeah, well, your microphone is never actually fixed, it always starts out small quiet and then it bursts into regular mode leading you think that it was fixed, but it's not fixed.

B

Oh so, scott, if you think I cut you it it's bad on me, I usually sometimes do I did it. I didn't I I didn't know.

G

um I'm turning up my oh it's funny. It just turned down on its own. It's saying I did it too high. I don't know, uh listen. um So let me try and ask my question um andy.

G

What would you think about moving some platform level testing into json b? Let's just say it's a cross blend of different containers, which would mean that I guess in s e test and mode, you wouldn't be able to run the x. You know the container level ones and it would be a lot more complicated, but you know in that you'd kind of need, a container that implements kind of like the platform level testing. But this is you know this is um like a platform level question that um has come up.

G

You know when we were discussing on the mailing list before and I'm just curious what you would think about if those tests wanted to be moved or someone wanted to move or the platform wanted to move other test in what had like I mean. I know our killian supports that that's no problem, but as a general thing, that's it's a lot more complexity and there's a lot more chicken and egg.

F

Yeah so yep, so what I, what I would suggest there is that usually we have usually we don't have uh tests that are involving every single spec in the platform. Usually it's usually it's two specs interoperating with each other. Usually you know very frequently it's like cdi plus something else.

F

Sometimes we have interactions between three specs in a single test case, but I think very pretty rarely more than three specs getting involved in a single test case um and I think for that ultimately, the way I see it and the way we do it in open liberty is, is we try to have no tests in no man's land? And what I mean by that is, you know really: no one designs, api and spec. Well, I guess people design spec at the platform level, but no one works on api at the platform level.

F

um So if there is a test case or some interaction that involves two or more specs, we should just move that test to one of the involved: specs repos and then for the in the case where you can test, you know, still have a compliant uh implementation without the other spec involved, then we can have conditional tests where they say- and we do this in micro profile as well, where we say where we actually do have. You know truly optional specs, where we say: okay run this test.

F

If the api for this other integrating technology is available, but if that other integrating technology is not available, then we'll just skip the test. So to give a concrete example, there are some json btck tests that interoperate with cdi, but it's perfectly valid to use json b without cdi. So what we've done?

F

What we can do there is, you know, check for the presence of a cdi implementation on the class path in all the test cases that involve uh json, b plus cdi, and then you know, if cdi is not present, then we just simply skip those test cases, but if cdi is available, then we would run those test cases.

F

Does that make sense, yeah.

G

That makes total sense, um so we're open to addition, like, let's say for ee10 to it, say some a few other specs tests at the platform level moving in and that that that's kind of like a fundamental question. I've had because we have like what like four million lines of um text or whatever in the platform tck- and you know we have a lot of you know- maybe there's two million lines of actual java source sources. We don't want to lose any testing um and maybe not all all specs will want to do this.

G

I know the servlet tc. Sorry, the servlet spec mailing list had a discussion.

G

um A brief discussion in the last few weeks about doing something similar um yeah, like json b, has done um so I I just I didn't get really a clear answer when we had talked about it before about the openness for platform level testing to move in which brings in complexity.

G

I agree I mean there's a whole layering diagram of what depends on what and they're just going to be some tck that maybe will have more than two or three you might end up with four or five. Like things that depend on a lot of other things, I guess so.

E

So, scott, I I think most of that has to really kind of um be derived from what does the spec say? What are the one of those requirements are listed in the tck user guide right? If, if those requirements need you know, if they call for different options, um you know optional features or running in different containers, then that tck should support that in many cases with the platform tck it's the platform that has those requirements in it, and that's where those are.

E

You know that's where those different modes come in and that's why sometimes like, for example, with jms you'll, see different numbers between the gms standalone pck, as opposed to what's in the jms vehicle uh in the platform test, and I don't expect that to change right.

E

What we need to do is come up with a way that we can um keep the core of the the standalone tcks associated with the apis, just like we've seen here uh and then have a way to import that into um into the platform pck and then the platform keep the k needs to add. Whatever its requirements are and then run that test in addition to the standalone test, so in that way, each one of them is separated, uh and you know you you can you I think the goal should be.

E

You can always just look at the teasing at the spec and look at the tck and grok everything that you're going to need to do based on those documents and that um you know those those tests right at the spec.

G

Level is what you're saying so yeah if, if so, no more platform level testing than this, the individual api spec calls for would be brought in directly into the spec tck folder, the spec levels, tck folder, the platform level test will be in the platform core tcka still. I.

B

G

D

I'll simplify what ed was saying is that, wherever the requirement lives, wherever that, whatever specification, that is that's where the tests should live in that in that right. So for.

H

D

For example, uh cdi has uh tests for the ejb integration that are optional from the cdi perspective from the ejb perspective cdi, reintegration is a requirement.

D

So, from the ejb perspective, all those optional tests are all required tests now, based upon that alone, and you would think okay well, if they're, if they're required for ejb and optional for cdi, we should move all those tests from cdi into ejb problem with. That is the specification that contains the requirements and how the integration should work is the cdi specification.

D

So it would be wrong to move the tests out of the cdi realm into the ejb realm and but leave the specification requirements in cdi such that we now have a weird relationship between the two of them where the ejb people have to do the tests, but the cdi people have to do the spec.

D

So if we wanted to move tests like that, we would actually have to move those specification requirements out of cdi into some sort of chapter or add-on spec for ejb.

D

So basically the spec and tck requirements should live together in whatever project governs them. Both.

B

Okay, then, I understand.

F

That cdi cdi is kind of a a special spec because it just kind of has its fingers and everything right. So it's.

E

It kind of defines.

D

F

Yeah, it kind of cdi kind of defines a framework for how to interoperate with a bunch of things, and it should it should be responsible for testing sure that that framework works properly. But going back to the example of cdi plus json b, I mean when you look at a scenario like that. Most of the you know, if you just look at the testing scenario, most of the ownership kind of falls under json b to get that interaction right. Assuming that you know cdi, isn't fundamentally busted and cdi extensions.

F

You know work in a generic sense. uh All the ownership falls in in json b, so I think it can kind of just be evaluated on a case-by-case basis. In my opinion, but moving away from, I think it's important to move away from having tests landing in no man's land um at the at the platform level, because then it's um it's unclear to the spec authors uh who who's responsible for maintaining those tests.

F

If they make a spec change- and I think frankly, they just get forgotten about until the last minute, because you know with with these standalone tcks, um I I can verify that the json btck works with every single change I make to the json b api and a json b implementation, um but you know we're not going to be able to run the full platform level tests on every single change like that.

B

I have a question here, and this is super a super good conversation, sure ed you're here and you have run you have started helping or running the tck somewhat. I saw something coming and the tck do you have an input on this because you are jumping onto this.

H

uh Yeah, I think um the I started. I just started uh figuring out how this whole thing works, so I tested it out with the jpa, because the jpa was one of the specs that I had some experience with earlier. In the sense I've used gpa in the past. So that's what I tried it out with, and I uh so I think this discussion is where, for example, the tests for each spec should live.

H

I wasn't really sure I basically pulled it out from the um I looked at the instructions for the jpa tck, so I think this discussion that we've had in the last few minutes is basically about where the tests should live and should be maintained, uh whether with the individual spec or at a centralized level.

H

So if that's what I understood correctly, um okay, so yeah I mean I didn't. I think I'm probably I'm not really sure about all the things that are involved right now, because I've only just started doing it because, but I think from what I've understood is because of all these dependencies and interdependencies involved, uh probably makes more sense to have each like uh get involved with the maintenance of their own tck.

H

I think it's more natural for someone who is probably new and kind of trying to get into the first time that the tck lives, along with the spec. So from that sense it's more natural. But I guess that you know there are more subtleties involved in, um as with like legacy projects and stuff like that,.

D

Yeah the status is you that that that uh um jpa test suite that you ran was actually generated out of the big platform, tck, okay and so the actual source for it doesn't live in the jpa project. um So we're able to produce a tck that looks like a standalone tck. I mean it is a standalone tck, but only in binary form really.

H

D

um And so you know what andy did was basically figure out how to move the actual source out of the platform tck into the individual specs. So that way the standalone tck produced is uh coming from source. That's under the control of the json b project, which is really fantastic. So there's a little bit of legacy there.

D

H

So when we so when we say platform, sorry uh just a quick question so when we say platform tck, so what does what does platform historically mean in context? So what what? What is supposedly supposedly traditionally belonging to platform versus, say, belonging to a certain spec.

D

um The the platform refers to the kind of the umbrella java, ee or jakarta ee specification itself. So there's uh the there's the jakarta ee full platform. It's not never really actually been called full, but it's basically the jacardi platform. Then there's jacardi web profile specifications.

H

D

Those those two specifications.

H

D

And and all the tests have generally lived there uh for convenience, I guess, but you know, however, the decision made was made a long time ago and just continued, um and so we end up with one big pile one big code base where all the tests live and then the the specifications and api apis are all sort of separate, and so after the donation, everyone had the same thought we needed to break up that big source thing of one. You know that one big mono repo as andy's been calling it.

D

You know we need to break that up into smaller bits and move the tests out into the specification projects that actually own that, so that that is a very major amount of work and that's that's what was being presented and- and uh so you have some experience, so you know because you attended this call. You now know why that that that jpa tck looks like a standalone tck, but actually isn't. Quite yet.

A

um I I have a one quick question for andy and how was the migration from from from the jt harness into jet unit and killing approach? Do? Was that be a I'm, even plugging or something to automate all the tests, or it was more more manually.

D

That was gonna be a good question as well.

F

Yeah, it was, I would say, the whole thing took me about an hour. um I didn't do any fancy, plug-ins or tooling. uh I basically just copy pasted it all the code out into like kind of a new skeleton maven project, and then it was basically a bunch of bulk find replace options. So um you know at it. You know you know, searching for all test methods that start with public void test and then just slapping an at test annotation in front of them.

F

um You know changing some of the assertions with find replace kind of yeah, just lots of find replace operations. I would say, um but I'm certainly aware that json b is one of probably the the specs that is most conducive to this kind of of migration and I'm not implying that every spec will will be this easy, but I think, like jsonp would be a great candidate for it, um and probably some of the newer specs like uh like the jakarta nosql one.

F

If that's, if that's coming around, I'm not sure what they're doing for tcks, um but it can be a good candidate for some specs for sure.

D

um Do you happen to have those finery places sitting in your clipboard.

F

No they're long they're long gone it.

D

Was it was nothing.

F

Fancy, though it was just some some a little bit of regex and find replacements.

D

Yeah yeah: well, if you could you make a list of the things that you remember having to adjust at least.

F

I I think I I made this change- probably I think was over a year ago or whenever I first started the thread so any stuff I had is long gone on that um right. I think anyone familiar with the tck and also familiar with junit should be able to um get it pretty pretty quickly. I think.

B

About documentation, andy.

D

Well, yeah, I was just you know, maybe because there's a lot of people who would want to do this and we have them all start from scratch for with nothing uh or, if possible, at some point in the next coming weeks, you could carve out an hour to rem, to pull them both back up and remember what you know just compare them right and left and go oh yeah. I had to change this. I had to change this and just kind of shoot out a small list.

D

That would be super useful.

F

Okay, yeah I'll uh I'll wait to do that until another spec comes forward with interest because I'm.

D

F

Not a fan of of uh write-only documents. If I'm convinced someone's gonna read it then I'll write it.

D

Right yeah, if it's worth the time uh in terms of timing, you know everyone's mentioned 10, but I do wonder we have to do 11 we. So we have to kick out um a jakarta e9 that supports java 11., and you know I wonder if that would be the ideal time to do. Some of this.

B

Work, we don't hear you, we.

D

Heard a no, but it was super quiet yes sing for a while until your microphone kicks in and then talk.

G

Microphone settings again and see what it's doing.

D

G

F

It's working, it seems.

G

Like when you turn.

F

To your left, we can hear you well for what it is. Okay,.

G

That is weird, um um so I I was just going to say: I mean we need to have discussions about like in the case of jpa, in the platform tck in the jpa source tree, there's an ee folder that makes the container level test mostly visible and obvious. So you know following the idea that the platform level test would stay, maybe in the core for jpa, but others would move. I guess I mean that that would be.

F

I would say: don't touch jpa at all. I mean this is this is really this is kind of a problem of sharpening the x versus chopping wood right. I think we still have a lot of wood to chop with um json b and some of the newer specs, but j things like jpa and servlet and ejb are mostly feature complete. They don't really have much wood to chop so to speak, so it doesn't really make sense to invest in improving the tools for those.

D

Well, I would disagree because I mean you know I would love for there to be. There is no actual standalone ejb tck, so I mean I I would love to see that happen. So we could release that spec.

B

Let's, let's close it.

D

So we could release it without having to wait for our platform, uh sure release.

B

So so so we have 1004, uh we need to close it, but andy. You still need to write uh some things in the agenda. So can we throw at you the task of copy paste, the entire agenda and just send it in a new thread, say you know, tck call the date as a subject and then copy paste. The entire agenda after after you are done and then also add the link to it and just nothing else. Can you do that publish.

D

B

We need to publish minutes.

F

uh What mailing list.

B

Of the of the tck ml list.

D

Okay, we've been publishing them.

B

Yeah, that is why we publish the the minutes and because you're the last one, you need to do some typing. Also.

B

um I wonder if you can join again um a future friday, because I believe you have so much knowledge and I think we have a disadvantage on the level that we need to convert you into understanding that some of us believe that documenting now is so important and there is much power onto that and you are one of the people that, through microprofile I bring in the coolest lightness on that side of the fence that we need to decay here.

D

I'm assuming andy that you didn't do anything with the signature tests, so basically, that part is still not freed up from the monorepo.

F

uh Yeah that got brought up on the mailing list as well- I think we'll have to I do- have to run, though so we can. We can hash that out next time, yeah, okay,.

B

Andy, if you can join again, and maybe next friday, that would be amazing.

H

A

Everybody bye bye. Thank you.