Eclipse OMR Architecture, 5 Nov 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: OMR Architecture Meeting 20201105

Description

Agenda:
* Clean up generation of load and store sequences for Power (#5630) [ @aviansie-ben ]
* Discuss future OMR dependence on Travis CI [ @0xdaryl ]

A

Welcome everyone to this week's omr architecture meeting today we have a couple of topics to discuss. The first is ben thomas, who I'll be talking about cleaning up generation of load and store sequences for power, so I'll turn it over to ben.

B

All right, so um the way we currently generate the load sequences from our nodes on power during tree evaluation is turning out to be duplicating a lot of code all over the place.

B

So, for instance, when we do a load, we have to create the memory reference corresponding to the um the load node.

B

There is an api for doing that on memory reference, we need to generate the instruction that actually performs the load, but then there's an additional step that is actually specific to power that we must take whenever we perform a volatile load and that's mostly open j9 specific.

B

So in that case, if our symbol, reference was volatile, we actually need to emit a memory fence, and the problem with this is that the code for generating these loads is actually duplicated a lot within the coach, because it's very common for us to have code gen optimizations, where a load which is only used in one location as the child of another node, may actually end up being emitted with a different load, op code, then the um then the load itself may imply such that we can perform both operations in one step.

B

So, for instance, we have a byte swap op code and if we have a byte swap of a load, we can do that in one instruction on power and the problem with this is that the code for doing that has to go in the parent, the parent, opcodes evaluator.

B

So in the example I gave that code would need to be in the byte swap evaluator, and what this has led to is a lot of duplication of these uh relatively small, actually code sequences. But it's actually very error prone. So I've actually found, while looking at the code gen over a dozen locations where we fail to correctly emit the um the memory fence after a volatile load, because whoever was writing that code did not think to check whether or not the symbol reference we were performing.

B

The load of was volatile and because of this, we've ended up with a lot of volatility bugs, and these things can actually be really nasty to track down, because they will lead to unpredictable data races everywhere, and there are some other minor problems. This has caused. So, for instance, because we create a memory reference that um is corresponding to a load node.

B

We need to keep track of what node's registers need to be kept live through the duration of the use of that memory reference, and that means that tr memory reference is actually needing to keep track of node information, which is very surprising, but we have to keep those registers live until the caller is done using the memory reference, and that also leads to the caller needing to manually tell the memory reference when it's done using it, and this causes some problems.

B

So it it entangles a lot of node evaluation related code in tr memory, reference which, on its face, should be a relatively simple class. But because of concerns like this, it has become far more complex than it needs to be, and it has been coupled with things such as unresolved in open j9, and it has all sorts of complex tree evaluation logic in it, and these sorts of things just add unnecessary weight to that class.

B

And in order to address all of these issues, what I'm proposing is to split up our concept of memory reference so that we no longer allow evaluators to create arbitrary memory, references corresponding to a load or store node, because it turns out that doing so is just inherently unsafe because of those volatility issues.

B

So the caller must remember to explicitly check for volatility, and it has to have the logic itself to do that.

B

So what I'm proposing in the power code generator is a new interface and I'm for right here, I'm going to call it load store handler where, instead of getting back a memory reference corresponding to a load or store what you do is you request that the load store handler, emit a load sequence or a store sequence?

B

You provide it with the node. You provide it with the op code, you'd like to do to use to perform the load or store and the source or target register, and it will handle emitting the instruction corresponding to the actual load or store, as well as constructing the memory reference keeping track of what nodes need to be kept live and then when they need to be when they stop being kept, live as well as emitting those memory.

B

Fences in the correct locations and another big benefit of this is by making load store handler into an extensible class. A lot of the handling for these weird corner cases of load and store sequences, mainly revolving around unresolved and volatile, can actually be sunk down into open j9.

B

So basically, this does remove some power from the evaluators, so they can't just generate a memory reference and use it. Multiple times turns out. That's actually not something you should be doing for a volatile at all, but in certain cases it may be useful, especially useful on sisk architectures like x, but not as useful on risk. Architectures like power, where our loads or stores are typically done in separate instructions.

B

So we generally don't have memory references as part of a complex instruction.

B

um So basically, what I'm looking for is any comments on potential problems. People might foresee. If we go this route or other things, people want to bring.

C

C

So how will this relate to read barrier and write barrier evaluations where we may be loading things that are, you know in the vm and then calling out to arbitrary sort of side effect handlers to notify you things? Is there going to be any change required in those kind of sequences.

B

There is going to be a slight change required in those sequences. The way I've handled that in the load store handler api is that it is while it's normal mode of operation is to generate the load or store sequence itself.

B

It does have a separate mode of operation where you can ask it to compute the effective address that a load or store would would operate on, and then you can then later ask it to perform the load or store separately on that address, and while that is a bit more error-prone, um that that flexibility is required, particularly in the right barrier, but the vast majority of locations will be using the simplified interface.

D

Do we allow uh memory references to be reused on on power in x86, because we certainly don't own nazi.

B

Not on power, however, there is a way to construct a copy, a clone of a memory reference on power which is done occasionally, so you can effectively use the same memory reference twice if you copy the original.

D

Yeah, that's the same way it works. So how does x86 solve the issue of node reference counts and memory?

D

A

From my perspective, I would have to go back and look at the code. I can't tell you off the top of my head.

A

C

It's not part of the code. I've looked at so.

D

Okay, because from what it seems, uh z does already one thing where it artificially increments reference counts to make it transparent for the user. uh So if x86 does something? Third and we'll have three different implementations across the cogens.

B

Yeah, um as I understand it, x, is currently doing the same thing as power where the memory reference itself is keeping track of the nodes that need to be decremented.

A

Later, okay to phillips point: if if there is a point of convergence here, um maybe we should aim for that.

A

I think the comments that uh some of the discussion that was happening in the issue seem to suggest that that's certainly a possibility between z and and power, and if that's the case there, then um same kind of thinking could be applied to x86.

B

um Yes, I definitely think there's some points that could be converged. um I'm not sure whether this would apply neatly to x because of its um it allows memory, references to just sort of be thrown onto nearly any instruction, and I believe, we're using that in some locations.

B

uh So you can sort of do memory, loads and stores within more complex sequences.

B

So if there may need to be different handling for x, uh but I know for sure that all of our risk architectures could definitely do the same thing as power, um also not sure about z.

B

But I would need to look into that.

B

C

Now isn't arm a bit more like x86 and it's rough memory, references they're not quite as powerful, but then that need to look like.

A

X86 yeah, the logic on 64 borrowed borrowed quite a bit from power. um So I think that even some of the issues that that then is found here with um with the barriers not being inserted properly, I think, would apply- may apply to art 64 as well.

A

So I took that as a as a to do to uh bring it up with uh the look myself or bring it up with some of the developers that have been looking at that code that were inspired heavily by the power implementation to see if the same issues um appear there because places where barriers are required for the memory model on ar-64 seem to be similar to what power requires, not as necessarily as strong but.

A

A lot of similarities.

C

Right just they have some pretty funky memory reference modes right that we may not be exploiting at the moment, but that that's right yeah. So if we were to exploit them, then it would end up looking more like the x86 than the than the current risk power style that it's dealing with.

C

It just seems unfortunate that there would have to be a dichotomy between the risk and the cyst in the in that design. I mean I understand from a pragmatic.

B

Level but from a conceptual level.

B

Yeah, I agree, it is definitely pretty unfortunate.

B

um There is one potential way of solving this where, instead of having the, um instead of having the load store handler uh api emit the load itself, you would provide it with a function that it would then call with its memory reference to perform the emission.

B

uh The problem with that is that it doing that nicely would rely on having c plus plus 11 lambdas available.

C

So for the memory reference generation in what you're proposing how it is um like well for one of the I have rip relative, I guess is the way I would describe it on x86 like so. How does pc relative addressing fit with this model? Does it fit naturally, or is there something weird about that.

B

um So that does fit naturally into it, so load store handler, has some internal apis that it would use to basically construct a memory reference based on a node. So as long as you can represent the rip relative memory reference as a tr memory, reference you're all good and you wouldn't need any special handling for that.

A

So so what is your next uh step here? So you've produced a sort of a proof of concept pr to accompany this issue.

B

Yes, I've got a pull request, uh work in progress uh that implements the api and switches all known locations in omr to use it where appropriate.

B

The main points of work that I still need to go through are getting open j9 to use this, particularly in the right barrier evaluators, which is where I I may run into some unfortunate issues, uh still working that one out.

C

Okay, the right barriers would seem to me to be the most difficult to prove out with the design right because of the arbitrary nature of some of the things that it has to do with side effects of doing the.

C

C

I'm especially thinking I get the use for field watch for.

C

A

um So, with this design will the notion of an unresolved memory reference the sunk into open j9, so right now they are.

B

Go ahead, they could theoretically be. The initial proposal doesn't actually do this, but it's definitely something that would be possible to do without breaking um the evaluators later.

C

A

Okay, a bit I'd want to spend some time with the the pr that you created. I haven't had a chance to go through that yet, but I think that will help connect some of the dots.

B

Here all right yeah, this is definitely uh a change in how we're thinking about uh node-based uh memory operations.

B

um So there's there's definitely some potential points where my design may be a little bit deficient, and I would like people to look through my proposed changes and look for anywhere that they they feel it might cause some.

A

A

Any more questions for ben about this uh proposal.

E

I just wanted to confirm this all applies only to memory references based on nodes, otherwise we'll be able to create memory. References with all these other old apis.

B

Yes, memory references that aren't constructed from either a load or store node will continue to work the same way. They always have.

A

Are there examples on power where those kinds of memory references are also reused.

A

You know I mean I think it's it's invalid, that you're saying that it's invalid for for node-based memory, references to be shared, but I think it's also invalid for non-node-based ones shared as well.

B

It's definitely invalid to have a point: two pointers to the same memory: reference on two different instructions. So I know for sure we don't do that. uh There is still the copying that we do on on non-node based memory references, but in the case of non-node-based memory references. That's generally fine, because you can assume that there's no data races going on there, usually we're usually with those we are loading data that we have internally sort of emitted within the compiler. So we know more about it than with an arbitrary, simple reference.

E

Oh, I think, then you know we cannot only use memory reference in any case because of register location.

E

uh Because those virtual registers will be replaced with real ones and then register now.

B

Yes, as I was saying, you cannot have two two pointers to the same memory: reference on two different instructions, but you can clone a memory reference to use it to use a memory reference and its clone separately.

E

A

A

Okay, so I guess, if there's, um if uh there's any further comments um for people want to study this a bit more, um there's that discussion in the issue and um some sample code in the in the poll request so we can um is, is the reason that it's a work in progress, because you wanted to get feedback on it or is it ready for review.

B

uh It's mostly because I wanted. I want to get feedback on the proposed interface early um and especially before I s, while I'm starting to go through the write and read barrier stuff in um in open j9, because that that may take me a week or two more. But I'd like to get some feedback early.

A

A

All right any more discussion on this topic.

E

uh Just a quick question like ben, so there is at the end as a result of this work, you're gonna, remove old interface so that it's all sort of clean and finished.

B

Sorry, I couldn't quite hear you. Can you repeat your question.

E

Oh sure, uh as a result of this work, you're gonna remove old interface like that node based memory, reference.

B

ah So um the original implementation of the uh load store handler will be using the old interface uh just so that we don't break open j9 for the time being. uh Yes, the plan is later that the node-based interface for tr memory reference will go away.

E

Yeah, I guess I just don't want both to exist for too long right.

B

Yeah, basically they're only going to coexist in my plan for as long as I need to to verify that everything continues to function.

E

Oh great great.

E

A

Okay, um all right thanks ben for uh for uh bringing that one forward. um Okay. So let's move on.

A

um Okay, so the the the next topic is about the omar project's usage of travis ci for those that may not have have been following goings-on at travis ci lately, so they recently changed their their pricing model um because of because of some issues that they were having with um with usage of their of their farm machines, um perhaps predominantly by those that were in the free tier, which of which omr is um so.

A

What they wanted to do was to try and restrict the amount of usage that um free projects were um were using and to try to drive them the ones that really needed the the throughput to drive them onto a paid plan.

A

um So that affects us, uh because one of the things that they're that they're going to be doing is that they're going to be imposing limits on the number of minutes that you can that each project can use in the free tier.

A

We would get about 10 000 credits, which translates to about a thousand minutes of of build time in practical terms, based on our current usage. This is about 90 builds or uh 90 check-ins, where a ci bill get launched on travis.

A

Expect that we would exhaust that in less than a month for sure just given the rate of check-ins that are happening on uh at eclipse omar. So clearly we would. If we were going to rely on a travis ci going forward. You know we would. We would certainly have to upgrade to a paid plan fairly early.

A

They do have some other credits that you can apply for, but it's not clear to me how how how generous those credits actually are and whether or not that they would be adequate for for this product for our project from an omr perspective, what we're really using? We only really have one build that happens at travis right now.

A

It's a it's a linter job that runs that basically verifies whether or not the use of extensible classes is correct from the omr perspective, and if it's not it'll it'll fail the build we could.

A

The rest of our builds are happening at um azure, so the work that that philip did um not long ago to to add the azure pipelines on mac, os and windows, and I think, there's a linux there as well, and then we have the rest of our jobs happening at a build farm at the university of new brunswick that we that we use the eclipse jenkins to to get jobs scheduled there.

A

So between those two, I think that there is probably enough capacity to handle moving the job off of uh travis ci into one of those environments and and and doing it there and then going forward deprecating uh our dependency on travis ci. um Even if we decided, we would still have to do some work. If we, even if we decided not to do anything and just continue on with the free plan, uh because the url is changing uh as well. The the travisci.org is going away.

A

We're gonna have to switch things to the dot com, probably not a big deal, but um it is something that we would have to do so.

A

um So I wanted to turn it over and see what um what people? If people have any thoughts about, uh you know, travis ci, whether that's something we you know, even though we're only using it for one build going forward, are there are sorry, even though we're using it for only one build now. Are there uh you know, do you see an expanded role there in the future or or what or do you think we should uh back away from it and move things on to the onto the unb farm.

F

Opinions I have a few questions. I guess I assume that 1000 credits is renewed monthly.

A

It's um from what I read: it's not clear, yeah.

F

That's when you use.

A

It when you use up your credits, what it says is: if you use up your credits, you're welcome to discuss with them um paid options, so it doesn't.

F

It doesn't specifically save, but on the on on the plans, page they're based on monthly, so I'm assuming it's renewed monthly. But it's again it's not clear yeah.

A

But but even then we're gonna blow through that pretty quick yeah um you know within within two weeks I would say, uh would would be ten thousand per project and not per org.

F

Or do we know that I don't know that.

D

That was my question as well, because historically, travis ui was by org.

F

Yeah and it's per oregon, where should be way more screwed?

F

um Can you can we set it to do manual launch rather than automatic, launch.

F

To limit how much it's running.

D

At that point, we might as well just disable it. I don't see anyone running that true.

F

And then another question I was: can we move it to jenkins, which you already said.

E

F

For the dot-com move, what exactly would we change for that.

A

And I think eclipse actually set those up for us too, so maybe actually have control over that.

F

A

I guess one uh potential complication for moving things to um onto the b farm are some of the dependencies that we have for running the linter job. So at the moment it is based on.

A

Maybe some other dependencies there as well. Not, I don't think there'll be a lot of problems installing things in the various nodes on the unb farm, but um you never know it's something. We'd have to look into.

A

There are more dependencies with that build. We incredibly replicate on azure pretty easily.

B

True yeah, I would really be in favor of keeping it with azure if possible, because as a linter job, I'd prefer that it run automatically and for every pr so that we don't get blindsided by something getting in. That happens, to break.

F

B

F

On jenkins automatically as well without a comment uh well, I was going to suggest this: could you put it into a docker container so that you can sort of control your own dependencies, and then you just run on any machine at jenkins that has docker on them.

F

But I mean if you want to azure, that's fine too. Whatever you want.

A

Philip, I guess you've had the most experience with azure. Do you think that's um the kinds of dependencies that are required to run that job on travis? Is it? Are they easy to get set up on.

D

It should be triggered. I think the bigger piece of work is getting the linter to work with cmake.

A

Okay is that um is that a requirement of the azer environment.

D

uh Not necessarily no third independent pieces work.

A

I mean it's a good thing to do anyway.

D

I think that's our last dependence on.

D

I think that's this last thing not running on cmake in terms of all our farm, jobs.

A

Yeah the cross compiled builds on 16 needs some work. There.

A

Too so I guess is: is there anybody in favor of.

A

Sticking with with uh travis die, or is there any possible use that we have for it going forward? It doesn't sound. It sounds like the the direction that we're thinking about is is to move away from it. Do the automatic builds on azure, perhaps even on the on the jenkins farm, initiated.

A

Manually, okay, so if we're going to do something we're going to have to do something by the end of the year, I believe.

A

Or we will quickly run out of our um we'll cook. We will quickly run out of our credits. I think.

F

D

I can volunteer too.

A

To basically track this piece of work to move us off of uh travis ci and uh discussion and other pieces of the pieces of work, for that can be, uh can you track there.

A

Okay, when you got things set up on azure philip um did you have to create an account there.

D

uh Yeah we created an organization.

A

You create okay, and are you the owner of that organization.

D

uh Yes, myself, um mark and charlie are all articles and anyone can be added if they ask.

A

Okay, no, I just want to just want to check that um it includes the project leads as well: okay, cool.

A

um Okay, so I'll get an issue created for this and any further discussion invite to have to happen there unless there's something that somebody wants to bring up.

A

A

Okay, um so we'll get that done. Are there any other discussion points that anybody wants to bring up before? We adjourn?

A

Okay, all right, uh that's the last topic for today, so um I guess we'll we'll end it now. So thanks everyone for participating and talk to you in a couple of weeks.