GeoSci.xyz SimPEG, 3 Nov 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SimPEG Meeting November 3rd

Description

SimPEG weekly meeting from November 3rd, 2021

A

Welcome again, everyone, as always, the notes are posted on the slack channel, there's a pin at the top there. I do so and you click records they're, also getting to the point where you might just be randomly calling on people seeing where people are actually at that kind of time, or at least a number of people right now is kind of manageable.

A

As far as agenda and announcements, I know you made a mention about the newsletter going out that you're, assuming you're always or wanting information about what people are up to with senpai down there.

B

Yes, absolutely I dropped the link in the ch or in slack, but I can pop it in here too, um and the talk this month will be craig miller um and he so the date we have. Let me just look, um but I believe it's the 19th it's a friday uh is what worked best with him um and it's at 3 p.m. I believe just double checking.

B

Yes, it'll be the 19th at 3 pm.

A

The announcements are.

C

We're just getting.

A

A quick report to kind of go through it we'll get to anything, so I have been some of you guys might have gotten pinged a little bit, but one of my students submitted a pull request to the gpg to fix two typos. I thought it was really neat that they did that.

A

It was really um to see that I was like oh okay, yeah. That is, I think we got that link fixed up. um What was the computation that geosci xyz? What's that doing,.

B

We started that a while ago the vision was to have sort of like a resource kind of on finite volume on inversion, um and we put a few examples and things like that in there, but we never really like. We haven't maintained it, um and so I think, there's good stuff in there. We shouldn't delete it, um but I also don't think that it's in a state that we want to be referencing pointing too many people to it.

A

I've also, john now that you gave it seems like you're, getting really close to the floor request. I've gone back that my head back into the mt with the bounded side of it, so that everything working again, because I think that's still part of.

C

The pr in there.

A

So sorry, which one so the bound the like one, the new one, oh the one, the boundary, um the 2d bounded problem- I've been revisiting that trying to get it push it through. Now that you're pretty close, so that should hope with some code coverage on that right on.

D

Thanks joe.

A

Just hooking up finishing hooking up the receivers and testing derivatives. That's it.

A

I made the one the receivers and one be a little bit more flexible for those.

A

At least for those pde ones, so the 1d receivers should be able to support both orientations, instead of just one of them before.

A

Just double checking by my estimate from the sim peg with the 1d earth of the xy impedance should have a uh what was it like: a 100 like a negative 130 or something negative one to be a negative value. That's the 45 degrees right, because the same peg gets y x, I believe, was 45 degrees.

A

X y was like negative, so 40, whatever 180 minus 45, was. Was that what you remember as well for a 1d earth.

D

As far as I.

A

Remember I think it's a little bit backwards, but it's still in line with what's impact's definition of coordinate systems are.

D

uh There was one other thing that uh I we do have tests for the row and the earth. Sorry the current resistivity in the phase for the xx and the yy, but, like you, you showed before. If you just change the model a little bit, it blows up the uh the yy and the xx, because.

C

D

Divide by zero- um I guess is this an offset or is there something that might be a little bit too dangerous? I guess.

A

E

To kind of stabilize those derivatives.

A

D

Or just have a more yeah, I don't know.

A

I think the stabilization could probably come in with like a noise.

C

A

F

Again, generally, uh I apply uh a floor, uncertainty on those on the diagonal components and then more of a percent driven on the off diagonals, and that seems to be fine. Oh.

D

That does okay, perfect cool.

F

Yeah, I I think, that's well that's sort of what I've advocated for as a as a best practice: yeah, okay, okay,.

A

um And then I've also been applying the cross-gradient regularization that I was working on to a few more models.

A

It's still pretty still seems robust to those checkerboarding issues that could have happened before so I haven't really been encountering that in any of the examples I've been running, which is good even with like a really high regularization on the cross. Gradient like it gets enforced really hard to see that checkerboarding issue, so I'm getting a little more time, I'm getting more confident in it. However, we do and we'll still need to spend more time, updating some or automating some more directives.

A

The way that those parameters that enforce the cross gradient are controlled right now, it's there's no directive to kind of update it or adaptively increase it or decrease it as needed, which makes a little bit harder to optimize. So that's also what I'm working on thinking about some of those directions. So you can automate that that's for you.

A

That's what I've been up to john how's it going on here.

D

uh Not too bad uh yeah, I the last week there pushed as much as I could there on the mtpr and everything, but that one test and then the 1d stuff there, but it looks like you've, got that all wrapped up and uh yeah. I decided to look back at the dcip processing code because yeah, it's just time for an update and uh yeah I've kind of generalized it enough that I can probably push something to the simpeg side again. Hopefully have some examples: someone can have some fun with it. Just a pure stacking.

D

Just give it a time series and it'll. You can do your windowing put whatever stacking kernel. You want into it yeah just some stuff that someone um yeah uh none yeah. I want to give a talk on the matter and the paralyzation of the inversion. There finally got some hard numbers and some good graphs and yeah for doing large scale, inversions getting some good results but or getting results back, but I just need to run some inversions now with uh with the larger scale data um yeah.

D

I'm finding that pretty much when you get the large-scale data you could you you have to mesh, you have to decouple the mesh. uh You can't just send a global mesh to everything and give it a few fewer sources and expect to get a good speed up like with 10 million cell mesh. If you give it 340 our sources or 20 sources, you get maybe a 10 speed up it's just too large. uh So now I'm I finally got some hard numbers, so I can get my boss to.

D

Let me go to the next decoupling now and uh yeah. I just wrote some stuff for the tensor tensor decoupling um yeah, it seems to the octree, is definitely going to be the better way you're going to save more, but the tensor. Does you actually do save quite a bit? Even if you decouple a tensor, it's it's significant enough, but yeah, hopefully neither december or january I'll fill in one of those talks there and we can show my results.

D

And that's about it.

A

Okay, no one else is on and put on anything.

A

You can go around to ask people.

B

I've got a question sure: um do we currently have any capabilities in simpeg for estimating volumes based on a cut off? I know some of that work was done with the camp project, um but is yeah. Do we do we have that in synpeg, or is that elsewhere? At this point.

A

I don't know of any instantly it just like. I want to estimate the volume of properties between this number and this number.

B

Yeah, yeah and potentially within a given sort of bounding box, so that if you have stuff in the padding or something like that, it's not not included.

G

It depends if you want to make sure they're connected or not right. If you want to identify separate bodies, because otherwise it's easier, you just do a threshold and a value and get the cell volumes.

G

But yeah, if you want to identify isolated objects, then that's a little bit more of a search, search problem right.

H

Don't you think you could do it in pi vista.

G

um I don't know if I says it capability. What do you know of their functions? Peter, you don't know there.

H

You can you can plot relatively easily bodies, depending on a value right, so you can have a threshold.

G

H

By defining two thresholds, I guess you could potentially see the volume in between I'm not sure.

G

Yeah, I know that's easy to index right to just do like anything like larger than the value who give me all the indices and.

C

G

Volumes from that, but I think what lindsey wants to do is to say you know this body and that one, what's the value of this guy, what's wrong with that guy.

H

B

How was that done for the camp project? Do you know.

G

Yeah well initially we did like uh we, the uh like you put a seed and then you look at neighbors right and it's kind of like a propagation propagation function, but uh it's very expensive and um and eventually, as you said like if your body like, is close to the edge of your domain, then you start like bleeding into like larger volumes, which you don't, then you need to threshold it. So, at the end of the day, it was just more efficient to do a bounding box and a thresholding, basically uh yeah.

G

What's your problem lindsay what you are trying to do? Well,.

B

Thinking, basically about exactly that problem, yeah.

G

B

Is uh estimating volumes for carbon mineralization.

D

B

From an l2 or lplq inversion, um yeah.

G

Put up like give me around this point: what's the volume something yeah.

A

A

Tivo, do you have any quick reports now pictures.

E

So we know some people being late. I just ran into into mark and was good to catch up with the volcanic and planetary science groups. Sorry, for that.

B

E

But you know not much for you not much quick reports. I've mostly been writing rather than coding these days.

B

Dom is that code kicking around that you used, for that. Is that something I mean, because we could think about like trying to clean that up, or at least have a first cut at it um in simpeg.

G

Yeah definitely can make it better yeah. I have a notebook somewhere uh yeah. It definitely needs a needs, love. It was pretty brute force. um It kind of reminds me joe, uh you know. Ideally, you would have a function like on the mesh on in discretized to be able to access all neighbors of a of a cell right then, maybe that that lookup function would get a lot faster. If we be able to do that.

A

Yeah, I know we have something like obviously the treatment she can look at: the neighbors of the cell birds and then a tensor mesh. We know the neighbors of the cell by definition.

G

Yeah yeah yeah.

G

Yeah yeah, it's the old issue of uh not just the not these neighbors, but also the the cross neighbors right, um yeah anyway,.

B

And tebow you, you did some of that for the geological segmentation, uh abstract right.

E

Like yeah, but like I like, because I was looking for like a certain number of neighbors and so on, like I just I'm like. Basically, there is a kd3 behind that I'm building with sci-fi, so I'm not actually using the property of the match. I'm building my own kd3 for each theoretical unit.

B

G

Yeah and the problem with kdh3 is, is just a distance measure right, so if you're in an archery mesh, if you have an actual change, you might not see the you might not see the uh the big neighbor next next door and then you mess up.

G

That's why using the discretize package is better because then it's really a connectivity between between two cells. Anyway, we can work on it I'll, send you what I have, but then some thanks.

B

A

I know I had written something somewhere to uh estimate and not necessarily estimate volumes, but just look at uh sense for sensitivity. Analysis on a tree, nice looking downwards from a from the top to find a sensitivity, cut off.

G

So you go down on that.

A

Yeah that one's a lot that one's a little bit easier because you're just looking in one direction, the entire time.

G

Actually lindsay now, I think about it. Maybe the easiest would be to use the uh the marching cube. The socket learn has a you know: you create iso surfaces, there's a marching cube like our algorithm, which is pretty fast, so maybe that could actually be a better a better trick.

G

I'm using it to create ice surfaces like you know, surface size surfaces, so maybe there's a way to leverage that that package.

B

That's a good point we'll check that out. Yeah.

B

A

Okay, so anything else.

G

I've been working on uh ip a lot doing a 3d iphone versions. Try to well manage the the uh the parallelization on that part. It's fairly easy right, because it's uh it's linear, um so the speed up is really nice. uh When we, you know, store sensitivities and uh and and block it um now, I'm looking into, uh I realize that a chunk of time is being used at evaluating the uh the data you know, creating all the prediction matrices for all the receivers.

G

I know that you did mini surveys joe to be able to reduce on the transmitter side right on the source sources sides, but we still do two for loops, which is you know, just creating a survey initially takes longer than creating most of the rest of the mechanics. So.

A

G

Have the feeling that we should probably have a survey evaluate rather than evaluating all the receiver sources? You know just create a giant production projection matrix, basically for all the all the receivers right.

G

I don't know why we're doing it like one at a time where we could just create you know one one bit projectors from field to to all the data. I don't know what you guys think about that.

A

It's another another level of optimization. It's just storing um like just storing a projection matrix for each electrode.

A

And then sending it out from there.

G

Yeah, but when we do the query on the fields object right, we could just do it on a one one operation rather than uh doing a for loop over all sources. All receivers, that's like super heavy in python, so I don't know just a thought.

G

Would be the the cost of.

D

G

All that, well, it's a sparse matrix right. I mean it's a sparse matrix that just goes from nodes or cell centers to uh to a point. So technically it's like you know uh four integers per uh poor data points, four or five entities per data points or something it's not uh that's not. I mean we're storing it anyway, right because we're.

C

E

G

B

How many receiver of like how many receiver objects are you creating? Is it more than one.

G

Oh yeah, I mean receivers uh like on a dc ip survey right. You could have like thousands of receivers so then you're doing a for loop over thousands. You know.

B

But they should like it's sort of one receiver object with all the locations or are you giving each each receiver only has a single location. This is something that is slightly confusing in our terminology.

B

Yeah, like basically as long as you're, either working with either a pole or a dipole receiver uh type it it can take multiple locations.

A

G

Oh you're, right yeah, it has multiplications yeah. So why huh yeah that's a good point. Maybe.

A

Because I have.

G

Many sources yeah yeah.

A

Yeah like if you only have like one receiver for one source like every one like you know, everything is paired up with that. Then it kind of breaks it down.

G

Yeah, fair enough, you know- maybe I just have many many sources. Yeah. If I have like a big, a big dipole, dipole survey, you're kind of uh going over several sources right still, you could still have like 100 sources, type thing but yeah I see which point yeah it's true. All the locations are stored under one receiver, which is confusing terminology, but uh yeah yeah, okay! Well, I don't know I'll have to dig deeper later. It's not origin, it works, inversion works. So that's all good.

G

And then oh yeah joe, so we have this pull request right. uh That is in front of tebow's uh tables pr. So I addressed your concern, maybe do another round and then uh we can merge it down. It.

A

Looked like it was all passing, but my only concern was one of the tests was taking a lot longer than it is normally so I was going to look at what was causing that I think it's just a slight difference in a convergence or something in a conversion. Example.

G

On 10 37 you're, looking at the run time.

G

Okay, 60 minutes is that the one you're talking okay.

A

G

I don't know yeah. I can't help you on that. One.

A

I just needed to look.

G

A

That that was the only thing that I was kind of that.

H

I was looking at.

G

Yeah because after this it's tebow's uh 10 10 53 that we need to to go in right.

A

G

Then we can carry on the rest, the rest of the work after.

A

Like all the tests fast, everything is good. It's just that that one.

A

That one took an hour when it normally takes place about 20.

G

Minutes did I even touch at that part of the code? It's just a it's just a pipeline problem.

A

Like it's running the tutorials, I think a couple tutorials.

E

So it might be that, because now I'm just thinking like now that you normalize the weight by the volume like if any of the example is using the same, was using the simple uh realization they should now use. That.

E

A

That it should be about the same now, but they should be the same, because the only difference is I'll have to look into a little bit more. But that's it was. That could be something yeah.

G

E

Like definitely with your change, you're done like like the pgi example, we need to switch from simple pgi to pgi. That's that's a certainty already, at least in the example in tutorial. We can just like replace all the simple pgi documents by pgi, and that would be that will fix that, like I did it on my branch, but we can. Maybe we can do it. We can do that specific portion and.

E

G

I look at all the tutorials and I I don't think that's the referencing sample anywhere. It's either tikkanav or sparse. So I I don't see. Why would that change anything but.

A

So, if they're, if they were referencing taken off before and now it's um and they were using an updated sensitivity, weighting right, some of those tutorials might have gotten double volume.

A

Like they might have been double volume, estimated before the volume terms before now, they're, not so just causing slight different convergence, I just need to go back and look at it check. Okay, but yeah. I just double checked uh it's about 30 minutes! It's like that. One test takes about 30 minutes now and, like you said it's about 50 minutes there, since it I'm pretty sure it's just a convergence thing. Yeah.

A

What are you been up to the last two weeks, four weeks or, however long it's been not.

H

Much four weeks, I think, a long time I was at I was a physical conference which was a kind of a new thing in the last few years, the eig in amsterdam that was kind of interesting. It was quite small and there was a dedicated small group that followed all the non-seismic stuff you uh after the first day, you knew each other and then yeah you stuck together, it's quite quite a small group but yeah. It was nice to to meet people in real life.

C

H

H

Well, everything is joined in some degrees right, you don't see, hardly anything, only em, let's say or only empty.

H

Slowly, getting dominated by machine learning stuff as well. So I guess the usual.

H

Yeah there was the one by ramco by daniele, colombo and robetta. They I think they published in geophysics this year. They do this joint inversion where joint means deterministic gradient-based, inversion with machine learning inversion, which I find very interesting so using the advantages of both to get to a hopefully global minimum.

H

That was kind of interesting to me, because I never thought of it or heard of it, so that was kind of new.

E

You have a complication around like uh I saw they were. They were at scg as well, but uh yeah like the the detail of the implementation. They need something.

C

uh Let me let me.

H

I'll put it in the chat, the the the article, I think it's geophysics this year I'll have to look, but I would assume that it's not straightforward, because you kind of well as usual. I guess the waiting will be the difficult part. How do you combine these two? I yeah I haven't looked in detail.

H

But yeah, how did how is this one week, one week going the new scheme? What is it a month now, a month and a half.

A

uh It's been going for the last month right right today. It's I we don't have too many people joining from new zealand, australia, time zones.

H

Like that well you're, neither from europe it's, I guess in either time zones it's only one or zero percent joining at the moment. Right.

A

Yeah so whatever seems like it's working out, okay for the vancouver group.

A

Just a matter of getting back into reminding like oh yeah, it's today is it four or three o'clock meeting versus 10.

H

It's good, I think it's it's nice! It's just this danger that yeah. If you miss one, then you miss a month, but then, on the other hand a month pass as quick. As you know,.

A

Any any other things you'd like to chat about.

F

um Maybe one quick thing so uh as I go through uh the api stuff, uh I realize that there's the the properties package is still very much alive in simpeg, so I mean we don't want to put too many things into one uh one project, but they do kind of go hand in hand. So I think I'd like to confer with you at some point on how we can exercise uh the properties uh and uh that I can continue on so yeah I'd like to do that before I start messing around with things too much.

F

Otherwise, I'm not really moving through in kind of a methodical fashion.

A

Yeah it does like we need to put that forward. If this is going, I even still looking at it. I don't think properties. Is that, like it's not that hard to get rid of properties, that is synthetic, it just takes some work.

F

Yeah, I don't think it's uh it's too heavy, but I mean you you already went through and did it for dispro ties, so I mean even just kind of if there's any little detail that I'm gonna miss going through it, the first time or sort of uh yeah just to be consistent with how it was done. Discretized is kind of what I want to accomplish.

F

G

Just need to do it for an one class right and then we can just do a small pr just for one one of them. The easiest one pick one any, and then we can all comment on like how we're moving from stuff from properties to the setter.

G

And then after this copycat, and we just go one class at a time. And then you know.

F

Change them yeah that actually makes sense. Just do it on a separate branch and then okay, so really that's kind of the first step, then is exercising properties before we would even yeah yeah. Do it.

G

A

G

A

That's why I'd recommend it uh like, starting with the api stuff, starting with all the functions and those things like documenting those, because those are not dependent on that right. Those just kind of build by themselves nicely.

F

Yeah yeah I mean I was, I guess I was starting the documentation from the base simpeg classes, because so much work is being done on individual simulations.

F

uh There was also projects being done with regularization, so I'm trying to I guess I was starting with the stuff I felt was not actively being changed by the community sort of where I started.

G

Well, those functions are technically not being touched too much right. All the utils yeah.

F

Oh, the utils are already documented okay and there's no properties class in there right, they're, just there's their functions so they're.

A

F

Class yeah: that's why it said that those are like the easiest place to start yeah.

A

Yeah, so that that's already.

F

Done and like and built locally so now I'm moving on to simped classes but uh yeah. I like that idea. So I'll uh I'll I'll start, a branch for exercising properties, start somewhere, small and then and get some feedback so that that would be good or.

G

Even create a branch that is like exercise properties in this class right so so then we are a module. Maybe we can do it per module, but.

A

G

That way, we can only like you know refactor. This part merge it in everything passes and then switch to the next one. Don't don't try to do it all at once, because it's going to be a mess.

F

Yeah yeah and then I'll just when I'm ready for each small piece I'll just be uh I'll just tug at people's shirts.

G

Yeah I've worked on the rag, so don't pick don't grab the don't do the regularization I'm already on it. Just do something else.

A

My other question is so it discretizes, we kind of. I tried to keep the serialization side of it still alive. um How often are we serializing sim type classes like how often are people doing that.

B

It would be we had that idea in mind, but we never actually got it materialized. So at this point I would say just don't.

A

Worry about it.

B

Don't worry about it and we'll fix it later.

A

I mean because most of the I mean the synthetic classes, I've never tried pickling them, but I imagine it would work.

E

A

I know you can pickle meshes with difficult things.

E

Like the negotiation features also is very easy and people that actually use that about.

A

Like create a gm.

A

Like it's not readable, but.

H

Yeah, but I think often you will run into issues if you then update the version of simpek and try to load it something from half a year ago.

C

H

A

So it's I imagine it could probably be one of the things where we keep it on the back burner like it's. It's still not.

A

I think we could probably be safe with just the living properties and not worrying about trying to implement the serialization side of it right now, because that adds a whole another a whole other level of things that I care about.

A

G

H

I miss that you completely removed it from discretize a hundred percent general properties yeah. That was a while.

A

H

Cool yeah, I knew you removed it. I just I thought you removed it in many things, but I wasn't aware that it's completely gone. Okay, cool.

E

Do you have an example of something that's not being able to something right now.

A

I have not, I mean.

A

So things that are not picklable are like.

A

Nested definitions of functions, um lambda functions are not picklable.

A

Things that basically functions that only exist at runtime, and I don't think that really occurs too often in simple.

B

One um quick idea that I'll throw out there we can see what folks think I wonder if actually starting devin, if you started in geoana, if that would be a simpler place, we've got properties in there, but it's not super heavy um and I suspect- and I don't think we're doing anything in terms of like observing objects or things like that.

B

So removing it there, I think, would be probably much more straightforward than simpek and would give you a better feel for well would give you a feel for it without um yet before sort of tackling the complexity of senpai.

F

Well, I do already have discretize as kind of my example for how that was done, um and I would say that I my I guess my ultimate goal is to finish the api for simpegs. So I don't really want to be diverted from that goal.

F

So uh yeah I mean if, if everyone thinks that's probably the best idea, I could go through it, but uh yeah. I feel like the quickest way to attain. My goals is to start with something simple and stim peg and just get feedback from there directly.

B

Okay, I mean, I think either either is fine. um Then if you, if you want to start in simpeg, then I would say start like start at the top. Don't go. Don't start at the base um start somewhere um like dcip or em um start start at the top and work down uh starting at the bottom is always harder because you flip something and everything else kind of falls. On top, so yeah.

G

It will ha all have to be done at some point anyway, so any any work done is.

A

I know when I did it to discretize. I did actually start with the base class and move outwards, but then I did it all at once. So.

G

B

G

That yeah, yeah and.

A

G

B

Yeah, sorry, yeah, no, no fair enough! I mean I think, for for some pig I would highly highly recommend that you start at the top. um I think if you start at the base, this is going to be a much harder project, so.

F

Yes, start start out with something that doesn't have any dependencies, or that is not a parent class of a bunch of stuff that will that's that's a pretty good idea, so yep, okay,.

A

A

Anything else anyone feel like turn chatting through.

H

How is the teacher teacher hidra stuff going through? I know.

E

It's getting there.

H

Working on a few other things at the moment, when can you show it off for christmas christmas event hit your.

G

H

A

That would be cool, but it has given me thoughts on how we're designing some of the internal functions, because the way some of our pde is constructed. They involve in inverting the mass matrices if some of our formulations in synthetic involve inversions of mass matrices, and that is not an easy thing to do with the tetrahedral meshes, because those mass matrices are all often like irregular, sparse matrices, they're, not just diagonal things, so the tetrahedrals only work with certain simulations, so they'll work with the nodal dc formulation.

A

They'll work with, um I think the electric field formulation. I have to double check, but there's only a few ones that will actually work and actually and also a few different types of receivers too, because some of the receivers are and fields are implemented with inversions of mass matrices.

A

So there's been something that's been on my mind.

C

G

I want to have a christmas tree and then do a dc experiment and you put power at the bottom of the tree and then all the light bulbs light up, and then it feels that's what I want for christmas june coat it out. Man.

A

If you wanted to do you, do you know the solutions for a uh like potential field solutions for touch for eagles?

A

Do you know where to find them.

G

Like it's a question.

A

G

No, I would have to look for no.

A

Well, if you, if you get bored- and you want to start coding up like that solution, that would be nice too, so it works for the integral. uh You know the integral equations in uh potential fields.

G

Yeah, if you feel like it, no, but uh I know for sure that the one with the with where it's like you have some faces, but just the tops that are curved, that one that one was I've, seen it but not sure for a pure tetra tetrahedra.

G

A

Know I know that I know there's out. I know it's out there and I know like there's some stability conditions too. That need to be satisfied, but I think if they're all just like nice meshes, it usually works like it's like. If you have like really long skinny tetrahedrals or something it's like kind of breaks down.

B

I think gimli might have a solution.

A

G

A

Well then, that's it! That's it we'll see y'all at three o'clock next week, 3 p.m. Next week,.

G

Perfect see you guys.

H