GeoSci.xyz SimPEG, 30 Jun 2021

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SimPEG Meeting June 30th

Description

Weekly SimPEG Meeting from June 30th, 2021

A

Yeah, nice to see everyone glad we all kind of made it on the other side of this heat wave up here in vancouver, so that was real.

A

I know I. I definitely had some trouble thinking here and there putting together some complex thoughts was a little rough.

A

So I kind of um just go right into my clicker report here I migrated some of the analytic functions from simpeg into geoanna and I added some more tests to make sure they were correct and like dieter.

B

And I were chatting.

A

About like signs and things making sure everything made sense there as well, but it looks good um also considering that we've been having a these amount of problems recently with the like mkl libraries or the pardiso solver in there like looking to it with that. By entail, um I wrote up a quick cycon wrapper for it.

A

With the goal of it being a little bit more rigorously built against the mko version and that it was a little bit more, it should always properly link up when you build when you have the content mode like it should always work properly.

A

So it's built against it in the same way that, like intel, has their own honda recipes for things like if you, if you haven't found, comment, there's this mkl service.

A

Package that exposes a lot of the underlying control functions for mkl things like setting the number of threads to use. um You can set threads used for individual processes like if you wanted to limit the number of threads that the fft routines used. You could do that specifically.

A

There's a bunch of other things like okay: you can get versions from that as well, but I was able to take keys from what they did there to figure out like how they were properly linking to them like their own mkl library, because I mean obviously they should know how to do it.

A

So I just kind of followed, along with those steps, and I sent out, sent it to a couple people to see if they could get it to build locally, which they, uh john and bomb were able to get it installed pretty easily on windows.

A

I know I've installed it on my mac or my windows machine as well. For my private I'm doing some github tests with all three operating systems, and I felt fine on there.

A

That's a good thing. Fortunately, though, if you're installing it from condo like it once we get it up there, um the user doesn't have to build it right. The links it's built, honda forge servers, build it for themselves and then.

B

It's already properly.

A

Linked up with how condo would expect it to be linked right, the same way that all those other things are so just a little bit more rigorous since how it's handling linking to the mkl and then, but that should make us not have to.

B

Does anyone know if there will eventually be a newer version of it in mkl? Is there a plan or they don't plan, to release anything open.

A

A newer version of pardeso, yes, I have no idea, I I doubt it. I'm not feeling like that. Pardeso project doesn't want to sell it to intel anymore.

B

A

But I also have not played with the pardesa project at all.

B

um No, yes, I thought, maybe maybe they do it like always if they have a stable release. That is, I don't know five years old, they might drop it out, but maybe not.

C

Joe, if like a because mkl is a big big library- and there are a lot of actually numerical kind of recipes that we could potentially like a kind of something like a matrix-vector product, they got actually kind of special functions.

C

Is that something that we can actually use in our end as well, or most of them are actually in in numpy, so they actually use the mkl? What's up? What's the situation.

A

So um with conda numpy is built usually with mko, as the blast like the.

A

The uh thing that I don't think that I think we could take advantage of with mko is they have a few sparse, matrix, vector routines that could potentially speed up like those operations like sparse matrix times, a vector compared to how they're implemented at sci-fi scifi they're, just they have their own simple implementations of them. Like I mean it's pretty fast, it's pretty, there's not too much.

A

You can do as far as optimizing a sparse matrix times a vector multiplication, but um I imagine the mkl libraries might be a little bit better optimized for the targeted.

A

C

For dense operation like what numpy is actually pretty much optimized and.

D

Numpy's, probably already using that right.

B

I I had a before I used number with empire mode. I did sometimes some benchmarking of it with running it with and without mkl and well. All my functions are using numpy and scipy, and there was quite a difference here, whether you use it or not, it always depends which functions you use, but so yeah numpy definitely makes use of it already. So I don't know how much you could gain.

B

Just what numpy often does is they do some checks, finite tests and stuff like this, which sometimes you could speed up, but yeah.

E

A

Yeah, like I at this point with this, I think we probably need to take a step away from that pi mkl library, because it's essentially dead and they're, not maintaining it or updating everything. So I made me to do this. I have, I think you finally have a discussion about where it should live, um whether it lives on its own or it lives as part of pimax solver.

A

I don't know um I I've been thinking about it a little bit and I guess for me, it kind of makes sense that it would be somewhat separate, considers just like meant to do this partiso interface um and then pi max solver could certainly rely on it. The same way that pi entail does.

C

A

C

If yeah, if the only propose, is actually handling purdue, so then yeah, I think it's probably fine in time it's solved. If that's the only case like if there, if there has any other general purposes, maybe it's worthwhile to make a separate thing.

A

Yeah, my only my only concern with that was adding like that compiled part of it to the finance solver.

A

That would that was my only like.

B

Googling python and party, so there seems since since pyromancer was built, I think there were quite there are quite a few projects that wrap part d. So I don't know if it could be possible to use another one.

A

Yeah I was looking at some of them. Most of them were just dealing with like the real matrices.

A

So that's why it's like. I just threw a fight on record together. It was pretty pretty robust.

A

I can send, I still need to like test it against things, to make sure it doesn't break anywhere too much too hard and that its performance is probably on par with what we had before.

A

Temporarily, I'm calling it pie diesel.

F

Got a good ring to.

A

It that's my quick report, like I said it wasn't.

E

Doing anything I.

A

Didn't have too much complex thought outside of just implementing a wrapper or something.

A

Anyone else have any updates on.

A

Theater, I think I would like to lead off.

B

Yes, I don't know who thought saw the I think tebow reacted to it and lindsay. It came up in a swung meeting recently about how long do meeting recordings live on the internet. How long should they live, and I thought it's maybe a good discussion to have with the government's policy uh yeah it was raised because of the justified concern that things can be taken out of place five years down the line or something. How long should everything be online?

B

Not everybody being the same easy going about it! Something like this, so I don't know what other people think lindsay shipped in and then we said, maybe we could have a after a while have them unlisted, so they're still on youtube, but you need to have the direct link and then after another, while completely remove them from the public view yeah. What do I think.

G

Well, in some ways the stuff that we talk about in our meetings, I guess there's like an intent to finish the projects we're talking about, let's say within six months, so that could be kind of an upper limit for that at a certain point, it doesn't necessarily seem like it's useful to have videos talking about things that have been like urgent master release documented. So uh at the very least, we could probably delete them after six months or not make them available freely after six months.

B

Who's going to watch old videos anyway, exactly I mean either you you follow up on a meeting within a week or two or you probably don't.

G

That's another way to look at it too. I don't think you're going to look up what we did three weeks ago and go yeah yeah.

B

What lane is.

E

Like I was just gonna say, yeah like if you like, unlisted or like put it private after a year, that's probably like, like a conservative position and but also considerate. So I have no problem with that and like it's true that also like, like this like right to be to be forgot and laws on internet varies from one country to another, but yeah. I think there is no.

E

I don't feel a need to archive everything forever, indeed, or at least not having it public forever.

B

I think what lindsay said in the chat there for maybe having it a bit longer. She, I think she linked in the governance to the meetings where we decided it.

B

So then there is a link, but that would also because it's the direct link, then it would still work if it's unlisted, so you cannot google it anymore, but you can still find it so yeah.

E

Yeah actually, maybe.

B

E

Yeah- and maybe there are some meetings that I was keeping longer like some meetings we do like it was actually someone doing a presentation, so that can probably stay up longer or like. If, like the meeting, we had about government and stu, where we really we're really putting the groundwork for the government and it's like, probably one that's worth keeping longer but somewhere. It's just like uh uh like a week two week, uh this discussion or like um like more casual yeah after six months or a year. It's probably fine, if it's at least unlisted.

A

I think that's perfectly fine. I guess I like the idea of having them just kind of like we don't need to keep records of every conversation we've had you know two years ago.

H

All my bonehead comments, yeah.

H

Hey, I don't know just a thought like this. I don't know if it would be possible to just um to just uh convert them to text. If we could query after this, uh you know ideas or topics that were raised. It would be easier than if it was uh on videos right. It was just. uh I don't know if it's possible to uh use youtube to.

E

Yeah yeah, it's it's possible, um I have I've actually used it like um is that uh youtube can generate uh automatics uh subscriptions.

A

E

Yeah and then and they're, not bad they're like the generated ones, are not bad. So just an example. So I did a uh like uh my training of on my of my phd defense with a simple group.

E

So that's one presentation that I was thinking, for example, that I would like to keep that one up as long as possible, but one thing that I did was that after that training and to like to prepare them like the real one, I was actually able to download the automatic captions and I actually use that automatic caption to see like okay. Here is what I said and what I forgot, etc, but uh yeah. So you can. You can download the automatic caption on on any videos and that's so like the text.

E

The textures search should be actually fairly easy to implement.

B

But would it it wouldn't have the names right like in recording meetings, you wouldn't see who said what? No? No, you will.

E

Just have like the audio yeah like the yeah, it will not say like this person is speaking because yeah youtube has no idea about that.

H

Yeah that would have might be. My only comment is that you know hey we're, actually not even capturing it now all right and we're kind of maybe something someone said something intelligent and a good idea, but we're just losing it, because, whatever it's piled under more videos, if we there's always a way to be able to capture that, but I don't know, I don't have time to work on this either.

A

There are notes that you.

B

Don't just volunteer, so that's fine, yeah.

H

If it wasn't remembered, maybe it was not worth remembering: maybe it's okay or if it's important, it's gonna, come up later again. Yeah someone else is gonna, rethink the same idea and it's gonna come back yeah.

A

Peter did you open up a like? It should probably be something on the governance thing or the community like yeah,.

B

Yes, I can do that like an issue in the governance or where do we have that when.

A

We're like well, we propose doing it like within a year or something at videos, they're, just publicly they're still available but unlisted, and then they go away after so long unless, like you decide otherwise for special presentations or yes,.

B

Yeah, we will do that.

B

Then what else did I have? I worked on the joint state with sogi and props to him, so it works fine now in empty 3d, at least for electric receivers. So I'm looking forward to what soki will do with it in the near future.

B

But it's it's for me. It was an amazing insight with the whole joint to see, even though the gradient was pretty good before, if you compared it to finite difference, if you don't do the exact joint, it's still different in some ways, it's kind of fascinating, uh because yeah, the the, even though we got the receiver response more accurate because we couldn't do the exact joint of this. The gradient was worse. So we have to get the receiver response less accurate, with a linear interpolation to become the better gradient which is kind of counter-intuitive buddy.

B

That was an interesting ride.

B

uh Yeah! That's it for me. I think. Do I have anything else.

A

I I think lindsey has posted a few things. I'm just going to say it here as well. um She's added any issues on the community repo on about the publication listing another about listing institutional partners and.

A

Another minor one on the formats of the file, whether it's markdown or rst institutional, I believe, if will probably have to deal with like how what do we consider an institutional partner? What benefits drawbacks things like that? What did they get out of it? What did we get out of it? What we're expecting.

A

So please take a look and read over this.

A

Let's see um here.

G

Yeah, so I guess I got a few things.

G

Obviously I've been doing some work on the doc strings, but uh whether or not it's in this meeting or if some of you have some time just afterwards, uh I would like to maybe briefly show you what I've done and get some comments and some feedback, and I've done one example doc string of of how I want to document differential operators on different meshes and I'd really just like to nail down what we think is the most important information or maybe like uh I don't know what kind of visual representation we like on this.

G

So it's just. uh I don't think it needs to take a really long time, but I think just getting a few people's perspectives on it and maybe deciding collectively kind of like a direction would be really useful uh for me.

G

So I mean I've gone through documenting the utilities and all of the the basic properties and methods for all of these. These meshes and given examples and stuff, but now the differential operators and that kind of stuff are uh yeah. We need to come up with a bit of a plan. Otherwise, it's just gonna, maybe not be uh done very efficiently, so yeah does anyone have time just after this meeting, if it's relatively short, maybe from like, I don't know up until noon or something.

A

Yeah- let's do that, so we can go back to them.

G

Yeah, that would be great. We could just kind of stay on the channel and I could screen share um the other thing that I was working on, uh because I had a little bit of time. Was this uh this em issue with rings right? We all know the you know.

G

You got the sensitivity and if you've got a very compact body, it's going to recover a ring shape and I was just trying to generate a really small scale, simple example uh in in simpeg that I could use to play around with, and I realized that we don't have a we're, not applying a preconditioner when we solve the gauss-newton system.

G

I don't think they're we're applying a preconditioner to that. It's basically, my example is crazy, crazy, slow I've only got about 30 000 cells and 49 um stations and a few frequencies, and each time I try and solve uh that problem. It's uh it's like two hours so yeah I I looked in. We have. uh We have a jacobi preconditioner for linear problems, that's very easy to construct, but we don't have one for the frequency domain or or time domain, and I'm wondering if this is something.

G

That's being put in when the mt stuff is being developed.

H

Yeah, if you're on the tiled simulations branch, uh because now we're forming j, we have jacobi. So you, uh if you switch that branch you'll be able to have access to uh jtj., that's what you need right. Gtj.

G

Yeah so I mean is that automatically been put into how you would solve the problem? Is that something I have to call as a directive like update preconditioner like how do I implement that.

H

Yeah well, there's an example. uh Look at the look at the uh I think, there's an fpm example in there: okay, so.

G

Tiled simulation branch.

H

Yeah and it needs to be right now- we haven't touched time domain. I mean, if you're really keen and want to start working on uh updating, doing the exact same implementation like we did for frequency to time. That would be definitely uh useful.

G

Okay, um let's see I'm running some stuff, that's going to take a few days on my current branch, but uh I will go back to that so uh and that example is that just in the examples gallery, or did you put it in as a tutorial.

F

It's in the examples, isn't it under examples right under simpek.

G

Okay, so it's in the examples gallery it's in the okay.

F

B

I don't know what that's yeah.

F

That's called gallery.

G

All right that would be fun to play around with uh yeah and then, apart from the doc strings, I mean one one small scale project I could kind of do pretty quickly is working on the the ios that connect sim peg to ubc. I don't know if anyone would find that particularly interesting, but we don't exactly have that and I know we can.

G

I know we can write a lot of things out to some compact file formats, but it is nice to be able to write out a file that maybe isn't as compact but actually lets. You see the organization of the data and and see the numbers.

G

So uh does that kind of make sense to you guys.

G

I'm not sure, maybe with like, I think you can write out to json or something like that. So you can write out to these compact easy to use file formats, but if you double click and open it you you don't really see. Oh here's, my receiver, here's the location.

G

It might be nice just to to have that, even though it's not necessarily sophisticated so yeah. I like the idea of having the link between the ubc and the simpeg codes. I find that pretty handy, so I might take a stab at creating something for that.

H

Yeah, I would say uh you know, be careful because uh you might start creating a new file format that no one's gonna want to use. um Just look around to see. What's there right.

G

Because uh well, I'm not! I'm not gonna invent my own file format. It's it's going to be directly ubc to simpeg right, and one of the benefits of this is, uh with these code validations that we're doing um being able to pretty easily go and and construct a mesh in a survey uh in using one of the coding packages.

G

Then just outputting it loading it into simpeg and re-running it. It's gonna make life a lot easier.

H

Yeah I hear you but again, uh like the number of ubc users, ubc em users are gonna use, simpex is nil right, it's just gonna, be you uh very few. People are gonna. Do that conversion because those file formats are, you know. Hopefully everybody starts using simpeg in the in the future and then the ubc codes are just duplicated and those file formats are such a pain in the ass to deal with. I don't know if it's really worth your time to really create ios for all those files.

H

I I don't know, that's just my opinion, because we're doing the same thing with that with uh gif tools. Right I mean.

I

Yeah so much time, customizing gif tools to read and write those files and no one using them so frustrating yeah it is. It is a good.

G

Point um I'm definitely not. I don't want to invent some kind of new file format for that um yeah. It's something to think about, because I do agree with you that the people are either using gif tools in geoanalyst and those people aren't really trying to then start learning simpeg they want to. They want to either click buttons or they're, more coding minded and just using simp peg anyway.

H

Maybe just use gif tools to convert it to a matrix form, you know and then, after those uh take it up, pick it up by python or something I don't know.

G

Okay, it's, it would still be nice to have some kind of I o that uh you'd actually be, maybe there'd be like a. I don't know, verbose, io or something like that where you could write out and see it. I still think people will want to open a file and and see what's in it, even if it's not the most compact.

B

And efficient, but I mean if you have hdf5, it's even compact efficient and you can still open it and see it. Okay,.

G

I'm not super familiar with that, but you could just.

B

Have to download to the viewer, but it's for free, so okay and it's for instance, people will know it because, like the newest, not even the newest, all the recent version of matte files from matlab are also hdf5 under the hood, and I think many things are hdf5 files.

G

I think okay, yes, I've seen those get kind of read in and read out for various things I worked on, but I haven't played with them specifically, but that that might be a pretty good solution to this. Then.

B

With dome you have to master in the house, so he can explain you everything about hdfi files.

H

Yeah, it's fully customizable, but the problem is that you know em. Surveys are complicated. You need to store so much metadata and then connections between things like just writing the format to yeah anyway. I.

B

Just meant, if you wanna, if just looking at it, is the thing, then nothing speaks against htf5.

H

No yeah, yeah or json for that matter, right.

B

But then it becomes very quick huge with the um true I mean very huge, fair yeah.

H

I think it's compressed yeah gff is compressed.

G

Yeah I mean it was pretty useful doing for the the potential fields and for dcip replica, because those are codes that a lot of people use, but people aren't really using the the em ones.

H

Yeah and there's not that many things you need to store right, it's basically xyz with uh with data. Next to it, that's pretty pretty darn easy, but yeah time domain, you need to store transmitter receiver configurations, the loops, the waveform.

H

Orientation of the loops like it adds up.

F

The file structure in the hdf5s, too I've been trying to convert our databases to hdf5 and they.

B

F

Up just because the way we because it's like there's a reading, then there's a dipole and then there's like what angles all those certain dipoles and just the way that you do the path it just. It goes from like 500 megabytes to like two gigs, it's crazy.

F

But again it's just all that metadata. You have to know exactly what everything is doing when you're at the acquisition. So but it's worth it I'd say because oh man, they're fast access and super nice.

H

Are you compressing.

F

I'm compressing what I can it's just like. We don't really have that big of like when you think of it. It's more of the path, that's more important. Otherwise, the big data you're actually recording on a dipole is just the either the time series uh that can be compressed or just a decay. So it's yeah yeah, but the.

D

F

Is hard yeah so.

G

Hypothetically then, so somebody comes to you with uh with a data file, some kind of let's say, text file that has all the information or all the locations of things in uh in a survey. And then you, you have the report, so you can figure out the loop size and orientation um we should have a ideally. We should have a straightforward way to be able to load that into simpeg so that you can start building stuff.

B

But there will be a million formats.

D

C

Yeah and in a practical sense that I would it seems, like it's not very worthwhile to figure out a general format of em data, because it it's actually not that useful, because at the end nobody will follow that and then it'll through a very different types of format, depending upon whatever survey. Even if you do the same survey but depending upon the company who is running the survey, they may have completely different.

B

Format so even within the same company.

C

Yeah, even the same company, so my guess is like uh I don't know if something may be useful having in an and syntax side about something like specific, maybe so because you may do a specific test of I don't know if you're doing an airborne, em then having something like that might be at least useful and it may be used because if that's not too specific, it's probably not used. I guess I think that's what's happening, maybe mac and dc, it's quite specific and it's actually a single format. It's pretty simple.

C

So, for instance, if you do an airborne em now I think it's actually quite simple, for instance, or let's say say smt or whatever specific survey, then that may be useful, and once you got that you can potentially generalize at one point, but I think starting from something specific and just making it work uh and that at least like it could be used. um So just just a thought.

G

Yeah, but I asked what I'm what I'm hearing is just the the file formats being used for em are so widely varied. There isn't going to be like we can't create a nice simple function. That's like a general I o for that, and so in the end, I guess the onus is on the user to script up, something that will load their particular format and if that's the case like that's just that's just life.

A

Yeah my opinion.

G

A

Is the only thing that I would add is that if you want to do something as far as writing out, it are information to files, it would be probably better served if we worked on serialization for them to make sure they survived.

G

Yeah we already have that. I mean if you've got two sim peg users and you created something in simpeg, and you want to share it with another simpeg user, just use serialize to output it and give it to them. That's way better.

A

And my point is, though, is I'm not sure if they are all properly implemented right now I get that handles it properly, so it could be a thing to check um like serialize and deserialize on every object like that to make sure it happens, that's something I don't think like. It tentatively supports it, but I don't think it works properly right now, okay,.

G

Yeah, I thought that would have been part of the tests.

A

Unfortunately, not like I'm pretty sure everything here, like everything will serialize, I don't think they all deserialize properly, potentially.

H

So they should be all unit tested. I guess all the serializers.

G

Okay, you guys written anything for like the edi format for uh for mt.

A

Because that's.

G

A

Reader and writers, I think um that had created for it um I think they're hidden somewhere in the empty folder yeah.

F

There's something there. I made a few modifications because it was using a really heavy. uh Like a lat, long converter library, um it was using like gdal when I just lightened it up using utm but yeah. It seems to be quite functioning. I believe I've been using it when I have edi files.

G

Yeah, I mean, I guess, even if it's not uh yeah, even if we don't have plans to implement stuff, like that, I guess the community would tell us if there was something that says hey where a lot of us are doing this is there a way to yada yada.

F

Our uh mt guy diaz um he's been involved with the community quite well and they're. Changing they're they're, going from edi files to hdf or hd5. uh Okay they've already started to sketch it out, but I don't think there's anything official yet but uh yeah. If you want some more information, I can try and get something from him sent to you.

G

Yeah, I was just uh something just trying to find something small to work on um if the the docstring stuff stalls a bit.

J

Pick an issue we have a thousand.

J

Any other seven.

B

I would, uh I think, the most value you would get if sim peg would very detail describe how our stuff is getting serialized and stored. So everybody who would get something into it would know how to write a file that can be read by simpac. That is the much better path than trying to make loaders for particular files unless there is a real request, but if, if you can, if you can really describe how we serialize our stuff, then anybody could write a file that can be read by simpac and that's much more worthwhile.

B

G

Yeah, actually, that's a really good point either.

B

Like uh the discretized tensor mesh, for instance, if we, I don't think it's a dogs, but it would just need the h. It would need the the origin. I think like three things, so everybody could write a file that then could be read by simpeg right if it's documented.

B

G

A good example.

B

I guess other things are more difficult, but still I think it would be the most valuable thing yeah. It's a great idea.

G

um Yeah, but that's basically it for me.

A

Okay, um john, so you got a few things there and then soggy after.

F

That yeah, I don't have too much productive stuff to report on just uh started, throwing some bigger problems at the in parallelization and for the dc problem and yeah. My mpi is blowing up. uh It can only handle two gigs uh or yeah. I don't know. I haven't quite figured out the air exactly, but as soon as you throw a big problem at it, it can't distribute it. It can't scatter it.

F

So I went back to the desk a little bit and was just playing around with it locally, because we've been running the mt locally and that I can distinctively see that they're running in parallel, but for some reason sometimes the dc it just sometimes it it'll start up and it'll run them parallel, but then sometimes I'm finding that they're staggered a bit and it's almost like they're, sequential, but I've kind of combated that a little bit I found a way to assign the affinity using das pretty much the same way as I did with multi-processing it's just like a operating system or an os uh library or module um and yeah.

F

When I first looked at it, yeah desk isn't actually setting affinities. So it's not telling which task to go to what core it just kind of tells it hey, just access it and whatever one's free, take it. But if I find I can improve it. If I can tell uh what task where to go, that seems to be improving uh for locally running anyways.

F

H

That the nanny, sorry john, isn't that the nanny that you're starting looking into or is that different.

F

I haven't gone in. That was what I was gonna do today. uh The nanny kind of is under the hood of a worker anyways, uh but yeah. I think you can do a little bit more work with the nanny, um but yesterday yeah, I was gonna. Try the nanny today, but I was trying scatter, because desk has a scatter option as well, but uh yeah. I quickly learned that that is no good it. uh The overhead is enormous. It's crazy.

H

So it depends what you scatter, though, right, because you want to scatter things that are permanent, but.

F

Yeah, that's true um yeah if I was just scattering like the objective function or like just the misfits and uh yeah. No, that's.

B

F

B

Yeah that one was.

F

Bad that that was just yeah if you got like solid numbers and things to go, and it's all right, but yeah, it's not like mpi mpi, has a good time scattering those at small problems, but as soon as you get to like a six million cell problem, it's yeah toast. Even if you.

H

Just want to scatter vectors right, you don't want to scatter objects. I think yeah.

F

So yeah I'm looking at me, I don't know, I don't know if scatter, if I try and restructure everything just to scatter vectors, if it's worth it but yeah we'll see I'm having pretty good luck with assigning the affinities within the das tasks.

F

That seems to be doing a good trick, but it's essentially doing what, if you had a cluster, uh it's just telling on the computer which ones to go to, whereas if we have, if you have das on the cluster, you just you're pretty much telling it that computer is yours and then you are having everything in parallel. So it's just kind of the local implementation. That's a bit of an issue.

H

F

uh But yeah I've been sticking my head in that all week, so that those five days and applied frequently or some days went by quick.

H

That's going to be important for everybody, because uh scheduling tasks, we're not yeah, as you said, we're kind of throwing everything everything at it right now, but uh we have to be more careful like you do this you and then don't start anything else until you've done this part right yeah. Otherwise we have too many instances of parties. So in that that starts kicking around.

F

H

F

Then you it's two different kind of modules too, under os for windows and for linux you can do it for both of them, so you just have to have a check to see. Oh, what am I what my operating system is and then it would implement the right thing. So there would have to be there's two different ones, but you can still make it general. You can add it in.

C

So john, are you like uh what is the connection with mpi and desk? Are you using separately or.

F

Yeah, I was, I was doing two separately. I was building like dom and I were working with desk and then I was playing around with mpi and mpi. I was finding was giving me better performance uh on a cluster. uh However, it just yeah as soon as you throw a big problem at it. It actually fails, which is kind of surprising.

F

uh I guess most of the mpi I've ever really seen used is on 2d mt, inversion and stuff, but I I don't know it just seems it seems strange and I've looked around and there's like a an addition to mpi or uh yeah mpi.

F

That's called big mpi, and it says that it's supposed to do big ones. It just kind of it kind of does what das does it just uh chunks everything, however, I tried it and it just gave me the same air output. That mpi was giving me that it's too big, so I don't know if I had like it says it was using big mpi but yeah. I don't know it just seems like it's a dead end right now, so I'm gonna yeah.

A

So john I'd be curious to see how you're implementing this with mpi. Oh.

F

Yeah um actually I'll shoot you my uh my repository, I have it on my forked repo.

A

Because I I like I'm curious, where you're breaking things up and what's getting sent around because like for me what I the way, I would imagine doing it with mpi right like I wouldn't.

A

I would just break things up based off of like sources and receivers. I wouldn't send around any j matrices I wouldn't send around any like. The only thing I would send around was maybe like I'd, send a vector when it has to get multiplied or something right.

F

Yeah, I think that's where it's going a little bit wrong, because I'm I'm scattering the the simulations so that the simulations go on to each uh cluster node and then that all its information has to do with the derivatives to do the deep threads.

A

F

So that might be, why, like I have maybe it is, I have to break everything down to scattering vectors. um You can't just yeah scatter a whole object. I guess.

A

Yeah that that's that's rough like it's, so uh let me see I'll, take a look at it and see what I can point out and what points. What jumps out to me.

F

Okay, yeah I'll I'll point you at what lines it's just in uh get deep, reds under the uh inversion problem, dot, pi.

H

Yeah not moving the mesh around would probably help a lot too. If we can just uh keep the vectors.

C

Yeah, I don't have any technical update, but uh we submitted the hu session for uh in near surface geophysics about open, open source. I think what was the title: uh uh yeah open source software for near surface geophysics and its application, so yeah.

B

C

Guys submit some abstract, so I can see you all there uh and if you have a good, so I need to have a two invited speakers to somebody who, in a high profile, I don't know doing some cool things. So if you guys have any suggestions, please let me know that would be very helpful.

H

I'm sure craig would love to uh to be there. He loves agu.

A

Okay, anything else. If not, we can like to see some of the have. Some devin show some of that uh doctoring, stuff and documentation that he's been building.

H

I'm gonna go back to work, see you guys have a good day.

H

A

If you guys need to head out like there's nothing, stopping you for sure, yeah yeah see you guys.

G

Just need to share screen powers whenever you can.

G

Yeah, so um this is kind of like the this is the landing page for the api reference. um Most of it was pretty structured based on what joe had there before, but we have our you know our mesh classes uh and then there will be sort of a kind of like a base, so base meshes uh a place where we have the discrete operators and then mix-ins. I guess this was a set of classes for like interacting with vtk or kind of interacting with external packages.

G

So if we want to take something, that's uh simple, formatted and then reorder it for for something else. That was a place to do it, and then we have all those utilities, but uh to I guess the stuff that I'm most interested in getting feedback on is stuff for uh operators.

G

So here we go so I I decided to do a little bit of a prototype one, um just the face: divergence on a tensor mesh, so um yeah we have kind of our one, our one sentence, descriptor that would uh that would show up in the big list of properties for that, for that particular mesh or that particular class, and then the returns. Everything is linked, uh and it would give you the the dimensions of this.

B

G

Little bit of an explanation uh and then something kind of brief about the operator. So in this case it's a second order: numerical uh divergence operator, it maps from the faces to cell centered, and once it's constructed it is stored permanently as a property of the mesh.

G

So I don't know if we want to elaborate too much on this description, but we do have a space under notes where we can provide some mathematical background if we want to so um that's some questions I have is how how much of this math do we want to put in the the main description, uh or do we even want to bother that stuff is sort of uh questions I have and then below? We can put in some kind of example, and for these operators it's nice to have some visual representation and so below.

G

I just uh there's there's a script that will basically just show the mapping, so it's going to map from the green the faces, and it gives a number of of the index of this this value in the uh in the original vector and then tells you where it's mapping in red at cell centers and then gives you something about the the structure of that uh the resulting matrix.

G

So this this one, I think, is a little is pretty straightforward um and I wanted to use this as a template for for other ones. But before I start going nuts, I wanted to get some feedback.

A

Well, I I mean it's definitely looking pretty good. I like how it I like that. I think the details at the beginning of it are great. um The.

A

I think the q is a little abstract defining it like. This is maybe a little bit abstract. um So it's like, I guess I would the way I would approach it, is to write it on something and show, but.

A

I I would look okay, this face off this, this space divergence matrix like we've, talked about before, like for each cell. It takes the sum of the values on the boundaries around the faces, started with their normals right and then adds them together, divided by area or something like that or multiplied by the area. Whatever it is,.

G

Yeah yeah, I mean in the end the first half of it like this is kind of yeah, the important part. That's just saying it. Just it's a second order. Divergence operator, it's going to yep yeah.

A

So, okay, I would like you know, I don't think we need it explicitly like. Okay, it's exactly diag v and verse. Q diags, um just I think, it'd be enough to just represent it in terms of like okay, it's a sum like for each cell. I some faces started with their thing like this is what it is doing.

G

A

G

Yeah, I thought about doing that and I didn't want to make it. um I didn't want to make it really long and then a lot of those details I decided to put in the tutorials, so that was this. This is kind of like this is the information that you need to be able to implement this and know what you're doing, but the details about like which faces ijk.

G

um We did kind of cover that in the tutorials, so I'm I'm less inclined to do that on on this.

A

I don't necessarily mean like okay, it doesn't have to be ijk for anything, just kind of a general thing like oh some of the faces like some over faces, or so I right some of her faces, yeah yeah, okay, at least I I don't know that kind of at least like representing them as their like most fundamental thing of what it's actually doing.

A

Possibly I don't know because even like looking at okay, like diagon v and verse q times diags um like I know what it's doing, I'm sure like some of us here know what it's doing, but it's for someone else looking at it, they'd be like what. How is this doing the divergence.

G

Yeah yeah- and I I wanted to include something like that where there was kind of that fundamental equation, where it's like x, face minus x, face over the the cell dimension plus y face minus y face over the y dimension. Plus I wanted to put that in, but in a way that wasn't going to lead to more ex more math or more explanation than is necessary.

A

Right, that's why I was like just it's. I would say: do it as like the.

A

Like I think that is pretty clear and that's about what it's doing here like that's, basically what it's doing here right. So I think that is just it's. This operator.

G

Can you remind me of that, I'm not sure.

A

G

Not talking, I was trying to write it.

A

G

A

What I'm talking about too, like you know,.

G

A

A song over faces of value dotted with the normal or like v, dot, n or something and then divide by the areas. Something like that. It's their I'm thinking about it. But it's something like that.

G

Oh because that's in in a lot of ways, isn't that what what that's doing yeah this basically represents is one over the volume times the the integral of the the outward normals time. Yeah yeah.

A

Yeah yeah: it's that's what I'm saying I just kind of like okay, it's just right in terms of sums and things and like go for each cell and then, if you wanted to, you could like put it like that, where q represents that you know minus one that represents the dot product there.

A

Okay, just I think there needs to be one more step, just kind of intuitively. So, like oh.

B

Okay, I see what.

A

It's doing now for someone who's coming at this, who doesn't quite have as much like. Okay, this is what they're doing for their.

A

This is how they buy.

B

Yes, I always struggle with simpek documentation because of the d's and c's and m's. And if you don't work in this space, it is very cryptic- and I guess probably more people are- are globally used to sums and integrals and and those things. So I I second chose point.

C

Who would you think was like who's the target audience for this? Like uh said, um I'm just curious like who are we targeting to read this? Is that somebody who's going to rewrite this code or like just going to use this and then just want to have a rough idea.

A

Yeah, I think the goal should just be like people should be able to use like look at the documentation and at least understand what it was implemented like not necessarily the intricacies of it just be like okay. Well, I see it's doing that operation. Okay, right right,.

C

And if, if that's the case, then I think that's starting with the continuous form and uh here's the disc reform and emphasize is this? How, besides, is that, I think, that's to me: that's actually the most important part. Okay, I've been putting.

B

C

Of number of cells and I'll put a number of faces, or something like that.

G

Yeah I mean that I think that part's pretty yeah useful, I mean basically, we have a. uh We have like a we're approximating like a continuous operator, we're saying we want a way to evaluate the divergence of a vector function and then we'll just say: if you the way: we've discretized it.

G

If you define the components of you on the faces, then you can get the divergence itself centers, because I don't I don't know we need to do much beyond that, because that's all you need to implement it, and you know you need to know that it's sparse. You need to be nice to know that it's second order.

G

um These are all important things for implementation, but all of that that stuff about like what's happening under the hood or like those, I don't know graduate level math details that kind of got covered in a tutorial where we we bring people in and say, here's a way that we can approximate the divergence operator and we're going to take this face minus this phase, like that, that's kind of been done in detail in a tutorial as like a an extended learning, so yeah. The other part, is some.

G

Some of these things like, I think you have a uh like a cell gradient operator and then that becomes actually a little bit more complicated because now you're interacting with different cells and implementing boundary conditions and stuff.

G

So that would be kind of.

G

Yeah, I guess I'm I'm trying. I think, I'm focusing more on implementation.

G

So I guess yeah the implementation, the mapping.

G

And like dimensions.

G

And what what function? It's it's! It's representing! What function is it discrete? What operation is it discretizing all right, so.

A

G

G

Want me to stop sharing.

A

A

I'll just drop an image in here somewhere.

A

um So what I'm thinking as far as like just showing it for this.

A

I'm gonna stop here for a second here, just something like I don't know that ish.

A

Is that kind of? Because that's exactly what it's doing right for each cell summing over all of the faces.

D

Well, it's one overvolt, but yeah yeah yeah.

A

Like to me that looks a little bit more intuitive than the you know, diag like one over diagon v times, q time- I guess like it I mean this is literally exactly what the code is doing right.

D

Like it's doing,.

A

Yeah and that's is that's like the simplest way I can kind of think to think about it and like okay. Well, that's pretty clear to me at least like okay. I see that it's doing that operation.

G

Yeah I mean it, I guess to me when I look at it, it doesn't look like a divergence. I know I I know it is, but for me I'm like where's the derivatives right.

C

So I think, that's probably worthwhile to why kind of having that integral form depth might be useful. So, like you can start with the continuous form, and this is basically same as the divergence and here's the discretized version, and you can clearly see that corresponding match in the code. So that's probably all you need to say.

G

Yeah, I mean I'll put that in if, if we think it's useful- um so maybe me so maybe under the notes we just start. Okay, here's here's, a continuous function that we want to discretize we're just like this is mpeg has a way to turn it into this form, this discrete form and for each cell.

G

This is the operation, that's actually being done and it is equivalent to divergence.

G

And then I mean once once: we've actually finalized the tutorials one of the the plans I had was to go back in and then actually link to those tutorials, so that, if you really do want a full explanation of what's happening in pictures and things, then it'll just take you to a tutorial where we really go into easing them into. What a discrete operator is.

A

Yeah, so even like just something sitting like that, like just to keep people that oh, it's the integral form of it like that, probably be sufficient as well like you don't even.

B

A

Put you might not have to put in like oh the whole, you know ddx, like the sum of all that, like I don't know, yeah okay, yeah yeah, just to make it clear that.

D

G

D

Is the representation.

A

That it is like, and then if you wanted to, you, could even link to like you know, wikipedia's divergence, page or something.

C

Yeah, so yes, it looks like that, like here's continuous form, what we're doing basically discretizing this and you said the dimensions and mapping, I think that's probably enough, I guess, as a user, I think.

G

And then, hopefully, they're not too. The integral forms are not too difficult to figure out for some of the ones that interact with adjacent cells, because, like right now, the the curl.

A

G

uh The face divergence, the nodal gradient, those are all pretty straightforward.

A

So gradient is the one that's kind of a little bit weird, but like we don't really we don't. We never use the cell gradient itself in the finite volume operators like what's for equations, for what, whenever we're forming like the pdes with the finite volume operators, we never actually use the cell gradient operator, that's defined.

G

Yeah that makes complete sense like you're you're, solving it on one cell and then.

A

Yeah yeah, whenever, when we do that cell gradient thing we're actually, whenever we do like a cell-centric thing like that, we use the node, we use the face divergence, operators.

G

Well, don't you yeah when you're, when you're saying like cell gradient, I mean, aren't you really just calling? uh What was it you're calling like the face? Divergence transpose, plus your boundary condition, something sort of like I that's that's one part I think I'm going to need to talk to you about is how you cleaned up the implementation of being able to do general boundary conditions on on everything, how that's being implemented.

G

C

Yeah, maybe there probably need some bifurcation about finest volume and find a difference.

B

I think cell gradient is.

C

Basically, the finite difference operator um yeah. So potentially it's an extra case, exceptional exceptional case, so you can potentially mention it.

A

Yeah and that one, I think, is probably probably sufficient just to say that, oh it's, you know forward cell minus the backward cell, divided by the difference between cell centers. That's what that is great.

G

Yeah, because when I was looking at um well, I was taking a brief look at it. It seemed like the face divergence, I mean, there's, no, you don't need to implement any boundary conditions. You just call the face divergence and the only input argument is self and then, when you had, let's say edge divergence, then you would be calling self and supplying a boundary condition whether or not it was neumann or something like that, and uh so there you're basically saying okay, I need to implement.

G

I need to provide it with some boundary conditions, because now I'm going to be interacting with locations that are outside my mesh. The point is that was just to to I guess: yeah. What would it be use nodal, gradient, transpose, plus.

A

A piece right: the idea is that you're, using the nodal gradient to approximate that edge sure, like you're, using a different transformation to get it to be in terms of the nodal gradient and then the boundary integral, as well like only in that weak form. Does that do those equations work like.

G

That yeah, oh, but it's and it's a little bit more interesting than that because of like uh do you factor out the mass matrix or do you keep it in.

A

That depends on like the pde.

C

Yeah yeah, I think that dev, if you get on that differential operator level except the cell gradient, you don't have to worry about any boundary condition. I think.

A

C

So you just can describe what it is and that boundary condition is handled in in an upper level when you're formulating or when you're actually and formulating your system. So I think, uh on that level, the only place that you need to mention about the boundary condition is the cell gradient, which is an exceptional case.

G

Yeah, um I guess again, I guess what I need to do is that I'm going. I have. I have this property or this functionality that's not hidden and it needs to be documented, for if it's visible to the user, I need to be able to say what it is and yeah that's kind of that's that's it. So some of those details actually kind of have to be put in there. I can't just kind of like gloss over them, because I need to document what is this?

G

So if I have a uh a cell gradient, is that actually going to be returning a cell gradient, or is it going to be returning a mass matrix times a cell gradient.

A

So that one I'm pretty is just the cell gradient.

G

Like all those.

A

G

Are what they say they are okay, so that's that's super to know that and you're saying that the cell gradient was actually computed with a finite difference, not a finite volume.

A

Pretty sure it wasn't and if not it was implemented in like okay, it's actually like they did it. It was done like by just doing like the face divergence transpose times the volume or something in which case it's based. It's the same, it's equivalent to saying that you know the finite difference from forward cells.

G

Yeah, well I mean the mass matrix is invertible so, like I think it's diagonal, so you'd be able to to basically like create it using the face divergence transpose plus your piece with the boundary conditions and then factor out the mass matrix to actually give you back a true cell gradient.

A

Kind of um my point is that if you do that, like v, it's like divergence transpose times uh volume right like that operator ends up being like the same divergence transpose times volume. um Then you have to divide by. Let's take the inverted math matrix like which is just.

G

Now you have to you, have to apply the inverse of the mass matrix to the boundary conditions piece right.

D

Yeah yeah, so you.

G

Basically, you'd have it as like: div transpose plus, and then you factor out the mass matrix times. The boundary condition piece that you had and that would give you control.

A

G

Divergence or something.

A

Negative divergence transpose most of the time. It's just like that.

A

Yeah I mean the important.

B

D

Is that at the.

G

End of the day, it's the actual cell gradient with boundary conditions of your choosing implemented, and I I can leave it at that. Yeah and and that's okay, because before things weren't necessarily factored out and it was kind of confusing. But now you made it make sense.

A

And if I like, just be like, oh the default cell gradient, I think, will probably just leave as like. Oh.

B

Yeah it's this.

A

And then it has a zero dirichlet assumption, there's no zero neumann assumption at the end. I think the default yeah assumption.

G

Yeah yeah and those details are important to put in their their implementation details yeah, okay!

G

Well, it sounds like um well, it sounds like you guys are overall, pretty. Let's see I'll, do this overall you're pretty happy with the level of detail being provided, but we'll add we'll add something first. That actually says this is the op of the operator, the continuous operator we want to discretize and replace this stuff with an integral form of how it's evaluated for a single cell.

C

Sounds good to me.

A

Right like for the nodal gradient right, it would just be like for every edge just take the difference of the you know the nodes at the boundary and then in the direction of the edge you have to edge direction times.

A

I don't know the absolute value of the difference.

A

Yeah. I can divide the forward minus the backwards.

B

A

B

Think it's it's great to have the code, but I wouldn't add the theory, but I would go to extreme because you have like three three classes of readers: the first one they never read the documentation anyway. They just ask the second one is the newbie, which is probably already overhelmed and just wants to have an example that he can copy paste. So this is really good to have the example and yeah.

G

B

Class is the the the one that goes straight to the code and looks there so yeah. I think it's good to have some theory and then maybe a link to a paper or a book or wikipedia and that's probably easier to maintain as well.

G

Yeah I mean I can always put something here when the tutorials are done. That says like for a full tutorial on differential operators like see here, because there's there's already something: that's really comprehensive everything yeah. Well, it's yeah, it's another! Well, it's uh it's in discretize, but this needs to get finished before the tutorial section, because it's going to link to the functions and stuff but there's a whole thing where there's like pictures of cubes and stuff, and it gives you the indices of faces and writes that if.

B

If, if you have this, don't do you want to rather link there.

B

Well, the thing is, you could read this in in in your terminal with your with the question mark. If you want to see then, but then the math is not easy to read, because it's plain text and if you read it on a browser anyway- and it will be just one- click away- yeah, that's another! That's another possibility as well, because math looks nice on the browser right, but then it's not good as a help string in in in any in many other ids.

A

That's why the notes are placed at the end of the help string.

G

Yeah yeah, so I guess you're saying if uh well.

B

I think it's good, I think the size is really good. I wouldn't elaborate yeah too much. I think.

G

I did just trying.

B

To keep notes, as short as possible,.

G

And I think we can do that. I think I think we can keep notes no longer than what it is right now.

B

Yeah, I think I did some much longer in my code at some point- and I probably wouldn't do this today and rather put that maybe in the docs but yeah. It's always.

G

B

In the docs, you can link from different places to there right and then in in yeah.

A

We haven't even gotten used to the layout here.

G

Yeah I've gotten used to the layout. I I even tried to rebuild everything from scratch to get the the tree mesh stuff to those docks to update and they didn't work that didn't didn't happen.

G

So I even I I closed it all closed everything and then just said: uh what was it the build, build extensions in place and then I so I think I deleted the pre-existing extensions rebuilt. The extensions.

A

G

Redid the setup.

A

I think the easiest way to like do an in-place installation of a package right now is just like pip install dot like dash e dot.

A

That's like the easiest way to do it in place installation to do what.

A

And in place install yeah, if you do it like.

B

A

You don't have to mess around on path and everything like it. Just works baby you're, saying doing it with pip.

B

I don't want to do with pip. No, you don't download from pip pip, it's just a command that installs it. You do. Pip dash minus e, now pip install dash minus a dash e dot, so it will install what it finds in this directory instead of from pi pi.

A

And the dash e means that installs it in like editable.

B

Mode or like yeah.

A

You make changes.

B

It updates yeah, it basically links to your folder. It puts into the paper library a link to your folder, so it uses that.

A

Yeah, so that's it's nice because you don't have to mess around with your python path and everything yeah.

G

So pip install minus e and then set up dot, pi or just pick install dash e dot dot. Is there a space between the e and the dot? Yes, the dot means in here.

B

In this current directory, yeah install this thing here, this thing exactly grab whatever you'll find and install it. For me, okay.

B

Yeah and if, if yeah any project that has a setup properly made and it works okay,.

G

B

G

G

Yeah, so that's uh that's basically it. I have a lot of doc strings to do, but we're making.

A

G

It looks really good, I think it's looking good did you do any for any more um comments, since the first round that you did last week, joe? uh No, I don't think so. I need I can go through it again and look through it. Oh, no, no, no worries um yeah. I went through those and and made those changes. So the things are the the formatting and some of the stuff is that's been corrected.

A

Sounds good yeah, it's definitely going to.

D

Be nice like, I think this is.

C

D

Better format and layout.

A

Overall than it was before, they're.

G

Just unpredictable.

A

Before they even look for anything like.

G

Yeah, I didn't even know what I was looking for. I was just seeing like information. It was nuts, um but this is actually quite helpful, so uh yeah thanks for the feedback that this uh really helps.

B

So when is that change going to land, because now the docs is still read the dogs or not? I don't even know. I still read.

J

G

J

Discretized and.

A

G

Yeah I mean hopefully as soon as possible.

G

I don't I don't have an actual. It really just depends on how quick the review process is on.

A

It right I'm gonna, get some stuff in earlier, which is like I'm glad we're able to get that and see it. Okay, let's, let's keep moving forward on that looks good, though.

G

Yeah yeah and when that's done, I'm it'd be nice to do it for, uh for simpek yeah.

D

Yeah, it would be nice just just yeah, it needs to be.

A

The point is, we have we'll, have discretize good and then we'll have a good reference point to go from.

G

B

It's also easy right: discretize is a more self-contained, smaller thing right.

A

Compared to simple as well like I don't know, I I thought he's probably feeling the same way but like we worked with enough to where it's like. Well, I understand I understand how everything's linked together. I understand like what happens when you, you know when you call something of the field when you index the field, I see what's happening in the background it's like. I can see all these things happening, but they're just not well documented, and it's like.

G

There's still still stuff uh in simpeg that I don't understand, I think some of it is the fields object, just how it's I just I haven't played around with that implementation of storing stuff yeah.

C

Yeah well, there's a lot of pieces and you use whatever you used to.

D

And assume everything's.

G

C

That's fine, I think that's actually that's the purpose. I think you don't have to understand everything but uh have a confidence, but that's actually working.

B

Unless it until it doesn't.

B

G

You have to start understanding.

B

G

I feel like if we prioritize it, we could be close in a couple of weeks.

G

Okay, I'm hoping so fill in all the doc strings and then do a final review and then maybe decide on some formatting related stuff.

D

Because I want to play with.

G

D

And things and fonts.

C

That sounds very fun.

A

Okay, my original past. That format was to try to make it look like the synthetic.xyz.

A

As far as like fonts and colors, and things.

G

um Yeah, if there's nothing else, then I'll get to it.

G

All right cool thanks again.

J

Have a good one, everyone.

A

Take care I'll.