SunPy Developer Discussion, 9 May 2016

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: SunPy Developer Discussion (25/4)

Description

https://meet.jit.si/sunpy

A

Okay, I don't want my husband.

B

Watch how Stewart I don't do so well with this chainsaw in my neck.

B

B

I'll fix that, for you.

B

Take your mind off it completely from the shoulders.

C

Alex, are you the Alex who is emailing me about light curve.

D

Yeah, that would be it then, but.

C

That's here, yeah.

D

C

D

What's your name Dan all right, so I didn't see your name so high. Then thanks for cuz.

C

D

C

No I read your email this morning and thought that might as well just wait for this meeting because it might be easier to discuss it here for a.

D

C

D

Definitely you.

E

Can start there at all yeah.

C

Well, have you been in on this Stuart a little yeah and will, if we're gonna, get Alex to to sort of start actually working on this, then we should have a little bit of agreement about how we go forward to save him working and then coming back later and other people sing. Hang honestly. This is not how I wanted this to happen, a talk so which I know it does happen. So.

C

So did Alex, have you have you told Stuart pretty much everything you've said to me I believe so.

D

C

D

I mean obviously a lot of it is from excuse original document. So Stewart went describe the problem to me specifically and then yeah I sort.

B

Of relied on to you and then obviously yes with respects to the.

D

Specific issue with using the time object in tables which I pretty sure, I told you about didn't I. It.

C

Did but I hadn't heard about the well hadn't tried to use a time object in tables like that before. So what? What is this problem? It's, like you, can't do anything yeah.

D

Basically, so uh so, if I'm, correct and I've asked him the dev forum as provide a forum case, I'm wrong or there's a workaround, but essentially if you are using objects within a column, so in this case the time object within the column, it will turn it into a mixing column, and if that object is immutable and the time object as files on before aware is immutable. So you can't change value once you created a time object, it's a time limit and that's it.

D

If you want to change the time, you would have to destroy that object and create a new one and because of that essentially Owen.

D

So the the time objects that stores the column is a time object, but with multiple values, but it is still immutable, and so essentially, if you want to do anything like rearrange or if you want to change any time value which obviously would be required for rearranging that I do sorting, Lee the columns and rows or equally, if you want to do anything like add a row or delete a row that you cannot do it directly. You can't just add a row by inserting into the object you have to destroy the entire time.

D

Object, that's table and start again, which is not very efficient. Obviously I mean I'll, admit on.

E

Your boat yeah, you wouldn't have to destroy the whole table, though right just that cog but yeah. It's so terrible. Well,.

D

As far as I'm aware, it seems because you can't so you can't change that table, because the column is kind of linked to the table. So right.

B

D

To add in a row, you would have to create a new table as far as I'm aware again, you know my understanding. I've used tables a little bit. My understandings reasonably limited, especially I've never used them. I've never used a cue table and the mixing column before, but as far as I could tell you do, have to recreate the whole table, because you've got a different number of rows in the new table. Potentially so.

C

That's a big deal for resale, but since you can't be done this, this seems like it's.

E

Wrong somehow, oh it's obviously.

D

Not like I office in these facing um yeah well, maybe I mean that that's way, obviously, it could be fixed potentially with the time object. I, don't know again, maybe I, maybe I'm just misinterpreting it, but it didn't it didn't. Let me do very much as soon as you added in the time into a column. It really restricted you with respects to things like resampling and similar.

D

It does mean that and as I suppose, the problem is it depends on how big your data set is and therefore how much of an issue you know really resampling would be. If you essentially double the amount, a memory usage you need so you've got the before and after that might not actually be ridiculous. Still, but I don't know again, that's where it's the devil's in the details and I'm, not a good enough expert to be able to say. Oh definitely, this will be a problem.

D

We need to pick two verses well, we can actually work around it and be good enough for now.

C

It was Bree, yeah I mean sounds like then. If you want no.

B

Idea, I'm thinking of closing time.

C

In seconds, or something like that as a float or a quantity which is kind of throwing away a lot of the you know, the usefulness of time object and height and handling of those. The.

E

Other thing to remember, of course, is that nothing you have true young.

E

Isn't always you can always go in and find the hidden tribute and change it.

D

That sounds like black magics. Do it is that is that not going against the laws of nature, probably.

D

Again, I don't know, I'm not an expert enough to be able to figure out the workarounds, so I'm sure there are ways around that, but it sort of yeah. It is a problem that, if I'm correct and it seems to be I, couldn't get it to work at all. Then it is something we need to consider in the implicate.

C

I mean really, as far as I can see what we want for poor sort of feel like I've, essentially, is just pandas. That unit aware. That's really the only to me anyway. That's that's the big reason to not go to try and think about changing from pandas. We really would like to be unit away. It's just so important and useful foot science.

C

D

Had suggested the two major reasons were unit aware, and not necessarily the time object, though having a higher accuracy in time, but planters can cry the Paris.

C

Is down to nanoseconds right, you can't go any further lower further than nanoseconds I can't.

D

Honestly say: I know what pandas locations are. Stuart told me that it was that the time is more accurate, but I don't know that I times.

E

More, what that I can't remember what it is but yeah it is. You I can't remember what pandas goes down to, but it goes. It goes.

C

Down some other side later.

E

C

I've been using a recently, it goes down to 90 seconds. Okay,.

D

So that's I, guess that's not not so much a problem line but.

C

I've tell me it's it's the unit awareness really. I mean I mean the switching from tables to find this, or whatever doesn't mean that, like this still doesn't need a refactor. So it's.

B

Not the only thing.

C

But it's kind of it's something: you'd rather have you'd rather be able to decide while you're doing the refactor anyway yeah otherwise.

D

You're just wasting a lot of time. I, don't.

E

Know, oh, the two things are quite separate my mind. Yeah.

C

E

Actual underlying data structure is relatively separate from the large-scale refactoring that needs to happen. There's.

D

Frozen, it's also.

C

D

Guess, support for things like coordinates and similar was essentially another reason for why pandas may not be quite so ideal, but I.

E

Think, like that say,.

D

That again, I support for coordinates was that something that was also you.

E

C

E

Put a strap, I corner objects in actually buy tables and stuff, but I don't know how important that weight, but I think there is there's an awful this. There is an awful lot to do on the at reply. Table from getting some of it done would be great, but it's more important to do be retaken.

D

E

Of the actual son by code first anyway, that doesn't mean to say that you can't move on into the Occupy staff, which is also very useful.

B

D

B

Pardons, like the amino agree with Stewart with the implementation detail, whose should be transparent to the Yuuzhan.

B

In what sense you know whether we store whether we use the pandas dataframe model has to pay table so to some extent they should be hidden from the from basic manipulations or the the object go to the Luzon.

D

Yeah, the interface and API can be you.

B

Know I supplies.

D

Without necessarily knowing exactly which implementations being used but essentially always saying, is, though, obviously developing the I API in the sense of I'm having an interface that we know is all see a job, but then we've got to implement it. Otherwise it's a meaningless and obviously the implementation will be critical on what unit pointing it on I'll see if we implanted it, for instance, on pandas at the moment and then decided in a year's time. Oh no implementing on a stripper tables, then that's going to require an effort, a.

E

Lot of a lot of the refactoring is stuff that isn't actually directly linked to the data underneath I. Don't think it's as much of a problem like there's. A lot of the refactoring is how you create the objects or yeah.

D

E

Doesn't actually matter what the data easy and underneath and at the moment you don't actually expose a lot of API that sells stuff for the data anyway. Ok, so like about the most exciting thing we do at the moment is like can take a tonight and that's only reason, like the actual sulfite light curve, API doesn't really expose much in the way of a.

E

Functionality that modifies the data to the user- mostly it's just applying and creation player- hmm ok.

D

C

Mean yeah I think, as you know, it's less exciting, but I think we you know it is that those modifications she was talking about that we need to get done first, just because what we ideally like data format to be and doesn't exist you so, like you steal you are you proposing an astro pipe time series class right, I.

D

Strip disappeared.

C

Okay with this that you used actually sent this link to me Alex, but I have seen it before hello message on this thing.

D

You there's a icon if.

C

You, oh yeah, I, see it yeah I've got it um so I mean as far as I can tell this is kind of like. Ideally what we'd like. Essentially it's yeah, it's the it's, the the most useful functionality of pandas with you know, unit support effectively, um but that doesn't yet exist. So I think, instead of trying to come up with a hack until something like it does exist, we should you know we try and do the refactoring the.

B

Less, what we've got yeah.

C

And don't worry too much about the data format I'm going to make the lack of more like map, for example, and all that works got to be done.

C

You know and like even like, even things, I still think that um you know that's being able to to use concatenate multiple files into one right. Go because at the moment to me that is.

B

Hello, yo, yo lace shift.

E

Yeah, it's all I had just asked her selling.

C

Around that my cookies Nigeria.

C

But yeah I was just saying, while you're gone that, ideally, what we'd like the data to be is something like your proposed asteroid, time series class and but since that doesn't exist, we should just focus on the the API changes or the the the actual object, creation changes or refactoring.

C

Instead of coming with a hack for the data.

A

D

Yeah I can go that it saves I, see a lot of ways. The big problem is re-implementing, the the underlying data storage. Suddenly it becomes difficult to be, like you say it's bit of a hack and there's a lot of unknowns.

C

To happen, yeah I think we should just stick with pandas for the course of this exercise, and just do it like the refactoring, that's somewhat independent to the day lecture and then the future project can be to either write that create and then implement some kind of a stroke, wide x, here's class.

C

But the answer by side on the light curve side, yep.

D

Let's go I, I have also just for the record. I've opened the question up on on the Astro fighter, dev forums so checking that I am right about that limitation with tables in time and also, therefore, potentially pushing the time series issue to some people's the forefront of some people's minds. At least so. I think that might come back with something, but all.

C

Right good, so it's actually been a while, since I looked at it in detail, but just you it's still with us.

C

A

I didn't hear any of that, so I was.

C

Just saying I was just asking if you're still with us, oh, but do you remember the details of the light curve, refactor that aren't linked to the data structure like the Nevada.

B

C

More like map right, it's just been a while since I've she looked at the details of that. Well,.

E

Rip out the data downloading code right and light coat factory and clean out the code.

D

John I've got the link to the actual online project proposal. Part.

C

Somewhere, oh you mean just this one right.

C

What's that one.

D

That looks like it yep. That's it. Ok, yep.

D

Yeah I'm from the looks of it it looks like a reasonably simple task. Then the difficult bit was going to be in three implementing any fundamental data storage, but I played around with pandas. Quite a lot, so I know how they work like you would have played around with a lot of maps now God too many maps.

C

Yeah well, no I mean I, think that would be well I, wouldn't know just to say it's easy, but for simple, but but it's certainly a lot simpler and if we can get that part of it done, then, then like in a much better shape for us to actually in the future, have a better data structure underneath it, which I think we need to do, but it's kind of like you got to break it down into sequential steps.

C

Otherwise it's just like I'm, just not sure that all that could be could be done in one sort of simple project. You know yep, my great that.

D

D

Of course, the the scary thing is actually I'm, not the only applicant now for the like every factor, so it was funny it was less than 24 hours after I put lightning on line ab c, very early group draft and then something somebody asked per day online on the seat. Okay, then, that's that strange line, let's see, I'll compete, I'm, fine, yeah.

C

Okay, good competition breeds excellence. Right, isn't that that.

A

C

Thanks I, oh good, so they say.

D

So is there anything else worth kids are doing for the project. I think I think that pretty much covers all of the questions that I have for now can't think of anything else necessary. I.

C

Think is actually allowed to get on with, with with what we discussed already so yeah.

D

That's cool okay, great.

C

Okay, so whatever other issues people wanted to talk about today, then I guess we can get to them.

B

Okay, I find the bug distant place week with the drawing grids and on some pie maps the you get. You get an air of a nice trip, I Adam.

B

Eventually- and you never used to have that and I wonder why this was not caught in our tests, so I don't think we don't have any tests formats when we draw a grid as I correct, it must be Donna.

C

B

Otherwise, this bug would have little pop top where the phone so.

C

Looking everywhere is the draw grid attribute. Is it in map factory? Matt is Matt base.

B

Now so there's no job it on yeah. So now.

C

I can't find waiter and.

B

Test my face so comments.

B

And so I have them you're going to say something. Sir I.

E

Personal point: 6.4 release, 2 pi PI.

E

Room next bug-fix release.

B

Never very simple fix for laptop pick. Then fold Alex facing in the laptop PK I wasn't actually and mapped optic. He could set draw a grid with only accept values which were just ants or floats will get through doc peak. But then, when you pass that on to rather function, the peak function calls then that would look for and as 2 pi quantities.

B

So there was no way to change the grid spacing and amount of peak so by PR in 1716 fixes that.

A

Sarah tells yes.

B

C

That looks good to me.

C

Surely that's one they can just be merged straightaway.

E

For it back pull that.

B

B

A

I, don't think so. I don't have anything else in particular ya.

C

Know the way, as the light go thing was really well resident, the.

A

Only thing that really needs doing and before nor put seven is some more examples with a gallery in the docks.

D

I've got a couple that I have mostly done here, which I can add to it, but I wanted you to help me won't add them stew to make so I don't make a silly mistake. Still still a little.

B

Apprehensive about.

D

Adding some fuzz no hurry so just when you're feeling better not like that.

B

Well, we can PR for the drug heads and black box cool.

B

A

Cool well, thank you very much. Thanks, guys, cool, that's a steal nice trying to speeches in it.

B

Better, hey is.

E

D

Get better teammate.

A

Cheers Turo.