DevoWorm Lab Meetings, 27 Jan 2020

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: DevoWorm (2020, Meeting 3): deep learning architectures (spatiotemporal models)

Description

DevoWorm meeting: January 27, 2020. Attendees: Richard Gordon, Vinay Varma, Devansh Batra, Yash Agarwal, Bradly Alicea, and Jesse Parent

A

Are you what's the hurry all.

B

C

Okay, you don't know what else is gonna show up, but welcome to the meeting for today today we're gonna have the ashen finish present on the deep learning architectures, so they're gonna tell us what they have found in their you know in research orientation, so I don't know else is gonna show up, but you know they can come in as they come in and so any is there any news from anyone over the last week. Anything they want to bring up issues, deadlines, other things.

A

A

A

But it's for high school school.

A

So, are they before things were musician and we had that.

A

Some issues, like the past's event, the song and.

A

C

You know something like that.

A

Experience yeah yeah.

C

Well, that's good, congratulations, yeah!

C

So anything else before we turn to the talk.

C

Dick did you have anything you want to bring up no.

D

C

All right so yeah I, don't forget. We have well next week. I, don't know what we have on the schedule, but we might have something from.

C

Yes, we might have something from Jessie and I might present something next week. I can't remember exactly on schedule. I think it might be usual actually was on the schedule.

C

So he said he wanted to do a talk on something I think biologically related. They don't know if he's still on, for that.

C

Anyways well will will arrange that over the PA, so there's well. Are you still on for your top next week? Do you know, is it still the same topic.

B

C

He's going to present a DNA fingerprinting and so we'll get to see a little bit about that I mean you know it's again, it's up to the presenter to make it. You know interesting and as well as presented before, so he can. He kind of knows how to.

C

So we're looking forward to that next week, but now we're gonna.

C

Learning architectures.

E

C

You're, on the same account: yes, okay,.

E

E

E

Difference between.

F

You can see the is.

E

E

E

You can use this.

E

Segments each segment.

E

E

Is the motion of objects between consecutive frames.

E

By relating movement of objects in cameras, so, for example, this bus is moving. These objects vectors to the pixel level movement of this bus.

E

Moving across the image and no notion is happening and put each exhibition.

E

In the next frame, so this is this is not about tracking sort of any individual objects and marking them. This is pixel element.

E

We want to model the motion. The motion of them see in this example you can see there are, that is showing.

E

E

E

E

You can see.

E

So- and there is another thing like sparks and then subdivision in space optical flow, you have you make some important points in the image. For example, you.

F

Find some points that I.

E

Put in like the corners and edges of objects and.

E

E

E

For example, in this paper they have estimate this model to estimate optical flows using a sedative frames over video, and this president, this is an instrument, so we passed both of them in the culmination neural network, and so they used first of all, they in the first variant they pass in the.

E

E

E

E

So what they did, they stand two two three three units in a.

E

Units so what happened? They were also looking for intermediate populations, so people not until flew in one states. They were.

E

Also in a break even snow in displacements, but introducing a subnetwork specializing on snow, so they fear of snow notions where they are you've seen that there is no notion, but the objects are moving quite slowly, so they were not captured earlier in the original unit. So they introduced a special subnet architecture which data soon so.

E

E

E

E

Shows, for example, this is a vehicle tracking in speed estimation, this used optimal.

E

E

E

E

E

Many women, so I am.

E

So fashion segments Asian branch tries to segment and single image, while the temporal coherence branch is. The main task of this is to maintain coverage between the previous field teams and the.

E

E

E

E

E

Reform the means at time T in the feature vectors learned from these images from the previous times now they might be some in some patient knows when you're trying to convert something.

E

It's a less number of parameters to a high number of pixel values. So, as you can see, the initial layers are quiet and there's the key technique.

E

As you keep advancing, the number of pixels gets reduced, as also the number of channels is.

E

E

E

E

C

Okay, great well, that's very good! Thank you. Let's take a look at the comments here in the chat, we're talking about the video, so you can share the link with us, so yeah I, like the presentation, if he could send me a copy of the slides and link to the video I, can make them available to the rest of the group just as a link so actually loop. If we're gonna have a chat about some of the slides, could you share your screen again?

C

I just want to make sure people had an opportunity to ask questions with the slides sort of on the screen.

E

C

So first question is: does anyone have any questions about the talk like anything that may be captured? Your attention were was something you were confused about.

E

E

Slowly, shuffle displayed something.

E

C

I guess so well, I had one question. Actually son is thinking about the in the.

C

In the diatom paper that we recently did and we did deep learning analysis of data, we also did a like a template version of the analysis of the vassal area. So you know the the cells were segmented and then they were tracked using templates of the cells. What you're suggesting here is, we could do something similar to that, but just using a deep learning architecture. So you would have- and you probably don't know this paper, but what we had was we had a colony of multi colony of single cells that operate together.

C

It's called it's a diatom, so they're sort of you know it's very: it's not a mic. It's a micro organism, but it's not a unicellular organism, the existing colonies. That are where the cells are, these long filaments that are stacked together and they move in a way that is collective in nature. So the cells move against one another and they have pretty complicated, oh yeah. So we did this paper on this organism called Bessel area, that's a diatom and they're marine or micro organisms, but they're, you know multicellular organisms.

C

So my question is from that is that in that analysis, that we did of those organisms we did sort of where we had we segmented the cells. And then we didn't use this each cell as a template and then looked at their relative motion. But in this case you can actually combine that templating method with a deep learning method to identify the cell or identify the features and then identify the motion of the features relative to one another.

A

Yes, I became their Dean ex-wife. That's what I doing in busy as you so put the pig in with us for the next I'll eat people's heads yeah.

C

So yeah, so you could well okay, so you could do that and it would be maybe more efficient, maybe now we'd have to find out, but uh so you can also you were talking about. Pre train models, a lot and as opposed to like a conventional training method. So could you elaborate a little bit more on that? What's, the is is like part of the envision pre-trained models in this literature. Is it like? Is this sort of standard that they do, or is this like something that they compare with like regular training?

C

I was a little confused about the whole training technique that they're using.

E

Focus on it is that so this is probably the people who generate the most number of times within in these.

A

Slides slow, this might just be a put in one of my attempts, so.

E

A

1-2-3 desert 405: these are the.

E

These are actually from the ResNet model. It is used in computer vision to get let's say, vector of outputs from a 3d 3d frames on.

E

E

We could like produce a completely mobile camera, which is which has bogus temporal and spatial parents, along with lots of weights of base combination, which may or may not be misunderstood, so that can be throughout for that. So, as I mentioned, there were four phases for ending with these four phases of training. This the first frame, the third and fourth actually work on the target, intercept the first two phases, which are done differently for temporal coherence, branch and special governance runs until in use.

E

They use a preset data set which are used for segmentation. Let me open up this profile. Show you what exactly MC.

E

Okay, so these are the four phases of painting which have been using this repeated and overlord expands. It is done over this deficit.

E

This is a standard data set. We could replace it with one of the biological data sets, which should immediately increase our performance and again there is partial segmentation branch. This also uses sending a dozen VMs on a 10k.

E

We could again change it to something which methods would be biological to me and then I guess this model could be used for segmentation throughout biological to me by simply changing daily. So we like the first three.

E

Phases of this model that.

E

C

Yeah I mean that that sounds good in the reason, I well I mean I guess. The thing I would point out is that in a lot of data and a lot of biological data, you kind of have a mix of temporal and spatial features so like, let's take something like cell division, if you're observing cell division in a time series that's out of it, you know you can't necessarily.

C

You know, use like past images to predict the future cleanly. There's gonna, be some correction. You're gonna have to make I mean I, guess you could use it to predict the future, but the image you know it's. It differs over time in terms of what's there you have maybe like fewer cells and more cells over time, but there are these division events that I don't know how.

G

The algorithm would handle soon in the future is, if you put it in the next frame from the previous few frames, it is.

E

A single frame I think you can easily next Phoenix like one twenty-fourth of a second from even the previous games.

C

Would probably work I mean it's good. A wall says it's going to be very costly in terms of computation.

E

C

Mean you know it's it's it's fine I just you know I, think about, like you know, I, just things to think about when you're mentioning these things, they're not as easy as maybe they seem well. I, don't know if they seem easy, but there's a lot of like.

F

C

Error behind all this I know with vinay, for example, we had some trial and error last summer with you know, getting things to run. You know in terms of just computational resources that he had at his disposal. He talked about going to like a supercomputer and doing it in a high-performance computing.

C

Resource, so you know they're these near these considerations. You have to make when considering these architectures but I like I, like the idea of you know having like spatial and temporal streams that can be done. You know so you're doing like a spatial you're doing segmentation and combining those two.

C

C

H

H

E

C

The second slide: if he go, it was talking about going from 2d to 3d.

C

No up the second slanted yeah, so he showed to deconvolution to deconvolution on multiple frames and then 3d.

C

Convolution is just like.

F

C

C

E

E

And certainly it is shape in between Afeni so to deconvolution. We have done these kinds of even less is that there is a bond moving from left to right and such a nice shape, but you can think that there are currents that will Forex shell cheat and get one. So in the mega layers, this type of complex dependencies can be done very easily, so yeah about the ocean. Yes,.

E

The third dimension is.

E

E

To channel channels, but this paper, which is explained.

C

That looks good. Well, thank you for presenting, and if you have any questions you can take them offline, you can put them on the open room slack if you, if Devon Chan Yosh, could send me the slides and a link to that video I can send it out to the rest of the group.

C

Otherwise, next week which wall is going to present on DNA barcoding and then, if you anyone has any other announcements or things they want to bring up, you welcome to do so again. We have think we have slots to fill for the rest of the spring. So if you want to do something longer, like a paper review or something you're welcome to schedule it, we can talk about that offline. So, thanks for attending everyone have a good week and talk to you next week or offline.