Numenta Live Streams, 20 Sep 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: Two Papers on Differentiable plasticity, one on Oscillations

Description

Paper review: https://arxiv.org/abs/1804.02464 "Differentiable plasticity: training plastic neural networks with backpropagation"

It is aimed to be a connection between the work we are doing, with structural plasticity through Hebbian learning, and continual learning.

Will possibly review a 2nd paper: "Backpropamine: training self-modifying neural networks with differentiable neuromodulated plasticity"
https://openreview.net/forum?id=r1lrAiA5Ym

Subutai, time-willing, will go over "Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties"

https://www.nature.com/articles/338334a0

A

A

So I'm I think I'm live, but the meetings not quite ready to start yet we're having a small change of plans. This is a research meeting I tried not to disrupt research meetings. Why are we going to a different, authoritative, camera? Okay, I, try not to disrupt research meetings. I just want to let him flow I just want to live stream. Whatever happens there, so it's hard to plan.

A

So what you're gonna see is differential plasticity, but we're going to review another one first I think on oscillations and then go into the differential plasticity at some point. Jeff has to leave because he has some engagement and he just wants to hear both of them.

A

I'm, not sure how this how well this is going to work, but I'm just going to put this on my room Lib for a minute while they're just setting up but I, wanted to get everything properly set up, so I'll be right with you. I should put like the I know, there's like a starting soon. That's it! No I used to have something said starting soon.

A

All over this is just change that to starting so.

A

Sorry this is what live streaming is like. So you know if you're gonna watch this channel get used to it a little bit alright. So if I do this.

B

A

And you don't have to pay much attention to that. That's my agenda. It's not like I'm, probably not gonna stream, all of it I might I might stream the thing of Jaden later on twitch. So if those are just joining, we're gonna start as soon as they're ready, but we might start with a different paper than the ones we were talking about. This is a live research meeting, so I just kind of go with the flow here, and here we go we're getting something just a moment.

A

Okay, I've got a good view. You guys, whenever you're ready.

A

Okay, all right we're on the air I.

C

Defended America small yeah.

D

Very small and many of the possible ones, particular each Solomon arbitrators and they.

E

D

Inside the same.

C

D

E

So so we're actually going to do this oscillation paper first something and then this would be hopefully very quickly and then then they'll switch to us.

C

Yeah so I just went across this.

E

Paper recently and I thought it was really interesting in light of the thousands some that we talked about, and we've talked a lot about oscillations and synchronizations, and but not a lot in terms of cortical columns and how they synchronize.

C

Possibly they talk a lot about simulation, yeah.

E

Alamos and good cells and reason, so this is a fluorine here that I'm sure you know a lot more about this area.

E

This funny coincidence happened so I I want to talk about this, and then the question came up in my mind as I'm reading this in light of our theories, I actually spoke with Peter in this morning. Real I'm asking my question. So I have an answer.

E

E

This can be, this paper can be somewhere.

E

Yeah so well also I think what he means by column here is in any column.

E

Because it a functional column, yeah but I think.

D

E

D

F

E

So it's basically oriented bars that it's being shown to a cat. You wanna have sometimes yes yeah, and there is looking at synchrony between columns that are close together and columns that are far apart and far apart means as far seven millimeters away.

E

That's pretty far, and so these would definitely be in different cortical columns. You know, and so the basic thing that he finds is that you can look at the receptive fields and when they're, nearby and they're and the stimulus is consistent with their preferred orientation. You get some synchrony, usually.

C

Near about what you.

E

Said two millimeters yeah: it's a big range of 0.4.

E

C

I think you've known about many.

D

Columns this makes.

E

Sense, because if you want to find too many columns that have the same orientations.

C

Jason many colleges are mountain, mini-cons I have the.

E

C

Orientation jump distance, okay,.

E

And what they basically find is that if two columns are seeing the same in their line, corresponding.

F

To this thing, responding to the same statement like their.

E

Faces suddenly become really surprised.

E

You give me my gamma, and so really they just show that.

E

C

Is so so, presumably these are everything's parts of the retina that are just not same whether it's displaced right. So what this is stimuli? Is it like a line that's going across and these are both aligned.

E

So they run both see.

B

Even though they can be like.

E

Vertical vertical I would right: nowit began to develop one cases. You have a bar here and the bar here: okay and their movie consisted in the or not or the.

C

Second case, if you have a single bar or I, was thinking like it's not like the only thing you could have or something like this. Would you have a bar here and a bar here? I know it may not be correlated all right, because I would have to be two parallel lines. This could be.

E

So we just talked about the scenario but also.

E

Yeah, so here you don't.

E

So here's like we would say it's like the same mob yeah, so I thought that was really interesting because we've talked about voting across article columns and so on, but we never really talked about synchrony and to me this is synchrony and our models. This was really interesting because a lot of our stuff happens through active and recommended excitements yeah. If you have the neuron and you have these active dendritic segments in order for this segment to invoke in an MVA spike or to fire, you have to be less than five milliseconds.

E

You have to have a coincidence, that of activity within 5 milliseconds and the best way to get that is through synchrony yeah. This is sort of it's almost.

C

C

E

And so this does not imply oscillations, but it'll be nice, secretly yeah, but so here there.

E

So I thought that was kind of an interesting all right. So my question was to him, so this is great, but this is descent lines. I heard about actual objects, yeah all right, so I want her to find out about things like what, if welcome you want, if you have this, but it's actually a same same object yeah. So you know something like this yeah, let's see a rectangle yeah and maybe one part is looking here now: the cars looking.

D

There yeah, but.

E

It's the same in all, but the same object and we see any evidence of synchrony. This is primary visual cortex of this, and so he then sent me the link to this.

C

E

Take a look at synchrony, but it's very I.

C

Mean in some sense we were almost already the cat yeah.

E

You know can be some random, so he said they.

C

Haven't quite done.

E

This but they've done natural stimuli and looked at similar things like this in.

C

D

E

So what they looked at- and this is a 2013 paper, so they looked at- is our movies like.

E

And then they presented stimuli from they had two different movies and they presented stimuli.

A

E

Just the two two pageants, but they could be from the same movie or from different movies, so they could be coherent or so kind of similar to that and, of course again, I've got this table literally twenty minutes ago and then really our store all of this, but they do a bunch of different they meet, but they base a conclusion and have been really go into. The more detail is that when they're from the same movie and they're co-parent, you get a lot more synchrony and the final activity. No. Thank you. This.

C

Is kind of an artificial stimuli in the sense that you're only presenting this passage? It's like it's like those images, they show you like, there's some circles and there's some lines and the circles and then, when the lines move, you realize there's an object behind the circles. Yeah no filter, it's kind of like that. It's.

E

C

E

Tried to control for this here and I think that's what is like how much context you see you see just at people meatballs. If you see some more contacts, so they try to control that need to read the paper anyway. This was pretty it's pretty.

A

Interesting and the fact that any of this is even.

E

Going on in primary visual cortex, you know certainty is not consistent with.

E

We want a little bit of B to it.

F

E

Are in cats just.

C

This very first one, two thousand two.

E

Thousand thirteen, do you have a book yeah? He complained to me.

C

Anyway, mm go.

E

Ahead yeah so saying this is a new research where, although it's very open and the reviews open and everything like that, they thought they would publish there, but it actually is not getting. No one knows about it, so it's not getting much tried. It's.

C

Not oh yeah, they wanted a little platform without editors. Oh this is this journal.

D

E

D

C

Morning, I think that's a.

E

Great idea, it's a great idea, but it's going to take some critical mass anyway. I think this is.

C

Yeah, there's no.

C

These Asians didn't show up. I couldn't argue that maybe it's because they have to NASA ties right hand and, but fact they do show up. This is still definitive.

E

Yeah I know models might be analogous to where we like grasp with a single yeah.

C

It's also sort of like saying when you're in essence eyes, you know in a coma. You know some sensations on your. You know.

C

It's very hard, you know what happens when it's an S, but have you seen these results and exercise? It's still see that me of that. If you didn't see results right, yeah.

E

Thank you. Okay,.

E

Paper up in case.

B

The reason I really fast is that, because the paper is not think you think just it's not very interesting for Harvard search, I would say so.

C

B

Paper using interesting, okay.

B

C

I'm just curious: if we don't think it's really knowledge of fairies is. Why did you choose that by just.

E

Stuff he's been working on for many years before.

B

So just this feels like bacon in so the main difference in his paper is.

B

This is just the same, welcome forward here, and the difference between the network in a regular is that the output of the network is also linked good for the next. This H of T here is good for the next, so this is between.

B

And this is an LST m and you may not recognize some modifications. Anything I just took the memory out of the cell and I had the same. They were going. No network has a problem before learning several time steps in a row, and then you get a loss. You have a problem by propagating that how they're repeating so the difference? The way Alice can fix this.

B

It adds a memory, and this memory is, along with painful, defines the output of that current time start that memory is modified, so we have three gates: neural stem and chair you, one of them defines how much of that memory you're gonna take into account. So it's like an input cake, and these two gates are the output gates and they will define how much of this neighbor you gonna forget and how much of what to learn here. You can add to that memory.

B

So basically, this line would be the wrong short-term memory, which is edited a week on their network to fix that and vanishing great in problem I'm, not sure.

D

If I follow this, ignore that the memory itself sort of partially.

B

B

D

That, because it's not less liking right and we have the input and the output right. So this this memory.

B

Is going directly and we have this to set of weights so where squares are weights here and they are defining at each time step what you gonna forget from your never know what you're gonna forget.

B

So these weights will be learned and they will change the neighbor that the memory is going to affect directly the next output, along with the next thing and I'll answer. The question.

D

Does that mean that whenever from no backwards, I'm traveling back in time, looking at those memories, so it's actually a temple trace of some sort.

E

It's like a temporal trace, Imagineer grammar, where something now can affect something concepts later right. It's very difficult to keep this trace around for ten steps difficult. So what these things will do is is some part of it will recognize, hold I need to remember this, and it will artificially just keep it alive for until the system says. Okay and.

B

I'll just keep that memory alive.

C

B

Parsing this paper there's a.

C

B

Network and you.

E

B

A memory component and the memory component here, who's behind and trace, and what he calls ahead and trace heavy interest yeah and the way it's calculated so the head and face it's the product of the pre-synaptic activation and postsynaptic activations. Just things with an output register paper words and the product of those are going to compose this hydrant race, and this handle face is updated from statue step so from time 0 to time step one it's going to flow here, and it's going to be updated with this with this correlation at that current time step.

B

So you can see it's kind of similar to the data stem and the heaven trace is also calculated. So you have the input times of wait plus they have been face times and after this, which is also set of weights that are going to define how relevant, how much are they taking to account ahead and paste into the calculation of girl so similar to the.

B

B

This small learning rate in these equations- it's switched by an actual gay, so it is okay, so that this is the Nero modulation perfect. So the neural localization is that this is going to tell you how much you can update at each time step as before. You only have like this global learning rate. Now in the second paper, you have a specific gate, so this is the so this is the again update rule.

B

This is the simple one which is just the learning rate, plus output, plus 1-minus learning rate and the video 7 phase, and this is the OTS truth. I was not computed, which is another way of updating heaven, so he is most phony and asynchronous in front of the problems. What.

C

Is the motivation college? Did they start officing we're trying to recreate allis-chalmers using back propagation, or do they just get this independent valves down? How does have some paper position.

B

Itself in his field, I was also I was also curious about that I'm, not sure I think.

E

Part of the idea is that when you update the weights in a typical back problem at work, everything gets the same burden great yes. What he's trying to do here is that have some of the weight, the more plastic wrap great function of.

C

Differential update for them for like what to remember, is not always the same, and instead of cleverly figured out what things should I keep in the trace. Not I mean this is how we normally have of this context is propagated throughout, and we do this with separate representations in every southern point of time. Here they did something equivalent, but just by saying this is the thing you need. This is the context you need to keep track of, to know what to put it in X, so it was a clever, different and so typical back propagation.

C

A lot of interpreting here. What you're saying is typical pop isn't.

A

It nothing like that. There's no idea of what's important.

C

What's not important and they're trying to add that idea to back propagation.

C

Using back propagation to to make its now there's no longer being uniformly applied all the time I.

B

Ask you to see how how.

B

C

Not saying is Jim Linnell still any super. The Santa Bangkok we're trying to make back talk about.

C

What's important but Ellis Jim has a concept of what was the important things in the past I.

E

Need to keep track of is that I'm trying to answer that's a fair assessment. Yeah I'll, take it and is using Co activity.

C

E

Yeah that works but yeah the main difference.

C

This crazy line here.

C

You think that heavy learning yeah well, why how's the heavy component going to try this differential knowledge I, should.

B

Probably make this just say the kind of test it isn't you so the kind of test is not this whole bunch of parents. Super network and dispenser show, like you know, a bunch of times no Wendell random, some random order or in the water. Yes, and then they keep the network.

B

That's it and they'll give the network just a big rated version of it and ask network to reconfirm this part. So.

C

Not really doing anything in time, they're just doing completion.

D

Just pointing out that the ideas nice, but they like them, was possible tossed with because the whole point of the scheme.

D

Where they're, trying and.

E

Even this trace implies that there's some temporal coherence.

D

Yeah, so that's a weird thing, but of course that pattern completion, the heavy component, will.

C

Do its job right, yeah, well,.

D

That's that now.

C

I'm Morgan intuition, why the reason, having a piece that I was trying to understand how the relevance of that would be a time I think Ellis. um Does it, but it's kind of clever I, don't remember it, but okay, so that that's essentially this someone, my curiosity or a confusion about it, because they're not really you thinking for time they passed yeah, which is odd.

E

I, don't know why you use a moving average like that. You can just compute the average if it's yeah.

C

B

E

E

Did you perhaps the components are important.

C

And did not take advantage.

C

I mean the idea, building a temple in predictive model of the world and and so different ways of encoding time in diction time, and my sequences is interesting. So we have a model.

A

That Alice Kamath.

C

A very interesting model to which I don't think it's for the biological, but it works well, and- and so this is a new way, I'd like to know it, but this sudden this doesn't Illustrated, it may be the base.

C

In some sense, this is what has been traditionally until fairly recently. This has been a backwater of a all machine, learning and neuroscience research to.

E

Deliver time-based packaging system hasn't been people's focus, one kind of loose connection to our a10 models.

E

In with the notion of permanence and the more Co activation you have, the stronger the permanence would be so, the less likely it's going to go from one to zero yeah, but if you're sort of in between you're going to be right near the threshold and you could easily go from one to zero now sort of related to here, you know if, if the correlation between two units is very strong, the weights are going to be very strong at last. My last second one, so here.

B

Yeah yeah, that's relationship being a seven, is same same question of the paper that make sense for distance. I was also not seen how they have.

C

B

C

Train for a while on that so.

B

Yeah the words of spot.

C

And so they sell the system with a good 3-1 in one spot. They put, you start the same spot and the middle.

E

More changes every time, but in kitchens.

C

Every time there's nothing the line well,.

E

You have to learn how to navigate in this, but.

C

How could you I guess the point? Is you can't I guess you'd say you just don't know you wouldn't go back yeah. If you change the location of the reward, then you're. Essentially you have to do a sort of random walk to find it. No.

E

No, no I think it's given. You know where you already know where their reward is. How do you get there.

F

C

D

How they there or how.

C

Do they specify the the locations.

D

And this immediately transported to around the location.

C

In the mist, so rather to start them in there.

C

Am I feeding into the.

C

Building a ruffles friend play cells and we're going to map of this world and then use the calculator to get someplace where it could be like a bacterium. A bacterium has a strategy, it goes forward and then it turns every turns more often when it doesn't find what it's looking for- and this is more like the bacteria might have.

D

C

It is the maze there's no concept of overall structure. It just has a few things you can do and it's coming up with a strategy.

E

Could do it just doesn't go back yeah.

C

But that's not that's tight, the bacterium, saying I'm not going to turn randomly I'm an attorney with the various bacteria doesn't know where he is in the world. He has a strategy for movement which is successful.

C

After two hundred iterations of recess, then the reward chains, but the reward location, is in.

A

The same place again and again, and it's being random melville opposed that somewhere. So it's tasks as a represent. What is the location of the reward using the array today, us together, be localized yourself, see.

D

C

Out how to get from it's.

D

Been given an inferior because.

C

It can also I'm trying to money is being given all of it represent. That could cost learn the task as given these parameters. I just know that.

C

Without actually having any concept was the overall structure is in the same way that the bacterium gets the reward know I give your structure is so I, maybe I'm wrong about that. But I don't know this meaning practice.

D

I'm just saying it is in theory an idea model could.

E

Learn the ideal scenario is going to do better than the bacterial scenario. So if it's.

F

Really minimizing.

C

This area, it's gonna, know that this could be an advanced back trip. It says: oh, not only do I just turn less often when I'm getting my mortis is given this set of inputs. I should just follow this pattern. Moving in it and maybe actually know just like a network can recognize it.

F

E

It is that whole it to.

D

Continually and refreshes itself and tries to do things is that with us he gives.

D

Off the whole paper right, I mean this is like, like.

C

So the mazes of the first papers.

C

Much less difference.

B

D

Mean I mostly paid attention to like the way I'm used to having my weights PD heavy weights. I mean somewhat optimized anyways, because I usually.

D

Try and the posterior and but but here it's sort of an additional component on top of the weight which.

D

E

This thing on their list: yes, this is purely a table, but it.

B

Does like this alpha I mean I'm in plans right right. It doesn't. Maybe the thing of it is gauge, but it's a fairly.

E

E

B

It doesn't have.

E

E

Right so it could learn that certain weights should be much more important right now.

F

If I know that right, so the idea is that you steal these paper. They still use back propagation to write.

F

Down these just is your rule right, monitor.

B

Clicks a plastic barometer to motivate.

F

It's very similar will be as seen in a computer where you want to find some of these orbits above each weight change.

F

So I guess this is just some remarks that you know you can estimate the importance of our synapse. Wait. Sorry, just looking at a pre and postsynaptic, essentially the two currents.

F

Know I was talking about me. It's cold.

F

You could somehow estimate importance of a particular wages who can get the free, and so it's kind of signal, I, don't know why.

A

F

Compared with other techniques, century are doing similar things to see that it's.

D

Just the method section chapter three.

D

F

D

There you have the head of X component, a green and then modulated, happy-end component, which has this this, and so you 91, was a little confusing. To me. Advice was the fact that, like I used to my waist.

D

Had been component to it, with some okay.

E

So here, there's no LS without the heavier component is just a standard record. Yeah.

D

E

A

No, we know the heavy is just.

E

D

Unless you build a different kind of air heavy, a load.

E

Of a you, have.

D

E

When I'm, not clear about is this alpha parameter here holds that.

E

E

B

B

Right then, you would expect.

D

Because, like a lot of activity would be explained by just correlation yourself, whereas if you highly temporal patterns of a super fast aging, you will expect that component.

E

I'm not expect the opposite. If it's very sad, you don't need this. Have you component at all, you can just do static return. That's fine!.

D

B

It's static, like you always showing the same thing. Then there is there's horrible and entrance cause got a fencing team.

E

We go, we show the same thing as a record that can learn it. Just fine I.

B

D

This is traditional.

E

Recurrent net, which is also required, so what this way is of you in if you use Lucas's own diagram right this self connection here right here, this other WI J's right and these are static networks right. This is the new.

B

Way to the bus, that's.

E

Great, but it's applied to this is these are recurrent ways the weights for the recurrent connections right. So, if you think about I draw a bunch of units, you know they're all fully connected here.

D

So if you can learn it.

F

Decide which ways to preserve yeah so.

E

This this, these weights here are learned right and they're they're always static, they're, not dependent on the inputs at all. All.

B

D

B

So these are learning using a data.

E

Set or training set, and you come up with the optimal weights area and then for a given input sequence then the only thing that can changes this heavier component, and so this thing can say.

E

Okay, if I'm saying let's say this, alpha is really high, then, if I'm seeing a lot of correlation between this guy in this guy- and this alpha is high for this connection, I'm going to pay a lot more attention to this collection, and if this guy alpha is really low, I'm going to pay less collection less for this specific input trace, so that so the heavy end piece can be very dynamic and dependent on the first static case.

E

You just don't need feature that you have this base yeah work there, but then I still want to have something if I'm being fed something different. I want to have some way of adapting to it, and this is a mechanism to at least partially do that right, and so you know it's clearly more powerful than there were.

B

B

E

I think his argument was that it just it learns a lot faster.

D

It's the signal, reward prediction, error yeah, but none of that is here. If you mean no protections.

D

You get dopamine release well with all the rewards that would be expected. So so dopamine is not a reward signal right, and so you wonder how this applies to this right. So how would this network recognize that? What it sees is not what it expected for them up regulated as the city I'm, not.

E

Sure so you know they don't really have any sort of reinforcement, learning, maybe.

B

I think the point of dopamine is the neuromodulation. Is here.

C

B

Net a either so area is going to control how much of the output here you're going to how much of these actual cultivation are going to use to take your trade, quick that hebbian component? Yes, peter, is gonna, so that would be the dopamine part of it. So well, it's not really! No.

F

E

Florian I think that's true, but it's triggered by a bit more prediction error. So, if you're expecting a higher reward and you get a higher reward, you're not going to get much dopamine. But if you're expecting a lower reward and you get a higher.

D

Reasoning, just being that, you know when you're offering your prediction your that book is wrong, so you need to change yeah, whereas if you were like to not expect to reward and you fewer rights to an expected one got it it's nothing. You should change nicely.

E

It is the same it very tight correlation with temporal difference.

E

D

That would be a great.

E

Topic for a future but I think just yet exploring that. Are you going to meet up? That's really good.

B

D

A

Anything else modulatory is.

D

Okay, weird yeah.

E

So does anybody.

D

Know why I laughs what they do? Yes,.

E

I'm gonna, when we stop the lights, treat me right: okay,.

A

Just just a moment, I'm going to stop the live streaming.

A

Okay, you're good.

C

A

Not being straight.

A

Okay, I just want to say thank you guys for watching and hopefully a tune in for our next livestream next week.