Numenta HTM Community, 2 May 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: HTM Chat - Forum Q&A - May 2 2019

Description

Broadcasted live on Twitch -- Watch live at https://www.twitch.tv/rhyolight_

A

At least one I'll do that real, quick and then get to work at 9:00. Tell me how this aggregate info help and improving the accuracy of a model.

A

That's such a hard question to answer. It seems so simple, but you have to know about data and patterns of data, especially sequential patterns and data. I mean imagine, lots of waves sort of occurring over time. Aggregation is sort of a way to sample reality so so that you can continue to keep the patterns represented in whatever the sample is so to know how to aggregate your data. You, you have to understand your data. It's different for each data set I would say so. I'm gonna leave it at that.

A

I, don't know, I, don't have a good answer for that, but I'll mark it and I'll post that video. It's my lazy way of answering questions, hello. It's on bento, we've interacted quite a bit over the years. I have just found an interesting paper at a nice 2019 conference author uses fast ret and the key points descriptions for images fast, retina, key points, I, don't think I've heard of that bunny has sparse. Just Hey look at that: it's like directly related to HTM binding of sparse distributed representations in hierarchical temporal memory.

A

New morphic AI lab Rochester Institute of Technology. Oh, this is nice.

A

Cool this is from nice, 2019 I, don't know that workshop. It's one of the workshops.

A

Day 3f, so it's one of these workshops.

A

The author was what was his name Luke, let's see if we can find it. No I was for the oud you, okay, so there's names not there, but they might not have it. They probably don't have them all listed out like the workshops. It's just organizing teams and the main speakers, but so that's cool announcement agenda, tutorials demos.

A

I'm not paying attention to whether this is an act. Hey hey mark I bet, oh you're, probably not there anymore, whoever's! Listening! If you know this, I've got a record. I have a record that has two copies of the same basic out. Basically, album, hey Mark, you might know this because you're sort of an audio file and you're an older generation that would appreciate appreciates this form of media. Yes, the king is in I've got one mono and one stereo record of the same. Album are those because this was recording like 50-something six.

A

A

Well, it says: copyright, 1972, but I know these recordings came from earlier, because they're all pre-recorded. This is like a anyway ones, mono ones, stereo were those actually too physical, physically different recordings in the in the time that they were recording, different tracks for mono and stereo, because I'm sitting here listening to the mono version, trying to compare it to my mental model of the stereo version that I hear all the time trying to figure out if it's actually different.

A

Yeah I'm just curious.

A

I'll, let me look through the presentation, HTM structure, region. Okay, we've got to be careful here. We want to say mini column, I think yeah. Let's call these mini columns, but we were bad at this. In the beginning, we were bad calling this. Oh, the record, lay the worst. Oh I'm, interesting, okay, so that's cool, that's obviously that's the value of having an extra record, because it's a totally different recording of the artist.

A

So that's neat hello, James I was I was just chatting about vinyl records with bit King and going over our forum, htm' forum, computers aren't real. Perhaps now what is real. Nothing is real if you want to learn what we're talking about. It's hierarchical to form memory, there's a link in chat there and explain sort of the concept, so they were cut separate least two different masters for pressing. They may be the same. Recording oh I figured that the interesting thing about that for someone.

A

Nowadays we would to have the only reason you'd want to have two recordings. One mono in one stereos.

A

If they were different recordings, I hope they were, but I haven't, like noticed a difference. Yet, in the background, just stack more layers, I, don't think that's a that's maintainable! Alright! Now I, don't think that that works. The way you think it works.

A

Yeah Mysterio was new. It was a new feature. Okay, I want to look through this special cooler structure, so these are many columns. This is their like potential pool, / receptive field. Each seller see the same feed-forward input and put flows through proximal synapses. You know, there's approximate connections, heavy and learning online learning, it's a good summary sort of breaking out what an SDR is as representation it represented in spatial for this nice pictures, I like it when people make their own pictures so until they have an understanding of things, non-local is distributed.

A

Non-Local asturian such an interesting way, to put it learns to have similar cuts, similar outputs similarity can be computed with a dot product and then the essential algorithm, explaining overlap comparisons and then inhibition I, don't think I've seen these out I think he's these algorithms. These aren't coming from our documents. I, don't think so. That's interesting too I mean, of course, there's lots of. We had this all came from intuition of looking at how neurons were working and trying to figure out what they were doing. So we didn't start with the math.

A

You know you've got to go, find the math that explains the biology. So it's interesting to see people finding the math for it right. That's cool and I'm, not gonna verify whether this is right, I'm not a mathematician, but this looks somewhat feeling of the sets like I said: I'm, not a mathematician, but.

A

Inhibition so we're talking about groups of neighborhoods, ok, the motivation. Our research is focused on bio possibility and emulation of the pathways in neocortex. I would agree with that.

A

Binding operation is a basis for content-addressable memory, which could help facilitate long term storage or feeble content. I've I think I've heard about this before, but I'm, not not enough to define it.

A

This must be his motivation, perhaps combined multimodal data without increasing dimensionality. Definitely that's. This is HTM as a way to do that for sure. We're gonna check out the asymmetric confidence thread. I have some things to look at that may be wrong. Yeah I'll pop over there about 10, more minutes, I, don't he's talking about the binding problem, vector symbolic, architectures, vector operations, difference different implementations of vs A's.

A

I am not familiar with these sparse, distributed, sparse pioneer, distributed representations, I. Think I. Thanks for the follow James interesting enough that you could stick around for the next one. You know I'm I have a schedule if you want to look at it and I try and put events up on my vets page. So if you're on Twitch look down and at my schedule, that's usually right but there's an event- a link to an events page that I've trying to keep totally updated.

A

So like there's an event in ten minutes, we're going to be working on a project I.

A

Can I can feel the sarcasm James?

A

Okay, so this is cool I, don't binding and unbinding I'm gonna I'm gonna skip some of those back. Simoni sparks STRs use a local vape on a mission official or vast retina keypoint. So this is what I have not frickin coding. I've never heard of this before faster retina key point, so we're looking at like motile multimodal data or no difference of gaussians I. Guess it's something. I have to look into. Is the free, fast retina, a key point vast retina?

A

A

Boy, okay, so it's like basically gonna have to read a paper. I'm, not gonna, read the paper, because if it's interesting, if somebody's interested in the fast threat in a free point, you think it's a valid thing to relate to HTM, which this guy obviously does, but basically I'm gonna. Ask someone to explain this to me on the floor.

A

If you want to I mean, is that I'm I'm not gonna, go down the rabbit hole right now, because I want I have to get work done today, so like 9:00 to noon, I'm gonna be working, and this is gonna. Take me down. This could be half an hour. You know, I, don't want to do that.

A

You did a scan of the paper. Okay, cool cluster analysis, finding experiments, I love, seeing this stuff, though, even though I don't understand what what it's doing I mean the thing. The ideas behind HTM are gonna, be behind a lot of things. You know of understanding more about how well you can do a lot of stuff with it I think. So. That's very cool, that's great! Really, that is an is really interesting for sure.

A

A

Asymmetric confidence: oh yeah, so let's go back so we can get a little history on this, so the problem originally was defined by phil goddard and he was trying to figure out why, when he fed fed HTM a sine wave, why didn't the anomaly confidence or whatever, whatever you want to call it he's, calling it a confidence but I think he's just plotting the anomaly score?

A

Why does it continue to go? Why does it go up and down with the data and I? Don't think anybody really understands why my my initial intuition was because there's because there's no reset, so it never knows what psyche, what weather is it in the one period sequence or the two period sequence or the three period sequence, because it's just building up more and more and more and more and every time it ceases, and this is the value again.

A

It can branch off and create, say: oh I, don't this might be a new sequence or the old sequence, so this one might be a new sequence. Oh this one might be a new sequence. It never recognizes that this is just one sequence repeated over and over again: okay, that's the that's! The core problem, I think that's probably causing this fluctuation.

A

That being said,.

A

Mark says yes: I asked some questions that he tried to change, not not not change the symptoms. Try the change. Okay,.

A

So it makes sense if the confidence was consistent, always good are always bad or always jittery, so I, don't know why what causes it to change characteristics over time, that's kind of weird. But that being said again, this is such a dynamical system. It's hard to anticipate what it's going to do honestly,.

A

Last two graphs unchanged of the finite values, how many I'm trying to get to I can't remember all the context and I want to thank because originally it was fill and then recently that we didn't really find an explanation for this as far as I could tell, but then pooling if I'm saying that right um looks like so I'm interested pulling.

A

If you're watching us did you do this from scratch, like yourself like, based on what Phil had done, I'd love to see the code, because this one might be of an easy codebase to sort of change some of the values and see how it affects these graphs.

A

If it's just if it's just simple to do that, like like I asked you, maybe change the jitter to a different value, and you did seem to do that pretty easily, but I still like what I was saying is I. Don't understand why anomaly score goes up and down with the wave unless it has something to do with it being unsure about well yeah anyway, let's read through what Vic King said. So we see stuff like this when doing integer math- and you were at an article about this too- that I thought I figured.

A

You would link here anyway on imp use, I, don't know the code and totally uninterested in digging through the nested dopey stuff. Yeah I don't want to you know it's not so bad in the spatial Pooler. Most of that logic is pretty in the Python code. Anyway, problems with signal versus absolute values come to mind yeah, so absolute value is being sort of a changing, a signal to only output, a certain range of it, but I don't know yeah.

A

So this reads to be a problem with comparisons and other common and related problem is shifting frames of reference during calculations picking the wrong intermediate value for further calculations. This could be related to what I'm, trying to say might be I I feel is this most suspicious thing is the fact that it doesn't know that it's one period over and over and over again it just it sees it as a continuously building sequence. That will never end the other. Two perennial problems are getting scaling wrong.

A

I think yeah note that the error seems to vary with the absolute value of the sine wave right and.

A

If the calculation was using this input to establish scaling of the value, the error would look something like what is shown.

A

This is based on a time varying value using the before, and one part of the calculation of the after, in a different part, can be maddening than the anything too difficult to find.

A

Yeah I think you're, like probably bolted down to the core problem, because every time step in the past is dependent on all of the time steps in the past previous to it, and because of we're seeing some randomness in this, because you have to establish the initial connections that are in a random way, but but I think you're just seeing sort of a oscillation of the system as it attempts to learn and unlearn about pattern. Unlearn Obul, because it does not recognize that the period is ending and repeating that's something.

A

That's a deficiency in temporal memory as it is implemented today in HTM. U you have to manually, tell it. This sequence has ended and that sort of cut severs the connections and then everything bursts again on the next input. So you can start new sequences that make if you were, if you were doing that at every step of the way here. Reset reset reset, it would not probably be doing, would probably get flat and really confident.

A

But I think what you're pitting out here mark is exactly right. I mean it's not necessary to pin down exactly why this is happening. In my opinion, it's just a a characteristic of the dynamism of the system that we've set up here and the fact that it's at least somewhat consistent.

A

It's probably a good thing: I'd rather see this than a completely noisy graph. Of course, I mean honestly. What you want to see is perfect predictions and no prediction, no anomaly likelihood at all. Okay, it's still going still going distracted with my own thoughts.

A

Confidence values are the ones that are abnormally fluctuating, I. Think I'm, pretty sure we're talking about anomaly scores right when you say confidence, values. Is that that's not a term we use I, don't know where that came up. I think Phil introduced it, but let's call it anomaly scores and that's what he's saying you sure that always scores in your plot.

A

So the prediction, blue confidence and orange anomaly scores in red this you're talking about not only likelihoods, so there were terminologies ofhere confidence is anomaly score and what you're calling anomaly scores is anomaly likelihood or an anomaly indication or some a flag, anomaly flag right.

A

What and you're say it's either one or zero okay, I'm just getting my term straight here, so he added noise. The way I suggested and it didn't really change things very much I mean there's, there's still it's not it's not! Well, actually, it does look quite a bit more random. Doesn't it but I still don't know what that means. Mark I'm, assuming that sine wave is the value between 1 and negative 1, just for diagnosis, diagnostics or whatever try between 0 & 1 makes no material difference to the results presents it.

A

Oh that's Phil, so Phil's been watching this and he was trying. I I must have missed this perp, assuming it's working correctly. I would agree unless there a bug in the software one of the possible bugs is issues with values.

A

I think you've misunderstood my Steven I'm, not suggesting there should be no material difference, I'm, stating that the above examples have all been executed, with a sine wave going from negative 1 to 1 and from 0 to 1, and that there's no material difference in the results.

A

I did not see where that was tested in a graphs above it seems Phil's, correct, I found no significant difference between the results going from 0 to 1 Robert. That's, okay,.

A

So we're just talking about moving the sine value and I'm, assuming that everyone is changing the encoding appropriately when they're doing this, because it's you're gonna have to change your encoders.

A

If the encoder I don't know what the encoder is for the sine, but if it's a scalar encoder, you know if you have to update the min and Max and stuff or it will cut the input, you know, it'll only give half of it: I don't it'll floor it or ceiling. It so make sure that you've updated your a coder. If you did this for the min and Max zero to one and not zero negative one to one but I assume, that's where you're doing.

A

They certainly are much higher. Aren't they let's compare where's the original one here, yeah.

A

Let's just save this I should I need to start soon. This. This is interesting, though, so that there definitely a higher, but these are if these are not only scores. They're not confidences. Oh I, bet anomaly. I bet. The confidence value is like the inverse of genomics core.

A

Okay, no dips below twenty percent versus thirteen okay, you did yeah. So how can you say? That's the same. That's a good point! If this isn't percent, then the value should be the same, no matter what the magnitude.

A

So so I'm confused about. Let's talk about terms we're going to maybe let's just get this terminology thing out of the way.

A

Up terms, just to make sure we are all talking.

A

Only score comes out of you pick a raw value between zero and one.

A

You want me to respond to the whole thread: uh-oh.

A

Oh, respond to the whole FERC gotcha gotcha yeah yeah, make sense.

A

Anomaly likelihood is the post process. We run to get it a vow that is easy to threshold, which is used to create anomaly.

A

Flags questions: what is confidence? Was it where they call it. Confidence.

A

Confident what is confidence to say what is confidence.

A

Here are the typical terms: I usually use okay, yeah, no problem. What is confidence and how are you getting the the flags and your red line, you're using anomaly likelihood.

A

Okay, I'll post a fit I'll post this video right here too, okay, so I'm gonna reply with this and I come back around I'll post a video to it. Did you oh yeah, yeah, that's a good one! Yeah! Did you reset the well?

A

Did you when you changed right when you changed your and put range from negative 1 1 to 0 to 1? Did you properly update the encoder in max okay?

A

Okay, all right now, it's time to get to work, don't forget this Bay or meetup. If you want to come, hang out with us, I hope you didn't turn off learning, that's shouldn't! You shouldn't turn off learning. It would be a bad mistake. I mean it's sort of self-evident, don't turn off learning.

A

So if you want to, this is an event to HQ, if you're in the Bay Area also be streaming this on Twitch.

A

There's a couple other things I wanted to talk about, while I have some some people online, because I see will is there I've been thinking and Christy and I have been thinking. This is actually her idea, but of going to a conference or two, and so here's something I've seen.

A

People on Twitch do other people that are like evangelists for Microsoft or whatever they'll they'll go to a conference and they'll line up some guests and they'll and they'll like rent a little room in the hotel and they'll have quick interviews with people like in the industry. So I was thinking about going to some of these AI conferences for a couple different reasons, one because we never go to these spaces I'm not going there to learn about machine learning.

A

Necessarily I would be there to to get a feel for what the field is like out there and what people are saying and basically to go network and meet people and see if there's anyone, anyone technically that's presenting anything that overlaps with, like the biological idea, ideas that we are are promoting and investigating at all and if so, have them on the show like bring them on to twitch and do a live stream while I'm there and talk to people.

A

Would anyone be interested in that because I think that might be fun, but I definitely don't want to don't want to like waste time prepping and going to a conference and doing something like this and putting something live on Twitch. If it's like hey, we just want to see you in front of your computer, okay cool. So at least that's one, that's one! That's that's good enough! Another one is the O'reilly a I conference in San Jose and that's I, think in September they don't have a too much of an. They do have some agenda.

A

No, they did not have speakers here yesterday, all right so now they've got speakers.

A

A

Facebook Berkeley I thought I knew that guy but I didn't anyway it would be cool if I can get other people. If you, if you look through, if you recognize any of these people on here and you think oh they're doing something related to Numenta they might, they might be interested in talking to us at a conference.

A

Let me know and I'll hit them up and I'll say: hey I've got a twitch channel I talk about AI and neuroscience, and do you want to come talk about AI neuroscience on the twitch show and life, and so I will do that. So let me pop this into chat. See here are the O'reilly speakers and the other one was called: rework and rework is like a I'd call: it half half executive, half tech. So it's one of those conferences. It's not a full-on tech conference.

A

It's got like a big half of its a management track, an executive track and they're talking about applying AI and the strategies behind building products with AI had all that stuff. So if I did this I'd probably go with our VP of Marketing Kristy Naver and she would like a ten those tracks and I'd go to the tech tracks. We would get a feel for what's going on in the industry. So that's the that's the idea there.

A

Okay, let's go to.