Numenta Live Streams, 18 Jun 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: BHTMS: UI tweaks and simplehtm SP learning

Description

Broadcasted live on Twitch -- Watch live at https://www.twitch.tv/rhyolight_

A

So I have already changed the code just a little bit from what is on master right now. So let's look into this and just in case.

A

Anyone wants to follow along, you can do that. I'm going to go live share should get a copy got it, so you can follow along if you want, if you have Visual Studio this vs code, which is a great ide honestly, all I've done here is I've. I've commented out the other things on this page and then I changed the update rate to 200 milliseconds instead of a hund. So we get this really fast update, because when we're doing the learning stuff, I don't need to do this.

A

Yet, actually, when we get to the learning stuff, it looks it should. It should look more interesting if it's updating faster, so I wanted to update it faster.

B

But we don't have to do that yet I want to get to the.

A

Other the easy things first, hopefully easy things. So, let's look and see what we're gonna do here, show the data point on the screen somewhere. That's that should be easy enough. So let's do this. First am I missing anything! Everybody can hear me. Okay, let me get rid of those everybody hears me. Okay, quit my QuickTime Player yup, okay, cool and thanks to our lack for coming back, I. Think I've seen you a few times on the channel I appreciate anybody who's coming back again, Joey this brand I've recognized several of you guys in chat.

A

Freeman 42 is out there genre bloom thanks everybody. This should be a fun episode, because I don't think it's gonna be too hard to enable the learning testing. It might be a little bit hard. But that's that's down here. That's down here, we'll get to this sort of at the end. I want to do some of the more you I stuff first. So let's do that. So, first, we're just we're just going to show the data point on the screen.

A

This should be pretty easy for, let's get the debugger up and running and I think I'm gonna, move it down on the bottom I'm on and and maybe make this a little smaller yeah cuz. That's all we need to see right now.

A

React doesn't like something okay and so I want to just be able to display the data point on the screen somewhere. I.

A

Imagine I can pretty hopefully easily do that because in the code, so let's look at the code on the spatial pooling page I am already getting an in component did update here's where the I know the data is changed right here right here, there's been an automatic data feed update. So if I, if I, if I add that to my state, just the raw data, so so I'll just call this data.

A

That's maybe I'll call it data point for current data. Let's call it current data I should be able to I've got these fixed means for later. I should be able to add this to the state right here current data, and then it should be this dot props, and that should be.

A

That should have a value and a time component and I think we'll be able to get that and and eat really easily display it like this, like I, might just put it right under here, we'll just put it in an h4 I, don't know, and then we'll say this dot state dot current data and and this isn't going to be the way it ends up. But objects are not valid as a react child no, oh I have to actually set like a value and a time. So so let's do that.

A

So so, let's call it current current. Well, current data value varied at a time. Oops. It's fine with me I'm flying blind here, because I don't have David David Truman duck D duck. Oh three he's usually in chat helping me with all the react stuff, but that's okay, I can handle this. I can handle this all right. So this is going to be current data value, and this will just be dot value and make this one time time now down here, I'm going to show them both will have current data time.

A

And let's just take this whole thing copy. It.

A

Values, so at this time I don't know if this is gonna display well in in line as a string.

A

Objects are not valued well, this shouldn't be an object. I didn't think it was an object, but let's go see what it is. Let's, let's, let's set a breakpoint well.

B

That's just debug right here.

A

And we'll see exactly what's going on here: okay, sooo, good, bye console, so this stuff props data is a time. Oh, it's probably just wants a string instead of a date. That's that's! No big deal um so um do I have a moment. I think I usually use moment. I do have a moment. Okay, so I can just wrap it in a moment and then send it off to string and I.

A

Just got distracted, foe yeah just comment this out: okay, so here where we set the time we're just going to set it as a string, so I think I could just say moment this.

A

And now I have to remember how to do it. How do I so I gotta? Look this up moment jas dave22 string.

A

This is probably super easy I always forget. It's just dot format, okay, so dot format. Let's see if that works,.

A

And you take the debugger out, because I bet it'll work. Nope a moment is how I have to import it. Of course,.

A

Import moment from moment, I think that's all I need. Hopefully there we go alright.

B

So there's our data.

A

Point we can format it later. I just want to make sure everyone knows actually, let's format a little better than that.

A

What does a good way to format it.

A

Parts get manipulate display. This is what we want display yeah. Let's use this format date string and we can just use that for now, we'll format this and I want I, don't want to use double quotes. I don't like double quotes in JavaScript I, just don't I, just they're, not I. Just don't like this is fine. This is fine. Okay, hello, hello, hey David! You are here, yeah, yeah, yeah, here's, the live stream link. I was just saying how I'm I'm flying blind here without is.

B

A

React world here you go I, don't think. I have to do anything tricky with react today. Hopefully all I'm doing right now is just making sure that the data point is yeah. So that's a that's a more readable I. Don't.

B

A

A 1 that's kind of weird, but I, don't really care so and we're just getting a value. That's I think between negative I, don't even know what the value is, what I decided is going to be. It doesn't really matter.

A

B

A

Let's make that a little less intrusive it doesn't need to be in a header does not need to be in a header so down here in the in the layout. Let's, let's take it out of this and we'll just put it in a div.

A

Oh just a cop pulled somebody over do not want all right, so this would just be a little date string narrative. So that's the date. That's the data that we're showing here. What is my console? Is it a favicon? Oh, it's just a reactive issue: I'm going to put my headphones on.

A

So I'm, not distracted by sounds except for the ones I want to be distracted by okay and I'll. Also open up discord.

A

The chat and discord in each team chat livestream Channel, so we are hooked up there if anybody wants to join their otherwise here we are okay, so one one things done: I, don't that's great check it out. Look. It was in progress for about five minutes, and now it's done next.

B

A

Gonna add some labels. So let's do this, the connection threshold, distribution center and spread these- and these are these things right here. So let's put them just in a little bit of a better format: I I, just I, just tossed them in I, didn't really think at all about about them. So let's put them in a in an actual structure. Let's put them in a list for now we'll put them in a list.

A

And we'll say connection threshold, and that's our up of this. This doesn't have to be beautiful, we'll do beautiful later now, it's just a functional connection, distribution.

A

Hello mark brown, the king, is in people still in him on you missed a good research meeting. I thought thought it was a good good one, but it'll be on the archive soon enough.

A

What was this one distribution center foot center of this pre-owned fusion.

B

A

Right, so that should look better all right. Okay, so as we change this, there's not gonna change anything yet back in the US. Welcome back glad you had safe travels by the way mark I changed again.

B

A

Changed the date for a discussion about fully connected networks, because this Friday I'm gonna be at the applied AI, Summit and I completely forgot about that. So it's gonna be next Friday I know I keep pushing it up, but I actually will not be able to even do it because I'll I'll be attending conference talks about applied AI which I'm honestly not looking forward to because I'm not interested in supplied deep learning, I'm interested in it intelligence.

A

Let's change this: let's change it, I, don't quite like the way this looks. So, let's move these to be first or else yeah like I said it's. Not it's not beautiful, but it's functional, I know. I would put this in you CSS to do this, but a bench eventually, but for now alright.

B

This is gonna, it's gonna look, okay right! They.

A

Might oh they're all gonna be different anyway, that's not even worth doing nevermind well we'll get to proper alignment at some point in the future. It doesn't matter it doesn't matter so I'm gonna put. The permanence is back right now, because I took these out because uh I changed the update rate to be really fast and it slowed the page down, but with just a one Hertz update rate, it shouldn't slow the page down too much. What is this big gap here? Oh yeah yeah I need to change the size of the potential pools.

A

So when I change this, this will change that. Okay and that and it will, it will also should change. So that's one of the things I want to do. I want to make sure when I change the connection threshold here it should also update Oh, see it is it's updating, I, think it's updating spatial boiler yeah. So, if I make this point for yeah.

B

So it's definitely so.

A

We're definitely getting a set point for it yeah, so.

B

We're definitely getting.

A

A reread I don't think we should be reinitializing the spatial puller when this happens, but we are so those those are the things that I want to go through when we change this that definitely read initializes a spatial cooler because it creates all-new permanence --is. When we change the threshold, we shouldn't initially initialize the spatial polar. That is something that we can update on the fly the threshold.

A

So that may be something that we have to add to the spatial cooler, API and simple HTM, and if so, I'm just going to write it down to do in a moment. Let's see how we're doing it in the code.

A

Okay, so we are looking at right connection threshold and.

B

That was I think.

A

One of our other tasks confirmed changes to SP initialization values changed the mini column, competition, okay, I'm gonna, put this in we're gonna be working on both of these essentially and I've got these I've done the labels, so we label them.

A

All right remember, there are deep learning. Has lessons to teach yeah pulls the big trick? Dl pulls from biology is connected Maps, even with point neurons boy. Powerful. Does some good lessons for HTML. It finally gets back to the age of HTM yep an hour. Don't worry already are we? Are they doing anything that we're not doing I'm, not that anything necessary that we're not doing because.

B

A

Maps too I mean we have connected maps.

A

Okay, so connection threshold changes. So when this, when the connection threshold changes here, I need to go into the permanence --is first of all and make sure that actually I'm going to this connection threshold change is something I need to look at in this page, because this is where we're doing it so right, clean up, SP, initialization, so I think this is a bit we might be doing. You do speech recognition. Oh definitely, not.

A

Definitely not, but I mean well. There. There are Richard. Crowder is working on doing speech, speech, recognition, stuff with temporal.

B

A

You're talking about there's this, the spate massive spatial pattern, recognition that deep learning gives us writer I, won't throw it out mark I'm going there to pay attention. It certainly does do I'm, not discounting it. Deep learning does a lot of amazing things. It's just it's! It's not what I'm interested in so I'm going there with an open mind to try and see what the current state of the art is and how deep learning is being applied today.

A

So I will report back on this stretch, channel about what I find here for sure, okay, so so this is, if there's, if there's new data, if there's not an SP, we create a new SP, look for bridges to HTM from deep learning. Absolutely definitely I will.

A

So. This is new data point. What we want is the down here, if there's a new connection threshold. Okay, so we're reinitializing, no matter what, but what I really want to do is say we're not. We don't want to reinitialize that there's a new connection threshold, all right! We because that's not something we need to recreate permanence is for so I'm going to create another L stiff and- and here we'll put that let's get rid of this.

A

Come I like word my line, wrap on here so so now. I need a way to update that connection threshold in the SP I'm, not sure I have a good way to do that. So, let's, let's look in the simple, HTM, real, quick and see how we're storing that connection threshold and I could just you know, just sneak a peek in and go change it and then write a note to when I get to the simple HTM to update the API, so that we can do that the right way.

A

So we should have something called a connection threshold. So what I'll do is simply just go change. It I'll just go change it I'll Joe I'll, sneak right into ops, because I should be able to go spi, ops, connection threshold and change it. That's the sneaky way to do it and it's not what I want to do so I'm going to write. So so, let's, let's make a note, because I have a project for simple HTML there.

A

I've I've got a planning thing for it: spatial pooling and I'm going to add a to-do item to add API to change connection threshold.

A

All right so we'll add that that's going to be something we're going to do today, but I'm not going to do it yet but I'm. What I'm going to do right now is a sneaky way. The sneaky way of doing it, which is right, yeah, sneaky way of doing it, is going to be just to jump in there and and and update the connection threshold.

A

So I should be able to say SP doc, whoops whoops whoops holy cow, it Otto commented connection threshold equals the state, that's connection threshold, so hopefully this will reinitialize the SP but change the way it's calculating its connections without reinitializing it. So, let's see if that worked. I should be able to see this. So if I change the connection threshold, let's change it to dot one. That means they're all going to be super all connected.

A

None connected.

A

Okay, what am I.

C

B

Yeah yeah, that's that's.

A

Right so lots of connections, that's right versus very few connections and I, don't think so. If I select this mini column, so we're out in the same mini column, here I put it to five.

A

Not oh five, five, so the potential pool should not change. Okay, so keep an eye on the potential pool. It doesn't change, that's good, so I changed it from like five to 0.15. The potential pool is the same. That means it was not reinitialized the permanence azar the same. It's only the it's. Only the calculation of whether the permanence value is connected or not. It represents a connection or not that's different, so I think that's right. I think.

B

A

We want so I think I think that's good and this one is updating in the same way, the permanent scissors part are still staying the same because when you initialize a special puller, you get a whole new set of preferences. All right, so that's I, think done.

A

Back to here so confirm changes s be initialized, values, change, the mini column, competition.

A

It's not what I was doing yes and that when I change the connection threshold, it does not the other ones. Definitely do so. If I looks, let me go back here and and if I change the center of the distribution I get new permanence --is or do I, not yeah I. Do because that reestablishes, though all of the weights. So that's right if I change the connection distribution, this is sort of the spread.

A

It also reestablishes potential pools and all the permanence --is I could that doesn't matter I mean if you're going to create recreate all the weights. You might as well recreate all the central pools too so I think it's fine to recreate them all. So.

B

This looks fine to me, I.

A

Think this is I'm, think I'm happy with that. Let's get rid of you all right.

B

So that's done. I'm.

A

Flying through these hoist selected mini column up onto the page, State, okay, so what I want to do here is you'll notice. Let me make this a little smaller you'll notice that when I select the mini column here, it does not get reflected in the bottom. One I want the same selected mini column to be displayed on all of these every diagram. So so we want this. Whatever is keeping track the state of that to be up at the page level. Right now, it's handled at the in the diagram levels like diagram state I.

A

Would you wish to clear your backlog that fast just create really easy issues?

A

It makes you look really efficient.

A

Hard things are hard easy things are easy. Okay, so to do this, to hoist this up, let's look and see where how we're doing this, because right now, I do not have a selected mini column state here, but what I want basically is selected any column.

A

B

A

That doesn't exist yet so in the permanence is, let's start with permanence is here. There is a selected well selected any column, and it's.

B

A

Then and then we change that based on a mouse handler, so the mouse handler is a d3 I think is a d3 I'm, not sure, let's go I think maybe it's not!

A

No! It's not it's not so.

B

A

So what we could do is essentially from this point from this mouse move. I. Think I've done this on one of the other diagrams are working on. Is we want to emit an updated state? Okay, so I've got examples of doing this. David.

B

Don't help me out yet I.

A

Can do this I think because I have an example of doing this in the I think it encoding numbers right so there's when I create the diagrams I can pass it there. It is like on update so I'm gonna do that and that will, if I call an update with the right value, we'll see how we're doing this. So this is simple: scalar encoder. So let's go look at simple and scalar encoder and on update as a function, we're gonna call with the value yeah.

A

So we're calling on update with the value and how does it know the only thing I'm confused about is: how does it know? That's the right value. Don't answer that.

A

This is just getting a value, so I, don't necessarily know what state is being set, but I could I could certainly say explicitly what it is so we'll try it we'll go along with this and we'll figure it out so for the permanence. Is we're going to do this here? The potential pool I, don't think it has a selected mini column. Does it potential pool yeah.

B

A

We need to do this for all three of these all three days. So, let's start with this one, we have our on update value.

A

And I'm, not oh I, put that on the wrong one, but they should both be the same and I might need to add selectively any column as a key to this I'm, not certain yet, but in potential pools. Let's go there. First, that's the first one we made.

A

Now we went to call right here instead of calling this update. I think we just need to call this dot prop stop on update with the selected mini column, and this is now we're not gonna. Have this state here anymore?

A

Oh, wait before I. Do this I know that here we go, that's where the value came from: okay, okay, I'm sure I'm starting to remember this. Now, since we named this value, that's how it knows to set that value, so I need to make sure that I've got I'm, giving it a property. And then that's that's what I'm, using here? Okay, so so I need to go back to spatial, polar pooling and we're.

B

Gonna give it a.

A

Property called selected mini column, that's going to be this dot state, dot, selected mini column and this I'm going to call selected mini column and that's what we're going to set okay so now in potential pool back to here, I'm a react pro yeah thanks man thanks for the encouragement okay, so this is going to be we're just going to have this local and on update selected mini-com. I could of course do this in line, but I think it's better to be explicit about what it is or updating.

A

So I know that when I call update, I'm, I'm, updating selected mini column, now anytime, I have I'm using selected mini column now I want to I want to remove this because it's going to be coming from the properties. So where else are we getting this? So this selective mini column? Is this dot props dot selected mini column? That may be the only the only place that we need. This is going to be this stuff. Props, whoops, stop props I've selected mini column, and this stop props. That's like okay, all right!

A

Let's see what broke see, what the broker sell, that adder is not a function. That's an interesting! uh Wasn't what I expected just refresh to make sure that that's okay! So, let's see, why would that be the case, so I'm calling treat cells with the sell? And it's not really a cell, because the data? Okay, let's debug, right here and see why we don't have the stuff props, that selected mini column and our.

C

Are you defaulting it on a value.

A

Undefined- that's probably MA, so if I just make it instead of defaulting to undefined default to zero, which is what I did original.

C

A

There we go. Thank you.

B

A

Okay, so alright, so this is not going to update because the other diagrams are still using diagram state. But now I could basically just do the same thing to permanence a--'s, where we did that we did here so we'll give it first of all a.

A

Just like this selected mini call them an on update the exact same lines and I might as well do it here, I'll, add it selected mini column and on update I, don't think I have it on update on either one of those. This is the only thing that these diagrams are updating in the page state is, is selected, mini column, otherwise they're all just doing local visualizations, okay.

B

So this is gonna break.

A

I, don't think this is gonna work, yet so I'm gonna take it off first and we'll work on permanence is real quick. This should be. This should be easy. It's basically of doing the same thing. We're gonna do Const.

A

Selected mini column and then everywhere else or we're using a selected main column. First, we're gonna get rid of this anywhere else. We're using it we're gonna, say this dot props. What's.

C

Is it Oh all update or update.

A

I have been calling it on update.

A

C

B

The others I was calling.

A

It on update, which.

B

Is I think did.

A

I not put that down here, yeah you're right. Thank you. Fun, update on update that was the previous function. I'm. Not that was a local function, update which is gonna, be called by react. Now. That's.

C

Also, on props say to be like this props.

A

You mean in the in the page.

C

Yeah, when you called it on the on the handle Mouse, where did this stuff prop stuff? Oh right,.

A

Thank you, yeah yeah got it yeah this stuff, props right; okay, next, David! Okay, that that's good! This one needs to be updated to.

A

Props dot any other uses.

A

Yeah the same thing here stop and then one more.

A

A

And another, you know what I should probably well I'll clean up later drop, stop.

A

All right, they're, all over the place, this one uses selected mini column, a lot. Okay, let's see how this works now, dopant errors. So now.

B

A

See yes, it worked. I can already tell so as we hover over this one. Our our selected state is now sort of cascades throughout this one's, not working, though, because I haven't hooked it up. So let's do that one next okay same deal and in Maycomb competition, we're gonna go, find select it any column for this anytime.

A

We're using that, let's just let's actually do it the easy way, this time and say anytime, we're saying this dot selected, mini column or replace it with this crops, dot selected mini column five times and then in the update or in the mouse move. Of course, this is wrong, but we can fix this Const select with mini-com props dot, on update selected minute.

A

ha Awesome. Ok! So that's exactly what I wanted so now, no matter what diagram we're dealing with it, we can look at all of them and they're all displaying the same mini column data. This is a little odd because I've I've switched out. Ike I put the mini column on the left here. Instead of the right, I should probably move that so that they're all the same. Let's do that that wasn't on my list of things to do, but that's ok.

A

This should be easy because it's a only it's in the mini column, competition and it's just a transpose transpose where, where are we transposing? Oh, that's for me, translate whatever ok, so so we're not going to do this on the competition anymore, we're going to do it on the input.

A

A

Input, okay, so so we're gonna do that here on this G group, so that should just swap them out there OOP, okay, so I need to do. Okay, I know, I know it. Did you hold on I need to grab input space, not input, so I actually put this in the wrong function, because I think there's an input space yeah yeah. So this is the wrong one. I wanted to put it here right.

A

Is that still that's the group that contains all if I want to move everything in that group there we go all right so now.

B

A

These sort of better synced and using the same dimensions and cool I'm, pretty happy with that. Let's, let's commit this: let's commit what we have so far: quick, quick review, I'll use of es code to do this, quick review of the.

A

So I'm gonna look at the page level, first I think in the spatial pooling we have hoisted some state for the mini column and we're now keeping track of the time and date value separately. What was that I'm? Sorry I just got a notification on my Stratocaster has arrived, I want to run home, I, want it in the street right now and run home, but I'm, not that's how dedicated I am I'm waiting for a text for my wife because she knows how excited I am okay,.

A

Right we're just setting the state we're used, we're adding the data point into the state and formatting it so that we can see it and then we are only reinitializing okay. So let me put a note here.

A

Let me put a note here. Fix me, simple, HTM ASPI needs to manage this state better or, let's say needs to provide API to change this.

A

That's easy we'll do that when we go to simply Jim and will update to a new version, because I don't want to keep jumping back and forth as I have to release it every time, so we're gonna wait till you have to get over there and do learning and we'll do it with learning.

A

That's just where we're putting the date and time and data point, and this is the updates for the selected mini column. This is also the same thing. This is the names of the parameters, and this is the mini qualitative that looks all good to me. Potential pools removed. This is all about wasting state. Okay, so now I'm going to look in here and see if there's any place, I can refactor not in that function. This function.

A

No. This function.

A

Okay, but then I think it was in.

B

A

These there is some places. I could pull it out. This one already has pulled into a value. This one I can pull into a value, so so, for example, I can take.

A

Const selected column.

A

So then, I'll just use this and then down here I can get rid of the doesn't make a little bit simpler. At least so I've got a local copy. It's not a copy. Local reference same same thing here: I can do generally I try to if, if I'm referring, if I'm, referring to like this dot, XY Z, whatever more than once within a function, I'll pull it out into the variable.

A

If I don't it's just because I'm being lazy, because it's I think the right thing to do and I am lazy, so it probably happens more than I like to admit this. One does not need to be pulled up that one's good, okay, so many column, competition. It's the last one.

A

That's our translation that we moved.

A

This is only one reference. This is two references, so do this.

A

Selected bitty column.

A

And the star here many called and this one's fine- that one's fine, okay, okay, so real, quick I, just want to make sure that it still looks good.

A

What I break mini-com competition, 1:32 I, think I just spelled something right selected. Oh.

B

Yeah I didn't equals.

A

This stock drops dot selected.

A

A

Let's work in let's work in this is this is working.

A

Okay, all right, let's commit this good, so I fixed a bunch of stuff here. What exactly did I did? I do so I'll list it all showing data point on the screen.

A

I'm just gonna put set several small UI changes.

A

Hoisted selected any column to page state.

A

Confirm changes just yeah only re in it SP when necessary. Sorry.

A

Labels pandas or SP, perhaps.

A

I thought that was a train show the data point on the screen. Show data point works for me now and I'm just going to push it's a master.

A

B

That's what I thought I.

A

Thought it's just my music or is this an actual train could not tell okay your ID uses to do as a navigation target. My probably does too I just don't know how to do it.

A

Okay, so we got all the easy stuff done now, it's time to do the harder thing, which is the learning part, so that was pretty good, so this is gonna, be okay, so we're gonna go to simple HTML we're gonna, we're gonna, add learning there and, like I, said the.

B

A

Part I think is going to be simple. It's the tests that are gonna be hard. It's always.

B

A

Art so so now we're going to go to yeah, sorry that was my github.

A

So now we're gonna go to simple, HTM and I think I have to reshare. My like I have to live share this one too I don't know if I could do two at a time, but I'm gonna try, let's try and live share, simple HTM. Let's try.

C

A

Yeah some of the some IDs automatically index like fix being, and to do for you this one probably does too it's just I, don't care enough right! Okay, start it on.

A

That's a good point.

A

I'll get to that stage, I'm still sort of tuning Visual Studio code.

B

A

I'm sorry guys I'm gonna eat my sandwich I'll. Let you guys stare stare at me while I eat my sandwich for a moment, cuz I'm hungry and it's not fun, to listen to me, try and talk and eat. At the same time,.

A

It's a really beautiful day here in Watsonville, California.

A

Excuse me really.

B

Whenever I think Jordan I think, like.

A

Blazing hot but I guess they do they have seasons there.

A

Jordan is really close to Israel right now: Zev seasons, so.

B

I guess that's not like Quade or there's quite a ways away from Saudi.

A

Arabia to quit.

A

B

A

I used to know a lot about that region of the world.

A

That was a long time ago.

A

B

You know more than I ever did I.

B

A

Knew about like what countries had terrorist groups operating within them, because that was my job snowing terrorist organizations.

A

I forgotten them all now intentionally.

A

What you've walked in the footsteps of Jesus, like a pilgrimage.

A

B

As an area rich in religious history,.

A

I wish I traveled more I. I did for a little while not a lot once the kids got out of high school I, probably travel more.

A

My wife just texted me and confirmed that my guitar has arrived.

A

B

Interesting analogy: I.

A

A

It can be overwhelming.

A

If the offender it's a it's an electric Fender Strat performer brand-new, it is at my house if I haven't opened it yet.

A

But when I get home I'm going to stream it, you got a P bass.

A

You should send me a picture. I love, pictures of guitars.

A

B

A

Have a bass someday.

A

What's a jazz neck is.

B

A

Fretless, do you know winamp I lived on I used to win amp exclusively for like five years in the late 90s early thousand 2000.

A

Olympic series one I'm gonna, look that up I know what a P bass looks like ooh.

A

That's pretty! Oh, my god. What I'm looking at here! That's a pretty bass.

B

Excuse me that.

A

Is a really nice.

A

There's a whole program called Rocksmith I, think it's for guitar and bass, or bass and playstation or xbox or pc. Listen right here looks like yours: it's really pretty I love, unique guitar shapes and I like the I like fat, bodied, electric guitars too. So that's nice. It's very nice.

A

All right enough, guitar talk, let's, let's do this thing that we're planning on doing here.

A

Okay, all right so we're actually gonna go to simple HTM and the last things the last tests that we were writing were about permanence --is. Let's see, let's see it wonder if I have a good outline for each but does not I got one.

A

So all my SP tests are in the same file, and so, let's maybe split this up a little, maybe take the mini column, competition. One out.

A

This is all this is about overlaps about permanence --is permanence is so so, let's, let's create a few files here.

A

Let's call this SP potential pool tests Jess, actually that's the one that will will keep so, let's say, firm firm and it's such an ass and create another one. Oops called ode, I put it in the wrong place. This should be here.

A

Yeah, okay, so another file will call SP.

A

And a sit mini column, petition tests.

A

Okay and we'll rename this one to.

A

Sp potential pool tests and then just scrape out what we, what we need here so upon SP initialization, for when creating let's just do this copy and paste will take everything out of here, except for potential pools. Connective tissue pools are the same. That's good when establishing initial premises so from here on out we're gonna. Take all this out. Performances I think it's. We can take the whole thing it's all of us out.

A

We do need to end that. That's all we need. Oh, my God. Look, oh look at mark space or quick nice.

A

Can I show the whole picture so.

A

The Jazz Bass is a narrower neck. Oh that's interesting, I didn't know that was a thing. Cool I, like the shape the P base shape, is nice too. Okay,.

A

Back to the work, so that's the potential pool tests. We we don't need d3 anymore for this one that remember, we were using that I think for a range. Let's go here, we'll paste everything and then take out all of the overlap compass to the perf everything but permanency. So I take this out.

A

Okay, permanent stuff will stay.

A

That this goes so starting here and pick this out.

A

Okay, so this has only got permanence tests in account and then this mini called competition tests. I don't have anything.

A

Take everything out, I, don't know that I'm gonna do any I'm, not.

B

Sure what I'm gonna do here.

A

Yet this one needs d3 that one's all good, so I should be able to run tests now and it should pass which do. I have yeah mocha all.

A

An exception has occurred message directory test, like I did for the command line, Oh.

A

Npm run test.

A

Okay, there's our permanence --is. That looks like that's working the.

B

A

Run, there's our potential pools, there's our coders yep okay. So now at least we have and I won't run it from I, don't need to run it from yes code. So what do we want to test here? We need to test that, first of all,.

A

I'm, not gonna, let's say describe a pull, pull and let's say during during spatial pulling mm-hmm I know this is a little bit of cruft, but it just it reads better. It reads better when we write the tests during the spatial pooling we want to I.

A

Guess it's during during the mini column, competition! What's called that during. Maybe we should write this in the mini comp competition and because that's when learning occurs so so I'm not sure. Oh, these are mini column. Competition test didn't I. Have some I feel like I skipped some tests, I meant I messed something up.

A

I had many comp competition tests that I did not.

A

Yeah yeah, okay yeah, so during mini-comic competition, I already have tests for this and I and and overlaps. Those need to be here too. So those need to be here. I almost took too much out when establishing.

B

The initial permanence.

A

A

A

B

This okay, so I need to meet I'm gonna run this again.

A

And make sure that we're, but.

B

A

Tests are properly run fifty one tests last time. It should be more than that, a lot more than that there there that's what I want they put. Saturations mini counts as most overlapped way, so I almost screwed up and deleted some of the most important tests, which would have been bad 71 tests.

A

Okay, so during the mini column, competition now we're gonna add a section. What's that was just clean up. So let's commit that just because so.

B

A

Remove this, and it should recognize that this was moved.

A

And he finally falcon gonna, run test for sure didn't screw anything up. 71 tests, yeah that was a good catch I think I would have caught it um when I reviewed the code. Hopefully.

A

A

All right so get commit.

A

Extracting tests into individual files by subject.

A

ah Just leave it I, don't push it I, don't need it yet.

A

Okay, so we've got our test properly overlap.

A

What we need to do is maybe I'm gonna see, if maybe there's a place in here. We can already do this.

A

This is just checking that all of the winners have higher overlaps than all the losers, and what we'll do is now. We want to make sure that the mini columns are learning and then we're gonna make a test to make sure you turn learning off.

A

B

A

It when it wasn't doing anything well, who knows I might I might never have caught it, because those tests just wouldn't be run, but it could, it could be, could have been yeah long time in the future when some, when I changed code that broke one of those tests that doesn't run anymore. That's when and it would have been ugly like I said it wouldn't I believe so.

A

I'm probably gonna have to write a new test, whatever what I wanted when I wanted to test here is I'm gonna write this down. This is a good time to write this down, so so the functionality we're talking about espy learning, I'm gonna, get to straight my head, but I'm going to I'm gonna work on here. What I want to do is, let us say, on new data and instead of data, let's say in coding input but I've been calling it on new and put.

A

Winner MCS winner MCS, for let's.

A

For winner MCS.

A

A

Over laughing overlapping input.

A

A

A

That's one test.

A

A

Connections, not overlapping and put our let's say: incremented incremented, incremented and decremented all right. So that's a good place to start. That makes sense. That's the your method of writing test versus good one I like working with a good test, suite yeah. That's the only way to do real, good, real, especially.

B

A

Code like, if you don't really understanding, if you don't really understand what you're doing the best way to figure out, what you really need to do is write a test first and then you can define exactly what it is you're trying to have the code accomplish, which is what I'm doing here for every winter, in winter, MIDI column after we see input, we need to look and see that any connections that are overlapping with the input, the permanence values get incremented and and I think we can test that and the opposite really gotta sit.

A

You know what I mean, okay, so how we're gonna write this? That's the question.

A

So I have a setup here. That's during mini column, competition I think I could probably add this in the during mini column, competition I. What I don't have yet are like permanence, increment and decrement values, but we all we can add that.

A

A

They don't teach you test-first development in school at least I didn't teach me I, I went I, took a pay cut to work at a software shop that did this type of development, because I knew that it was something that I needed to learn early in my career, and it was worth it. It was absolutely worth it so describe our with with X amount of winning mini column. So I don't care how many we many columns there are, but I could.

B

A

I'm just trying to think I'm just thinking in my head, I could I could write my test within the context of this. Many a winter count and input saturation, but it's gonna make the test run. I mean it's gonna make the test suite run pretty long, but I mean it's more resilient. It's more resilient and redundant for the tests to test all these different conditions, although it's not necessarily necessary, necessarily necessary.

A

Yeah I, don't do see, that's the thing. I never do see.

A

All right so so I think we could probably write a test here right sort of at this level, and we know it's going to get run. 1 2, 3 4 times, 1, 2, 3, 4 16 times, that's the thing it'll be run 16 times and that's not exactly the best thing, but at least it'll cover all of the different possibilities.

A

So when I hear a C joke, tell us a joke or not do I want to hear a C joke. Let me see: do I have a sound for that.

A

What's this supposed to do.

A

Open the pod bay doors, please Hal I'm, never gonna use that sound I, don't know why I have it.

A

Okay, so it I'm gonna.

B

A

Put another describe so on assuming we're gonna, we don't have any new input. Do we? How are we yeah? We have input, count yeah, we're making random input here so I'm gonna describe here.

A

On new and put just like I wrote on the board here for winter mini columns.

A

Describe or winter any columns two strings enter a bar. The first one says give me a beer, Scott dock.

A

That is a see, does strings poorly yeah, it's one of the C's super fast I I know you. Could you could be off track up? Let's focus on this mini-comic competition thanks.

B

A

The joke Falco keep moving on for winter, mini columns and and I sort of want to also create one for loser. Many columns.

A

Okay and what we'll do for loser mini columns is basically the opposite right, ensure, let's just do let's say.

A

Permanence a--'s are never changed its cuz. That's the that's the way it works in HTM for us, so we'll get to that. So.

B

For winter mini columns.

A

Connections, overlapping and put our incremented by permanence Inc.

A

All right and I'm going to copy and paste this and we're gonna do the same thing for losers. Connections over let Oh connections, not overlapping connections, not overlapping input are decremented by permanence, Dec.

A

I'm just using permanent sync & Dec, because that's what we use in new pic that we could have better names for those. Maybe.

A

No worries about cars. You can tell jokes, it doesn't bother me. Okay,.

A

So first thing I want to do is create some input that overlaps the spatial pullers many columns.

A

So I could do this randomly or I could just create an input like that has maybe just one bit on right. Maybe I'll I could create an input with one bit or half the bits or something, but with a range that I know that's explicit and that I can test. So maybe I can test. I could turn half the bits on go halfway through the structure, know that and as a condition of the test.

A

After it's seen that so now, I'm going to have to like get a copy of all the permanence values for all of the many columns before it sees the input and and then.

A

Give it the input and go back through the mini columns before each one, each one of its Maps sort of to the input make sure that the only the connections that overlap with the inputs were incremented and they were recommended by a specific value. Right, I think that's what I have to do so. So what I'll do is I'll create input like this, but I won't. Do it randomly then we're going to do this so I get the winners.

A

This way and I know the indices of the winners and I'll go manually into each one of those. If the permanence isn't test that they have changed by the decrement or increment. Whatever excuse me, so we're gonna write this as if it worked the way we wanted it to work. So I'm gonna have a permanence increment.

A

Point 1, permanence, decrement, 0.05 and and I want to make it different just because I'm I want to test it. I want to make sure that there, because I'm gonna test each one and if they're the same one it could be, it could be using the wrong value. The other thing I definitely want to test, is I.

A

Just thought of it and then I forgot it textures are layer recommended by firmness and chromatic.

A

We should also test.

A

Ok, the reverse of this. Oh I already did the loser. Many columns permanent Cesare, never changed. Ok, so I think I've got all my bases covered here when I look through I'm gonna give.

B

A

I'm gonna I'm gonna, look at all of the winter mini columns, look at every one of the for each mini column for each connection, if it's overlapping check the permanence and make sure that it was specifically changed. Otherwise, not and I can honestly do this in the same function.

A

I don't need to loop twice over. This I can do that. I can loop through all the connections and do this all. At the same time,.

A

B

We'll try and do that. Ok, but first, let's do.

A

Let's add the input Wow or did I just do I not copy it.

A

Okay, so I want I want this.

A

So there we have our winners, and this is not going to be random.

A

We're not using an input saturation.

A

We're gonna, let's set it initially to one and we'll just say well, I could do this in a condition. I can say if input count is less than excuse me.

A

Excuse me X. If X is less than input count /, we will push a 1, otherwise 0, so the first half of the input will be 1. The second half will be 0 roughly and I'll just use the same condition to check and make sure that they were committed before I input. This before I run the competition I need to get permanence, --is, okay and we'll say original perms.

A

Okay, afterwards we're gonna, say new herbs.

A

So this is basically it so this is what we're going to compare the original terms in the new perm. So we're going to get the original perms run a line, get the new perms and then this is basically going to run through and say for.

B

The winner, but there are many columns.

A

We could we could voice this up here. Perhaps it's sort of more fitting there?

A

Okay, so connections overlapping. The input are incremented by so for the for the winners so we'll go.

A

Perhaps we'll do this winners for each winner, I think this has an index and a permanence value. I think.

A

I'm pretty sure this is, it has an index and a permanence value and I think I can.

B

A

I put this inside of this loop I think I can't yeah.

B

I'm pretty sure I can.

A

So for each winner.

A

Now I'm going to say each winner is a mini column, so I'm going to go, get the the perm that let's say.

A

When I call this all perms original, all perms from you know all original perms and all new perms, all new curves, so now original firm for this mini call. Virginal perms for this mini column, is all original perms winner index I think that should be the mini column index. So this should be the permanence of the mini column index before it was changed. New perms equals all new perms winner index.

A

Let's see I'm come I'm, not sure this is right here, I'm going to log this.

A

And then we're gonna run I want to figure out what was wrong with my run here, I want to run the current file. Why did this work last time?

A

No such directory test, MOCA apps.

A

A

I don't understand: why did that I think.

B

A

Problem last time and I just didn't deal with it, so I can always I can always run this.

A

Of course it's gonna take too long to run oh, but that was my. Those are my winners, yeah, okay, so that gives me the index and the and the complete overlap.

B

Which? I don't really.

A

Need for this or do I do I want the over there yeah.

B

Yeah yeah, so I want.

A

To go through all of those all of that many columns connections to input space. If it's in the overlap, then I need to make sure that it's incremented, if it's not I, need to make sure it's decremented. Okay, so now so I am gonna use the overlap years. This is kind of perfect okay.

A

So let's get this overlap. Fools winner overlap.

A

Let's, let's be explicit index equals winter index, and so here this.

A

Okay, get out of the way here so for winter, many columns for each winner, we've got the overlap and index connection, it's overlapping it, but our incremented.

A

A

Again, I might end up doing this outside of this yeah cuz I don't need to be in there. So I have the overlap. I've got all the new firms on the old firms so for each so I'm going to go through the entire input space right, so the full input so input dot for each input bit and input index.

A

And put bit and put index so so this is where we're going to do the comparison. Here's where we're going to do the the actual testing oops.

A

All right so connections overlapping. They are incremented by permanence, increment all right. So if I'm not sure, if this test is going to work properly, well we're going to find out, we might need to restructure it a little bit because I don't usually write my it's statements like this, but we'll find out.

A

So if input index is in the overlap- and it's not contains its I always forget, array contains a JavaScript. Javascript already contains it's. Something includes that's right. Okay, if includes so. If our overlaps include this index, then it should be incremented.

A

A

B

Here I already know the.

A

Perm so let's say this is a constant I. Don't know why I'm doing this, these are constants I just have the my brain just learned to use let, instead of afar and now I'm just unconscious everywhere.

A

Original perm is original perms.

A

A

Okay and new perm equals new firms and put in Dex and put index alright.

A

A

There's an overlap that means that the new perm assert. Here's where you get to the assertion assert equal I want to assert that I expect the new perm to be the original.

B

A

Plus, oh dude, what did I say? What did we increment 0.01 I'm gonna pull this up? I'm gonna hoist this up so Const permanence increment equals zero point. One firm decrement equals 0.05 and then we're going to use that in our test P. I okay.

A

So now in our test, that's what we're going to say, plus the permanence increment and there's a certain of it. So winner mini column at I like to put as much detail as I can in the error messages where, because it certainly helps with debugging winner mini column,.

B

A

Mc index was not fermented.

B

A

A

A

All right, let's put overlapping overlapping and put index otherwise.

A

So this is, this is basically like. This is where I should probably just write this in the same place. Connections overlapping imprints are incremented by permanence.

A

All right, I'm, just gonna, say, connections.

A

Say say: permanence is: are properly updated when learning is on, where we'll just leave it at that, because the assertions tell the story that we're trying to tell I don't have to describe it that much because else I'm basically going to do the opposite of that. If it doesn't include it, this should be minus permanence, decrement and then say winter Minnie column at was not decremented.

A

Decremented for non-overlapping infant index, all right, I think that test is what I want to test, and it's certainly not going to work. It shouldn't work and I should call this overlaps, because that's what I called it here all right. So, let's watch this fail, hopefully should fail.

A

Yeah, so if it's definitely failing and I didn't get my oh that's wait. What's going on there, permanence is a properly updated when learning is on.

A

A

I think I don't know, what's got, permanence is probably up there and learning spawn. What am I doing around here? I got something screwy.

A

A

So I don't want this to do that every time. So, let's not, let's not. Let's, let's move this. We don't want this to run every time. Let's move this way up here that should work.

A

It should just be one test, a lot of assertions, but one one in statement. That's better! Okay! So now now we'll see the failure, a lot better, I think all right.

A

I think I need to move all these, oh yeah, so this needs to come right. Yeah, that's right! So.

B

There should be a failure now.

A

And we should see it, a failure, messages and purposes are probably updated.

B

And they're not because.

A

We're not doing it. Obviously, another thing we should do is.

A

Nothing exciting learning.

A

Let's just call it learn. True, ok, so we're going to send that into and then we're gonna have an opposite test.

A

Is that where I wanted it yeah.

A

Using the same SP for all these tests and I'm gonna set up a negative test with learning off.

A

Okay, so I will I'll do that here, input saturation when it comes with most overlock ones, described on new input.

A

A

I'm going to add another condition here that says like when learning is off when learning when learning is disabled, we don't want to see what we just tested for. Okay,.

B

So this is gonna mean.

A

We're gonna have to create another mini column or another SP here and we're just going to turn learning off we'll use. Everything else is going to be the same. I'm gonna create a new SP and that's the one that we're going to test here. Learning is false.

A

B

And I really should say.

A

On new input with learning enabled and then we'll and then we'll create, you know we'll make this one on new input when learning disabled. That makes sense that makes sense, snapshot and compare the isolated changed elements. I.

A

Don't know I mean that's what I'm doing right I'm getting the perms before and after and just comparing them, but I but I, but I have to create a new spatial. Pooler I, don't have to I mean I can just turn learning off. You can turn learning on and off. So let's do that instead, instead of instead of what I'm about to do, let's say: SP disabled learning.

A

That should be that's. That's good as a general test tool snapshot to compare to isolated changed elements as a general test tool. Yeah I could do that. I want to I want to be really explicit at first and maybe will end up pulling out some more general testing stuff as we go along I. Don't think this is too too complicated.

A

On and learning yeah so I'm gonna do this same. I'm gonna I'm.

B

A

This up keep using the word hoist a lot but I'm into hoisting. Right now,.

A

So I'm creating input for these next two tests here and we'll just reuse. It.

A

It really doesn't matter whether this is random or not. Does it, it doesn't I mean I. Could this could very easily be random I, don't know why it doesn't matter I'm not specifically. Looking for this is fine. I can do this and just use it use it randomly.

A

So one new book and learning is disabled.

A

I'm, basically gonna run the same thing: I could create a I could certainly get a creative function. That does this. For me,.

A

Yeah I got off by one index problems: bad tests.

A

All right, are you see a problem with one of my testers mark I, don't quite see what you're telling me. Oh.

B

A

Kinds of errors, I'm looking for yeah yeah.

A

So what I should really do? What I really sort of need is a function that.

A

We'll compare the two data for basically, the two permanence is the before and after permanence values of all the many columns given the overlaps. Let me just do this explicitly. First I think that's that's going to be the easiest thing to do for me to understand, and then maybe we will do it explicitly. Next, okay, so we've disabled learning on the SP we get the permanence.

A

Is it doesn't matter what we've seen before we're gonna get the permanence --is and that's what we're going to start from that's the comparison we start from well run the competition get the winners, new premises and now permanence is our will say, not updated when learning is disabled.

A

And let's change this one.

A

I, don't need to save when learning is disabled, because that's part of this, when, when learning is disabled,.

A

I'm learning it's the co-princes are properly updated. I could turn this off. Remove that.

A

So this is just gonna ensure that we are not changing anything but they're the same, because.

A

In our mini column, it's just to say at input index was updated and learning was disabled and this is we're gonna do the same thing, no matter what this assertion is, what we're gonna do for everything. If anything changes ever we're going to throw an error or we're gonna fail. The test.

A

B

This is going to fail right here.

A

Because there's no disabled learning function right away.

A

So, let's go and now we can go change the code.

A

So first of all, we want to have a have a parameter called whether we're learning or not.

A

Mark what can you get on discord and explain to me where you can't yet sorry I agree? This is really. This is just for the first cut, but I mean as long as the overlap functionality is working.

A

This shit's, this should be okay for the front, for every step of the way, I think.

A

A

So I'm creating so I'm going to default to learning being on I. Think I don't have a test for that, I mean how much do you want to test default is on, should I write a test that learning goes on by default. I could I, don't think I'm going to.

A

I call this learning enabled equals the stop, stop.

A

What did I call? It learn.

A

Yeah, okay, all right! So so that's where we're going to always go to this learning, enabled and I'm going to add a function, disable learning this.

A

Learning enabled people's false or true. So, let's see if this is that's not gonna work. Let's say if learn.

A

Learned is not equal undefined.

A

Otherwise, true so I wanted to default. The true.

A

Okay, you're a very paranoid you're, it's good to be paranoid as a software engineer, I think I think we're gonna find out, see we're gonna visualize. This almost immediately and we're gonna see this right away and we should be able to see it learn right away. So once we get this tested on a sanity check, tit I think it's doing what I think it's doing we'll go, release a version and fix it all right and and hook it up.

A

It's already hooked up well release a version that learns essentially until it to learn, and we should see over time the many columns grow affinity towards certain subsections of the spatial input.

A

A

I'm also going to add an enabler.

A

Okay, so we're always gonna go to that so now, let's run the test and we'll see a different failure. Keynote.

A

So now we should get a failure, not an error.

A

We are, this is gonna, say these permanence is aren't being updated, the little lights aren't blinking did I know that moving. Can every property includes a undefined? Oh, that's, not what I expect it includes that was on this overlaps function. Oh I know why it's because I said.

B

A

And I think it's singular, actually overlap. Let's call it make them all singular, find out.

A

As soon as this, as soon as this past is gonna reward myself with some Japanese candy I love this stuff, it's I, don't know how to say it whoo-chow, but it's a it's Cola flavored and it's fizzy and it's got little gummy Cola things in it. They're delicious here we go winter mini column at 58 was not decremented. Okay, so now we're getting real errors that we expect to get okay. So let's make it work.

A

Okay! So where are we gonna do this and here's our perms? This is where we're calculating the overlap. Here's there's a competition, we're not gonna. Do this efficient yeah! You get! You know what I'm talking about this. These are awesome, particularly the Japanese candies are so good, but this one in particular, I really love the fizzy Cola gummy histor I get them at Daiso in San Jose to die. So it's like a Japanese dollar store thousands of different crazy items. It's so interesting.

A

A

So we get our overlaps, but we're not learning. So when we compete I think we do have to do this in calculate overlap, so in calculate overlap. We get all of our permanence --is and get these permanence --is, because we're keeping them in this array called permanence --is and for each permanence.

A

This is we're calculating overlap, but we can. We only want two, so here's the winners. So what we're going to have to do is.

A

This is not going to be efficient, okay, but it's gonna be right, so we're gonna, we're gonna, calculate the winners and then for each of the winners. We're going to go. Do this get let's say if this dot learning enabled.

A

Winners, if learning is enabled winners for each winner.

A

Okay, we're gonna say: let's get our perms. Let's start firms come on stuff terms. This dot permanence is.

A

I'm gonna call this all perms. Just so I know that I, you can use the index for this, so the winner has an index mini-com index all firms winner index.

A

So these are only the winners.

A

And for each one of these I have to go through, the permanence is again for each.

A

And this is going to be just a permanence and an input index. Oh no! This is going to be a pool index.

A

That's! Okay! That's! Okay! It's just it's a lot of indexing, but it'll work.

A

Eventually, it work.

A

Okay, so now I got to go to the pool which I've already got right. I have the pool do I, not a ton of the pools. I've got permanence is crap.

A

Okay, so we'll get pulls all pools.

A

Touch pools and the potential pools are byte index as well.

A

So I should get my pool here is all pools minor index. So those are the potential pools.

A

Yeah, okay, so now in the pool, I have to say I said this pool index. That's how I know this is going to be my input index because remember the permanence is don't have an input to the index. The permanence is associated with a potential pool or a pent-up array or the links. The indices to the index, little-little I, know I, know, I, know I already.

B

Have it in put index don't.

A

I or do I index, oh no wait. I just have a I forgot. This is.

A

There, okay! So that's my input index. Okay,.

A

So now I need to get the permanence value.

A

If, if there is a connection, so that's that's the.

B

Next thing: isn't it but.

A

I've already done the connection bit. Oh.

A

Right right, okay, so so so actually, how did I think about this? The winners.

B

A

Overlaps from the winners now I have to use the overlaps.

B

A

Just bear with me: you guys I got to work. I got to work through this in my head, so there's the overlap.

A

Lenard up, but people's wintered up overlap.

A

Okay and these indices are pointed to the input index so now so I know the overlap now. Okay, so now I have the input index here. If the overlap includes this looks familiar to the tests that I wrote right. That's that's what happens when you do test-driven development if the overlap that means you're doing it, that means it's probably gonna work. If overlap includes the input index.

A

That's when we want to increment the permanence okay, so then we're going to increment the permanence increment perms. Only of.

A

Connections that overlap okay, so because I've already done the comparison just and the overlaps are already calculated. So now I'm going to say all perms winter debt index.

A

It's just all the premises and and and.

A

Now all my firms, so then I have to find the right permanence. Isn't it so I have to go to the pool index and then I have to add plus equals the decrement or the perp. The increment, which I which I haven't, which is in the options? Okay, so I'm, gonna, I'm gonna, put.

B

This up here, Const.

A

And it's increment, stop stop permanence increment haunts permanence.

A

Decrement equals this obstinance decrement okay, so this is going to add the permanence increment.

A

The data structures are not fixed sizes. That's why I'm doing this pool thing. The potential pool is not fixed, so the permanence a--'s are tied to the size of potential pool. So that's why I'm doing this extra sort of third, it intermediary link to the potential pool.

A

Okay, friends, here, okay, so increment, otherwise this is only for winter columns, otherwise we're going to decrement it. So the same, basically, a synthetic decrement.

A

Thanks for following along mark appreciate, it decorate perms for non-overlapping.

A

Connections. Okay, so this is going to be minus equals permanence. Decrement now comes the moment of truth.

A

Then it didn't work. Okay, we'll find out mine, I.

A

Think we're pretty close, though well.

B

A

Is when we get to the point, so these are all the same things. So winner was not decremented and there's probably the same for increment. Oh, oh, these are old ones.

A

It was not decremented, so the first one worked: okay, let's, let's try this again. So if I were, if I comment this out, does everything pass so.

B

A

B

A

B

A

Have been weird.

A

Come on now, I know these tests are long. Okay was not admitted for overlapping input index.

A

This probably has something to do with the way that I am accessing. The permanence is I, think I know why.

A

B

Do this, let's.

A

Spate spongebob, okay, I'm gonna do this: instead, instead of all pulls in all perms, I'm gonna make a reference to me and and wherever I'm saying all pools and all perms. Yes, the pools are fine because I'm not messing with those that's but I'm assigning permanence, so I'm going to.

A

I'm not sure if this is this is fine. Oh.

A

So I already I have the permanence here. I'm gonna take so here's what I'm gonna do me dot. Permanence is I'm. Gonna, go straight towards straight to the data structure. I want a mini column index pool index.

A

And that should be the permanence that I just saw going to say, equals permanence plus permanence increment right. So the permanence is was a list or as an array with many columns over the mini column index and that's going to give me a list of.

A

Values, indexed against the pool and I'm going to reassign it to the permanence that I should be at plus the incrementing. Let's see if this works.

A

Should be the same, this is just a.

A

That I don't want to tear spongebob right now you have a kaboom sound I. Have no I, don't have a kaboom sound out of cow I.

A

Should have it so that didn't work permits these are not being properly updated.

A

B

Interesting: okay,.

A

A

So, first of all, let's make sure learning is enabled. Let's.

B

Make sure this is working.

A

Learning so I should see a bunch of learning. Learning learning learning is not enabled I mean. That seems what it that seems like what it is. They're all sites I'd see all those learning is just now. Okay,.

B

So that's good because.

A

That's hopefully gonna be an easy fix. Learning is not, which also means I liked the way I was doing this earlier. It doesn't matter.

B

A

Enabled so, let's see in this test I'm creating- where am I calling been able learning? Am I with learning enabled am I, not I I, never called learning enabled didn't enable learning. No, how does that possible? I thought I, could've swore I had a oh I. Just have disabled learning, but I put learn equals true here, okay, so it's really just it's something. I'm doing here like this is wrong.

A

Let's just leave it like this about this affair. This will pass probably good. I, probably just did a stupid if condition Ruby not.

A

All perms, it's not fun to wear special puller to.

A

Oh yeah, that's because I removed it so all I have to do here is just me: permanence, --is.

A

It's weird that that didn't trigger last time.

A

Okay, so that wasn't it ice misspelled permanence is here.

A

And the test just, but only in the test, I'm.

C

B

Bad at misspelling.

A

Permanency I misspelled permanence in several times her menses place. Okay, just keep me from Oh, crazy, okay, we're so close, we're so close, we're so close, but we're not getting somehow learning enabled is not working.

A

This table Stern.

A

Is true here but did I create a new SP anywhere the SP.

A

My learning is disabled on Demento and learning is disabled.

B

We're not even.

A

To this one, yet we're not even getting to this one yet we're failing here.

A

So, let's initially I should what I probably should have done anyway, said I said: learn is false on this one, because we're not testing learning in any of this stuff, and when we get to this, one will explicitly turn learning on when learning enabled so.

B

A

Will explicitly turn learning on and then in the next one we're explicitly turning learning back off so we're initially not learning we're gonna run the competition tests and we turn learning on and run the learning tests and then turn it back off and run for continue running learning tests that are disabled, I, don't know if that's gonna fix anything, it's just a little bit more clear, clearer, doesn't fix anything. So are we.

A

Are we even getting in here are not console.log still, we are learning.

A

And if we're not, we.

B

Are learning do.

A

You see that we are learning.

B

A

Wait a minute, that's not when we should be learning.

A

We shouldn't be learning when we're calculating the overlaps.

A

A

Whoa, that's not what I learning and able to Sable the naval disabled.

A

Jerem any calm competition when learning enabled that says: disabled disabled, disabled, disabled, okay. So yes, so that's not right! But what is this learning disabled enabled coming from.

A

There's something weird in my test here: this is false, find out references.

A

B

A

A

Before I go nuts, let me I'm going to disable these I'm. Just gonna comment. These tests out for a moment, I want to see what I see as learning disabled or not. Learning is enabled why it should be disabled okay. So this is my problem: I've got like I've got it backwards.

A

See this is the only place I'm calling an able learning and I've haven't commented out and I initially have learning as false. So there's something wrong if I've got learning and abled here, something certainly wrong in the spatial Pooler.

A

A

And SP is earning enabled.

A

This is typical. The way I do should.

A

Some people like to run debuggers I like to do console logging, ok, undefined undefined false true, true true, where is the true coming from.

A

Why would it ever be undefined? Oh.

A

Yeah because learning is undefined, let's just say: if I was going to do this simple, if learning heart enabled equals undefined.

A

Do we want to default to learning or not I, don't know, let's I keep going back and forth about this. Let's say false, let's say no say no, we're not going to there. We are not going to David's in the house.

A

So I'm gonna take us off I'm, gonna, say well now, I'm gonna be explicit about it. We're gonna, we're gonna call send it in explicitly false here explicitly true here. No no! No! This is this is where I want it to be false, false, that's. That was one of my problems and then we're, and then we call a naval here. So learning should always be false in this test.

A

False false false there we go false, false false and now that's that's what I want to see? Okay, all right! So thanks David for the clean up there.

A

Okay, so that's what I wanted to see there. So, let's turn this: let's slowly turn these tests back on and see what happens.

A

This is one test on new input when learning enabled so we're going to call SP learning enabled and then we're.

B

Not going to see this because.

A

This is, is this only in the constructor, then able learning I'll make that true, disciple learning we'll make that false.

A

Alright one more time with feeling broke, something I got a syntax error somewhere, it's probably because of I didn't.

B

Comment something out.

A

Yeah, it's probably this.

A

A

A

Unable to say old, unable to say.

A

So I don't know why it's alternating I, don't see, and in this one it still says: disabled, that's screwed! This is screwing with me what the heck's going on here. First of all, why is it alternating.

A

If I take this out, Oh a.

C

Lot of times, I see that in tests, when I have a single equal sign, instead of a double equal sign, oh really: well, are you setting it instead of.

A

Let's see, let's see well, this is where I'm setting it that's where it's supposed to be. Where else are we and otherwise I'm, just saying if learning enabled.

A

It's only a couple places I'm, even touching this. It's like every call changes it. Why would that happen?.

A

A

I'll figure it out in a minute, so we're saying learning is enabled.

A

That is I'm taking.

A

I'm just pulling it I'm, just I'm uncommenting everything that every test I really don't want to run.

A

And Pitt's a truce many calls with most over that wins, I, don't care about that.

A

On the input, when learning enabled.

A

Like this whole thing, I can I can take out we're just trying to isolate only the code. That's failing are the only the tests. That's failing so I know that there's nothing interfering with anything else. This learning is enabled because, because I called learning enabled oh.

A

This this that's false. Okay, this should also be false, but then I called enable learning here, which makes sense, so it should be enabled, but when I get into the actual checks.

A

That's okay, so so these are printouts a lot of times. You'll see the console logs come out before the test results and I expect it to be enabled, based on everything that I'm showing here. So, let's.

A

I'm gonna even put more logging in here fermenting I should see increments allons in addition to yeah. So a good thing fermenting, but the permanence is don't seem to be updating and it could be maybe that I'm just I'm hitting the wrong index. So, let's, let's be more explicit about we're doing here, I'm gonna, say incrementing.

A

Let's see what this looks like, maybe maybe it's not a value. Maybe it's wrong. Oops.

A

We'll figure this out we're almost there. Suddenly everything.

B

Will start working okay.

A

Well, it found the value okay um before incrementing after incrementing, we should see a difference right.

A

Stop it before before incrementing after they're.

B

A

By hat by 0.4, okay, wait a minute wait, a minute. The actual errors were getting. Oh there's a through some index wrong or or no no, it's not! That's! That's fine! That's expected it's difference of 0.05, which is right.

A

A

Let me check my test.

A

Make sure the original permanence original perms is coming from get permanence --is unless get permanence --is oh get provenza's needs to return a clone.

A

That's what it is it's returning. A reference get permanence is needs to always return a clone. Does that make sense if I slice, this I.

A

Wonder if that fixes, it.

A

A

I'm, just that's right! This.

B

A

See I'm just wondering I was thinking them like if I'm calling it permanence is here and I'm, calling it again and if I have all updated this.

A

Obviously, when I run the compete step, if it updates and now my reference just has the updated values, maybe I just need to do a deep clone, because this is a it's, not a deep clone. A slice isn't a deep clone right. So I need to do like a deep clone here which isn't it's an array of arrays. So I need to do a deep clone of array of array, JavaScript array, deep, deep, deep copy.

C

I thought I thought permanence was an array of numbers. It's.

A

In it, well, it's an array, indexed by many column index, so like this, this permanence is, is an array of.

A

A

Wait a minute: that's an array of.

A

Permanence values, indexed by I've, had forged mini columns, so there's one for each mini column and it has.

A

Its Oh okay hold on I gotta work is not my head too many data structures. It's too many data structures, okay, so perms. First of all, it's it's an array of mini columns, so you know 0 1, 2 3. This is mini column index and each one of these that's the index. Each one of these is a scalar value. No, no.

B

Each one of these contains another array and.

A

This is an index of the pools.

A

No! No, that each.

B

One of these is.

A

Is the the permanence values like point one's going to join on whatever, and these indexes are the pools, the pool indexes so.

B

It's an array of arrays the.

A

First, arrays is bordered by mini column. The second array for each one of those the sub array is indexed by the pool.

A

So it's an array of arrays, so I think I do need to do it like a one level, deep clone right, I.

A

Said I shouldn't have to yeah yeah it's. um This does make sense. I think this is the problem, so the this slice won't work.

A

I don't need it's an object, a neurologist, an array of an array array from creates a new or a instance from an array. Maybe I can just do that. I'll! Try that first, let's try that first.

C

That so do you want that to return from get permanence, as always, or do you want to do that to what.

A

Yeah I: don't want get permanent system to return. Anything that's referenced. I want it to be always a immutable object. You.

B

A

Something something that they can change and it won't change the spatial cooler and if the spatial Pooler changes, something it's not reflected in whatever they got from dignity of permanence, no links, no references.

C

A

Maybe that'll work.

C

Yeah there's a couple: let's.

A

Try this first that didn't work.

A

Cuz they're super close can.

C

I, try something yeah cool a lot of times. People do like this, like JSON, that parse a string goodbye but I, don't like doing that. Just cuz it's memory, just.

A

Do it do it and let's make sure that it works? Oh yeah, I guess we do Matt I've seen the jason method before but you're right, it's it's kind of suboptimal.

A

Okay, so this will create a new data structure. Is that it you're gonna, try yeah.

C

Let's try it: let's.

A

Try that I hope this is the issue I'm.

C

A

C

Have following along so what? What we'll see if you going on within the test? Well,.

A

Let me clean something up: real quick and I'll. Explain: yeah I putting I've got a print statement somewhere. That's where is it now learning.

B

A

I've I've convinced myself that learning rather enabled or not is working.

A

B

So the issue is, after.

A

Incrementing yeah.

A

So I'm trying to turn this down.

A

So when learning is on every time, we do rerun the competition we want to go through all the winner, many columns and an increment or decrement, the permanence --is of their weights, and so how I'm doing this is for each one of those.

A

If learning is enabled for each mini column going through all the permanence is for that column, because they're indexed by the mini column index, which is here and for each one, I, should get the permanence value and then the index of the potential pool I go to the potential pool to get the input index and that's what I look at for in the overlap. If that input index is in the overlap, then I know that this this permanence needs to be incremented, and so the problem I keep thinking, is and I'm. Looking.

B

A

And and when it so I just uncommented this, but this was working right so before incrementing is a different value and increment it and I go look at the permanence --is and it's been updated, so the permanence is, is certainly being updated. I'm going directly into the data structure by reference to do this, update I'm, taking its permanence that I'm adding or subtracting the value that needs to be. But what I'm thinking is wrong here is in my test.

A

Up front I'm, getting all the original permanence --is I'm running the competition and getting all the new permanence ha's and if they're all references. If these are rich firms of references, they will be updated as soon as I run this compete, then all my originals are now updated, so I need whatever returns from get permanence --is to be a snapshot in time and should never be updated. No matter what the spatial blur does. At that point,.

A

B

It's not happening.

A

The way I'm expecting it to.

A

C

A

Just like foreach so I'm going to go through all the winners.

A

Original perms.

C

Does that show it's true.

A

Let's see, let me do let me do this, so it doesn't run a bunch of failures.

A

C

Okay, so they're not gonna, be the same reference, and let's here now run this.

A

Okay, that's also false yeah.

C

Okay, so they'll be different, so.

A

That's not the issue.

A

So what I thought was the issue is not the issue. What is my problem kingdom.

C

Hecklers, like compete yeah, let.

A

Me put this: let me review this test with me, real quick, because I think this is the right. The right logic, I go for each winter column. Now then I go through the input array, all right, it's because for each winter column, I want to look at the input array and then loop through each one I'm, not even looking at the bit I don't care about the bit because I've already done the overlap.

A

Calculation I just want to go to the input array, get the original permanence value and the new permanence value for that input bit and assert that if this was an overlapping.

B

A

That it was incremented right, otherwise,.

B

A

Decremented that makes sense, I think.

B

A

Makes sense this is the crux of the test like this. This is the test of whether we're learning or not. Is that- and this is what keeps failing and I, don't understand why.

A

Is saying the new permanence should be the original permanence plus the increment? Otherwise, if it wasn't part of the overlap, it should be decremented and it's and it's acting like the test failures are acting like permanence. --Is were never changed, at least that's what I think it was doing. Let me run it. Oh my.

B

A

Did these pass because I comment something out I hope so.

A

Whoa, what's going on yeah yeah, so.

B

A

Just getting failures- and it should be- they should be like either 0.1 or 0.05 off each one of these. So that's 0.5 off all the decrements or 0.5 off because it expected it to be.

A

Yeah expected it to be point: 4, 6,.

A

But it was actually higher.

C

Doesn't listen, those are going to compete, function actually.

A

This is all the learning code right here.

A

That's all that I have added really what.

C

The winners gives you a an array of indexes of the columns that are where.

A

It's it's an array of objects and, and each one has an overlap and an index. The index is the mini column index of the winner, the overlap and is an array of indexes to the input. So it's so it gives you a way to say from this mini column. Here's here's how it's mapped to the input. Here's all the overlapping input bits.

C

So all pools is the column. It's.

A

The potential pools indexed by mini column.

B

A

B

A

I need that, because the the potential pools actually has the map to the input, because the permanence is are tied to the pools. The permanence is are the same length as the pools. You know what I mean.

C

And this permanence is for each one of these permanence is. This is going to be the actual permanence value, and then this is the index of what.

A

The index to the potential pool, okay.

C

So the input potential index know.

A

Potential pool index, and then the input index is is, is that as part of the potential pool right.

C

Yeah, that's something yeah, so that'll be the index most convoluted, all right, yeah.

A

It's because the permanence is tied to the structure of the pool. That's why we're going through this extra sort of step through the pool to get the input index, because I didn't want to create the pools I wanted to create a list of indices for the pool not just like didn't cover the entire input space. It's just a list of indexes to which the mini column connects to my connectivity and then the permanence is associated with that.

A

C

What is this overlap here? This is, it's.

A

A it's a list winner.

C

A

A

So Cao Lu can look at calculate overlap for that, but it goes it looks. It should be in this indexes to the input array and it's which indexes in the input array are currently overlapping. The many Collins connections right.

C

So term here is really just this way exactly so.

A

I initially had plus equals there, mm-hm I, don't know why I took it out because I it wasn't working and I was just trying trying different things, but yeah.

B

A

But that, but it shouldn't matter and and if I go in here it I can we should see it- let's, let's verify this console.log okay term arm and then actually I'm going to get input index this. So you can see.

A

If it index, so that should be something that maps to the input array.

A

That's that's what I would expect, because there's I think the input is wait. What size is my input? Let me make sure in the test. Anyway. How big is my input? Input count default, input, count which is 500 so that that's what I would expect indices up to 500 and there's 499 so yeah. So that's what I expect not to be.

A

Something there's something strange going on.

C

The only thing weird about this is you're looping on this array and then you're changing that array, I.

A

Know yeah, so that's gonna.

C

So yes, what you would really need to do is slice. This I mean not that everybody won't mind it, but if you slice that that way, you're not editing the array that you're looping through but I, don't see him how that would cause this issue.

A

Yeah I mean it could I could try it. We could just try it like that, so that we're operating in a coffee, all right, yeah.

C

A

C

Mean none of these are gonna, be called toys because they're, each indexed by their own column,.

B

A

B

A

Think something we think.

A

Me I'm thinking, maybe I'll try something like really what, if I.

A

Damn I expected to be to be passed this by now. I, don't know it's. This may be one of the situations where I have to sleep on it and come back and I'll immediately know. What's wrong, I was hoping to get the visual visualization working.

A

Take through this again, learning is enabled for each winter column is.

C

This right cool index, I think.

A

So here, let's let's, let's.

B

Print it out to.

A

A

It shouldn't be the same: it's not the same, so yeah I think that's right, because there's less pools and there are input you.

C

A

It's pool as a subset of the input, so they should go up slower, I! Think that's right, so that.

B

A

A

That's one index input.

A

Let me see, let me try something with this test again, you would. You did something that made me think. Maybe we could simplify this okay and say instead of these tests. Okay, let's just say, assert, deep, equal or is it if they? If that overlap includes it, we're going to say not deep, equal I think it's up the thing not deep equal and we'll just do all of them: new perms, original perms and no perms, so that should that should pass so likes. Okay. So if there's an overlap, the firm should be different. Okay.

A

So let's test that.

A

Probably take out that console logging, but.

A

Did did that work I, think that worked so maybe I just have maybe I've just screwed up my assertion. Somehow: oh okay, okay, so that's promising! Let's, let's do is deep. Equal whoops I think it's just deep evil, deep, equal new firms, original perms I, want to see that work I'm going to take out those print statement: sighs, okay! So if this passes that may run the right track, then maybe I just had something wrong in my assertion: indexing.

A

No that failed. So we're failing on the else.

A

Overlapping the overlap includes this input index.

A

B

No I'm doing this at the wrong line: yeah yeah, yeah.

C

Yeah this this actually.

A

Isn't a valid test? It's just this! If we're learning, no matter what we should have a different different firms, so that's that's where that should be so.

B

A

That should pass and.

C

Then you want to do the same thing for here: Terry yeah, I'm,.

A

Trying to check the individual values in this loop and make sure only the permanence is that were that should be updated. We're updated so I'm going to keep this and say.

A

Just say always be updated. It's already having a condition.

A

Okay, so that pass so sir properly updated, they are updated and then.

B

A

So I can actually add the opposite here, this other one for the winners. So for the for the non winners. Oh you know what I should be doing is looping over the the whole input on the outside with the winter loop on the inside. Let me let me let me change this a little bit then I can then I can include the test for the loser. Many columns in the same place, okay, bear with me here.

A

I'm gonna, restructure this a little bit and sometimes the restructuring sort of makes it clear what we should be doing so I'm gonna take this I'm gonna I'm gonna start at the at the input level so for the whole input and then and then we'll do the winter column for each.

A

Okay, so for every so.

A

And no, no I'm, not gonna. Do that, like this anymore.

A

Well, what I do okay, so I'm going to loop through each one of these bits and the and I'm also going to do this winner, indices, where's.

A

Okay, so those are just the indexes that, when, if winter in, to see, if not count, indices includes input index, this is where we're going to check. This is what we're going to do. One one assertion: I, want to make sure that.

A

A

Okay, so I don't have the new perm area, but this is essentially I think what I want to do so for for the? If, but if this input is part of one of the winter columns.

B

A

So then, it's so that I'm gonna check if any of the winter columns and I'm gonna go into that one or column. Oh I,.

B

A

Okay, you know what I'm gonna I'm gonna sleep on this I'm gonna sleep on this and we're gonna come back. Cuz I have other things: I gotta do today, I've been streamer for like five hours or so so I'm gonna sleep on this David thanks for your help. Mark thanks for your support. I think this is one of those things where I'm gonna I'm gonna wake up tomorrow, I'll be like oh.

B

A

Know so we'll put this in a branch. This isn't simple htm'. This is important that we get this right, because this is a part of you know our learning for spacial our spatial Pooler, so I'm gonna put this in a branch called spatial, polar learning, I'm gonna, add both of this stuff and say partial, partially, no.

A

Let's say: half half written learning tests that don't pass. That's pretty obvious. Okay, git push origin, SP learning. So if you want to go get that branch, you can go check it out. I feel like this is gonna, be something else. I'll come back and within half-an-hour figure out like tomorrow. So if that happens, I'll fix it we'll get it in place and then we'll do the visualization on the next on the next live stream.

A

That makes sense, because this is I feel like my intuition is like this is a banging my head moment and all I need is a good night's sleep and everything will fall into place and that's just a programming thing. That's uh so.

B

Let's, let's do.

A

That yeah, that's all this is a these types of data structures are tricky for sure.

A

Okay, let's, let's go share the love with somebody else on on twitch I, don't think, oh by the way, by the way I mentioned that I got a new guitar, so I'm gonna unbox it on Twitch later and here's the here's, the URL. If you want to watch me so and hopefully.

B

A

Hours I'll be doing that and otherwise, let's go see who's doing science stuff anybody doing science stuff, oh, let's go to Simula simulate, he's always doing algorithm work. So let's go read him.

A

You guys can go check out and see what he's doing he's always he's always doing some interesting stuff, he's a nice guy and and again, thank you very much for watching and for supporting the stream be sure to follow. If you haven't already.

A

B

A

The stream of the Stratocaster unboxing, you can always catch it on the video on demand. Part. So I'll see you guys later and take care, probably not this week, because I've got conferences later this week, but definitely next week, we'll finish this up and the visualization will be cool when we do it. When you have learning on your cool, you can tell that it's learning so we'll get to that point. Take care everybody.

C

Hey I just don't know how we would do this transpose exactly that's the thing I couldn't figure out, so this mean we trade.