Numenta Live Streams, 4 Jun 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: BHTMS: Spatial Pooling Potential Pools Diagram D3js / Reactjs

Description

Broadcasted live on Twitch -- Watch live at https://www.twitch.tv/rhyolight_

A

It's a lot of sirens now all right, bringing up this okay, so I, know I, go through this almost every time, but that honestly for anybody new. This is the vision that I'm shooting for all right. It's a little bit cartoony, but this is sort of the.

A

This is what I'm shooting for to explain: htm' sort of in the context of neurobiology I want and that me I'm not going to use these exact, graphics or anything. But this is the direction I want to come into this spatial pooling conversation talking about something that's happening in one layer of a cortical column, or at least with one type of cell in a cortical column and how it handles feed-forward input, hop and its relationship with feed-forward input.

A

The idea, then, to create an input space. You know, saying you can only sample from so much of the feed-forward input and to talk about the mini column, competition that will be occurring within one of these spaces, and so the first part of that is is taking just one of these mini columns and sort of breaking out its relationship to the input space.

A

I'm not sure if these, what what I said right here, I haven't run this by anybody who has better neuro, not neurological knowledge than I do, but if, like these may not be going in the right direction, I don't know well, but essentially each one of these mini columns like this has a potential pool of input that it might get connected to. It can't connect to everything in the space, so you have to I mean biologically it will never connect to all of those neurons even that are within.

A

You know the the area of neurons that are coming into it. It has part of it. Thanks mark I'm, glad you like thee yeah, that's what I thought I was feeling like they I was getting going in the direction wrong, but the idea is right, I think so, assuming this manically encoded sensory input is coming through here.

A

We're gonna create this streaming encoder, which we have. We have done. Okay, let's, let's take a quick look at what we have done so far. First of all, building HTM systems- readme, is here and I will throw this into chat. I always think the mouse is not where it is to us all right. So here's a link to the document that I'm looking at there's a music store downstairs. So you're gonna hear some electric guitar.

A

He tests out guitar, sometimes okay, so the here's, the Heroku deployment. So we can look at this. It takes a few seconds.

A

It takes a few seconds because it's a it's a free free deployment, so this.

B

Was our latest.

A

Deployed app that anybody can see that if you follow that link- and we just did this spatial pooling thing and it should be moving alright, so we actually have a streaming input occurring here and this diagram is responding to that stream changing over time. So all of the remaining diagrams that we're going to work on from here, basically almost all of them from here on out, are going to be attached to this streaming data component, so everything will be moving at. However fast.

A

We want it to be move moving, because this data stream is going to be moving. The data stream is, like you know, looks like this. It's a typical spatiotemporal data stream is one scalar input through time. Okay, hop Jim, it's hot Jim data, and if you don't know what that means, go to the forum and search for hop Jim, so all this stuff is going to be streaming. The combined input like we just saw is streaming so this is live updating as the input changes eventually we'll have this display.

A

So you can see it stream to the next thing that we're doing is the same sort of streaming deal but potential pool we're going to show the potential pools. So we're going to start with this combined encoding, that's all gray and we're going to overlay based on the selection of a mini column. So we've already done that have to make some decisions here. How many many columns we want to have? We already have the input space sort of defined.

A

So now we have to decide how many many columns do we want to have, and how do we decide what cells in the input space is going to be selected to write or it's going to have access to? So that's how we decide the potential pools and what I want to do is, as the user hovers over this space, a mini column is selected and its potential pools are displayed over top of another space, which represents the input so essentially we're gonna. Have this grayed out that's going to be the input space?

A

Well, have another we'll have this over to the right? Okay and we'll have another box. That is this. That has a mouse interaction, so, as the cursor hovers over that you can, you can see for each mini column or, as you can select a mini column essentially, and it will show its potential pool. So that's the plan, and the other little bit here is- is just some additional marks on there to highlight not only the potential pool but the inputs that were either observed or ignored, because that's that's also important too.

A

You know, because of these potential pools exist, some mini columns will never connect to some patterns if they, if they're, not representative, represented or able to be represented within its potential pool so there's by it by narrowing down the the field of the mini column, to only some of the input, your your sort of your enforcing selectivity right you're, not good you're you're, losing information. Okay. This is just an example of how the spatial Pooler loses information all right. Okay. So how do we do this?

A

Let's get down to how you say: brass tacks as that's correct, let's see, NTA we're in building HTM systems where we do I have what do I have different here.

A

Oh I never committed that did I Falco says is the mini column, so the mini column is each of the little squares on the area on the left. Yes,.

C

A

Are these are the mini columns in an and just to represenative array and actually I'm going to I? Don't want to mark this one up. We're in else did I marked this up there. This.

D

A

Document there.

A

C

A

Think that was it. Let me go to this one I'll mark this one up all right yeah, so this this.

C

A

On many columns over here- and this is our on the right as the input space, so so, let's get to our code here.

A

The first thing to do is share a session. Let me make sure that I'm all up to date and I apparently have changes that I didn't stage which that's a bummer I, don't know what oh yo. That was for something else, so I'm gonna leave that that's something that David was working on, but I'm gonna, so I'm gonna leave that I'm, actually gonna I'm gonna stash it I'm gonna check out Dave.

E

A

Don't know he showed he was just showing me how to do something with reacting. Divs I think I'm gonna remove it because he showed me and it's got reference to it. So I'm gonna check it out, get checkout pages and then I'm just gonna. Remove this combined. Oh I can check this out.

D

A

Playground is new alright, so let me do a live share and now I should be able to paste here if anybody wants to follow along with the code and let me get a server up and running so that we can follow so that you guys can follow a law. Here's my server and, if I remember correctly, I can do this share server share the server here and now. Anyone who connects to that live share link will be able to run the server which is right here on localhost, so loops, and so here's we're gonna start.

A

We're gonna continue working with a spatial Pooler spatial pooling document. Here, let's, let's fill up the space we can fill and, of course, in our debugger open all right, so we're gonna start we're gonna, create a new diagram and.

A

Aside from the combined encoding and we'll call it potential pools and it's gonna, it's going to be one big SVG, 500 width, I'm, trying to make them all 500 width and with two groups of cells within them, just like just like I showed here, so the SVG is going to encompass both of them. So it's gonna like wrap like this, and the whole thing is going to be in this SVG, so I need to make sure that I can fit it in there properly and it may be for the space of space sake of space.

A

We move that somewhere else or something, but for right now, I just want to get the these two sections working. So, let's see where we were at with our code and let's turn some music back on now that I'm not guitaring I.

D

A

To make sure that that's not too loud for anybody.

A

Okay, how's the volume on that turn it down a little bit more.

A

So I assume that the that these are layers in the same section of cortex, no they're, so the only one of these represents a layer of cortex only the many columns. This represents feed-forward input throughout the right side. Here, yeah this one. This is feed-forward input. This is the only one representing layer of cortex. Does that make sense, Jeff speculated mini comms could exist across layers. Yes, they definitely do.

A

They do exist across layers, they're not observed in all layers, but if there's, if they exist in some and not in it, but not in the middle but then down lower I mean we probably just don't see them, they probably exist in all layers. Just because.

E

A

Don't see them doesn't mean they're, not there. It's one of the problems with neuroscience.

A

Yeah so feed-forward input, I'm, not gonna, assume anything about the feed-forward input and, in fact, I'm not going to soon it comes from the same layer. Perhaps this part of the input comes from one layer. This party input comes from somewhere else. This part comes from somewhere else, so we won't make any assumptions we'll just assume that each one of these mini columns has a potential pool into the same area of input wherever it's coming from.

A

They have to be sort of grouped together, somehow like topologically, so that's they can reach each other, all right so to the code. What we have here, we're looking at I've, made a toggle button. We are in the spatial pooling page, so this is reactor gun order. We've got a react application and we're going to create some d3 diagrams. On top of this react application.

A

And the last one we created was this combined encoding and that's that's the last bit that we saw, but we're going to also create- and this is our toggle for it. So I'm going to program.

A

Another encoding I'm, assuming this is going to be potential pools and I'm going to I'm going to set it up, like I, think it's going to exist.

A

This is another diagram. Diagram width is the same.

A

Okay, so what do I need to specify sorry, sorry about the guitar sound downstairs. What do I need? What about are you gonna need to specify to this? Well, I'm gonna need to tell it first of all how many many columns there are all right, so mini column, count.

A

We usually say 2048 that might be too big to do the visualization and they also need to specify.

A

There's one other thing is that for each mini column, if I'm looking at the mini column and its relationship to each one of these input cells, what are the chances that it was? It's gonna be connected and I. For no note, it's actually more.

E

A

D

A

Of that input is going to be in the potential pool, that's what it is and I think in in new pic I think we call this connected percent thing like that, so initially we'll start with one right which is 100%.

A

So let's just do that. For now, we should see a fully.

B

Connected layer.

A

Well, not in that way, not in the Boltzmann machine. Wait: okay,.

C

So this doesn't exist yet so we.

A

Need to like this is just going to break everything, so what I'll do is I'll come in here want to steal.

D

This one because I'm going to use some parts.

B

A

One, this isn't diagrams new file. This is funny I'm. Looking at the wrong screen, yeah new file.

E

Potential potential.

A

Pools JSX I, never thought I was going to be. Writing JSX.

A

Never do okay, okay,.

A

I will not need most of this stuff. Only this I'll use on/off color combine into color and then we'll have a like PP color, which is the potential pool color, which is going to be very light, yellow lemon, yellow or something.

A

Keep the padding the same okay, so we're not gonna have any coding we're gonna have actually.

B

We are we're gonna.

A

Have we're gonna see here's the thing. Do I want to I'm gonna have a combined encoding in here, so it's gonna have to be updated with the data which, which makes sense, I need this data and I will have it going so yeah.

B

A

This and this data is, is representing another thing that we set up last week, which was a high order component right though, but it's called with scalar data, and this is what actually generates the data, so this is going to be generating data which consists of value and time, and this will just progress over time. So we're going to depend on on this coming in as berra, this being updated, this data component being updated for potential pools at some consistent regulation, one Hertz, for example. It's probably good. It starts out.

A

Okay, so we're gonna need a mini column, count we're going to connect how much how connected it's going to be. Let's just let's make this more interesting, so we can tell about 50%, 85 I, think what we generally do is 85, and then we know it's gonna get data, so it is gonna need an encoder, just the which encoder is it going to need only well, it's gonna. Do all these encoders it's going to need everything!

A

That's the right! Yellow, okay, he's found me good, yellow-headed, think lemon yellow. That was a stretch, I just kind of guessed.

A

Okay, so I removed my encoders, but I. Definitely I'd still need them. There's there's another thing: I could do but out, but I won't it's it's. It's fine to just I, don't mind doing this. How they think about it. We are. We are not going to use those other colors, but we will encode this data in the same way. Fact, let's close this, let's make this a little bit smaller and split to the right, so we can see our combined encoders side by side. Potential pull bind encoding.

A

So all this stuff is correct. We're still going to do this, render output cells I'm going to call something different: it's not outfits cells anymore, let's call it its its input space and your input space and the input space is always going to be. This is fine. I think encode will be fine.

A

So the input space, the color, is always going to be the same.

A

Didi, that's right. We're setting up a little data object, she's the encoding itself via high order component. um You know, I was just thinking about that it could that way it could be encoded like the scalar decoder updates, the encoding updates and just adds something else to the data which would be an encoding.

C

A

I mean what what I need it's still different from the combined encoding, because the encoding I need isn't broken up into pieces. The combined encoding needs it broken up into pieces, which requires even more information. So, let's.

D

A

It for now, like one of the things I, continue to realize as I make more visualizations than that each one is going to be unique and it's okay to copy and paste some code, sometimes there's just a few unique things and to some of this stuff, but we'll see if you see stuff like that, it's good definitely good to bring it up.

A

Yeah I need color data or you could pass it as an object with each of coding as its own attribute yeah.

E

C

A

C

A

Well, maybe we decide to do that later, maybe say to do that later, but for right now, let's try and get this potential pools as simple as possible, because the data.

A

That's right, since this joint color bit that we can remove from the encode. So when we encode here we're not going to do the joint color thing, this will be simpler. It's just always going.

C

A

A

It's just going to be PPE color, no combined combined color, which is great and I'm gonna. Keep this up here. Yeah I already have it so I have a combined color. It's always going to be the combine colors color.

A

Combined color or off color, that's basically.

A

So I can pull this off and take some function.

A

E

A minute this there's.

A

Probably better way to do this, some encoding.

A

Now I do need to do it. I just I can provide the same function.

A

And that's basically, this.

A

Just make sense, I'm doing the same thing to all. No, oh, no I can't I have to do it right here.

A

So I copied the rock thing. Let me start with 4-h.

A

Sorry I'm keep that's all.

C

A

I just want in the 4-h the color function.

A

D

We're just doing.

A

This basically for each one we're just coloring at the same color just so we can match like the the data pattern that d3 is looking for and each at the end of that after we do that, we'll have filled out our encoding I think that should work.

A

So now we'll get only so. This color is going to work out and let's look at the render. So so this output cells gonna call this input space that how fit cells.

A

And that's: why we'll call this anything space.

A

And okay: let's see what broke.

A

Did you pull? Oh, yes, I just have to import potential pools and the spatial pool air.

A

Prop types fix this real, quick.

A

This is a react thing. Okay, we do need an ID, we do need data, but we also sending something else. Mini column count.

A

This all have diagram with two.

E

B

A

So we've got a warning here: can't perform a react. State update on an unmounted component.

A

I'm not sure if that means this isn't no op, but it indicates a memory leak to fix, cancel our subscriptions and asynctasks, and the component will unmount so I've got something in component will mount that I'm not gonna doing a component did mount I.

A

Just David to the rescue so something an.

D

A

C

Do I need to do this, isn't doing anything.

A

Different than the other one yeah.

F

I think it's just a warning from before when the page that the HM art, the hot module, was it dismounted and remounted, and then it probably called updating after a minute. So I was like, if you refresh the page.

A

Yeah, all right so I don't have a toggle here, but I have exactly what I want I'm.

E

A

Gonna comment the other one out, while we're working on attention pool so we're just looking at the one thing actually Oh.

A

Okay, so this is.

C

Making the space.

A

And I'm gonna write prose for all this I just have not gotten around to it. Yet.

A

Okay, this should be clean. Now, look at potential poets, all right, so we have our one. So now we have two. So here's where I'm gonna draw the I.

A

Want to have a debug style.

A

Nice I forget how to do this, don't.

D

A

I'll, do a react style attribute right.

A

Is it inline Styles.

F

Nobody just put like a style sheet like yes, yes, that's file or whatever, and then you just set the class name. Ego no.

A

No I I went to actually just oh, oh, you mean okay, you.

F

Can do it like this, though I could do it either way, but.

A

I think I already.

F

A

I think I already have so I could do something like that and just update my stylesheet and make sure I have. That seems fun. That's gonna.

E

C

E

A

This is where I have been putting things.

A

I just want to see the SVG border. Okay,.

C

A

That I can work out the layout. So that's.

A

Because we're gonna have to we're gonna, have input space and that's gonna, be on the right. We're gonna have another one whoops, that's going to be the mini columns.

A

And I'm just gonna translate these put them in the right place. I could do it here. Well, we'll see. First of all, since this is work, I don't need you anymore. Did you split this.

C

A

Actually, I do because I want to decide how well it will just do it won't change it. First I want to allow the MIDI column parameter to change, but for right now, we'll just get it working first, so so the first thing I need.

A

So this is tricky because I'm not sure I want this diagram to display a state of the spatial Pooler, but but honestly, I think I feel like the state needs to be passed in what I told it to do was pass in the count of the many columns, but I could just have it pass in many columns and create the many columns.

A

A

All right and then we'll assume, we'll have to make some assumptions about the structure of this data. You know, but.

A

Because when I change, the / when I change, so here's a couple things that I want the user to be able to do, I want them to be able to change the number of many columns and the thing responds and also change the connected percent. So every time someone changes the number of mini columns or the connected percent that all of the central pools have to be reevaluated reinstated and I'm I'm wondering where the best place to do.

A

That is because I'm sort of eventually I'm going to create a spatial Pooler and code that initializes a spatial Pooler in another project that does the initialization and the state is going to be accessible from it. Some objects, you know like an SP object and I'll be able to look into its many columns and for each one look through its potential pools.

A

So honestly, I'm wondering I mean that that manipulation, the changing of the mini column and the changing their connected percent, is going to pre instantiate, the spatial, Pooler and I'm. Starting to wonder. If I should do all this within the potential pools, diagram or I should do it outside of the tension force diagram, it's just an illustration. So we could just. We could start and put the logic in the potential pools diagram, because what we're?

A

What I'm trying to explain, is the logic of creating the potential pools and perhaps, in the end, we'll figure out how to you know, translate that into the spatial pool or class or code base that we actually create. That runs spatial pooling, because at this point we don't have to run a spatial pool or we just have to create this data structure.

A

And so let's just let's try and do it the simplest as we can right now, which would be to isolate it within this potential pools. So what that means is when it mounts we know well, anytime, there's an update. We're gonna have to re-create this data structure. So I'm gonna have a we are well, you know we have an encoding and that's working. We're also gonna. Have many columns here. Did I check this out yeah. So this is we're. Gonna put this back to count what.

C

I'm gonna undo.

A

What I just did this is gonna.

C

A

Count for now we might, we might come back and change this, but so we'll keep a local state object for many columns, we're just going to update anytime someone changes. One of these, so we're going to be recalculating here inside the potential pool, diagram, I, think it'll be the easiest thing to do, and coding is working.

C

A

So in the update, this is where we create the encoder. So we need a spot here that, just like in code and hit put space, we're gonna have something else which is going to be a like set up to potential pools and then the stop render potential pools. Okay, I'll just try to do something similar, so set of potential pools, render such a pools.

A

Okay. So what we're going to do here is we know that we have many columns and it's at this point it's going to be there's going to be nothing in them, so so, essentially for our props. We're gonna create a new array. Well, we're gonna create that new array and you want for each one of those. We're gonna create a new set of potential bullets.

A

So the easiest way to do that, perhaps, is just to say this dot, props, mini column, count clout.

A

I didn't: where did I put clout.

C

Good call count: okay,.

A

Okay, so yes, how do you loop a certain amount of times in harmony, JavaScript I wrote count with an M, oh I, think you're good.

F

You do you care about performance. No.

F

A

F

Do a note right here, up I'll put it in their new.

A

Array with that amount of count and then for each.

F

Now, if you can never could everything would be undefined, they wouldn't be initialized, so you live a ring. You actually need to wants it. That's.

A

Okay, if they're uninitialized, all I want, is to run a function so many times the number of mini column, Cal.

D

A

Tell where you are.

A

Shouldn't this work, this seems like I, should work.

A

Up because I'm gonna do actually.

A

The city seems nuts, but bear with me.

A

That's all I want to do.

A

For each one of these I'm, going to I'm going to create a data structure and we're gonna have something called potential pool and it's going to be empty. Initially,.

E

F

Because, what's weird about that, if you just initialize the array for some reason that elements are undefined, so when you did a for each, they won't, they won't loop over the index and the index are still work, but the items won't be in their size, so it won't loop over anything. That's weird.

A

Okay, all right: well, it's a good catch before it bit me okay, so this is the this is where we decide. This is where we create the array of such pulses. So what we want to do here is go through the entire input space and.

D

A

That's gonna be so that the data that, where do I, need to know how big in the input space is, it should be in now it's in data.

A

It's in the encoding.

F

A

F

A

A

That we certainly didn't coder oops any.

F

Question for you is a are you? Is this just like a quick array object, just stuff hold this list of potential bulbs or just gonna, be like an initial spatial board. So it's.

A

Just a quick array and eventually.

F

A

What I'll use for deserving spatial cooling, eventually the same concept but but I'm not trying to optimize anything I'm trying to make it understandable. So.

F

Do you do you prefer having an array indexed by the input space that has like a list of all the columns and their permanence, or do you go from the columns and then each column has a list of all the input space, Maps yeah.

A

So so each mini column here, because this is a mini column, it's going to have a potential pool. That's an it's a list of indices towards the input space from that mini column, to the input space.

A

So this is going to be a random selection of cells, indices from that input, space and I guess I go.

F

The other way I go from the input space. You.

A

Have it called, but it matches better the diet or the the concept I'm trying to share here is maybe maybe you're right. Maybe it's because I'm it doesn't matter which way you map it, but when you're doing the computation that makes more sense feels like to be centric to the layer of cells.

A

Alright, then think.

B

A

From an input space centric place, because we're going to think about it from the aspect of many columns, how it's related to input space and then many columns how they're related to themselves because it when we get to the temporal memory portion of it, we're going to talk about how those cells are related to other cells within other mini columns.

F

So I see that as to supper, okay, I got you yeah.

A

It's two separate, but but I feel like it's it's if I make I need to make it layer centric. You know a relationship of the layer to this versus a layer to something else to itself so I'm gonna do I'm gonna. Do it this way, I'm gonna, say each each mini column has a relationship to the input space. It has a potential pool potential connections.

A

It might have a connection on that's what this is gonna represent, so so, let's create this should be a easy, one-off function to write, create potential pools, great potential pool and all we need to know is the input space, size and.

A

The connected percent, so this props connected percent and however I get the size which I can't remember and coding that in I think so then it's this dot and coding den so I need to create this function.

A

It's not even related to this. You know there we can create this river just connect the percent and size connected.

A

So this is going to return an array. Let me work through this in my head for a minute, all I need to do is for each for each potential input. Just you know randomly select randomly create a number, and if it's above or excuse me below the connected percent, we will return that index. So this should be a list of indices whatever, like so I need to go through this and again I can do it the same way. So I don't really care about performance right now, right I can do the same sort of deal.

A

I want to do something so many times and I want to decide for each one of those times.

A

Let's do this step by step out, let's just call cool and we'll return, the pool at the end and then for each one of these didn't like that.

F

We tend to be a map right here. There's never come the map.

A

F

A

Gonna return a list of indices.

A

The pits the pool.

B

A

Be a list of index indices indexes to the input space which ones are available for connection? Why isn't.

B

D

B

This doesn't work.

A

Obviously this is maybe sighs.

A

This is, did I do something wrong here.

A

Just wanna I just want to loop size times and I'm going to decide for each one.

A

Yeah for each for each index, if I'm going to if I'm gonna, know this, how did I do this and I did this in HTM school I can't remember how actually.

B

I think I just asked all.

F

Right I think you just need a semicolon in this case since, but does a lot of what I use is not as like a lip, no guys a lot of things like this. Like times all kinds of functions, I.

A

Sit low a lot I got rid of it when harmony came out as I thought, I could I thought everything would work them.

A

So really I need a okay.

F

So what you probably reduce- you probably want to reducer here.

E

A

Let me let me think this through: this is for one mini column: I just want to loop through every bit in the input space. I just want to select basically a percentage of those bits randomly a percentage of this into so for each one, I can quickly choose a random number and if it's below all I added.

A

So in your chant you're saying I should do I should how would reduce help with us for.

F

Building the return object, so you basically as you you start with like an empty object, would what's what's the object supposed to look like that gets returned one.

A

Dimensional array.

D

A

This is for one mini column. This create potential pool, gets called for for each mini column, I.

D

F

This is the easy thing, so then, let's.

A

Not do it with reduce cuz I can do it. Just with this. All I have to do is say that random, equal random, isn't that gonna give me between 1 and 0 and.

A

F

I just I just use this, but, like you do the same thing, because I use this a lot who plays like just a P.

A

B

A

The person that's the actual chance, but I need to create I need to call random every time. So I have to do this in here. No.

F

No, you would just call Pete, you keep that function up.

A

Oh okay, I gotcha, gotcha, so you're the functional so we'll just so you're just saying. If P of connected.

E

A

Then, or is it my Python or JavaScript, is it ad not a pen, let's.

C

A

I need to index here just a sec.

A

Okay, so this should give me potential pools.

F

So this this won't hold any like permanence values, just.

A

You know we haven't even gotten to the permanent values we won't today. This is just to select what possible connections we might have firmness values within its each one will have a permanent value at some point, but for right now we're only showing the pools okay.

A

So, let's at least debug here and see that we have many columns.

A

All right, they.

C

Call it's not defined.

A

A

A

Somewhere else, I did that too I think.

C

A

So this many columns.

A

Such a full array of nothing well one of just zero. We got something going on here and let's check this out. Let's stop here so creates potential pool. We've got to connect a percent point, eight five, a size of undefined okay. So that's the problem.

A

This start coding, it doesn't exist.

A

Right so we it has, oh I have to I, haven't coded, though it should exist, so I, don't know why it doesn't and code here constant coding this up coding. Let's stop there make sure we're hitting that first, you are. This coding equals something so there's we have this time coding.

A

Oh I, don't have this. Obviously the encoding needs to be sent in because I'm pulling it I pulled it off. No, no, no, all right it should be. There should be unless this was correct.

B

Not sure why I don't have is.

F

A

Calls this I'm fine, that's weird! That's.

F

Because it's it's not bound to this, depending on where it's coming from well.

A

It's coming from update.

F

And then, what calls update.

A

Component did mount and component did update I, don't.

E

Know just component.

A

Did update, it should be bound.

A

F

E

A

What was I sitting for this time? Encoding yeah, it's they're, just uh coating us here so.

A

Still, this still doesn't seem to be working. It's because I've got another function here, hold on.

A

This right so I think I, don't understand why you need to do this, but that doesn't make sense to me, because this is already it should already be bound I'm, calling it from another member function through this. This shouldn't be bound now. I can understand if this doesn't work, because let's try this actually, let's just say in because this is what we really want in this time. Coding down or I should really call it size, and then this should really call that.

A

I think that this issue is coming from right here.

A

Size is still undefined. This doesn't make much sense to me right now, because oh the encoding has no. Oh, it's linked. That's not in its length! That's what the problems.

D

C

A

Check that out, but I should at least have this dot many columns in each one. Hopefully, now has a fishbowl yeah one two, three four five six phone so but it includes yeah, 85%, okay, yeah, two, fifty two, forty five to fifty three I, don't remember how many are there, but it'll look pretty obvious when we get it rendered, whether it's right or not. If we make this, if we change this to like 0.08 five percent, it should be obviously smaller.

A

But we'll see we'll see you in a minute: don't need to rush run through this turn-off.

A

Try to turn this off: okay, okay,.

D

So let's render let's.

A

Render some things so I'm gonna take a page from this book, and this is initially.

D

A

Look silly just gonna render, on top of each other.

E

Let's see if I can maybe.

A

I should put the render functions closer to each other.

C

This updates and.

A

They decided to clean this up a little bit but spent some time doing that in a moment.

D

A

The render component and mount update I feel like she'd, be she'd, be up at the top.

A

Okay and we have update some potential polls and code under input space under potential points all right, so this is going to be.

E

There's a pool I.

A

Think that's what I called it. No mini calls right.

B

A

This is the mini column. We're gonna render the potential pool overtop of the coding, so we're gonna actually wants to yeah. This makes sense yeah, so the pools are actually be on top of the coding, but we have to have. We have to have a mini column selected, so there's going to be a selected mini column and the mouse-over is going to change that and that's just going to be an index, and then the surrender potential pools is going to render over this existing input space.

A

So we're going to have two two arrays of rectangles written into this input space, one. That's the combined encoding, that's gray and another one which is a yellow overlay on top of it and they're both going to be on the same space. One we've already got, the other one is written as render potential pools, we're going to put right on top of it I'm, so we're gonna have a group of cells over top of another group of cells.

A

So this isn't going to be many columns. This is going to be potential pools. I did I, didn't, set that up exactly right potential pool, so we're going to have an input space group and we're going to have potential pool.

A

All right and the potential pool yeah we'll leave it at that, and so this is going to be an actually in a different location, but these two are going to be overlaid on top of each other.

A

All right, so I've already done all this jazz. So in this case the fill is always going to be well. Not always. How are we doing our data? So so we can't use this encoding. We're going to use well we're gonna have to go, find the selected MIDI column. So, let's, let's hard good this for a moment, we know many columns.

A

Okay, new columns go, look at well selected, any column doesn't exist yet, but it will I'm just going to do this for now. This will eventually be a sort of global thing.

A

Well, just select the first one shouldn't matter: okay, so this is gonna, be a list of indices, which is different. Excuse me, which is.

E

Which is not what.

A

This this got last time, so we're gonna have to decide where to render the squares based on the index, not the count all right, so this is going to be a little tricky or or we could convert it into an array of on on and off bits at with, with all the indices.

A

I, don't know, if that's what I want to do- let's just let's just play around with this and see if we can get this to work so.

A

We know that all of these are going to be.

A

All of them are going to be the PP color, just a percent opposite central pool color the stroke, I'm gonna put none. This is gonna, be sort of an overlay I, don't think we need a stroke. Width and fill opacity is one we might change that. But, okay, so that's the X&Y. That's gonna be tricky, so width and height they're gonna be fine. So this.

G

X and y is not.

A

Gonna work right so it looks weird but I think it'll render something. So, let's check it out.

A

We're not I, don't know, okay. Well, let's see, let's inspect and see what didn't happen. So, oh yeah, so maybe we maybe we didn't select it properly. So potential pool and my calling render potentials. Yes.

A

Is it getting the right thing? Let's find out.

E

A

Is yes, so there is it's getting the SVG G element.

A

Okay, these selected mini columns 0, so we're gonna go with the very first one which has a potential pool. Oh yeah I have to call dot potential pool. So that's not a big deal right here. So it's not just this. It is also dot potential pool.

A

Let's break this out, so we could see the data that we're getting okay, so I think this will now render something, but it won't look right. Thank you rid of the debugger yeah okay. So it's like I said it's rendering something that doesn't look right because we're using that this is the number of indices, not the right locations of indices. So this is where we need to inspect right here.

A

So we're not looking at the data we're looking at the index, so we want to start looking at the data instead of it, so so I think it might be as simple as switching out the index for the data. If that's the case, that would be flipping brilliant. Yes,.

A

It's that simple, oh okay, that makes me pretty happy, but that's not 85%! That's like the inverse of 85%! So, let's, let's do this greater than right. I bet.

F

You change it to zero I.

A

Changed it to oh, I did you're right, sorry, yeah.

F

That should be less than you're.

A

Right right, how I should know now not to doubt you? Okay, that's right so whoops.

A

So now it should be very small. So, let's open oh wait, I changed something. Let's stand! Oh yeah I made it a 25 okay, so that's 0.85. Let's make it 15 that should be yeah, okay, cool, so that that's potential pools.

A

So it's not quite exactly how I want it to be. First of all, the potential pools are changing in each time step. They should not. So that's one thing we have to figure out. The potential pool should be created on initialization and then on update. They should no longer change ever again.

A

So let's try and see if we can figure that out because then, when we think.

F

So because the update right under this update, your calling yeah.

E

F

Should be a male right, yeah.

A

So so this is going to happen once this set up potential pools is only going to happen and component did mount.

A

A

So I have to do the encode first, but I can do the encode once I can do the code once it's not gonna hurt anything I think this will work all right. My Auto formatting is really screwed me up.

A

It doesn't like I, haven't gotten encode the encoders set up right. What else do I have to do opponent.

E

F

Can also just check to see if the arrays already initialized, if it is, then don't set up an issue and keep it in the update.

A

A

Hold on side I'm, not understanding why?

A

Why am I getting this air.

F

Probably cause it's on now and your you didn't create the encoder daylight. Oh.

A

Where do I create the encoders? Oh I create them here.

A

So this stuff won't be available and in component did not.

F

Island, it should be that.

A

Was code was choking because it said there was no time. Oh, we haven't gotten any data, there's no data, yeah.

D

A

D

A

Ignore, if there's no data, we just ignore not update.

A

I think I can do that because a.

B

Just going to say, if.

A

This drops that data time.

A

Okay, alrighty.

A

C

A

Every property potential pool of undefined and render petition pools once okay, so this is actually this might be legit, maybe I, don't have a select mini column, I think I'm, just getting things on the wrong order. I'm a rendering potential pools from update, so I have to render infant space, but there's no many columns yeah, because oh yeah, the.

F

You need to put that in code back in the mail I need.

A

A

The I no longer have setup potential pools anywhere, so setup potential pools needs to go in. He did mount like you said right here, I, don't know I accidentally.

C

A

Thank defence, so there's no encoding and.

A

I need to have an encoder need to have the encoding setup before I can do the potential pools. I need RG.

F

You can just put that as the else and update me the first time then do all that.

E

F

Diddy encode bird.

A

If there's no time wait a minute I'm only doing this, if there is time, if there's note, if there's no data, there's no point doing anything because I don't have an encoding I, don't know how long the encoding is. I can't do anything.

A

So the first time.

F

So what what did you do? First, you put this sets up here, but you said you needed the encode to be called for.

A

The only reason I need encode to be called first is so I know how long the encoding is. That's the only reason, that's something that could be hard-coded honestly I mean I, can look, I, can look at these encoders and add them up, but why don't I just do that?

A

I'm gonna. Do that I think that's probably because, because the size is, if there's no encoding, I can still calculate the length of it and and that's just by going to the encoders and and counting the W, so I can go like I know. This is just not efficient.

A

You're saying if no.

E

A

Okay, this I don't know you're, saying that will only run once then, as long as it only runs once future.

C

F

Know actually, if you don't actually just now, because you only needed that one, it was in the mail right.

A

Yeah I only need to set up central polls to run once and then mount it should run in mount, but the problem there is that we haven't coded anything in mount and so there's no link, so I'd have to manually count count the links, which is what I was about to do. Okay,.

F

But so this will come in here. This will create the encoding yeah.

B

F

Then this will say: have I initialized the Pretender Bulls. Yet, if not then set them up and then that and then the encoding will be there when I get to setup thento book, you just use this yeah.

A

That should work missing, sighs because I took it out so then I just have to said, like I removed. This coating.

B

A

B

A

This shouldn't change now there we go good good good. This is what I wanted to see. Okay, let's change the opacity, so you can see this little better.

A

A

Right for potential pools: let's make this point five, so we can see the gray under it there we go so now. This is perfect. This is.

B

A

Closer, this is what I want to see so one potential pool and then- and we show the input streaming behind it. So this is like the view of one mini column. It only sees what's within the potential pool, and this should probably be I'm going to move this under what I bet. This will look better like this and then make the oh I can't color whites for the.

A

Or I could there's a couple ways: I could do this.

A

Potential pools I'll make.

A

100% and then put oh quit, fill opacity here we go how close I have to play around a little bit with the opacity of stuff, but it's it's something like this I I might even change. So the colors look a little bit different, but well I'll mess around that later. But this is basically what I want. I want the the data running in the background- and this is one potential pool for one mini column.

A

So after this we'll well we'll make this smaller and we'll move it to the side and we'll start working on the mini column, diagram that we can hover over and I mean it's. It's largely unnecessary to make this but I feel like it's tangible. When you can move through the space and click on a mini column or select the mini column, you can see each one has a different potential pool. That's that's what I want to show.

A

So, let's take a break for a little while because I'm starting to get a little burnt out right now so I'm gonna put on this loop I'm going to commit the code that we have right now and push it to the repo sorry guys, I've been sort of ignoring chat. Now, remember you have the sounds if you, if I'm ignoring you to use one of those, sounds so that all remember your okay.

D

I'm, adding the layout.

A

Component pages CSS- actually I'm gonna do this I'll do this, so you can see, will use okay, I.

B

A

Color this whole potential pool class is new, I, think I've cleaned it up, but there's more stuff we're gonna do in here. So it's good enough for now, and all we're doing here is adding that I have this commented up for now, which is fine, so this works for me. Let's push this.

E

Initial potential pool.

A

A

Get push not even I'm, not even peeing and I'm. Just gonna push it alright.

A

So that's cool I'm, happy I'm, pretty happy with that. I'm gonna take a quick break and stretch a little bit.

A

All right I'm, turning my turning that off I'm gonna play some records. Instead,.

D

Ii, it's grateful.

A

B

A

B

A

Is my record cam can't see my face.

A

Wes Montgomery jazz, guitarist, I.

A

Love this I gotta play it I think that's Montgomery's!

A

B

Is one of my favorite records.

A

Ya did go way up this.

D

One my favorite.

A

Records it's great. It's really montane. It's such a chill! That's just.

D

A

Record I could listen to it over and over it's one of those. So that's what I'm gonna do.

E

A

Okay, so we got the the first part of this working, which is awesome, they've, simplified, a method in the meantime.

A

See where you're at.

A

Two of their time, given the connection.

B

A

Okay, so this is basically the same thing: you're just or just say, good potential pool. Yeah, that's easy! I can do that as long as it still works.

E

D

A

Right so it's just a filter yeah that makes sense. Okay,.

A

Onward onward, let's go to the next part. The next part is just this, which is basically we have to draw the other box. So let's translate these two groups, I think I'm gonna. Do this manually I?

A

Think it's okay to do stuff manually manually, meaning like hard coding, visualizations I've.

E

Done enough, visualizations no.

A

I think manual tuning- it's not only necessary, but it's I think you end up wanting to do it after a while, because the diagrams are so tiny different, like everything's a little bit different. So what I'm gonna do is I'm gonna split this space in half.

A

One way I could do it is I could add a translation.

A

I know that the diet, the the width is going to be 500. I really do know that, because that's a restriction of the design space that I have that I'm going to need to fill. So we could add a translation here. I think you do it just like this translate.

A

There's a transform, transform equals, translate.

A

Let's see if, let's see if this works, I shouldn't move it yeah, okay, so it moved it.

A

This is something that we can also do on the fly. If we want to do it, programmatically right now, I'm.

D

A

I'm gonna move it over half the space. I can do that for both of them and then I'm going to make the cell height smaller.

A

Keep in mind that this, the the actual size of this is like this.

A

So we can't get too small, and this is 500 pixels. That's not it's not that big right, but it just it seems to be a pretty typical width for visualizations I. Don't want to make too much bigger I want it to be able to display on lots of screens.

A

So, let's, let's go find cell width.

A

So, as we were doing, the input space.

A

Find the right place yeah so in both of these we're going to get the cell width based upon the diagram with and the number of columns, and that seems to include the padding.

A

So let's just divide this by two again.

A

Okay, let's divide the other one by two.

A

All right, it's not it's, not amazing, but it works right.

A

Okay, so at least we have the space put as we put it aside and then we're going to render the other one, which is of any columns so to render the many columns. We don't need any information, except how long how big the mini column array is. So we have to make sure this is done. After this, after we set up potential pools, we can set up many columns.

A

I, don't even need to set up any columns. I can just render them.

A

Cuz they're already set up all that's all it's really gonna be is a bunch of white rectangles with a mouse with a hover effect with a hover interaction. So it's gonna be really simple, really similar to render input space.

A

So, let's just copy and paste that hope my mics on is everything good. You guys hear me: okay, it's not the ray LaMontagne's, not too loud render many columns. This is good. This is good. This is going to be many columns. Finally,.

G

I'm sorry, Dave I'm, afraid I can't do that loud.

A

Enough forgot I had the volume I'm so loud, okay,.

C

A

Is that what I called it just many columns? Yes, many columns.

A

This is fine. This is so wit. This will make it the same size all. Actually, um this is gonna be a lot it's smaller, because we're not doing encoding this. It's gonna be many columns, so these cells will be smaller if we're using a buttload of columns. It's a technical term.

A

This is gonna be by index, not by data, because these are simply objects with a potential pool. So all we care about is the index, not the data, so this should render something right here.

A

That's interesting: it's not exactly what I thought it would do.

A

D

I think there's just an.

A

Artifact of the so everything is gonna, be the fill. Let's see, the fill is gonna, be none every time at this point, so that should all be none, and these should be the same. This should be a square I. Don't know why it's not a square. So, let's, let's step into here.

A

See what happens so with is for diagram with us for 20 columns 45, as the many columns length is 2048 35. Oh I know what it is. It's that we're still using the encoding as the data source. It's not. We want to use the many columns as a data source, so that should fix it.

A

That's what we want.

A

Drinking my water.

A

Okay, so now we need a mouse-over, we need, first of all, let's do a mouse-over and we're going to be able to select, let's write a function that will select a MIDI column based on an index, and then we can call that function.

A

Render render render setup update.

A

And if this works I'll be ready to render render lots of rendering this is alphabetical. Okay,.

A

So, let's make one called select mini column by index like this whoa. What happened.

A

Keep jumping somehow to David's position.

D

A

Stop it's not you David I, don't know. What's the my vs code is doing something strange?

A

Okay assuming so so this select mini column.

A

We're gonna want to property here, selected, mini column,.

A

We'll just start with zero. Actually, let's, let's start with.

A

I'm gonna start with something we want to show something so we'll start with zero, and we.

C

A

Somewhere else.

A

Where are we setting that there? It is no sign of potential pools.

A

That's right so now we can take this out and just put this got selected many columns. Okay,.

A

Okay, everything should be okay, so one other thing we want to do before we get too far is where we render the mini columns. We do want the field to be different in one case.

A

Okay, so the fill will be different if the index.

A

So if the index is the same as.

A

Well, let's, let's put this here constant selected many columns, this select did, did I. Do that right, yeah.

D

B

A

Selected many colonies, Coffee sounds good, I don't have any I could make some actually. So if this is a selected mini column, we're gonna return.

A

Selected color, otherwise return.

B

A

No, no, no none. None.

A

Tim's always coming in at optimizing my code, all right. There we go he's in line. The whole function works for me, okay, so now we should clean the spit up.

E

Just like this right.

A

So now we should see the first one: I don't have a selected color. So let's.

C

A

Color equals red for now Falco. You can look me up a nice red color. If you want there, it is okay, so that's the first one. So you can see we got it right there.

A

So that's good.

C

A

So now we want an on hover and we're gonna. Typically we're gonna. Do that and component didn't mount.

A

B

C

B

D

A

Going to do that here down in where we've rendered the mini columns. So here's where we will add a.

A

Mouse-Over handler on this G, so we can do like G dot.

A

Okay, how about done this before I'm gonna go look! How I did hovers.

D

Before as I know, it's we.

A

Did it in simple scalar encoder.

A

A

Your outfits, all those where's, my mother, on this- handle out the powder area. So how did I set this up? uh Sir? So that's that's so I need to set up an a mouse move.

A

Yeah so I'll set up a mouse move over the whole thing over the whole SVG yeah and then tell it to basically this exactly yeah, and this is selecting whether it does one or the other. So this is pretty close to what I want. So I'll put this in potential pull down here on the SVG.

A

This so onmousemove, and in this case I, don't think I need to select anything I'm, just gonna I'm, just gonna defer to this that handle.

A

It's called handle mouse move.

A

Don't even need to do this now, since I'm, not using that I here can just do this now. I need to add the hello Mouse move function like.

E

Handle mouse move.

A

All right and now all this stuff take out, and it's just.

A

Just want to see this working okay: where did we break he's not defined yeah? That's because I don't need this that and allow smooth RV console logging? Yes, okay, so we're getting the mouse move. So I only want you know. You know even better I could put this on Mouse move on the mini column group right there, like that. That's.

B

A

Everything because then I'm specifically targeting only one so when I mouse move off this, it doesn't matter but I mouse move over here, I get it okay, so that's great I can do that. So.

A

And we don't work, can I get off of this yeah? How did I do with it last time? I can get e paychecks and what I really need to know is the index if I could get the current target. That would be so easy. I don't really want to translate. Pixel cursors location I would rather just get the target I'm over, but that seemed to be a tall order. Last time I mean ideally all I want a top target, but that doesn't work.

A

I wish it worked so bad. It did work perfect. Why didn't that work like that did not work? The last time I tried this. Okay, so can I identify its index. No, but I can.

A

And I can add, I can add a data attribute to it that I could just stash the index on probably the easiest thing to do when I'm creating this mini column array.

B

A

Just say last: bang attr.

A

Actually there may be a d3 data. D3.Js data attribute adding data attribute condition. No.

D

It's just a powder: okay, I, just wonder if there's a shortcut.

A

Okay, data index and.

E

That is going to be.

A

B

Do I get my dear index.

A

Did index yes there there, you see them right here, data index, all right, so I'm getting me! This should be 2048 or 2040 siding this.

B

A

Right, perfect, perfect, all right, so I can get from my hover event.

E

Should be able to say II.

A

Actually, just this dot selected mini column equals P dot target.

A

I'm not sure the scar.

A

It's a good attribute, no selected mini column of undefined.

A

Yeah because it's.

D

A

Bound so this I do understand why I need to bind this.

A

Now I've forgotten how to do it?

A

How was it was.

E

A

D

A

Glad did I did in another, one I think David did it for me. Actually Drakken, then and I was in losing scalar thought.

A

That's scaler over that diagram. How here does handle cell.

A

Oh, oh yeah I did not have to do with that in this one, because I.

A

Called it differently.

A

Because I actually called this dot, so I need to bind this properly and I forgot. How to do it? Well, then, I'm sure quickly will react. Data.

E

Function, binding Jack, yes, function, binding.

A

It's this really easy pattern. I just can't remember the there, it is equals friends, that's it so people's there, okay! So this! Hopefully this will bind.

B

B

A

Pump should be selected mini column.

A

D

Yeah, it's zero because.

E

A

Never actually updating anyway e target get attribute data index. Let's see if we can do that.

E

E

A

Can I get to my console? Is it really way down there e dot.

A

A

I never know what frickin object.

B

I actually have.

A

A hold of get attribute that should be on a yeah.

B

A

Attribute data index yeah.

C

B

A

That works, get attribute data index number.

A

I don't see this should work, I think.

C

A

So in this case, why why not.

A

Selected mini column, this not selected many common this. So it's still. This is undefined. I don't have this okay.

E

David I think I need you.

A

To help me out here, I'm having a binding problem, I thought see down here in my return, I put on Mouse move this handle Mouse move and then I tried to do this, but it didn't seem to work.

A

I'm close, it's just that this I thought this would bind and it didn't bind.

F

A

You see where I'm at.

B

A

A time where I think I have a problem, this onmousemove should.

F

A

And then call it with the okay yeah, okay,.

F

Yeah then you know, then this can be a regular method. It doesn't have to be it then.

A

I, don't have to make it that thing all right. You can.

A

All right, I thought I would that's the way it was and I was confused by it, but this will work. Fine.

C

Yeah something's still wrong.

B

What's happening.

A

So here II.

A

To pull a selected mini column, this deselected mini column, whoa.

E

A

B

A

That's why, on top of everything else, okay, now it should work and then we'll just set selected mini column and we'll have to rerender to see it.

A

All right, yeah now we've got the indices flowing. That's what I want to see, and so at this point instead of logging, what you call.

E

Select actually did I treat I.

A

Create a function called select, many column, I did but.

C

I, don't really.

A

Need to what I'm gonna do is just run and update that one thing so I set the selected mini column and around this dot render many columns things work.

D

A

We go okay, see how oh I also need to re-render this infant's face as well, because the potential pools will not update by themselves. Maybe I should just call update. Let's see what see what happens if I just call it.

D

A

That's right, that's what I want.

D

A

Everyone has a oh, it's, not one other thing, one other thing.

A

Update is not calling.

A

Render mini columns has to happen: okay, okay, okay, okay, hold on.

B

A

Think I need to put this here. I want to see if this breaks I think it will. What is that.

A

D

Gonna break no.

A

It didn't break okay, great, it didn't break. So we got two things going on here. We have one I'm moving the selected mini column and that's changing the potential pools that are showing, which are the yellow boxes and I'm not doing anything and the inputs changing so I can look at ones.

A

Particular many columns, potential pool or I can or I can look through all of the different potential pools, so every mini column has its own potential pool, which was one of the things that I really tried and trying to explain with this visualization, and we can see it's sort of all happening.

A

It's a little bit slow on the update, but it's probably because I have yeah it's it's because I'm at the console open. So this is fine. This is great now, when I click on one I'm gonna have I'm gonna. Do something different. That's the next thing. Okay, all right we're getting there we're getting there. Folks.

A

Okay, sorry, does that mean yeah? That's me.

A

All right so, okay, I'm gonna, commit this. Now all we did was change potential pools. We got a selected color.

A

Red is fine. For now, we've got the potential pool deal selected, mini column, render the mini columns.

A

Potential pool so with I cut in half, so we could fit two diagrams on the screen, so both of them are set. Artists are half of what they were in previous visualizations. Here's the render MIDI calls function, which is very similar to the other functions, but we're we're filling dependent on what columns selected.

A

Our past is also different.

A

Okay, that makes sense, remove that to use the local Steve and handle mouse move is now bound. Okay, so we'll also add a handle mouse click soon enough, but for right now, I think we can can push this added mouse hover of any column, selection.

D

E

A

Should be able to see our latest code deployment here, I think yeah. So this is the previous one. As soon as we push this, this one will go, live.

E

A

You should be able to see the latest version of this soon enough right here as soon as the CI builds, and it should look like this alright. So the next thing is this I mean we don't have to do any more data we just have to. We don't have to deal with data anymore. We simply have to decide whether to put something in the rectangle and we're gonna do that dependent on whether the current combined encoding is overlapping.

A

The potential pool of the mini column and and like I, said I would like it to look like this. This bottom part here so.

D

A

To have check, marks and X's something like that, so on top of this and I'm not exactly sure how I want to do this, we could we could do more. We could do this with text elements actually and do one more overlay on top of everything else, and just do an overlay with text boxes and essentially use the same logic we have been using. That seems like to be the easiest thing to do, so.

A

Let's try that first, instead of drawing more rectangles or trying to like throw some more things inside of the rectangles, let's just simply.

E

A

Create one more of these geez and I'm gonna, put on top of everything else. I think I, don't know what's on top, so this will call. This will be called I, don't know, I, don't know what to call this Czechs Texas silly it it it's supposed to.

A

It's an overlay for input, whether to note whether it was overlapping with a potential pool or not. Let's, just let's call it overlap overlay over.

B

A

Or just call it overlay, I, don't know.

G

Overlay overlay.

A

Works for me and I want to think too much about this okay. So if we have an overlay, we're gonna have to render the overlay and it's only going to render if we have potential pools so we'll do it here.

A

No, no, no rope we'll do it here. Overtop of potential pool is we're going to say, render overlay.

A

And let's just take the potential polls and copy it and paste it and update it, render overlay okay, so we're gonna. The diagram with will be fine by the way. Anybody, if anybody is wondering we are building HTM systems, here's a little bit of a description of that and a link to more information.

A

We are rendering an overlay so that we can show these green and and white marks over top of this visualization we're working on here. So I've got it. So you can, you can select a mini column. It will show its potential pools on the right.

A

This hovers a little bit slow or off something's, weird here and I'll deal with that later so, and this should update live with checks and X's.

A

So we're not going to use recs, we are going to use text text text texts, so this is not going to be erect. I think all this is fine, except this. This is a potential pool. We call this overlay I. Think overlay I think we're gonna get this done. I have like one more hour before I really I need to take off and I think we're gonna get this done. At least this bit that it better keep show I might not get the legend done, but that's okay, we'll have the majority of it.

A

A

Okay fill stroke, we're not gonna, have a fill. There won't be a stroke. There will not be a fill opacity, there will be an x and a y. There will be a width and a height.

A

What there will be inside of the text. There will be an add, checkmark or an X okay. So, and this is going to be a text, this is going to be a text and I. Think, let's do a quick check, SVG text element I do believe you can give it X&Y, yeah, XY and then even yeah, or you have more control over that.

A

So I think X&Y is good and we're just simply going to put a checkmark or an X in each one of these okay.

A

And that's gonna be I. Think text I think this.

C

A

I can say, let me check. Let me check on my Jack d3.js text enter.

A

Let's just say that text text elements.

A

This is just SVG text element, but how do you I think it's just HTML text, HTML I think you just say: dot, HTML pen or now it's text, so you put dot text and you yeah yeah, I'm other so dot text and we can. It will get a D I, so we'll get the data and then we're going to decide based on what we return here.

A

So if I return here, hi that should write hi a bunch of times over top of all the stuff I mean it should I probably didn't do it right, but that's what it should do.

C

Once I get this.

A

G

A

This is overlay, render overlay overlay translates, let's see if we can find it. It's gonna, be here inspect.

A

It's gonna be on top hair, does whose the overlay, but there's nothing, either some o te. How to do that.

A

Te XT, that's weird: okay,.

C

There we go so there's all the highs.

A

Obviously, I want this down here on top of everything. Okay, so there's all the highs.

A

So I'm just writing on everything right now. This is weird because there's still some tes hair still. What is this te.

A

Something's wrong come on.

A

It's not refreshing just a moment, there's something wrong like it's frozen there yeah something froze. Let's go back to it, special pooling, there's all the highs. I want to see. I expected there to be two texts in every box and there's not text in every box. Why not? There's not text in every box? Oh I.

B

A

Okay, because of the data that I'm sending it is this: what I should what I need to be sending it?

A

Okay, so now I need to decide how I'm going to create this data object and what's going to be in it. Basically, what I want in it is.

B

A

I think I might just pre-process the data a little bit and send in an array that basically has weather what should be in it. Should it be an X or Y, but then.

D

C

A

I, don't think I. Do you want to do that.

A

Let's try this I just want to see what what happens when I do some of these things? Okay,.

E

D

C

A

So zero one, two three four.

A

We also do text SVG, sighs, whoops, sighs, SVG font size such as font, size, 16, yoga.

C

C

A

There we go so this is. This is weird. Oh.

A

Oh yeah, that's right, so texts, text elements are counted from the bottom at the top. The wrecks are from the top. The text ailments from the bottom so I have to add the cell width, because width and height doesn't translate here. So this doesn't work or width and height I. Don't think works for text. So let's not take that out and it shouldn't change anything. What I do need to do is for the Y I need to add.

A

The cell width so that I can lower them all down. Whoo. That's not that's! Not! Oh! Oh! That's! Yes! I multiplied! It I think this there we go so now now we're more we're better aligned here.

A

So, interestingly enough, it's not displaying the ones. It's only displaying the ones.

A

Okay, so I'm gonna need an initial array of every single element in the input space. So I can't do what I'm doing here. The data that I'm going to send it is going to be the encoding.

A

This encoding dot length. No, no, no, no just the just.

C

A

So there won't be a D. Alright, excuse me: there will there there will be a D, but it, but it will be on or off so this is not. This will still work, but it's the text bit that I'll need to sew.

A

This should still work, we're not no it's because I'm relying on D and what I really want to rely on his I hear that's right, so so I'm not relying on the DA I'm, relying on the index again right so now, I get a count for every single one, which is what I want audio and discord is super choppy.

A

Like I, don't know what to I, don't know how to fix it. Sorry.

B

A

Could mute it and listen to the twitch stream, which hopefully isn't that bad.

B

A

Okay, so now I need to decide what to put in these texts, programmatically and I'm, going to do that inside of this text bit here, I'm going to decide so we're gonna have.

A

So let check equal.

E

A

Actually I think I don't want it. Emoji I want a symbol. I.

A

Think it's just checked heavy checkmark. That's what I want heavy checkmark there's another one called heavy X! Isn't it there? It is heavy ballot, X, okay. So these are the two things that I want check.

A

X ballot, that's what it seemed to be called. Okay,.

A

How did I get to the symbol? Viewer? It's it's. A Mac thing show emoji and symbols.

A

I think ctrl command space will also give you a yeah ctrl command space, and then you can expand it. That's on a Mac, so in one case, I want to show the check in another case, I want to show the ballots so I'm gonna make this the check. No, no, no out initially is gonna, be nothing and then I'm gonna put the check on if this index is in the potential pool and there's currently a bit in the encoding okay.

A

So if the encoding is on so first of all, if D, which is on or off that I know, that's one thing: if, if D- and let's do this first, if D out equals check so this this should at least then I'll return out. This should at least give us checks for that's not what I wanted I.

E

A

To be, what is D D is gonna, be should be on or off.

E

I think it's I think I'd.

A

Have to say, if D does not equal undefined.

A

No, okay, hold on.

A

Something weirds going on D is a color. Oh that's right! Oh I forgot I did this so that doesn't make sense. This done coding is I, don't want the colors here. This stunts encoding is full of colors. It's already full colors. That's right! I forgot it's already. Full colors I I want the original encoding for this I. Don't want the one that's full of in cut and full of full of colors. So when I do the encode function here.

A

Yeah, this is where I'm doing it.

A

I, don't really I don't want to have to look well, I know what I can do I'll just say if it's not the off-color, that's easy enough. Okay, I.

C

Can do this this.

A

Is we'll keep it simple? Keep it simple if.

E

It's not the off-color.

A

A

If it's not the off-color will output a check, I still have a debugger I. Do it's not the off-color up for the check here we go all right.

C

All right so now this is just the.

A

Encoding basically, but this isn't gonna know if I need I, also need to overlap the potential pool. So if it's not the off-color and.

A

So now I have to think about the mini columns, this dot any columns, dot, I, don't know many columns, ah I.

A

A

Selected mini column, right selected, mini column, selected mini column, so net dot.

B

Potential pools.

A

Dot potential pool, and now we have to look at the index index of eye.

A

Close, it kind of be probably many columns of undefined. This is undefined.

A

Right, so let me just get the let me let.

G

A

The this is in render overlay.

A

Yeah, so this I should be able to say this whole thing: I should be able to just call potential pool, and then I can find that here, I.

E

A

Okay, it's not quite not quite let's see, let's see here, potential pool.

A

What's it look like okay, it's got an array of indices.

A

85% index of I and X of zero, oh.

A

I'm not checking I'm just saying indexof does not equal undefined, if so, if I caught ya, otherwise it's undefined. So if this does not equal undefined. So if, if is if the off colors I.

A

Think this is it: let's see if this works.

E

E

A

I'm gonna I'm close here this is bear with me, I'm only going to put a check out if I got this part right and now I got to look at if the potential pools the potential pool has an index of I that it's a check.

A

It's not equal undefined. Okay. So let me what if I do this? Oh.

A

That's working.

B

It's working, it's.

A

Working yeah yeah, it's working! Yes, that.

C

A

Perfect yeah, yeah, okay. So oh all the checks. Yes, I changed.

F

It to include, but are.

A

You you yes, okay, includes, instead of and okay perfect, that makes sense. Well, it works now. No thank you. Okay, so now, I just have to have the condition for the X. The X is our condition, so these are only showing on the ones that are going to be observed. Inputs and I want to put the X's in the in the overlaps where they're not observed so.

B

These are only.

A

Going to happen in.

A

Where it's not off so I think I can I can keep this here and create, and then this is.

A

Like so alright.

B

B

A

So the the entire input should have yeah I think this makes sense, and now so I just got to make one green and one white and make them a little bit bigger, and that should be that all right.

A

So let me change. Let me tweak a few things. Is we're good we're good we're they're good teamwork Dave's? Still in there? Can you clean it up? Yes, please clean it up, go for it I'm going to mess around with some styles and colors. You can clean that up. I think all I want to do here is maybe update the font size pan, give it a little bit of padding.

A

So that's what I'm just going to mess with your padding a little bit.

E

A

A

Now, of course, this is an economic, not quite gonna work until then. In the meantime, let's make our font size a little bit bigger I.

A

Always write things right, code, convoluted and then come back and refactor later.

A

See what we got here, okay, I, can still make this a little bit. Bigger, I think the X. This is not quite the same size as the check okay, but let's do the attribute for color.

A

And again, I think di.

A

We're gonna have to do the same thing. Cuz we're gonna have to make it green. If it's on otherwise off. Let me see text SVG text, color, I,.

A

Think there's a color.

A

It's the fill all right, there's yeah.

B

A

Fill fill fill so not color, it is fill okay and we're basically to do the same thing. This works for you how that works. That's great Dave! Thank you, so I'm going to do something similar here, we're going to fill.

A

We're going to call this green, otherwise,.

B

B

Otherwise, none.

B

A

Brooke expected token to 35 what.

C

About myself here.

A

A

Got a syntax error somewhere I can't figure out where it is. Is it text.

A

What is this something.

D

E

It doesn't make sense.

A

Oh, is that it my extra princess right that makes sense that does make sense: okay, okay, so the white worked and the green did not.

D

A

Alright, let's figure out why the green didn't work.

A

F

E

Sure the green.

D

E

Was that David.

F

Are you sure the green didn't work and the other colors just my school.

E

A

My just itch I, don't see green.

E

A

Says fill green but I'm not sure why it doesn't look green.

A

B

F

Look is it so red.

A

Even why it doesn't work. Oh I, wonder if the checkmarks heavy check symbol can't be colored I.

A

Don't know, that's that's odd. I, don't know the other.

E

F

A

F

It a make it a V or something similar yeah.

E

A

Capital V well.

B

I'm gonna leave it.

A

To simple's there yeah.

B

D

Is it's the check.

A

Mark symbol can't be coloured, but the X can that doesn't make sense. Okay,.

E

Let me see if I can find.

A

A different symbol like a collar.

E

A

You can't use the heavy check. Maybe this that's the heavy check, maybe this one just a normal check right.

A

That works, so in that case, let's set the font size, higher and I hope I hope I can do font weight, I'm, not sure, but I'm gonna try, okay, SVG SVG text font. Wait not sure if that worked or not font weight, yeah.

B

A

I worked and it's bold, what can I give it normal or bold that it you.

F

You can either give it a bold bolder. A number yeah.

A

F

500 600 700 all.

A

Right, let's usually, don't use the numbers, let's see what it looks like I want it to be pretty bold.

A

Older yeah, okay, it's good enough for now. I am gonna, move them a little bit so minus ten. That should move it up a little out to the whoa. That's not what I wanted I forget! These are really small.

A

More like plus two.

A

Okay and then this wise we're gonna do three, maybe two there we go all right, that's pretty close. It's pretty close to the original design. Originally I was only showing that on click, but I could just show it all the time I.

A

Mean it shows you how certain columns will develop an affinity to certain areas of the input space simply because its potential pools it won't, allow it to connect to other places that have prominent patterns in them. Let's, let's change this to spatial pooling hold on my wife's text to make the.

E

B

A

A

Spatial pooling: where did it go there? We go I'm, gonna change this. Let's make this smaller. Just wanna see what this looks like with some different parameters for fun.

A

So that's more like 50%, so you can see a lot more being I'm gonna change. The opacity now I can mess around with some of these opacities.

A

Okay, this is the input space. What if I made it one.

A

It's on top, so I feel like the input space should be on the bottom and then we should render on top of it.

C

A

So we want this to be up one. The potential pool should render on top of it now, okay, and then we make the potential pool opaque.

A

Or less less opaque sorry, so it's opacity I'll make it like 65% all right, and then we have the X's on top of everything and it's fully opaque.

A

Yeah I, don't even have an opacity here that looks pretty good I'm, pretty happy about that I'm actually I'm going to.

G

A

The opacity of the potential pool, but let's move it down to 50 by five great all.

B

Right successful.

A

Day for sure, a successful day, I got everything we wanted done, David thanks again for your help. You're really helping me, keep my code clean and doing the react thing in the right way, but I like this there's there's a few weird interaction bugs, but we can we'll deal with those later, but this is exactly what I, what I wanted to show as we continue with the spatial Pooler, we will keep this idea of having a selected mini column because we'll have future diagrams that build upon that example.

A

After this again, this is also given a selected mini column, we're going to show a heat map. This is where we to get into permanence, and so the next, the next session of BH TMS, which I think will be Thursday, I, think I already have this scheduled. Only the next one is going to be initial permanence diagram.

A

That's a little bit premature, because I've got to do this this bit of the receptive field here this bit before we do initial permanence is we'll see we might have to sort of do these both at the same time, because this is sort of a precursor to this. But in order to create the permanence is we have to figure out a technique to instantiate permanence --is, so these are sort of part and parcel with each other. But this is the simple version.

A

Given the permanence is exist, we should be able to upon selecting a mini column, show all those permanence is in sort of a heat map, type of way over top of the potential fools. So, instead of these potential pools that were showing here, instead of putting the checks and the X's over top of them, we will put- we will render them as a color between yellow and red or green and red, or something that indicates how permanent that synaptic connection is.

A

That's the idea, so that'll be the next thing that we do, but we have to establish initial permanence is first, and that takes a little bit of thought. So that's what this whole thing is about, but essentially I already have this I've already built this visualization that we're going to be working on in the input space here and there's there's a real, easy function. You can do to establish these initial connections using just this sort of what did I call it.

A

It's just a simple distribution: it's just a normal distribution around the center point and you can affect how spread how how wide the spread.

B

A

This is not something that's in nupoc. This is not part of new pic. This is something that I just came up with, because it's it's not hard to come up with schemes to devising initial permanence. It's just you could do it in a lot of different ways. I. This is this goes along with the spirit of HTM, which is basically to try and create and I'll talk about this when we get there, but we're trying to create sort of a distribution.

A

That's heavily weighted around a center point, because you want you want these connections to be on the brink of learning. You know, as soon as data starts flowing, you want connections to strengthen right away and others to fall, fall away straight away straight away.

A

So, like you, don't want something like this. That's, oh, that's too many connections. This is not enough connections, there's a sweet spot. You know and depending on how how you want your learning just to kick.

G

A

You might want to change this around so anyway,.

D

That's what we're going.

A

To be working on next, so keep keep track up. With my events, don't forget about Wednesday Bay Area Meetup. I will be live-streaming this from the office. I'll have a short talk, I think Jeff Hawkins will also have a few things to say.

A

Yeah I think that's it okay. So let's commit what we have got here and I'm pretty happy with how far we got okay, so last bits was: we've rendered the overlay I changed. Some opacity changed some opacity we're under overlay function and then we just rearranged. This put an input. Space and overlay put the overlay on top okay, so I am going to let's look and see. If we did this, we still don't have we still don't have I mean this is totally done like we don't have a legend so.

A

See I'm gonna, I'm gonna enter a note about this. Like add legend I'm still sort of testing out the github projects, so I'm gonna stage, this task add legend to potential pools diagram and that didn't like create like this. This I would say it's pretty pretty done, I'm pretty happy with this. How.

B

A

B

Do I turn it off.

A

B

Away all right.

A

These were, we did both of these I'm gonna call. This done but I'm gonna hand this done, because all because this is you know the exes, the checks and the checkmarks are the boxes. So those are both done and this one is not and I'm, but I haven't created it as an issue yet but I'm gonna leave it here is just a note and I may initially. I may eventually I'll probably turn it into something. I may do it in the background. Okay,.

G

But at least I haven't known.

A

Okay, so finished potential pool diagram. Let's just leave it that, except for the legend, but I made a note. I made a thing for that. So now we will.

E

A

Push up stream master.

A

Yep and nobody got jinxed I didn't jinx myself by saying you had no problem. This will be easy. I did, however, use up nearly well three and a half hours that took three and a half hours a little break, but that's not too bad I I'm happy with this I'm I think this is great. I could even put you know what I should do. I should put back the other one so that we have them both and I can take the debug thing out.

A

Yeah take the debug style out.

A

All right, great, very happy so I will all commit that to real quick.

A

Small tweaks style content push.

E

A

All right anybody have any questions or comments or anything about this work session. Let me know and I'm going to wrap it up here in the next five or 10 minutes wrap the stream up. If anyone also wants to rate anyone in particular, let me know that too and I will hey. Let me run the credits if I can find them. I, don't know that there's many credits to run.

A

Or that they're even going to clear to come, I got.

B

One more follower of the stream thanks.

A

Conda 3p from Finland, so I appreciate it. It's been fun, I.

A

Will be back later this week, I think either Thursday or Friday I haven't decided. I will stream a big long stream on building HTM systems, either Thursday or Friday.

G

Just a little note, when you smell the rain, you have to wait about five or six seconds before you actually deliberate. You always cut off your last words. You say by hand, look.

A

Okay, I will do.

E

That next time, thank you like.

G

To wear a walkie-talkie, you, you look heavyweight and then you release the button and I.

A

D

A

All right over I don't know who I don't know who to read: there's nobody right now: online I usually watch except brackets age, but I rated her last time. Let me take a look here.

A

No, no science only only Rockets ages, the science but I guess I could send send you guys to her. Anybody have any other suggestions: she's, pretty fun.

A

She's just chatting I'm not sure what she's doing we'll go we'll go there. Why not? Okay! Thanks again, you guys.

A

Rock it rock I to this channel is intended for mature audience. Cuz she's a she's, a cursor. She curses.

B

A

Sure, alright, so we'll head over there in 9 seconds, tell her we said hi, she likes my name because it's the name of a mineral so use your use, the brain, a moat, let her know where you came from thanks. Everybody take care, we'll see you see you on the next stream, which will be Thursday or Friday, like I, said again doing initial premises and real receptive fields going on within the potential pools. Here we go. Take care, wait, 5 seconds.

D

And cool bearer of the curse right.

G

It's great, are you doing next mana.

D