Numenta Live Streams, 4 Jun 2019

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: HTM Chat Time - Neuroscience paper review (FEP), forum chat

Description

Broadcasted live on Twitch -- Watch live at https://www.twitch.tv/rhyolight_

A

A

What I was going to do so I'm having trouble installing one of our community applications called detailer from Marty on the forums? So I was going to look at that. This is a Marty, the teeth, the pink teapot there's new Taylor. It's it's relayed backwards in case you're wondering, but it's a C++ implementation of HTM Marty says it's very fast, but I wasn't able to get it built on my Mac and I'm, not sure why well I'm, not sure what my problem is.

A

I really would like to try and run this just to see how hard it is to build something with maybe put a little. You know quick video together on doing something with it, but I haven't gotten a bill. Yeah I saw there was a docker file, though, and he's got examples. I haven't looked at the examples. Yeah, maybe I should okay, so we're doing so. This is a paper rock scissors and it's dependency. I, don't know what tiny DNN is, but I don't read, see very well so feel free to.

B

Correct me, if.

A

I say anything: it doesn't sound right, cuz I'm, not a I've, never written a line of c-sharp C++ I've written some c-sharp. It.

A

Okay, so we're constructing a network, we've got an RNN flare. We're RNN to me means recursive, neural network. Is that what this is? This is tiny DNN. Does that mean dense neural network I'm, not sure what the what the acronym is quite or standing for I'm? That's what there usually are in the machine learning world construct, Network and in underscore GRU I know. This is supposed to be like a parallel system.

A

Gpu ready right, tensor, ready, I'm, not super familiar with tensor flow, but I understand the idea of tensors at least tiny DNS header only dependency, free, deep learning framework and C++ 14 Thanks. Well, that's well code. All the things that owed somebody changed their name: Thank You, New York, who were you before.

C

A

I'm not sure the.

C

Glory Canon, okay, cool.

D

A

Never made any assumptions of what that was in reference to so I won't.

B

D

Marty suppose yes I've.

B

A

It up there, that's where I got this link to so I was looking at some of his examples. So I'm like I, want to try and get this running, but I don't think it's really appropriate for an HTM chat time session, because it's going to be highly technical.

A

So anyway, it's like I got a it spritz. Some environmental problem with my Mac and Mojave and I'm having problems with C++ compilation on Mojave I always have some problem with my back. Okay, so.

C

A

An HTM player, so this is for paper-rock-scissors. This may be a better place to look at so I'm, just trying to interface an RNN player and an HTM player. So we were gonna, have oh I, see so he's competing they're playing against each other, so he'll have an RNN, that's why he uses the tiny DNN and they're gonna play against each other. So the HTM player has a last state temporal memory, depth, okay. So this is the construction, the instantiation of the player and we compute, given let the a last opponents move so we're learning.

A

The sequence of the opponent's last throws paper. Rocker scissors I'm wondering well. This is probably just a simple example, but it probably.

B

A

Your last move was to maybe that's what last state provides, so that would be it so this is essentially an HTM system, learning change of state over time. That's encoding, your move and the last opponents move and predicting I assume I'm, not sure. If it's going to predict your your next move or the opponent's next move, it could possibly predict both and I sort of weigh them against each other to make a decision on what to do next.

A

This is cool, though I like this.

D

A

Yeah, the moves are paper-rock-scissors converts agent predictions to agent move yeah.

D

A

That our predictions are going to be our next move, essentially.

A

So this these are implementations for the departing thing, or is it just a general classifier? It might just be part of the general classification right.

D

So we initialize both.

A

Keep tracks of the scores and then run ten thousand twenty wait, how many twenty times ten thousand so two million right, two hundred thousand two hundred thousand so.

D

I wonder what it comes out to see how this is what I wanted to.

C

A

Just I want to get this set up, but I'm not gonna. Do it right now, I just wanted to kind of show. That's something I'm interested in working on at the moment how this a Taylor thing, because it's, but one of the thing that that's interesting, is the tightly integrated tensors. You know any any. Any um progress we can make towards like creating an HTM station in within the language of today's deep learning is, is progress, I think so see these tensors support broadcasting, GPU acceleration and basic indexing? That's a that's! A big win!

A

That's a big win! So that's cool!

A

Maybe I can try and synchronize when Marty can join the stream yeah. Maybe we'll work something out and we'll do one you can help well. I have to I really need to get it running on my own, I can't depend on him I've already p.m. Tim several times trying to get it working so once I get it working, maybe we could do a session I, don't want it to be like a debugging session I'd. Rather it be something more useful to people who are watching out there, but.

B

That's a good idea. In fact, I'm gonna write it down.

A

And probably don't let you guys watch me right. I, keep lots of notes. Where are my notes? That's for NAB.

B

A

It again synchronize the time with Marty right, Taylor.

A

On stream, this is assuming that I have already gotten it: working: okay, oh yeah, okay! So next I was going to talk about this Delta, a coder, someone on the forum posted about a Delta encoder, oh by the way I always have to here's. Our I'm gonna get rid of this here's! What we're working on later? The create special folders diagram.

A

Let me just do this: let's say that all right, sometimes I just need to pull things off the screen and I still want to look at the same screen, so we're our rear, buckets in bits, Delta and covering here. We go.

A

I've now put together a Delta encoder that I think works as expected. Okay, so Delta encoder is supposed to so I've never made a delta coder. So let me caveat that. That's definitely something we should probably add to building HTM systems at some point. I have never actually made.

D

A

There is I, think, there's information about it: um slash papers, I'm pretty sure in the.

A

D

B

That was it in sparse.

A

Distributed representations, the coding data for each team systems- this is a great document. Scott pretty wrote this.

A

Here, I'll paste it in chat. If anybody wants to read it, but this is what I built all of my encoders off of was reading this document. This came a lot of these sections are verbatim going into building HTM systems, because that's exactly what we're describing and.

C

I took a lot of.

A

These diagrams and turn them into interactive visualizations, so big thanks to Scott, okay, so here's a section on the Delta encoder.

A

So, let's read through what Scott says about the doulton encoder come on I'm having I'm having browser difficulties system on here. Okay, a delta encoder is designed to capture the semantics and I'm reading right here get my mouse emulator up. Reading.

D

A

Here the semantics of the change in a value rather than the value of itself so but but the thing that I don't understand initially about this is that you have to have a starting point so you're first, you have to have your first input, can't be encoded essentially or has some encoding for null or something like that, and then.

C

A

From there, how do you judge the relative distance, like as things jump around? That's that's my initial questions. Having never read into this before this technique is useful for modeling data that has patterns that can occur in different value ranges and may be helpful to use in conjunction with a regular American coder.

A

Excuse me, translation is, you might want to encode to two values or several values as scalar encodings and also encode a Delta between those values? If you, if you know like there, is some relationship between them, a simple example of where you might use a delta encoder is when trying to predict a value. That's constantly increasing.

A

We've talked about this on the forum before. If you search the forum, Fargo says, sounds like a deviation curve. That sounds like the right term. I, don't know, send me a link if there's yeah statistics, there's probably a group of deviation, curve statistics, standard.

D

Deviation curve.

A

Yes, every one.

A

Standard deviation, curve or just any deviation curve, not sure that it matters more general term, but this is more of us. This is more of a static thing. It looks like.

A

D

Says the retina.

B

A

Delta in a lot of places, he I wouldn't be surprised. Oh, this is just about this. No I wouldn't say it's necessarily like this standard deviation curve. I think because it could be I mean the data might plot out like that, but I mean it depends on the on the data. I was thinking more along the lines of these, like ever increasing value idea.

A

Say you have something that continues to go up like a cumulative value, so you could turn that into a simple problem and just and and for every time step you just have a delta, a number: that's increase or decrease. That's essentially what this is, what we're encoding and it's it's like a it's really.

B

Simple I think you.

A

Can potentially encode it as a scalar value starting from zero and how much are sounding from any point that you want to and just encode an offset from that point as a scalar value using any of our other scalar encoding techniques. Right I, think that would work fine.

A

Okay, the predictable pattern in data with this characteristic is not the value in the values themselves, but the change in the values is it smaller, larger and by how much while irregular and American coder would give different representations for each new value of the Delta encoder would produce overlapping encoding so for values that increase increased or decreased from the previous value by similar amount.

A

So it's going so an HTM system would be able to identify the patterns in that change, that it wouldn't be able to identify it within a an accumulating value right, because if you can't see those sequences in the accumulating value, you can see it in if you plot the change to the value, because it might always be positive, but you'd still see you know up and down to it, even if it's always positive and the valleys always increasing.

A

If you had a temperature sensor on a piece of machinery at Delta and quartered recognized temperature patterns caused by machine behavior, even if the absolute Grange was different due to the surrounding weather patterns, right yeah,.

B

A

That so I think what he's saying there is you could use the same. Do the same analysis in many different places, independent on the context of the temperature. Just by doing a delta.

A

Okay, so let's read this picture this question from our eager from Sydney Australia: was it an elephant? Yes,.

B

A

Is he has been around since April 30th anyway, he's put together a delta encoder? He thinks works as a spec it, or at least at least it seems reasonable in context. I would appreciate some feedback, let's see so this setup he's saying an inn and a sparsity, so the so the first thing so I'm gonna give you some feedback as I go along the encoding that this encoder creates does not have to be sparse, I think.

B

A

Of the big misconceptions that people have when they're, when they try and write an encoder for an HTM system, the spatial Pooler that we're going to talk about.

D

Later this afternoon, it.

A

Will apply sparsity you could make this 50 percent sparse I wouldn't go much higher than that just because, but because you lose some of the semantics of the value you're trying to encode when you go to high, but you don't have to you, don't have to encode sparsity here, I see, that's why you have the W of 10, so I'm, assuming this is the output that you're trying to get and arranged in buckets so range from 0 to 232 and I'm. Assuming, since you have the same amount of buckets, you've got one bucket per value.

A

Yeah, don't think that's I, don't think I would do it that way. I think you can do this simply by using a scaler, encoder and honestly just subtracting well, the I mean having a delta and encoding it with a scaler encoder, but you do have to have like a starting point. This seems to be like the minimum. Delta is zero. The maximum Delta is 232, so we're assuming there's only increases to this value, and what is this plotting?

A

Let's see table measurements change, so this is 0 to 40,000, and this is like some distribution very long tail. What okay. So this is 0. This is the min and the max.

A

So these are values, so this must be must be a sample distribution from a bunch of Delta's from the data like so most of them are very small, but there are some that are quite large, all right, there's a long tail on this, so almost all of them are under 10, but there are some that go all the way out so with how it's set up at the moment, I'm getting the following: STR break down bits 0 to 20, represent 0 to 10.

A

That's 0 to 20 represent 0 to 10, ok, so yeah I, don't think I would I, don't know that I would do it this way. Let me go back and read the rest of this. A regular American quarters representation for others and 60s would not share much if any overlap with representations from the 70s, but the Delta encoder produced similar coatings. Let's look at the Delta encoder. Let's just this will be easy because it's probably super super simple I'm gonna.

A

Just let go look a new pic I think it's just I think it should just be encoding a delta I'm, pretty sure. That's what we're gonna find when we look at the Delta encoder I'll bet it's just going to be taking two values: subtracting them and encoding it as a scalar.

A

Okay, the Delta encoder codes differences between successive scalar values instead of encoding the actual values. So it's probably just going to keep a 1 just going to keep a memory of the last value and subtract them. It returns an actual value in decoding and not a delta. ah So that's interesting! That's on decode own! So yeah it sounds like a pre-processed encoder.

A

Let's see what this takes min value, max value periodic, which I guess makes sense. No.

C

A

Guess you could make it periodic in radius resolution named for Boston divorced okay. So when we get in code into array, we get input and output. The input.

A

We're using it but.

A

Are we not using the input, oh yeah, if it's, if input here.

B

D

A

Equals input minus the preview previous value and then we use the Delta and adapt us and.

B

We're using an adaptive, scalar encoder.

A

And encoding the Delta, so so, that's all that our Delta, a coder seems to be doing, is just subtracting and and using an adaptive, scalar encoder, which.

C

Is what I suspected.

A

It would probably be doing so.

C

If you can understand.

A

Scalar encoding are eager, Rob, Rob eager. If you can understand scalar encoding, you should be able to understand the Delta, so what I would suggest?

A

Well I mean we could we could point him towards building HTM systems, which is what we're building right now and we have a bunch of encoders here, and this is mostly complete for this one section I would say so. I could point them here. So.

B

I think I'll do that if anyone has any.

A

Better ideas, because what I mean we're working on this documentation right now, it could also point them to the document on encoding. So let's just give him some references. I wasn't planning on doing like a full form Q&A session here, but workout Rob, I'm, not sure you're going about it. The right.

B

A

Delta encoder should be very similar to a scalar encoder.

A

Except you're encoding the difference between two scalar values. Not they values themselves, see the.

A

Let's, let's here's us to say here are some references.

A

Coding there we go encoding data for a steam systems.

A

I'm going to specifically see the Delta encoding was it called.

A

The Delta encoder okay, see the Delta encoder.

A

A

New pic Delta encoder, that is here, okay and what was the oh yeah yeah.

A

Let's call it the HTM s, encoders scalar, encoders number encoding.

C

A

I'm gonna say this: this doesn't contain a Delta encoder example yet, but these these scalar examples should help alright.

A

Okay, I'm mark that well I'll. Add the video to this. Also, okay and back to the first in paper. I'll, do in a moment right now, I think I need to let's go to here again still trying to get I'm still getting used to my buttons. I need to look at my calendar for a minute, because we should have an HTM hackers hangout coming soon. I forgot the last one. I totally forgot honestly I got so wrapped up in all this twitch stuff.

A

I totally forgot to even schedule it so I'm working on that hold on Google Calendar got this on Google Calendar, pretty sure yeah. So there should be a hackers. Hang out this Friday at nine I. Think I should be able to do that. Let me check I, don't think I've events at the moment.

A

What was the planning I'm doing this Friday yeah Thursday Friday, nothing, okay,.

B

A

We're gonna have a hackers hangout plant in my Falco, my planning on adding what to the website. Sorry I might have missed that hackers.

B

Hangout 9:00 a.m. I. Definitely the hackers.

A

Hangout the Delta encoder Oh to building HTM systems. um It should be but I don't have a juror for it or a juror I, don't have an issue for it. Yeah.

D

But definitely.

C

A

We are pushing through we're trying I'm trying to do spatial pooling stuff right now, so I pushed I skipped a lot of encoding stuff that I wanted to do, but we're not gonna. Do it. Okay,.

A

Now, if now I've forgotten, what we're gonna do all right, all right, I'm, gonna, I'm gonna, create a hackers. Hangout I go to do this on YouTube. This is a pain YouTube's, not nearly the like the live or that's live events. What you guys can see this I, don't mind I'm, trying to schedule a live event. This changes all the time. It's.

B

So frustrating.

A

Okay, ischium hackers hangout, and this is going to be not today. This is going to be Friday not today. Why does it do that? Friday at 9:00, okay and the end time is gonna, be 10, I, always good at an hour, sometimes I go over. Sometimes we go short. Okay,.

A

Obviously, that okay.

B

A

Have this on the form yet I keep forgetting I want to.

B

Whoops whoops whoops, sorry I am.

A

Like I said, I'm not used to my buttons yet hi, everyone I can't show my form screen online, because I get private messages and stuff and I don't want to show too much. If someone's trying to chat with me about something privately. So I have to be careful about what to what I'm, showing here so I'm just going to create a new topic. This isn't oxidants and it is ischium hackers, hangout topic.

A

No agenda yet because I just haven't this is going to be a live event. I'll think about this right now, I just want to get it on the books once I get it on the books come on so think about it. Is that good enough discourse? Thank you.

A

All right, I always have to do this because I'm creating an event over here. So I'll put the link to the forum post there. Jeff Hawkins is not going to be there.

A

Open-Source community.

A

Public, don't have any advanced settings. Okay, create an event.

A

That's that's what your mom says thanks about her yeah. It does change all the time, though it's frustrating like and it'll change to something entirely different and the things that used to mean something don't anymore. So, okay, so here's the link, I can use, and now I got to go back to.

A

We go back to this standby.

A

Here you go hackers hangout, I reckon this here.

B

A

We're gonna put the date to June 7.

A

Alright, alright, alright, alright.

A

Alright dealt encoder decoding spatial over diagrams all right. Every time got it got the hawkers hang out the other thing. I, don't know how interested you guys are going to be in this, but I are trying to figure out how to think about the free energy principle where's. The papers here does and someone on Twitter recommended that I read this paper. I'm, sorry I'm not done with it yet, and so that's what I'm doing I'm reading this paper.

A

I'll have to find a link to it. If you guys want to.

C

Let's see Alan Kristen.

C

B

Auto basis, oh ho ye. Yes, here.

A

We go yes, thank you for your auto. Correcting me I think this is the one I got it from ok. So if anybody does want to read along with me, this is what I'm, looking at okay, so I'm, trying to understand the three energy principle and I'm, not very good at it, and.

A

It's a bit of a struggle for me every time, I, try and understand the free energy principle. I always get to a point where I'm, like obviously like I get I'm like I, feel like when I do finally understand it. It's I've just read so many words that have.

B

A

To some a concept that is inherently intuitive and it.

D

A

That's what it seems like to me perhaps I'm wrong, though, because I'm right I'd really like to be corrected, but the hope, but the idea of predictive processing and everything feels like it's. Whenever.

B

I read about it: I'm like yeah. That's.

A

Obviously, how it works, I feel.

B

Like that's, how it.

A

Makes sense what I don't like about it? Is it still sort of fails to explain what happens within the modularity for the modalities like it like using this principle, you should be able to split up the brain into these nice little sections and say this section. Does this this section? Doesn't this this section? Does this optimize those tasks using whatever you.

D

A

Typically, they use Bayesian networks and stuff to try to optimize those those boxes and then tie them together, don't be like a brain to create a brain.

A

And you know with throwing in these the URLs unavailable, I'm. Sorry I, don't know you.

C

B

Have to search for.

A

A

So I'm trying to understand what one of the things I wanted to I wanted to read. This is because Mike Allen is a well-respected neuroscientist I, follow him on Twitter, and so he was the primary author on this with Carl friction and I, like a lot of the things that Mike Allen says so I wanted to try and read this, and so this is where I'm at here I don't quite understand, but I've only read through this one time and my highlighting style is blue means, I I get it and it seems important and red.

A

Is it I, don't get it and it seems important and the and other than that is sort of just a free-for-all? Any.

B

A

Here, I'm, looking at some specific things, any dissent, predictive processing Theory, must take into account this continuum of views and associated theoretical commitments. So.

B

What I don't understand.

A

About this is radically inactive, enin and active I, don't quite understand what he's meaning by inactive, how to position the question of how to position predictive processing with respect to inactive and embodied cognition, I.

C

Understand what embodied.

A

Cognition means I think because I think that's something that we also claim to need you you, you need it, but when we say embodiment we mean I, think when most people say embodiment, they don't mean having a physical body, they just mean having the ability to take action that affects the environment, at least I'm, hoping that's the type of embodied cognition that Mike Allen's talking to him about here.

A

Somebody spamming me on whisper chat. That's weird! Ok, radically, inactive I, don't know what that means body. Ok, so let me try and reread this again here we present a basic review of of neuroscience right here: scientific cognate, cognitive and philosophical approaches to predictive coding to illustrate how these range from solidly cognitive cognitivist applications. So there again, this paper also has like a dichotomy of cognate ism versus connectionism.

A

That's that's represented here as well. I think that this may be the first time we talk about it, but there's there's like the idea of connectionism versus cognitivism and here several times, but we're saying solid, solidly, cognitivist applications and activists of somebody's Wikipedia dat Foreman. Thank you having causal power. Oh oh I like that term, then, if that's what it means that makes sense to me. Cognition arises through a dynamic interaction between an acting organism and its environment.

A

Claims that our environment is one in which we selectively create through our capacities to interact with the world that makes sense to me. Okay. So we're talking we're talking about the same thing here when I think about embodiment. It's similar to that. Okay. So inactive basically means having causal power that would be great in the environment that you're acting upon.

A

That would make sense to me if you want to use the Judah pearl. You know causality terminology, I, think that makes sense.

A

Uno momento I have to attend to something in a quick. So one moment let me fix something: I do I do still have a day job, so I have to do that.

A

Someone's asking something: let's see.

A

Okay, this is just about. We have a new intern and he's got to sign the contributors, licensing agreement.

A

Just like everybody who writes good for us, okay, I hope that works.

A

All right back to what we were talking about here so at another autopoietic. So let's, let's Google that because auto what was it it's like poop there, it is autopoietic systems, auto, pius, poiesis, auto, meaning self poesis, meaning creation or production, o self, creating system people reproducing and maintaining it self reproducing okay, hold on maintaining itself sure reproducing I. Don't usually think about that.

A

So that's what we're talking about when we say from cognate ism to auto-pay poesis towards a computational framework for the embodied mind. Why are we talking about auto places in this case yeah I'm making it, but why would something be making a copy of itself unless maybe they're talking about this.

A

From a less alarmist standpoint and just saying when you train an AI, you can copy it over and over and and those things can then go learn new models but I.

A

Take, for example, the idea that you train something that can move in an environment in a very generic environment, and then you have some like base base agency, that that knows space and time and it's body, and now it can be pulled out of that that environment and put into something similar enough, and at least one where the sensors work the same way and the space-time laws of the environment work the same way. Then you should be able to transfer that learning into whatever environment.

A

If the environments are similar enough, meaning space and time and sensor set are the same. But then each one of those copies can they learn. Can then specialize can like learn. Other things you know go into different environments, go one, go, for example, a desert environment, a forest environment whatever and then continue to interact, and one will start learning. Oh, this is how I navigate in sand. This is how I navigate over big plants.

A

Whatever it's that sort of idea, if that's the type of idea, we're talking about and I, can get on board with it, if formal synthetic account of how internal representations are rise from autopoietic, self-organization, so I'm, not so sure about that. That doesn't make sense to me.

B

A

Falco says yes or distribute an idea like a meme, Dawkins and Dennett had that as a working theory for a while yeah I read The Selfish Gene that really changed how I thought about life and everything the direction life was taking and what life was doing.

A

Certainly thought about life is more in terms of knowledge, transfer, accumulation organization. More than anything else,.

A

But I'm not that maybe it is a maybe they're talking about mimetic transfer. I, don't know it doesn't. If you for mimetic transfer, you need language, though so I don't know. If.

B

I would jump.

A

All that way, I don't know if I jump that far okay.

B

A

Break this down again here we present we first present a basic review of neuroscientific cognitive and philosophical approaches for coda illustrate how these range from solidly cognitivist applications with a firm commitment to modular internal istic mental representations.

A

So this cognitivist approach is more of what I see from the first and literature, where we've got a lot of boxes and a lot of things happening in the boxes and things signing linking them together.

A

A

That's what I think about when I think of cognitivist cognitivism am I wrong in that. Does that make sense? At least that's what this description says: the modular internal istic mental representations and that's on one side of it. The cognitivism to a more moderate views emphasizing the importance of body representations, so I can see that being sort of the other side of the argument. Body representations, yeah, I, think that's probably important and finally, to those who fit comfortably within radically inactive, embodied and dynamic theories of mind.

A

D

So I guess I assume that's what they're going to present now.

A

Because they're talking about the the one side, the cognitivist approaches to the other side, the body representations to somewhere in the assembler that's more comfortable, where they're calling radically an active, embodied and dynamic theories of mind and innocente predictive processing theory, and they were saying of attention or consciousness. You must take into account this continuum of views and associated theoretical commitments. Ok, I think I can understand that a little bit better now I thought it through sounds like blow load of buzzwords to me.

A

Could it could be like I said most of the other and I was excited to read something that wasn't mainly from Kristin, because a lot of this stuff that I read from first and as seems to be a lot of complicated terminology describing something that's very simple, and unless.

B

I'm understanding it wrong, but free.

A

Energy principle, as a final point, we illustrate the free energy principle attempts to dissolve tension between internalist.

B

A

External list accounts of cognition: that's loaded, that's loaded! What does that mean? This is this is what I was? What I have a hard time getting past by providing a formal synthetic account, a formal synthetic account synthetic account of how internal representations arise from autopoietic self-organization. This doesn't make any sense to me this whole. This.

A

None of this makes sense to me if anybody can help me understand this. What does this mean? What is this tension between internalists and external list accounts of cognition?

A

What is internal I mean I, don't understand these two camps.

E

Formal synthetic account sounds to me like they want to make a model yeah.

A

D

A

Right of how internal representations arise from autopoietic self-organization, so I could say that we're doing something similar we're trying to create a formal synthetic account of how internal representations arise, but not from an auto-pay etic self-organization we're doing it from sensory motor interaction like if you, if you took this out and replace it with arise from sensory motor interaction, that's sort of what we're doing right. Yeah.

B

A

So I don't know what from autopoietic sort self-organization means yet. So this is something I'm still don't understand.

A

I'm gonna turn the music down a little there's too many devices and.

E

No poetic and self-organization. Is that not twice this thing? Is that another, when you say twice the same thing all.

A

Right well, self-organization doesn't imply replication, we're autopoietic implies replication.

E

A

So the fep fuss furnishes empirically productive process theories by which to guide discovery through the formal modeling of the embodied mind.

E

And I just add it's okay, actually that one sentence means they're, trying to make a model of a system that is designed intrinsically. You replicate, oh.

A

B

A

Of the system that's designed to triticale to replicate and.

E

By design that mean by nature, it doesn't mean by does.

A

That does it mean replicate like so that must mean by procreation right. Does that the type of thing that they're talking about I'm.

E

Only I'm only trying to understand this well I, don't know this. It sounds to me like any system that just a lot of interesting things but ultimately wants to procreate like basically what the human does. You could argue. That's that the human does that yeah. It needs to learn language and it needs to know where food is, but ultimately it wants to make baby yeah.

A

It wants to move pass along its genetic material. That's that's.

B

At least the way.

A

Intelligence emerged in biological life is as a way to better pass along genetic material. I'm.

E

Not saying I agree with that, but I think that's what they try to at least present all.

A

Right that could be so: let's try and move on, because I have okay. So what is interesting here?

A

Predictive processing theories of mind, then I granted I have not read this Clark Christin Hawaii paper, but they're there, noting the common ground with this idea of top-down error, minimizing predictions as the key locus of information processing, as opposed to more classical accounts, which emphasize feed-forward feature recognition.

A

D

I I highlighted that blue but I don't necessarily understand it.

A

Anymore, top-down airman, so I understand the error, minimizing aspect of the free energy principle and of I think deep learning, deep neural networks in general and that's very top-down, and this is as the key locus of information processing. So this on. But this is almost trying to link the backpropagation method is because that's what I think about when I think of the top-down error minimizing predictions I feel like that seems like back propagation to me and that posed some more classical accounts, which emphasized feed-forward featured recognition. That was before back prop I.

A

Think we're talking about wasn't talking about features at the low levels that move up and are put together and move up and put together and move up or put together that.

E

Make sense: okay,.

A

Of these, some predictive approaches are strictly cognitivists. Here we go again and cognitivist, meaning boxes of different processes linked together, emphasizing internal modular representations right, so each box has a different representation for whatever it's currently optimizing right.

A

Other more recent variations in doors connectionism to varying degrees. Are we talking about this move this spectrum from cognitivism to connectionism? Perhaps counter-intuitively predictive processing has also appealed to concepts and mechanisms from embodied inactive and dynamical systems. Theory approaches. Predictive processing has also appealed to concepts and mechanisms from embodied and active and dynamical Systems Theory approaches again.

B

So I think this.

A

Is a very complicated sentence trying to say that.

A

Are they trying to say that predictive processing works with any in either fashion, whether you using it in a connect, could augment amidst functionality or a connection? It's connectionist method, I. Think that's what they're trying to say. You can still use predictive processing, particularly.

C

Those that come.

A

Under the rubric of inactive inference, which I would assume means sensory motor I mean anything if we're talking about inactive the ability to have agency within an environment. You have to talk, I mean that includes sensory motor integration, the ability of an agent's to learn space, essentially what we're always talking about about grid cells and navigation and all that stuff yeah.

E

A

Anybody else have any thoughts on this. Is this and I'm also interested? Is this something that you would like to see on the stream? Because I read a lot of papers like this and I come and I mark up a lot of papers like this too and I? Wonder if it's interesting to other people for me to go through and sort of discuss them afterwards, because it sort of helps me because I like to get another person's opinion, if I, because I don't always discuss these papers with anybody, I just read them.

E

A

E

Absolutely fantastic that I'm not really reference yeah.

A

I mean you're good target audience falco there's somebody who's really engaged and has a good understanding of pretty much the theory and the direction we're going so I.

E

Think Marc Marc Brown is very. Do this well, not not initially in in this paper, but in also variable Y and.

B

E

Know that Oh mom had a opinion about this a long time ago in one class he mentioned this and even he told me to watch a video about it. That I might my reaction about. It was if it sounds very new wavy yeah.

B

E

Serious you know, but I can't be wrong and I'm often wrong, so say that.

A

Yeah, it's very high level and say tying this, so there's different kinds of like neuroscientists, there's neuro scientists that operate very close to the wet biology and are trying to understand how neurons specifically different types of neurons are interacting specific types of things and then I mean, but but you also have to think about neurons as populations of and the population effects of neurons, and you can do so much high-level, theorizing research and it's and in many ways it's the type of stuff that we're doing.

A

But it's hard to pin this down to specific biological concepts right. That's what I have trouble with with the free energy principle like you can tie it down to some broad ideas of processing in the brain, but it's, but it seems really hard to tie down two distinct biological processes and things that are happening at the cellular level and we always try I mean we started there. You know trying to build up the idea of how sequences are created like at this cellular level, so I always feel like we're grounded in a way. You know.

A

Our theory is at least, is, grounded in something like this is how we build up representations as we move sensors through time, and here there's actually a neural mechanism. We see working to do that and I always missed that when I, when I read the free energy stuff.

E

The thing is why I, don't please regardless, is that these guys have oh yeah.

A

No, these are smart, guys.

E

Exactly yeah I mean I mean there must be something there. Maybe it's wrong, but there must be something no.

A

I think there's something there, but I think it's I think it's I, think it's hard to nail down and, like I said from what I read about it like they're doing a lot of the models have a lot of math and you know they're trying to optimize these different processes and when it comes down to it, feels like what the right the thing that I learned from this is that you know the the brains doing what I always thought it was doing and in the first place you know, I would say.

B

A

It's it's, it has certain urges and things, and it's trying to optimize by taking actions in its environment to reach goals. But that's what I keep coming back to and when I read for energy is there's.

C

All different ways.

A

And processes descriptions of that, but in lots of different ways.

D

A

Anyway, it's I wouldn't say it's wrong: it's it's just hard for me to understand and and tie it down to biology. That's why I wanted to read this because Micah talks a lot about the insular and first ins been talking about the insular lately, which is a part of I, think it's a cortex insular cortex, which is um it's.

C

Like an associative area.

A

E

It's the one that relates emotions. That's.

A

Right interception and emotion, stuff, so it's it's an entryway from the body sensations of the body, interception proprioception, maybe not proprioception, but definitely interceptions emotions to to to have a representation in the overall representation of your brain right. It could be. It could be like, like one sensory input processes, your vision. This could be something that processes that and it adds its representation to the environment that you're in the objects that you're talking about.

A

You know you might have an object that invokes emotion when you see it, that's because of that part of your brain right.

E

A

Very very small.

E

But very important the way I've read about it is I'm very interested in that into a narcissism and the only neurological correlation with narcissism I found have to do with.

B

E

Narcissists have apparently have it's not certain that they have a.

A

Smaller insular.

E

Descending into la insula means Islands, and there was a part that is smaller and that part that regulates our empathy, and so, if you have a narrow, insula, a reduce from that I'm, not saying that is correct. The deduce from that that people are not capable of feeling this the same way that.

A

A

No but yeah, but it totally makes sense because it's.

C

A

Your ability to your empathy is your is like modeling, another person. If you can't model emotions of another person or if you have smaller ability to model the emotions of another person. That means you're not as empathetic right, yep.

A

Well, Charles, not Charles. Is it Chuck, I'm, sorry, I know you from Twitter I'm gonna put this on hold here. Oh, so, here's the here's, a free energy print. Is this a good video to watch Cristina interview, I'll put it I'll, put it to the side and watch it offline? Oh free!

A

It's autocorrect for Kristen, yeah I might I've watched a lot of his interviews. I think I might have seen this yeah I've seen this interview, but again I mean it's very hard for me to take his description of it and I. Don't know nail it down to neural processes. You know it's. It always comes in the form of here's. Here's the interesting thing: here's the interesting the thing I liked about this paper. Let's get to that. The thing I liked about this paper was something like this I liked. This graphic I like this graphic.

A

So this idea of because I think this supports you know the thousand brains idea, in my opinion, if you're talking about all of these different models from coming from all these different parts of your cortex sort of circling around and and I would think of hierarchy is like the fringes being the low levels just like, as you see here, the fringe is being the low levels and and as you go up towards the center you're, going up the hierarchy.

A

I like this model, because this this makes sense to me because this is how it looks like it's messy right. It's super messy everything, there's a lot of things going to different places, but I, but I like this diagram of like everything coming to the middle and as you come to the top of the hierarchy, that's an important place right. That's where I think I don't know, I, don't know, I'm just saying that, but I, but I like this. This makes sense to me.

A

Whatever is is here where it says self model. You know this is to be understood, I think but I like that diagram, a lot because it incorporates your proprioception and your interoception wherever that's coming from and it's a general model, so you can, you can plug and play. You can say: okay well, I want a system where I've got this sensory hierarchy coming in over here and in this sense we have you coming in over here and there's something and you can sort of by placing them in the right place.

A

You give them some some connection to each other. You know some lateral connectivity by placing in the right place sort of around this. So I like that aspect of paper.

A

Almost at noon, I'm going to start I'm gonna do whoops, don't know which button to push I'm gonna do I'm gonna start at one doing, building HTM systems. So in the meantime, before that, how many pages is a paper? This first in 1, 24, 24.

A

But I haven't read all of it yet and I don't want to.

A

You see some of my notes, so let me keep through I have a little bit time, so I keep coming through right, so I wrote. This is I, understand, I! Think what inactive means, which is good. This is what I thought that was what I needed to understand: free energy principle. It's not only fundamentally inactive and embodied.

A

Which seems similar, those two terms seem very similar to me, but further offers a synthetic, empirically productive resolution to it. Long-Standing disagreements between internalist and external list viewpoints. Let's talk about internalist and externalist viewpoints, I think I know what the difference here is now internalist I, don't know which is which I would I would assume that the internalist viewpoint is that each one of these modalities or models or whatever in this system, has an internal black box right.

A

So you can think of it as a Bayesian, a Bayesian parameter system tuning system- and you don't know if you can't look into it and understand what it's thinking it just optimizes I think that's the internal viewpoint, external viewpoint, I think means you have to understand the processing and what's going on there, it's it. It's not I think that's what that means, but I'm not certain from modularity. To the dynamic mind, the ability to predict future states is essential for the efficient control of perception in action.

A

This notion is a long history in cognitive science, for example, talks about Helmholtz, efference, copies of motor commands. All of this I understand yeah, yeah, yeah I. Don't think this is I think this is just a reference to a model that is way too simple.

A

Our perception of the world was as much inference as sensation. Yeah I could I agree with that.

A

Okay, initial action initiation generates a copy of the motor command. Yes, yes, yes, our conscious sense of agency depends on the comparison of expected in actual actual States. I agree. Okay, what does this fit along these lines? Chris frif argued that the sense of agency depends upon the interaction of two feed-forward comparators.

A

Maybe that's what we were talking about here, I think, no, that's something different: okay, two feet: Ford comparators one comparing desired and predicted states to generate a feeling of control and another, comparing predicted and estimate estimates and another, comparing predicted in estimate states to generate self ascription.

A

Okay, so two comparators, so this is more old brain type, stuff I! Think! Oh, it's Falco's got a definition of internal ism.

A

Okay, that thesis, that reasons are to be identified with objective features of the world, so I guess I would be an internal and external internal ISM is no fact about. The world can provide reasons for action independently of desires and beliefs. That doesn't make any sense to me. I. Think external ism is the way to go right.

A

Reasons are to be identified with objective features of the world I.

D

A

That seems like a very philosophical debate and let's skip it because I don't think it matters and ascription is the attribution of something to a cause that makes sense so assigning causality, Tyron and nice. Thanks for joining us, I'm reviewing a paper neuroscience paper, that's sort of an neuroscience slash computer science paper by Micah Allen, who is a prominent neuroscientists and Carl frist in who is the inventor of the free energy principle.

A

So this is like this is a couple years old that I would still call it pretty bleeding edge, neuroscience AI type stuff most, that this is not necessarily state-of-the-art, deep learning, it's more theory, research, I'm trying to understand the free-energy principle. That's my my goal here. So I've read a good portion. This paper and I'm just reviewing it with people to discuss some bits of it. That I don't understand.

B

D

A

Was I? um Okay here we go, I was trying to understand this pit. I got spun up this weekend. Reading about truth, starting from a criteria of truth, that's a good thing to to get spun up on and honestly, if you're looking for truth, you're you're on the right path, if your goal in life is to find truth, that's a good generic goal and it'll lead you in some interesting places. It'll lead you to some places. You didn't think you'd go his internal ISM, related hume, stuff about terminal goals.

A

I, don't know, I haven't read Hume heard referred to a lot okay, so these two feed-forward comparators I, don't quite understand this, but I think this is much more old brain I'm, not sure that this is right. If it's psychology or not I just don't know.

A

Okay, so this model explains schizophrenic. Delusions, philosophically speaking, comparator based models are unambiguously cognitivist, which makes sense, because they're boxes of processes right information processing proceeds within the confines of the brain by compartmentalized modules. This exactly what I usually think about what I want to think about the first and about most of the firsts and stuff these. This emphasis on functional, localization and modularity led directly to attempts to identify various comparators and brain.

A

So you can look at the brain and see all these different places in your cortex, where you know certain things are happening and- and a lot of this is to try and put a box around this and say this is doing this process. This is doing this process and then figure out the ways they're connected.

A

That I think that's the cognitivist method of going about this. But let's move on to these comparator models. Crucially, comparator models are agnostic regarding the specific mathematical computation underlying their function, which can be explained by a variety of Bayesian methods or non-bayesian Bayesian or non-bayesian mechanisms. Okay, such models are thus ultimately functionalist in nature.

A

Particular behaviors are explained to appeal to a revision of module. Modality-Specific prediction errors: okay,.

B

So yeah, so this is.

A

Where I put, this is basically what I think about names of processes surrounded by boxes and arrows, and this is one of the things that Mark Brown complains about I think when we talk about FTP.

A

So what does this mean? Although the elegance of comparator models has led to experimental and clinical applications across a variety of cognitive, social and affective domains, it should be clear that they are predictive processing revision.

A

It should be clear that they are predictive processing revision of cognate ism, rather than a radical new paradigm, predict a process, a revision of cognitive okay.

A

So so that's okay! So these talking about the old way, all right so he's just saying: there's nothing special about this old stuff. It's.

D

A

I guess it's so I think he's just sort of some mating: the FTP, the old idea of FTP, whether it's be easier or non-bayesian whatever. So the radical predictive process saying the connectionist approach. This is I think the meat of it that I'm trying to nail down. What is this all about? There's a connection is the approach?

A

Is internal ISM related to you, though, I already read that the predictive coding, implicit in comparator models of motor or social controls is clearly cognitivist. Rich internal models which explain a world hidden from the agent do the functional work of cognition such applications of predictive coding are best at best, ignore, embodied and inactive cognition and aren't worst irreconcilable. I can agree with that.

A

So he's sort of dissing the comparator models and saying this is the connectionist approach is the way to go, which is good because that's what a lot of the neuroscientists seem to be going. I talked to car Conrad chording recently, and he said he would consider himself a connectionist I think I think that's what he said. It's check the interview.

A

A

So I got I, hadn't, quite understood all the rest of this. You can see it's not quite as marked up you can tell where I started. Losing I want to understand this RPP thing, but I have but I don't understand it yet and.

B

A

Think I could still go on. I do have to go grab lunch at some point and I want to start broadcasting building HTM systems in one hour about so I have about one hour, so I'm just going to keep reviewing code for it. For any of your new folks out there I'm trying to understand the free-energy principle.

B

A

I've always had a hard time understanding the free-energy principle, which is Karl, frist and crow. First and joint by reading this paper, primarily authored by Mike Allen, who is a neuroscientist I, follow on Twitter and from this idea of moving away from cognate ISM to connectionism, but I still haven't quite understood. This link to autopilot police's yet, which is I, think the ability to replicate so from cognate ISM to like the ability to replicate towards the compute computational framework.

A

For the embodied mind, they still don't quite understand the title, so I obviously haven't understood the paper. Yet where was I yeah I think this is where we're at so.

A

Just a quick aside, I've got I'll be streaming for most of the day. My name is Matt Taylor by the way I work for Demento. If anybody's new here I will be doing I'm just doing HTM chat right now, which is the blue area and I'm talking about this Myka first of paper, I've already done the hackers hangout. So all this other stuff is done after lunch, I'm going to build, build HTM systems, we're going to talk about potential pools and receptive fields for many columns. So hopefully you guys can join me for that.

A

This will be the first real spatial pooling session of building HTM systems, and it should be fun.

A

It's the first thing you got to do if you're gonna build a spaceship learners, figure out those potential pools and receptive fields.

A

You can ask question anytime Falco, you.

C

Don't even have to ask a.

A

Question to ask a question.

E

Yeah, but this money's away from the free that's.

A

Okay, I'm getting a little distracted by it anyway. So what's up.

E

I've been thinking very much about it's no problem, any orientation problem, Geoff and Marcus were talking about yeah and.

E

Being over cooking my brain the weekend, I really am kind of lost in lots of tangents, and there is one thing: I came up. The whole reason why we are trying to link features to a grid is to compensate for changes in what we see in what we feel so that when.

D

I move a bit the.

E

Object that I am looking at in front of me is still the same object, even though I get other data on my retina or I feel different things. My.

A

Brain compensates for the changes.

E

D

Decision of all these points, yeah and so I need a.

E

Map that is compensating for the rotation and translation right. That's the reason why we're trying to order you guys are trying to find something is also something else that changes all the time and that is not influencing not influencing our way. How we see your model and at the scholar when, when the Sun breaks through and my shoes change, colors all of the sudden I'm, not wondering hey. What is that in? That's!

E

That's that's on my feet. It's something different! Oh! No! Oh! No! It's my shoes! It's in my shoes, so changing of color has probably the same impact, maybe even a bigger impact on online making models and- and so maybe the same.

E

The same adaptation or the same compensation should also compensate for changes in colors and I. Just I would just want to throw it in there, and maybe maybe you can think about it. Maybe there's something well.

A

I think that something like color so I think about like this there's there's a space around you. So your brand is representing space. It has to represent that space and you can think about space as an infinite, potentially infinite number of features that can be sensed at each place each spot in space. So it's like a high dimensional array. You know at in this 3d location around you there at this point, there's a whole lot of features and color.

A

For example, the color of my glasses or whatever, maybe I can find something. It actually has it color it doesn't really matter, but the color is a feature that you sense and that's and in that space and the way that color changes based on the environment is I think is, is a it's a property of honestly the the planet that we live on right. You part of your mother, like when the color changes, your brain is never confused.

A

If you, if you have something sitting on a table and a cloud comes over and it suddenly looks like it's more green than yellow, you know, but just because it the the light change the characteristics of the light of the environment changed your object represents representation is the same. You recognize it's the same object, but it's representation to your senses. Changes depending on the context of the object, so I I, don't think color is special. I. Think color is just like a texture. It's the same.

A

It's the same type of attribute of an object at a space in the objects frame like orientation. It is special I think it is special in that and I think that it's special because we have because it exists on this planet right. We all life on this planet evolved under the forces of gravity and gravity, gives us a default orientation by.

C

Pulling downward.

A

Right so I think there's a natural orientation for most objects, and that is the orientation under which we generally see them in reality, because gravity enforces sort of objects to to to have an orientation right and I. Don't think color is the same thing.

E

Yes, but when you say that orientation is special. That is because, if, if our orientation changes the data that comes into our retina and fire, certain neurons has to be compensated for in such a way that the right neurons gets called the pound. So I look at my phone and I turn my phone around and I still have to fire those normal to say no, it's still the same phone. This is not a motor object, so it's special because we are changing something that our brain has to recognize.

E

It's still the same model and I think anything that does that, should it count down as the same kind of special. If you say my screen, all the sudden lights happen becomes white. That's the dramatic change! I'm sure a lot of neurons in my head are going to fire differently, but somehow, at some level some neurons have to say no, it's still the same phone, it's just all now, but it's still the same phone.

A

B

A

Doesn't that doesn't make color special I don't understand like color is a purse if ik aspect of a vision, you know some people are colorblind and they don't have like a messed up representation of reality. Necessarily they just have less detail. I.

A

Would I still would not argue that color color is just an aspect of reality and when you're, when you're doing object, modeling color is a feature of the object. Orientation is not a feature of the object. It's it's an inherent vector right, it's the it's! How the reference frame of the instance is oriented in space as it intersects something else it has to intersect something else. It's definitely special. The color of something will be different in different environments or in different contexts depending on the environment, but it's not like a core I.

A

Think it's not something we have to to account for in the architecture of the layers that we're trying to do. I think orientation is something we have to account for, like I, don't think there's. There are certainly cells in your brain that fire, when read, is read when you see read, see something that's written right, but I think that's more of an intersection with objects that you're that you're you.

B

Know viewing if.

A

This were like white sunglasses: I might have an intersection of some cells that are representing white, because I know that these are white and it's a little odd color to have sunglasses anyway.

A

But the orientation of the sunglasses are not specific to sight and color a specific to sight and.

A

Orientation is also not specific to your somatic senses. Either. I can tell how these are oriented based on how they feel, in my hand, right side up now, they're upside down, I can't.

D

Tell what color they are, but.

A

I can always tell what they're oriented how they're oriented I think its core I think the orientation is a core concept and object. Recognition has to be represented within each cortical column. That's what we're saying! I.

B

Think I still think that's true, I think color.

A

Is different, so Justin I still didn't get your first name color.

D

Stability is hard.

A

An edict, a computer vision system, yeah and you're I think at our eyes, is much more efficient and blue banana says it's. It is very specific and the colors we accept as valid for an object depending on its material. Well yeah. You could certainly see a pink rabbit and be surprised right because you've never seen a pink rabbit before or something to that effect.

A

But if you see a few pink rabbits, you're gonna stop being surprised by it right because you will have learned to associate the color with an object and you can do that with anything. How cool is it? That's a random guy like me from Finland, could just tune up into top research. Like don't call this isn't top research.

A

If you want top research, let me show you where you can get it: I'm, not I'm, not even a researcher I'm, not even typing, on the right keyboard here, I'm, not a researcher for Numenta, but I can show you where you can see all the research videos I do record research videos for Numenta.

A

This is the most turn.

C

A

Thought it was here, playlists give it to research meetings here yeah. So here's where you can yeah.

B

I'm trying to get useful that I have the webcam okay.

A

That was that's the first one. You can run right through them. There's several and I'm recording at least one a week. Typically two a week, they'll probably be one Wednesday and Friday. There will be one not today, though, but this is where you can get your fill of research.

A

But yeah I try and stick around the research meetings. Honestly, I, don't follow everything in the research meetings because I have other things like I have to do, but it is cool that some random guy in Finland can show up and tell what I'm doing every day, I think I love the whole twitch concept.

A

I think it's very neat and so I'm excited to be sort of working in the space because honestly go out there and look on Twitch and there's very few people talking about this type of work, especially artificial intelligence, machine learning, neuroscience I. Think there's one other channel doing neuroscience: it's I know beta. He does neuroscience stuff Chris thanks I like to know who I'm talking to I. Couldn't forget your I.

A

Couldn't remember your first name, but this this channel is pretty unique on Twitch, especially since we're doing real research and you can tune into those research meetings every week. So I'm glad you recognized that tune around. Give me a follow. If you stick around long enough, I will gift you a subscription that goes for anybody for as long as I can afford it. There's a cool experiment: chris says: there's a cool experiment where a scene is shown to viewers under different lighting conditions. It contains random stuff, including bananas, a color palette.

A

The view consistently identifies the bananas as yellow, yet they see the same, yellow patch on the color palette change under a different light. However, the yellow panel on the color palette is actually a slice of banana peel and perceptually changes, color, obviously but stays stable when it's part of a banana yeah that makes sense yeah. That's.

C

A

Way, it's totally works should check out the forum yeah. We have a forum. Thank you for the link, my demon and.

D

There's always Suraj Suraj is, is.

A

A great guy I like him, he did a video on Numenta once and it's funny, because when I didn't even know this, but I started engaging Suraj to do something on Numenta before he did it and and I. When I started, making HTM, School, videos and I didn't know. He had some video from the past where he like he came. He got a little dissing on the meds. I can't remember what it was like.

A

He was talking about companies who didn't do anything or didn't produce anything, and he said like yeah Numenta or something which is which was weird, because we hadn't been putting some papers out and we open sourced our coachmen, but afterwards he did a video on memento, which I was very happy to see. We got a ton of views based on that video, like more views than I've ever gotten on YouTube and it's all came from suraj weed and there was a New York Times article as well, that got good use, but Suraj did better.

A

Suraj gave us more views than anybody and.

B

A

Video wasn't quite accurate, they called he called nunu pick an autoencoder, but it's much much different, really and just not ochre bring.

D

A

The follow from Finland appreciate that, okay, let's go back, we are have a little bit more I'm gonna, try and get through on this paper from mica Allen and Karl. First and and then we will well I'm gonna, take a break for lunch. I'll keep streaming and I'm gonna go grab something to eat and come back and do building HTM systems and condo if you're interested or anybody new. This is what I'm going to be doing after lunch.

A

I'm going to be actually doing these visualizations we're gonna, be making the first steps toward building a spatial folder, which is to identify a set of mini column, potential pools and receptive fields, and that's gonna be interesting, because I'm.

C

A

If we'll get this whole thing done, the potential the potential pools will be easy. The receptive fields, however I, don't know I'm, not sure I'm using the right terminology yeah. This will all be easy. This will all be easy because I'm not sure I'm using the right terminology. This is really just the receptive. The potential pools, I think that's it so I think we will be able to get through all of this, but that should be fun and we'll be using d3. I think this will be easy.

A

Honestly, Falco I think it'll be easy, because I know how to do all this stuff, except maybe the Czechs and XS, but that should be easy too. Okay, I think I'm, gonna I'm, ready to jinx myself. This will be easy. We're gonna knock this out. We.

B

Did the last one and.

A

I was happy that we got that done pretty good, pretty quickly. I think you'll be surprised how easy it's gonna be. Let's do a quick overview of this incase if anybody's new. This might be a good, a good introduction. Does anybody new? Does anybody have no idea like what how htm' works or anything that I'm talking about when I talk about mini columns and temporal memory, and since memory or spatial pooling the HTM neuron distal dendrites active dendrites?

A

Anything like that, the we've got 10 viewers now I just wondered who's actually um interested in an overview or not I used to be able to tell there we go, but the idea is we're looking into a layer of the cortical column and each one of these layers has a bunch of feed-forward input that it can sample from, and there is this mini column competition between the mini columns in this to represent the input coming in so Condon watched a video and read an arc a while back. Well, you definitely go to HTM school. That's.

B

A

I'm trying to get the link in here so that I have a whole video series about this, but I'm gonna. This is going to be an interactive document, so each one of these mini columns has a different potential pool of connections and that's what we're going to be visualizing, we're going to take just one mini column, we're going to focus on one mini column, which is just a group of pyramidal neurons and we're gonna zoom in and we're gonna look at just its potential pool, which you can see on the right here.

A

This this one mini columns, potential pool and we're gonna visualize it that way.

A

B

Did this we did this one.

A

Last time, I'm Mark Brown one at chess. Congratulations.

A

I was gonna, go to lunch soon mark, but no, but it won't take me very long and then we're gonna come back and do this part I keep I'm very excited about this, because I know this will be fun and then.

C

A

Have the receptive fields bit, which is a little bit different, because we need connection strength right and for connection strengths. We have to decide the initial permanence. So then it gets a little bit a little bit hairier because we're going to be creating more data structures as we do this initially we're just going to have a data structure where each point represents a mini column and each point has a list of ones or zeros for a list of indices that represent its potential pool.

A

Each mini column is gonna, have one of these and then, as the input comes streaming through this screen, because we're going to have a streaming scalar data thing and that I know you've read the papers mark about the free energy principle yeah. It's just an exercise of me trying to understand it a little bit better anyway. This is what we're getting into potential pool receptive field, and then this is going to be the fun part, because we have to build a spatial cooler to actually represent the mini competition.

A

So I'm excited about that I, don't know when we're gonna get to it, probably pretty soon, but we got to do this these two first, but judging from how fast our last one went, I think we're gonna speed through so.

A

Yeah Marc Brown I've been reading this paper. You.

B

Maybe you didn't maybe.

D

You haven't read this I.

A

Liked this approach, because Micah Allen is the main author and he's a pretty prominent neuroscientist at least I know him by reputation and I'm trying to understand his take like this seems like sort of his take on free energy and how free energy could be applied to connectionist models versus cognitivist models.

A

That's what I'm getting at here and that, but I still haven't quite gotten to the meaty details of what he calls a radical predictive processing. I. Just that's! That's the next part that I, really I think I need to read this part offline because I don't get it yet. I've got a few things I like this model, though, and what do you think about this chart? Marc Brown this? Would it interested in your opinion on this? If you want to pipe in, but.

C

I really like this.

A

Delete some of this stuff I really like this, this presentation of sort of the hierarchy, with the fringes being the outer layers and they're being connected like that, can I get the encoders for the hypothalamus outputs I'm that not yet I, don't know yet anyhow.

A

So that's a strong framework to drive and learning well know bit mark is referring to I. Think the fully connected networks that we're eventually gonna be talking about hold on I've got I, have it on the calendars for June 21st under our events here fully connected deep learning networks, hot field networks, Boltzmann machines and how they might exist in brains and marks.

A

Gonna kind of join join me for that I think over discord over chat, I'm gonna go over how they why they were implemented in it originally in deep learning systems and why they're not used anymore, and then we can talk about marks thread on the old brain. Oh I, keep doing this. I won't.

B

Talk about your fear, I'll bring it up later.

A

Mark says this works the same way as a stret as the threads thrust streams. We're learning from both ends, transmog regarded mark as the threads I think.

D

A

A typo in it I cannot parse it I can't parse the typo learning from both ends you're talking about the fully connected Network. Are we talking about this graphic that we're talking about.

A

In this case is not so much is the fruit in the frist and paper, not so much to both ends of parallel guidance for learning.

A

So you've read this mica Allen one I'm still confused about this auto poesis I, don't see that happening anywhere, I, don't understand how he's using that in.

C

A

From all auto poet, poetic self-organization, what what does this mean? I, don't I'm really confused about that part of it still auto poiesis meeting like self-replication in some fashion, but I don't understand how that concept relates to this radical predictive processing, maybe I'm, just not there yet, maybe maybe I just have to get through this session all right. We might start the building HTM systems session a little sooner than normal.

A

Because I am ready and I need to go grab something to eat. First.

A

So let me take a quick break here and.

C

Then I'll decide what to do.

A

Mark can't get on discord today. It's already, let me know that he will be after June 21st.

A

A

I'll be back I'm.

B

A

Gonna use the restroom I'll be right back.

A

Okay, scratch that I'm, actually gonna go to lunch I. Think let me just change the text here. I.

A

Think I can just do this.

A

There you go so.

B

I think this should.

A

Still loop, I'm gonna go grab, something and I will be back to chat and then we'll switch over to building.

D

Htm systems excuse me.