Numenta Featured Applications, 30 Nov 2015

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: AI-909 - Taylor Peer

Description

2015 HTM Challenge Application submission (ineligible for prizes).

A

This is AI 909 by Taylor, pier and I'll mention again: Taylor works for cortical I, oh so this is ineligible for prizes, but it's still a cool application.

B

A

B

Ai now nine, it's a drum machine, app that created over the past couple weekends for the nomeds HTM challenge. It is three channels for a kick drum, snare and hi-hat and it runs as a web application, so it works in a computer tablet or most devices that have a browser. You can program a drum beat on it by turning the drum steps on and off so like this.

B

Playback and temple controls are here.

B

But what makes this different from other drum machines is that it can learn patterns and generate new ones by using the HD MZ bility to predict sequences. So the user has two buttons to interact with the HTM and generate by learning a pattern. It saves it to the current memory location and when the user presses the generate button, a subsample of the learn patterns in the current memory location are used to train the HTM and output, a new predicted pattern and by.

C

Changing memory locations.

B

The user can store different kinds of training sequences separately, so you could train the HTM to compose different kinds of musical styles and different memory locations and then switch back and forth between them. So I'll go ahead and let it learn the current beat once I do that I see the sequence displayed in the bottom part of the screen so so far this is the only sequence it knows in his memory location and since it won't be able to generate anything interesting with just one training example: I'll teach it a few other views.

B

And so now that is a bunch of examples to learn from I can start generating new sequences with it.

B

And so the way this works internally is pretty simple. The three drums annals getting coated together into a single numerical sequence, so silence on a beat is represented by this number. Zero I think a Hyatt is a one kick and a higher together out of 5 and so on, and then those integer representations just get put through a scalar encoder and from there the HD and the patterns using the spatial and temporal coolers so to generate a new pattern.

B

It just predicts step by step the next integer value in a sequence and then that predictive value gets fed back into the HTM as input until its predicted in entire 16 step drum beat, and so even with just three channels.

B

The a 909 is able to come up with some interesting and varied compositions and I think it's easy to imagine how this could be expanded upon, first of all, by adding more drum channels or eventually also applying the same concepts to tonal music, to use the HTM to write entire compositions and all the code I wrote for this is on get up so feel free to take a look at it. Make improvements build upon its whatever you want to do.

B

The back end is all in Java, so I'd like to thank my coworker David Ray for his contributions to the Java ports of the ATM, as well as everyone else in the new pic community.

A

Taylor come up to the mic and take questions from the panel. Let me just say that I'm gonna put this on on my kid's iPad, because they would really like this. You.

B

Have to have it running on a computer, though, in the backgrounds oh one.

A

Of these days, right, okay, open up the comments, tell.

D

There's a lot of fun I ask a question online about this Bell SEO similar when you're too I just try to understand how much the HTM is adding and so one way you could ask that questions like did you did you use the memory function to say, okay, I'm, going to give some nice slow things on one grouping in another. You know they really sort of show two different classes of rhythms and then see if the HTM in those two different classes that you could tell I'm just trying to state.

D

How do you know how much they seem is doing, and one way would have been to do like two types of patterns in the two different memory functions. If I understood it correctly, did you try that yeah.

B

Exactly I did try that myself, I didn't show it in the video but yeah. It does learn the differences when you train it with like one style of music in one memory, location and then switch to another one. It does output different kinds of beats, yeah.

D

And then the second question would could I tell the difference in this and a monkey doing it. I mean it's like. No, they all sound good to me. So I'm, not sure if that's just your talent in the indie HTM or if some ignoramus like I would did it would work out k too. You know yeah.

B

Evaluation is kind of a difficult thing here. I just played around with it till I thought it sounded cool. That was it other.

D

Patterns that don't sound cool, yes,.

B

It it is able to output gibberish to that that did happen in the beginning. Okay,.

D

Good, very nice.

D

Well, it makes it take some three months to learn if they were, you want get out, but they just like banging out something I just I have no idea how to evaluate the performance. Yeah.

B

I think that's a general problem with algorithmic composition, I! Don't.

D

Know people one way is by using those multiple memory types and you clearly see if it's catching. You know it's at least learning and getting things there's so much of the things that has been trained on yeah.

B

A

One way to evaluate its to play it at a rave and see how long people dance.

E

Okay, can you explain you when you trained it, you had a series of sequences that was used to train. So what was it that was similar among those sequences they were, you said they had a certain style was the beat or what oh.

B

Yes, well to me, though, I mean they were all like on a drum bass, breakbeat sequences that I drained it with so I'm yeah they're, all stylistically similar, oh.

F

So I'm, this is really cool. Add a very slick, presentation and I want to play with it, but I think I'm missing something at the heart of the generate function. I'm wondering if you could slow it down for me step by step. What's exactly going on there by, I think what's happening is that you are creating these sequences and then the HTM is learning them and then, when you press generate it's going through them and then asking for what would the next thing be? They're, not.

B

In a pretty next step, so it it generates the first, the pre Queen's randomly or not random, leave it takes the most probable one from the training sequences and then it says: okay, what at sh t aime? What's next and in the sequence, but.

F

It's not looping this sequence so because then it would learn it perfectly and just think that the first week.

D

I think the question is I had the same question. The answer online is hot. What prevents and just playing back what it learned earlier, but.

F

It's not because it's not a loop, it's left he's you fed in this example, for was it for seek. Was it for differences that are concatenated then or oh yeah?.

B

Yeah exactly they're.

F

Concatenated together and while you get written it.

B

Resets after each one but yeah, okay,.

F

And then at the end, you're saying what's next, but it doesn't know what's next, so it's to make its best gas based on yeah.

B

Basically, okay I, didn't explain in the video I explained it to Jeff and an answer online. It trains it every time you press generate it. Trains it on a different subsample of the training training.

E

B

That's why I doubt put something different every time, because otherwise it would just learn like the average beats over all of them, and that would be kind of boring to just generate one thing and then never get anything.

F

That makes an abstraction in theory. It would be the style like you said it.

B

Basically, you give it a bunch of sequences and then, when you generate two takes a random subsampling of those trains, the HTML nets, and then it uses that to predict a new one right.

F

So interesting, if you had one that had a lot more channels and it fit in like a Bach fugue. What would at the end of that? What would it would yet.

B

That would be a little bit more complex than this, but.

F

Theory that would be an interesting experiment. No yeah.

B

Maybe at the next sakkath on so.

G

In some sense, is it's taking an interpolation or averaging of the training samples, yeah.

B

I'm sure, internally, that's basically what it's doing its learning like kind of sub patterns or you know, what's a fitting. What beats follow other beats sample so.

E

The inputs you encode it as as numeric values from 0 1 through 7, whatever 0 through 7 yeah.

B

E

You constrain the output, how did the HTM know not to generate 15 or something as an output and how? How did you interpret that oh I.

B

Thought that was kind of in Arrington in the way the what is it heaved? It's never seen 15 before yeah. Exactly so I thought that.

D

It even though he says the numbers is really treating like categories. Okay,.

E

Yeah I got a gig, nobody, just cattle guys, yeah yeah.

D

Okay, gonna miss leading to stay out with 27. It's like now. It's just write something one.

C

Thing I was confused about was the memory locations? How did that tie to HTM s? What was the mapping between memory locations.

B

Oh, that's kind of that separate that has nothing to do with the HD I. Just um when you train you give it example: sequences they're stored in a memory location. So that's why you could train it to do other kinds of beats by storing them in another memory location. So it's not HTM memory. It's good is.

A

There only one model than running yeah.

B

Exactly that's so.

A

You could you could train a model on one style and then another style, and potentially you could create a combination of styles. You.

B

Know true, you could do that to it cool.

D

I'd love to stay it with with you, know, I think I'm and did some work with MIDI ones would be. We really love to see it. You know not just the rhythm but also a melody as well and and then you know in deceit, then you'd really have a better sense. How creative it's! If it's, you know how good it is, but that'd be a nice extension yeah.

B

Terms of this was actually the first application I ever made with the HTM. So over the time working on this I realize other people been doing audio. To say, I would be interesting to put this together with people working on tonal, music and then because.

D

It's a great interface I mean I, mean I, would look it's fun to play with right, so it'd be a great way of experimenting with like oh well, if I teach at this sequence, in this sequence, in this sequence, and and then put that in one memory, how well does it generalize like that I mean you'd, get a better sense. At least I would get a better sense of some of its attributes, but it's a nice interface for doing that. So very.

A

Cool cute make you Taylor.