Numenta Numenta Journal Club, 13 Sep 2022

Previous Meeting Next Meeting

⏯

youtube image

►

From YouTube: NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks

Description

Guest speaker Burak Gurbuz talked about his recent work with Constantine Dovrolis that was presented in ICML 2022: “NISPA: Neuro-Inspired Stability-Plasticity Adaptation for Continual Learning in Sparse Networks.” He started the presentation with an overview of the biological aspects of continual learning, then introduced NISPA and shared experimental results.

Paper: https://arxiv.org/abs/2206.09117
- - - - -
Numenta has developed breakthrough advances in AI technology that enable customers to achieve 10-100X improvement in performance across broad use cases, such as natural language processing and computer vision. Backed by two decades of neuroscience research, we developed a framework for intelligence called The Thousand Brains Theory. By leveraging these discoveries and applying them to AI systems, we’re able to deliver extreme performance improvements and unlock new capabilities.

Subscribe to our News Digest for the latest news about neuroscience and artificial intelligence:
https://numenta.com/news-digest/

Subscribe to our Newsletter for the latest Numenta updates:
https://tinyurl.com/NumentaNewsletter

Our Social Media:
https://twitter.com/Numenta
https://www.facebook.com/OfficialNumenta
https://www.linkedin.com/company/numenta

Our Open Source Resources:
https://github.com/numenta
https://discourse.numenta.org/

Our Website:
https://numenta.com/

A

Like my name is brock and I'm a phd student in georgia, tech, I'm working on like intersection of machine learning, neuroscience and network science, but I'm machine learning student, so I work more heavily on the machine learning side of things, but I try to get some inspiration from neuroscience plus we use network science because deep learning, architectures are neural networks and we believe we can analyze them from the lens of network science too.

A

So I work with professor constantin, dolores, and this is a joint work today I will talk about our recent icml paper, which is called nispa neuroinspired stability, plus this adaptation for continual learning in sparse networks.

A

So I will briefly talk about biological aspects of continuous learning. Then I will explain some of the related works in continuous learning domain. Then I will follow by explaining our algorithm nispa.

A

Then I will share some experimental results and finally, I will highlight some of the future directions. We plan in this work.

A

So artificial neural networks achieve great performance on wide variety of tasks, but they primarily rely on fixed data sets and stationary environments in continued learning. However, we have sequential tasks that change in time, so the environment is dynamic.

A

Under such continued learning scenario, models forget previously learned information upon learning a new task, and this is the one of the main challenges in continuous learning and it's called catastrophic forgetting because of this catastrophic, forgetting problem: artificial neural networks like a critical characteristic of general intelligence, which is sequentially learning more and more information over time.

A

In contrast, the brain exists in continued learning and it offers the only existing proof. We know that efficient content learning is possible because of this observation, we study four mechanisms of the brain that we think that are relevant for continued learning and can be adapted to design of artificial neural networks.

A

So, first mechanism is biological. Neural networks are sparsely connected.

A

Now I will give like two examples from smaller creatures. They are unique because we have access to wiring diagram of these creatures at neuron level. So first example is this c elegans, which is a tiny nematode and its whole neural network has danced to less than four percent.

A

Second, we have the dorosofilap or fruit fly. We have the full map of the optic middle of this fly and it is dance. Its density is only six percent, and this sparsity is also the case in more complex brains such as memory brain for example. They are also sparsely connected here. We see a plot of synoptic density of the human brain, and please note that, like from now on, when I say brain, I will be referring to mammarian brain for the rest of the presentation so like at birth.

A

The human brain is partially connected after the birth. We observe a significant snaps formation and the dust of the brain peaks around age of six.

A

After this period, we observe a synaptic pruning driven by experiences. This phase is quite interesting because, during this phase, new.

B

A

Directly affects how the brain is wired and after this pruning phase, because the brain becomes sparse again and its density stays relatively constant through healthy and through lifetime in healthy adults.

A

So to sum up, the brain is sparsely connected and we observe a relatively constant level of sparsity for the most of the life.

A

Unlike their biological counterparts, artificial neural networks have dance connectivity.

A

Here. I share like two of the main building blocks for most of the deep learning architectures.

A

First on the left, we have fully connected layers, as their name stands for, like they are fully connected between layers, so layers are densely connected on the right. We have convolutional layers, despite the weight sharing. They are conceptually dense because the each unit operates on all future maps of the previous layer, so they are also sparse and.

B

A

A

Second, the mammalian brain has dynamic, topology brain runs through two types of plasticity. The first one is functional plasticity, which adjusts the synoptic transmission rates between neurons and second, we have structured plasticity which rewires the neural circuitry.

A

While learning tasks in artificial neural networks, we observe a static topology, they have weight updates which might be considered as analogous to functional plasticity, but there is no room for connection rewiring, because layers are densely connected and all possible connections are already there. There is no room for changing the topology.

A

B

A

Brain has selective plasticity, so some connections are dedicated for remembering all tasks or old experiences, while others are dedicated to learning new information.

A

Here, I will share an experimental result on mice.

A

First, researchers put mice in a scenario in which they require learning new motor skills.

A

Then they study how pyramidal neurons in the mouse cortex are altered. In response to this novel experience.

A

Here we see the formation of new spines after a learning period, and here we have some more quantitative results and, as we can see, exposure to no experiences triggers more spine formation compared to.

B

A

A

And here we see how this process unfolds in time. First, while learning news new tasks or new information, the brain forms new spines. However, only a tiny fraction of new connections survive the first few weeks after learning.

A

Then these survived connections are stably maintained later in life, to remember the associated skill, so, for example, in this figure the red dot is kind of a frozen connection. It is a permanent mark in the brain that is associated for the learned skill, but we also have lots of gray dots which can be modified for learning new information, so they are plastic, so this indicates that learning leaves small but permanent marks on brain connectivity.

A

To sum up in the brain, we have task specific, stable connections for remembering tasks. Here I show them with this colorful dots. However, we also have plastic connections that cannot leave information and they they are indicated by these grey dots.

A

In contrast, artificial neural network in artificial neural networks, all weights are equally plastic. By this I mean we apply same learning rate to all parameters, and that's why there is no stability to remember previously seen examples or tasks.

A

That's why deep learning models require revisiting all tasks frequently, otherwise they just forget them, because new information overwrites learn knowledge, and this is the catastrophic forgetting problem.

A

Finally, the memorial brain continues learning more and more tasks with fixed number of neurons. Of course, there is neurogenesis in few restricted brain regions, but we know that growing. The brain is not the primary mechanism for learning.

A

In contrast, deep learning architectures, it is very common practice in deep learning that we add new modules to the architecture as we need to learn more and more tasks.

A

So now I will briefly explain some of the research directions in continued learning. The first direction is regularization methods.

A

These type of methods apply an additional term in the loss, function and idea is that we identify and protect important weights that are associated with the previous tasks.

A

Second, we have architectural methods and I divide them into two categories. The first one is expanding networks. Here we dynamically allocate new neurals or new resources for new tasks, and we have. We also have fixed network architecture. Methods and idea is that different parameters for will have different parameters for different tasks, so we will isolate certain parts of the network from each other to avoid interference between different knowledge.

A

Finally, we have replay methods here we store or generate all task samples, and we frequently replay them to remind our model about the previous scene tasks.

A

This comes with significant overhead and it is sometimes not possible, for example, because of the privacy concerns. We can't just store the whole data sets. For example, we know that brain employs some form of replay, but most of the proposed continued learning methods uh use some kind of replay. That is not biologically classified, for example, they store images in pixel level. We know we know that this is not happening in the brain.

A

B

A

Continued learning.

B

A

With the biological possible techniques that employees replay, but since our work nispa focuses on the structure side of the brain, I will not go details of this. That kind of related works or like how brain employees are playing so now. I will give few examples about these three two classes of content learning methods. The first one is an example for regularization.

A

So, for example, memory where synapses is one of the famous examples here we have important score for each weight and it determines the elasticity of the corresponding weight.

A

Regularization methods have selected plastics like the brain, because each weight has different plasticity and they can learn multiple tasks with without adding new modules. However, they have fixed and dance topologies or units are highly entangled.

A

This means even a slight change in the weights affect all other units, and that's why small changes accumulate throughout the network and these models still suffer from forgetting if the sequence is very long.

C

Second, barack, I have a quick question on that. I don't know if you want interruptions along the way or yeah yeah. So would you include synaptic intelligence in this category here? Yes, I would eat.

A

It okay yeah, for example, also elastic weight, consolidation.

A

So second example is progressive, neural networks. This is one of two main or like most well-known example of architectural methods with expanding networks.

A

The idea is that, after learning a task, we freeze the whole network, then we with new tasks. We expand the network by adding horizontally connected sub networks.

A

Here we have selected plasticity and dynamic topology, but the problem is that here the overhead is huge. The network size grows linearly with every new task and this method becomes quickly interactable.

A

My final example is continual learning, via neural pruning method or in short, clmp.

A

This is a good example of an architectural method with a fixed network, so.

B

A

Idea is that, at the end of each task, we select important units for the associated task. We isolate them by removing some connections and finally, we freeze the connections of this isolated sub network.

A

So instead of network growth, here we distribute available resources across different tasks, but it comes with few problems. The first problem is that this method works with dance connectivity, so isolation is very hard. We draw lots of connections to remove one part from other.

A

Also, as I said, this selection and isolation happens at the end of a task. So there is a performance loss because of this dropping operational removal of connections, and there is no recovery phase that can gain the performance back.

A

So network does not recover from this damage because of the connection, removal and finally, one there's one counterintuitive thing about this method- is that network loses connections with every task, because we need to remove connections to isolate certain parts. So, as the number of things we need to learn increases, the number of parameters are decreased, so we quickly route of capacity to learn future tasks.

A

Now I will talk about our methods. Nispa combines previously explained. Concepts of the memory in brain first, like the brain, is by uses sparse networks with fixed density to mitigate forgetting problems.

A

Second, spot dynamically changes, its network topology when learning a new task and third similar to selected plasticine. The brain is by freezes specific important connections to retain previously learned information and, finally, like the mammalian brain, dispellers, more and more tasks over time with fixed number of units, so a number of units or like connect number of units are fixed and number of connections are also fixed, but we change the topology of connectivity.

A

So in summary, nispa sequential learns multiple tasks. It does not need to revisit all the examples to remember them and it accumulates more knowledge into a fixed number of units.

A

Our method starts by dividing all units into three categories.

A

The first category is plastic units, all units start as plastic, and they are dedicated for learning future information.

A

Then we have stable units.

A

Their role is to preserve previous learned information and they serve as a memory for our methods.

A

Finally, we have candidates table units, they are kind of a transition state between plastic and stable units, and their role is to learn the current task that the model sees here is the transition diagram for these three states.

A

First, plastic units can become candidate, stable units or vice versa, and this is based on the activation of units for training. Examples on the given task.

A

Second, a candidate stable unit may become stable unit at the end of a task and stable unit is kind of terminal state in this transition diagram. So once a unit becomes stable unit, it will never become plastic unit, and I will talk about this point in the future. Works too.

A

Okay, as I said, we use activations for these transitions, so one may ask whether activation of a unit is a valid intrigue indicator of its contribution to a task and prior work suggests that the answer to this question is yes, so here I present a very simple example on any state asset, but the same observation holds in more complex architectures and data sets as well. So here we see that removal of highly active units has a detrimental effect on the performance of network.

A

Another concern might be whether too many units will be selected as stable. Of course, this will not leave enough elastic units for subsequent tasks, but experiments suggest that using rally activation leads to highly skilled perlar activation distribution.

A

So here I showed this phenomenon on mnist. But again the same observation holds in more complex data sets as well as, as you can see, we have large number of units with very low activation. They are on the left. Tail of this distribution and few units are in the right tail, which highlights that these units are highlighted.

A

This suggests that we can preserve most of the activation in the network by selecting only few units, so we will still have many plastic units for future tasks.

A

Another important point is that this skewness is enforced by value. Therefore it is, the skewness is high, even at the initialization, and it remains high throughout the training. So we can exploit this phenomenon at any particular time during the training.

A

So the next question is when we make this selection or transition between plastic to candidate stable units. So we start by dividing each task: training into multiple phases. With few epochs candidates table units are selected before each phase.

A

Algorithmically we have two steps for selection. The first step is we compute the total activation for all training samples.

A

Second, we find a small set of units that capture at least a fraction tau of layer activation here tower varies during varies dynamically during the training and it serve as a dynamic threshold for this selection.

A

So, as I said, we varied tau dynamically and we use relatively simple strategy. We just use cosine scheduling to decide on power. The motivation is as follows: the force initially we keep power high to give all units a chance to become stable.

A

This is because activations can vary in early phases, because our model is just start learning the task, so it is better to start with a large tower.

A

As the training proceeds activations becomes more stable and we start resting the selection even more so we will only have a few stable units.

C

A quick question on this, so it sounds like just make sure I understand correctly here.

B

C

Tau determines, does it determine which units are candidates or which units actually become stable, which units become.

A

Candidates become.

B

A

This selection, from like stable to sorry plastic to candidate stable units, is based on the activations during yeah.

A

I will go do like how you go from candidate stable into stable in, like few slides, so.

C

A

C

But even here, um okay, that that's helpful. Even here, though, even at the extreme end, it still seems, like you know, sixty percent of the units or eighty percent of the units could still become candidates. Oh okay, this is the tau value. Oh okay, it's not the percentage yeah. So.

A

We find a small set of units that capture, for example, 70 percent of to their activation right. Right now is 0.7 yeah.

B

There is a correlation.

A

Between number of units and tall, but it is not like direct correlation. Yes,.

C

If it's very skewed, like you showed earlier, it could still be a very small number of.

B

Actual units yeah.

A

So after the selection phase, we do rewiring and rewiring consists of dropping step which preserves the which helps preserving the knowledge, and we also have growing phase which promotes learning so in while dropping we drop connections from plastic units to candidate stable units.

A

This ensures that change in plastic units do not affect the candidate stable units, because, after this operation, candidate stable units do not receive any input from plastic units.

A

However, candidate stable units can still share knowledge through outgoing connections with plastic units.

A

Another way to interpret this is that dropping forms, input, output, paths that only pass through candidates table units, so we construct a sub network that consists only with candidates table units, and this sub network is dedicated to learn the task at hand.

A

After this dropping operation we perform growing and for growing we have two options.

A

First, we can drop grow, a connection between two plastic units. This type of connection growth, promotes learning new representations or we can grow a connection from stable to plastic units. This can be either candidate, stable or stable in it. I am just showing candidates table here. This type of connection group promotes reusing, previously learnt information.

A

Note that we drop for each dropped connection. We grow a sec same number of connection in the same layer, so we preserve the layer, wise density, which in turn preserves the density of the overall network.

A

We didn't uh like we have two type of connection group, but we didn't optimize how we choose between these two types, so it is basically random and I will discuss this point later in the future. Works.

A

Okay, we perform multiple rewiring cycles for each task and once the task ends, final set of candidate units becomes stable.

A

Here, our motivation is to form stable structures in the network that will serve as a lifelong memory for the learned task, just like the stable spines in the mammarian brain and recall that we enforce smaller tau as training proceeds, and we have this skewness in the activation.

A

So once we are at the end of the task, we will only select few stable units because we will have very small number of candidate stable units. So we still have many plastic units reserved for learning, more information.

A

After promoting candidates table units to stable units, we freeze the connections between stable units and this guarantees that grant and descent for future tasks will never alter stable units because they do not receive any input from plastic units and we freeze the other connections. So they will never change and serve as a memory.

C

And you freeze the weights as well or just the connectivity.

A

We freeze the weights yeah this.

B

Balloon connections.

A

Means the rate is frozen, so no updates, but hopefully we are not like freezing lots of parameters because we have few units and they are sparsely connected. So the intuition is that although we lose some capacitor of learning with freezing, it is not that dramatic.

C

Okay, but there's still going to be some maximum number of tasks that you can learn.

B

A

Based on that skewness thing, yeah yeah, we propose lots of hyperparameters to tune like the plasticity or like the stability, but since we have fixed number of units, fixed number of connections at the end of today, network will become like mostly frozen and will not have enough capacity to learn. From one point.

A

So, to sum up, we divide each task into multiple phases.

A

At the initialization we start with a randomly proven network, and all units are plastic.

A

Before each phase we select candidates table units initially nispa selects many candidates stable units, because tao is high as training proceeds. We start selecting fewer and fewer candidates table. Units selection is always followed by connection rewiring, and this selectional rewind cycle repeats multiple times until further phases start decreasing the performance so intuitively.

A

Instead of doing early stopping in epact level, we are doing early stopping in the phase level and once the task ends, we promote candidate, stable units to stay volumes and we freeze the connections between them and the same cycle repeats for the future task, but note that we carry stable units from previous tasks to future tasks. So that's why they serve as a memory for our method.

C

A

C

One more clarification on the so and then during this entire process, you're doing standard backdrop.

B

C

You know other than the fact that some weights are frozen.

A

Yeah some ways are frozen. We do rewiring sometimes before each phase, but besides that we use grand and descent.

A

So now I will briefly explain our evaluation scenario. We generate a sequence of tasks by slicing existing benchmark data sets into smaller tasks.

A

Each task introduces a set of new classes.

A

The task boundaries are available to the learners and we assume there is no access to samples of the previous tasks. So every task is visited only once and they are never revisited again by the model.

A

And last but not least, we provide task identifiers during testing.

A

So, first we start with the control, experiments, suppose new spacer p parameters and t tasks to learn. Then we have two control designs.

A

The first model is single task learner with p parameters. We denote it by stl and this is an upper bound because it corresponds to dedicating all parameters to a single task.

A

Second, we have single task learner with prt parameters. We call this str iso.

A

This is like a lower bound because it corresponds to partitioning network into three equal sized sub networks that are totally isolated from each other, and one thing to note is that this str and ester iso has the same depth with nispa. We are not playing with the depths in this experiment, so here we see that nispa nearly matches the performance of of stl and it significantly outperforms scrizo.

A

This highlights that this past success is not only due to parameter isolation, but sharing knowledge across tasks with this new connection that we grow is crucial part of the success.

A

But one thing to note is that we observe if the sequence is very long as as in the case of c420 tasks, the accuracy gap increases as nispa sees more tasks and, as you can see on the right, the problem is that we run out of plastic units, especially in the later layers, for example in the last convolutional layer or the penultimate linear layer before the outputs.

A

This is because these units have more specialized or higher level features that are less shareable and display eventually a lot of capacity in these layers, and we see a performance gap.

A

Next, we compare nispa with other continued learning methods. Here I present three regularization methods and one architectural method clmp, which is which I mentioned in the related works.

A

As you can see in this power performs baselines significantly and it is using 10 times up to 10 times fewer parameters, because we have sparse connectivity and the space lines have dense connectivity, so we are more efficient and we still have better performance.

A

Next, we investigate the optimal density level in which nispa performs the best. We found that optimal density is determined by multiple factors, such as data set, number of tasks, the domain of the tasks and modular architectures.

A

But although optimal density varies, it is always towards sparse networks.

A

When density is too low, the models suffers from underfitting because we don't have enough parameters, but when the dust is too high, we have too many entangled units and catastrophic, forgetting problem becomes much harder to solve.

A

To sum up, sparse is essential for nispas success in continued learning.

A

Finally, we analyze the effect of task similarity on the number of stable and plastic units to control the similarity between tasks. We have two task scenarios. The first task is the standard m list, and the second task is the amnesia permuted pixels.

A

We choose this task, because random permutations are equally challenging for multiplayer perceptron, because features are fed in a flattened, vector and multiplayer. Perceptron is not aware of like structured input, so all tasks are equally challenging. This means only variable is the in this experiment is how many pixels we permit, which determines the similarity of tasks. So we have this permutation knob that we can tweak in order to get similar tests for very different tasks.

A

Here on the x-axis, we have the degrees of permutations and on the y-axis we have number of additional stable units selected after the second task, and here we observe that, if tasks are similar, this automatically reuses existing knowledge and it does not select additional stable units.

A

So, in conclusion, displays a neuro inspired, continual learning approach it uses sparsity and dynamic connectivity to avoid catastrophic, forgetting and it outperforms baselines using up to 10 times fewer parameters.

A

However, there are lots of open questions about nispa. The first thing is that we know that brain selectively forget things, especially if they are not recalled frequently so as humans. We do not have perfect memory, so in future work we will explore strategies to unfreeze some of the stable units when the number of remaining plastic units drop below a certain level, so our hope is to have controlled forgetting mechanism which can return some capacity back to the model if we are seeing lots of tasks.

A

Second, as I mentioned, our connection growth has two type of connections, but besides that it is mostly random. So we we plan to use some network theoretical metrics to guide our connection growth decisions.

A

Third nispas starts with a randomly prone network and it evolves the topology while learning new tasks. So we plan to study evolution of this topology by using uh like from the lens of network science. We will analyze input, output paths and other network properties such as node degrees.

A

Maybe we can find some interesting patterns as or motives in the network that associated with lifelong memories over we can diagnose some of the problems in nispa and finally, besides sparse connectivity, we know that brain employs, sparse coding or sparse activations.

A

We want to explore strategies to simultaneously exploit both type of sparsities, for example. Maybe with sparse coding, we can have more skewed activations. This will lead selecting even fewer stable units and we can save more plastic units for future.

A

C

Great. Thank you barack.

D

Thanks that was great, so I I have uh one question um so conventionally.

D

In the early days, when they trained excuse me networks, they noticed features low-level uh little image, processing, features evolving in the lower layers and then gradually going up to more general stuff.

D

So one of the things that strikes me is that when you have a set of tasks- and you do your network analysis as to pause, the topology of where the stable units are, whether you ever considered moving the boundary of the layer, because you keep the sparsity constant between the layers, but maybe whether you can compress say the earlier layers to those common sub networks that are being used by all the tasks and then allow the other networks expand outwards to handle the special cases so kind of creating a a formal separation between those things that are general and those things that are task.

D

A

I see that's quite interesting point. We didn't consider that because, like as.

B

A

Have like we are applying our algorithm locally in each layer. The one motivation is that if we use global information, let's say we analyze all the units and make decisions based on that.

A

Probably it will be less biologically possible because we know that neurons make decisions locally, and this is one of the reasons that we try to keep our algorithm just based on local activations. Instead of let's say, removing one part of or like having isolation in the upper parts and having like real using in the shared features.

A

This will require our algorithm to be making some global decisions. I guess I.

D

Think well, or at least layer to layer decisions because in in some sense the whole notion of a finite set of layers is not necessarily biologically plausible.

B

A

Yeah definitely.

D

So what I'm saying is is that you know anything you do to move the boundaries there I think is, is fair game because of what what the whole notion of layers structure imposes upon the whole learning problem to begin with,.

A

Yes, okay, yeah, that's quite interesting thought yeah yeah, probably in the brain. We don't have this kind of like fixed number of their structure. So yeah I got your point. Yes still. We can suggest that this is biologically possible if we just have like trans, like the movement from what, like removing, maybe from one layer to adding it to the next layer, is still biological possible.

D

Yeah yeah, I mean you can consider those those people have networks that have a common layer. Then they fan out to other sub layers where there's no crossover between them. It depends upon how the architecture of the brain is is wired, and so you have some degrees of freedom there to isolate things and still say yeah well, okay, biologically possible. Everything is not even potentially connected to everything else. So how do we?

D

How do we? How do we kind of you know, allow that to evolve in that kind of natural fashion? But anyway you you? You have a great framework there in which to explore a bunch of these ideas. So I applaud you for that.

A

Yeah yeah thanks for a great suggestion. Actually, this could be like a very good future direction to investigate more, and I think this kind of approach will be very interesting for the people in neural architecture search domain, because some people investigate this type of strategies to evolve the network based on the task getting like optimal structure for the human task so yeah. This would be a very interesting direction to take.

C

Yeah, this is really interesting. Thank you. I forget in your benchmark. Did you compare against the uh non-continual learning version of the thing? So if you train the network on all the tasks simultaneously, um I forget, if you oh, okay,.

A

Yeah we have that kind of analysis, but not in this slides. The actual there's, like a small implicit uh caveat about com, okay, it is called like some people called multitask learner, which learns all the tasks at once and the problem.

C

Yeah, I mean just just this standard batch training of scifir10. You know just uh because that that also gives an upper bound uh of what you can reach it's. You know it's using the same number of parameters as this current yeah.

A

But yeah, but the network is that this last item in the evaluation scenario. So we provide task identifiers during testing so.

B

A

Let's say instead of in c farther: instead of learning one versus line classification, our model learns binary classification, but if you use multitask learner, it will learn a much complex task. Just like a single model, it will learn one versus nine classifications.

C

B

C

Okay, so your output is only two you're, only comparing two things at a time here, two.

A

C

A time I see you, you don't.

A

Have plus, for example, yeah hundred, we have five classes for each task, then we also have some experiment with emness which is like x and that version of amnes with handwritten letters here we have like we go up to 15 classes per task, but still comparing it with the standard learner is not a fair game for, and it is very easy to outperform it, because it performs much complex tasks.

C

Yeah yeah: it's it's trying to compare one against nine as that's supposed to.

B

C

I see so so you do outperform that.

A

B

A

But like intuitively, it is not the same task.

C

It's not the same yeah yeah I get it yeah.

A

And all in the actual people, we also extend nisba with replay methods. We use very simple, replay, very simple, replay mechanism on top of nispa. It is not like.

B

A

We are not claiming that part is biologically possible, but we also show that this, our framework can beat other replay methods with very simple repair, on top of it, so like replay, is kind of orthogonal to nispa and nisba can benefit from replay and outperform that kind of methods too. So we have like more extended evaluation in the actual paper right.

C

Have you thought about how you can remove this limitation of task identifiers provided, okay,.

A

It is like one of the main challenges and I hope that we will find a way, but there are lots of problems with that because first, if we do not perform any type of replay, we're using the previous information causes interference. So there is like a unavoidable trade-off between reusing the representations and mixing two classes between each other, so replay solves it, but it is biologically impossible.

A

My hope is that we will come up with a mechanism to maybe replay the activations or some reminder of the previous tasks instead of just storing the previous samples inspired by the neuroscience, and we can eliminate this problem, because the main problem is that nispa does not use the task task identifiers at any layer except output there. So we only use task identifiers to have a isolation in the output layer, so I think, with some additional tricks.

A

We can avoid this problem because actually, most of the continual learning methods use this kind of task identifies throughout the architecture and it's a big limitation.

C

I think it is because it's not very useful if you have to know it during testing. You.

A

C

Practical scenario.

A

Yeah, I think it depends on the task, for example, now I'm showing in the classification task, but let's say we are using reinforcement, learning tasks in that kind of scenario it is maybe more makes us assume we will have task identifiers, even during testing.

B

C

Yeah but um reinforcement learning introduces a whole set of other challenges.

B

C

Yeah, extending this to reinforcement, learning uh would be interesting and how you uh yeah.

A

B

A

Was talking this with some people that works in reinforcement, learning and talking about how we can use this algorithm in the reinforcement learning setting, for example, one problem is that in the reinforcement learning it takes huge amount of time to learn the first policy that will make things much easier, but since we make connection rewiring, maybe removing single connection will make us forget this very important initial policy. So we will stuck in this loop where we're trying to learn the same policy over and over again, and we can't stop learning to task. So.

A

Okay, reinforcement learning comes with lots of problems, and it will be very interesting and challenging to apply most of the continued learning, algorithms to reinforcement, learning settings.

C

Yeah we used- um I don't know if you've seen our paper from earlier this year, but we applied this to uh applied our continued learning thing to multi-task reinforcement learning. So it wasn't a continual learning scenario, but it was a multi-task learning scenario, a reinforcement, learning scenario um yeah. We got it to work, but there are definitely a lot of challenges. The reinforcement learning stuff is extremely noisy as well.

C

What are the challenges? Cool? Have you thought about a simpler question? Maybe is applying this to convolutional networks? Oh okay, actually, that's more straightforward. I think.

A

We are using convolution neural networks for cfr, 10 and c400.

A

I think the notion is just how you define units in neural networks, and I am just showing it in like multi-layer perceptron setting, but once you define the idea of units, the same ideas extends to convolution neural networks and actually, in the paper I explained how you can transl translation get a transition from multiplayer perceptron convolution neural networks very easily.

C

And what do you consider a unit in that case then? So each.

A

Filter then it becomes like connections become matrices instead of these single connections and then dropping becomes setting all metrics to zero. Instead of like setting one weight to zero.

C

So each filter is considered a individual unit. Yes,.

A

Yeah, that's the.

C

A

B

A

Like the idea is basically the same.

B

A

Cool one may be interesting thing it would be. Is that if this, I I'm not sure whether this algorithm may scale to very very deep architectures. So we experiment with the relatively smaller convolutional neural network, and it would be interesting to test it on, let's say, rest net 50 or that kind of architecture right.

C

I don't think I think one interesting thing is: you know you showed the the number of of units that are available um gets more challenging with convolutional networks, because typically a much smaller number of filters, yeah.

B

C

Have uh 32 or 64 or 128 filters, as opposed to you, know,.

A

C

000 units or something.

A

Yeah, maybe one thing is that convolutional networks come with a challenge, but I think they are more shareable. So right, maybe we can run out of plastic units in some years, but since they learn to extract some abstract features, they will serve well, even though they are like frozen and like. We always assume that we will be seeing same tasks from same domain. So since we are seeing like images old in every test, maybe there's lots of this share between tasks and.

C

Yeah yeah, particularly at the earlier layers yeah, so.

D

I have a question: uh do you use conventional dropout while you're training these.

B

D

Okay, so when you were talking about the difficulty in reinforcement learning that, if you drop a connection, you might spend an ordinary amount of time trying to relearn it. I'm just kind of curious. If you have a connection that you decide to drop, whether you do it probabilistically, rather than a hard drop and seeing if the network can change its mind if you wish, because it tries it with and without multiple times.

A

Okay, I think I got your point like I think, if we let the model decide on the connection uh rewiring itself, instead of us like putting some rules to it, there will be goal of optimizing the current task and it's a chicken and egg problem, because we don't want everything to be optimized for the current task, and the reason is the reason that we make a hard drop is that we want to ensure these plastic units do not spread their change to like the stable structures in the other parts.

A

So I think we need to have that kind of hard drop, because this, if we make it probably it becomes kind of regularization method, and, to be honest, I don't know like regularization methods. That is a very robust, catastrophic, forgetting most of the time they failed in those sequences.

D

Well, I I guess it would be a way.

D

Maybe it's not a uh an autonomous mechanism but it'd be way to decide if it's low in the current task, but it has utility in other tasks.

D

You have a gradual way as you slew, through the task trainings of the sag. I've marked this thing as potentially not useful, but if I make if I disappear entirely, I have no chance of reinforcing it again.

A

D

So to the goal of allowing this thing to be a general learner, it might allow you a softer edge to that decision. Point.

A

I see, probably this cat also paved the way for one of the items that I discuss in future works, which is selective unfreezing strategies, for example, yeah.

D

Yeah exactly but you could you could do it in a principled manner that you say I've marked this thing. You know I mean it could be proportional to how much it's low activation. If it's dead dead. You know, then you say: okay low probability for coming back, but if I'm right on the threshold, maybe I want to have a softer edge to it, even though not on off it's kind of like, rather than modifying the weight. So it's always there.

D

You have a soft edge. That's a probabilistic thing where it's sometimes there, sometimes not so it becomes it's potentially possible. The network learns how to be robust against it being there or not, but without you know, catastrophically killing it, so it could never recover.

A

I see that's quite interesting. Yeah, probably we want to have that kind of mechanism, because now we we are very concerned with the stability and we make very hard decisions to have stability, but this is not happening in the brain. We gradually forget things. We don't need.

B

A

Memory for continual learning- I guess.

D

Yeah so anytime, I see something: that's a hard threshold. I instantly my uh you know, antennae go up and say: okay,.

B

E

um I have just a general intellectual curiosity question: um have you thought about looking at how your method affects model calibration or adversarial adversarial robustness? It seems like um enforcing sparsity and uh and whatnot might actually change uh something about the logits at the last step of the model, I'd kind of be curious if you get something like better calibration for free with this method,.

A

Oh, I am not very aware of that type of research so to bands. I I didn't look into that.

B

Okay, maybe like.

A

Some pointers, some papers that I can read it would be great if you can share yeah yeah.

E

A

C

Okay, thank you very much. I need to go on to my next meeting, but thanks so much for coming and presenting to us. This is really cool.