當針對一系列學習問題進行優化時，卷積神經網絡會遭受災難性的遺忘：當它們滿足目前訓練示例的目标時，它們在先前任務上的表現将急劇下降。在這項工作中，我們引入了一個新穎的架構來通過條件計算解決這個問題。我們為每個卷積層配備特定于任務的選通子產品，選擇要應用于給定輸入的過濾器。這樣，我們實作了兩個吸引人的特性。首先，門的執行模式允許識别和保護重要的過濾器，進而確定先前學習的任務的模型性能不會損失。其次，通過使用稀疏性目标，我們可以促進選擇有限的核心集，進而保留足夠的模型能力來消化新任務。現有的解決方案在測試時需要了解每個示例所屬的任務。但是，在許多實際情況下可能無法獲得此知識。是以，我們另外引入了一個任務分類器，該分類器預測每個示例的任務标簽，以處理其中無法使用任務預告片的設定。我們在四個持續學習資料集上驗證了我們的建議。結果表明，無論是否存在任務預言，我們的模型始終優于現有方法。值得注意的是，在Split SVHN和Imagenet-50資料集上，我們的模型的w.r.t.精度提高了23.98％和17.42％。競争方法。
原文标題：Conditional Channel Gated Networks for Task-Aware Continual Learning
原文：Convolutional Neural Networks experience catastrophic forgetting when optimized on a sequence of learning problems: as they meet the objective of the current training examples, their performance on previous tasks drops drastically. In this work, we introduce a novel framework to tackle this problem with conditional computation. We equip each convolutional layer with task-specific gating modules, selecting which filters to apply on the given input. This way, we achieve two appealing properties. Firstly, the execution patterns of the gates allow to identify and protect important filters, ensuring no loss in the performance of the model for previously learned tasks. Secondly, by using a sparsity objective, we can promote the selection of a limited set of kernels, allowing to retain sufficient model capacity to digest new tasks.Existing solutions require, at test time, awareness of the task to which each example belongs to. This knowledge, however, may not be available in many practical scenarios. Therefore, we additionally introduce a task classifier that predicts the task label of each example, to deal with settings in which a task oracle is not available. We validate our proposal on four continual learning datasets. Results show that our model consistently outperforms existing methods both in the presence and the absence of a task oracle. Notably, on Split SVHN and Imagenet-50 datasets, our model yields up to 23.98% and 17.42% improvement in accuracy w.r.t. competing methods.
原文作者：Davide Abati, Jakub Tomczak, Tijmen Blankevoort, Simone Calderara, Rita Cucchiara, Babak Ehteshami Bejnordi
原文位址：https://arxiv.org/abs/2004.00070
用于任務感覺的持續學習的條件通道門控網絡(CS.CV).pdf