Keras Layer自定義

簡單層實作

實作一個簡單層需要首先繼承 layers.Layer 類即可,如下是官方網站上的例子:

from keras import backend as K
from keras.engine.topology import Layer
import numpy as np

class MyLayer(Layer):

    def __init__(self, output_dim, **kwargs):
        self.output_dim = output_dim
        super(MyLayer, self).__init__(**kwargs)

    def build(self, input_shape):
        # Create a trainable weight variable for this layer.
        self.kernel = self.add_weight(name='kernel', 
                                      shape=(input_shape[1], self.output_dim),
                                      initializer='uniform',
                                      trainable=True)
        super(MyLayer, self).build(input_shape)  # Be sure to call this somewhere!

    def call(self, x):
        return K.dot(x, self.kernel)

    def compute_output_shape(self, input_shape):
        return (input_shape[0], self.output_dim)
           

如上所示, 其中有三個函數需要我們自己實作:

• build() 用來初始化定義weights, 這裡可以用父類的self.add_weight() 函數來初始化資料, 該函數必須将 self.built 設定為True, 以保證該 Layer 已經成功 build , 通常如上所示, 使用 super(MyLayer, self).build(input_shape) 來完成
• call() 用來執行 Layer 的職能, 即目前 Layer 所有的計算過程均在該函數中完成
• compute_output_shape() 用來計算輸出張量的 shape

正常DL都是一個forward, backword, update 三個流程,而在 keras 中對于單層 Layer 來說,通過将可訓練的權應該在這裡被加入清單`self.trainable_weights中。其他的屬性還包括self.non_trainabe_weights（清單）和self.updates（需要更新的形如（tensor, new_tensor）的tuple的清單）。你可以參考BatchNormalization層的實作來學習如何使用上面兩個屬性。這個方法必須設定self.built = True，可通過調用super([layer],self).build()實作

loss 以及參數更新

詳細檢視了下 add_weight 函數實作如下（keras/engine/topology.py）：

def add_weight(self,
                   name,
                   shape,
                   dtype=None,
                   initializer=None,
                   regularizer=None,
                   trainable=True,
                   constraint=None):
        """Adds a weight variable to the layer.
        # Arguments
            name: String, the name for the weight variable.
            shape: The shape tuple of the weight.
            dtype: The dtype of the weight.
            initializer: An Initializer instance (callable).
            regularizer: An optional Regularizer instance.
            trainable: A boolean, whether the weight should
                be trained via backprop or not (assuming
                that the layer itself is also trainable).
            constraint: An optional Constraint instance.
        # Returns
            The created weight variable.
        """
        initializer = initializers.get(initializer)
        if dtype is None:
            dtype = K.floatx()
        weight = K.variable(initializer(shape),
                            dtype=dtype,
                            name=name,
                            constraint=constraint)
        if regularizer is not None:
            self.add_loss(regularizer(weight))
        if trainable:
            self._trainable_weights.append(weight)
        else:
            self._non_trainable_weights.append(weight)
        return weight
           

從上述代碼來看通過 add_weight 建立的參數，通過 regularizer 函數來計算 loss, 如果 trainable 設定 True ，則該生成的 self._trainable_weights, 可以通過 regularizer 來建構 loss

具體訓練過程參見: keras/engine/training.py