關于在神經網絡訓練中使用批量歸一化（batch_normalization）時遇到的參數

文章目錄

• 批量歸一化
• • 參數 momentum
  • 參數 epsilon
  • 參數 training
• 執行個體

批量歸一化

def batch_normalization(inputs,
                        axis=-1,
                        momentum=0.99,
                        epsilon=1e-3,
                        center=True,
                        scale=True,
                        beta_initializer=init_ops.zeros_initializer(),
                        gamma_initializer=init_ops.ones_initializer(),
                        moving_mean_initializer=init_ops.zeros_initializer(),
                        moving_variance_initializer=init_ops.ones_initializer(),
                        beta_regularizer=None,
                        gamma_regularizer=None,
                        beta_constraint=None,
                        gamma_constraint=None,
                        training=False,
                        trainable=True,
                        name=None,
                        reuse=None,
                        renorm=False,
                        renorm_clipping=None,
                        renorm_momentum=0.99,
                        fused=None,
                        virtual_batch_size=None,
                        adjustment=None):
           

參數 momentum

移動平均線的動量

參數 epsilon

小的浮點數添加到方差中避免除以0

Small float added to variance to avoid dividing by zero.
           

參數 training

要麼是Python布爾值，要麼是TensorFlow布爾标量張量(如一個占位符)。是否在訓練模式下傳回輸出(使用目前批次的統計資料進行規範化)或推理模式(使用移動統計資料進行标準化)。注:請務必設定此選項參數正确，否則您的訓練/推理将不起作用正常。

training: Either a Python boolean, or a TensorFlow boolean scalar tensor
      (e.g. a placeholder). Whether to return the output in training mode
      (normalized with statistics of the current batch) or in inference mode
      (normalized with moving statistics). **NOTE**: make sure to set this
      parameter correctly, or else your training/inference will not work
      properly.
           

執行個體

training = tf.placeholder_with_default(False, shape=(), name='training')

hidden1 = tf.layers.dense(X, n_hidden1, name="hidden1") #全連接配接層，在DNN中視為隐藏層也可
bn1 = tf.layers.batch_normalization(hidden1, training=training, momentum=0.9) #批量歸一化,tensorflow使用batch_normalization（）函數來實作中心化和歸一化輸入，但是你必須自己計算均值和标準方差
bn1_act = tf.nn.elu(bn1) #計算線性指數函數，對輸入值進行處理

hidden2 = tf.layers.dense(bn1_act, n_hidden2, name="hidden2") #全連接配接層
bn2 = tf.layers.batch_normalization(hidden2, training=training, momentum=0.9)