反向傳播(Back Propagation)與神經網絡(Neural Network)

圖檔來自:http://galaxy.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html

程式實作參考:http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/

實驗資料來自于:http://yann.lecun.com/exdb/mnist/

感謝。侵删。

1.前向傳播與反向傳播

• 1.1 網絡結構

  

• 單個神經元的結構

  

• 1.2前向傳播算法

  

  

  

  

  

  

• 1.3後向傳播算法

  

  

  

  

  

  

• 1.4權值更新

  

  

  

  

  

  

2.神經網路的實作

• 程式使用資料擷取位址:https://raw.githubusercontent.com/lxrobot/lxrobot-s-code/master/data178x197.npy

  #!/usr/bin/env python2
  
  
  # -*- coding: utf-8 -*-
  
  """
  Created on Thu Jul 19 19:11:28 2018
  @author: rd
  """
  from __future__ import division
  import numpy as np
  
  def sig(_z):
      _y=/(+np.exp(-_z))
      return _y
  
  def predict(model,X):
      W1, b1, W2, b2 = model['W1'], model['b1'], model['W2'], model['b2']
      z1 = X.dot(W1) + b1 
      #a1 = np.tanh(z1) 
      a1=sig(z1)
      z2 = a1.dot(W2) + b2 
      exp_scores = np.exp(z2) 
      probs = exp_scores / np.sum(exp_scores, axis=, keepdims=True)
      return probs
  
  def get_accuracy(model,X,Y):
      probs=predict(model,X)
      pre_Y=np.argmax(probs,axis=)
      comp=pre_Y==Y
      return len(np.flatnonzero(comp))/Y.shape[]
  
  def get_loss(model,X,Y,reg_lambda):
      probs=predict(model,X)
      # Calculating the loss 
      corect_logprobs = -np.log(probs[range(X.shape[]), Y]) 
      data_loss = np.sum(corect_logprobs) 
      # Add regulatization term to loss  
      data_loss += reg_lambda/ * (np.sum(np.square(model['W1']))+ np.sum(np.square(model['W2'])))
      loss = /X.shape[] * data_loss     
      return loss
  
  def nn_model(X,Y,nn_hdim,nn_output_dim,steps,epsilon,reg_lambda):
      np.random.seed() 
      W1 = np.random.randn(X.shape[], nn_hdim)  
      b1 = np.ones((, nn_hdim)) 
      W2 = np.random.randn(nn_hdim, nn_output_dim)
      b2 = np.ones((, nn_output_dim)) 
  
      model={}
  
      for i in xrange(steps):
          ###forward propagation
          Z1=np.dot(X,W1)+b1
          #a1=np.tanh(Z1)
          a1=sig(Z1)
          Z2=np.dot(a1,W2)+b2
          #softmax output
          exp_score=np.exp(Z2)
          prob = exp_score/np.sum(exp_score,axis=,keepdims=)
  
          #Backward Propagation
          delta3=prob
          delta3[range(X.shape[]),Y]-=
          dW2 = np.dot(a1.T,delta3)
          delta2=np.dot(delta3,W2.T)*(-np.power(a1,))
          dW1 = np.dot(X.T,delta2)
  
          #update the weight value
          dW2+=reg_lambda*W2
          dW1+=reg_lambda*W1
  
          W2+=-epsilon*dW2
          W1+=-epsilon*dW1
  
          if i%==:
              model = { 'W1': W1, 'b1': b1, 'W2': W2, 'b2': b2}
              print "The {} steps, Loss = {:2.5f}, Accaracy = {:2.5f}".format(i,
                         get_loss(model,X,Y,reg_lambda),
                         get_accuracy(model,X,Y))
  
      return model
  
  def main():
      """
      The data is saved in a 57x197 numpy array with a random order,
      197=14*14+1,14 is the image size, 1 is the label.
      """
      datas=np.load('data178x197.npy')
      np.random.seed()
      np.random.shuffle(datas)
      sp=int(datas.shape[]/)
      train_X=datas[:sp,:-]
      train_Y=datas[:sp,-]
      test_X=datas[sp:,:-]
      test_Y=datas[sp:,-]
  
      reg_lambda=
      epsilon=
      steps=
      nn_output_dim=   
      nn_hdim=
      model=nn_model(train_X,train_Y,nn_hdim,nn_output_dim,steps,epsilon,reg_lambda)
      print"The test accuracy is {:2.5f}".format(get_accuracy(model,test_X,test_Y))
  if __name__=='__main__':
      main()
  >>>python nn_model.py
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The  steps, Loss = , Accaracy = 
  The test accuracy is 
             

refer

[1] http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/

[2] http://galaxy.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html

[3] http://yann.lecun.com/exdb/mnist/