天天看點

反向傳播(Back Propagation)與神經網絡(Neural Network)

圖檔來自:http://galaxy.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html

程式實作參考:http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/

實驗資料來自于:http://yann.lecun.com/exdb/mnist/

感謝。侵删。

1.前向傳播與反向傳播

  • 1.1 網絡結構
    反向傳播(Back Propagation)與神經網絡(Neural Network)
  • 單個神經元的結構
    反向傳播(Back Propagation)與神經網絡(Neural Network)
  • 1.2前向傳播算法
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
  • 1.3後向傳播算法
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
  • 1.4權值更新
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)
    反向傳播(Back Propagation)與神經網絡(Neural Network)

2.神經網路的實作

  • 程式使用資料擷取位址:https://raw.githubusercontent.com/lxrobot/lxrobot-s-code/master/data178x197.npy
    #!/usr/bin/env python2
    
    
    # -*- coding: utf-8 -*-
    
    """
    Created on Thu Jul 19 19:11:28 2018
    @author: rd
    """
    from __future__ import division
    import numpy as np
    
    def sig(_z):
        _y=/(+np.exp(-_z))
        return _y
    
    def predict(model,X):
        W1, b1, W2, b2 = model['W1'], model['b1'], model['W2'], model['b2']
        z1 = X.dot(W1) + b1 
        #a1 = np.tanh(z1) 
        a1=sig(z1)
        z2 = a1.dot(W2) + b2 
        exp_scores = np.exp(z2) 
        probs = exp_scores / np.sum(exp_scores, axis=, keepdims=True)
        return probs
    
    def get_accuracy(model,X,Y):
        probs=predict(model,X)
        pre_Y=np.argmax(probs,axis=)
        comp=pre_Y==Y
        return len(np.flatnonzero(comp))/Y.shape[]
    
    def get_loss(model,X,Y,reg_lambda):
        probs=predict(model,X)
        # Calculating the loss 
        corect_logprobs = -np.log(probs[range(X.shape[]), Y]) 
        data_loss = np.sum(corect_logprobs) 
        # Add regulatization term to loss  
        data_loss += reg_lambda/ * (np.sum(np.square(model['W1']))+ np.sum(np.square(model['W2'])))
        loss = /X.shape[] * data_loss     
        return loss
    
    def nn_model(X,Y,nn_hdim,nn_output_dim,steps,epsilon,reg_lambda):
        np.random.seed() 
        W1 = np.random.randn(X.shape[], nn_hdim)  
        b1 = np.ones((, nn_hdim)) 
        W2 = np.random.randn(nn_hdim, nn_output_dim)
        b2 = np.ones((, nn_output_dim)) 
    
        model={}
    
        for i in xrange(steps):
            ###forward propagation
            Z1=np.dot(X,W1)+b1
            #a1=np.tanh(Z1)
            a1=sig(Z1)
            Z2=np.dot(a1,W2)+b2
            #softmax output
            exp_score=np.exp(Z2)
            prob = exp_score/np.sum(exp_score,axis=,keepdims=)
    
            #Backward Propagation
            delta3=prob
            delta3[range(X.shape[]),Y]-=
            dW2 = np.dot(a1.T,delta3)
            delta2=np.dot(delta3,W2.T)*(-np.power(a1,))
            dW1 = np.dot(X.T,delta2)
    
            #update the weight value
            dW2+=reg_lambda*W2
            dW1+=reg_lambda*W1
    
            W2+=-epsilon*dW2
            W1+=-epsilon*dW1
    
            if i%==:
                model = { 'W1': W1, 'b1': b1, 'W2': W2, 'b2': b2}
                print "The {} steps, Loss = {:2.5f}, Accaracy = {:2.5f}".format(i,
                           get_loss(model,X,Y,reg_lambda),
                           get_accuracy(model,X,Y))
    
        return model
    
    def main():
        """
        The data is saved in a 57x197 numpy array with a random order,
        197=14*14+1,14 is the image size, 1 is the label.
        """
        datas=np.load('data178x197.npy')
        np.random.seed()
        np.random.shuffle(datas)
        sp=int(datas.shape[]/)
        train_X=datas[:sp,:-]
        train_Y=datas[:sp,-]
        test_X=datas[sp:,:-]
        test_Y=datas[sp:,-]
    
        reg_lambda=
        epsilon=
        steps=
        nn_output_dim=   
        nn_hdim=
        model=nn_model(train_X,train_Y,nn_hdim,nn_output_dim,steps,epsilon,reg_lambda)
        print"The test accuracy is {:2.5f}".format(get_accuracy(model,test_X,test_Y))
    if __name__=='__main__':
        main()
    >>>python nn_model.py
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The  steps, Loss = , Accaracy = 
    The test accuracy is 
               

refer

[1] http://www.wildml.com/2015/09/implementing-a-neural-network-from-scratch/

[2] http://galaxy.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.html

[3] http://yann.lecun.com/exdb/mnist/

繼續閱讀