分類問題
在二維空間，實際上可以等效于拟合最佳直線，将所有點分類
三圍空間，就是拟合最佳平面
擴充到n維空間
滿足n維空間的各向量次元
W:(1xn), x:(nx1), b:(1x1)
感覺器
把方程式進行編碼，形成下圖的節點連結方式
把方程式參數提取出來作為權重，特征向量作為輸入節點，方程結果作為輸出
把偏置值也作為一個權重(對應特征為固定值1)，最後檢查結果是否大于0,如果是，傳回1,否則傳回0；
提煉出通用公式：
階躍函數 Setp Function
輸入值不小于0,則輸出1，否則輸出0；
用感覺器實作簡單邏輯
AND運算
%matplotlib inline
import pandas as pd
# TODO: Set weight1, weight2, and bias
weight1 = 1.0
weight2 = 1.0
bias = -1.5
# DON'T CHANGE ANYTHING BELOW
# Inputs and outputs
test_inputs = [(0, 0), (0, 1), (1, 0), (1, 1)]
correct_outputs = [False, False, False, True]
outputs = []
# Generate and check output
for test_input, correct_output in zip(test_inputs, correct_outputs):
linear_combination = weight1 * test_input[0] + weight2 * test_input[1] + bias
output = int(linear_combination >= 0)
is_correct_string = 'Yes' if output == correct_output else 'No'
outputs.append([test_input[0], test_input[1], linear_combination, output, is_correct_string])
# Print output
num_wrong = len([output[4] for output in outputs if output[4] == 'No'])
output_frame = pd.DataFrame(outputs, columns=['Input 1', ' Input 2', ' Linear Combination', ' Activation Output', ' Is Correct'])
if not num_wrong:
print('Nice! You got it all correct.\n')
else:
print('You got {} wrong. Keep trying!\n'.format(num_wrong))
print(output_frame.to_string(index=False)) Nice! You got it all correct.
Input 1 Input 2 Linear Combination Activation Output Is Correct
0 0 -1.5 0 Yes
0 1 -0.5 0 Yes
1 0 -0.5 0 Yes
1 1 0.5 1 Yes OR 運算
NOT (‘非’)
%matplotlib inline
import pandas as pd
# TODO: Set weight1, weight2, and bias
weight1 = 2.5
weight2 = -4.5
bias = 1.0
# DON'T CHANGE ANYTHING BELOW
# Inputs and outputs
test_inputs = [(0, 0), (0, 1), (1, 0), (1, 1)]
correct_outputs = [True, False, True, False]
outputs = []
# Generate and check output
for test_input, correct_output in zip(test_inputs, correct_outputs):
linear_combination = weight1 * test_input[0] + weight2 * test_input[1] + bias
output = int(linear_combination >= 0)
is_correct_string = 'Yes' if output == correct_output else 'No'
outputs.append([test_input[0], test_input[1], linear_combination, output, is_correct_string])
# Print output
num_wrong = len([output[4] for output in outputs if output[4] == 'No'])
output_frame = pd.DataFrame(outputs, columns=['Input 1', ' Input 2', ' Linear Combination', ' Activation Output', ' Is Correct'])
if not num_wrong:
print('Nice! You got it all correct.\n')
else:
print('You got {} wrong. Keep trying!\n'.format(num_wrong))
print(output_frame.to_string(index=False)) Nice! You got it all correct.
Input 1 Input 2 Linear Combination Activation Output Is Correct
0 0 1.0 1 Yes
0 1 -3.5 0 Yes
1 0 3.5 1 Yes
1 1 -1.0 0 Yes 用感覺器實作邏輯運算 - XOR （“異或”）
A: AND
B: OR
C: NOT
通過簡單邏輯的重新組合，可以表達更複雜的邏輯。
感覺器是神經網絡的最基礎單元，
如何拟合最佳感覺器方程？
先給待拟合參數任意初始值，然後檢查所有學習資料
正确分類的點，不做處理，凡發現錯誤點，更新參數，使分類直線更靠近自己
調整直線，更新參數的方法，類似線性回歸中的拟合直線
感覺器算法實作
import numpy as np
# Setting the random seed, feel free to change it and see different solutions.
np.random.seed(42)
def stepFunction(t):
if t >= 0:
return 1
return 0
def prediction(X, W, b):
return stepFunction((np.matmul(X,W)+b)[0])
# TODO: Fill in the code below to implement the perceptron trick.
# The function should receive as inputs the data X, the labels y,
# the weights W (as an array), and the bias b,
# update the weights and bias W, b, according to the perceptron algorithm,
# and return W and b.
def perceptronStep(X, y, W, b, learn_rate = 0.01):
# Fill in code
for i in range(len(X)):
y_hat = prediction(X[i],W,b)
if y[i]-y_hat == 1:
W[0] += X[i][0]*learn_rate
W[1] += X[i][1]*learn_rate
b += learn_rate
elif y[i]-y_hat == -1:
W[0] -= X[i][0]*learn_rate
W[1] -= X[i][1]*learn_rate
b -= learn_rate
return W, b
# This function runs the perceptron algorithm repeatedly on the dataset,
# and returns a few of the boundary lines obtained in the iterations,
# for plotting purposes.
# Feel free to play with the learning rate and the num_epochs,
# and see your results plotted below.
def trainPerceptronAlgorithm(X, y, learn_rate = 0.01, num_epochs = 25):
x_min, x_max = min(X.T[0]), max(X.T[0])
y_min, y_max = min(X.T[1]), max(X.T[1])
W = np.array(np.random.rand(2,1))
b = np.random.rand(1)[0] + x_max
# These are the solution lines that get plotted below.
boundary_lines = []
for i in range(num_epochs):
# In each epoch, we apply the perceptron step.
W, b = perceptronStep(X, y, W, b, learn_rate)
boundary_lines.append((-W[0]/W[1], -b/W[1]))
return boundary_lines
if __name__ == "__main__":
# perform perceptron
data = np.loadtxt('data.csv', delimiter = ',')
print("data.type:",type(data),data.reshape)
X = data[:,:-1]
y = data[:,-1]
regression_coef = trainPerceptronAlgorithm(X, y) #傳回訓練過程多組系數
# plot the results
import matplotlib.pyplot as plt
plt.figure()
X_ = data[:,0]
y_ = data[:,1]
data1 = [x for x in data if x[2] > 0]
data0 = [x for x in data if x[2] == 0]
X_min = X_.min()
X_max = X_.max()
data0_ = np.array(data0)
X0_ = data0_[:,0]
Y0_ = data0_[:,1]
data1_ = np.array(data1)
X1_ = data1_[:,0]
Y1_ = data1_[:,1]
plt.scatter(X1_, Y1_, zorder = 3, c='b')
plt.scatter(X0_, Y0_, zorder = 3, c='r')
counter = len(regression_coef)
#print(regression_coef)
'''
想畫多根直線，直覺看分類線移動過程，代碼有問題
for W, b in regression_coef:
counter -= 1
color = [1 - 0.92 ** counter for _ in range(3)]
#print(color)
Y_min = X_min * W + b
Y_max = X_max * W + b
if Y_min > 1:
Y_min = 1
X_min = (1 - b)/W
if Y_min < 0:
Y_min = 0
X_min = (0-b)/W
if Y_max > 1:
Y_max = 1
X_max = (1 - b)/W
if Y_max < 0:
Y_max = 0
X_max = (Y_max - b)/W
print([X_min, X_max],[Y_min, Y_max])
#plt.plot([X_min, X_max],[Y_min, Y_max], color = color) #color=[0.5, 0.5, 0.5]
'''
W, b = regression_coef[-1]
plt.plot([X_min, X_max],[X_min * W + b, X_max * W + b], color=[0.5, 0.5, 0.5])
#plt.show()
plt.savefig("Perceptron.png")