機器學習實戰之 -- NaiveBayes

• NaiveBayes
• • 一、工作原理
  • • 1.條件機率：
    • 2.樸素貝葉斯成立的條件機率假設：
    • 3.後驗機率：
    • 3.樸素貝葉斯分類器：
  • 二、核心代碼

NaiveBayes

一、工作原理

1.條件機率：

P ( X = x ∣ Y = c k ) = P ( X ( 1 ) = x ( 1 ) , . . . , X ( n ) = x ( n ) ∣ Y = c k ) ,   k = 1 , 2 , 3 , . . . , K P(X=x|Y=c_k) = P(X^{(1)}=x^{(1)},...,X^{(n)}=x^{(n)}| Y=c_k) , \:k = 1,2,3,...,K P(X=x∣Y=ck​)=P(X(1)=x(1),...,X(n)=x(n)∣Y=ck​),k=1,2,3,...,K

2.樸素貝葉斯成立的條件機率假設：

P ( X = x   ∣ Y = c k ) = P ( X ( 1 ) = x ( 1 ) , . . . , X ( n ) = x ( n )   ∣ Y = c k )                          = ∏ j = 1 n P ( X ( j ) = x ( j )   ∣ Y = c k ) P(X=x\:|Y=c_k) = P(X^{(1)}=x^{(1)},...,X^{(n)}=x^{(n)}\:| Y=c_k)\\ \:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:=\prod_{j=1}^nP(X^{(j)}=x^{(j)}\:|Y=c_k) P(X=x∣Y=ck​)=P(X(1)=x(1),...,X(n)=x(n)∣Y=ck​)=j=1∏n​P(X(j)=x(j)∣Y=ck​)

3.後驗機率：

P ( Y = c k   ∣ X = x ) = P ( X , Y ) P ( X )                                                             = P ( X = x   ∣ Y = c k ) P ( Y = c k ) ∑ k P ( X = x   ∣ Y = c k ) P ( Y = c k )    ,      k = 1 , 2 , 3 , . . . , K P(Y=c_k\:|X=x) =\frac{P(X,Y)}{P(X)}\\ \:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:\:= \frac{P(X=x\:|Y=c_k)P(Y=c_k)}{\sum_kP(X=x\:|Y=c_k)P(Y=c_k)} \:\:, \:\:\:\: k = 1,2,3,...,K P(Y=ck​∣X=x)=P(X)P(X,Y)​=∑k​P(X=x∣Y=ck​)P(Y=ck​)P(X=x∣Y=ck​)P(Y=ck​)​,k=1,2,3,...,K

3.樸素貝葉斯分類器：

y = f ( x ) = a r g max ⁡ c k P ( Y = c k ) ∏ j = 1 n P ( X ( j ) = x ( j )   ∣ Y = c k ) ∑ k P ( X = x   ∣ Y = c k ) P ( Y = c k )    ,      k = 1 , 2 , 3 , . . . , K y = f(x)=arg\max_{c_k}\frac{P(Y=c_k)\prod_{j=1}^nP(X^{(j)}=x^{(j)}\:|Y=c_k)}{\sum_kP(X=x\:|Y=c_k)P(Y=c_k)} \:\:, \:\:\:\: k = 1,2,3,...,K y=f(x)=argck​max​∑k​P(X=x∣Y=ck​)P(Y=ck​)P(Y=ck​)∏j=1n​P(X(j)=x(j)∣Y=ck​)​,k=1,2,3,...,K

注意到，上式中分母對所有的 c k c_k ck​都是相同的，是以，

y = f ( x ) = a r g max ⁡ c k P ( Y = c k ) ∏ j = 1 n P ( X ( j ) = x ( j )   ∣ Y = c k )    ,   k = 1 , 2 , 3 , . . . , K y = f(x)=arg\max_{c_k}{P(Y=c_k)\prod_{j=1}^nP(X^{(j)}=x^{(j)}\:|Y=c_k)} \:\:,\:k = 1,2,3,...,K y=f(x)=argck​max​P(Y=ck​)j=1∏n​P(X(j)=x(j)∣Y=ck​),k=1,2,3,...,K

二、核心代碼

from numpy import *

def loadDataSet():
    '''
    Desc：建立實驗樣本
    '''
    postingList=[['my', 'dog', 'has', 'flea', 'problems', 'help', 'please'],
                 ['maybe', 'not', 'take', 'him', 'to', 'dog', 'park', 'stupid'],
                 ['my', 'dalmation', 'is', 'so', 'cute', 'I', 'love', 'him'],
                 ['stop', 'posting', 'stupid', 'worthless', 'garbage'],
                 ['mr', 'licks', 'ate', 'my', 'steak', 'how', 'to', 'stop', 'him'],
                 ['quit', 'buying', 'worthless', 'dog', 'food', 'stupid']]
    classVec = [0,1,0,1,0,1]    #1 侮辱性, 0 正常
    return postingList,classVec

def createVocabList(dataSet):
    '''
    Desc：處理dataset傳回不重複詞表
    '''
    #建立一個空集合
    vocabSet = set([])  
    for document in dataSet:
        # 用于求兩個集合的并集
        vocabSet = vocabSet | set(document)
    return list(vocabSet)
# 
def setOfWords2Vec(vocabList, inputSet):
    '''
    Desc：詞集模型，構造詞向量
    vocabList：詞庫
    inputSet：待轉換詞
    '''
    # 建立一個與詞彙表等長的向量，所有元素都為0
    returnVec = [0]*len(vocabList)
    # 周遊輸入文檔中的每個詞
    for word in inputSet:
        # 判斷詞是否在詞彙表中
        if word in vocabList:
            # 将詞在詞彙表中出現的位置，對應的标記在等長0向量中
            returnVec[vocabList.index(word)] = 1
        else: print("the word: %s is not in my Vocabulary!" % word)
    return returnVec
           

# 
def trainNB0(trainMatrix,trainCategory):
    '''
    Desc：樸素貝葉斯分類器訓練函數
    params:
            trainMatrix:文檔矩陣
            trainCategory:标簽向量
    return:
            
        
    '''
    # 文檔行數（向量個數）
    numTrainDocs = len(trainMatrix)
    # 詞的個數
    numWords = len(trainMatrix[0])
    # 計算侮辱性文字的先驗機率：P(y=1)
    pAbusive = sum(trainCategory)/float(numTrainDocs)
    # 構造兩個與詞彙表相同長度的向量，元素均為1，避免算多個機率的乘積為0。
    p0Num = ones(numWords); p1Num = ones(numWords)      #change to ones() 
    p0Denom = 2.0; p1Denom = 2.0                        #change to 2.0
    # 周遊所有的向量
    for i in range(numTrainDocs):
        # 如果是侮辱性文字
        if trainCategory[i] == 1:
            # 統計侮辱性詞出現的頻率
            p1Num += trainMatrix[i]
            print(trainMatrix[i],'trainMatrix[i]====')
            print(p1Denom,'before====p1Denom')
            # 統計行中所有詞數
            p1Denom += sum(trainMatrix[i])
            print(p1Denom,'p1Denom====')
        else:
            p0Num += trainMatrix[i]
            p0Denom += sum(trainMatrix[i])
    # 太多很小的數相乘，避免下溢出或者浮點數舍入導緻的錯誤
    p1Vect = log(p1Num/p1Denom)          #change to log()
    p0Vect = log(p0Num/p0Denom)          #change to log()
    return p0Vect,p1Vect,pAbusive
           

listPosts,listClasses = loadDataSet()
myVocaList = createVocabList(listPosts)
trainMat = []
for postinDoc in listPosts:
    trainMat.append(setOfWords2Vec(myVocaList,postinDoc))
p0v,p1v,pAb = trainNB0(trainMat, listClasses)
p0v,p1v,pAb
           

def classifyNB(vec2Classify, p0Vec, p1Vec, pClass1):
    '''
    Desc：計算後驗機率分類
    vec2Classify：詞向量
    p0Vec：類别為0的條件機率
    p1Vec：類别為1的條件機率
    pClass1：類别為1的先驗機率
    '''
    # 已知p1求後驗機率
    p1 = sum(vec2Classify * p1Vec) + log(pClass1)    #element-wise mult
    # 已知p0求後驗機率
    p0 = sum(vec2Classify * p0Vec) + log(1.0 - pClass1)
    # 分類
    if p1 > p0:
        return 1
    else: 
        return 0
        
def testingNB():
    '''
    Desc：測試算法
    '''
    listOPosts,listClasses = loadDataSet()
    myVocabList = createVocabList(listOPosts)
    trainMat=[]
    for postinDoc in listOPosts:
        trainMat.append(setOfWords2Vec(myVocabList, postinDoc))
    p0V,p1V,pAb = trainNB0(array(trainMat),array(listClasses))
    testEntry = ['love', 'my', 'dalmation']
    thisDoc = array(setOfWords2Vec(myVocabList, testEntry))
    print(testEntry,'classified as: ',classifyNB(thisDoc,p0V,p1V,pAb))
    testEntry = ['stupid', 'garbage']
    thisDoc = array(setOfWords2Vec(myVocabList, testEntry))
    print(testEntry,'classified as: ',classifyNB(thisDoc,p0V,p1V,pAb))
           

testingNB()
           

[‘love’, ‘my’, ‘dalmation’] classified as: 0

[‘stupid’, ‘garbage’] classified as: 1

持續更新中…