python技巧 - 了解multiprocessing及其用法的執行個體

本文用執行個體代碼說明如何了解python的多程序工具multiprocessing，并給出Process、Value、Array、Lock、Queue和Pool方法的應用示例。最後一個Pool用法的執行個體，可以看成是由前面幾個示例演化而成的，從中可以了解multiprocessing的内在機制。

示例1： Process的用法

from multiprocessing import Process,Value, Array, Lock

def add_operation(number):
    for i in range(100):        
        number.value += i 
        
if __name__ == "__main__":

    shared_number = Value('i',0)
    print('從 %s 開始相加...'%shared_number.value)

    p1 = Process(target=add_operation,args=(shared_number,))
    p2 = Process(target=add_operation,args=(shared_number,))

    p1.start()
    p2.start()

    p1.join()
    p2.join()

    print(' 相加到 %s 結束'%shared_number.value)
           

運作結果：

從 0 開始相加...
 相加到 9900 結束           

注意：不同電腦的運作結果不一樣，以下示例2、示例3也如此

示例2：Lock的用法

from multiprocessing import Process,Value, Array, Lock

def add_operation(number,lock):
    for i in range(100):        
    			with lock:    
   			        number.value += i 
					 #lock.acquire()  
					 #number.value += i 
					 #lock.release()
        
if __name__ == "__main__":

    lock = Lock()
    shared_number = Value('i',0)
    print('從 %s 開始相加...'%shared_number.value)

    p1 = Process(target=add_operation,args=(shared_number,lock))
    p2 = Process(target=add_operation,args=(shared_number,lock))

    p1.start()
    p2.start()

    p1.join()
    p2.join()

    print(' 相加到 %s 結束'%shared_number.value)
           

從 0 開始相加...
 相加到 9900 結束           

示例3：Array的用法

from multiprocessing import Process,Value, Array, Lock

def add_operation(numbers,lock):
    for i in range(100):   
         for i in range(len(numbers)):
            # numbers[i] += 1   # 可以用下面的with 語句替代          
            with lock:
                numbers[i] += 1          
        
if __name__ == "__main__":

    lock = Lock()
    shared_array = Array('d',[0.0,100.0,200.0])
    print('從 %s 開始相加...'%shared_array[:])

    p1 = Process(target=add_operation,args=(shared_array,lock))
    p2 = Process(target=add_operation,args=(shared_array,lock))

    p1.start()
    p2.start()

    p1.join()
    p2.join()
    
    print(' 相加到 %s 結束'%shared_array[:])
           

從 [0.0, 100.0, 200.0] 開始相加...
 相加到 [189.0, 289.0, 388.0] 結束           

示例4：queue的用法

from multiprocessing import Process,Value, Array, Lock, Queue

def evalute_square(numbers,queue):
    for i in numbers:
        queue.put(i*i)

def convert_negative(numbers,queue):
    for i in numbers:
        queue.put(-1*i)
        
if __name__ == "__main__":
    numbers = range(1,6)
    q = Queue()

    p1 = Process(target=evalute_square,args=(numbers,q))
    p2 = Process(target=convert_negative,args=(numbers,q))

    p1.start()
    p2.start()

    p1.join()
    p2.join()
    
    while not q.empty():
        print(q.get())

           

1
4
9
16
25
-1
-2
-3
-4
-5           

示例5：Pool的用法

from multiprocessing import Process,Value, Array, Lock, Queue, Pool

def evluate_cube(number):
    return number*number*number
        
if __name__ == "__main__":
    numbers = range(10)
    pool = Pool()

    # map,apply,join,close
    result = pool.map(evluate_cube,numbers)

    print(result)
           

[0, 1, 8, 27, 64, 125, 216, 343, 512, 729]