Java線程篇(十一)：Fork/Join-Java并行計算架構

并行計算在處處都有大資料的今天已經不是一個新鮮的詞彙了，現在已經有單機多核甚至多機叢集并行計算，注意，這裡說的是并行，而不是并發。嚴格的将，并行是指系統内有多個任務同時執行，而并發是指系統内有多個任務同時存在，不同的任務按時間分片的方式切換執行，由于切換的時間很短，給人的感覺好像是在同時執行。 

Java在JDK7之後加入了并行計算的架構Fork/Join，可以解決我們系統中大資料計算的性能問題。Fork/Join采用的是分治法，Fork是将一個大任務拆分成若幹個子任務，子任務分别去計算，而Join是擷取到子任務的計算結果，然後合并，這個是遞歸的過程。子任務被配置設定到不同的核上執行時，效率最高。僞代碼如下：

<code class="hljs livecodeserver has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">Result solve(Problem problem) {
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (problem is small)
        directly solve problem
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">else</span> {
        <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">split</span> problem <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">into</span> independent parts
        fork <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">new</span> subtasks <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">to</span> solve <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">each</span> part
        join all subtasks
        compose <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">result</span> <span class="hljs-built_in" style="color: rgb(102, 0, 102); box-sizing: border-box;">from</span> subresults
    }
}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li></ul>      

Fork/Join架構的核心類是ForkJoinPool，它能夠接收一個ForkJoinTask，并得到計算結果。ForkJoinTask有兩個子類，RecursiveTask（有傳回值）和RecursiveAction（無傳回結果），我們自己定義任務時，隻需選擇這兩個類繼承即可。類圖如下： 

下面來看一個執行個體：計算一個超大數組所有元素的和。代碼如下：

<code class="hljs cs has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">import java.util.Arrays;
import java.util.Random;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveTask;

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">/**
 * @author: shuang.gao  Date: 2015/7/14 Time: 8:16
 */</span>
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">class</span> SumTask extends RecursiveTask<Integer> {

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> final <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span> serialVersionUID = -<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6196480027075657316</span>L;

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> final <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> THRESHOLD = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">500000</span>;

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[] array;

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> low;

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> high;

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-title" style="box-sizing: border-box;">SumTask</span>(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[] array, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> low, <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> high) {
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">this</span>.array = array;
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">this</span>.low = low;
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">this</span>.high = high;
    }

    @Override
    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">protected</span> Integer <span class="hljs-title" style="box-sizing: border-box;">compute</span>() {
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> sum = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>;
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">if</span> (high - low <= THRESHOLD) {
            <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 小于門檻值則直接計算</span>
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> (<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> i = low; i < high; i++) {
                sum += array[i];
            }
        } <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">else</span> {
            <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 1. 一個大任務分割成兩個子任務</span>
            <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> mid = (low + high) >>> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>;
            SumTask left = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> SumTask(array, low, mid);
            SumTask right = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> SumTask(array, mid + <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, high);

            <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 2. 分别計算</span>
            left.fork();
            right.fork();

            <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 3. 合并結果</span>
            sum = left.<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">join</span>() + right.<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">join</span>();
        }
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> sum;
    }

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">public</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">void</span> <span class="hljs-title" style="box-sizing: border-box;">main</span>(String[] args) throws ExecutionException, InterruptedException {
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[] array = genArray(<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1000000</span>);

        System.<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">out</span>.println(Arrays.toString(array));

        <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 1. 建立任務</span>
        SumTask sumTask = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> SumTask(array, <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, array.length - <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>);

        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span> begin = System.currentTimeMillis();

        <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 2. 建立線程池</span>
        ForkJoinPool forkJoinPool = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> ForkJoinPool();

        <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 3. 送出任務到線程池</span>
        forkJoinPool.submit(sumTask);

        <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 4. 擷取結果</span>
        Integer result = sumTask.<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">get</span>();

        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span> end = System.currentTimeMillis();

        System.<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">out</span>.println(String.format(<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"結果 %s 耗時 %sms"</span>, result, end - begin));
    }

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">private</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">static</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[] <span class="hljs-title" style="box-sizing: border-box;">genArray</span>(<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> size) {
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[] array = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">long</span>[size];
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">for</span> (<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int</span> i = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>; i < size; i++) {
            array[i] = <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">new</span> Random().nextLong();
        }
        <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> array;
    }
}
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li></ul>      

我們通過調整門檻值（THRESHOLD），可以發現耗時是不一樣的。實際應用中，如果需要分割的任務大小是固定的，可以經過測試，得到最佳門檻值；如果大小不是固定的，就需要設計一個可伸縮的算法，來動态計算出門檻值。如果子任務很多，效率并不一定會高。 

未完待續。。。

參考資料

http://gee.cs.oswego.edu/dl/papers/fj.pdf 

https://docs.oracle.com/javase/tutorial/essential/concurrency/forkjoin.html 

https://www.ibm.com/developerworks/cn/java/j-lo-forkjoin/ 

http://www.ibm.com/developerworks/cn/java/j-jtp11137.html