天天看點

神經網絡的收斂标準有最優值嗎?

神經網絡的收斂标準有最優值嗎?

作一個5分類的三層網絡,分類9*9的圖檔,收斂标準從0.5到6e-8,共47個收斂标準,每個收斂标準收斂199次,共收斂了47*199次。取平均值統計平均性能pave。觀察pave是如何随着收斂标準改變的。

得到的表格

f2[0] f2[1] f2[2] f2[3] f2[4] 疊代次數n 平均準确率p-ave δ 耗時ms/次 耗時ms/199次 耗時 min/199 最大值p-max 平均值标準差
0.4854799 0.1858553 0.272492 0.2602853 0.2563134 1084.7739 0.6110079 0.5 70.015075 13949 0.2324833 0.8237011 0.2530133
0.5326435 0.12101 0.1954744 0.1860563 0.1582058 1643.7136 0.778164 0.4 78.79397 15680 0.2613333 0.8505546 0.0234568
0.6958971 0.0446005 0.1856664 0.1821425 0.1503321 1914.8794 0.8280271 0.3 83.59799 16654 0.2775667 0.8669002 0.0191745
0.7952586 0.0322357 0.1595491 0.1555259 0.1254544 2235.7487 0.8500412 0.2 89.135678 17738 0.2956333 0.8898618 0.0184869
0.3876615 0.0156595 0.0816177 0.0503286 0.5872436 3160.2261 0.8985676 0.1 104.77387 20850 0.3475 0.9208017 0.0088857
0.0611409 0.0047451 0.0363387 0.0046498 0.9119949 6747.4322 0.9403243 0.01 166.92462 33218 0.5536333 0.9453201 0.0018007
0.3821803 4.17E-04 0.0911098 0.0609155 0.4675613 28730.467 0.9551943 0.001 550.80905 109626 1.8271 0.9620549 0.004127
0.3118726 3.98E-04 0.1211546 0.0457933 0.5227548 31145.402 0.9568312 9.00E-04 587.94975 117033 1.95055 0.9624441 0.0036683
0.2867465 3.69E-04 0.1261143 0.0858857 0.5026527 34822.437 0.9582198 8.00E-04 644.41709 128254 2.1375667 0.963417 0.0029477
0.3319181 3.20E-04 0.1361159 0.0406462 0.4925607 39842.181 0.9591331 7.00E-04 738.93467 147048 2.4508 0.96439 0.002471
0.3067792 2.99E-04 0.1159715 0.0656649 0.5126319 45219.985 0.9600444 6.00E-04 814.98995 162229 2.7038167 0.9657521 0.0024014
0.3268212 2.69E-04 0.1209193 0.0756559 0.4774668 53172.774 0.9615122 5.00E-04 951.00503 189281 3.1546833 0.9675034 0.0029077
0.251422 0.0052492 0.1811337 0.0605513 0.5025688 63857.302 0.9637104 4.00E-04 1132.3015 225328 3.7554667 0.9696439 0.0029834
0.2413307 0.0102346 0.2061994 0.0353675 0.5075742 80150.191 0.9662254 3.00E-04 1408.1156 280231 4.6705167 0.9747033 0.00291
0.1759618 1.33E-04 0.3669061 0.1257354 0.3317159 113404.32 0.9705993 2.00E-04 1971.5276 392334 6.5389 0.9766492 0.0027735
0.0301947 6.71E-05 0.5176071 0.1508071 0.3015366 178704.5 0.9755657 1.00E-04 3079.9296 612906 10.2151 0.9807356 0.0020734
0.0352193 6.16E-05 0.5728769 0.1558284 0.2362122 189645.84 0.9762081 9.00E-05 3289.3618 654583 10.909717 0.9801518 0.0017323
0.0301881 5.37E-05 0.5326786 0.1708962 0.2663572 199240.38 0.9767577 8.00E-05 3124.5779 621799 10.363317 0.980541 0.0016933
0.0201314 4.96E-05 0.5176013 0.1909908 0.2713782 213000.2 0.9770862 7.00E-05 3870.0804 770149 12.835817 0.9820977 0.0017903
0.0201276 4.13E-05 0.5377 0.1608375 0.2814242 232538.81 0.9775898 6.00E-05 4183.5427 832531 13.875517 0.9819031 0.0019222
0.0251499 3.54E-05 0.4321747 0.2914793 0.2512723 261837.84 0.9784054 5.00E-05 4671.7035 929680 15.494667 0.9824869 0.0017131
0.020118 0.0050529 0.4673476 0.2864494 0.2211226 293045.84 0.97917 4.00E-05 4496.6231 894833 14.913883 0.982876 0.0017905
0.0150892 2.13E-05 0.5125703 0.206044 0.266342 341946.9 0.9802202 3.00E-05 5834.7286 1161121 19.352017 0.9836544 0.0016258
8.67E-06 0.0050394 0.467342 0.2964908 0.2311632 408835.39 0.9810798 2.00E-05 6995.1508 1392042 23.2007 0.9846274 0.0016328
4.22E-06 7.36E-06 0.2663364 0.4673396 0.2663348 576604.23 0.9824663 1.00E-05 9819.1457 1954028 32.567133 0.9856003 0.0014906
0.005029 6.68E-06 0.3567873 0.3718623 0.266335 590518.09 0.9824595 9.00E-06 10070.03 2003949 33.39915 0.9861841 0.0016341
3.66E-06 6.13E-06 0.361812 0.4221129 0.2160835 609672.87 0.9826296 8.00E-06 10578.678 2105158 35.085967 0.9854057 0.0014217
0.0100532 5.28E-06 0.3517614 0.4321629 0.206033 649084.43 0.9828379 7.00E-06 11378.653 2264389 37.739817 0.9863787 0.0014723
2.51E-06 4.55E-06 0.3919619 0.402012 0.2060328 688141.75 0.9829699 6.00E-06 11112.593 2211411 36.85685 0.9863787 0.0013737
0.0050274 3.74E-06 0.366836 0.3919614 0.2361828 739405.14 0.983187 5.00E-06 13885.628 2763247 46.054117 0.9863787 0.0013701
0.0050268 2.85E-06 0.3165845 0.4221118 0.2562828 814895.43 0.9833523 4.00E-06 12441.407 2475855 41.26425 0.9875462 0.0013463
0.0100515 2.26E-06 0.3115589 0.4221114 0.2562826 898370.47 0.983451 3.00E-06 15068.241 2998588 49.976467 0.9867679 0.0012352
0.0100511 0.0050265 0.3065335 0.4422116 0.2361817 1018724.4 0.9834647 2.00E-06 17163.116 3415468 56.924467 0.9865733 0.0014891
0.0100506 6.75E-07 0.2462316 0.4422113 0.3015079 1244358.3 0.9834852 1.00E-06 20607.568 4100909 68.348483 0.9865733 0.001324
0.0100506 6.44E-07 0.3216084 0.4522616 0.2160808 1291161.7 0.9834999 9.00E-07 21746.186 4327501 72.125017 0.987157 0.0014341
0.0050255 5.71E-07 0.2512566 0.432161 0.311558 1360011.7 0.9835537 8.00E-07 22496.085 4476721 74.612017 0.9867679 0.0014877
0.0050254 4.83E-07 0.2412064 0.432161 0.3216083 1402576.1 0.9834344 7.00E-07 24104.799 4796856 79.9476 0.9865733 0.0014402
0.0100505 4.13E-07 0.3015078 0.4371861 0.2512565 1430741.6 0.9834246 6.00E-07 24127.779 4801430 80.023833 0.987157 0.0014815
0.0050253 0.0100506 0.3366836 0.3718594 0.2763821 1539160.9 0.9834148 5.00E-07 26017.894 5177575 86.292917 0.9867679 0.0013904
0.0100504 2.66E-07 0.2964825 0.4170855 0.2763821 1618461.8 0.983409 4.00E-07 27559.96 5484442 91.407367 0.9867679 0.001473
0.0100504 2.00E-07 0.2864323 0.4020101 0.3015076 1702883.1 0.9833728 3.00E-07 29320.739 5834845 97.247417 0.9867679 0.0013969
0.0150755 1.47E-07 0.2663317 0.4221106 0.2964825 1843820.1 0.9831616 2.00E-07 31024.427 6173862 102.8977 0.9865733 0.0013638
0.0050252 0.0050252 0.2763819 0.3919598 0.3216081 2188509.1 0.9828672 1.00E-07 37078.025 7378528 122.97547 0.98813 0.0015414
0.0150754 0.0100503 0.2914573 0.3517588 0.3316583 2266508.3 0.9827519 9.00E-08 38511.402 7663774 127.72957 0.9875462 0.0015187
0.0201005 5.58E-08 0.2964824 0.3567839 0.3266332 2273906.5 0.9828144 8.00E-08 38618.528 7685090 128.08483 0.9861841 0.0014832
0.0100503 4.94E-08 0.2713568 0.3969849 0.3216081 2310469.9 0.9827157 7.00E-08 39082.382 7777409 129.62348 0.9867679 0.0014673
0.0150754 0.0100503 0.2512563 0.3668342 0.3567839 2424313.6 0.9827822 6.00E-08 42822.03 8521589 142.02648 0.9863787 0.0015439
神經網絡的收斂标準有最優值嗎?

這張圖是δ=5e-5到δ=6e-8的pave曲線,相當直覺當δ=8e-7時網絡達到峰值,峰值是0.983554。

神經網絡的收斂标準有最優值嗎?

這張圖是δ=1e-5到δ=6e-8的pave曲線,表明對三層網絡收斂标準是有最優值的,超過最優值以後網絡性能是下降的。

《估算卷積核數量的近似方法》實驗表明卷積核數量是有最優值的

《平均分辨準确率對網絡隐藏層節點數的非線性變化關系03》實驗表明隐藏層節點數是有最優值的。

因為有最優值的存在,如果将網絡的收斂标準設定為某一疊代次數,則将疊代次數調大并不必然導緻網絡的性能改善。如将收斂标準設為6e-8,平均性能隻有峰值δ=8e-7的0.999216.但疊代次數卻是峰值的1.78倍,耗時是峰值的1.9倍,也就是用了1.9倍的時間卻換來性能下降萬分之8.或者也可以解釋成導緻網絡性能下降的一個原因恰恰是疊代次數過多了。

繼續閱讀