3x3,5x5,7x7,9x9卷积核性能比较
在用7*7的卷积核分类9*9的图片到底应该用几个卷积核?中得到了一个经验结论：卷积核越大，网络的性能上升区间越大；上升区间越大，性能峰值越大。要按此分类9*9的图片，9*9的卷积核就是极限性能最优的卷积核。这次就验证这个观点。
(mnist 0,2)-con(9*9)*n-30*2-(1,0)(0,1)
用9*9的卷积核分类mnist0，2，卷积核数量从2到33个，收敛标准6e-5。统计平均值，比较卷积核数量对分类性能的影响。
得到表格
|
f2[0] |
f2[1] |
迭代次数n |
平均准确率p-ave |
δ |
耗时ms/次 |
耗时ms/199次 |
耗时 min/199 |
最大值p-max |
平均值标准差 |
|
|
2 |
0.8592554 |
0.1407446 |
3583946.2 |
0.9776667 |
6.00E-05 |
2062400.8 |
410417767 |
6840.2961 |
0.9900596 |
0.0076349 |
|
3 |
0.6934459 |
0.3065541 |
38925.899 |
0.9799669 |
6.00E-05 |
30715.769 |
6112448 |
101.87413 |
0.9910537 |
0.0083689 |
|
4 |
0.4673405 |
0.5326595 |
30166.276 |
0.9824595 |
6.00E-05 |
31589.251 |
6286261 |
104.77102 |
0.9910537 |
0.0076396 |
|
5 |
0.4170945 |
0.5829055 |
25983.362 |
0.9837433 |
6.00E-05 |
33737.673 |
6713801 |
111.89668 |
0.9915507 |
0.0059125 |
|
6 |
0.2864561 |
0.7135438 |
23770.226 |
0.9840255 |
6.00E-05 |
37091.392 |
7381199 |
123.01998 |
0.9915507 |
0.0055148 |
|
7 |
0.2613328 |
0.7386673 |
22559.327 |
0.9833237 |
6.00E-05 |
42215.553 |
8400902 |
140.01503 |
0.9910537 |
0.0057736 |
|
8 |
0.2261607 |
0.7738393 |
21344.186 |
0.9832063 |
6.00E-05 |
44424.935 |
8840569 |
147.34282 |
0.9910537 |
0.0057323 |
|
9 |
0.2110868 |
0.7889131 |
20384.533 |
0.9830989 |
6.00E-05 |
47688.075 |
9489940 |
158.16567 |
0.9910537 |
0.0064783 |
|
10 |
0.2060627 |
0.7939372 |
19761.482 |
0.9828541 |
6.00E-05 |
51445.075 |
10237576 |
170.62627 |
0.9900596 |
0.0057135 |
|
11 |
0.1457677 |
0.8542324 |
19322.889 |
0.9820224 |
6.00E-05 |
55209.754 |
10986747 |
183.11245 |
0.9910537 |
0.0064363 |
|
12 |
0.2161118 |
0.7838883 |
19832.055 |
0.98194 |
6.00E-05 |
64321.513 |
12799991 |
213.33318 |
0.9905567 |
0.0063398 |
|
13 |
0.1156199 |
0.8843801 |
18791.593 |
0.9822997 |
6.00E-05 |
63586.332 |
7926090 |
132.1015 |
0.9905567 |
0.0063122 |
|
14 |
0.1658659 |
0.8341341 |
18955.724 |
0.9827193 |
6.00E-05 |
70956.271 |
14120303 |
235.33838 |
0.9900596 |
0.0060133 |
|
15 |
0.185964 |
0.8140359 |
18780.759 |
0.9821348 |
6.00E-05 |
73026.568 |
14532291 |
242.20485 |
0.9905567 |
0.0061449 |
|
16 |
0.1357185 |
0.8642816 |
18840.497 |
0.9814855 |
6.00E-05 |
81444.03 |
16207362 |
270.1227 |
0.9900596 |
0.0084691 |
|
17 |
0.1558167 |
0.8441832 |
18937.834 |
0.9827243 |
6.00E-05 |
106632.71 |
21219920 |
353.66533 |
0.9905567 |
0.0059273 |
|
18 |
0.1960134 |
0.8039867 |
18645.412 |
0.9810459 |
6.00E-05 |
88521.121 |
17615710 |
293.59517 |
0.9905567 |
0.0080427 |
|
19 |
0.1809402 |
0.8190597 |
18927.709 |
0.9818251 |
6.00E-05 |
122419.96 |
24361584 |
406.0264 |
0.9900596 |
0.0061403 |
|
20 |
0.1457675 |
0.8542325 |
18760.226 |
0.9820524 |
6.00E-05 |
122352.5 |
24348153 |
405.80255 |
0.9905567 |
0.0067007 |
|
21 |
0.2060623 |
0.7939377 |
18865.905 |
0.9812956 |
6.00E-05 |
103280.36 |
20552815 |
342.54692 |
0.9905567 |
0.0060445 |
|
22 |
0.1759154 |
0.8240846 |
19115.874 |
0.9818726 |
6.00E-05 |
131887.82 |
26245685 |
437.42808 |
0.9910537 |
0.0077266 |
|
23 |
0.1658656 |
0.8341343 |
18498.497 |
0.9810833 |
6.00E-05 |
120268.38 |
23933414 |
398.89023 |
0.9895626 |
0.0065564 |
|
24 |
0.160841 |
0.8391591 |
18965.899 |
0.9817452 |
6.00E-05 |
139977.06 |
27855444 |
464.2574 |
0.9900596 |
0.0073353 |
|
25 |
0.1156199 |
0.8843801 |
18914.126 |
0.9811932 |
6.00E-05 |
136068.09 |
27077554 |
451.29257 |
0.9905567 |
0.0080618 |
|
26 |
0.1759148 |
0.8240852 |
18712.136 |
0.9807736 |
6.00E-05 |
130852.5 |
26039658 |
433.9943 |
0.9905567 |
0.0084431 |
|
27 |
0.1306936 |
0.8693063 |
18862.322 |
0.9817727 |
6.00E-05 |
152705.07 |
30388311 |
506.47185 |
0.9900596 |
0.0074685 |
|
28 |
0.1608408 |
0.8391593 |
19022.915 |
0.9820224 |
6.00E-05 |
154029.97 |
30651970 |
510.86617 |
0.9905567 |
0.0074131 |
|
29 |
0.1507922 |
0.8492078 |
19243.367 |
0.9808211 |
6.00E-05 |
166377.2 |
33109072 |
551.81787 |
0.9895626 |
0.0101928 |
|
30 |
0.1708907 |
0.8291093 |
19727.829 |
0.9817003 |
6.00E-05 |
165774.29 |
32989084 |
549.81807 |
0.9900596 |
0.0069316 |
|
31 |
0.1407426 |
0.8592574 |
19221.327 |
0.9816378 |
6.00E-05 |
175660.87 |
34956519 |
582.60865 |
0.9900596 |
0.0082442 |
|
32 |
0.1759154 |
0.8240847 |
19524.286 |
0.9810983 |
6.00E-05 |
206296.99 |
41053101 |
684.21835 |
0.9900596 |
0.0075721 |
|
33 |
0.1256688 |
0.8743311 |
19937.794 |
0.9821898 |
6.00E-05 |
178003 |
35422597 |
590.37662 |
0.9900596 |
0.0072098 |
将pave画成图
很明显当卷积核数量为6的时候网络性能达到峰值。这个结论与前面的经验关系完全不符，这个最优值大于3*3卷积核的4个，小于5*5卷积核的16个。
|
2分类 |
3*3 |
5*5 |
7*7 |
9*9 |
|
性能上升区间 |
4 |
16 |
55 |
6 |
|
p-ave |
0.9838731 |
0.987322 |
0.987867 |
0.984025 |
|
耗时min/199次 |
11.6074 |
102.3604 |
821.3148 |
123.02 |
对二分类9*9尺寸的mnist0，2，平均性能峰值最大的卷积核是7*7.
|
平均准确率p-ave |
||||
|
6.00E-05 |
6.00E-05 |
6.00E-05 |
6.00E-05 |
|
|
3*3 |
5*5 |
7*7 |
9*9 |
|
|
0 |
0.981171 |
0.981171 |
0.981171 |
|
|
1 |
0.975916 |
0.978588 |
0.976048 |
|
|
2 |
0.981326 |
0.983376 |
0.981191 |
0.977667 |
|
3 |
0.983633 |
0.985159 |
0.983136 |
0.979967 |
|
4 |
0.983651 |
0.986268 |
0.98426 |
0.98246 |
|
5 |
0.983289 |
0.986143 |
0.984795 |
0.983743 |
|
6 |
0.983506 |
0.986323 |
0.986064 |
0.984025 |
|
7 |
0.982744 |
0.986605 |
0.986086 |
0.983324 |
|
8 |
0.982694 |
0.98689 |
0.986218 |
0.983206 |
|
9 |
0.981885 |
0.98697 |
0.986083 |
0.983099 |
|
10 |
0.980983 |
0.986988 |
0.985991 |
0.982854 |
|
11 |
0.981401 |
0.987225 |
0.986111 |
0.982022 |
|
12 |
0.98214 |
0.986988 |
0.986328 |
0.98194 |
|
13 |
0.987295 |
0.986571 |
0.9823 |
|
|
14 |
0.987132 |
0.986858 |
0.982719 |
|
|
15 |
0.987065 |
0.98695 |
0.982135 |
|
|
16 |
0.987322 |
0.986715 |
0.981485 |
|
|
17 |
0.987227 |
0.987055 |
0.982724 |
|
|
18 |
0.98672 |
0.986471 |
0.981046 |
|
|
19 |
0.987137 |
0.986778 |
0.981825 |
|
|
20 |
0.986988 |
0.986765 |
0.982052 |
|
|
21 |
0.986855 |
0.986953 |
0.981296 |
|
|
22 |
0.986728 |
0.981873 |
||
|
23 |
0.98719 |
0.981083 |
||
|
24 |
0.987053 |
0.981745 |
||
|
25 |
0.98706 |
0.981193 |
||
|
26 |
0.98685 |
0.980774 |
||
|
27 |
0.98665 |
0.981773 |
||
|
28 |
0.987227 |
0.982022 |
||
|
29 |
0.986795 |
0.980821 |
||
|
30 |
0.986586 |
0.9817 |
||
|
31 |
0.98675 |
0.981638 |
||
|
32 |
0.98691 |
0.981098 |
||
|
33 |
0.986768 |
0.98219 |
||
在21个卷积核以内对比4个尺寸卷积核的性能
9*9卷积核的性能显著的小于5*5和7*7卷积核，与3*3卷积核的性能相当。当卷积核数量大于4个以后9*9卷积核的性能略好于3*3卷积核。在21个卷积核以内比较这个4个尺寸的卷积核，可以得到
5*5>7*7>9*9>3*3
也就是5*5卷积核在21个以内性能最好，但7*7卷积核由于有55个的性能上升区间最终将以8倍耗时取得万分之5的性能优势。这4个卷积核极限性能最好的是7*7，但效费比最高的应该是5*5.
由这4个实验也可以得出一个经验规律，对2n+1*2n+1或者2n*2n尺寸的图片,卷积核最优尺寸为2n-1*2n-1，在小于等于2n-1的范围内，卷积核越大，网络的性能上升区间越大：上升区间越大，性能峰值越大。
关于卷积核的实验整理
用7*7的卷积核分类9*9的图片到底应该用几个卷积核?55个
(mnist 0,2)-con(7*7)*n-30*2-(1,0)(0,1)
(mnist 0,2)-con(5*5)*n-30*2-(1,0)(0,1)
(mnist 0,2)-con(3*3)*n-30*2-(1,0)(0,1)
/**/
估算神经网络卷积核数量的近似方法
3*3卷积核5分类