LeNet5各層偽代碼
[導(dǎo)讀]LeNet5的第一層卷積運(yùn)算，輸入圖像大小為inH * inW，卷積核c1CNum個(gè)，輸出feature map大小為c1H * c1W, 偏置個(gè)數(shù)、feature map個(gè)數(shù)與卷積核個(gè)數(shù)相等。輸入圖
LeNet5的第一層卷積運(yùn)算，輸入圖像大小為inH * inW，卷積核c1CNum個(gè)，輸出feature map大小為c1H * c1W, 偏置個(gè)數(shù)、feature map個(gè)數(shù)與卷積核個(gè)數(shù)相等。輸入圖像inmap：inH * inW，卷積核：c1conv: c1CNum * 5 * 5, 輸出圖像：c1map: c1CNum * c1H * c1W, 偏置c1bias：c1CNum
functioin?ForwardC1: ????for?ith?convolution?kernel: ????????for?hth?row?in?feature?map?i: ????????????for?coth?col?in?feature?map?i?row?h: ????????????????令curc1為c1map?+?i?*?c1H?*?c1W?+?h?*?c1W?+?co ?????????????????令指針curc1指向位置的值為0 ?????????????????for?cr?in?ranges?5: ?????????????????????for?cc?in?ranges?5: ?????????????????????????令curc1指向位置的值自加?(inmap[(h+cr)*inW?+?co?+cc]乘以c1conv[i*5*5?+?cr?*?5?+?cc]) ?????????????????????endfor?cc ?????????????????endfor?cr ????????????????令curc1指向位置的值自加?(c1bias[i]) ????????????????將curc1指向的值輸入到激活函數(shù)，輸出賦給它自己 ????????????endfor?co ????????endfor?h ????endfor?i endfunction?ForwardC1
S2層池化，輸入圖像c1map: c1CNum * c1H * c1W， 輸出圖像：s2map: s2Num * s2H * s2W，其中s2Num = c1CNum，池化權(quán)值: s2pooling: s2Num，偏置：s2bias: s2Num
function?ForwardS2: ????for?ith?s2?feature?map: ????????for?hth?row?in?feature?map?i: ????????????for?coth?col?in?feature?map?i?row?h: ????????????????令curs2為s2map?+?i?*?s2H?*?s2W?+?h?*?s2W?+?co ????????????????令curs2指向的值?為以下四個(gè)值的均值乘以s2pooling[i]?加上s2bias[i]： ????????????????????c1map[i,?h?*?2?*?c1W,?co?*?2],?c1map[i,?h?*?2?*?c1W,?co?*?2+1],?c1map[i,?(h?*?2?+1)?*?c1W,?co?*?2],?c1map[i,?(h?*?2+1)?*?c1W,?co?*?2?+?1] ????????????????將curs2指向的值輸入到激活函數(shù)，輸出賦給它自己 ????????????endfo?co ????????endfor?h ????endfor?i endfunction?ForwardS2
C3層卷積：輸入圖像：s2map:?s2Num * s2H * s2W，輸出圖像：c3map: c3CNum * c3H * c3W，卷積核：c3conv: c3CNum * 5*5，偏置：c3bias: c3CNum:
function?ForwardC3: ????初始化一個(gè)數(shù)組s2MapSum:?s2H?*?s2W，各元素為0 ????for?hth?in?ranges?s2H: ????????for?coth?in?ranges?s2W: ????????????for?ith?in?ranges?s2Num: ???????????????令s2MapSum[h,?co]?自加?s2map[i,?h,?co] ????????????endfor?i ???????endfor?co ????endfor?h ????for?ith?in?ranges?c3CNum: ????????for?hth?in?ranges?c3H: ????????????for?coth?in?ranges?c3W: ????????????????令curc3為c3map?+?i?*?c3H?*?c3W?+?h?*?c3W?+?co ????????????????令curc3指向值為0 ????????????????for?ch?in?ranges?5: ????????????????????for?cc?in?ranges?5: ????????????????????????curc3指向的值自加?（s2MapSum[h+ch,?co?+?cc]?乘以?c3conv[ch,?cc]） ????????????????????endfor?cc ????????????????endfor?ch ????????????????令curc3指向的值自加c3bias[i]然后輸入激活函數(shù)，返回值賦給curc3指向的值 ????????????endfor?co ????????endfor?h ????endfor?i endfunction?ForwardC3
S4池化層：輸入圖像：c3map: c3CNum * c3H * c3W，輸出圖像：s4map: s4Num * s4H * s4W，其中s4Num等于c3CNum，池化權(quán)值s4pooling: s4Num，偏置：s4bias: s4Num:
function?ForwardS4: ????for?ith?in?ranges?s4Num: ????????for?hth?in?ranges?s4H: ????????????for?coth?in?ranges?s4W: ????????????????令s4map[i,h,co]的值為以下四個(gè)值的均值乘以權(quán)值s4pooling[i]加上偏置s4bias[i]: ????????????????c3map[i,?h*2,?co*2],?c3map[i,?h*2,?co*2?+?1],?c3map[i,?h*2+1,?co*2],?c3map[i,?h*2+1,?co*2+1] ????????????????將s4map[i,h,co]的值輸入到激活函數(shù)，將返回值賦給它自己 ????????????endfor?co ????????endfor?h ????endfor?i endfunction?ForwardS4
C5卷積層：輸入圖像：s4map: s4Num * s4H * s4W, s4H = 5, s4W = 5，輸出圖像：c5map: c5CNum * c5H * c5W, ?c5H = 1, c5W = 1, ?卷積核：c5conv: c5CNum * 5 *5，偏置：c5bias: c5CNum：
function?ForwardC5: ????初始化一個(gè)數(shù)組s4MapSum:?s4H?*?s4W ????for?h?in?ranges?s4H: ????????for?co?in?ranges?s4W: ????????????for?i?in?ranges?s4Num: ????????????????s4MapSum[h,co]?自加?s4Num[i,?h,co] ????????????endfor?i ????????endfor?co ????endfor?h ????for?i?in?ranges?c5CNum: ????????for?h?in?ranges?c5H: ????????????for?co?in?ranges?c5W: ????????????????令curc5為c5map?+?i?*?c5H?*?c5W?+?h?*?c5W?+?co ????????????????令curc5指向的值為0 ????????????????for?ch?in?ranges?5: ????????????????????for?cc?in?ranges?5: ????????????????????????curc5指向的值自加（s4MapSum[h?+?ch,?co?+?cc]?乘以?c5conv[i,?ch,?cc]） ????????????????????endfor?cc ????????????????endfor?ch ????????????????curc5指向的值自加c5bias[i]然后輸入到激活函數(shù)，返回值賦值給curc5指向的值 ????????????endfor?co ????????endfor?h ????endfor?i endfunction?ForwardC5
全連接輸出層：輸入圖像：c5map: c5CNum * c5H * c5W，輸出圖像：outmap: outNum, 全連接權(quán)值：outfullconn: c5CNum * outNum，偏置: outbias: outNum:
function?ForwardOut: ????for?i?in?ranges?outNum: ????????令curout為outmap?+?i ????????令curout指向的值為0 ????????for?ch?in?ranges?c5CNum: ?????????????curout指向的值自加（c5map[ch]?乘以outfullconn[ch,?i]） ????????endfor?ch ????????令curout指向的值自加outbias[i]然后輸入到激活函數(shù)，將返回值賦值給curout指向的值 ????endfor?i endfunction?ForwardOut
反向計(jì)算
輸出反饋：
function?BackwardOut: ????for?i?in?ranges?outNum: 輸出的第i個(gè)值的誤差outdelt[i]等于第i個(gè)輸出outmap[i]減去期望的第i個(gè)值label[i]，然后乘以輸出的激活函數(shù)的導(dǎo)數(shù) 偏置的誤差outbiasdelta[i]等于輸出誤差outdelt[i] endfor?i for?h?in?ranges?c5CNum: ????????for?co?in?ranges?outNum: ????????????fulldelta[h,?co]?等于C5層第h個(gè)神經(jīng)元的值乘以outdelt[co] endfor?co endfor?h endfunction?BackwardOut
C5層反饋：
function?BackwardC5: ????for?h?in?ranges?c5CNum: 令臨時(shí)變量curerr為0 ????for?co?in?ranges?outNum: ????每個(gè)輸出神經(jīng)元誤差outdelt[co]乘以輸出全連接權(quán)值fullconn[h,co]，curerr自加此乘積 endfor?co 神經(jīng)元誤差c5delta[h]為c5map[h]神經(jīng)元激活函數(shù)導(dǎo)數(shù)乘以curerr 偏置誤差c5biasdelta[h]等于神經(jīng)元誤差c5delta[h] for?ch?in?ranges?s4H: for?co?in?ranges?s4W: 第h個(gè)卷積核的第ch,co個(gè)算子誤差為s4MapSum[ch,?co]乘以C5層第h個(gè)神經(jīng)元誤差 endfor?co endfor?ch endfor?h endfunction?BackwardC5
S4層反饋：
function?BackwardS4: ????初始化數(shù)組c5convSum?：5*5 ????for?i?in?ranges?c5CNum: for?h?in?ranges?c5H: for?co?in?ranges?c5W: c5convSum[h,co]自加第i個(gè)卷積核對(duì)應(yīng)位置c5conv[i,?h,?co]乘以C5第h個(gè)神經(jīng)元誤差 endfor?co endfor?h endfor?i for?i?in?ranges?s4Num: ????for?h?in?ranges?s4H: for?co?in?ranges?s4W: S4層神經(jīng)元誤差s4delta[i,h,co]等于其激活函數(shù)的導(dǎo)數(shù)乘以c5convSum[h,co] endfor?co endfor?h 偏置誤差為對(duì)應(yīng)map的神經(jīng)元誤差均值 池化權(quán)值誤差等于（S4層神經(jīng)元誤差乘以C3層對(duì)應(yīng)位置神經(jīng)元均值）再求均值 endfor?i endfunction?BackwardS4
C3層反饋：
function?BackwardC3: 初始化數(shù)組s2MapSum：s2H?*?s2W for?i?in?s2Num: for?h?in?s2H: for?co?in?s2W: s2MapSum[h,co]自加s2map[i,h,co] endfor?co endfor?h endfor?i for?i?in?ranges?C3CNum: for?h?in?ranges?c3H: for?co?in?ranges?c3W: C3神經(jīng)元誤差c3delta[i,h,co]等于其激活函數(shù)導(dǎo)數(shù)乘以對(duì)應(yīng)池化權(quán)值再乘以對(duì)應(yīng)位置的S4神經(jīng)元誤差 endfor?co endfor?h 偏置誤差為對(duì)應(yīng)map的神經(jīng)元誤差均值 for?ch?in?ranges?5: for?cc?in?ranges?5: 令c3convdelta[i,?ch,?cc]為0 for?sh?in?c3H: for?sc?in?c3W: c3convdelta[i,?ch,?cc]自加s2Map[ch?+?sh,?cc?+?sc]乘以c3delta[i,sh,?sc] endfor?sc endfor?sh endfor?cc endfor?ch endfor?i endfunction?BackwardC3
S2層反饋：
function?BackwardS2: ????初始化數(shù)組c3convSum:?s2H?*?s2W for?i?in?ranges?c3CNum: for?h?in?ranges?c3H: for?co?in?ranges?c3W: for?ch?in?ranges?5: for?cc?in?ranges?5: c3convSum[h+ch,?co+cc]自加c3conv[i,ch,?cc]乘以C3第i個(gè)map中[h,co]處神經(jīng)元的誤差 endfor?cc endfor?ch endfor?co endfor?h endfor?i for?i?in?ranges?s2Num: for?h?in?ranges?s2H: for?co?in?ranges?s2W: 對(duì)應(yīng)i個(gè)map的[h,co]處神經(jīng)元激活函數(shù)導(dǎo)數(shù)乘以c3convSum[h,co]得到對(duì)應(yīng)位置S2的神經(jīng)元誤差 endfor?co endfor?h 令偏置誤差為對(duì)應(yīng)map的神經(jīng)元誤差均值 池化權(quán)值誤差為對(duì)應(yīng)map?i中， for?h?in?ranges?s2H: for?co?in?ranges?s2W: 池化權(quán)值誤差s2poolingdelta[i]自加S2神經(jīng)元誤差s2delta[i,h,co]乘以以下四個(gè)元素的均值： c1map[h*2,?co*2],?c1map[h*2,?co*2+1],?c1map[h*2+1,?co*2],?c1map[h*2+1,?co*2+1] endfor?co endfor?h endfor?i endfunction?BackwardS2
C1層反饋：
function?BackwardC1: for?i?in?ranges?c1CNum: for?h?in?ranges?c1H: for?co?in?ranges?c1W: C1神經(jīng)元誤差為其激活函數(shù)導(dǎo)數(shù)乘以對(duì)應(yīng)池化權(quán)值乘以對(duì)應(yīng)S2神經(jīng)元誤差 endfor?co endfor?h 偏置為對(duì)應(yīng)map神經(jīng)元誤差的均值 for?ch?in?ranges?5: for?cc?in?ranges?5: for?h?in?ranges?c1H: for?co?in?ranges?c1W: ????對(duì)應(yīng)卷積核的誤差c1convdelta[i,?ch,?cc]自加inmap[h+ch,?co+cc]乘以c1delta[i,?h,?co] endfor?co endfor?h endfor?cc endfor?ch endfor?i endfunction?BackwardC1
這里有幾步是我沒(méi)有理解LeNet5架構(gòu)導(dǎo)致的錯(cuò)誤，在研究了博客后，通過(guò)調(diào)試代碼，才逐步了解。
下載文檔
