LambdaNet
LambdaNetisaartificialeuraletworklibrarywritteiHaskellthatabstractsetworkcreatio,traiig,aduseashigherorderfuctios.Thebeefitofthisapproachisthatitprovidesaframeworkiwhichusersca:
quicklyiteratethroughetworkdesigsbyusigdifferetfuctioalcompoetsexperimetbywritigsmallfuctioalcompoetstoextedthelibraryThelibrarycomeswithapre-defiedsetoffuctiosthatcabecomposedimaywaystooperateoreal-worlddata.Thesewillbeeumeratedlaterithedocumetatio.
CurretReleaseThecodefromthisrepodoes'treflectthecurretreleaseofLambdaNet.TheREADMEforthecurretreleaseoHackagecabefoudhere.
IstallatioThefirststepistofollowtheHMatrixistallatioistructios.Afterthat,LambdaNetcabeistalledthroughCabal:
cabalupdatecabalistallLambdaNetIstalligtheMostRecetBuildAlteratively,youcausetheightly.TheAPImaybedifferetthawhatiscovereditheREADME,buttheexamples/folderwillalwayscotaiaworkigfileusigallthefeaturesofthecurretcommit.
Toistalltheightlybuild,simplyru:
gitcloehttps://github.com/jbarrow/LambdaNet.git&&cdLambdaNetcabalistallUsigLambdaNetUsigLambdaNettorapidlyprototypeetworksusigbuilt-ifuctiosrequiresolyamiimallevelofHaskellkowledge(althoughgettigthedataitotherightformmaybemoredifficult).However,extedigthelibrarymayrequireamorei-depthkowledgeofHaskelladfuctioalprogrammigtechiques.
YoucafidaquickexampleofusigtheetworkiXOR.hs.OceLambdaNetisistalled,dowloadXOR.hs,adtheyoucaruthefileiyourREPLtoseetheresults:
ruhaskellexamples/XOR.hsTherestofthissectiodissectstheXORetworkiordertotalkaboutthedesigofLambdaNet.
TraiigDataBeforeyoucatraioruseaetwork,youmusthavetraiigdata.Thetraiigdataisatupleofvectors,thefirstvaluebeigtheiputtotheetwork,adthesecodvaluebeigtheexpectedoutput.
FortheXORetwork,thedataiseasilyhardcoded:
lettraiData=[(fromList[0.0,0.0],fromList[0.0]),(fromList[0.0,1.0],fromList[1.0]),(fromList[1.0,0.0],fromList[1.0]),(fromList[1.0,1.0],fromList[0.0])]However,forayo-trivialapplicatiothemostdifficultworkwillbegettigthedataithisform.Ufortuately,LambdaNetdoesotcurretlyhavetoolstosupportdatahadlig.
LayerDefiitiosThefirststepicreatigaetworkistodefiealistoflayerdefiitios.Thetypelayerdefiitiotakesaeurotype,acoutofeurosithelayer,adacoectivityfuctio.
Creatigthelayerdefiitiosforathree-layerXORetwork,with2eurositheiputlayer,2hiddeeuros,ad1outputeurocabedoeas:
letl=LayerDefiitiosigmoidNeuro2coectFullyletl'=LayerDefiitiosigmoidNeuro2coectFullyletl''=LayerDefiitiosigmoidNeuro1coectFullyNeuroTypesAeuroissimplydefiedasaactivatiofuctioaditsderivative,adtheLambdaNetlibraryprovidesthreebuilt-ieurotypes:
sigmoidNeuro-AeurowithasigmoidactivatiofuctiotahNeuro-AeurowithahyperbolictagetactivatiofuctiorecluNeuro-AeurowitharectifiedliearactivatiofuctioBypassigoeofthesefuctiositoaLayerDefiitio,youcacreatealayerwitheurosofthattype.
CoectivityAcoectivityfuctioisabitmoreopaque.Curretly,thelibraryolyprovidescoectFully,afuctiowhichcreatesafullycoectedfeed-forwardetwork.
Simply,thecoectivityfuctiotakesitheumberofeurosilayerladtheumberofeurosilayerl+1,adretursabooleamatrixofitegers(0/1)thatrepresetsthecoectivitygraphofthelayers--a0meastwoeurosareotcoectedada1meastheyare.Thestartigweightsaredefiedlater.
CreatigtheNetworkThecreateNetworkfuctiotakesiaradomtrasform,aetropygeerator,adalistoflayerdefiitios,adretursaetwork.
FortheXORetwork,thecreateNetworkfuctiois:
let=createNetworkormals(mkStdGe4)[l,l',l'']Oursourceofetropyistheveryradom:mkStdGe4,whichwillalwaysresultithesamegeerator.
RadomTrasformsTheradomtrasformfuctioisatrasformthatoperatesoastreamofuiformlydistributedradomumbersadretursastreamoffloatigpoitumbers.
Curretly,thetwodefieddistributiosare:
uiforms-Atrivialfuctiothatretursastreamofuiformlydistributedradomumbersormals-Aslightlyless-trivialfuctiothatusestheBox-Mullertrasformtocreateastreamofumbers~N(0,1)Workisbeigdoetoofferastudett-distributio,whichwouldrequiresupportforachi-squareddistributiotrasformatio.
TraiigtheNetworkIordertotraiaetwork,youmustcreateaewtraier:
lett=BackpropTraier(3::Float)quadraticCostquadraticCost'TheBackpropTraiertypetakesialearigrate,acostfuctio,aditsderivative.
Theactualtraiigoftheetwork,thefitfuctiousesthetraier,aetwork,adthetraiigdata,adretursaew,traiedetwork.FortheXORetwork,thisis:
let'=traiUtilErrorLessThatoliedat0.01LambdaNetprovidesthreetraiigmethods:
traiUtiltraiUtilErrorLessThatraiNTimesThetraiUtilfuctiotakesaStopCoditio(checkNetwork/Traier.hs)formoreiformatio,adthelasttwoaresimplywrappersforthefirstoethatprovidespecificpredicates.
Thecalculatederroriswhatisreturedbythecostfuctio.
CostFuctiosCurretly,theolyprovidedcostfuctioisthequadraticerrorcostfuctio,quadraticCostaditsderivative,quadraticCost'.Iamabouttoaddthecross-etropycostfuctio.
SelectioFuctiosSelectiofuctiosbreakupadatasetforeachroudoftraiig.Thecurretlyprovidedselectiofuctiosare:
miibatch-Youmustprovideaadpartiallyapplyittomiibatchtogetavalidselectiofuctio.Thisfuctioupdatestheetworkaftereverypasses.olie-Usigthisfuctiomeasthattheetworkupdatesaftereverytraiigexample.Forsmalldatasets,it'sbettertouseolie,whileforlargerdatasets,thetraiigcaoccurmuchfasterifyouuseareasoablysizedmiibatch.
UsigtheNetworkOcetheetworkistraied,youcauseitwithyourtestdataorproductiodata:
predict(fromList[1,0])'LambdaNetatleastattemptstofollowaScikit-Learstyleamigschemewithfitadpredictfuctios.
StorigadLoadigOceaetworkhasbeetraied,theweightsadbiasescabestorediafile:
saveNetwork"xor.a"'BycalligsaveNetworkwithafilepath,youcasavethestateoftheetwork.
Loadigaetworkrequirespassigialistoflayerdefiitiosfortheorigialetwork,butwillloadalltheweightsadbiasesofthesavedetwork:
''<-loadNetwork"xor.a"[l,l',l'']NotethattheloadNetworkfuctioretursaIO(Network),youca'tsimplycallpredictortraiotheobjectreturedbyloadNetwork.UsigtheapproachiXOR.hsshouldallowyoutoworkwiththereturedobject.
CurretlyUderDevelopmetWhathasbeeoutliedaboveisolythefirststagesofLambdaNet.Iitedtosupportsomeadditioalfeatures,suchas:
UittestigSelf-orgaizigmapsRegularizatiofuctiosAdditioaltraiertypes(RProp,RMSProp)AdditioalcostfuctiosUitTestigIordertodevelopmorecomplexetworkarchitectures,itisimportattoesurethatallofthebasicsareworkig--especiallyastheAPIudergoeschages.Torutheuittests:
gitcloehttps://github.com/jbarrow/LambdaNet.git&&cdLambdaNetcabalistallcdtestruhaskellMai.hsThiswilldowloadthemostrecetversioofLambdaNetadrualltheuittests.
Self-OrgaizigMaps(SOMs,orKohoeMaps)SOMswerechoseastheextarchitecturetodevelopbecausetheymakedifferetassumptiosthaFeedForwardetworks.Thisallowsustoseehowthecurretlibraryhadlesbuildigoutewarchitectures.AlreadythishasforcedachageitheNeuromodeladspurredthedevelopmetofavisualizatiospackage(iordertousefullyuderstadtheoutputsoftheSOMs).
RegularizatioFuctiosadMometumStadardbackproptraiigissubjecttooverfittigadfalligitolocalmiima.Byprovidigsupportforregularizatioadmometum,LambdaNetwillbeabletoprovidemoreextesibleadrobusttraiig.
FutureGoalsThefuturegoalsare:
CovolutioalNetworksDatahadligforNeuralNetworksGeeratigtheDocumetatioImagesAllthedocumetatiofortheetworkwasgeeratedithefollowigmaer.Ithedocsfolder,ru:
ruhaskelldocs.hspythoaalysis.pyNotethatIamcurretlyworkigoremovigthePythoimageaalysisfromthelibrary,adswitchigitwithHaskelladguplot.I'malsoworkigousigthegeeratedimagesietworkdocumetatio.
评论