Thisrepositorycontainscodecorrespondingtotheseminarpaper:
D.Stutz.IntroductiontoNeuralNetworks.SeminarReport,HumanLanguageTechnologyandPatternRecognitionGroup,RWTHAachenUniversity,2014.
Advisor:PavelGolik
Update:Thecodecanbeadaptedtoallowmini-batchtrainingasdoneinthisfork.
MNISTDatasetTheMNISTdatasetprovidesatrainingsetof60,000handwrittendigitsandavalidationsetof10,000handwrittendigits.Theimageshavesize28x28pixels.Therefore,whenusingatwo-layerperceptron,weneed28x28=784inputunitsand10outputunits(representingthe10differentdigits).
ThemethodsloadMNISTImagesandloadMNISTLaelsareusedtoloadtheMNISTdatasetasitisstoredinaspecialfileformat.Themethodscanbefoundonlineathttps://ufldl.stanford.edu/wiki/index.php/Using_the_MNIST_Dataset.
MethodsandUsageThemainmethodtotrainthetwo-layerperceptronistrainStochasticSquaredErrorTwoLayerPerceptron.Themethodappliesstochastictraining(ortobepreciseastochasticvariantofmini-batchtraining)usingthesum-of-squarederrorfunctionandtheerrorbackpropagationalgorithm.
function[hiddenWeights,outputWeights,error]=trainStochasticSquaredErrorTwoLayerPerceptron(activationFunction,dActivationFunction,numberOfHiddenUnits,inputValues,targetValues,epochs,batchSize,learningRate)%trainStochasticSquaredErrorTwoLayerPerceptronCreatesatwo-layerperceptron%andtrainsitontheMNISTdataset.%%INPUT:%activationFunction:Activationfunctionusedinbothlayers.%dActivationFunction:Derivativeoftheactivation%functionusedinbothlayers.%numberOfHiddenUnits:Numberofhiddenunits.%inputValues:Inputvaluesfortraining(784x60000)%targetValues:Targetvaluesfortraining(1x60000)%epochs:Numberofepochstotrain.%batchSize:PloterrorafterbatchSizeimages.%learningRate:Learningratetoapply.%%OUTPUT:%hiddenWeights:Weightsofthehiddenlayer.%outputWeights:Weightsoftheoutputlayer.%Theabovemethodrequirestheactivationfunctionusedforboththehiddenandtheoutputlayertobegivenasparameter.Iusedthelogisticsigmoidactivationfunction:
functiony=logisticSigmoid(x)%simpleLogisticSigmoidLogisticsigmoidactivationfunction%%INPUT:%x:Inputvector.%%OUTPUT:%y:Outputvectorwherethelogisticsigmoidwasappliedelementby%element.%Inaddition,theerrorbackpropagationalgorithmneedsthederivativeoftheusedactivationfunction:
functiony=dLogisticSigmoid(x)%dLogisticSigmoidDerivativeofthelogisticsigmoid.%%INPUT:%x:Inputvector.%%OUTPUT:%y:Outputvectorwherethederivativeofthelogisticsigmoidwas%appliedelementbyelement.%ThemethodapplyStochasticSquaredErrorTwoLayerPerceptronMNISTusesboththetrainingmethodseenaboveandthemethodvalidateTwoLayerPerceptrontoevaluatetheperformanceofthetwo-layerperceptron:
function[correctlyClassified,classificationErrors]=validateTwoLayerPerceptron(activationFunction,hiddenWeights,outputWeights,inputValues,labels)%validateTwoLayerPerceptronValidatethetwolayerperceptronusingthe%validationset.%%INPUT:%activationFunction:Activationfunctionusedinbothlayers.%hiddenWeights:Weightsofthehiddenlayer.%outputWeights:Weightsoftheoutputlayer.%inputValues:Inputvaluesfortraining(784x10000).%labels:Labelsforvalidation(1x10000).%%OUTPUT:%correctlyClassified:Numberofcorrectlyclassifiedvalues.%classificationErrors:Numberofclassificationerrors.%LicenseLicenseforsourcecodecorrespondingto:
D.Stutz.IntroductiontoNeuralNetworks.SeminarReport,HumanLanguageTechnologyandPatternRecognitionGroup,RWTHAachenUniversity,2014.
Copyright(c)2014-2018DavidStutz
Pleasereadcarefullythefollowingtermsandconditionsandanyaccompanyingdocumentationbeforeyoudownloadand/orusethissoftwareandassociateddocumentationfiles(the"Software").
Theauthorsherebygrantyouanon-exclusive,non-transferable,freeofchargerighttocopy,modify,merge,publish,distribute,andsublicensetheSoftwareforthesolepurposeofperformingnon-commercialscientificresearch,non-commercialeducation,ornon-commercialartisticprojects.
Anyotheruse,inparticularanyuseforcommercialpurposes,isprohibited.Thisincludes,withoutlimitation,incorporationinacommercialproduct,useinacommercialservice,orproductionofotherartefactsforcommercialpurposes.
THESOFTWAREISPROVIDED"ASIS",WITHOUTWARRANTYOFANYKIND,EXPRESSORIMPLIED,INCLUDINGBUTNOTLIMITEDTOTHEWARRANTIESOFMERCHANTABILITY,FITNESSFORAPARTICULARPURPOSEANDNONINFRINGEMENT.INNOEVENTSHALLTHEAUTHORSORCOPYRIGHTHOLDERSBELIABLEFORANYCLAIM,DAMAGESOROTHERLIABILITY,WHETHERINANACTIONOFCONTRACT,TORTOROTHERWISE,ARISINGFROM,OUTOFORINCONNECTIONWITHTHESOFTWAREORTHEUSEOROTHERDEALINGSINTHESOFTWARE.
Youunderstandandagreethattheauthorsareundernoobligationtoprovideeithermaintenanceservices,updateservices,noticesoflatentdefects,orcorrectionsofdefectswithregardtotheSoftware.Theauthorsneverthelessreservetherighttoupdate,modify,ordiscontinuetheSoftwareatanytime.
TheabovecopyrightnoticeandthispermissionnoticeshallbeincludedinallcopiesorsubstantialportionsoftheSoftware.Youagreetocitethecorrespondingpapers(seeabove)indocumentsandpapersthatreportonresearchusingtheSoftware.
评论