v3.0.1forC/C++andMATLAB/Octave
MIToolboxcontainsasetoffunctionstocalculateinformationtheoreticquantitiesfromdata,suchastheentropyandmutualinformation.ThetoolboxcontainsimplementationsofthemostpopularShannonentropies,andalsothelesserknownRenyientropy.Thetoolboxalsoprovidesimplementationsoftheweightedentropyandweightedmutualinformationfrom"InformationTheorywithApplication",S.Guiasu(1977).Thetoolboxonlysupportsdiscretedistributions,asopposedtocontinuous.Allreal-valuednumberswillbeprocessedbyx=floor(x).
Thesefunctionsaretargetedforusewithfeatureselectionalgorithmsratherthancommunicationchannelsandsoexpectallthedatatobeavailablebeforeexecutionandsampletheirownprobabilitydistributionsfromthedata.
Allfunctionsexpecttheinputstobevectorsormatricesofdoubles.
Functionscontained:
EntropyConditionalEntropyMutualInformationConditionalMutualInformationgeneratingajointvariablegeneratingaprobabilitydistributionfromadiscreterandomvariableRenyi'sEntropyRenyi'sMutualInformationWeightedEntropyWeightedMutualInformationWeightedConditionalMutualInformationNote:allfunctionsarecalculatedinlogbase2,soreturnunitsof"bits".
MIToolboxworksondiscreteinputs,andallcontinuousvaluesmustbediscretisedbeforeusewithMIToolbox.Real-valuedinputswillbediscretisedwithx=floor(x)toensurecompatibility.MIToolboxproducesunreliableresultswhenusedwithcontinuousinputs,runsslowlyandusesmuchmorememorythanusual.Thediscreteinputsshouldhavesmallcardinality,MIToolboxwilltreatvalues{1,10,100}thesamewayittreats{1,2,3}andthelatterwillbebothfasteranduselessmemory.Thislimitationisduetothedifficultiesinestimatinginformationtheoreticfunctionsofcontinuousvariables.
======
Examples:
>>y=[11100]';>>x=[10110]';>>mi(x,y)%%mutualinformationI(X;Y)ans=0.0200>>h(x)%%entropyH(X)ans=0.9710>>condh(x,y)%%conditionalentropyH(X|Y)ans=0.9510>>h([x,y])%%jointentropyH(X,Y)ans=1.9219>>joint([x,y])%%jointrandomvariableXYans=12134======
Allcodeislicensedunderthe3-clauseBSDlicense.
Compilationinstructions:
MATLAB/OCTAVErunCompileMIToolbox.mfromthematlabdirectory.LinuxCsharedlibraryrunmakex86ormakex64for32-bitor64-bitversionsrespectively.runsudomakeinstalltoinstallMIToolboxinto/usr/local/liband/usr/local/includeWindowsCdllinstallMinGWfromhttps://sourceforge.net/projects/mingw-w64/addMinGWbinariesfolderstoPATH,e.g.mingw/bin,mingw/msys/binrunmakex64_wintocompilea64-bitWindowsdll.UpdateHistory
08/02/2017-v3.0.1-BugfixtoensureANSICcompatibility.07/01/2017-v3.0.0-Refactoredinternalstoexposeintegerinformationtheoreticcalculations.10/01/2016-v2.1.2-RelicensefromLGPLtoBSD.AddedcheckstoensureinputMATLABtypesaredoubles.02/02/2015-v2.1.1-FixeduptheMakefilesoitinstallstheheaderstoo.22/02/2014-v2.1-Fixedacoupleofbugsrelatedtomemoryhandling.AddedamakeinstallforcompatibilitywithPyFeast.30/07/2011-v2.00-Addedimplementationsoftheweightedentropyandweightedmutualinformation.MorecleanupofMexentrypointtofurtherchecktheinputs.08/11/2011-v1.03-MinordocumentationchangestoaccompanytheJMLRpublication.15/10/2010-v1.02-FixedbugwhereMIToolboxwouldcauseasegmentationfaultifaxby0emptymatrixwaspassedin.Nowprintsanerrormessageandreturnsgracefully.02/09/2010-v1.01-FixedabuginCMIM.mwherethelastfeaturewouldnotbeselectedfirstifithadthehighestMI.07/07/2010-v1.00-InitialRelease.
评论