EvoTorch是一个直接构建在PyTorch之上的高级进化算法库，由NNAISENSE创建。EvoTorch旨在加速进化算法的研究和应用，并专门支持NeuroEvolution。
pipinstallevotorch使用EvoTorch，可以解决各种优化问题，而不必担心手头的这些问题是否可微。EvoTorch可解决的问题类型包括：Black-box优化问题（连续或离散）强化学习任务监督学习任务等等EvoTorch中提供了各种进化计算算法：基于分布的搜索算法：PGPE：基于参数探索的策略梯度。XNES：指数自然进化策略。SNES：可分离的自然进化策略。CEM：交叉熵方法。基于群体的搜索算法：SteadyStateGA：一个完全精英化的遗传算法实现。还支持多个目标，在这种情况下，其行为类似于 NSGA-II。CoSyNE：协同突触神经进化。上面提到的所有这些算法都是在PyTorch中实现的，因此可以从PyTorch的矢量化和GPU功能中受益。此外，在Ray库的帮助下，EvoTorch可以通过将工作负载分散到以下方面来进一步扩展这些算法：多个CPU多个GPURay集群上的多台计算机black-box优化示例fromevotorchimportProblemfromevotorch.algorithmsimportSNESfromevotorch.loggingimportStdOutLogger,PandasLoggerimportmathimportmatplotlib.pyplotaspltimporttorch#Declaretheobjectivefunctiondefrastrigin(x:torch.Tensor)->torch.Tensor:A=10(_,n)=x.shapereturnA*n+torch.sum((x**2)-A*torch.cos(2*math.pi*x),1)#Declaretheproblemproblem=Problem("min",rastrigin,initial_bounds=(-5.12,5.12),solution_length=100,vectorized=True,#device="cuda:0"#enablethislineifyouwishtouseGPU)#InitializetheSNESalgorithmtosolvetheproblemsearcher=SNES(problem,popsize=1000,stdev_init=10.0)#Initializeastandardoutputlogger,andapandaslogger_=StdOutLogger(searcher,interval=10)pandas_logger=PandasLogger(searcher)#RunSNESforthespecifiedamountofgenerationssearcher.run(2000)#GettheprogressoftheevolutionintoaDataFramewiththe#helpofthePandasLogger,andthenplottheprogress.pandas_frame=pandas_logger.to_dataframe()pandas_frame["best_eval"].plot()plt.show()强化学习示例fromevotorch.algorithmsimportPGPEfromevotorch.loggingimportStdOutLoggerfromevotorch.neuroevolutionimportGymNE#Declaretheproblemtosolveproblem=GymNE(env_name="Humanoid-v4",#SolvetheHumanoid-v4tasknetwork="Linear(obs_length,act_length)",#Linearpolicyobservation_normalization=True,#Normalizethepolicyinputsdecrease_rewards_by=5.0,#Decreaseeachrewardby5.0num_actors="max",#UseallavailableCPUs#num_actors=4,#Explicitsetting.Use4actors.)#InstantiateaPGPEalgorithmtosolvetheproblemsearcher=PGPE(problem,#Basepopulationsizepopsize=200,#Foreachgeneration,samplemoresolutionsuntilthe#numberofsimulatorinteractionsreachesthisthresholdnum_interactions=int(200*1000*0.75),#Stopre-samplingsolutionsifthecurrentpopulationsize#reachesorexceedsthisnumber.popsize_max=3200,#Learningratescenter_learning_rate=0.0075,stdev_learning_rate=0.1,#Radiusoftheinitialsearchdistributionradius_init=0.27,#UsetheClipUpoptimizerwiththespecifiedmaximumspeedoptimizer="clipup",optimizer_config={"max_speed":0.15},)#Instantiateastandardoutputlogger_=StdOutLogger(searcher)#Runthealgorithmforthespecifiedamountofgenerationssearcher.run(500)#Getthecenterpointofthesearchdistribution,#obtainapolicyoutofthatpoint,andvisualizethe#agentusingthatpolicy.center_solution=searcher.status["center"]trained_policy=problem.make_net(center_solution)problem.visualize(trained_policy)
更多示例可以在这里找到。
评论