DeepLearningforCoderswithfastaiandPyTorch:AIApplicationsWithoutaPhD-thebookandthecourse
WelcometoPracticalDeepLearningforCoders.Thiswebsitecoversthebookandthe2020versionofthecourse,whicharedesignedtoworkcloselytogether.Ifyouhaven'tyetgotthebook,youcanbuyithere.It'salsofreelyavailableasinteractiveJupyterNotebooks;readontolearnhowtoaccessthem..
HowdoIgetstarted?Ifyou'rereadytodiveinrightnow,here'showtogetstarted.Ifyouwanttoknowmoreaboutthiscourse,readthenextsections,andthencomebackhere.
Towatchthevideos,clickontheLessonssectioninthenavigationsidebar.Thelessonsallhavesearchabletranscripts;click"TranscriptSearch"inthetoprightpaneltosearchforawordorphrase,andthenclickittojumpstraighttovideoatthetimethatappearsinthetranscript.ThevideosareallcaptionedandalsotranslatedintoChinese(简体中文)andSpanish;whilewatchingthevideoclickthe"CC"buttontoturnthemonandoff,andthesettingbuttontochangethelanguage.
Eachvideocoversachapterfromthebook.TheentiretyofeverychapterofthebookisavailableasaninteractiveJupyterNotebook.JupyterNotebookisthemostpopulartoolfordoingdatascienceinPython,forgoodreason.Itispowerful,flexible,andeasytouse.Wethinkyouwillloveit!Sincethemostimportantthingforlearningdeeplearningiswritingcodeandexperimenting,it'simportantthatyouhaveagreatplatformforexperimentingwithcode.
Togetstarted,werecommendusingaJupyterServerfromoneoftherecommendedonlineplatforms(clickthelinksforinstructionsonhowtousetheseforthecourse):
Colab:ApopularfreeservicefromGoogle.Googlehavechangedthenotebookplatformquitealot,sokeyboardshortcutsaredifferent,andnoteverythingworks(e.g.muchofchapter2doesn'tworkbecauseColabdoesn'tsupportcreatingwebappGUIs)Gradient:UnlikeColab,thisisa"real"JupyterNotebooksoeverythinginthecourseworks.Italsoprovidesspacetosaveyournotebooksandmodels.However,sometimesthefreeserversgetover-loaded,andwhenthathappensit'simpossibletoconnect.IfyouareinterestedintheexperienceofrunningafullLinuxserver,youcanconsiderDataCrunch.io(verynewservicesowedon'tknowhowgooditis,nosetuprequired,extremelygoodvalueandextremelyfastGPUs),orGoogleCloud(extremelypopularservice,veryreliable,butthefastestGPUsarefarmoreexpensive).Westronglysuggestusingoneoftherecommendedonlineplatformsforrunningthenotebooks,andtonotuseyourowncomputer,unlessyou'reveryexperiencedwithLinuxsystemadminstrationandhandlingGPUdrivers,CUDA,andsoforth.
Ifyouneedhelp,there'sawonderfulonlinecommunityreadytohelpyouatforums.fast.ai.Beforeaskingaquestionontheforums,searchcarefullytoseeifyourquestionhasbeenansweredbefore.(Theforumsystemwon'tletyoupostuntilyou'vespentafewminutesonthesitereadingexistingtopics.)OnebitthatmanystudentsfindtrickyisgettingsignedupfortheBingAPIfortheimagedownloadtaskinlesson2;here'sahelpfulforumpostexplaininghowtogettheBingAPIkeyyou'llneedfordownloadingimages.
Isthiscourseforme?Thankyouforlettingusjoinyouonyourdeeplearningjourney,howeverfaralongthatyoumaybe!Previousfast.aicourseshavebeenstudiedbyhundredsofthousandsofstudents,fromallwalksoflife,fromallpartsoftheworld.Manystudentshavetoldusabouthowthey'vebecomemultiplegoldmedalwinnersofinternationalmachinelearningcompetitions,receivedoffersfromtopcompanies,andhavingresearchpaperspublished.Forinstance,IsaacDimitrovskytoldusthathehad"beenplayingaroundwithMLforacoupleofyearswithoutreallygrokkingit...[then]wentthroughthefast.aipart1courselatelastyear,anditclickedforme".HewentontoachievefirstplaceintheprestigiousinternationalRA2-DREAMChallengecompetition!Hedevelopedamultistagedeeplearningmethodforscoringradiographichandandfootjointdamageinrheumatoidarthritis,takingadvantageofthefastailibrary.
Itdoesn'tmatterifyoudon'tcomefromatechnicaloramathematicalbackground(thoughit'sokayifyoudotoo!);wewrotethiscoursetomakedeeplearningaccessibletoasmanypeopleaspossible.Theonlyprerequisiteisthatyouknowhowtocode(ayearofexperienceisenough),preferablyinPython,andthatyouhaveatleastfollowedahighschoolmathcourse.Thefirstthreechaptershavebeenexplicitlywritteninawaythatwillallowexecutives,productmanagers,etc.tounderstandthemostimportantthingsthey'llneedtoknowaboutdeeplearning--ifthat'syou,justskipoverthecodeinthosesections.
Deeplearningisacomputertechniquetoextractandtransformdata–-withusecasesrangingfromhumanspeechrecognitiontoanimalimageryclassification–-byusingmultiplelayersofneuralnetworks.Alotofpeopleassumethatyouneedallkindsofhard-to-findstufftogetgreatresultswithdeeplearning,butasyou'llseeinthiscourse,thosepeoplearewrong.Here'safewthingsyouabsolutelydon'tneedtodoworld-classdeeplearning:
Myth(don'tneed)TruthLotsofmathJusthighschoolmathissufficientLotsofdataWe'veseenrecord-breakingresultswith<50itemsofdataLotsofexpensivecomputersYoucangetwhatyouneedforstateoftheartworkforfreeDeeplearninghaspower,flexibility,andsimplicity.That'swhywebelieveitshouldbeappliedacrossmanydisciplines.Theseincludethesocialandphysicalsciences,thearts,medicine,finance,scientificresearch,andmanymore.Here'salistofsomeofthethousandsoftasksindifferentareasatwhichdeeplearning,ormethodsheavilyusingdeeplearning,isnowthebestintheworld:
Naturallanguageprocessing(NLP)Answeringquestions;speechrecognition;summarizingdocuments;classifyingdocuments;findingnames,dates,etc.indocuments;searchingforarticlesmentioningaconceptComputervisionSatelliteanddroneimageryinterpretation(e.g.,fordisasterresilience);facerecognition;imagecaptioning;readingtrafficsigns;locatingpedestriansandvehiclesinautonomousvehiclesMedicineFindinganomaliesinradiologyimages,includingCT,MRI,andX-rayimages;countingfeaturesinpathologyslides;measuringfeaturesinultrasounds;diagnosingdiabeticretinopathyBiologyFoldingproteins;classifyingproteins;manygenomicstasks,suchastumor-normalsequencingandclassifyingclinicallyactionablegeneticmutations;cellclassification;analyzingprotein/proteininteractionsImagegenerationColorizingimages;increasingimageresolution;removingnoisefromimages;convertingimagestoartinthestyleoffamousartistsRecommendationsystemsWebsearch;productrecommendations;homepagelayoutPlayinggamesChess,Go,mostAtarivideogames,andmanyreal-timestrategygamesRoboticsHandlingobjectsthatarechallengingtolocate(e.g.,transparent,shiny,lackingtexture)orhardtopickupOtherapplicationsFinancialandlogisticalforecasting,texttospeech,andmuchmore...WhoweareWeareSylvainGuggerandJeremyHoward,yourguidesonthisjourney.We'retheco-authorsoffastai,thesoftwarethatyou'llbeusingthroughoutthiscourse.
Jeremyhasbeenusingandteachingmachinelearningforaround30years.Hestartedusingneuralnetworks25yearsago.Duringthistime,hehasledmanycompaniesandprojectsthathavemachinelearningattheircore,includingfoundingthefirstcompanytofocusondeeplearningandmedicine,Enlitic,andtakingontheroleofPresidentandChiefScientistoftheworld'slargestmachinelearningcommunity,Kaggle.Heistheco-founder,alongwithDr.RachelThomas,offast.ai,theorganizationthatbuiltthecoursethiscourseisbasedon.
Sylvainhaswritten10mathtextbooks,coveringtheentireadvancedFrenchmathscurriculum!HeisnowaresearcheratHuggingFace,andwaspreviouslyaresearcheratfast.ai.
Wecarealotaboutteaching.Inthiscourse,westartbyshowinghowtouseacomplete,working,veryusable,state-of-the-artdeeplearningnetworktosolvereal-worldproblems,usingsimple,expressivetools.Andthenwegraduallydigdeeperanddeeperintounderstandinghowthosetoolsaremade,andhowthetoolsthatmakethosetoolsaremade,andsoon…Wealwaysteachingthroughexamples.Weensurethatthereisacontextandapurposethatyoucanunderstandintuitively,ratherthanstartingwithalgebraicsymbolmanipulation.
ThesoftwareyouwillbeusingInthiscourse,you'llbeusingPyTorchandfastai.
We'vecompletedhundredsofmachinelearningprojectsusingdozensofdifferentpackages,andmanydifferentprogramminglanguages.Atfast.ai,wehavewrittencoursesusingmostofthemaindeeplearningandmachinelearningpackagesusedtoday.WespentoverathousandhourstestingPyTorchbeforedecidingthatwewoulduseitforfuturecourses,softwaredevelopment,andresearch.PyTorchisnowtheworld'sfastest-growingdeeplearninglibraryandisalreadyusedformostresearchpapersattopconferences.
PyTorchworksbestasalow-levelfoundationlibrary,providingthebasicoperationsforhigher-levelfunctionality.Thefastailibraryisthemostpopularlibraryforaddingthishigher-levelfunctionalityontopofPyTorch.Inthiscourse,aswegodeeperanddeeperintothefoundationsofdeeplearning,wewillalsogodeeperanddeeperintothelayersoffastai.Thiscoursecoversversion2ofthefastailibrary,whichisafrom-scratchrewriteprovidingmanyuniquefeatures.
WhatyouwilllearnAfterfinishingthiscourseyouwillknow:
Howtotrainmodelsthatachievestate-of-the-artresultsin:Computervision,includingimageclassification(e.g., classifyingpetphotosbybreed),andimagelocalizationanddetection(e.g., findingwheretheanimalsinanimageare)Naturallanguageprocessing(NLP),includingdocumentclassification(e.g., moviereviewsentimentanalysis)andlanguagemodelingTabulardata(e.g., salesprediction)withcategoricaldata,continuousdata,andmixeddata,includingtimeseriesCollaborativefiltering(e.g., movierecommendation)Howtoturnyourmodelsintowebapplications,anddeploythemWhyandhowdeeplearningmodelswork,andhowtousethatknowledgetoimprovetheaccuracy,speed,andreliabilityofyourmodelsThelatestdeeplearningtechniquesthatreallymatterinpracticeHowtoimplementstochasticgradientdescentandacompletetrainingloopfromscratchHowtothinkabouttheethicalimplicationsofyourwork,tohelpensurethatyou'remakingtheworldabetterplaceandthatyourworkisn'tmisusedforharmHerearesomeofthetechniquescovered(don'tworryifnoneofthesewordsmeananythingtoyouyet--you'lllearnthemallsoon):
RandomforestsandgradientboostingAffinefunctionsandnonlinearitiesParametersandactivationsRandominitializationandtransferlearningSGD,Momentum,Adam,andotheroptimizersConvolutionsBatchnormalizationDropoutDataaugmentationWeightdecayImageclassificationandregressionEntityandwordembeddingsRecurrentneuralnetworks(RNNs)SegmentationAndmuchmore
评论