个人介绍
1. Experience of building a large Distributed web crawler system (50,000 lines code, including 500+ different websites or applications, 10million data extracted)
2. Extracted data from 30+ dark websites.
3. Experience of popular websites extracting: Kuaishou, JD, Weibo, Taobao, Amazon, Bilibili
4. Data mining and visualization
5. CAD&CG: OCR, Tensorflow, Opencv, Pillow, Captcha cracking
6. Other Skills: Scrapy, puppetter, redis, celery, Mongodb, Appium, Selenium, Django, linux, Semantic Web, Ontology, Vue.js, Javascript, HTML, JAVA, Python
7. Other Skills_2: 接码,打码,代理,各种验证码破解,js破解,逆向,移动应用爬虫等
工作经历
2016-09-30 -至今平安科技安全工程师
平安集团信息安全部-安全工程师 1. 负责威胁情报的数据获取/分析/可视化/数据运营 2. 独自开发测试运营大型分布式爬虫系统,爬取数据源500+, 数据1000万以上 3...
教育经历
2012-09-01 - 2016-06-30浙江大学计算机科学与技术本科
浙江大学-计算机科学与技术专业