工作经历
1970-01-01 -至今EastridgeSenior Software Engineer
Work on Python Pyramid based microservice layer. Mainly do REST API end-points develop and debugging, unit testing by writing test cases. Also do ETL data pipeline with stacks: Airflow + Spark + MySQL + Snowflake.
教育经历
1970-01-01 - University of SaskatchewanComputer Science硕士
资质认证
技能
This project is just a demo to show how ETL is implemented in Spark. Use ETL stack: Airflow + spark to implement data pipeline. (read from data sources, transform data, write to a destination) Airflow is for workflow management. Spark is for data transformation.
Use unit test in Pytest framework to test REST APIs end-points in tests folder in Django framework.