Data Engineer - Data Warehouse

5-7 Years

Save

Early Applicant

Job Description

Job Description

Define data model convention and governance
Design, develop and maintain data pipelines (external data source ingestion jobs, ETL/ELT jobs, etc)
Design, develop and maintain data pipeline framework (combined open source and internal software to build and govern data pipelines)
Create and manage data pipelines infrastructures
Continuously seek ways to optimize existing data processing to be cost and time efficient
Ensure good data governance and quality through build monitoring systems to monitor data quality in data warehouse.

Job Requirements

Minimum 5 years experience in Data Engineering, Warehousing
Fluent in Python and advanced-SQL
Preferably familiar with data warehouse environments (eg: Google BigQuery, AWS Redshift, Snowflake)
Preferably familiar with data transformation or processing framework (eg: dbt, Dataform, Spark, Hive, etc)
Preferably familiar with data processing technology (Google Dataflow, Google Dataproc, etc)
Preferably familiar with orchestration tool (eg: Airflow, Argo, Azkaban, etc)
Understand data warehousing concept (eg: Kimball, Inmon, data vault, etc) and experience in data modeling and measure + improve data quality
Preferably understand basic containerization and microservice concept (eg: Docker, Kubernetes)
Having knowledge in machine learning, building robust API, and web development will be an advantage
Able to build and maintain good relationship with stakeholders
Able to translate business requirements to data warehouse modeling specifications
Able to demonstrate creative problem solving skill
A team player who loves to collaborate with others and can work independently when needed