Data Engineer/Senior Data Engineer
Search Jobs
Search by job, company or skills
Search by job, company or skills
Role: Data Engineer/Senior Data Engineer
Location: Kuala Lumpur(Onsite)
Duration: 12 Months (Renewable) Contract
Databricks.
Implement scalable data processing and transformation logic in PySpark and SQL on Databricks (Delta
Lake / lakehouse patterns). Ingest and integrate data from diverse batch and streaming sources into
ADLS Gen2 and Delta Lake.
Build curated, documented datasets for analytics and ML (support dimensional models where
appropriate).
Implement automated data quality checks, unit/integration tests, and validation gates. Monitor pipeline
health, implement logging, alerting and dashboards; perform RCA and production incident remediation.
Optimize pipeline performance and cost: cluster sizing, job scheduling, partitioning, caching and query
tuning.
Implement and maintain CI/CD for data jobs and infrastructure (Terraform/ARM/Bicep, Azure DevOps or
GitHub Actions). Candidate knows Cantonese .
Interested candidates can connect on +6586533349 (WhatsApp chat only)
Job ID: 151629003
Skills:
Pyspark, Sql, databricks, Data Engineer
Skills:
Hadoop, Pyspark, Spark, Data Warehousing Concepts, UNIX, Python, Sql
Skills:
Java, Spring Boot, Sql, Linux systems, command line operations, Big Data operations
Skills:
Unix, Data Manipulation Language, Tableau, Json, Python, AWS, Java, Power Bi, Csv, Scala, Apache Spark, Automation, Sql, Git, Linux, Xml, Databricks, Azure, Agent Orchestration, Data storage subsystems, MS Fabric, Normalized dimensional data modelling, Event-Driven Workflows, ETL workflows
Skills:
Devops, Pyspark, Databricks, Azure, Python, Sql, AWS, Infrastructure as Code, DataOps, SRE, CI CD
