Data Engineer

Hello,

Role Title: Data Engineer

Experience: 5 - 10 years

Location: Singapore

Contract Duration: 12 months & extendable (Extension is subject to project requirements.) 

Work Mode: Work From Office (WFO) role.

Note: Candidates must have hands-on AWS and Databricks operations experience.

Role Overview

We are looking for a Data Engineer to join our existing data engineering team and accelerate the implementation and operationalization of our enterprise data platform built on Databricks. You will help migrate legacy data pipelines to Databricks, establish operational excellence frameworks, and ensure seamless integration with our healthcare information systems and clinical data workflows.

This is a hands-on, operations-focused role. You will own the reliability, performance, security and cost efficiency of production data pipelines on AWS, while mentoring junior team members and collaborating closely with infrastructure and compliance teams.

Key Projects

•        Healthcare Data Analytics Platform modernization

•        Migration of existing ETL processes from traditional data warehouse systems to a Databricks Delta Lake architecture

•        Real-time streaming analytics for clinical monitoring systems

•        Advanced analytics infrastructure supporting machine learning operations (MLOps)

Responsibilities

Platform Operations

•        Monitor and maintain production data pipelines to ensure 99.9% uptime and optimal performance

•        Implement logging, alerting and monitoring using application monitoring tools

•        Run regular health checks on pipeline performance, job execution times and resource utilization, and proactively resolve bottlenecks

•        Manage incident response for pipeline failures, including root cause analysis, resolution and post-incident reviews

•        Establish and maintain disaster recovery procedures and backup strategies for critical data assets in the Databricks environment

•        Perform regular performance tuning of Spark jobs and Databricks cluster configurations to optimize cost and execution efficiency

•        Coordinate scheduled maintenance windows and system upgrades with minimal business impact

•        Manage user access controls, workspace configurations and security policies

•        Maintain documentation for operational procedures, runbooks and troubleshooting guides

Delivery and Collaboration

•        Lead the migration of legacy data pipelines to Databricks

•        Support integration with existing healthcare information systems and clinical data workflows

•        Work closely with infrastructure teams and compliance officers

•        Mentor junior team members

Required Qualifications

•        Degree in Computer Science or Computer Engineering

•        Minimum 5 years of experience in system operations, compliance and management

•        Hands-on project experience on the AWS platform (primary requirement)

•        Project experience in cloud operations or cloud architecture

•        AWS certification (Associate or Professional level)

•        Hands-on Databricks operations experience on AWS

Core Technical Skills

•        Proficiency in the Databricks platform, including workspace management, cluster configuration and job orchestration

•        Strong expertise in Apache Spark within Databricks (Spark SQL, DataFrames, RDDs)

•        In-depth understanding of data warehouse concepts, data profiling, data verification and advanced analytics techniques

•        Strong knowledge of monitoring, incident management and cloud cost control

Technology Stack

•        Databricks

•        AWS cloud services and architecture

•        IDMC (Informatica Data Management Cloud)

•        Tableau

•        Oracle Database management

•        MLOps practices within Databricks

•        Amazon SageMaker integration with Databricks

•        DataRobot platform integration

•        STATA (advantage)

If this role isn't the right fit for you at the moment, but you know friends or colleagues who might be interested, please feel free to pass this opportunity along!

Important Note:

> Please share your resume in word format with [Confidential Information]

> Important Note: If this requirement is not a match for you, please refer to your friends.

> Interested professionals can reach out to me for Confidential Discussion @ +65- 6950 5535