Data Engineer-Augmentation Consultant

Data Engineer-Augmentation Consultant

This Data Engineer role is designed to augment our existing data engineering team to accelerate the implementation and operationalization of our enterprise data platform built on Databricks. The successful candidate will be responsible for leading the migration of legacy data pipelines to Databricks, establishing operational excellence frameworks,and ensuring seamless integration with existing healthcare information systems and clinical data workflows.

. Monitor and maintain production data pipelines to ensure 99.9% uptime and optimal performance

. Implement comprehensive logging, alerting, andmonitoring systems using Application monitoring tools

. Perform regular health checks performance, job execution times, and resource utilization to identify and resolve bottle necks proactively

. Manage incident response procedures for pipeline failures, including root cause analysis, resolution, and post-incident reviews

. Establish and maintain disaster recovery procedures and backup strategies for critical data assets within the Databricks environment