Search by job, company or skills

Elliott Moss Consulting Pte Ltd

Senior Data Engineer – Databricks

8-16 Years
SGD 9,500 - 11,000 per month
Save
new job description bg glownew job description bg glow
  • Posted 22 hours ago
  • Be among the first 10 applicants
Early Applicant
Quick Apply

Job Description

Job Summary

We are looking for an experienced Senior Data Engineer – Databricks to design, develop, and maintain scalable data pipelines on the Databricks platform. The role requires strong expertise in PySpark, Databricks, and modern data engineering practices, along with experience modernizing legacy data pipelines and building high-performance data solutions in cloud environments.

Key Responsibilities

  • Design, build, and maintain scalable data pipelines using Databricks and PySpark.
  • Develop end-to-end data workflows including data ingestion, transformation, and consumption.
  • Optimize Spark jobs, cluster configurations, and pipeline performance.
  • Manage and orchestrate workflows using Databricks Jobs and notebooks.
  • Refactor legacy ETL pipelines into modern PySpark-based ELT frameworks.
  • Implement data quality checks, monitoring, and error handling mechanisms.
  • Design and maintain Delta Lake tables with proper optimization strategies.
  • Collaborate with data architects, analysts, and infrastructure teams to deliver reliable data solutions.
  • Troubleshoot and resolve production data pipeline issues.

Mandatory Technical Skills

  • Strong Data Engineering fundamentals including ETL/ELT pipeline design.
  • Hands-on experience with PySpark (DataFrames API, Spark SQL, performance tuning).
  • Experience with Databricks platform including workspace, clusters, notebooks, and job orchestration.
  • Knowledge of Delta Lake features such as ACID transactions and schema evolution.
  • Strong Python programming for data processing and automation.
  • Experience with data modelling (Dimensional modeling, Data Vault, or Lakehouse architecture).
  • Strong SQL skills for data transformation and analysis.
  • Experience with cloud platforms (Azure, AWS, or GCP).
  • Experience with Git version control and CI/CD practices.

Professional Experience

  • Minimum 8 years of experience in data engineering or related roles.
  • 2–3 years of hands-on experience with Databricks.
  • Experience building and maintaining production-grade data pipelines at scale.
  • Proven experience modernizing legacy data pipelines.
  • Experience working in Agile development environments.

Certifications (Mandatory)

  • Databricks Certified Data Engineer Associate
  • OR
  • Databricks Certified Data Engineer Professional

More Info

Job Type:
Function:

About Company

Elliott Moss in the past six years has grown and so did our ambition to provide skilled talents to all major MNCs and SMEs around the globe. We now have three operational offices strategically place around the globe to met our clients need.

We primarily deal with SAP (Principal Offices) and other IT companies in consulting, resourcing and training. We work with over 40 SAP Offices worldwide providing them with Trainers & Consultants for high-end Consulting assignments.

Our primary mission and vision is to provide you with the best suitable employees who bring success to your organization and allows us to grow together.


•Vision - To be a valuable contributor to the success of our clients and candidates in the next few years.

•Values - To reflect the qualities of our clients with a culture of passion, energy, enthusiasm and innovation

Job ID: 148477201

Similar Jobs

Singapore, Cecil Street

Skills:

SqlPythonData ModellingPysparkData governance and security best practicesDatabricks PlatformData quality frameworks and testing methodologiesWorkspace AI AgentDelta Lake

Singapore, Paya Lebar

Skills:

data vault PysparkPythonAWSSpark SQLData ModellingSqlGitGcpDatabricksAzureEtlData pipeline design patternsPerformance optimizationStreaming data processingELT processesOptimization TechniquesStructured StreamingDevOps practicesSchema evolutionLakehouse architecturesWorkspace AI AgentDimensional modellingData Engineering principlesDataFrames APIDelta LakeACID transactions

Singapore

Skills:

PysparkEtl

Singapore, Bukit Merah, Jalan Bukit Merah

Skills:

data engineering Data ModellingPysparkSqlELTDevopsGitGcpDatabricksData GovernanceAzurePythonAWSEtlData Quality FrameworksStreaming Data ProcessingDelta Lake

Singapore

Skills:

PysparkApache SparkKafkaELTGitAzure Data FactoryDatabricksAzureAzure DevOpsEtlDatabricks Structured StreamingSpark performance tuningParquet formatGreat ExpectationsAzure Key VaultMLflowUnity CatalogData LakehouseDelta LakeAzure Event Hubs