Senior Data Engineer (Databricks)

SHELL INFOTECH PTE. LTD.

Singapore, Paya Lebar

7-10 Years

SGD 7,800 - 12,000 per month

Save

Posted 3 hours ago
Be among the first 10 applicants

Early Applicant

Job Description

Job Description
Data Engineering: Strong foundation in data engineering principles, ETL/ELT processes, and data pipeline design patterns.
PySpark: Proven hands-on experience developing data pipelines using PySpark, including DataFrames API, Spark SQL, and performance optimization.
Databricks Platform: Practical experience with Databricks workspace, cluster management, notebooks, and job orchestration.
Workspace AI Agent: Knowledge of Databricks Workspace AI Agent capabilities and integration.
Data Modelling: Experience implementing data models including dimensional modelling, data vault, or Lakehouse architectures.
Delta Lake: Understanding of Delta Lake features including ACID transactions, schema evolution, and optimization techniques.
Python: Strong Python programming skills for data processing and automation.
SQL proficiency for data querying and transformation
Experience with cloud platforms (Azure, AWS, or GCP)
Knowledge of streaming data processing (Structured Streaming)
Familiarity with DevOps practices and CI/CD pipelines
Experience with version control systems (Git)
Requirements:
Must have experience in data engineering or related roles
Hands-on experience with Databricks platform
Proven track record of refactoring legacy code to modern frameworks
Experience building and maintaining production data pipelines at scale
Background working across multiple data sources and formats
Experience in agile development environments
Databricks Certified Data Engineer Associate OR Databricks Certified Data Engineer Professional
Additional Certifications (Preferred)
Databricks Certified Associate Developer for Apache Spark
Cloud platform certifications (Azure Data Engineer Associate, AWS Certified Data Analytics, or Google Cloud Professional Data Engineer)
Relevant data engineering or big data certifications.