Senior Data Engineer / Data Architect (Databricks)

What You Will Be Doing

• Design and build production-grade data solutions and applications using Databricks.
• Develop scalable data pipelines, ingestion frameworks and distributed data processing solutions.
• Lead or contribute to the end-to-end architecture, design, build and deployment of enterprise data platforms.
• Work directly with customers to understand business and technical requirements and translate them into workable solutions.
• Guide customers on Databricks architecture, implementation and platform best practices.
• Integrate Databricks solutions with customers existing cloud, data and application environments.
• Support the delivery of big data, analytics and, where applicable, ML/AI solutions.
• Work closely with project managers, architects, engineering teams and customer stakeholders to ensure successful delivery.
• Troubleshoot implementation issues and work with Databricks engineering/support teams where required.
• Develop reusable frameworks, accelerators, reference architectures and technical documentation.
• Participate in technical discussions, workshops and whiteboarding sessions with both technical and senior business stakeholders.

Key Requirements

• Strong background in Data Engineering, Data Platforms, Big Data or Software Engineering.
• Strong hands-on experience with Databricks and Apache Spark.
• Proficiency in Python and/or Scala.
• Good understanding of distributed computing and Spark architecture/runtime.
• Experience designing and deploying production-scale data architectures and pipelines.
• Experience with at least two major cloud platforms – AWS, Azure or GCP, with strong expertise in at least one.
• Familiarity with CI/CD and production deployment practices.
• Strong enterprise customer-facing and technical consulting experience.
• Able to manage technical scope, timelines, stakeholders and delivery outcomes.
• Strong communication, documentation and solution-design skills.
• Databricks certification is preferred.
• Experience with MLOps, ML/AI models or AI APIs is an advantage.