Senior, Data Platform Architect

Overview:

We are seeking an experienced Senior Data Platform Architect to lead the design and implementation of scalable data solutions on Azure Cloud, with a strong emphasis on Databricks. The ideal candidate will possess deep hands-on expertise in cloud-native data platforms, infrastructure as code (IaC), and modern data architecture principles. This role demands a strategic thinker with hands-on technical skills and the ability to collaborate across teams to drive innovation and efficiency.

Key Responsibilities:

· Architect and implement scalable data platforms using Databricks on Azure, ensuring performance, reliability, and security.

· Design and manage cloud-based data infrastructure using IaC methodologies.

· Collaborate with data engineers, cloud architects, and business stakeholders to deliver robust data solutions.

· Able to define and enforce best practices for data platform governance, and security through access control and data governance technologies.

· Provide technical leadership and mentorship to engineering teams.

· Align data platform architecture with enterprise-wide strategies and business goals.

· Optimize cloud resource usage and manage cost efficiency across data workloads.

Qualifications:

· Minimum of 8 years of experience in cloud data platform architecture, with a strong focus on Azure.

· Proven experience in designing and deploying Databricks-based solutions.

· Strong understanding of data engineering, networking, and cloud architecture.

· Hands-on experience with IaC tools and cloud automation.

· Experience with data modeling, ETL/ELT pipelines, and streaming architectures.

· Familiarity with data governance, metadata management and compliance frameworks (e.g., PDPA, GDPR, HIPAA).

· Bachelor's or master's degree in computer science, Engineering, or a related field.

Required Technologies & Skills:

· Deep expertise in Microsoft Azure services, especially Azure Data Lake, Azure Data factory, Azure Synapse, and Azure Databricks.

· Capable of designing access management and controls for related Azure and Databricks services

· Capable to designing data governance controls into data platform operations

· Strong proficiency in Databricks (Spark, Delta Lake. Delta Share, ML flow, Unity Catalog).

· Experience with IaC tools such as Terraform, ARM templates.

· Experience in GitHub or any other DevOps tools.

· Proficiency in scripting and automation using Python, PowerShell, or Bash.

· Familiarity with cloud security and data governance practices.

· Experience with Data pipeline observability tools in a cloud environment (e.g., Azure Monitor, Log Analytics, Grafana).

· Experience with ETL/ELT tools (e.g., Azure Data Factory, Apache Airflow).

· Understanding of metadata management tools (e.g., Azure Purview, Informatica).

· Exposure to machine learning workflows and integration with data platforms.

· Strong documentation skills and ability to define architectural standards.

· Excellent communication and stakeholder management skills.

· Experience in Agile/Scrum environments and project management tools (e.g., Jira)