Data Engineer / Lead (Azure Data factory, PySpark)

Key Responsibilities

1. Provide technical leadership for the architecture, development, and operation of the company's data platform based on Azure Databricks and the Azure data ecosystem. 2. Lead the end-to-end design of data pipelines, ingestion frameworks, ELT/ETL processes, and Lakehouse data models (Bronze/Silver/Gold). 3. Partner with data source system teams to design efficient, reliable, and secure data integrations. 4. Oversee the implementation of data governance, access control, and metadata management using Unity Catalog. 5. Ensure platform scalability, performance optimization, and cost efficiency, including cluster management, workflow automation, and FinOps best practices. 6. Drive the establishment and enforcement of coding standards, data engineering best practices, CI/CD workflows, and platform-level operational guidelines. 7. Guide the team in troubleshooting, incident resolution, and continuous improvements of data platform reliability. 8. Mentor data engineers and analysts, enabling them to adopt best practices in Databricks, Delta Lake, and distributed data processing. 9. Collaborate with business units to translate data needs into scalable technical solutions. 10. Other responsible assigned tasks.

Qualifications

. 5+ years of experience in data engineering, data platform development, or related technical roles. . Strong hands-on experience with Azure Databricks, Delta Lake, Spark, and Unity Catalog. . Solid understanding of Azure services such as Data Lake Storage Gen2, Azure Data Factory, Key Vault, Event Hub, Synapse, or Azure Functions. . Experience in building scalable data ingestion, ETL/ELT, and workflow orchestration. . Strong understanding of data modelling, Lakehouse design, and distributed computing. . Experience with CI/CD pipelines (Azure DevOps/GitHub Actions). . Proven ability to lead technical discussions and drive architectural decisions. . Strong communication and stakeholder management skills.