Job Summary
We are seeking a highly skilled and hands-on Technical Lead to drive the design, delivery, and operational excellence of our data and analytics platform. This cloud-native environment is purpose-built for advanced data engineering and analytics workloads. The Tech Lead will bridge solution architecture, project delivery, and engineering execution. In addition to leading a team, you will actively design, code, review, and troubleshoot complex solutions on AWS and Databricks, ensuring alignment with architectural standards, project timelines, and business objectives.
Key Responsibilities
- Technical Leadership & Delivery
- Lead a team of data engineers in designing and delivering end-to-end data pipelines (batch, streaming, event-driven).
- Actively contribute through coding, reviewing, and optimizing solutions in Python/Scala, Spark, and SQL on Databricks and AWS.
- Translate architectural blueprints into actionable tasks and oversee execution.
- Troubleshoot and resolve production issues, ensuring high availability and reliability.
- Ensure solutions meet performance, security, governance, and compliance standards in regulated domains.
- Oversee code quality, CI/CD workflows, observability, and DevOps practices.
- Collaboration & Stakeholder Management
- Partner with the Solution Architect to validate and refine solution designs, including developing prototypes/POCs.
- Collaborate with the Project Manager to align delivery milestones, manage dependencies, and mitigate risks.
- Work closely with domain experts, data scientists, and external vendors to integrate AI/GenAI use cases into production systems.
- Innovation & Continuous Improvement
- Drive adoption of modern practices in data engineering, MLOps, and automation to enhance productivity.
- Identify and resolve bottlenecks in pipelines, infrastructure, and delivery processes.
- Mentor and guide team members while leading by example through hands-on technical contributions.
Qualifications
Education
- Bachelor's or Master's degree in Computer Science, Data Engineering, Data Science, Information Systems, or a related discipline.
Experience
- 7+ years in data engineering, cloud-native solutions, or related technical roles.
- 3+ years in a technical leadership capacity, balancing leadership with hands-on contribution.
- Strong expertise with AWS data services and Databricks (Delta Lake, Spark, Unity Catalog, etc.).
- Proven experience delivering secure platforms in regulated domains (aviation, finance, or public sector preferred).
- Familiarity with Agile/Scrum methodologies and cross-functional collaboration.
Preferred Qualifications
- Experience with AWS Databricks, Delta Lake, streaming data, and medallion architecture.
- Exposure to AI/GenAI concepts or agent-based architectures.
- Knowledge of aviation data standards (ADS-B, ARINC 424, flight schedules) or willingness to learn.
- AWS certifications in Data Analytics, Big Data, or Machine Learning.
- Experience with real-time data processing and high-volume systems.
