Job Description
As a Data Engineer, you will be responsible for developing and maintaining scalable data pipelines, integrating multiple data sources, and ensuring high-quality data processing for analytics, reporting, and machine learning tasks. The role involves collaborating with analytics, product, and infrastructure leaders to advance the company's data and analytics platform, while ensuring data governance, monitoring, and reliability standards are met.
What will you do
- Develop and maintain ETL pipelines for centralized data storage systems
- Integrate data from databases, APIs, log files, streaming platforms, and external providers
- Develop data transformation routines to clean, normalize, and aggregate data
- Apply data processing techniques to handle complex or inconsistent datasets
- Contribute to frameworks and best practices for code development and deployment
- Implement data governance in alignment with company standards
- Partner with analytics and product leaders to design and operationalize pipelines
- Collaborate with infrastructure leaders to advance cloud-based data platforms
- Explore new tools and techniques leveraging Azure, Databricks, or related platforms
- Monitor data pipelines to detect and resolve issues promptly
- Develop monitoring tools, alerts, and automated error-handling mechanisms
Qualifications
The ideal candidate should possess:
Essential Qualifications
- The position is open to wide range of data engineering experience level including fresh graduate
- Good experience designing data solutions including data modeling
- Hands-on experience with data processing jobs using Talend, PySpark and SQL
- Experience orchestrating data pipelines with Talend, ADF, Airflow, or similar tools
- Experience with both batch and real-time data processing
- Proficiency in SQL including advanced features, with exposure to more than one data systems, such as SQL DB, Cloudera
- Understanding of DevOps tools, Git workflows, CI/CD or MLOps pipelines
Preferred Attributes
- Domain knowledge of defense logistics, including its organisation and processes
- Experience and ability to apply Scrum practices in a practical project context
- Good knowledge of SAP system functionally with prior experience of working on ABAP code
- Strong problem-solving and collaborative mindset
- Some knowledge and experience of using Qlik for BI development
