Data Engineer AMK

Job Responsibilities

• Analyse and migrate existing workflow integrations to new platforms.
• Develop workflow orchestration, including triggers, routing, retries and exception handling.
• Build automation for batching, pagination, looping and throttling.
• Design and maintain ETL pipelines for file-based data ingestion into relational databases.
• Perform schema validation, data quality checks and error reconciliation.
• Develop data pipelines using cloud-based big data platforms and distributed processing frameworks such as Spark.
• Support Lakehouse-based data environments for batch and streaming workloads.
• Implement logging, monitoring and alerting for data pipelines and workflows.
• Troubleshoot issues and participate in root-cause analysis and continuous improvement.
• Work with application teams to understand data requirements and upstream/downstream dependencies.
• Ensure data pipelines meet performance, reliability, security and governance requirements.

Job Requirements

• 3-5 years of relevant experience in Data Engineering, Integration Engineering or a related role.
• Experience with workflow orchestration or integration platforms.
• Hands-on experience with ETL and file-based data ingestion.
• Proficient in SQL and relational databases such as SQL Server.
• Experience with Spark or other distributed data processing frameworks.
• Experience with Python or other programming/scripting languages.
• Understanding of batch and streaming data processing.
• Experience with cloud platforms and managed data services.
• Experience working with REST APIs, JSON and CSV.
• Understanding of CI/CD for data and integration workflows.
• Good understanding of data quality, error handling and pipeline reliability.

Good to Have

• Experience with Lakehouse table formats, incremental processing and time travel.
• Experience with Flink or Spark Streaming.
• Familiarity with analytical query engines.
• Experience in banking, financial services, manufacturing or other regulated environments.
• Knowledge of data observability and data quality tools.