Data Engineer

We are seeking a skilled and hands-on Data Engineer to design, build, and operate modern cloud-native data platforms and backend systems. This role combines strong software engineering, cloud infrastructure, and data engineering expertise to deliver secure, scalable, and high-performance solutions across application and data layers.

You will work closely with Product Managers, Business Analysts, Solution Architects, and QA Engineers to deliver robust technical solutions that support operational and analytical use cases across the organisation.

Key Responsibilities

1. Data Platform & Pipeline Engineering

• Design, build, and optimise scalable data pipelines (batch and near real-time), including:

  • Data ingestion

  • Transformation

  • Orchestration

  • Data quality controls

• Develop reliable data flows to support operational systems, analytics, and reporting.

• Implement and integrate enterprise data platforms such as:

  • Operational data stores

  • Data lakes

  • Data virtualisation solutions

• Ensure data integrity, performance optimisation, and resilience across pipelines.

2. Backend & API Development

• Develop and maintain backend services and APIs to enable secure data access and system interoperability.

• Implement secure authentication and authorisation mechanisms (e.g., OAuth, JWT).

• Support microservices-based architectures and cloud-native application deployment models.

3. Cloud Infrastructure & DevSecOps

• Deploy and manage data and application workloads using AWS-native services such as:

  • ECS / Fargate

  • API Gateway

  • Lambda

  • S3

  • RDS

  • IAM

  • CloudWatch

• Design and maintain secure CI/CD pipelines integrating:

  • Automated testing

  • Security scanning (SAST, dependency scanning, container scanning)

  • Policy enforcement

• Automate infrastructure provisioning using Infrastructure-as-Code (e.g., Terraform, CloudFormation).

• Ensure secure configurations, least-privilege access controls, and proper environment isolation (SIT, UAT, PROD).

4. Security & Compliance

• Implement and enforce security controls across the software delivery lifecycle, including:

  • Authentication and authorisation

  • API security

  • Encryption (in transit and at rest)

  • Secrets management

• Support secure coding practices and vulnerability management.

• Ensure compliance with organisational IT governance and cloud security standards.

5. Observability & Operational Excellence

• Implement observability and monitoring across:

  • Applications

  • Data pipelines

  • Cloud infrastructure

• Configure logging, alerting, and performance monitoring to proactively detect and resolve issues.

• Troubleshoot and resolve complex technical issues across data, application, and infrastructure layers.

• Participate in code reviews to maintain high-quality, secure, and maintainable codebases.

6. Agile Delivery & Continuous Improvement

• Participate actively in Agile ceremonies including Sprint Planning, Daily Stand-ups, Reviews, and Retrospectives.

• Work closely with QA teams to ensure sufficient automated test coverage.

• Continuously evaluate and adopt emerging cloud, data engineering, and DevSecOps practices to improve delivery efficiency, security, and reliability.

Qualifications

Educational Background

• Degree in Computer Science, Computer Engineering, Information Technology, or related discipline.

Experience

• Minimum 5 years of relevant experience in software engineering, data engineering, DevOps, or related roles.

• Hands-on experience delivering at least 2-3 full Agile project lifecycles.

• Strong experience with cloud platforms (preferably AWS).

• Proven experience implementing CI/CD pipelines using tools such as GitLab, Jira, Confluence, SonarQube or similar.

• Hands-on experience with Infrastructure-as-Code (Terraform and/or CloudFormation).

• Experience working with cloud-native architectures, microservices, containerisation (Docker, Kubernetes), and modern deployment patterns.

• Strong programming skills in one or more languages such as Python, Node.js, C#, JavaScript, or TypeScript.

• Experience building and integrating secure APIs.

• Practical experience in data engineering concepts including:

  • Data transformation

  • Pipeline orchestration

  • Data modelling

  • Enterprise data platforms

• Strong troubleshooting skills across application, data, infrastructure, and security domains.

• Experience supporting secure software delivery and vulnerability management.

• Experience working in enterprise-scale or regulated environments is advantageous.

• Exposure to emerging technologies such as Large Language Models (e.g., OpenAI, Amazon Q, Gemini) is a plus.