Data Engineer

As a Data Engineer, you will be part of the Data Management and Operations team, responsible for maintaining a resilient data infrastructure, ensuring data quality, and managing end-to-end data pipelines. You will work closely with internal teams, external partners, and AI/ML engineers to support data ingestion, model deployment, and the delivery of high-quality data solutions.

Key Responsibilities

Infrastructure & Cloud Engineering

• Design, build, and manage scalable data infrastructure using AWS and other cloud platforms.

• Evaluate and adopt emerging tools to enhance system performance and capabilities.

Data Pipelines & Modelling

• Develop and maintain data models and pipelines for ingesting, processing, and distributing large datasets.

• Ensure consistency of data schemas, governed access, and reliable data flows.

• Work with vendors on secure API integrations, including API specifications and documentation.

Data Quality & Reliability

• Lead data quality frameworks to ensure accuracy, completeness, and reliability.

• Troubleshoot pipeline issues, support incident resolution, and contribute to post-incident reviews.

AI/ML Collaboration

• Partner with AI and ML teams to deliver optimised data pipelines for model training and deployment.

• Ensure infrastructure supports advanced analytics and machine learning workloads.

Documentation & Knowledge Sharing

• Maintain clear and comprehensive documentation on data architecture, processes, and standards.

• Build accessible resources to support consistent data practices.

Automation & Integration

• Automate data preparation tasks to reduce manual effort and operational risk.

• Implement seamless integration solutions to improve overall efficiency.

Governance & Stakeholder Engagement

• Collaborate with data governance teams to promote high-quality, trusted data assets.

• Engage with business stakeholders to refine requirements and co-develop solutions.

• Support the creation and upkeep of curated data catalogues.

Training & Continuous Improvement

• Conduct training sessions to uplift data competencies across teams.

• Establish feedback loops with data consumers to enhance pipeline performance.

Requirements

• Strong foundation in data engineering, including version control, release processes, and scalable pipeline development.

• Proficiency in Python and SQL exposure to AI/ML development or deployment is an advantage.

• Experience handling large, complex datasets and working with data warehouses.

• Independent, collaborative, and able to work effectively in cross-functional environments.

• Strong analytical and problem-solving skills.

• Excellent communication skills, both verbal and written.