Data Engineer

Responsibilities:

. Test and validate data pipelines for all new releases.

. Ensure data integrity and completeness after deployments.

. Debug and troubleshoot pipeline issues.

. Implement fixes and document root causes.

. Optimise pipeline performance and efficiency.

. Write clean, maintainable Python code with unit tests.

. Collaborate with data scientists, engineers, and stakeholders.

. Support AWS-based pipeline infrastructure (e.g., S3, Lambda, EC2, CloudWatch).

Requirements:

. Proficient in Python (unit testing, clean code practices).

. Experience in data pipeline development and troubleshooting.

. Familiarity with AWS services (S3, Lambda, EC2, CloudWatch).

. Strong debugging and problem-solving skills.

. Attention to detail and focus on data quality.

. Good to have:

. Basic understanding of data science workflows.

. Experience with CI/CD and version control (e.g., Git).