Senior Data Engineer (Python & Software Quality)

Job Description

Role Summary: We are seeking a highly skilled Senior Data Engineer to join our team, focusing on the end-to-end lifecycle and maintenance of critical data products. This role is ideal for a software engineering specialist who is not only a proficient Python coder but also an advocate for rigorous testing and high-quality software standards. You will take ownership of maintaining scalable data pipelines, ensuring system reliability through automated testing, and collaborating across technical teams to deliver stable data solutions.

Key Responsibilities:

ETL/ELT Excellence: Design, build, and optimize robust data pipelines, ensuring they are efficient, scalable, and easy to maintain.

Software Quality Assurance: Drive the reliability of data products by implementing comprehensive unit and integration testing frameworks.

Automation & Monitoring: Partner with cross-functional teams to automate deployment workflows (CI/CD) and establish proactive monitoring/alerting for data workloads.

Storage & Performance: Troubleshoot complex pipeline failures, optimize data storage structures, and resolve performance bottlenecks to ensure high system availability.

Technical Stewardship: Translate complex business requirements into maintainable technical designs and contribute to the long-term architectural stability of data assets.

Requirements

Education: Bachelor's or Master's Degree in Computer Science, Information Technology, or a related technical discipline.

• Experience: At least 8 years of professional experience in Software Engineering or Data Engineering.
• Coding: Expert-level proficiency in Python is a strict requirement.
• Testing: Proven track record of implementing unit and integration tests in a production environment.
• Environment: Strong familiarity with DevOps practices, CI/CD tools, and Agile/Scrum methodologies.
• Analytical Skills: Exceptional problem-solving abilities with a focus on data integrity and pipeline performance.
• Communication: Excellent interpersonal skills with the ability to work effectively as a team player in a collaborative environment.