Data Engineer

Must-have technical skills

AWS, SQL, PySpark, Apache Iceberg, dbt and Apache Airflow.

Key Responsibilities

. Build and maintain data pipelines using cloud technologies and distributed systems.

. Work with data transformation frameworks and pipeline orchestration tools.

. Manage large volumes of data and optimise data processing.

. Design and implement effective tests, including unit, component, integration, end-to-end and performance tests.

. Coach team members on coding practices, design principles and implementation patterns that produce high-quality, maintainable solutions.

. Manage stakeholders and prioritise work effectively across multiple workstreams.

Requirements

. Formal training or certification in data engineering concepts, together with applied experience.

. Recent hands-on professional experience as a data engineer.

. Experience coding in a recent version of Python.

. Hands-on experience with AWS, SQL, PySpark, Apache Iceberg, dbt and Apache Airflow.

. Experience with cloud technologies and distributed systems.

. Experience with data transformation frameworks and data pipeline orchestration tools.

. Experience managing large volumes of data and optimising data processing.

. Understanding of event-based architecture, data streaming and messaging frameworks.

. Experience designing and implementing effective software tests.