Senior Data Engineer (Contract)

We are seeking a competent Big Data Developer to join data engineering team of a large Financial institution. The successful candidate will be responsible for designing, building, and maintaining big data pipelines and platforms that support critical business and analytics initiatives. This role requires a solid foundation in big data technologies, programming, and data management principles-along with the discipline to write clean, testable, and secure code within a CI/CD environment.

Key Responsibilities

• Design, develop, and optimise big data processing workflows using HDFS, Trino, and S3 bucket storage.

• Write maintainable and scalable Python code for data ingestion, transformation, and validation.

• Collaborate with data architects and data scientists to implement robust data models and data designs.

• Develop and optimise SQL queries across multiple database platforms including MSSQL, Oracle, and PostgreSQL.

• Build and maintain CI/CD pipelines, incorporating code vulnerability scanning and security best practices.

• Write and maintain basic Linux shell scripts for automation and job scheduling.

• Work with various integrated development environments (IDEs) to develop, debug, and test code.

• Troubleshoot performance issues across data pipelines, storage layers, and query execution.

• Document data flows, system designs, and operational procedures for knowledge sharing and audit purposes.

Qualifications and Experience

Mandatory

• Bachelor's degree in Computer Science, Information Technology, or a related field.

• Minimum 5 years of professional experience in big data engineering.

• Excellent working knowledge of big data platform technologies:

  • HDFS (Hadoop Distributed File System)

  • Trino (formerly PrestoSQL)

  • S3 bucket storage (or equivalent object storage)

• Strong experience with Python programming for data engineering tasks.

• Hands-on experience with CI/CD pipelines and familiarity with code vulnerability assessment tools.

• Experience working with various IDEs (e.g., IntelliJ, PyCharm, VS Code, or Eclipse).

• Solid understanding of data management, data design, and data architecture principles.

• Working experience with SQL on MSSQL, Oracle, and PostgreSQL databases.

• Basic knowledge of Linux shell scripting (bash, sh).

Interested candidates please email your latest resume to [Confidential Information]