Big Data Engineer

Job Summary:

We are seeking an experienced Big Data Engineer to develop and support enterprise big data platforms while translating business and system requirements into scalable technical solutions. The role involves working with technologies such as HDFS, Trino, S3, Python, SQL, and CI/CD pipelines, while ensuring adherence to coding, security, and performance standards. The ideal candidate will have strong problem-solving and communication skills, with experience working in Agile environments and collaborating with cross-functional teams to ensure timely and stable project delivery.

Job Responsibilities:

• Drive development projects for big data platforms.
• Understand system and functional requirements and translate them into technical implementations.
• Translate business requirements to FSD documentation.
• Adopt best practices like Test-Driven Development (TDD) and continuous Integration (CI)
• Ensure adherence to standards (incl. code, security and performance).
• Participate in reviews and meetings and provide updates on project progress
• Take responsibility for ensuring that risks and issues are identified and managed closely and drive all stakeholders to deliver on time and to the required quality standards
• Ensure production stability and timely communication with stakeholders

Job Requirements:

Technical Skills

• Bachelor's degree in Computer Science, Information Technology, or related field
• At least 5 years of experience in big data engineering
• Excellent knowledge and working experience of Big data platform and related technologies - HDFS, Trino, S3 bucket
• Experience with programming languages like Python
• Experience with CI / CD pipelines, code vulnerabilities
• Experience of working with various IDEs
• Excellent understanding of data management, data design and data architecture
• Working experience of SQL (MSSQL, Oracle, Postgres)
• Knowledge of basic Linux shell scripting commands

Professional Skills

• Strong understanding of software development life cycles, Agile methodologies, and version control systems like Git
• Excellent problem-solving skills, with the ability to analyse complex issues and develop creative solutions
• Strong communication and collaboration skills, with experience working with cross-functional teams