Senior Data Engineer

Responsibilities

• Design, develop and deploy large-scale batch and real-time data processing pipelines.
• Build and optimise distributed data processing solutions.
• Deploy, manage and scale data workloads in Kubernetes environments.
• Manage data storage, quality, schema, metadata and lineage.
• Implement data checkpointing, recovery, backup and disaster recovery solutions.
• Ensure data integrity, retention and high availability across distributed storage systems.
• Maintain CI/CD pipelines and support automated deployment processes.
  
  

Qualifications

• Minimum 5 years of relevant data engineering experience.
• Hands-on experience with Apache Spark, Apache Flink and Apache Kafka.
• Proficiency in Java or Scala and SQL.
• Experience with Kubernetes and S3-compatible object storage.
• Knowledge of data quality controls, schema management, data lineage, checkpointing and recovery.
• Familiarity with GitLab and CI/CD practices.
• Experience with MinIO or equivalent S3-compatible storage solutions.
• Knowledge of Bronze/Silver/Gold data architecture and data integrity controls.
• Experience in high availability, backup, disaster recovery and inter-site synchronisation.
• Experience with Apache Airflow or equivalent workflow orchestration tools.
• Familiarity with SAFe and safety-critical or mission-critical systems.
  
  

Other Information


Beyondsoft Consulting (Singapore) Pte. Ltd is committed to being an equal opportunity employer and provides equal employment opportunities to all employees and applicants. We strive to cultivate a workplace that celebrates diversity and inclusion, where individuals of all backgrounds—regardless of nationality, ethnicity, religion, age, gender identity, sexual orientation, or any other distinguishing trait—can succeed and thrive. We prohibit discrimination and harassment of any type with regard to race, color, religion, age, national origin, disability status, genetics, sexual orientation, gender identity, or expression. This policy applies to all terms and conditions of employment, including recruiting, hiring, and the entire employee lifecycle. We are focused on creating an environment where everyone can reach their full potential.

Employment offers from Beyondsoft Consulting (Singapore) Pte. Ltd. are contingent upon the successful completion of any required pre-employment processes, in line with applicable laws and regulations. Beyondsoft Consulting (Singapore) Pte. Ltd. does not ask for any recruitment fees, nor does it request any unauthorized payments from candidates as part of the hiring process.