Data Engineering Lead

BAU Lead - Data Engineering (12-month Contract)

About the Company:

Our client is a Singapore-based Insurance organisation.

• Role: BAU Lead - Data Engineering (12-month Contract)
• Remuneration: Up to SGD 7,200 per month
• Location: Singapore

Key Responsibilities:

Our client is looking for a seasoned BAU Lead - Data Engineering to lead our data operations and support function in a cloud-first environment. The ideal candidate will bring strong technical leadership, hands-on expertise in AWS and Snowflake, and proven experience in managing Business-As-Usual (BAU) data platforms, production support, and continuous improvement initiatives.

• Develop tools to improve data flows between internal/external systems and the data lake/warehouse.
• Work with stakeholders to understand needs for data structure, availability, scalability, and accessibility.
• Build robust and reproducible data ingest pipelines to collect, clean, harmonise, merge, and consolidate data sources.
• Understanding existing data applications and infrastructure architecture
• Build and support new data feeds for various Data Management layers and Data Lakes
• Evaluate business needs and requirements.
• Support migration of existing data transformation jobs in Oracle and MS-SQL to Snowflake.
• Lead the migration of the existing data transformation jobs in Oracle, Hive, Impala, etc. into Spark, Python on Glue, etc.
• Able to document the processes and steps.
• Develop and maintain datasets.
• Improve data quality and efficiency.
• Lead Business requirements and deliver accordingly.
• Collaborate with Data Scientists, Architect and Team on several Data Analytics projects.
• Collaborate with DevOps Engineer to improve system deployment and monitoring process.
• Experience in critical production support and how the BAU functions is preferred.

Qualifications & Competencies:

• Lead a team of data engineers in managing day-to-day BAU operations, production support, incident management, and platform stability.
• Strong AWS knowledge in terms of designing new architecture and providing optimised solutions for existing ones. (S3, Glue, DMS, MWAA, AMS, IAM).
• In-depth knowledge with respect to Snowflake and its architecture.
• Prefer prior Experience in a BAU environment.
• Good knowledge of Airflow and MWAA.
• Hands-on experience in SQL/Python/Pyspark.
• Expertise in optimising techniques in cloud environments.
• Should have a vision for data strategy and be able to deliver the same.
• Should be able to lead the design and implementation of data management processes, including data sourcing, integration, and transformation.
• Able to manage and lead a team of data professionals, providing guidance, mentoring and foster a collaborative and innovative team culture focused on continuous improvement.

Qualification:

• At least 8+ years of Data Engineer Experience
• Bachelor's qualification in a computer science or STEM (science, technology, engineering, or mathematics) related field.
• At least 5+ years of recent hands-on professional experience (actively coding) working as a Lead handling support & production issues.
• 2+ years of experience with large-scale datasets, data lake and data warehouse technologies such as AWS Redshift, Google BigQuery, Snowflake. Snowflake is highly preferred.
• At least 3+ years of experience in ETL (AWS Glue), Amazon S3, Amazon RDS, Amazon Kinesis, Amazon Lambda, Apache Airflow, Amazon Step Functions.
• Professional experience working in an agile, dynamic and customer-facing environment is required.
• Understanding of distributed systems and cloud technologies (AWS) is highly preferred.
• Understanding of data streaming and scalable data processing is preferred.
• Strong knowledge in scripting languages like SQL, Python, UNIX shell and Spark is required.
• Understanding of RDBMS, data ingestion, Data flows, Data Integrations, etc.

QUESSCORP SINGAPORE PTE. LTD

Registration No. 199801439D

Disclaimer: The company is committed to ensuring the privacy and security of your information. By submitting this form, you consent to the collection, processing, and retention of the information you provide. The data collected (which may include your contact details, educational background, work experience and skills) will be used solely for the purpose of evaluating your qualifications for the position you're applying for. Your data will be stored securely and retained for the duration necessary to fulfill our hiring process. If you are not selected for the position, your data will be kept on file for a limited period in case future opportunities arise. You have the right to access, correct, or delete your data at any time by contacting us at Quess Singapore | A Leading Staffing Services Provider in Singapore (quesscorp.sg)

This is in partnership with the Employment and Employability Institute Pte Ltd (e2i).

e2i is the empowering network for workers and employers seeking employment and employability solutions. e2i serves as a bridge between workers and employers, connecting with workers to offer job security through job-matching, career guidance and skills upgrading services, and partnering employers to address their manpower needs through recruitment, training, and job redesign solutions. e2i is a tripartite initiative of the National Trades Union Congress set up to support nation-wide manpower and skills upgrading initiatives. By applying for this role, you consent to Quesscorp Singapore's PDPA and e2i's PDPA. By submitting your application for this position, you consent to the collection, use, and disclosure of your personal data by Quesscorp and its recruitment partners, including Employment and Employability Institute Pte Ltd, for the purpose of i) employment facilitation and ii) data and statistical analysis. You may find out more information on the PDPA policies at Quesscorp website and [https://www.e2i.com.sg/pdpa/].