Data Engineer

Avensys is a reputed global IT professional servicescompany headquartered in Singapore. Our services include enterprise solutionconsulting, business intelligence, business process automation, and managedservices. Given our decade of success, we have become one of the top trustedproviders in Singapore and service a client base across banking and financialservices, insurance, information technology, healthcare, retail, and supplychain.

We are currently looking to hire a Data Engineer. This is anexciting opportunity to expand your skill set andachieve job satisfaction and work-life balance. I've included moredetails below.

JobDescription:

The Data Engineer will be responsible for designing,developing, and maintaining scalable and reliable data pipelines on Databricksand cloud platforms. The role requires integrating diverse data sources,ensuring high-quality data processing, and supporting analytics, reporting, andmachine learning workloads. The role involves collaborating closely withanalytics, product, and infrastructure teams to enhance the company's dataplatform while adhering to best practices for governance, monitoring, and reliability.

What will you do

- Develop and maintain ETL pipelines for centralizeddata storage systems (e.g. Delta Lake).

- Integrate data from databases, APIs, log files,streaming platforms, and external providers

- Develop data transformation routines to clean,normalize, and aggregate data

- Apply data processing techniques to handle complexor inconsistent datasets

- Contribute to frameworks and best practices for codedevelopment and deployment

- Implement data governance in alignment with companystandards

- Partner with analytics and product leaders to designand operationalize pipelines

- Collaborate with infrastructure leaders to advancecloud-based data platforms

- Explore new tools and techniques leveraging Azure,Databricks, or related platforms

- Monitor data pipelines to detect and resolve issuespromptly

- Develop monitoring tools, alerts, and automatederror-handling mechanisms

- Analyse business requirements and identify dataextraction requirements

- Attend requirement grooming and refinement sessionswith users

- Develop and maintain ETL pipelines for ingestion,transformation, validation, and loading

- Optimise performance and batch scheduling

- Develop dashboards, reports, scorecards, and datavisualizations

- Perform SIT, data validation, data profiling andconfirm data accuracy

- Validate completeness and consistency of ETL Loads

- Support UAT and production implementation

Qualifications

The ideal candidate should possess:

- 3 or more years of experience in data engineeringwith scalable pipelines

- Strong experience designing data solutions includingdata modelling and distributed computing architectures

- Hands-on experience with data processing jobs usingPySpark, Spark SQL, and Databricks notebooks/jobs

- Experience orchestrating data pipelines with ADF,Airflow, or similar tools

- Experience with both real-time and batch dataprocessing

- Experience building pipelines on Azure, with AWSexperience beneficial

- Proficiency in SQL including window functions andperformance optimization

- Understanding of DevOps tools, Git workflows, andCI/CD pipelines

- Familiarity with Scrum methodology and experienceworking in Scrum teams

- Ability to apply Scrum practices in a practicalproject context

- Strong problem-solving and collaborative mindset

- Experience with streaming technologies such asApache Kafka, Apache Flink, or AWS Kinesis

- Ability to design and implement real-time dataprocessing pipelines

Databricks Certified Data Engineer Associate andDatabricks Certified Data Engineer Professional are preferred

PrivacyStatement: Data collected will be used for recruitment purposes only.Personal data provided will be used strictly in accordance with therelevant data protection law and Avensys privacy policy.

WHAT'SON OFFER

You will be remunerated with an excellent base salary and entitled to attractive company benefits. Additionally, you will get the opportunity to enjoy a fun and collaborative work environment, alongside a strong career progression.

To submit your application, please apply online or email your UPDATED CV in Microsoft Word format to [Confidential Information]. Your interest will be treated with strict confidentiality.

CONSULTANT DETAILS

Consultant Name: Zayn Atif
Avensys Consulting Pte Ltd
EA Licence 12C5759