Big Data Engineer

Interested candidate please share your updated copy of resume at [Confidential Information]

( To receive a quick response, kindly leave a message on the WhatsApp number +65 96310776

******************************************************************************************************

Looking for an experienced Big Data Operations Engineer (10+ years experience) with strong knowledge of Python, Kubernetes, kafka, Spark, Distributed Storage like S3/HDFS etc. Should be able to work independently and have experience in monitoring tools for platform health, daily data ingestion pipelines, consumer applications.

SKILLS REQUIREMENTS OF THE POSITION

Technical

• At least 5 years of experience in cluster environment operations (Kubernetes, Kafka, Spark, Distributed Storage like S3/HDFS)
• Great Python proficiency
• Expert on processes and topics around Data Ops
• Experience in monitoring tools for platform health, daily data ingestion pipelines, consumer applications
• Experience working on Linux based infrastructure incl. advanced scripting
• Expert SQL skills, preferably in mixed environments (i.e. classic DWH and distributed)
• Day to day maintenance of the DBs and load processes, CI-CD pipelines etc.
• Several years of work experience in a similar capacity in the financial industry
• Strong experience in operating and working with complex data environment
• Good to have experience on Dataiku and Tableau

KEY FEATURES OF THE POSITION

Functional/Technical

• Daily support and troubleshooting of data pipelines from ingestion to consumption Ensure proper thresholds are met for data availability
• Actively support release cycles and advocate on changes to avoid production disruption
• Maintain and model JSON based schemas and metadata to re-use it across the organization (with central tools)
• Act as a Data Engineer to ensure corrective measures are implemented (historizing tables, running dependencies, catching quality issues and similar)
• Be responsible operationally of Common Data Model tables in a separate access zone
• You are part of the agile setup and support development team
• Manage the operating model to ensure support levels and effort sharing of operational tasks (e.g. monitoring of service availability and performance) with
• 1st and 2nd Level Support
• Responsible to provide 3rd level support (event management like Incident & Problem Mgmt) for the IT Service and (together with the IT Service Owner) for the application related data pipelines
• Continuously improve the service availability, performance, capacity & knowledge management

****************************************************************************

EA Personnel Registration Number: R. No: R1114530

Singapore Employment Agency Licence No: 11C3373