We are looking for a Data Engineer to join our analytics team. The role involves developing and maintaining data pipelines, ETL/ELT processes, and analytical data warehouses to support business and cross-functional requirements. The candidate will also be responsible for data preparation, database optimisation, schema design, and improving stored procedures, batch jobs, and JavaScript-based processes.
Responsibilities
- Develop, construct, test and maintain data architectures such as databases, data warehouses and large-scale data processing systems
- Design and develop data pipelines/systems for data modelling, mining and production
- Ensure the data architecture is in place to support routine and ad-hoc requirements of data analytics team, stakeholders and the business
- Leverage on variety of programming languages and data crawling/processing tools to make raw data clean and highly available for use in descriptive and predictive modelling
- Recommend and implement ways to improve data quality, reliability, flexibility and efficiency
- Ensure data assets and data catalogs are organized and stored in an efficient way so that information is easy to access and retrieve
- PL/SQL and SQL Tuning and optimization of newly develop and existing applications
- Flexibility for critical issues: Exhibit flexibility to provide support for urgent, high-priority incidents or system failures that may occur outside of core business hours.
Requirements
- At least 3 years working experience in data architecture, data warehousing, data processing, data modelling and ETL/ELT, familiarity with real-time streaming solutions.
- At least 3 years working experience in Kubernetes-based DevOps practices, with experience in container orchestration, CI/CD pipelines, and microservices deployment.
- Working experience in database development (Oracle SQL/PLSQL)
- Working experience in AWS cloud environment, familiar with solutions such as EC2, S3, EMR, Redshift, Athena, Kinesis
- Programming knowledge in Python, R, SQL for data cleaning, processing and aggregation
- Proficiency in one or more of the following: Java, Hadoop, HDFS, Apache Airflow, Apache Spark, Scala, Hive, Pig
- Hands-on experience in data crawling, data modeling, data lake formation and data warehouse construction
- Hands-on experience in collecting and massaging structured/unstructured/semi-structured data
- Basic knowledge of Oracle database architecture
- Ability to build thorough algorithms and deploy machine learning & statistical models will be a plus
- Be proactive in work
- Strong communication skills
- Excellent time management skills
- Team player
Required Education (including Major):
- Bachelor's or Master's degree in Computer Science, Computer Engineering, Software Engineering, Database Administration or related Technical major
- Related Certificated in developer/data science/database/machine learning will be a plus
