We are looking for SeniorData Engineer role, and this role based here in Singapore.
Job Description
Key Responsibilities:
- Developing and documenting technical solution designs
- Test automation and tools such as Great expectations or Delta Live tables
- Build and release management e.g. Azure Devops Pipelines, Azure Devops Release Manager
- Source code management e.g. Azure Devops, Git
- DevSecOps approach and tools for cloud (including automation of build, deployment, environment management and operational tooling for Azure PaaS and IaaS)
- Agile delivery methodologies such as SCRUM or Kanban
- Knowledge and work management tools (e.g. JIRA, Confluence)
Required Skills & Qualifications:
- SQL Server database design and development (SQL, relational and dimensional modelling)
- Analytical querying using technologies such as SSAS cubes or columnar data stores
- ETL tools such as SSIS, Alteryx, Azure Data Factory
- Building data pipelines using code e.g. Pyspark
- Modern cloud-based data architectures and technologies such as Data Lake, Delta Lake, Data Lakehouse e.g. Azure Data Lake Gen 2, Azure Delta Lake
- Lakehouse architecture using Databricks, Hudi or Iceberg
- NoSQL using CosmosDB, MongoDB, DynamoDB etc.
- Python scripting, including data engineering and machine learning libraries
- Scripting languages such as powershell, bash, DAX, M-language etc. Note - We primarily use SQL, Pyspark and Python in our day-to-day work along with some coding around DAX, Bash, Powershell and other scripting languages.
- Distributed computing mainly around Spark
- Big data files formats e.g. Parquet, Avro, ORC etc.
- Big data technologies such as Hadoop, Hive, Map-Reduce, Partition design, performance tuning in Spark etc.
- Reporting and visualisation tools such as SSRS, Tableau, Power BI or Excel
