- 1-year contract, renewable
- Government project
- Hybrid work arrangement
The Senior Data Engineer will mentor data engineers to build and maintain robust infrastructure and systems across digital ecosystem. The team will be instrumental in supporting the development of data pipelines and infrastructure that ensure clean, accurate, and timely data is available for business analytics, decision-making, and AI applications.
Responsibility:
- Guide and plan data engineering processes for digital systems.
- Lead technical discussions with the data engineering team and stakeholders to shape the future of digital products.
- Lead and advise agency on data engineering strategy, architecture, and implementation aligned with the Digital Government Blueprint (DGB2.0).
- Provide technical leadership across multiple product teams by establishing data architecture standards and developer operations best practices that ensure consistency, quality, and enhanced system reliability across all data initiatives.
Data Infrastructure Support
- Assist in designing and developing data pipelines and architectures to collect, process, harmonize and store data from various source systems across digital products
- Support the building of data pipelines that integrate data from multiple platforms and services
- Help maintain data lakes and database infrastructure to support analytical, reporting, and AI workloads across the ecosystem
Data Quality and Monitoring
- Implement data validation, cleansing, and normalisation processes under supervision to ensure data quality and integrity
- Monitor data pipelines and systems to identify potential issues and performance bottlenecks
- Assist in maintaining data governance frameworks that support compliance and data security requirements
System Maintenance and Documentation
- Support the maintenance of data infrastructure supporting digital products
- Assist in troubleshooting and resolving data-related issues under guidance from senior team members
- Create and maintain documentation of data engineering processes and system configurations
Collaboration to facilitate business requirements
- Work closely with senior data engineers, product teams, analysts, and stakeholders to understand business data requirements into technical specifications and scalable solutions
- Design, develop and deploy data tables, visualisation and marts for business reporting
Professional Development
- Stay current with emerging technologies and trends in data engineering through training and mentorship
- Participate in continuous learning opportunities to develop technical skills and domain knowledge
- Contribute to process improvements and suggest enhancements to existing data systems
- Participate in knowledge sharing sessions and contribute to team learning initiatives
Requirement
- A Bachelors Degree, in Computer Science, Data Science, Engineering, Information Technology, or related technical field.
- 5+ years of experience in data engineering, cloud infrastructure, platform engineering or related technical roles
- Experience with any cloud platform (AWS, Azure, or GCP)
- Knowledge and experience business intelligence tools such as Power BI, AWS QuickSight, or similar platforms
- Experience with data visualisation tools and techniques
- Familiarity with agile development methodologies
- Previous project experience involving data processing or analytics
- Possess any of the certifications would be advantageous: AWS Certified Data Engineer, Databricks Certified Data Engineer
Technical Skills
- Proficiency in at least one programming language such as Python, Java, or Scala for data processing and scripting
- Basic knowledge of database systems, both relational and non-relational, and their query languages (SQL knowledge essential)
- Advanced knowledge of data modelling and schema design principles
- Familiarity with data integration and ETL (Extract, Transform, Load) concepts and processes
- Basic experience with version control systems such as Git
Cloud and Platform Knowledge
- Basic familiarity with cloud platforms, preferably AWS services such as S3, EC2, and RDS
- Exposure to big data technologies and frameworks such as Spark or similar platforms
- Experience with Databricks and implementing batch/real-time data pipelines
- Understanding of data governance policies, access controls, and security best practices in government environments.
- Knowledge in data science, statistical analysis and/or machine learning techniques to allow better collaboration with researchers and analysts to support their workflows within the Data Platform ecosystem.
