The Data Engineer is responsible fordesigning, building, and maintaining data infrastructure that supportsanalytics, applications, and enterprise reporting needs. This includesconstructing scalable pipelines, managing data flows across systems, andenabling reliable storage, integration, and retrieval of information acrosscloud and on‑premise environments. The role involves close collaboration withcross‑functional teams to deliver data‑driven solutions that are robust, secure,and high-performing.
Key Responsibilities
1. Data Architecture & Development
- Design, create, and deploy data models, tables, views, and data marts across data warehouses, operational data stores, data lakes, and data virtualization environments.
- Build backend APIs and database structures to support applications and data products.
2. Data Extraction & Transformation
- Perform data extraction, cleansing, transformation, and blending across multiple sources.
- Conduct web scraping activities when needed, using appropriate automation tools.
- Ensure data reliability, accuracy, and consistency through well-structured data processing workflows.
3. Data Pipelines & Processing
- Develop and maintain large-scale batch and real-time data pipelines using modern data engineering frameworks.
- Implement ETL/ELT processes leveraging tools such as SSIS, AWS Glue, AWS DMS, Python, Lambda, ECS tasks, EventBridge, Spring, or similar technologies.
- Integrate data from distributed systems and eliminate data silos through scalable and secure data flows.
4. Collaboration & Product Delivery
- Partner with Project Managers, Data Architects, Business Analysts, Frontend Developers, Designers, and Data Analysts to deliver end-to-end data-driven solutions.
- Participate in Agile ceremonies and contribute to continuous integration and delivery practices.
- Engage in pair programming and participate in code reviews to ensure coding standards and quality.
5. Infrastructure, Cloud & Big Data
- Work with cloud platforms such as AWS, Azure, or Google Cloud to build and operate cloud-native data solutions.
- Design and manage data pipelines in big data environments using tools like Spark, Hadoop, Kafka, RabbitMQ, or similar technologies.
- Ensure systems are resilient, scalable, and suitable for production workloads.
6. Data Governance & Security
- Apply data governance principles, ensuring compliant access, secure storage, and proper data handling practices.
- Implement security best practices for data assets and support policy enforcement across systems.
Required Technical Skills
Data Processing & Programming
- Proficient in data cleaning and transformation using SQL, Python (pandas), R, or equivalent tools.
- Strong skills in ETL/ELT pipeline development and workflow orchestration.
Databases & Storage
- Hands-on experience designing and operating relational and non-relational databases such as SQL Server, PostgreSQL, MySQL, MongoDB, Cassandra, Athena, SQLite, VoltDB, or S3-based storage.
- Familiarity with data warehouse, data lake, data mart, and data virtualization concepts.
System Design & APIs
- Solid understanding of system design, data structures, algorithms, and backend development.
- Comfortable working with REST APIs, HTTP protocols, and integrating various data sources.
Big Data & Distributed Systems
- Knowledge of distributed systems and tools (Hadoop, Spark, Kafka, etc.).
- Ability to implement streaming, micro‑batch, or batch processing pipelines.
Web Scraping & Automation
- Experience with web scraping technologies like BeautifulSoup, Selenium, CasperJS, PhantomJS, Node.js, or similar tools.
Security & Data Governance
- Familiar with access control, encryption, audit logging, and other data security practices.
Tools & Environment
- Comfortable working in both Windows and Linux environments.
- Experience using CI/CD pipelines and participating in DevOps-oriented workflows.
Soft Skills & Attributes
- Detail‑oriented with a strong focus on problem solving and analytical thinking.
- Good communication skills, capable of discussing technical topics with both technical and non‑technical stakeholders.
- Collaborative mindset and ability to work as a bridge between engineering and analytics teams.
- Self‑motivated, adaptable, and comfortable managing work in a fast-paced Agile environment.
Please submit resumes to [Confidential Information] with the following details in MS Word format:
- Position applying for
- Current remuneration
- Expected remuneration
- Notice period
John Goh Meng Chye
EA License No : 06C4642
EA Reg No : R1102621
We regret that only shortlisted candidates will be notified.
