Responsibilities:
Data Pipeline Development & Optimization:
- Design, develop, and maintain scalable data pipelines for real-time and batch processing.
- Implement efficient data extraction, transformation, and loading (ETL) processes.
- Optimize data workflows for performance, scalability, and reliability.
Graph Database & Network Analysis:
- Work with graph databases such as Neo4j and NetworkX to model and analyze complex relationships.
- Develop and optimize graph-based data structures for efficient querying and analysis.
- Implement algorithms for network analysis, anomaly detection, and pattern recognition.
Graph Database Management & Data Modeling:
- Design and implement graph database solutions tailored for data engineering use cases.
- Develop data models that support efficient data storage, retrieval, and analysis within graph databases.
- Ensure data integrity, consistency, and security through best practices in graph database management.
Big Data & Streaming Frameworks:
- Work with big data frameworks such as Hadoop and Spark for large-scale data processing.
- Develop and optimize data lake architectures to support efficient data storage and retrieval.
- Utilize streaming frameworks such as Apache Kafka and Apache Flink for real-time data processing.
Collaboration & Cross-Functional Integration:
- Work closely with data scientists, analysts, and software engineers to translate business requirements into technical solutions.
- Collaborate on the integration of data pipelines with machine learning models and analytical applications.
- Provide technical mentorship and guidance to junior team members.
Data Quality & Governance:
- Implement data validation, monitoring, and governance strategies to ensure high data quality.
- Establish best practices for data lineage, documentation, and version control.
- Ensure compliance with data privacy and security policies.
Requirements:
Experience:
- 5+ years of professional experience in data engineering or related fields.
- Proven experience in designing and maintaining large-scale data pipelines and processing frameworks.
- Experience working with graph databases such as Neo4j and tools like NetworkX is required.
- Experience in cybersecurity, finance, healthcare, or other high-stakes domains is a plus.
Skills:
- Proficiency in Python, with strong expertise in data manipulation libraries (Pandas, NumPy, Pytorch Geometric, Networkx, etc.).
- Strong understanding of graph databases, network analysis, and related algorithms.
- Expertise in Cypher query language for graph database operations.
- Experience with big data frameworks such as Hadoop and Spark.
- Knowledge of data lakes architecture for scalable data storage and retrieval.
- Experience with streaming frameworks like Apache Kafka or Apache Flink for real-time data processing.
Software Development & Engineering:
- Strong understanding of data structures, algorithms, and distributed computing concepts.
- Experience with API development and integration.
- Familiarity with version control (Git) and automated testing frameworks.
Preferred Qualifications:
- Experience in designing and deploying graph-based data solutions.
- Certifications in data engineering, big data technologies, or database management.
- Knowledge of cybersecurity data analysis and anomaly detection techniques.
Work Location: Jurong East
