The Engineer, Data should have a proven track record of delivering data pipeline solutions and architecture. He/She should also understand business requirement and able to build reliable data infrastructure using big data technologies. Ideally, you are someone who enjoys optimizing data pipeline, automating and building from scratch.
Job Responsibilities
- Architect, develop, and deploy end-to-end internal automation solutions and software tools, leveraging AI, web scraping, and custom logic to create impactful data products
- Take full ownership of the data engineering lifecycle, from conceptualization and data modeling to the implementation, CI/CD, and operational management of data systems
- Design, build, and maintain robust, scalable data infrastructure, including real-time and batch ETL/ELT data pipelines, using orchestration tools like Apache Airflow
- Develop and manage the organization's data lake and data warehouse, implement sophisticated data models and applying modern data transformation practices with tools like dbt
- Automate and optimize data processes by integrating disparate systems, APIs, and third-party services to ensure seamless data flow and synchronization
- Establish and enforce best practices for data governance, quality, and lineage, implementing automated monitoring and alerting to ensure system reliability
- Collaborate with cross-functional teams to translate business requirements into technical specifications and production-ready software and data solutions
Job Requirements
- Minimum Bachelor's degree in Computer Science, Computer Engineering, Software Engineering, or equivalent is preferred
- At least 5 years of relevant tech, app development or data engineering working experience
- Deep understanding of software engineering principles, system architecture, and object-oriented design, with proven experience building production-grade applications
- Expert proficiency in Python, including experience with data manipulation libraries (e.g., Pandas, NumPy), and frameworks for web scraping (e.g., Scrapy) or AI/ML (e.g., TensorFlow, PyTorch)
- Proficiency with workflow orchestration tools like Apache Airflow for scheduling and managing complex data pipelines
- Experience with data transformation tools like dbt, and a strong understanding of modern ETL/ELT design patterns and principles
- Strong experience in data modeling techniques (conceptual, logical, physical) and designing scalable data warehouse schemas
- Extensive hands-on experience with a major cloud platform (AWS, GCP, or Azure), including its native data services, compute, and storage solutions
- Strong command of DevOps practices, including building and managing CI/CD pipelines and using infrastructure as code (IaC) tools like Terraform or CloudFormation
- Advanced SQL skills and experience designing and optimizing relational databases (e.g., PostgreSQL, Oracle, SQL Server) and data warehouse schemas
- Practical experience with containerization and orchestration technologies such as Docker and Kubernetes for deploying and scaling applications
- Experience in building AI agents or developing solutions using Generative AI models and frameworks (e.g., LangChain, LlamaIndex) is a significant plus
- Excellent ability to translate complex technical concepts into clear business context for stakeholders at all levels
- Strong ownership mindset with the ability to work independently and manage projects from inception to completion
