Design, develop, and maintain robust data pipelines that extract, transform, and load data from diverse sources into target systems.
Perform data extraction, cleaning, transformation, and flow management to ensure data accuracy and consistency.
Build, launch, and maintain large-scale batch and real-time data pipelines using modern data processing frameworks.
Integrate and consolidate data across silos in a scalable, secure, and compliant manner.
Work in an Agile environment, adhering to continuous integration and delivery practices.
Requirements
Bachelor's degree in Computer Science, Software Engineering, or a related field.
3-5 years of experience in ETL/data integration projects.
Strong skills in data cleaning and transformation using scripting languages (mandatory: SQL, Python additional advantage: R).
Hands-on experience building ETL pipelines using tools such as SQL Server Integration Services (SSIS), Python, and Snowflake.
Experience with cloud platforms such as AWS or Azure and big data environments.
knowledge of AWS Lambda, ECS, EventBridge, AWS Glue, or Spring is a plus.
Proficient in database design and management, including SQL, AWS S3, RDS familiarity with PostgreSQL, Athena, MongoDB, MySQL, SQLite, VoltDB, Apache Cassandra is a plus.
Familiarity with CI/CD pipelines, DevOps practices, and automation tools (Ansible, Terraform, Puppet, Vagrant) is advantageous.
Knowledge of data modeling, storage infrastructure (Data Marts, Data Lakes, Data Warehouses, Data Virtualization), and efficient data retrieval.
Understanding of REST APIs, web protocols, and general system integration.
Comfortable working in both Windows and Linux development environments.
