We are seeking a Data Engineer with strong experience in AWS, Databricks, and Informatica to design and build scalable data pipelines within a cloud-based analytics environment. The role will focus on data integration, transformation, and enabling advanced analytics by building reliable data infrastructure.
Key Responsibilities
- Design and develop scalable ETL/ELT pipelines for processing large datasets.
- Build and maintain data pipelines using AWS and Databricks.
- Develop data integration workflows using Informatica.
- Implement data ingestion and transformation processes from multiple data sources.
- Optimize data pipelines for performance, scalability, and reliability.
- Support data lake and data warehouse architectures.
- Collaborate with data scientists, AI engineers, and analytics teams to enable advanced analytics use cases.
- Ensure data quality, governance, and security standards are maintained.
Required Skills
- Strong experience in AWS (S3, Glue, Redshift, EMR, Lambda or similar services).
- Hands-on experience with Databricks and Apache Spark.
- Experience developing ETL workflows using Informatica.
- Strong proficiency in SQL and Python.
- Experience working with large-scale data processing and data lake architectures.
- Knowledge of data modeling and data pipeline optimization.
Preferred Skills
- Experience with data warehousing and analytics platforms.
- Familiarity with workflow orchestration tools such as Airflow.
- Experience supporting machine learning or analytics pipelines.
- Understanding of data governance and security frameworks.
