Cloud Data Engineer

Job Description
The Cloud Data Engineer is responsible for designing, building, and maintaining cloud-based data systems. The role involves developing data pipelines, managing data storage solutions, and ensuring data quality, security, and performance using AWS, Databricks, and Informatica IDMC.

Main Duties

• Design and maintain data storage solutions such as data lakes, databases, and data warehouses using AWS and Databricks.

• Develop and manage data pipelines for data ingestion, transformation, and processing.

• Integrate data from multiple internal and external data sources.

• Build and maintain ETL/ELT processes to prepare data for analytics and reporting.

• Monitor and improve performance of data pipelines and queries.

• Ensure data security, access control, and compliance with data protection standards.

• Support data governance and metadata management activities.

• Automate routine data processing and monitoring tasks.

• Maintain documentation for data systems and workflows.

• Work with cross-functional teams to understand data requirements and deliver solutions.

• Identify and resolve data-related issues.

• Optimize cloud resource usage to control costs and improve efficiency.

Employment Requirements

• Bachelor's degree in Computer Science, Information Technology, Data Engineering, or a related field, or equivalent work experience.

• At least 5 years of experience in data engineering or a related role.

• Experience with AWS cloud services.

• Experience with Databricks and/or Informatica IDMC.

• Programming experience in Python, Java, or Scala.

• Strong knowledge of SQL and experience with relational and NoSQL databases.

• Experience with data modeling and data integration.

• Strong problem-solving and communication skills.

Assets (Preferred but Not Required)

• Experience with Apache Spark or Hadoop.

• Knowledge of Docker or Kubernetes.

• Experience with CI/CD pipelines and version control tools such as Git.

• Experience with data visualization tools (e.g., Tableau, Power BI).

• Relevant cloud or data engineering certifications.