Data Software Engineer, you'll be part of a team who are passionate about learning and prototyping cutting-edge technologies and convergence initiatives with latest Big Data services on Spark, Scala, Databricks, PySpark and leverage GitHub to support things like CI/CD and integrations
- 6-10 years of experience as a Data engineer who works on enterprise standard frameworks like Spark, Scala, Databricks, PySpark.
- Development experience in unit and integration test cases
- Intermediate level of Database (SQL) skills to develop SQL queries, functions
- Good Understanding on CI/CD Pipeline i.e. GitHub Actions
- Strong knowledge of version control tools, preferably GitHub
- Basic Knowledge on Linux/Unix environment (basic commands, shell scripting, etc.)
- Demonstrated ability to thrive in an enterprise Agile/SCRUM environment
- Implement data governance practices, data lineage, and metadata management to ensure data accuracy, traceability, and compliance
- Monitor and optimize data pipeline performance, troubleshoot issues, and implement necessary enhancements
- Implement monitoring and logging mechanisms to ensure the health, availability, and performance of the data infrastructure
- Experience using Collaboration Technologies: GitHub, Jira, Confluence
- Experience using Object-oriented languages Java
- Experience working with testing tools and Automation test needs
