Job Description:
- Join a fast paced data engineering team where you ll build and optimize large scale data processing solutions that power analytics and decision making across the business
- In this role you ll use Spark and Scala to design reliable high performance pipelines collaborating closely with data engineers analysts and platform teams to deliver clean trusted datasets
- You ll work on challenging problems like performance tuning handling complex transformations and ensuring data quality at scale while contributing to a culture that values ownership continuous improvement and knowledge sharing
- If you enjoy turning raw data into well structured production ready assets and want to make a measurable impact through scalable engineering this is a great opportunity to grow and lead through hands on delivery
Key Responsibilities:
- Data Engineering Development
- Design develop and maintain Spark based batch processing pipelines using Scala for large datasets
- Implement efficient transformations aggregations and joins ensuring correctness and scalability
- Write optimized SQL for data extraction validation and reconciliation across sources and targets
- Performance Quality Reliability
- Tune Spark jobs partitioning caching shuffles memory executor settings to improve runtime and cost efficiency
- Build data quality checks and validations to ensure accuracy completeness and consistency of outputs
- Troubleshoot production issues perform root cause analysis and implement preventive fixes
- Collaboration Delivery
- Work with stakeholders to understand data requirements and translate them into technical solutions
- Participate in code reviews follow engineering best practices and contribute to reusable components
- Document pipelines logic and operational runbooks for maintainability and onboarding
Technical Requirements:
- Technology Big Data Data Processing Spark Technology Java Apache Scala
Additional Responsibilities:
- Bachelor s degree in Computer Science Engineering or a related field or equivalent practical experience
- 5 9 years of overall experience with strong hands on development in Spark and Scala
- Solid experience writing and optimizing SQL for analytics and data processing use cases
- Strong understanding of distributed processing concepts data transformations and performance considerations
- Ability to debug and resolve issues in data pipelines with a focus on reliability and quality
Preferred Skills:
Technology->Big Data - Data Processing->Spark,Technology->Java->Apache->Scala
