Job Description:
- oin a high impact engineering team where you ll build and optimize large scale data processing solutions using Python Apache Spark and Scala
- This role is ideal for someone who enjoys solving complex data challenges improving performance and delivering reliable pipelines that power analytics and downstream applications
- You ll collaborate closely with data engineers platform teams and stakeholders to translate business needs into scalable implementations while promoting clean code testing discipline and continuous improvement
- If you re motivated by ownership enjoy mentoring others and want to work in a culture that values collaboration learning and measurable outcomes this opportunity offers the space to make a visible difference both through technical contributions and by raising the bar for engineering excellence
Key Responsibilities:
- Data Engineering Development
- Design develop and maintain scalable batch stream data pipelines using Spark with Scala and Python
- Implement efficient transformations joins aggregations and data quality checks for large datasets
- Build reusable frameworks utilities to standardize pipeline patterns and reduce delivery time
- Performance Reliability
- Tune Spark jobs partitioning caching shuffles memory executor settings to improve performance and cost efficiency
- Troubleshoot production issues perform root cause analysis and implement preventive fixes
- Ensure reliability through robust logging monitoring hooks and failure handling strategies
- Collaboration Delivery
- Work with stakeholders to refine requirements and deliver well documented production ready solutions
- Conduct code reviews enforce best practices and mentor team members on Spark Scala Python patterns
- Contribute to CI CD friendly development practices including testing version control and release readiness
Technical Requirements:
- Technology Analytics Packages Python Big Data Technology Big Data Data Processing Spark Technology Big Data Data Processing Scala
Additional Responsibilities:
- Bachelor s degree in Computer Science Engineering or a related field or equivalent practical experience
- 5 9 years of experience in data engineering or backend development with strong hands on delivery ownership
- Strong proficiency in Python and Apache Spark for large scale data processing
- Solid experience with Scala for Spark based development and production grade implementations
- Working knowledge of Hive and SQL based data querying processing concepts
Preferred Skills:
Technology->Big Data - Data Processing->Spark,Technology->Java->Apache->Scala,Technology->Analytics - Packages->Python - Big Data
