Role: Data Engineer
Work Location: Bangalore (Hybrid)
Hire Type: Contract/FTE
Years of experience: 4-8 years
Shift Timing: IST Hours with some overlapping
Mandatory Skills
Scala/Python (Pyspark), Big Data Knowledge, Scripting (Shell/Python), GCP Data Tech Stack, Airflow, CICD, Kafka, SQL/NoSQL
What You'll Do:
- Develop data pipelines/data frameworks/applications/apis using industry best practices.
- Drive Proof of Concept (POC) and Proof of Technology(POT) evaluations.
- Build data system and data quality monitoring process and ensure health of the big data system.
- Review KPIs, Monitoring and Alarms for successful production system.
- Work with the stakeholders to understand the business & reporting needs – Propose, design and develop creative data solutions that provides required insight, improve/automate processes and adds value.
- Influence and educate the team with your experience, idea and learning.
- Stay current with the latest development tools, technology ideas, patterns and methodologies; share knowledge by clearly articulating results and ideas to key stakeholders.
- Publish and update technical architecture and user/process documentation.
What you'll bring:
- Bachelor's degree in Computer Science or related technical field.
- 5+ years of relevant experience on Scala/Python (PySpark), Distributed Databases, Kafka with solid hands-on multi-threading, functional programing etc.
- A good understanding of CS Fundamentals, Data Structures, Algorithms and Problem Solving.
- Professional hand-on experience in Sql and Query Optimization.
- Experience in building frameworks for data ingestions and consumptions patterns.
- Experience with Orchestration tools like Airflow ( preferred), Automic , Autosys etc.
- Hand-on experience in Data processing and Data manipulation.
- Expertise with GCP cloud and GCP data processing tools, platforms and technologies like GCS, DataProc, DPaaS, BigQuery, Hive etc.
- Experience with Streaming data via Kafka or structured streaming etc.
- Exposure to lambda Architecture.
- Excellent communication skills for collaborating with teams.
- Appreciate productivity and care deeply about helping others work more effectively and efficiently.
If interested, please find your resume on [Confidential Information]
