Data Engineer- Senior II

Designs, develops, and maintains scalable data pipelines and systems to support analytics, machine learning, and business intelligence within the logistics domain. Collaborates with cross-functional teams to gather data requirements and deliver reliable, high-quality technical solutions. Responsibilities include optimizing data infrastructure, ensuring data integrity, and proactively monitoring pipelines to resolve issues efficiently. Also provides mentorship to junior engineers and provides technical excellence across all phases of project development, from proof of concept to full-scale production.

Job Description

Roles & Responsibilities

• Data Engineering:
  Design, develop, and maintain robust ETL (Extract, Transform, Load) processes to integrate data from diverse sources.

• System Architecture for Analytics:
  Build scalable data systems that support analytics, machine learning, and business intelligence use cases within the logistics domain, in close collaboration with solution architects.

• Cross-Functional Collaboration:
  Work closely with data analysts, product owners, architects, and other stakeholders to understand data requirements and deliver effective technical solutions.

• System Optimization:
  Continuously enhance existing data infrastructure to improve performance, scalability, and reliability.

• Data Quality Assurance:
  Implement validation and monitoring processes to ensure data accuracy, consistency, and integrity across systems.

• Pipeline Monitoring & Issue Resolution:
  Proactively monitor data pipelines and address issues promptly to minimize disruptions and ensure smooth operations.

• Mentorship & Team Development:
  Support the growth of junior data engineers through code reviews, debugging assistance, and career development guidance.

• Project Participation:
  Contribute to projects across all stages-from proof of concept (POC) and minimum viable product (MVP) to full-scale production-within teams ranging from small groups to large cross-functional units

Experience with the following preferred:

• Python Programming & Software Engineering

• Data Warehousing & Data Modeling

• SQL and relational databases

• Concurrency and Multithreading

• Cloud and Azure

• CI/CD

• Version Control