Sr Data Engineer

As a Data Engineer at Huron, you must have knowledge of building and operating data pipelines. You will be building data pipeline solutions by designing, adopting, and applying big data strategies and architectures. You must have knowledge of (or the ability to learn quickly) scalable system implementations with a focus on complex data processing and analytics pipelines. You must demonstrate an understanding of data integration, data transformation, data modeling, and data cleansing. The Data Engineer must be able to demonstrate innovative approaches to complex problems which deliver industry-leading experiences for business stakeholders. We are looking for those with a strong computer science foundation and the ability to quickly get up to speed with technologies they may not be familiar with. The ability to learn and grow quickly is a must.

RESPONSIBILITIES

• Design and develop a logical data model for a large multi-tenant application with intent to facilitate consistency and efficiency.
• Collaborate with analysts and internal clients to understand the goals of the overall system data architecture, and further develop on business needs/issues.
• Test/troubleshoot problems and conduct root cause analysis.
• Own communications with technical contacts at software vendors to escalate software issues, recommend enhancements, and find sustainable workarounds to issues.
• Work in tandem with analysts and other stakeholders to develop and execute necessary processes and controls around the flow of data to meet data governance standards.
• Verify accuracy of table changes and data transformation processes
• Deliver fully tested code prior to prod-deployment when appropriate.
• Recommend and implement enhancements that address system performance requirements, streamline processes and improve data integrity.
• Create sound technical documentation and train peer developers on this documentation as development completes.
• Additional duties as assigned to ensure company success.

QUALIFICATIONS

• Strong data modeling experience (Data Vault / Star Schema)
• Must be a hands-on individual who is comfortable interfacing frequently with product stakeholders to understand business needs and translating those needs to technical engineers.
• Display sound problem-solving abilities in the face of challenges, with the ability to look at the big picture when designing solutions.
• Strong SQL, python skills
• Infrastructure-as-code exposure is a benefit (CloudFormation/Terraform)
• Deep understanding of Data Lake/Lakehouse principles and technologies (Athena/Presto/Hive/Glue Catalog)
• Ability to optimize performance through advanced modeling, query tuning, and infrastructure optimization techniques.
• Strong computer science fundamentals
• Familiarity or knowledge of Cloud ecosystems (AWS, Azure, or GCP).
• Familiarity or knowledge of Big Data technologies such as Spark and Presto/Trino.
• Experience in designing and implementing innovative data integration solutions, utilizing Python with Spark clusters.
• Familiarity with architectural patterns for data-intensive solutions
• Experience with batch-style data processing and familiarity with real time data processing
• Ability to learn and optimize use of new software as introduced by data architects.
• Familiarity with consulting and traditional application design
• Excellent written and verbal communication skills
• Benefits to have experience with Agile Methodology.
• Possess excellent interpersonal and organizational skills.
• Able to manage your own time and work well both independently and as part of a team.

TECHNOLOGIES WE USE

Cloud (AWS) / Data Warehouse (Redshift / Athena) / SQL/ dbt / Python / Data extraction and Integration Tools (AWS DMS, Glue/Lambda/ECS) /AWS SDK / Cube.js / Visualization Tools (AWS QuickSight) / DevOps Tools (Azure DevOps/AWS CDK/CloudFormation)

EDUCATION EXPERIENCE

• 4 or more years of experience
• B.E., B. Tech or MCA preferably in Computer Science, Engineering or Science/Technology-based discipline

Education

UG: B.Tech/B.E. in Production/Industrial

PG: MCA in Computers