Senior Data Engineer
Job Description & Requirements
We are seeking a highly skilled Senior Data Engineer to join our growing team of analytics experts and take on the additional responsibilities of a Team Lead. As a Senior Data Engineer, you will be responsible for building and optimizing data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up to ensure optimal data delivery architecture is consistent throughout ongoing projects. They must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. In addition, the candidate will lead a team of data engineers and collaborate with cross-functional teams to deliver high-quality data solutions.
Data Analyst / Engineer Job Responsibilities:
Lead a team of data engineers, providing guidance, mentorship, and support to achieve project objectives and professional growth.
- Work closely with stakeholders to understand their data needs and translate them into technical requirements.
- Lead the adoption of agile methodologies and facilitate effective communication and collaboration within the team and across departments.
- Manage project timelines, resource allocation, and priorities to meet deadlines and deliverables.
- Develops and maintains scalable data pipelines and builds out new API integrations to support continuing increases in data volume and complexity.
- Building required infrastructure for optimal extraction, transformation and loading of data from various data sources using AWS and SQL technologies
- Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it.
- Writes unit/integration tests, contributes to engineering wiki, and documents work.
- Performs data analysis required to troubleshoot data related issues and assist in the resolution of data issues.
- Works closely with a team of frontend and backend engineers, product managers, and analysts.
- Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition
- Designs data integrations and data quality framework.
- Designs and evaluates open source and vendor tools for data lineage.
- Works closely with all business units and engineering teams to develop strategy for long term data platform architecture.
Data Analyst / Engineer Qualifications / Skills:
- Building analytical tools to utilize the data pipeline, providing actionable insight into key business performance metrics including operational efficiency and customer acquisition
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
- A successful history of manipulating, processing and extracting value from large disconnected datasets.
- Working knowledge of message queuing, stream processing, and highly scalable big data data stores.
- Experience with or knowledge of Agile Software Development methodologies
- Excellent problem solving and troubleshooting skills
- Process oriented with great documentation skills
- Excellent oral and written communication skills with a keen sense of customer service
Education, Experience, and Licensing Requirements:
- BS Computer Science or a related technical field.
- Experience with object-oriented/object function scripting languages: Python, Java,C++, Scala, etc.
- Advanced work SQL knowledge or experience (No-SQL experience is a plus).
- Experience with schema design and dimensional data modelling.
- Experience working with either a Map Reduce or an MPP system on any size/scale.
- Experience designing, building, and maintaining data processing systems.
- In-depth knowledge of data modeling, data warehousing, and ETL principles.
- Demonstrated experience with cloud-based data platforms (e.g., AWS, Azure) and associated services (e.g., S3, Redshift).
- Familiarity with data visualization and business intelligence tools (e.g., Power BI) is a plus.
- Excellent problem-solving skills and the abilit to work in a fast-paced, dynamic environment.
- Strong leadership abilities, including the ability to motivate and inspire a team, foster collaboration, and resolve conflicts.
- Excellent communication and interpersonal skills, with the ability to convey complex technical concepts to non-technical stakeholders.
