Solution Architect

This role requires strong technical knowledge of enterprise data platforms, excellent stakeholder management skills, and proven experience delivering complex projects within the banking and financial services industry.

• 10+ years of experience.

• 5+ years managing Enterprise Data Lake, Data Warehouse or Big Data delivery initiatives.

• Proven experience in Hadoop/Cloudera environments.

• Experience managing multiple concurrent project deliveries and BAU portfolios.

• Banking and Financial Services experience preferred.

Data Engineering & Integration

. Design, develop, and maintain scalable data pipelines within the Enterprise Data Lake platform using Cloudera CDP, Hadoop, Spark, Hive, Java, Python, and SQL.

. Develop high-performance batch and near real-time data ingestion frameworks for onboarding enterprise source systems into the Data Lake.

. Develop complex transformation logic using Spark, HiveQL, SQL, and Java-based processing frameworks.

. Design and implement data ingestion patterns supporting structured, semi-structured, and unstructured datasets.

. Build and maintain batch and near real-time data integration solutions.

. Implement source-to-target mappings and ingestion specifications.

Hadoop & Cloudera Platform Development

. Develop and support solutions within the Cloudera Data Platform ecosystem.

. Build and maintain data pipelines leveraging:

. Hadoop HDFS

. Hive

. Impala

. Spark

. Kafka

. YARN, Oozie

. Perform deployment, troubleshooting, and performance tuning of applications running on the Hadoop ecosystem.

. Partner with platform teams to ensure optimal cluster utilization and workload management.

Data Quality & Governance

. Ensure adherence to Data Lake onboarding standards and governance requirements.

. Investigate and resolve data quality issues and production incidents.

. Ensure compliance with enterprise data governance standards.

DevOps & Platform Automation

. Integrate data engineering solutions into CI/CD pipelines using Git, Jenkins, Azure DevOps, or equivalent tools.

. Automate deployment, testing, monitoring, and operational support activities.

. Develop reusable frameworks and utilities to improve engineering productivity and platform scalability.

Production Support & Incident Management

. Provide L3 support for Data Lake application.

. Perform root cause analysis for production incidents and implement permanent fixes.

. Monitor pipeline execution, cluster health, and workload performance.

. Participate in release implementation, deployment validation, and post-production support activities.

. Strong hands on experience with Hadoop ecosystem and data pipeline orchestration.

. Hadoop/HDFS

. Hive

. Spark

. Unix/Linux scripting

. Shell scripting

. ETL/ELT development experience.

. Data modelling fundamentals.

. Performance tuning and optimization.

. Experience in Data Warehousing and Data Lake environments.

. Strong Java / Python competency with experiences.

. Strong SQL development skills.