Position Overview

We are looking for an experienced Data Engineer with deep hands?on expertise in Cloudera (CDH/CDP) and Apache NiFi to design, build, and operate scalable data pipelines within a regulated banking environment. The ideal candidate understands distributed data processing, real?time ingestion patterns, and secure data handling consistent with banking compliance standards.

The role will collaborate closely with Data Architecture, Platform Engineering, Application Teams, and Information Security to ensure reliable, high?performance data movement across the enterprise.

Key Responsibilities

Design, build, and maintain end?to?end data ingestion and transformation pipelines using: Apache NiFi for flow orchestration, Spark (batch/streaming) for transformation, Hive/Impala for processing and analytics.
Implement real-time and batch ingestion patterns using Kafka, NiFi, Sqoop, APIs, and file-based ingestion.
Develop reusable ingestion templates, processors, and flow patterns for consistent onboarding.
Work within Cloudera CDH/CDP environments, leveraging services such as HDFS, YARN, Hive, Impala, Ranger, Kafka, Oozie, and HBase.
Integrate pipelines with data governance tools, metadata catalogues, and secure storage zones.
Collaborate with Cloudera administrators to optimize service-level performance for data workloads.
Implement data quality checks, schema validation, and reconciliation logic.
Optimize Spark jobs, SQL queries, and NiFi flows for performance and cost efficiency.
Troubleshoot bottlenecks across compute, storage, and network layers.
Ensure adherence to banking regulatory standards (e.g., MAS TRM, GDPR, PCI-DSS, internal audit policies).
Implement strong security patterns: Kerberos, Ranger policies, encryption, tokenization, masking.
Maintain data lineage, audit systems, and access control logs for governance and compliance.
Develop monitoring dashboards for NiFi pipelines and Cloudera services.
Write automation scripts using Python, Shell, or Ansible.
Support release cycles, CAB approvals, deployments, and production support during incidents.

Qualifications

Bachelors/Masters in Computer Science, Engineering, or related discipline.
3-8 years of experience as a Data Engineer in large-scale distributed data environments.
Strong hands-on experience with:
Apache NiFi (flow design, processors, controllers, templates, versioning)
Cloudera CDH/CDP ecosystem
Spark (PySpark or Scala), Hive, Impala
Kafka (producers, consumers, schemas)
Experience with HDFS, distributed computing, and data modelling concepts.
Strong programming skills: Python, Scala, SQL, Shell scripting.

Preferred Banking Domain Experience

Data engineering experience supporting:
Payments
Customer 360 platforms
Regulatory reporting
Familiarity with:
Banking-grade network segmentation (DMZ, SDZ, HDZ)
Secure API and file transfer patterns
Strict SDLC processes (CAB approvals, ITIL operations)

Soft Skills

Strong problem-solving skills with the ability to troubleshoot end?to?end pipeline issues.
Ability to work in cross-functional teams and communicate technical concepts clearly.
Detail-oriented, structured, and comfortable working under regulatory pressure.

Certifications (Nice to Have)

Cloudera Data Platform Data Engineer Certification.
CCA Spark and Hadoop Developer (CCP/CCDH).
NiFi or data integration certifications.
Cloud certifications (AWS/GCP/Azure Big Data).

Data Engineer - Banking

Senior Data Engineer - Banking Domain