Responsibilities:
- Design and manage streaming architecture
- Design, implement, and optimize event driven architectures using Apache Kafka
- Define topic structures, partition strategies, replication policies, and retention configurations
- Ensure scalability and high availability across clusters.
- Develop Kafka based applications
- Build producers, consumers, and stream processing applications using Java or Python.
- Implement real time data pipelines using Kafka Streams and Kafka Connect.
- Optimize message throughput, latency, and serialization formats such as Avro or JSON.
- Kafka cluster administration and performance tuning
- Monitor brokers, partitions, and consumer lag using observability tools.
- Perform capacity planning and cluster scaling.
- Handle fault tolerance, leader elections, and disaster recovery scenarios.
- Deploy and manage Kafka workloads on Kubernetes.
- Configure StatefulSets, persistent storage, and autoscaling policies.
- Maintain secure container environments and optimize resource allocation.
- Design and manage API exposure through API gateways such as Kong, Apigee, or Gravitee.io.
- Implement authentication, rate limiting, routing, and API lifecycle governance.
- Integrate event driven services with REST and microservices ecosystems.
Requirement:
- Minimum 8 years of working experience , and 3 years of banking/fintech exposure is good to have
- Familiarity with monitoring tools (e.g., Grafana, Kafka Manager) and logging frameworks to ensure system performance and reliability.
- In-depth knowledge of Apache Kafka architecture, including topics, brokers, producers, and consumers.
- Proficiency in Kafka APIs, such as Producer and Consumer API, Kafka Streams API, and Kafka Connect API.
- Strong skills in one of languages such as Java / Python for developing Kafka applications.
- Familiarity with scripting languages like Bash, PowerShell for automation tasks.
- Possesses in-depth knowledge of Kubernetes
- Experience with any one of API Gateway technologies (e.g., Kong, Apigee, Gravitee) for managing and securing APIs.
- Strong scripting and automation skills for managing Kafka infrastructure and deployments
- Ability to document processes, configurations, and findings clearly for future reference and knowledge sharing
EA Reg. No. 25C2690 | EA License No. R1330510
