Big Data Engineer

Platform Operations

. Operate and support CDP Private Cloud environments across Dev/Test/Prod.

. Perform BAU activities: monitoring, patching, upgrades, configuration tuning.

. Administer Cloudera Manager, alerts, health checks, and diagnostics.

. Ensure platform stability, availability, and SLA compliance.

Hadoop Ecosystem Support

. Support core services: HDFS, YARN, Hive, Impala, Spark, Kafka, NiFi, HBase.

. Manage HDFS quotas, snapshots, replication, and lifecycle policies.

. Support ingestion pipelines and troubleshoot platform-level issues.

Performance & Capacity

. Monitor and tune Hive/Impala queries and Spark workloads.

. Manage YARN queues and resource allocation.

. Perform capacity planning for compute and storage growth.

High Availability & Resilience

. Operate HDFS HA, Resource Manager HA, and Zookeeper.

. Support service failover, balancing, and resilience testing.

Security & Governance

. Administer Apache Ranger (RBAC, data masking).

. Integrate with LDAP/Active Directory.

. Enforce encryption (TLS, HDFS TDE) and audit logging.

. Support governance tooling (Atlas, lineage, audits).

. Ensure compliance with enterprise security standards (e.g. IM8, PDPA).

Monitoring, Automation & Incident Management

. Configure Cloudera Manager alerts and monitoring thresholds.

. Develop automation scripts (Shell, Python).

. Support log analysis (ELK, Splunk).

. Participate in 24x7 on-call, incident response, and RCA.

. Maintain runbooks and operational documentation.

Backup & Disaster Recovery

. Manage HDFS backup and replication (snapshots, DistCp).

. Support metadata backup and recovery.

. Participate in DR drills and recovery validation.

Required Skills & Qualifications

Education : Degree/Diploma in IT, Computer Science, or Engineering.
Experience : 2-5+ years experience administering Cloudera / Hadoop platforms.

EA License # 14C6941