IT Storage Engineer (HDFS)

Key Responsibilities:
- Design, deploy, and maintain HDFS clusters to support enterprise-scale data applications.
- Monitor cluster performance, manage storage capacity, and ensure high availability and fault tolerance.
- Implement data security, access controls, and encryption for HDFS data.
- Troubleshoot and resolve issues related to HDFS, including data node failures, replication issues, and performance bottlenecks.
- Manage data ingestion pipelines and optimize data storage formats (e.g., Parquet, Avro).
- Support and work with data engineering and analytics teams to ensure reliable data delivery and transformation workflows.
- Automate cluster operations using scripting (e.g., Bash, Python) and orchestration tools.
- Conduct upgrades and patching of Hadoop ecosystem components (HDFS, YARN, Hive, etc.).
- Maintain documentation of architecture, configurations, and best practices.
- Ensure compliance with data governance and data privacy policies.

Qualifications:
Bachelor's degree in Computer Science, Information Systems, or a related field.
3-5+ years of experience with Hadoop ecosystem, particularly HDFS administration.
Strong understanding of HDFS architecture, replication, and fault tolerance.
Experience with Cloudera, Hortonworks, or Apache Hadoop distributions.
Proficiency in Linux/Unix system administration and scripting (Bash, Python, etc.).
Familiarity with related components: YARN, Hive, HBase, Spark, Oozie, and Zookeeper.
Experience with monitoring tools like Ambari, Cloudera Manager, or Nagios.

Preferred:
Hadoop certification (e.g., Cloudera Certified Administrator for Apache Hadoop - CCAH).
Knowledge of cloud-based big data platforms (AWS EMR, Azure HDInsight, GCP Dataproc).
Experience with containerization (Docker/Kubernetes) for big data workloads.
Exposure to data lake architectures and data governance tools.

Start Date: Feb/March 2026 for 6 months