Cluster Management

Install, configure, and maintain Hadoop clusters (Cloudera, Hortonworks, or Apache Hadoop)

Manage core components including HDFS, YARN, MapReduce

Perform cluster scaling, upgrades, and patching

Administration & Monitoring

Monitor cluster performance, health, and usage

Troubleshoot and resolve cluster issues, failures, and bottlenecks

Implement alerting and monitoring tools (Ambari, Cloudera Manager, Grafana)

Security & Governance

Configure and manage Kerberos authentication

Implement data security policies using Ranger / Sentry

Handle user access controls and permissions

Data Management

Manage data storage, replication, and backup strategies

Optimize HDFS performance and storage utilization

Handle data ingestion and integration workflows

Performance Optimization

Tune Hadoop cluster performance (memory, CPU, storage)

Optimize YARN resource allocation and scheduling

Identify and resolve performance issues in jobs

Integration & Support

Support tools in Hadoop ecosystem:

Hive

Spark

HBase

Sqoop / Flume / Kafka

Work closely with data engineers and developers

Required Skills & Qualifications

Core Technical Skills

3–6 years of experience in Hadoop Administration

Strong knowledge of:

HDFS, YARN, MapReduce

Hadoop ecosystem tools

Experience with Linux system administration (mandatory)

Big Data Ecosystem

Familiarity with:

Hive, Spark, HBase

Kafka, Sqoop, Flume

Understanding of distributed computing concepts

Tools & Platforms

Experience with:

Cloudera Manager / Ambari

Monitoring tools (Nagios, Prometheus, Grafana)

Basic scripting in Shell / Python

• Primary skills:Technology->Big Data - Hadoop->Hadoop Administration

Experience with cloud-based Hadoop (AWS EMR, Azure HDInsight, GCP Dataproc)

Knowledge of DevOps practices (CI/CD, Docker, Kubernetes)

Experience in high availability (HA) and disaster recovery (DR) setups