Hadoop Administrator and DevOps Engineer with 4 years of expertise in cluster configuration and CI/CD automation. Achievements include leading the development of a MAPR Hadoop POC cluster, which enhanced system reliability and minimized downtime. Automated CI/CD pipelines in GitLab using YAML and Bash/Shell scripting, resulting in a 25% reduction in manual tasks and a 20% increase in productivity. Proficient in Python, Bash, Git, GitHub Actions, and monitoring tools to optimize operational efficiency.
Constructed MAPR Hadoop POC cluster and reconfigured environments for diverse testing and validation.
Automated CI/CD pipelines in GitLab using YAML configurations and Shell scripting, streamlining deployment workflows.
Developed Python and Bash automation tools, reducing manual tasks by 25%.
Installed, upgraded, and configured large clusters across data centers with effective backup/restore strategies.
Enabled high availability for core services including Warden, Zookeeper, Hive, and Spark.
Resolved over 150 ticketed incidents, leading remediation efforts for cluster vulnerabilities.
Monitored cluster dashboards and conducted health checks to minimize downtime.
Managed volumes, mirrors, and snapshots while addressing mirror failures and disaster recovery issues.
DevOps and CI/CD: GitLab CI/CD, GitHub Actions, YAML configurations, automated deployments
Automation: Python and Bash scripting, task automation
Big Data technologies: Hadoop, Spark, Hive
Version control: Git and GitHub
Containerization: Docker workflows
Monitoring tools: Icinga, Splunk, ServiceNow
Database management: SQL and disaster recovery strategies
- Languages & Tools: Python, Bash, SQL, Git, GitLab, GitHub, Docker
- Big Data: Hadoop, HDFS, YARN, MapReduce, Hive, Hive Metastore, Spark, Tez, Oozie, HBase, Pig
- DevOps & Automation: GitLab CI/CD, GitHub Actions, YAML workflows, caching/matrix builds, approval
gates
- Monitoring & Ops: Icinga, Splunk, ServiceNow, Linux system administrationapplicable), Icinga, Splunk, ServiceNow, MCS, Linux administration, basic Python, SQL