Big Data Platform Administrator - Apache Spark & Airflow… - Jobeax
Vacancy description
Big Data Platform Administrator - Apache Spark & Airflow in Chennai, भारत
Samridhi Placements
Chennai, भारत
Job Description :Key Responsibilities :- Administer and support Hadoop ecosystem components (HDFS, YARN) across Production, Test, and Development environments- Manage and maintain Apache Spark and Airflow infrastructure- Monitor cluster health, troubleshoot performance issues, and optimize system performance- Implement and maintain High Availability (HA), Disaster Recovery (DR), backup, and restore strategies- Manage logging, monitoring, and alerting systems using Prometheus, Grafana, and ELK Stack- Perform Linux (Ubuntu) system administration, including patching, upgrades, and security hardening- Develop and maintain automation scripts using Shell and/or Python- Handle Kerberos authentication setup and maintenance- Support incident management, root cause analysis (RCA), and ensure SLA adherence- Manage capacity planning and system scalability- Follow CI/CD and DevOps best practices for platform improvements and deploymentsRequired Skills :- Strong experience with Hadoop ecosystem (HDFS, YARN)- Handson expertise in Apache Spark and Apache Airflow- Proficiency in Linux system administration (Ubuntu), including security hardening- Scripting skills in Shell and Python for automation- Familiarity with monitoring and observability tools: Prometheus, Grafana, ELK Stack- Knowledge of High Availability, Disaster Recovery, backup, and restore concepts- Experience with Kerberos authentication setup and maintenance (ref:hirist.tech)
... Professional Skills : - Experience : 8 to 10 years in software engineering and big data development, including experience leading client-facing delivery teams.- Big Data Stack : Strong hands-on experience with Hadoop (HDFS, MapReduce, YARN), Apache Spark (Core, SQL, Streaming), Apache Kafka, and HBase.- Architecture : Deep ...
... https://jobeax.com/link/wJIgo45kO3tINlTo Skills & Experience :- 6 - 11 years of experience in Big Data, Data Engineering, or a related technology domain.- Strong hands-on expertise in Apache Spark and Scala.- Strong understanding of Big Data concepts, distributed computing, and parallel data processing.- Experience developing large-scale data processing, ...
... https://jobeax.com/link/wJIgo45kO3tINlTo Skills & Experience :- 6 - 11 years of experience in Big Data, Data Engineering, or a related technology domain.- Strong hands-on expertise in Apache Spark and Scala.- Strong understanding of Big Data concepts, distributed computing, and parallel data processing.- Experience developing large-scale data processing, ...
... Hadoop ecosystem technologies for distributed data processing and storage.- Integrate data from multiple sources including databases, APIs, files, and streaming platforms.- Build and manage data orchestration workflows using Azure Data Factory (ADF).- Optimize Spark jobs, Databricks clusters, and data pipelines for performance ...
WEKA Data Platform Administrator About the job WEKA is architecting a new approach to the enterprise data stack built for the age of reasoning. NeuralMesh by WEKA sets the standard for agentic AI data infrastructure with a cloud and AI-native software solution that can be deployed anywhere. It transforms legacy data silos ...
Big Data Executive Start Date Starts Immediately CTC (ANNUAL) ₹ 2,00,000 - 2,20,000 ₹ 2,00,000 - 2,20,000 /year Experience 1 year(s) 1 year(s) Apply By 13 Nov' 26 Posted 7 days ago Job 28 applicants About the job Key responsibilities: 1. Enter and maintain accurate client and company information in internal systems. 2. ...
... https://jobeax.com/link/wJIgo45kO3tINlTo Skills & Experience :- 5+ years of experience with AWS Services including RDS, AWS Lambda, AWS Glue, EMR, S3, and related big data technologies.- 5+ years of experience working with Apache Spark, Spark Streaming, Kafka, and Hive.- Strong experience in SQL and NoSQL databases (MySQL, ...
... scalable, reliable, high-performance data pipelines and data platforms.EDUCATION:Bachelor's degree in computer science, Software Engineering, MIS or equivalent combination of education and https://jobeax.com/link/NfKu17xn3PQ6gnFL SKILLS:Python, PySpark, Hadoop, Spark, Hive, Data Engineering, Big Data, SQL (ref:hirist.tech) []
... containerized environments.- Troubleshoot production issues and continuously improve platform https://jobeax.com/link/AMuFfOH9R4FgRv8m : - 5+ years of experience in Data Engineering or Big Data Engineering.- Strong hands-on experience with Apache Spark and Scala.- Experience building and maintaining ETL pipelines.- Familiarity ...