Big Data Platform Administrator - Apache Spark & Airflow in Chennai, भारत
Samridhi Placements
Chennai, भारत
Job Description :Key Responsibilities :- Administer and support Hadoop ecosystem components (HDFS, YARN) across Production, Test, and Development environments- Manage and maintain Apache Spark and Airflow infrastructure- Monitor cluster health, troubleshoot performance issues, and optimize system performance- Implement and maintain High Availability (HA), Disaster Recovery (DR), backup, and restore strategies- Manage logging, monitoring, and alerting systems using Prometheus, Grafana, and ELK Stack- Perform Linux (Ubuntu) system administration, including patching, upgrades, and security hardening- Develop and maintain automation scripts using Shell and/or Python- Handle Kerberos authentication setup and maintenance- Support incident management, root cause analysis (RCA), and ensure SLA adherence- Manage capacity planning and system scalability- Follow CI/CD and DevOps best practices for platform improvements and deploymentsRequired Skills :- Strong experience with Hadoop ecosystem (HDFS, YARN)- Handson expertise in Apache Spark and Apache Airflow- Proficiency in Linux system administration (Ubuntu), including security hardening- Scripting skills in Shell and Python for automation- Familiarity with monitoring and observability tools: Prometheus, Grafana, ELK Stack- Knowledge of High Availability, Disaster Recovery, backup, and restore concepts- Experience with Kerberos authentication setup and maintenance (ref:hirist.tech)
... Professional Skills : - Experience : 8 to 10 years in software engineering and big data development, including experience leading client-facing delivery teams.- Big Data Stack : Strong hands-on experience with Hadoop (HDFS, MapReduce, YARN), Apache Spark (Core, SQL, Streaming), Apache Kafka, and HBase.- Architecture : Deep ...
... to support data platform requirements.- Maintain technical documentation and follow data engineering best https://jobeax.com/link/wJIgo45kO3tINlTo Skills :- Apache Spark, PySpark, Hadoop, SQL, Python/Scala/Java, ETL, Big Data ProcessingSupported Skills :- Kafka, Hive, HDFS, Airflow, AWS/Azure/GCP, Data Pipeline Optimization, ...
... to support data platform requirements.- Maintain technical documentation and follow data engineering best https://jobeax.com/link/wJIgo45kO3tINlTo Skills :- Apache Spark, PySpark, Hadoop, SQL, Python/Scala/Java, ETL, Big Data ProcessingSupported Skills :- Kafka, Hive, HDFS, Airflow, AWS/Azure/GCP, Data Pipeline Optimization, ...
... to support data platform requirements.- Maintain technical documentation and follow data engineering best https://jobeax.com/link/wJIgo45kO3tINlTo Skills :- Apache Spark, PySpark, Hadoop, SQL, Python/Scala/Java, ETL, Big Data ProcessingSupported Skills :- Kafka, Hive, HDFS, Airflow, AWS/Azure/GCP, Data Pipeline Optimization, ...
... https://jobeax.com/link/wJIgo45kO3tINlTo Skills & Experience :- 6 - 11 years of experience in Big Data, Data Engineering, or a related technology domain.- Strong hands-on expertise in Apache Spark and Scala.- Strong understanding of Big Data concepts, distributed computing, and parallel data processing.- Experience developing large-scale data processing, ...
... and optimize scalable big data solutions in an AWS-based https://jobeax.com/link/EG3PG5FSTAqKZnRA Responsibilities : - Design, develop, and maintain scalable big data solutions and data pipelines on AWS.- Build and optimize data processing jobs using Apache Spark with PySpark or Scala.- Develop and manage ETL workflows ...
VAST Storage AdministratorExperience : 6+ YearsLocation : Hyderabad / PuneJob Summary :We are looking for an experienced VAST Storage Administrator with strong hands-on experience in the VAST Data Platform, VAST storage administration, NAS/SAN storage, and VAST REST https://jobeax.com/link/u5l3iDAdRIPcmNG1 Responsibilities ...
... interactive visualizations etc) which can offer ways to improve a business thus affecting business decisions. Must Have: - Strong design and coding skills using Big Data technologies such as Hadoop Spark(Pyspark) Hive and Airflow. - Experience with at least one cloud platform or service such as Azure or AWS. - Data warehousing ...
Big Data Executive Start Date Starts Immediately CTC (ANNUAL) ₹ 2,00,000 - 2,20,000 ₹ 2,00,000 - 2,20,000 /year Experience 1 year(s) 1 year(s) Apply By 13 Nov' 26 Posted 7 days ago Job 28 applicants About the job Key responsibilities: 1. Enter and maintain accurate client and company information in internal systems. 2. ...
... optimize operational efficiency at https://jobeax.com/link/8KuEYorn4an4hpi5 Responsibilities :- Architect and maintain robust data pipelines using Python and Apache Spark to ensure seamless ingestion and processing of high-volume datasets for downstream analytics.- Orchestrate complex workflows using Airflow to guarantee data ...
... Airflow, Datawarehousing, Data https://jobeax.com/link/jJwPeXOR5xzgNWwu skills :- Strong hands-on experience in Python Development.- Extensive experience with PySpark / Apache Spark.- Expertise in Data Engineering, SQL Unix + Big Data (Hadoop + Hive), Airflow, Datawarehousing, Data modelling and data processing frameworks.- ...