Big Data Platform Administrator - Apache Spark & Airflow… - Jobeax
Vacancy description
Big Data Platform Administrator - Apache Spark & Airflow in Chennai, भारत
Samridhi Placements
Chennai, भारत
Job Description :Key Responsibilities :- Administer and support Hadoop ecosystem components (HDFS, YARN) across Production, Test, and Development environments- Manage and maintain Apache Spark and Airflow infrastructure- Monitor cluster health, troubleshoot performance issues, and optimize system performance- Implement and maintain High Availability (HA), Disaster Recovery (DR), backup, and restore strategies- Manage logging, monitoring, and alerting systems using Prometheus, Grafana, and ELK Stack- Perform Linux (Ubuntu) system administration, including patching, upgrades, and security hardening- Develop and maintain automation scripts using Shell and/or Python- Handle Kerberos authentication setup and maintenance- Support incident management, root cause analysis (RCA), and ensure SLA adherence- Manage capacity planning and system scalability- Follow CI/CD and DevOps best practices for platform improvements and deploymentsRequired Skills :- Strong experience with Hadoop ecosystem (HDFS, YARN)- Handson expertise in Apache Spark and Apache Airflow- Proficiency in Linux system administration (Ubuntu), including security hardening- Scripting skills in Shell and Python for automation- Familiarity with monitoring and observability tools: Prometheus, Grafana, ELK Stack- Knowledge of High Availability, Disaster Recovery, backup, and restore concepts- Experience with Kerberos authentication setup and maintenance (ref:hirist.tech)
... Professional Skills : - Experience : 8 to 10 years in software engineering and big data development, including experience leading client-facing delivery teams.- Big Data Stack : Strong hands-on experience with Hadoop (HDFS, MapReduce, YARN), Apache Spark (Core, SQL, Streaming), Apache Kafka, and HBase.- Architecture : Deep ...
... https://jobeax.com/link/wJIgo45kO3tINlTo Skills & Experience :- 6 - 11 years of experience in Big Data, Data Engineering, or a related technology domain.- Strong hands-on expertise in Apache Spark and Scala.- Strong understanding of Big Data concepts, distributed computing, and parallel data processing.- Experience developing large-scale data processing, ...
... Hadoop ecosystem technologies for distributed data processing and storage.- Integrate data from multiple sources including databases, APIs, files, and streaming platforms.- Build and manage data orchestration workflows using Azure Data Factory (ADF).- Optimize Spark jobs, Databricks clusters, and data pipelines for performance ...
WEKA Data Platform Administrator About the job WEKA is architecting a new approach to the enterprise data stack built for the age of reasoning. NeuralMesh by WEKA sets the standard for agentic AI data infrastructure with a cloud and AI-native software solution that can be deployed anywhere. It transforms legacy data silos ...
Big Data Executive Start Date Starts Immediately CTC (ANNUAL) ₹ 2,00,000 - 2,20,000 ₹ 2,00,000 - 2,20,000 /year Experience 1 year(s) 1 year(s) Apply By 13 Nov' 26 Posted 7 days ago Job 28 applicants About the job Key responsibilities: 1. Enter and maintain accurate client and company information in internal systems. 2. ...
... https://jobeax.com/link/wJIgo45kO3tINlTo Skills & Experience :- 5+ years of experience with AWS Services including RDS, AWS Lambda, AWS Glue, EMR, S3, and related big data technologies.- 5+ years of experience working with Apache Spark, Spark Streaming, Kafka, and Hive.- Strong experience in SQL and NoSQL databases (MySQL, ...
... scalable, reliable, high-performance data pipelines and data platforms.EDUCATION:Bachelor's degree in computer science, Software Engineering, MIS or equivalent combination of education and https://jobeax.com/link/NfKu17xn3PQ6gnFL SKILLS:Python, PySpark, Hadoop, Spark, Hive, Data Engineering, Big Data, SQL (ref:hirist.tech) []
... containerized environments.- Troubleshoot production issues and continuously improve platform https://jobeax.com/link/AMuFfOH9R4FgRv8m : - 5+ years of experience in Data Engineering or Big Data Engineering.- Strong hands-on experience with Apache Spark and Scala.- Experience building and maintaining ETL pipelines.- Familiarity ...
About the team The Atlassian Platform Administrator, Desktop Systems P4, is a senior technical administrator responsible for the reliable, secure, scalable operation and evolution of Zillow’s Atlassian platform. The role owns complex administration and configuration across Jira, Confluence, Jira Service Management, Loom, ...