Softility - Big Data Lead - Hadoop/Apache Spark in Hyderabad, भारत

Role : Big Data LeadLocation : Hyderabad (Offshore)Experience : 8 - 10 YearsShift Timings : Night Shift (5 : 00 PM - 2 : 00 AM IST) / US EST PrimetimeRole Overview : We are looking for a Big Data Lead with 8 - 10 years of experience to design, scale, and manage our high-volume data pipelines and real-time streaming https://jobeax.com/link/Amgxzy7gmYbPGc16 this role, you will serve as the primary technical point of contact for US clients, lead engineering teams, and ensure data systems run reliably during US overlap https://jobeax.com/link/EWAGgEgr8isMoZFQ & Strategy : - Guide architecture standards and data governance practices.- Mentor developers, train tech leads, and build a strong engineering https://jobeax.com/link/IzdiRuPEDm9cn8k5 Development : - Build and optimize data processing applications using Hadoop, Spark, Kafka, and HBase.- Manage batch/streaming performance, low-latency querying, and partition https://jobeax.com/link/HJ3J7bahcw0HWFhe Engagement : - Act as the technical anchor for client accounts.- Translate technical performance and architecture plans into clear updates for https://jobeax.com/link/9alDK8qMvtSj3mN7 Management : - Lead troubleshooting and root-cause analysis (RCA) during system outages or performance bottlenecks.- Set up monitoring and automated recovery https://jobeax.com/link/0kSopwW8F6ag7jjD Technical & Professional Skills : - Experience : 8 to 10 years in software engineering and big data development, including experience leading client-facing delivery teams.- Big Data Stack : Strong hands-on experience with Hadoop (HDFS, MapReduce, YARN), Apache Spark (Core, SQL, Streaming), Apache Kafka, and HBase.- Architecture : Deep understanding of distributed systems, real-time message streaming, NoSQL schema design (HBase compaction/region tuning), and data lake design.- Performance Tuning : Proven capability in tuning Spark jobs, memory allocation, execution plans, and Kafka partition throughput.- Programming : Proficiency in Scala, Java, or Python, along with strong SQL skills.- CI/CD & Orchestration : Hands-on experience with Git, CI/CD pipelines, and workflow managers like Airflow or Oozie.- Communication : Clear verbal and written communication with the flexibility to work night shifts and overlap with US https://jobeax.com/link/DnRO02YSTsV4WkV8 Certifications : - Databricks Certified Apache Spark Developer / Data Engineer Professional- AWS Certified Data Engineer or GCP Professional Data Engineer (ref:hirist.tech)