Back to list

Vacancy description

Softility - Big Data Lead - Hadoop/Apache Spark in Hyderabad, भारत

Softility

Hyderabad, भारत

Role : Big Data LeadLocation : Hyderabad (Offshore)Experience : 8 - 10 YearsShift Timings : Night Shift (5 : 00 PM - 2 : 00 AM IST) / US EST PrimetimeRole Overview : We are looking for a Big Data Lead with 8 - 10 years of experience to design, scale, and manage our high-volume data pipelines and real-time streaming https://jobeax.com/link/Amgxzy7gmYbPGc16 this role, you will serve as the primary technical point of contact for US clients, lead engineering teams, and ensure data systems run reliably during US overlap https://jobeax.com/link/EWAGgEgr8isMoZFQ & Strategy : - Guide architecture standards and data governance practices.- Mentor developers, train tech leads, and build a strong engineering https://jobeax.com/link/IzdiRuPEDm9cn8k5 Development : - Build and optimize data processing applications using Hadoop, Spark, Kafka, and HBase.- Manage batch/streaming performance, low-latency querying, and partition https://jobeax.com/link/HJ3J7bahcw0HWFhe Engagement : - Act as the technical anchor for client accounts.- Translate technical performance and architecture plans into clear updates for https://jobeax.com/link/9alDK8qMvtSj3mN7 Management : - Lead troubleshooting and root-cause analysis (RCA) during system outages or performance bottlenecks.- Set up monitoring and automated recovery https://jobeax.com/link/0kSopwW8F6ag7jjD Technical & Professional Skills : - Experience : 8 to 10 years in software engineering and big data development, including experience leading client-facing delivery teams.- Big Data Stack : Strong hands-on experience with Hadoop (HDFS, MapReduce, YARN), Apache Spark (Core, SQL, Streaming), Apache Kafka, and HBase.- Architecture : Deep understanding of distributed systems, real-time message streaming, NoSQL schema design (HBase compaction/region tuning), and data lake design.- Performance Tuning : Proven capability in tuning Spark jobs, memory allocation, execution plans, and Kafka partition throughput.- Programming : Proficiency in Scala, Java, or Python, along with strong SQL skills.- CI/CD & Orchestration : Hands-on experience with Git, CI/CD pipelines, and workflow managers like Airflow or Oozie.- Communication : Clear verbal and written communication with the flexibility to work night shifts and overlap with US https://jobeax.com/link/DnRO02YSTsV4WkV8 Certifications : - Databricks Certified Apache Spark Developer / Data Engineer Professional- AWS Certified Data Engineer or GCP Professional Data Engineer (ref:hirist.tech)

Similar vacancies

More like this role

  • On-site
  • Full-time
... scalable data ingestion and processing frameworks to handle structured, semi-structured, and unstructured datasets - Solve complex business problems by leveraging data engineering best practices and advanced technologies What you need - SQL - ETL - Python - Spark - Azure Data Factory - Hadoop Good to have - LLMs - Linux - CI/CD ...

Pune, भारत

  • Full-time
Leadership data science role at Capco in Pune. Needs 10+ years. What you'll do - Develop high quality, secure and scalable data pipelines using spark, Scala/ python on Hadoop or object storage - Leverage new technologies and approaches to innovate with increasingly large data sets - Drive automation and efficiency in Data ...

Pune, भारत

... Big Data repositories and enforce standards and principles in the Big Data space. The candidate will be part of an exciting, fast paced environment developing Data Engineering solutions in the data and analytics domain. Role • Develop high quality, secure and scalable data pipelines using spark, Scala/ python on Hadoop or ...

Pune, भारत

... Big Data repositories and enforce standards and principles in the Big Data space. The candidate will be part of an exciting, fast paced environment developing Data Engineering solutions in the data and analytics domain. Role • Develop high quality, secure and scalable data pipelines using spark, Scala/ python on Hadoop or ...

Pune, भारत

  • Full-time
  • IT
Responsibilities - Design, development and testing of components / modules in TOP (Trade Open Platform) involving Spark, Java, Hive and related big-data technologies in a datalake architecture - Contribute to the design, development and deployment of new features & new components in Azure public cloud - Contribute to the ...

Bangalore, भारत

  • Full-time
... implementing modelled attributes and contributing to brainstorming sessions - Build and maintain a robust data engineering process to develop and implement self-serve data and tools for Visa’s product management teams and data scientists What you need - Spark - AWS - Data warehousing - ETL - Azure Data Factory - Hadoop Good to have ...

Bangalore, भारत

Zorba Consulting India

Big Data Engineer

  • On-site

Rs 10 - 19 lakhs p.a.

... and Apache Spark. - Experience with Big Data technologies and the Hadoop ecosystem. - Strong understanding of data ingestion, transformation, and distributed data processing. - Experience integrating multiple data sources and developing scalable data pipelines. - Understanding of distributed data replication strategies. ...

Hyderabad, भारत

Zorba Consulting India

Big Data Engineer

  • On-site

Rs 10 - 18 lakhs p.a.

We are looking for an experienced Big Data Engineer with strong software engineering and distributed data processing skills. The ideal candidate should have hands-on experience with Java, Apache Spark, Python/Scala, and the Hadoop ecosystem to build scalable, reliable data solutions. Roles & Responsibilities Analyze complex ...

Hyderabad, भारत

PradeepIT Consulting Services

Big Data Developer

  • Full-time
... implement highly scalable and distributed big data - Architect HBase schemas and data models to accommodate evolving business - Develop complex Hive queries and data processing pipelines to transform raw What you need - Machine learning - Spark - Hadoop - Hive Job details - Location: Bengaluru - Experience: 7–10 years - Education: ...

Bangalore, भारत

Zorba Consulting India

AWS Big Data Engineer

Rs 10 - 19 lakhs p.a.

... deploy scalable Big Data applications and data engineering solutions on AWS. Develop API-driven backend services to manage, monitor, and govern large-scale batch data processing applications. Build and maintain data pipelines using Apache Spark, Python, Scala, SQL, Hive, and Hadoop. Work with AWS services including S3, EC2, ...

Hyderabad, भारत

  • On-site

Rs 10 - 19 lakhs p.a.

... development. 9. Follow secure development practices, automated testing, and CI/CD processes. Mandatory Technical Skills: - Programming: Java, Scala, Python - Big Data: Apache Spark, Hadoop, HDFS, MapReduce, Cassandra - AWS: EMR, EC2, ECS, S3, Airflow, Step Functions, API Gateway - API Development: REST APIs, SOAP Web Services, ...

Hyderabad, भारत