Vacancy description

Big Data Engineer in Hyderabad, भारत

Zorba Consulting India

  • On-siteWork from the office

Rs 10 - 19 lakhs p.a.

Hyderabad, भारत

1. Utilize strong software engineering skills in Java, Python, Scala, and Apache Spark to analyze complex systems and design scalable data solutions.
2. Design, develop, and maintain scalable Big Data applications and distributed data processing solutions.
3. Integrate new data sources, tools, and technologies into existing Big Data ecosystems.
4. Implement scalable, reliable, and fault-tolerant distributed data ingestion and replication strategies.
5. Work with large datasets using Apache Spark and various components of the Hadoop ecosystem.
6. Collaborate with cross-functional teams to design, develop, test, and deploy data tools supporting business and product requirements.
7. Provide technical guidance and mentorship to onsite/offshore developers on architecture, design, and implementation.
8. Own end-to-end product features across the software development lifecycle, from development and testing to production deployment.
9. Evaluate emerging Big Data technologies and develop prototypes to improve data processing architecture and performance.
10. Automate data engineering workflows, testing, deployment, and operational processes.
Required Skills
- Strong hands-on experience in Java, Python, Scala, and Apache Spark.
- Experience with Big Data technologies and the Hadoop ecosystem.
- Strong understanding of data ingestion, transformation, and distributed data processing.
- Experience integrating multiple data sources and developing scalable data pipelines.
- Understanding of distributed data replication strategies.
- Strong object-oriented programming, design, and analysis skills.
- Experience with Agile/Scrum development methodologies.
- Knowledge of secure application development practices.
- Understanding of automated QA and testing requirements for Big Data applications.
- Ability to translate technical and business requirements into effective technical solutions.
- Excellent written and verbal communication skills.
- Apache Airflow experience is preferred.

Similar vacancies

More like this role

Zorba Consulting India

Big Data Engineer

  • On-site

Rs 10 - 18 lakhs p.a.

... testing requirements for Big Data applications. Excellent written and verbal communication skills. Good to Have Experience with Apache Airflow. Experience with Big Data workflow orchestration and automation. Experience mentoring developers and providing technical direction. Preferred Candidate: Big Data Engineers / Java ...

Hyderabad, भारत

  • Full-time
... scoping and designing analytic data assets, implementing modelled attributes and contributing to brainstorming sessions - Build and maintain a robust data engineering process to develop and implement self-serve data and tools for Visa’s product management teams and data scientists What you need - Spark - AWS - Data warehousing ...

Bangalore, भारत

Zorba Consulting India

AWS Big Data Engineer

Rs 10 - 19 lakhs p.a.

... Apache Airflow. The ideal candidate will have hands-on experience developing scalable data engineering solutions, API-driven backend services, and distributed Big Data applications. Key Responsibilities Design, develop, test, and deploy scalable Big Data applications and data engineering solutions on AWS. Develop API-driven ...

Hyderabad, भारत

... Engineer – Scala We are looking for an experienced Big Data Engineer with strong expertise in Scala and Apache Spark to design, develop, and optimize scalable data-processing solutions. The ideal candidate should have hands-on experience with distributed data platforms, large-scale ETL/data pipelines, and big data technologies. ...

Pune, भारत

... Engineer – Scala We are looking for an experienced Big Data Engineer with strong expertise in Scala and Apache Spark to design, develop, and optimize scalable data-processing solutions. The ideal candidate should have hands-on experience with distributed data platforms, large-scale ETL/data pipelines, and big data technologies. ...

Pune, भारत

  • Full-time
... scalable data pipelines using spark, Scala/ python on Hadoop or object storage - Leverage new technologies and approaches to innovate with increasingly large data sets - Drive automation and efficiency in Data ingestion, data movement and data access workflows by innovation and collaboration What you need - Python - ETL ...

Pune, भारत

  • Full-time
Responsibilities - Build and implement end-to-end data warehouse, data lake, and lake house projects in big data environments. - Design complex data pipelines using Spark with Scala, Python, or Java on Hadoop or object storage. - Work with Oracle and Netezza databases and apply strong SQL skills. - Participate in Agile/Scrum ...

Pune, भारत

... perspective gives us a competitive advantage. Position: Data Engineer Location: Pune, Maharashtra Job Description The Data semantic solution team is looking for a Big Data Engineer to drive our mission to unlock potential of data assets by consistently innovating, eliminating friction in how users access data from its Big Data repositories ...

Pune, भारत

  • On-site
  • Full-time
... Develop scalable data solutions to collect, ingest, transform, and store large volumes of structured and unstructured data - Monitor, optimize, and troubleshoot data pipelines to ensure high availability, reliability, scalability, and performance What you need - SQL - ETL - Python - Spark - Databricks - Azure Data Factory ...

Pune, भारत

  • Full-time
Data science role at Capco in Pune. What you'll do - Design, develop, and maintain scalable Big Data applications and data pipelines - Develop high-performance data-processing solutions using Scala and Apache Spark - Build and optimize ETL/ELT pipelines for large volumes of structured and unstructured data What you need ...

Pune, भारत