Vacancy description

AWS Big Data Engineer in Hyderabad, भारत

Zorba Consulting India

Rs 10 - 19 lakhs p.a.

Hyderabad, भारत

We are looking for an experienced AWS Big Data Engineer with strong expertise in Python/Scala, Apache Spark, AWS cloud services, and Apache Airflow. The ideal candidate will have hands-on experience developing scalable data engineering solutions, API-driven backend services, and distributed Big Data applications.

Key Responsibilities

Design, develop, test, and deploy scalable Big Data applications and data engineering solutions on AWS.

Develop API-driven backend services to manage, monitor, and govern large-scale batch data processing applications.

Build and maintain data pipelines using Apache Spark, Python, Scala, SQL, Hive, and Hadoop.

Work with AWS services including S3, EC2, EMR, Lambda, DynamoDB, and API Gateway.

Design and enhance Apache Airflow DAGs for workflow orchestration, scheduling, dependency management, monitoring, and failure handling.

Develop scalable batch and real-time data processing solutions.

Perform performance tuning, optimization, troubleshooting, and production support for distributed data applications.

Participate in architecture reviews, code reviews, testing, and deployment activities.

Collaborate with data engineers, backend developers, QA, DevOps, and product teams in an Agile/Scrum environment.

Implement automation, CI/CD practices, and operational improvements to enhance system reliability.

Mandatory Skills

5+ years of experience in Big Data Engineering, distributed systems, or enterprise application development.

Strong hands-on experience with AWS cloud services, particularly S3, EC2, EMR, Lambda, DynamoDB, and API Gateway.

Strong programming experience in Python or Scala.

Hands-on experience with Apache Spark, Hadoop, and Hive.

Strong experience in Apache Airflow, including complex DAG development, scheduling, dependencies, monitoring, and error handling.

Strong SQL and data engineering fundamentals.

Experience developing APIs, backend services, or API-driven applications.

Experience with Git, Jenkins, and CI/CD pipelines.

Good understanding of batch processing, data transformation, data quality, and performance optimization.

Experience working in Agile/Scrum environments.

Good to Have

Exposure to LLMs, Generative AI, Agentic AI, or AI-assisted engineering.

Experience with REST APIs, microservices, event-driven architecture, and serverless applications.

Knowledge of Infrastructure as Code and automated testing.

Experience with data governance, metadata management, data lineage, and data observability.

Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical discipline.

Key Skills

AWS | AWS EMR | Apache Spark | Python | Scala | Apache Airflow | Hadoop | Hive | SQL | S3 | EC2 | Lambda | DynamoDB | API Gateway | REST APIs | Microservices | Git | Jenkins | CI/CD

Similar vacancies

More like this role

  • Full-time
  • IT
Mid-level data science role at JPMorgan Chase in Bengaluru. Needs 3+ years. What you'll do - Analyzes, develops, tests and supports cloud, big data and reporting solutions required to fulfil business and technology requirements - Troubleshoots technical issues by breaking down problems and ensuring permanent resolution ...

Bangalore, भारत

  • Full-time
  • IT
... to work in collaborative teams to achieve organizational goals. - Proven technical proficiency in Big data stack – HDFS, Spark, Impala, etc. with expertise in AWS including S3, EMR, SQS, etc. - Experience in some Industry Standard Reporting Tool like Cognos/Sigma/Thoughtspot etc. with knowledge in Advanced SQL and data ...

Bangalore, भारत

SUPERSOURCING TECHNOLOGIES PRIVATE LIMITED

Senior Big Data Engineer - Azure/AWS Databricks

... looking for experienced Big Data Engineers and Databricks professionals who can design, develop, and optimize scalable data engineering solutions using modern Big Data technologies and cloud https://jobeax.com/link/UuxHBYkbQjke5gQz ideal candidate should have strong hands-on experience with Apache Spark, Python, and Databricks, ...

Gurgaon, भारत

  • Full-time
... scoping and designing analytic data assets, implementing modelled attributes and contributing to brainstorming sessions - Build and maintain a robust data engineering process to develop and implement self-serve data and tools for Visa’s product management teams and data scientists What you need - Spark - AWS - Data warehousing ...

Bangalore, भारत

Zorba Consulting India

Big Data Engineer

  • On-site

Rs 10 - 19 lakhs p.a.

1. Utilize strong software engineering skills in Java, Python, Scala, and Apache Spark to analyze complex systems and design scalable data solutions. 2. Design, develop, and maintain scalable Big Data applications and distributed data processing solutions. 3. Integrate new data sources, tools, and technologies into existing ...

Hyderabad, भारत

Zorba Consulting India

Big Data Engineer

  • On-site

Rs 10 - 18 lakhs p.a.

... testing requirements for Big Data applications. Excellent written and verbal communication skills. Good to Have Experience with Apache Airflow. Experience with Big Data workflow orchestration and automation. Experience mentoring developers and providing technical direction. Preferred Candidate: Big Data Engineers / Java ...

Hyderabad, भारत

  • Full-time
Data science role at Capco in Pune. What you'll do - Design, develop, and maintain scalable Big Data applications and data pipelines - Develop high-performance data-processing solutions using Scala and Apache Spark - Build and optimize ETL/ELT pipelines for large volumes of structured and unstructured data What you need ...

Pune, भारत

  • On-site
  • Full-time
... platforms for batch and real-time data processing - Build scalable data ingestion and processing frameworks to handle structured, semi-structured, and unstructured datasets - Solve complex business problems by leveraging data engineering best practices and advanced technologies What you need - SQL - ETL - Python - Spark - Azure ...

Pune, भारत

  • Full-time
... scalable data pipelines using spark, Scala/ python on Hadoop or object storage - Leverage new technologies and approaches to innovate with increasingly large data sets - Drive automation and efficiency in Data ingestion, data movement and data access workflows by innovation and collaboration What you need - Python - ETL ...

Pune, भारत

  • On-site
  • Full-time
... Develop scalable data solutions to collect, ingest, transform, and store large volumes of structured and unstructured data - Monitor, optimize, and troubleshoot data pipelines to ensure high availability, reliability, scalability, and performance What you need - SQL - ETL - Python - Spark - Databricks - Azure Data Factory ...

Pune, भारत