Back to list

Vacancy description

Infosys - Big Data Engineer - Apache Spark/Hadoop Ecosystem in Noida, भारत

infosys ltd

Noida, भारत

Education :- Bachelor's or master's degree in Computer Science, Information Technology, Engineering, or a related https://jobeax.com/link/nnF4EyWTLvU0XkJo and Technical Requirements :- Strong experience in big data engineering, data processing, and distributed computing.- Proficiency in Apache Spark, PySpark, and Hadoop ecosystem technologies.- Experience developing scalable data pipelines for batch and real-time data processing.- Strong programming skills in Python, Scala, Java, or similar languages.- Hands-on experience with Hive, HDFS, Kafka, and SQL.- Knowledge of ETL/ELT processes, data ingestion, transformation, and data integration.- Experience working with structured, semi-structured, and unstructured datasets.- Understanding of distributed systems, data partitioning, performance tuning, and optimization.- Familiarity with cloud-based data platforms such as AWS, Azure, or GCP.- Experience with Git, CI/CD pipelines, and workflow orchestration tools such as https://jobeax.com/link/KcMYm5qjAOje61Jj Responsibilities:- Design, develop, and maintain scalable big data pipelines and processing frameworks.- Build efficient batch and streaming data workflows to process large volumes of data.- Develop data ingestion and transformation processes from multiple source systems.- Optimize Spark jobs and distributed data processing workloads for performance and scalability.- Implement data quality checks, validation, and error-handling mechanisms.- Integrate data from various databases, applications, and external sources.- Monitor pipeline performance and troubleshoot data processing issues.- Collaborate with data architects, analysts, and engineering teams to support data platform requirements.- Maintain technical documentation and follow data engineering best https://jobeax.com/link/wJIgo45kO3tINlTo Skills :- Apache Spark, PySpark, Hadoop, SQL, Python/Scala/Java, ETL, Big Data ProcessingSupported Skills :- Kafka, Hive, HDFS, Airflow, AWS/Azure/GCP, Data Pipeline Optimization, CI/CD (ref:hirist.tech)

Similar vacancies

More like this role

  • Full-time
... implementing modelled attributes and contributing to brainstorming sessions - Build and maintain a robust data engineering process to develop and implement self-serve data and tools for Visa’s product management teams and data scientists What you need - Spark - AWS - Data warehousing - ETL - Azure Data Factory - Hadoop Good to have ...

Bangalore, भारत

Zorba Consulting India

Big Data Engineer

  • On-site

Rs 10 - 19 lakhs p.a.

... Automate data engineering workflows, testing, deployment, and operational processes. Required Skills - Strong hands-on experience in Java, Python, Scala, and Apache Spark. - Experience with Big Data technologies and the Hadoop ecosystem. - Strong understanding of data ingestion, transformation, and distributed data processing. ...

Hyderabad, भारत

Zorba Consulting India

Big Data Engineer

  • On-site

Rs 10 - 18 lakhs p.a.

We are looking for an experienced Big Data Engineer with strong software engineering and distributed data processing skills. The ideal candidate should have hands-on experience with Java, Apache Spark, Python/Scala, and the Hadoop ecosystem to build scalable, reliable data solutions. Roles & Responsibilities Analyze complex ...

Hyderabad, भारत

Zorba Consulting India

AWS Big Data Engineer

Rs 10 - 19 lakhs p.a.

... deploy scalable Big Data applications and data engineering solutions on AWS. Develop API-driven backend services to manage, monitor, and govern large-scale batch data processing applications. Build and maintain data pipelines using Apache Spark, Python, Scala, SQL, Hive, and Hadoop. Work with AWS services including S3, EC2, ...

Hyderabad, भारत

  • Full-time
... you'll do - Design, develop, and maintain scalable Big Data applications and data pipelines - Develop high-performance data-processing solutions using Scala and Apache Spark - Build and optimize ETL/ELT pipelines for large volumes of structured and unstructured data What you need - Scala - SQL - ETL - Spark - Hadoop - Kafka ...

Pune, भारत

  • On-site
  • Full-time
... scalable data ingestion and processing frameworks to handle structured, semi-structured, and unstructured datasets - Solve complex business problems by leveraging data engineering best practices and advanced technologies What you need - SQL - ETL - Python - Spark - Azure Data Factory - Hadoop Good to have - LLMs - Linux - CI/CD ...

Pune, भारत

  • Full-time
... python on Hadoop or object storage - Leverage new technologies and approaches to innovate with increasingly large data sets - Drive automation and efficiency in Data ingestion, data movement and data access workflows by innovation and collaboration What you need - Python - ETL - Hadoop - Data warehousing - Spark - Scala Good ...

Pune, भारत

  • On-site
  • Full-time
... Monitor, optimize, and troubleshoot data pipelines to ensure high availability, reliability, scalability, and performance What you need - SQL - ETL - Python - Spark - Databricks - Azure Data Factory Good to have - LLMs - CI/CD - Shell scripting - MongoDB - Git - Hadoop Job details - Location: Pune - Experience: 5–8 years ...

Pune, भारत

  • Full-time
Mid-level data science role at Citi in Pune. Needs 4+ years. What you'll do - Responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team What you need - Spark - SQL - Python - Hadoop - ETL - Linux Good to have - FastAPI ...

Pune, भारत

... Big Data repositories and enforce standards and principles in the Big Data space. The candidate will be part of an exciting, fast paced environment developing Data Engineering solutions in the data and analytics domain. Role • Develop high quality, secure and scalable data pipelines using spark, Scala/ python on Hadoop or ...

Pune, भारत

... experience in Scala . - Strong experience with Apache Spark / Spark SQL . - Good understanding of Big Data and distributed computing concepts . - Experience with Hadoop ecosystem technologies , including Hadoop/HDFS and Hive. - Experience building large-scale ETL/data engineering pipelines . - Strong SQL skills. - Experience ...

Pune, भारत