Infosys - Big Data Engineer - Apache Spark/Hadoop Ecosystem in Noida, भारत

Education :- Bachelor's or master's degree in Computer Science, Information Technology, Engineering, or a related https://jobeax.com/link/nnF4EyWTLvU0XkJo and Technical Requirements :- Strong experience in big data engineering, data processing, and distributed computing.- Proficiency in Apache Spark, PySpark, and Hadoop ecosystem technologies.- Experience developing scalable data pipelines for batch and real-time data processing.- Strong programming skills in Python, Scala, Java, or similar languages.- Hands-on experience with Hive, HDFS, Kafka, and SQL.- Knowledge of ETL/ELT processes, data ingestion, transformation, and data integration.- Experience working with structured, semi-structured, and unstructured datasets.- Understanding of distributed systems, data partitioning, performance tuning, and optimization.- Familiarity with cloud-based data platforms such as AWS, Azure, or GCP.- Experience with Git, CI/CD pipelines, and workflow orchestration tools such as https://jobeax.com/link/KcMYm5qjAOje61Jj Responsibilities:- Design, develop, and maintain scalable big data pipelines and processing frameworks.- Build efficient batch and streaming data workflows to process large volumes of data.- Develop data ingestion and transformation processes from multiple source systems.- Optimize Spark jobs and distributed data processing workloads for performance and scalability.- Implement data quality checks, validation, and error-handling mechanisms.- Integrate data from various databases, applications, and external sources.- Monitor pipeline performance and troubleshoot data processing issues.- Collaborate with data architects, analysts, and engineering teams to support data platform requirements.- Maintain technical documentation and follow data engineering best https://jobeax.com/link/wJIgo45kO3tINlTo Skills :- Apache Spark, PySpark, Hadoop, SQL, Python/Scala/Java, ETL, Big Data ProcessingSupported Skills :- Kafka, Hive, HDFS, Airflow, AWS/Azure/GCP, Data Pipeline Optimization, CI/CD (ref:hirist.tech)