RARR Technologies - Big Data Engineer - ETL/Apache Spark in Chennai, भारत

Role Overview :We are looking for an experienced Big Data Engineer with strong hands-on expertise in Apache Spark and Scala to design, develop, and optimize large-scale data processing solutions. The role will involve building scalable data pipelines, processing high-volume datasets, optimizing distributed workloads, and collaborating with data and technology teams to deliver reliable data https://jobeax.com/link/XnflZD2dGSZYgXFG Responsibilities :- Design, develop, and maintain scalable big data processing applications using Apache Spark and Scala.- Build data processing and transformation pipelines for large and complex datasets.- Develop efficient distributed data-processing solutions with a strong focus on performance, scalability, and reliability.- Optimize Spark jobs, transformations, queries, memory utilization, and cluster workloads.- Analyze data-processing requirements and translate them into scalable technical solutions.- Work with structured and unstructured data and implement appropriate processing and transformation techniques.- Troubleshoot data pipeline failures, performance bottlenecks, and distributed processing issues.- Develop reusable components and follow software engineering and data engineering best practices.- Collaborate with Data Engineers, Architects, Analysts, and business stakeholders to understand requirements and deliver data solutions.- Participate in code reviews, testing, deployment, and production support activities.- Maintain technical documentation covering data pipelines, processing logic, and operational procedures.- Contribute to continuous improvement of data processing frameworks, performance, and engineering https://jobeax.com/link/wJIgo45kO3tINlTo Skills & Experience :- 6 - 11 years of experience in Big Data, Data Engineering, or a related technology domain.- Strong hands-on expertise in Apache Spark and Scala.- Strong understanding of Big Data concepts, distributed computing, and parallel data processing.- Experience developing large-scale data processing, transformation, and ingestion solutions.- Strong understanding of Spark architecture, execution models, partitioning, transformations, actions, and performance optimization.- Experience troubleshooting and tuning distributed data-processing workloads.- Good knowledge of SQL and relational data concepts.- Strong analytical and problem-solving skills.- Good understanding of software development lifecycle, testing, version control, and deployment practices.- Ability to work effectively in a collaborative, fast-paced engineering environment.- Strong communication and stakeholder-management skills. (ref:hirist.tech)