Vacancy description
India, Bengaluru
Big Data Engineer - Hadoop/Spark/Scala in Bengaluru, India is listed on Jobeax. Browse 30,000+ vacancies available.
Role Overview : We are seeking a seasoned Big Data Engineer to join our high-performing data platform team. In this role, you will be responsible for architecting, developing, and maintaining robust data pipelines that process massive datasets to fuel our analytical engines. You will work closely with cross-functional teams, including data scientists, product managers, and infrastructure engineers, to translate complex business requirements into scalable technical solutions. By optimizing our Hadoop and Spark ecosystems, you will directly influence the speed and reliability of our data-driven decision-making processes, ensuring that our stakeholders have access to high-quality, actionable insights that drive business https://jobeax.com/link/KgVLdM5cThorPSD1 Responsibilities : - Design and implement scalable data pipelines using Scala and Spark to ingest and transform large-scale datasets, ensuring high availability and performance for downstream analytics.- Optimize complex SparkSQL queries and Hadoop jobs to reduce processing latency, directly improving the efficiency of our data infrastructure.- Collaborate with engineering teams to integrate real-time data streams using Kafka, enabling low-latency data availability for critical business applications.- Maintain rigorous data quality standards by implementing automated testing and monitoring frameworks, ensuring reliability for all internal and external data consumers.- Partner with stakeholders to identify data bottlenecks and implement innovative engineering solutions that support the long-term scalability of our Big Data https://jobeax.com/link/1dkmzToQOwmnlB8O Skillset : - Demonstrated expertise in building distributed data systems using Hadoop, Spark, and Scala, with a deep understanding of performance tuning and memory management.- Proven ability to write complex SQL queries and develop efficient data models that support diverse analytical use cases.- Strong proficiency in Python for scripting and automation, complemented by hands-on experience in managing data flows through Kafka.- Exceptional communication skills with the ability to articulate technical concepts to non-technical stakeholders and work effectively within a collaborative, fast-paced environment.- A Bachelors or Masters degree in Computer Science, Engineering, or a related quantitative field, reflecting a strong foundation in data structures and algorithms.- Ability to thrive in a hybrid work model across our Chennai, Bangalore, Pune, or Mumbai offices, demonstrating high levels of self-motivation and adaptability to evolving project requirements.- 5 - 8 years of professional experience in Big Data Engineering, with a track record of delivering high-impact data solutions in enterprise environments. (ref:hirist.tech)
Similar vacancies
altysys
... contributing to the design and development of robust, scalable data pipelines and solutions. Key Responsibilities: - Design and develop scalable data pipelines and engineering solutions - Write clean, maintainable, and efficient code using Scala and Spark - Work with Azure or on-premises Hadoop ecosystem for data processing - Solve ...
India, Gurugram
Mastercard
... Big Data repositories and enforce standards and principles in the Big Data space. The candidate will be part of an exciting, fast paced environment developing Data Engineering solutions in the data and analytics domain. Develop high quality, secure and scalable data pipelines using spark, Scala/ python on Hadoop or object ...
India, Pune
MaimsD Technology
... maintain scalable big data pipelines and architectures to support robust data solutions.- Collaborate with cross-functional teams to understand and deliver on data requirements and business objectives.- Implement data integration, transformation, and processing solutions using Python and relevant big data technologies.- ...
India
Sigmoid
... sets from a variety of sources - Experience with Hadoop, Spark, or similar stack a must - Experience with functional and object-oriented programming, Python or Scala a must - Ability to collaborate with a diverse set of engineers, data scientists and product managers - Experience in agile methodology Development and support ...
India, Bengaluru
Vensure Employer Solutions
... you an all-around data enthusiast with a knack for ETL We're hiring Data Engineers to help build and optimize the foundational architecture of our product's data. We've built a strong data engineering team to date, but have a lot of work ahead of us, Migrating from relational databases to a streaming and big data architecture, ...
India, Thiruvananthapuram
Norwin Technologies
... investigations. Design scalable cloud solutions using AWS and/or Azure . Ensure data quality, governance, reliability, monitoring, and compliance. Partner with Data Scientists, Fraud Analysts, and Engineering teams to deliver data-driven solutions. 4+ years of experience in Data Engineering or Big Data Engineering. - 2+ years ...
India, Bengaluru
blackbaud india
... delivery and improve quality and actively experiment with emerging capabilities as they evolve. Have 5+ years of data engineering experience - Experience with big data technologies such as Spark, Databricks, Delta lakes, Hive, etc. - 4+ years of experience in core languages such as Python, Scala or Java (Preferably Python) ...
India, Rāniganj
Referrals Only
... leading the system design and implementation of technical solutions. Working with data excites you; You have created Big Data architecture, can build and operate data pipelines, and maintain data storage, all within distributed systems. You have a deep understanding of data modeling and experience with modern data engineering ...
India, Bengaluru
Recruitment Room
$ 62400 p.a.
... build, and maintain scalable data pipelines and architectures for large-scale data environments. - Translate business and technical requirements into reliable data engineering solutions. - Develop data integration, transformation, and processing workflows using Python and appropriate big data technologies. - Build, manage, ...
India
Recruitment Room
$ 62400 p.a.
... build, and maintain scalable data pipelines and architectures for large-scale data environments. - Translate business and technical requirements into reliable data engineering solutions. - Develop data integration, transformation, and processing workflows using Python and appropriate big data technologies. - Build, manage, ...
India
PayPay India
... facilitate smooth data flow and integration. Enforce best practices in observability, data governance, security, and regulatory compliance Minimum 7 years as a Data Engineer or similar role. Hands-on experience with Databricks, Delta Lake, Spark, and Scala. Proven ability to design, build, and operate Data Lakes or Data Warehouses. ...
India, Gurugram
Publicis Sapient
20k
... Architect batch and real-time data pipelines leveraging modern big data frameworks and cloud-native services. Provide technical leadership and hands-on guidance to engineering teams, ensuring solution quality, scalability, security, and operational excellence. Establish best practices in data engineering, including coding standards, ...
India, Bengaluru
About the Role : We are looking for a Big Data Engineer to design, build, and operate large-scale data pipelines and analytical infrastructure that transform high-volume raw data into reliable, query-ready datasets for analytics, reporting, and data-driven https://jobeax.com/link/HouIpt6AtbK1KWOU data platform ingests and ...
India, Gurugram
Auriga IT
... similar). Experience in building data pipelines or ETL processes. Familiarity with REST APIs and backend frameworks (Flask/Django is a plus). Understanding of data structures and algorithms. Good To Have Experience with big data tools (Spark, Hadoop). Knowledge of cloud platforms (AWS, Azure, or GCP). Exposure to machine ...
India, Jaipur
Airbus pioneers
... Support / Customer Experience : Minimum 3 years in a customer-facing support or service role (specifically working with European customers). Technical Skills Big Data & Frameworks: Strong hands-on experience with Apache Spark, PySpark (Mandatory), and Spark SQL. Lightweight Data Processing: Practical experience with lightweight ...
India, Bengaluru
ALTEN Calsoft Labs
... architecture reviews. Participate in incident response and operational excellence initiatives for the data warehouse. Continuously learn mindset and apply new Hadoop ecosystem tools and data technologies. Required Skills and Experience - Proficiency in Hadoop ecosystems such as Spark, HDFS, Hive, Iceberg, Spark SQL. - Extensive ...
India, Bengaluru
... R, Java Strong in statistical & machine learning concepts Knowledge of Python Libraries - SciPy, NumPy, Pandas, I Python, Scikit-learn Knowledge of distributed big data processing (PySpark, Jupyter, Linux, AWS) Hypothesis testing, insights generation, root cause analysis, factor analysis Statistical model (predictive & prescriptive) ...
India, Bengaluru
ShrinQ Consulting Group Inc
The ideal candidate will use their passion for big data and analytics to provide insights to the business covering a range of topics. They will be responsible for conducting both recurring and ad hoc analysis for business users. Responsibilities Understand the day-to-day issues that our business faces, which can be better ...
India, India
... years of experience in the field of data engineering - Graduate degree or higher with courses in programming, data analytics - Good programming skill in Python/scala or any scripting language - Good knowledge of SQL is a required - Basic knowledge and understanding of big data - Experience in Azure - Excellent written and ...
India, Bengaluru