Vacancy description
India, Bengaluru
Big Data Engineer - Hadoop/Spark/Scala in Bengaluru, India is listed on Jobeax. Browse 30,000+ vacancies available.
Role Overview : We are seeking a seasoned Big Data Engineer to join our high-performing data platform team. In this role, you will be responsible for architecting, developing, and maintaining robust data pipelines that process massive datasets to fuel our analytical engines. You will work closely with cross-functional teams, including data scientists, product managers, and infrastructure engineers, to translate complex business requirements into scalable technical solutions. By optimizing our Hadoop and Spark ecosystems, you will directly influence the speed and reliability of our data-driven decision-making processes, ensuring that our stakeholders have access to high-quality, actionable insights that drive business https://jobeax.com/link/KgVLdM5cThorPSD1 Responsibilities : - Design and implement scalable data pipelines using Scala and Spark to ingest and transform large-scale datasets, ensuring high availability and performance for downstream analytics.- Optimize complex SparkSQL queries and Hadoop jobs to reduce processing latency, directly improving the efficiency of our data infrastructure.- Collaborate with engineering teams to integrate real-time data streams using Kafka, enabling low-latency data availability for critical business applications.- Maintain rigorous data quality standards by implementing automated testing and monitoring frameworks, ensuring reliability for all internal and external data consumers.- Partner with stakeholders to identify data bottlenecks and implement innovative engineering solutions that support the long-term scalability of our Big Data https://jobeax.com/link/1dkmzToQOwmnlB8O Skillset : - Demonstrated expertise in building distributed data systems using Hadoop, Spark, and Scala, with a deep understanding of performance tuning and memory management.- Proven ability to write complex SQL queries and develop efficient data models that support diverse analytical use cases.- Strong proficiency in Python for scripting and automation, complemented by hands-on experience in managing data flows through Kafka.- Exceptional communication skills with the ability to articulate technical concepts to non-technical stakeholders and work effectively within a collaborative, fast-paced environment.- A Bachelors or Masters degree in Computer Science, Engineering, or a related quantitative field, reflecting a strong foundation in data structures and algorithms.- Ability to thrive in a hybrid work model across our Chennai, Bangalore, Pune, or Mumbai offices, demonstrating high levels of self-motivation and adaptability to evolving project requirements.- 5 - 8 years of professional experience in Big Data Engineering, with a track record of delivering high-impact data solutions in enterprise environments. (ref:hirist.tech)
Similar vacancies
AngelONE
... analysis methods Good knowledge of machine learning tools such as Scikit-Learn, Pytorch/Tensorflow Experience in Deep Learning applications is a plus Experience with Big Data technologies, e.g.: Spark, Hive, and Presto, and ETL pipelines is a plus Diversity, Equity and Inclusion at Angel One At Angel One, our culture is rooted ...
India, Bengaluru
artech l.l.c.
... Time Series Forecasting techniques and real-world demand planning use cases. - Hands-on experience with Databricks , including Spark, notebooks, and distributed data processing. - Practical exposure to Machine Learning and Deep Learning model development and deployment. - Experience working with large, complex datasets in ...
India, Bengaluru
USEReady Technology Pvt. Ltd.
Role Overview : We are looking for a Senior Azure Data Engineer with strong expertise in Databricks and Azure data services. The role involves designing, building, and optimizing scalable, secure, and high-performance data platforms, leveraging modern Lakehouse architecture and real-time data processing https://jobeax.com/link/NEGihx4n1Wqwy714 ...
India
... cost.- Big-data and pipeline fluency: Advanced SQL plus distributed processing for large spatial workloads (Spark or Dask), and building reliable, repeatable data pipelines.- Productionizing models: Experience turning models into deployable, real-time APIs in collaboration with engineering - clean, tested, well-documented ...
India, Pune
... data engineering and demonstrate proficiency in data pipeline development, programming, data modeling, and governance. Key Responsibilities Design and implement data pipelines using AWS and Databricks. Manage and orchestrate data workflows, ensuring efficient data ingestion processes. Utilize Delta Lake for data storage and ...
India, Pune
Questhiring
... problem-solving skills. Technical Skills Cloud: GCP, BigQuery, Google Cloud Storage Programming: Python, SQL Data Processing: Dataflow, Apache Beam, Dataproc, Spark/PySpark Orchestration: Cloud Composer, Apache Airflow Transformation: Dataform, dbt Engineering: Git, CI/CD, Automated Testing, Monitoring Data Engineering: ...
India, Haryāna
Ethos Life
... role Ethos is seeking a Data Engineer to join our Data Platform team and build the internal infrastructure, services, and tooling that the rest of the company's data stack runs on. This is a software engineering role with a deep data background. You will design and operate distributed workflow systems on Temporal and Airflow, ...
India, Bengaluru
StarRez
1400
... & Data Platform. You’ll design and maintain reliable, secure, and scalable data pipelines and models that power a new generation of reporting, insights, and data-driven products across the StarRez ecosystem. This role sits at the intersection of data, product, and engineering. You will work closely with Data Analysts, ...
India, Hyderabad
Walmart Global Tech India
... programming across one or more platforms, languages and frameworks (Java/Python, Spring, Springboot, Microservice), data structure fundamentals and relational/NoSQL databases, Cloud Computing, Linux system and networks Must have good knowledge of big data tools like GCP Big Query/ Azure Data Bricks. of experience with SQL such ...
India, Bengaluru
Allianz Services
... leadership across re-engineering and migration workstreams, contributing to decommission planning for legacy components Core Technical Skills Azure Synapse & Data Platform Mandatory hands-on expertise with: Azure Synapse Analytics (Pipelines, Spark Pool, Dedicated SQL Pool) Azure Data Lake Storage Gen2 (ADLS Gen2) Delta ...
India
... or equivalent - Experience: 4 - 8 years of experience in data engineering or data application development (ETL/ELT/BI) - 2+ years of experience in cloud-based data platform development - Expertise in building Azure-based data pipelines, including: - Azure Data Factory / Synapse - DataBricks / Synapse Spark Pool - Cosmos ...
India, Pune
Prudent Technologies and Consulting, Inc.
... Experience with data quality tooling (SODA, Collibra, or similar) Exposure to cloud platforms (Azure, AWS, or GCP) Experience in a regulated or enterprise-scale data environment Prior experience mentoring or leading a small pod of engineers EXPERIENCE - 8+ years in data engineering, with at least 4+ years focused on Snowflake ...
India, Hyderabad
... LLMs, AutoGen, OCR, Databricks, PySpark, Python, Azure DevOps, and enterprise AI platforms. The role requires strong hands-on experience across SQL Server, Azure Data Factory, Azure Data Lake, Microsoft Fabric, Logic Apps,Power BI, GitHub, and Data Modeling. Required Skills: - 10+ years of Software Engineering / Data Engineering ...
India, Bengaluru
... checks Contribute to and help shape company-wide data governance standards Collaborate with analytics, BI, and business teams to deliver trusted, well-modeled data 5–7 years of hands-on data engineering experience - Proven production data engineering experience at scale - Strong Python and SQL skills - Deep analytical warehouse ...
India, Bengaluru
BAJAJ FINSERV HEALTH
... Engineer will own the end-to-end operationalisation of machine learning, large language model (LLM), and agentic AI workloads on the Bajaj Finance Enterprise Data Platform — a 5PB+ medallion lakehouse built on Azure Databricks and Unity Catalog. This role sits at the intersection of data engineering, model lifecycle management, ...
India, Pune
Job Summary We are looking for a skilled Data Engineer with a strong background in building scalable, high-performance data pipelines on Google Cloud Platform (GCP) . The ideal candidate will have hands-on experience with BigQuery , Airflow , and cloud-based ETL workflows , along with solid programming expertise in Java ...
India, Mannady
consultancy
... https://jobeax.com/link/fYGlpInxuIv5PeS1 & Skills :- 4 - 7 years in data engineering: SQL, Python, ETL/ELT orchestration.- Cloud data platform experience (Azure preferred): pipelines, storage, APIs.- Strong understanding of data lake and data architecture.- Data modelling for analytics (star schema) and data quality frameworks. (ref:hirist.tech)
India, Mumbai
Applied Data Finance
... Amazon S3, Amazon S3 Tables (Apache Iceberg), Amazon EMR, Amazon Athena (preferred). Snowflake/Databricks desirable.- Data Architecture & Modeling : Enterprise Data Modeling, Dimensional Modeling, Data Vault, Semantic Layer, MDM, KPI & Metric Modeling.- Data Platform Engineering : Apache Spark (PySpark), Python, Advanced ...
India
R Systems
Senior Data Engineer Starts Immediately Competitive salary Competitive salary Experience We are looking for a Senior Data Engineer to join our team. In this role, you will design, build, and optimize scalable data pipelines on the Databricks Lakehouse Platform. You will partner with data science, analytics, and business ...
India
Infinite Computer Solutions India Pvt. Ltd.
... platform ecosystem including Workbench, Connect, and integration patterns with R and Python environments Proficient in distributed computing technologies and big data analytics, including hands-on expertise with Python, Spark, SQL, and data transformation pipelines Strong understanding of MLOps/ModelOps principles, practices, ...
India, India