Vacancy description
India, Bengaluru
Big Data Engineer - Hadoop/Spark/Scala in Bengaluru, India is listed on Jobeax. Browse 30,000+ vacancies available.
Role Overview : We are seeking a seasoned Big Data Engineer to join our high-performing data platform team. In this role, you will be responsible for architecting, developing, and maintaining robust data pipelines that process massive datasets to fuel our analytical engines. You will work closely with cross-functional teams, including data scientists, product managers, and infrastructure engineers, to translate complex business requirements into scalable technical solutions. By optimizing our Hadoop and Spark ecosystems, you will directly influence the speed and reliability of our data-driven decision-making processes, ensuring that our stakeholders have access to high-quality, actionable insights that drive business https://jobeax.com/link/KgVLdM5cThorPSD1 Responsibilities : - Design and implement scalable data pipelines using Scala and Spark to ingest and transform large-scale datasets, ensuring high availability and performance for downstream analytics.- Optimize complex SparkSQL queries and Hadoop jobs to reduce processing latency, directly improving the efficiency of our data infrastructure.- Collaborate with engineering teams to integrate real-time data streams using Kafka, enabling low-latency data availability for critical business applications.- Maintain rigorous data quality standards by implementing automated testing and monitoring frameworks, ensuring reliability for all internal and external data consumers.- Partner with stakeholders to identify data bottlenecks and implement innovative engineering solutions that support the long-term scalability of our Big Data https://jobeax.com/link/1dkmzToQOwmnlB8O Skillset : - Demonstrated expertise in building distributed data systems using Hadoop, Spark, and Scala, with a deep understanding of performance tuning and memory management.- Proven ability to write complex SQL queries and develop efficient data models that support diverse analytical use cases.- Strong proficiency in Python for scripting and automation, complemented by hands-on experience in managing data flows through Kafka.- Exceptional communication skills with the ability to articulate technical concepts to non-technical stakeholders and work effectively within a collaborative, fast-paced environment.- A Bachelors or Masters degree in Computer Science, Engineering, or a related quantitative field, reflecting a strong foundation in data structures and algorithms.- Ability to thrive in a hybrid work model across our Chennai, Bangalore, Pune, or Mumbai offices, demonstrating high levels of self-motivation and adaptability to evolving project requirements.- 5 - 8 years of professional experience in Big Data Engineering, with a track record of delivering high-impact data solutions in enterprise environments. (ref:hirist.tech)
Similar vacancies
Weekday AI
... for an experienced Data Engineer to design, develop, and maintain scalable data platforms and pipelines using AWS, Apache Spark, Python, SQL, Kafka, and modern data engineering frameworks . The role will focus on building reliable data solutions for high-volume batch and real-time workloads, including data ingestion, transformation, ...
India, Mumbai
Databricks
20k
... community. 12+ years in data engineering, data science, technical architecture, or a similar pre-sales / consulting role. - 8+ years hands-on experience with Big Data and AI technologies, including Apache Spark™, data engineering, data science, and modern AI/ML workloads. - Strong hands-on architectural experience — able ...
India, Bengaluru
14k - 30k
... and experience working with Python/Scala and PySpark Experienced in Azure Data bricks, Azure Blob Storage, Azure Data Lake, Delta lake. Experience working with Spark SQL Experience in creating pipelines and databricks dashboards Experience with Azure cloud environments Experience with acquiring and preparing data from primary ...
India, Noida
Axtria - Ingenious Insights
... Learning models and their https://jobeax.com/link/MJvWAZuK638tHuBb RESPONSIBILITIES : Tech Stack & Skills : - Experience in model development using Python/PySpark libraries. Development on Databricks or Dataiku DSS is a plus.- Strong experience on Spark with Scala/Python/Java.- Proficiency in building, training, and evaluating ...
India, Gurugram
Axtria - Ingenious Insights
... Learning models and their https://jobeax.com/link/MJvWAZuK638tHuBb RESPONSIBILITIES : Tech Stack & Skills : - Experience in model development using Python/PySpark libraries. Development on Databricks or Dataiku DSS is a plus.- Strong experience on Spark with Scala/Python/Java.- Proficiency in building, training, and evaluating ...
India, Noida
... least 2 years in a technical leadership role overseeing data engineering or data platform teams. - Strong proficiency in big data technologies, including Apache Spark/PySpark, Apache Airflow and Apache Kafka. - Experience with programming languages such as Scala and Python for developing robust data pipelines. - In-depth ...
India, Bengaluru
Accenture
... Azure, or GCP for data engineering workflows. Strong proficiency in PySpark, Spark, or similar frameworks for building scalable data pipelines. Understanding of Big Data architectures, data storage, and data processing concepts. Familiarity with cloud-native data storage solutions such as S3, Blob Storage, BigQuery, or Redshift. ...
India, Bengaluru
Walmart Global Tech India
... working with big data and ETL development 3-5 years of experience in big data analytics technologies like Spark/Spark streaming, , Kafka streaming, Elasticsearch, Bigquery etc. Extensive years of experience in RDMBS/ NoSQL databases, Enterprise Messaging Application (Kafka) and Big data (Spark) Hands-on experience in designing ...
India, Bengaluru
Databricks
20k
... look for - 12+ years in data engineering, data science, technical architecture, or a similar pre-sales / consulting role. - 8+ years hands-on experience with Big Data and AI technologies, including Apache Spark™, data engineering, data science, and modern AI/ML workloads. - Strong hands-on architectural experience — able ...
India, Bengaluru
Crescendo Global Leadership Hiring India Pvt Ltd
Consultant Data Engineer Databricks | Azure | PySpark | Spark SQL - Bengaluru / Hyderabad / GurugramLooking for an experienced Data Engineer with strong expertise in Azure Databricks, PySpark, Spark SQL, Azure Data Factory (ADF), ADLS, ETL, and SQL. You will design scalable data pipelines, build modern Lakehouse solutions, ...
India, Bengaluru
Crescendo Global Leadership Hiring India Pvt Ltd
Consultant Data Engineer Databricks | Azure | PySpark | Spark SQL - Bengaluru / Hyderabad / GurugramLooking for an experienced Data Engineer with strong expertise in Azure Databricks, PySpark, Spark SQL, Azure Data Factory (ADF), ADLS, ETL, and SQL. You will design scalable data pipelines, build modern Lakehouse solutions, ...
India, Gurugram
48k SGD - 84k SGD / year
... (Docker, Kubernetes) - Experience with real-time data processing, anomaly detection, and time-series forecasting in production - Experience working with large datasets and big data technologies like Spark and Kafka to build scalable solutions - A self-starter mentality, with the ability to take ownership of projects from ...
India, Mumbai
Smart Energy Water
Job Description – Data Engineer Experience: 5–7 Years Role: Sr. Data Engineer Department: Data Engineering / Product Engineering Role Overview We are looking for an experienced Data Engineer with 5–7 years of hands-on experience in ETL/ELT, database engineering, Big Data processing, and complex bi-directional data integrations ...
India, Noida
ganit inc.
... Data Engineering Excellence : Design and implement data pipelines using formats like JSON, Parquet, CSV, and ORC, utilizing batch and streaming ingestion. Cloud Data Migration Leadership : Lead cloud migration projects, developing scalable Spark pipelines. Implement Bronze, Silver, and gold tables for scalable data systems. ...
India, Bengaluru
... the ability to thrive in a fast-paced environment. - Clear communication and technical collaboration skills. Preferred Skills (Nice to Have) Familiarity with big data processing tools like Spark or PySpark. Basic understanding of data warehousing concepts and dimensional data modeling. Exposure to version control systems ...
India
... Fabric Implementation: Build data pipelines, notebooks, and Spark jobs within Microsoft Fabric.- Data Ingestion & Integration: Develop ingestion pipelines (Azure Data Factory or Fabric Pipelines) from various sources.- Data Transformation & Modeling: Implement data transformations using Fabric Dataflows Gen2, Spark SQL, and ...
India, India
Rs 22 - 60 lakhs p.a.
... accessibility and usability. Troubleshoot data-related issues and optimize data processing workflows. Skills and Qualifications - Bachelor's degree in Computer Science, Data Science, Statistics, or a related field. - 8-15 years of experience in data analysis or a similar role. - Proficiency in Databricks and Spark for big data processing. ...
India, Chennai
Valuelabs
... Develop highly available data ingestion and processing systems for large datasets. Collaborate with cross-functional teams to deliver data solutions. Ensure data quality, performance, reliability, and scalability. Required Skills: - 7+ years of Data Engineering experience. - Strong experience in Data Modeling, Data Warehousing, ...
India, Rāniganj
KPI PARTNERS INDIA PRIVATE LIMITED
... Python, DatabricksQualifications :- Bachelor's degree in Computer Science, Data Science, Engineering, or a related field.- Proven 6+ years of experience as a Data Engineer, with a focus on ADF, Pyspark, Python, and SQL.- Proficiency in SQL and experience with programming languages such as Python or Scala.- Familiarity with ...
India, Hyderabad
KPI PARTNERS INDIA PRIVATE LIMITED
... Python, DatabricksQualifications :- Bachelor's degree in Computer Science, Data Science, Engineering, or a related field.- Proven 6+ years of experience as a Data Engineer, with a focus on ADF, Pyspark, Python, and SQL.- Proficiency in SQL and experience with programming languages such as Python or Scala.- Familiarity with ...
India, Bengaluru