Vacancy description
India, Bengaluru
Big Data Engineer - Hadoop/Spark/Scala in Bengaluru, India is listed on Jobeax. Browse 30,000+ vacancies available.
Role Overview : We are seeking a seasoned Big Data Engineer to join our high-performing data platform team. In this role, you will be responsible for architecting, developing, and maintaining robust data pipelines that process massive datasets to fuel our analytical engines. You will work closely with cross-functional teams, including data scientists, product managers, and infrastructure engineers, to translate complex business requirements into scalable technical solutions. By optimizing our Hadoop and Spark ecosystems, you will directly influence the speed and reliability of our data-driven decision-making processes, ensuring that our stakeholders have access to high-quality, actionable insights that drive business https://jobeax.com/link/KgVLdM5cThorPSD1 Responsibilities : - Design and implement scalable data pipelines using Scala and Spark to ingest and transform large-scale datasets, ensuring high availability and performance for downstream analytics.- Optimize complex SparkSQL queries and Hadoop jobs to reduce processing latency, directly improving the efficiency of our data infrastructure.- Collaborate with engineering teams to integrate real-time data streams using Kafka, enabling low-latency data availability for critical business applications.- Maintain rigorous data quality standards by implementing automated testing and monitoring frameworks, ensuring reliability for all internal and external data consumers.- Partner with stakeholders to identify data bottlenecks and implement innovative engineering solutions that support the long-term scalability of our Big Data https://jobeax.com/link/1dkmzToQOwmnlB8O Skillset : - Demonstrated expertise in building distributed data systems using Hadoop, Spark, and Scala, with a deep understanding of performance tuning and memory management.- Proven ability to write complex SQL queries and develop efficient data models that support diverse analytical use cases.- Strong proficiency in Python for scripting and automation, complemented by hands-on experience in managing data flows through Kafka.- Exceptional communication skills with the ability to articulate technical concepts to non-technical stakeholders and work effectively within a collaborative, fast-paced environment.- A Bachelors or Masters degree in Computer Science, Engineering, or a related quantitative field, reflecting a strong foundation in data structures and algorithms.- Ability to thrive in a hybrid work model across our Chennai, Bangalore, Pune, or Mumbai offices, demonstrating high levels of self-motivation and adaptability to evolving project requirements.- 5 - 8 years of professional experience in Big Data Engineering, with a track record of delivering high-impact data solutions in enterprise environments. (ref:hirist.tech)
Similar vacancies
... managing large-scale data pipelines - Strong proficiency in Python, PySpark, Spark, Databricks, Delta Lake, PowerBI and SQL, with hands-on experience using - Azure Data Factory, Azure Pipelines, Azure DevOps, and Git for version control. - Proven experience working with Azure Databricks and PySpark/Spark for data engineering ...
India, Delhi
Hexaconcepts
Role Overview:Seeking a highly skilled Senior Data Engineer with deep expertise in Databricks, Real-Time Data Processing, Lakehouse Architecture, and AI-driven solutions. The role involves designing, developing, and supporting scalable data platforms and streaming pipelines, leveraging modern Databricks capabilities such ...
India, Chennai
3SHOOL TECHNOLOGY CONSULTANTS
... Airflow workflows and production data science infrastructureIDEAL PROFILE:Looking for candidates with strong hands-on experience in:- Data Operations- Production Data Engineering- Data Platform Support- Data Pipeline Monitoring & Troubleshooting- Not looking for purely development-focused Data Engineers or DevOps/SRE https://jobeax.com/link/V6ZwWogXLIs5s3d8 ...
India, Pune
3SHOOL TECHNOLOGY CONSULTANTS
... Airflow workflows and production data science infrastructureIDEAL PROFILE:Looking for candidates with strong hands-on experience in:- Data Operations- Production Data Engineering- Data Platform Support- Data Pipeline Monitoring & Troubleshooting- Not looking for purely development-focused Data Engineers or DevOps/SRE https://jobeax.com/link/V6ZwWogXLIs5s3d8 ...
India, Mumbai
HiringBlaze
... and results-driven Data Engineer to join our growing Data Engineering team. You will be instrumental in ingesting, transforming, and delivering high-quality data to enable data-driven decision-making for our clients. This role is ideal for someone who thrives in a consulting environment, enjoys solving complex data challenges, ...
India, Hyderabad
NatWest Group
... in driving value for our customers by building data solutions. You'll be carrying out data engineering tasks to build, maintain, test and optimise a scalable data architecture, as well as carrying out data extractions, transforming data to make it usable to data analysts and scientists, and loading data into data platforms. ...
India, Chennai
JMAN Group
... Experience with containerization technologies (e.g., Docker, Kubernetes) - Experience with CI/CD pipelines and DevOps methodologies. - ETL or ELT: Azure Data Factory, Databricks, Synapse, dbt (any two – Mandatory). Data Warehousing: Azure SQL Server/Redshift/Big Query/Databricks/Snowflake (Anyone - Mandatory). Data Visualization: ...
India, Mannady
$360 USD - $460 USD
... industrial IoT time-series data is preferred.- Experience in Reinforcement Learning, ANN/RNN/LSTM, Gradient Boosted Machines, SVM, Ensemble techniques.- Experience in NoSQL DBs, Hadoop, analytics tools : Power BI, Tableau, etc. , and data pipelines for ML: Apache Spark Streaming, Storm etc., is preferred (ref:hirist.tech)
India, Bengaluru
... patterns and scalable AI capabilities • Mentor engineers on design principles, reliability, and prompt lifecycle management Required Skills - 4–8+ years in AI/ML engineering with focus on GenAI applications - Strong expertise in Databricks (Delta Lake, Spark SQL, PySpark) and Snowflake for scalable data/AI solutions - Proficiency ...
India, Bengaluru
... evaluating and implementing data-engineering and software technologies - In addition, you have experience in programming languages and frameworks: SQL, Python, Spark, Databricks (Delta Lake) - Experience in data storages - SQL and NoSQL databases, Azure Data Lake Storage- and in developing data solutions, models, API and ...
India, Bengaluru
... clean and reliable data. Remote What The Senior Data Engineer Will Own Pipeline Engineering: Build and maintain high-throughput ETL/ELT pipelines that ingest data from various sources into our Lakehouse. Code Quality & Tooling: Drive the adoption of dbt for transformation and PySpark for heavy lifting. You will be responsible ...
India, India
Costco IT
... years’ experience engineering and operationalizing data pipelines with large and complex datasets. - 5-8+ years’ hands-on experience with Data Build Tool (DBT), Dataflow & Dataproc (Spark) - 5-8+ years’ experience working with BigQuery, Google Cloud Storage, AlloyDB and Spanner - 5-8+ years’ experience with Data Pipeline, ...
India, Hyderabad
Impetus Consultrainers
... members. We are seeking experienced Data Engineering professionals with 4–6 years of hands-on expertise in Big Data technologies, with a focus on building scalable data solutions using Google Cloud Platform (GCP). Key Skills & Experience: Strong programming skills in Python and Bash Solid understanding of SQL and data warehousing ...
India, Gurugram
Role Overview : We are seeking a seasoned Data Engineering professional to spearhead our data architecture initiatives and drive the evolution of our analytical ecosystem. In this role, you will be responsible for designing robust data models and optimizing our Snowflake-based data warehousing environment to support complex ...
India, Mumbai
... Shell Scripting. Strong analytical skills for data engineering tasks. Familiarity with tools likeApache Airflowfor data migration workflows. Experience with database systems such asOracle,MySQL, orDB2. Knowledge of legacy data migration from diverse sources. Previous experience with data migration to platforms like ServiceNow. ...
India, India
QUANTIUM ANALYTICS PRIVATE LIMITED
... Python for data engineering workloads.- Demonstrated experience building ETL/ELT pipelines that transform and process large volumes of structured and unstructured data.- Solid understanding of data warehousing concepts and data modelling techniques (e.g. dimensional modelling, slowly changing dimensions).- Hands-on experience ...
India, Hyderabad
Fractal
... bringing passion and customer focus to the business. Data Engineer (data integration and streaming platforms) Data Engineer with strong experience in event-driven data integration and streaming platforms, ideally with hands-on expertise in Confluent Kafka and CDC technologies. Experience Strong Data Engineering background, including ...
India, Bengaluru
Capco Technologies Pvt Ltd
... contributing to large-scale data projects on Google Cloud Platform. You will work closely with senior engineers and data analysts to build and maintain robust data pipelines, helping to turn raw data into actionable https://jobeax.com/link/d6zqLcygnqsI43u7 Responsibilities:- Develop, test, and maintain data pipelines and ...
India, Bengaluru
... focus on Java, Microservices and Kafka Java Microservices Back end Developer Java Sprint boot Microservices with expected knowledge of Kafka, Basic knowledge of Database. Experience with TDD and BDD. Must have experience with one of cloud (GCP, AWS or Azure). Also experience with Kubernetes and containers - Hands-on experience ...
India, Bengaluru
Role Overview : We are seeking a seasoned Data Engineering professional to spearhead our data architecture initiatives and drive the evolution of our analytical ecosystem. In this role, you will be responsible for designing robust data models and optimizing our Snowflake-based data warehousing environment to support complex ...
India, Chennai