894 spark jobs in India.

... helps local businesses scale, and enables global technology companies to push the frontiers of robotics and AI. Backed by world-class investors like Benchmark, Spark Capital, Craft Ventures, Greylock, Y Combinator, and others, we’re looking for exceptional talent to reimagine the way the world works. Role Type: Full-time ...

... migration, testing, and production rollout initiatives. - Troubleshoot and optimize data pipeline performance, reliability, and scalability. Must-Have Skills - Apache Spark (PySpark and/or Scala Spark) - Apache Iceberg - Informatica Data Integration - MinIO Object Storage - Denodo - SQL and Data Modeling - Data Warehousing Concepts ...

... Expert-level hands-on coding experience in Spark/Scala, Python or Pyspark.- In depth understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, RDD caching, Spark MLibT/event-driven/microservices in the cloud.- Deep experience with distributed computing with spark with knowledge of spark runtime.- ...

... critical modules, conduct code reviews, and guide team members on best https://jobeax.com/link/wJIgo45kO3tINlTo Skills : - Strong experience in Azure Databricks, PySpark, and Spark SQL.- Hands-on experience in Azure Data Factory (ADF) and pipeline orchestration.- Strong understanding of Apache Spark architecture, including RDDs ...

... the modernization of the EBV platform and cDB2 migration.- Administer and support platforms leveraging DB2W and DB2U technologies.- Support enablement of EDA Spark/Unified Data Analytics after platform modernization.- Perform ongoing platform administration, monitoring, and operational support.- Handle ad-hoc platform-related ...

... real-time data ingestion and processing pipelines.- Build and optimize ELT/ETL frameworks leveraging Synapse Pipelines, Dedicated SQL Pools, Serverless SQL Pools, Spark Pools, PySpark, and SQL-based transformations.- Drive the implementation of modern data platform architectures, including Data Lakehouse, Medallion Architecture, ...

... platforms. Key Responsibilities: - Build and maintain scalable data pipelines using AWS, Python, SQL, and Databricks. - Work with Snowflake, Apache Airflow, Spark, Kafka, and AWS Kinesis for data processing and orchestration. - Implement containerisation and deployment using Docker and Kubernetes. - Manage CI/CD pipelines ...

... security, and compliance best practices within Databricks, Power BI, and API environments. Technical Skills: - Proficiency in Databricks, including Delta Lake, Spark SQL. - Strong programming skills in Python, Scala, or Java. - Experience with Apache Spark for big data processing. - Knowledge of SQL for querying and transforming ...

... solutions. - Machine Learning Integration: Collaborating with teams to integrate machine learning models into data pipelines for enhanced data processing. - Utilising Spark & Kafka: Implementing and working with Spark and Kafka for real-time data processing and analytics. What we are looking for? - Proficiency in Python, SQL, and ...

Seeking Strong Middle Data Engineers with deep experience in distributed data systems, Spark-based data processing, and production-grade data platform engineering. These engineers will be expected to operate independently, own technical solutions end-to-end, and contribute to both system design and implementation in a highly ...

... Requirements Requirements - Strong programming knowledge in Python, including functions, OOP, exception handling, and data structures. - Hands-on experience with PySpark DataFrames, Spark SQL, transformations, and actions. - Good understanding of Apache Spark architecture and distributed computing. - Strong SQL skills, including ...

... digital and human https://jobeax.com/link/NoCrBgvx6y0zx771 diverse, global teams bring deep industry and functional expertise and a range of perspectives to spark https://jobeax.com/link/qWMhFjcvQeZkpGcV delivers solutions through leading-edge management consulting along with technology and design, corporate and digital ...

... production support.- Batch and streaming pipeline fundamentals.- Orchestration using Airflow, Composer, Control-M, ADF, dbt, or equivalent.- Big data processing using Spark, PySpark, Beam, Dataflow, Dataproc, Databricks, EMR, or equivalent.- Cloud storage and data lake concepts.- Data quality, reconciliation, schema validation, ...

... performance tuning. - Snowflake data modeling. - Apache Airflow experience developing and testing pipelines in Airflow and Python. - Apache Kafka experience. - Apache Spark experience, including hands-on PySpark. - Experience with Iceberg and Parquet data formats. - AWS experience, particularly S3 data lakes, partitioning, and lifecycle ...

... https://jobeax.com/link/V9v5V1Tltw2JcjY6 Responsibilities : - Build services that integrate directly with client products and external vendors.- Work with modern data technologies such as Hadoop, Spark, DBT, Dagster/Airflow, Atlan, Trino, Databricks, Snowflake, and AWS.- Identify, design, and implement automation of manual processes, optimizing data delivery ...

About the Role : We are looking for an experienced Data Engineer with strong hands-on expertise in Python, SQL, Spark/PySpark, and geospatial data processing to build and maintain scalable, production-grade data pipelines. The ideal candidate should have experience working with geospatial/spatial data such as latitude/longitude ...

... support reliable Airflow workflows. The mandatory requirements are 3+ years of Python, advanced SQL, Snowflake data modeling, and hands-on experience with Airflow, Spark/PySpark, and AWS S3. RECRUITMENT PROCESS 1. Application — Share a few details about your experience and background. 2. Coding Challenge — If applicable, complete ...

This is a remote position. SPARK+ is looking for a motivated Junior QA Engineer to join our remote team and contribute to the testing and quality assurance of multiple digital products and platforms, including SOLO Network . SOLO Network is a skills-first digital ecosystem by SPARK+ connecting learning, verified credentials, ...

... experience building analytics platforms; 5+ years hands-on Databricks experience; Databricks certification required - Production expertise with Databricks, Apache Spark/PySpark, Python, SQL, Delta Lake, and scalable ETL/ELT pipelines - Cloud platform experience (Azure, AWS, or Google Cloud) with IAM/RBAC, secrets management, ...

... testing for data pipelines and data platforms. Validate ETL/ELT workflows, data transformations, business rules, and source-to-target data accuracy. Test Apache Spark/PySpark, Databricks, and Delta Lake data processing workflows. Perform advanced SQL-based data validation, reconciliation, and data-quality testing. Validate ...