687 spark jobs in India.

... high-throughput ETL/ELT pipelines that ingest data from various sources into our Lakehouse. Code Quality & Tooling: Drive the adoption of dbt for transformation and PySpark for heavy lifting. You will be responsible for writing modular, reusable code that follows strict CI/CD practices. Implement 'Data SLAs.' You will build the ...

... https://jobeax.com/link/LsX0MHwADPrlj8n9 technical skills in data profiling, analysis, and data management using modern tools and environments (Python, R, SQL, Spark, cloud platforms) Understanding of data lineage concepts and experience with lineage analysis, metadata management, and data cataloging Excellent communication ...

... pipelines integrating sources like ERP systems, IoT devices, iHistorian, web services, and external APIs Leverage modern data engineering stacks (Python, SQL, Spark, Delta Lake, Databricks, Azure) to enable large-scale, high-performance processing Collaborate with stakeholders and IT teams to deliver impactful, business-aligned ...

... performance tuning and query optimization- Good understanding of programming concepts with Python- Strong hands-on experience in implementing data pipelines using PySpark or Spark Scala- Hands-on experience with Databricks, including development, optimization, and management of large-scale data processing and analytics workload- ...

... development. Familiarity with cloud platforms like AWS, Azure, or GCP. Knowledge of data engineering best practices and scalable architectures. Experience with Apache Spark or PySpark. Knowledge of Docker, Kubernetes, and CI/CD pipelines. Experience with data governance and metadata management. Exposure to machine learning data ...

... hands-on experience with AWS services including Lambda, Glue ETL, Athena, S3, DynamoDB, Step Functions, EventBridge, SNS, and SQS. Deep experience in Apache Spark (PySpark/Scala) development, unit testing, and performance optimization. Strong Python programming skills using libraries such as pandas, requests, json, and ...

... reliable software delivery. Key Responsibilities: - Hands on experience working on Databricks, creating data pipelines, transformation rules using Unity Catalog, PySpark, SparkSQL - Hands on Databricks Devs – Deployments, CICD, Performance Management - Hands on Databricks Infrastructure like creating resources (Catalogs, Volumes, ...

... architectural guidance across their international footprint. Become an expert in, promote, and recruit contributors for Databricks-inspired open-source projects (Spark, Delta Lake, MLflow, Unity Catalog) across the developer community. 12+ years in data engineering, data science, technical architecture, or a similar pre-sales ...

... machine learning tools such as Scikit-Learn, Pytorch/Tensorflow Experience in Deep Learning applications is a plus Experience with Big Data technologies, e.g.: Spark, Hive, and Presto, and ETL pipelines is a plus Diversity, Equity and Inclusion at Angel One At Angel One, our culture is rooted in Diversity, Equity, and Inclusion. ...

... tooling. You'll help us modernize our batch ETLs and lead our push into streaming and real-time fraud detection. We run on AWS and make extensive use of Scala Spark, dbt, and Terraform. Build and expose the data lake/lakehouse so teams can understand business and product performance and make better decisions. Design and ...

... drive both member satisfaction and business growth. Key Responsibilities: - Oversee daily operations, ensuring a seamless experience for all members - Maintain a sparkling clean, secure, and efficient workspace - Become the friendly face of The https://jobeax.com/link/J4ymvmVu0lkw6kdt, fostering a sense of community, and building ...

... modeling knowledge - Fact/Dimension, Normalization, Medallion https://jobeax.com/link/ZwDzpQ6hJ7LcOcF6 to Have : - Finance, Risk, or Compliance data experience.- Spark / Hive / Hadoop experience.- YAML-based pipeline configuration.- CI/CD and Git workflows.- API integrations.- Experience with large-scale production data platforms. ...

... orchestration tools.- Familiarity with Microsoft Purview (or equivalent) for data governance, cataloging, and lineage, plus working knowledge of Azure security, networking, and identity (Entra ID, Private Link, Key Vault, RBAC).- Hands-on scripting/programming in Python, SQL, and/or Spark, and IaC experience. (ref:hirist.tech)

... title: Data Engineer Location: Bengaluru, India Long-term contract Job Overview: We are seeking a Data Engineer with 3–5 years of hands-on experience in Spark and Spark SQL, strong SQL expertise, and solid batch processing experience. Must have skills – Spark, Spark SQL, at least 5-8 years of working experience with Spark. Very ...

... architectural guidance across their international footprint. - Become an expert in, promote, and recruit contributors for Databricks-inspired open-source projects (Spark, Delta Lake, MLflow, Unity Catalog) across the developer community. What we look for - 12+ years in data engineering, data science, technical architecture, or ...

... Analytics, Lending, or Fintech. - Production-grade fluency in Python and advanced, highly analytical SQL. - Modern Stack Experience: Hands-on experience scaling data workflows over frameworks like Snowflake, Databricks, Spark, dbt, or similar platforms. - ROC-AUC curves, model drift scenarios) to non-technical stakeholders.

... large datasets, distributed computing, and modern data architectures. Key Responsibilities Design, develop, and maintain big data pipelines using Hadoop and Spark Build scalable data processing frameworks using Scala and Spark Develop and optimize Hive queries and SQL scripts for large datasets Work with distributed storage ...

... Quality & Security: SonarQube, TDD (JUnit/Mockito), Secure Coding Practices Frameworks & Libraries: Spring (Boot, Data, Security, Batch, Integration), Apache Spark (RDD, Spark SQL, DataFrames/DataSets) Key Responsibilities: Architect & Build: Design and construct high-throughput, low-latency data processing pipelines using ...

... is being finalised with the client. We expect to work with an on-premises stack along these lines, or close equivalents: - Distributed processing and query — Spark, Trino or equivalent - On-premises object storage — MinIO, HDFS or equivalent - Relational stores — PostgreSQL, Oracle, SQL Server - Orchestration — Airflow ...

... design for analytics and reporting Performance optimization and scalability Preferred Experience Databricks Delta Lake experience Azure Synapse Analytics Python for data engineering Spark SQL optimization Real-time data streaming Data governance and metadata management Agile/Scrum development model CI/CD pipeline experience