894 spark jobs in India.

... metadata/catalog solutions, secrets management tools, developer productivity platforms, and ETL orchestration frameworks. Nice-to-Have: - Experience with Hive, Hadoop, Spark, or related big data technologies. Benefits · Competitive salary and performance-based bonuses. · Comprehensive insurance plans. · Collaborative and supportive ...

... Learning models and their https://jobeax.com/link/MJvWAZuK638tHuBb RESPONSIBILITIES : Tech Stack & Skills : - Experience in model development using Python/PySpark libraries. Development on Databricks or Dataiku DSS is a plus.- Strong experience on Spark with Scala/Python/Java.- Proficiency in building, training, and evaluating ...

... Personally design and build end-to-end batch and streaming solutions, including CDC ingestion, Auto Loader, Databricks Workflows/DLT, ADF orchestration, and PySpark/SQL medallion transformations, using realistic enterprise data and production-grade engineering practices. - Review production code and pipeline design at the ...

... working (3 days work from office)- Experience : 5 - 8 Years (Minimum 5 years- AI Data Engineer)- Mandatory Skills : DVC (Data Version Control) and Airflow, Apache Spark, Flink, and Kafka, Advanced level Python and AI logic and Rust (or C++), Vector Database Mastery like configuration of HNSW indexes, scalar quantization, and ...

... secure, and reliable data pipelines and data integration solutions. - Develop, optimize, and support datasets, data models, and transformation processes using SQL, Spark-based technologies, and cloud data platforms. - Support business intelligence, analytics, and reporting solutions, including dashboard and reporting capabilities. ...

... Your skills and experience Must-Have Skills - Strong proficiency in Java and/or Scala for big data application development. - Extensive experience with Apache Spark for large-scale data processing. - Demonstrable knowledge and hands-on experience with Google Cloud Platform (GCP) services, specifically: - Airflow for workflow ...

... relational databases preferably SQL Server, SSIS, SSAS (OLAP Cubes), ADF, - Good programming skills in languages such as R, Python, Scala, Databricks (including spark, delta tables, unity catalog) - Knowledge in data governance, security, architecture - Experience with CI/CD workflows preferably azure devOPS and git - Strong ...

Key Responsibilities :- Design, develop, and maintain scalable ETL pipelines using PySpark.- Build robust data ingestion frameworks for processing data from multiple source systems.- Develop efficient batch and near real-time data processing workflows.- Perform data extraction, transformation, cleansing, validation, and ...

... vector databases, and NLP. - Hands-on experience with cloud AI platforms such as Azure AI, AWS SageMaker, or Google Vertex AI. - Knowledge of data engineering, Spark, Databricks, and data pipelines. - Experience with model deployment, MLOps, CI/CD, and model monitoring. - Proven experience leading data science teams or large-scale ...

... scripts.- Experience in building dashboards and reports using Microsoft Power BI.- Experience in transforming data using Azure Data Factory.- Experience working with Spark and Azure Databricks.- Ability to balance and prioritize multiple conflicting requirements with high attention to detail.- Strong verbal/written communication ...

... solutions in Azure. - Experience building microservices-based cloud architectures at scale. - Experience implementing event-driven architecture using Kafka, Spark, or similar technologies. - Hands-on software development combined with architecture and design experience. - Knowledge of cloud-native architecture, distributed ...

... documentation, implementation, testing, and deployment of data-intensive applications. - Experience with automated data transformation and data curation using dbt, Spark, Spark Streaming, and automated data pipelines. - Experience with workflow management and orchestration tools such as Airflow, AWS Managed Airflow, Luigi, or ...

... engineering, help us optimize and solve some of our greatest challenges! How You'll Make an Impact - Build and own production-grade batch ELT pipelines in Databricks/Spark (15-minute cadence during working hours; hourly off-hours) - Design, scale, and operate our lakehouse using medallion patterns (Bronze/Silver/Gold), including ...

... and data platforms - Develop cloud-native applications on GCP (App Engine, GKE, Cloud Run) - Build event-driven and data-processing pipelines using Kafka and Spark - Contribute to LLM/Agentic AI and RAG-based automation use cases - Drive application performance, resiliency, fault tolerance, observability, and security best ...

... our adaptable work locations and schedules enable us to employ professionals worldwide. Our teams and clients motivate us to deliver exceptional services and spark innovative ideas that fuel our ongoing achievements. With clients from diverse backgrounds and industries, our people encounter intellectual challenges daily. ...

... Comprehensive knowledge of cloud platforms and their AI/ML services. - Strong experience with data architecture, real-time streaming, and big data technologies (Kafka, Spark, Hadoop). Solution Design Expertise : - Mastery of solution architecture methodologies including TOGAF, enterprise architecture frameworks, and design patterns. ...

... Python, R, or Java with experience using libraries like TensorFlow, PyTorch, Scikit-learn, or Keras.- Data Tools : Strong knowledge of SQL/NoSQL databases, Hadoop, Spark, or other big data platforms.- AI/ML Expertise : Hands-on experience with supervised, unsupervised, and deep learning techniques, as well as reinforcement learning. ...

... real-time distributed data processing systems and low-latency, high-throughput pipelines and serving systems. - Hands-on experience with Apache Flink, Kafka, Spark Streaming, or similar streaming technologies. - Strong proficiency in Java-based backend development and distributed system architecture. - Experience deploying, ...

... roles. - Hands-on experience with big data processing frameworks, data lakes, and cloud-native services. Skills: Core Skills: - Proficiency in Python, SQL, and PySpark for data processing and manipulation. - ​​​​​​​Proven experience in Databricks and Apache Spark. - ​​​​​​​Expertise in working with cloud platforms like Azure ...

... strategic vision and implementation of scalable, cloud-native Lakehouse architectures using technologies such as Databricks, Delta Lake, Unity Catalog, DLT, and Spark for high-volume batch and real-time streaming workloads.- MLOps & LLMOps Standardization : Establish standardized, production-grade MLOps frameworks along with ...