582 spark jobs in India.

• Hybrid

... pipelines, and support enterprise-scale data processing workloads. Key Responsibilities Administer and support Cloudera CDP/CDH platforms, including HDFS, Hive, Spark, YARN, Hue, and CDE. Develop, deploy, and optimize PySpark and Python-based data processing solutions. Build and maintain CI/CD pipelines using Jenkins and GitHub/Bitbucket. ...

... customer-facing support or service role (specifically working with European customers). Technical Skills Big Data & Frameworks: Strong hands-on experience with Apache Spark, PySpark (Mandatory), and Spark SQL. Lightweight Data Processing: Practical experience with lightweight engines such as Polars and DuckDB . Skywise Ecosystem: ...

... generation from mapping documents) Databricks & Lakehouse Hands-on experience with Azure Databricks (Delta Live Tables, Unity Catalog preferred) Strong Apache Spark skills (PySpark / Spark SQL) Experience migrating workloads from legacy data warehouse or Synapse environments to a Databricks Lakehouse Ability to re-implement ...

... complete data management & processing systems Experience working with big data and ETL development 3-5 years of experience in big data analytics technologies like Spark/Spark streaming, , Kafka streaming, Elasticsearch, Bigquery etc. Extensive years of experience in RDMBS/ NoSQL databases, Enterprise Messaging Application (Kafka) ...

... concepts, Azure, ADF, Python). Experience with governed data environments and well-defined metric/rule management practices. Preferred Qualifications Experience with Spark/Delta optimization (partitioning, file sizing, performance tuning). Experience in healthcare/Life Sciences is preferred but not required. Knowledge/experience ...

... knowledge of cloud infrastructure such as GCP/AWS preferred - Experience working with large, complex data sets from a variety of sources - Experience with Hadoop, Spark, or similar stack a must - Experience with functional and object-oriented programming, Python or Scala a must - Ability to collaborate with a diverse set of ...

... across our data engineering practices. 4+ years of data engineering experience, building and managing large-scale data pipelines - Strong proficiency in Python, PySpark, Spark, Databricks, Delta Lake, PowerBI and SQL, with hands-on experience using - Azure Data Factory, Azure Pipelines, Azure DevOps, and Git for version control. ...

• Remote
• Hybrid
• On-site
• Full-time
• Contract

48k SGD - 84k SGD / year

... experience) Shift : (GMT+05:30) Asia/Kolkata (IST) Opportunity Type : Remote Placement Type : Full Time Indefinite Contract(40 hrs a week/160 hrs a month) (*Spark, Generative AI models, LLM, rag, AWS, Docker, GCP, Kafka, Kubernetes, Machine Learning, Python, SQL We are looking for a Senior Machine Learning Engineer who ...

... using at least four GCP services among Data Flow, Data Proc, Pub Sub, BigQuery, Cloud Functions, Composer, GCS Proficient hands-on programming experience in Spark/Scala (python/java) Proficient in building production level ETL/ELT data pipelines from data ingestion to consumption Data Engineering knowledge (such as Data ...

... fast paced environment developing Data Engineering solutions in the data and analytics domain. Develop high quality, secure and scalable data pipelines using spark, Scala/ python on Hadoop or object storage. Leverage new technologies and approaches to innovate with increasingly large data sets. Drive automation and efficiency ...

• Hybrid

... prompt lifecycle management Required Skills - 4–8+ years in AI/ML engineering with focus on GenAI applications - Strong expertise in Databricks (Delta Lake, Spark SQL, PySpark) and Snowflake for scalable data/AI solutions - Proficiency in Python (v3.11+), LLM APIs (OpenAI, LangChain, LangGraph) - Hands-on experience with ...

• Remote

... Excellent problem-solving and debugging skills. Strong communication and collaboration skills. Desired Skills: Experience with Scala frameworks like Play, Akka, or Spark. Experience with cloud platforms (AWS, Azure, GCP). Experience with containerization technologies (Docker, Kubernetes). Experience with Agile development methodologies ...

... with REST APIs and backend frameworks (Flask/Django is a plus). Understanding of data structures and algorithms. Good To Have Experience with big data tools (Spark, Hadoop). Knowledge of cloud platforms (AWS, Azure, or GCP). Exposure to machine learning libraries (Scikit-learn, TensorFlow, etc.). Experience with version ...

... models) - Solid experience with Time Series Forecasting techniques and real-world demand planning use cases. - Hands-on experience with Databricks , including Spark, notebooks, and distributed data processing. - Practical exposure to Machine Learning and Deep Learning model development and deployment. - Experience working ...

... including Lakehouse, Synapse, Pipelines, and Dataflows with 7+ years experience.- Strong understanding of Delta Lake, Parquet, and OneLake storage.- Knowledge of Spark Notebooks, KQL, and Data Engineering within Fabric.- Experience with source control integration (Git) and CI/CD pipelines.- Familiarity with Azure Data Factory, ...

... productionization while mentoring fellow team members. 5+ years of experience in data engineering with significant hands-on work in Databricks - Strong proficiency in PySpark, Spark SQL, Delta Lake, and Delta Live Tables - Advanced skills in Python and SQL - Experience with Unity Catalog, cluster administration, and at least one major ...

Rs 22 - 60 lakhs p.a.

... Computer Science, Data Science, Statistics, or a related field. - 8-15 years of experience in data analysis or a similar role. - Proficiency in Databricks and Spark for big data processing. - Strong SQL skills for data querying and manipulation. - Experience with data visualization tools like Tableau or Power BI. - Knowledge ...

... including Lakehouse, Synapse, Pipelines, and Dataflows with 7+ years experience.- Strong understanding of Delta Lake, Parquet, and OneLake storage.- Knowledge of Spark Notebooks, KQL, and Data Engineering within Fabric.- Experience with source control integration (Git) and CI/CD pipelines.- Familiarity with Azure Data Factory, ...

... data science, or related experience - Proficiency (2–4 years) in SQL, R, Python, Excel, etc., for effective data manipulation - Hands-on experience with Snowflake and Spark/Databricks, adept at Query Profiles and bottleneck identification - Apply creative thinking to solve real-world problems using data-driven insights

... minimal https://jobeax.com/link/d98s5MNtpPjtjH4I Stack & Requirements :- Hands-on experience with Databricks and modern Lakehouse architecture.- Expertise in PySpark, Spark Structured Streaming, SQL, NoSQL databases, Delta Lake.- Proficiency with Unity Catalog for governance and security.- Experience building AI agents, copilots, ...