Back to list

Vacancy description

Azure Databricks Architect - ETL/PySpark in Bangalore, भारत

SP Staffing Servcies

Bangalore, भारत

Work Location : Pan IndiaTotal Experience : 8 - 20 yearsRole Overview :Design and implement end-to-end data architecture using Azure Databricks and Microsoft Azure https://jobeax.com/link/v8qrFLRTZgSSorDg Responsibilities :- Architect scalable data lakehouse solutions using Delta Lake and medallion architecture (Bronze, Silver, Gold).- Define data ingestion, transformation, integration, and orchestration strategies.- Develop and optimize complex data pipelines using PySpark, Spark SQL, and Python.- Design and implement ETL/ELT pipelines using Azure Data Factory and Azure Databricks.- Implement data governance, security, access control, and compliance using Unity Catalog, RBAC, and Azure security services.- Optimize Spark workloads, cluster configurations, job performance, and cloud costs.- Integrate Databricks with Azure Data Lake Storage Gen2, Azure Synapse Analytics, and other enterprise data sources.- Establish CI/CD, deployment automation, monitoring, and data quality standards. (ref:hirist.tech)

Similar vacancies

More like this role

Hawk Sense Business Solution pvt. ltd.

Azure Data Engineer - PySpark/Databricks

Role : Azure Data EngineerExperience : 5.5 - 8.5 YearsMandatory Skills : - Python- PySpark- Databricks- Azure Services - ADLS, Azure Data Factory (ADF), Azure Databricks (ADB)- Advanced SQLKey Responsibilities : - Design, develop, and maintain scalable ETL/ELT data pipelines.- Ensure data quality, reliability, and timely ...

Bangalore, भारत

Calfus Technologies India Pvt Ltd

Calfus - Lead Data Engineer

... requirements and translate them into technical specifications and architecture https://jobeax.com/link/SH3XIHB7XPgGBBvi Integration & Pipelines : - Build and oversee ETL/ELT pipelines using Azure Data Factory, Airflow, SSIS, or equivalent tools.- Use Azure Databricks and PySpark for large-scale data processing.- Own pipeline reliability ...

Pune, भारत

ARES OPERATIONS INDIA PRIVATE LIMITED

Ares Management - Senior Associate - Data Engineering

... enterprise-scale data pipelines using Azure Databricks, PySpark, Delta Lake, Python, SQL, and Azure Data Factory, including both batch and streaming workloads.- Hands-on Databricks experience, including notebooks, workflows/jobs, Spark concepts, Delta Lake patterns, and production support.- Python and/or PySpark for transformation, validation, ...

Bombay, भारत

Merrill Communications

Senior Data Engineer

  • Full-time
Job Description: Responsibilities: - Develop & Optimize Data Pipelines - Build, test, and maintain ETL/ELT data pipelines using Azure Databricks & Apache Spark (PySpark) . - Optimize performance and cost-efficiency of Spark jobs. - Ensure data quality through validation, monitoring, and alerting mechanisms. - Understand ...

Chennai, भारत

  • Full-time
... Develop and maintain backend services and REST APIs using Python frameworks such as FastAPI, Flask, or Django What you need - Python - SQL - Machine learning - ETL - Databricks - Spark Good to have - Azure Data Factory - LLMs - Airflow - Kafka - Azure - Docker Job details - Location: Hyderabad - Experience: 12–17 years - ...

Hyderabad, भारत

ScaleupAlly

Data Engineer

  • Full-time
... support. - Design data models (dimensional SCD Medallion Bronze/Silver/Gold) that match how the business actually uses the data. - Build and maintain scalable ETL/ELT pipelines using Python SQL PySpark and Azure Data Factory on Azure Databricks or Microsoft Fabric. - Implement batch incremental/CDC and streaming loads and ...

Noida, भारत

  • Full-time
  • IT
... you'll do - Build backend APIs and scalable data pipelines (Python, PySpark) - Work with modern data lakehouse/warehouse tech (Iceberg, Delta Lake, Snowflake, Databricks) - Orchestrate workflows (Airflow) and optimize big data frameworks What you need - Python - ETL - Spark - Snowflake - Databricks - Airflow Good to have - ...

Bangalore, भारत

  • Full-time
  • IT
  • QA
... experience with ETL concepts & technologies, Data migration testing, Data warehouse testing, and BI Reporting. Hands on experience in working with CSV and JSON using PySpark. High level of Proficiency in writing complex SQL queries in Azure data bricks from Business requirements to read data from source DB hosted in Azure and compare ...

Pune, भारत

  • IT
  • QA
... complex joins and CTEs for deep data validation.- ETL & DWH : Deep understanding of ETL and Data Warehouse https://jobeax.com/link/foMcZv4qpsOyQhfA to have :- Databricks Experience : Hands-on experience with Databricks Notebooks and cluster management.- Programming : Strong skills in Python, specifically PySpark and Pandas ...

Pune, भारत

  • IT
  • QA
... complex joins and CTEs for deep data validation.- ETL & DWH : Deep understanding of ETL and Data Warehouse https://jobeax.com/link/foMcZv4qpsOyQhfA to have :- Databricks Experience : Hands-on experience with Databricks Notebooks and cluster management.- Programming : Strong skills in Python, specifically PySpark and Pandas ...

Indhur, भारत

AMNET DIGITAL PRIVATE LIMITED

Data Architect

  • Full-time
... data warehousing and analytics. - Strong hands-on architecture expertise in Snowflake and/or Databricks . - Excellent SQL skills and experience with Python PySpark or equivalent data processing technologies. - Strong understanding of cloud platforms such as Azure AWS or GCP. - Expertise in data modeling dimensional modeling ...

Hyderabad, भारत