Azure Databricks Architect - ETL/PySpark in Bangalore,… - Jobeax
Vacancy description
Azure Databricks Architect - ETL/PySpark in Bangalore, भारत
SP Staffing Servcies
Bangalore, भारत
Work Location : Pan IndiaTotal Experience : 8 - 20 yearsRole Overview :Design and implement end-to-end data architecture using Azure Databricks and Microsoft Azure https://jobeax.com/link/v8qrFLRTZgSSorDg Responsibilities :- Architect scalable data lakehouse solutions using Delta Lake and medallion architecture (Bronze, Silver, Gold).- Define data ingestion, transformation, integration, and orchestration strategies.- Develop and optimize complex data pipelines using PySpark, Spark SQL, and Python.- Design and implement ETL/ELT pipelines using Azure Data Factory and Azure Databricks.- Implement data governance, security, access control, and compliance using Unity Catalog, RBAC, and Azure security services.- Optimize Spark workloads, cluster configurations, job performance, and cloud costs.- Integrate Databricks with Azure Data Lake Storage Gen2, Azure Synapse Analytics, and other enterprise data sources.- Establish CI/CD, deployment automation, monitoring, and data quality standards. (ref:hirist.tech)
... quality, security, and reliability.- Mentor engineers and collaborate with analytics, BI, and business https://jobeax.com/link/b98mNqN2dpm2gqBN Skills : - Azure Databricks- PySpark and Python- SQL and query optimization- Azure Data Factory (ADF)- Delta Lake- ETL/ELT pipeline development- Performance tuning and data architecture- ...
... scalable ETL / ELT pipelines using Azure Databricks, PySpark, SQL and Azure services.- Integrate data from SAP S/4HANA and other enterprise applications into Azure Data Lake.- Develop curated data layers using Delta Lake and lakehouse architecture principles.- Implement incremental data processing, pipeline orchestration, ...
... DevOps/GitLab), Git- Unity Catalog, Data Governance & SecurityResponsibilities : - Design, build, and optimize scalable batch and real-time data pipelines on Azure.- Develop ETL/ELT workflows using Databricks and Azure Data Factory.- Implement Lakehouse architectures using Delta Lake and Medallion (Bronze-Silver-Gold) layers.- ...
... hands-on experience with Azure Databricks.- Excellent knowledge of PySpark and Apache Spark.- Strong programming experience in Python.- Strong experience developing ETL/ELT data pipelines.- Hands-on experience with Azure Data Lake Storage (ADLS Gen2).- Good knowledge of Azure data services and cloud architecture.- Strong SQL ...
... Azure Databricks, PySpark, and Spark SQL.- Hands-on experience in Azure Data Factory (ADF) and pipeline orchestration.- Strong understanding of Apache Spark architecture, including RDDs and DataFrames.- Experience in designing and developing scalable data pipelines and ETL/ELT workflows.- Strong SQL skills, including complex ...
... are looking for an experienced Databricks Data Architect to lead the architecture, design, and implementation of enterprise-scale data platforms using the Databricks Lakehouse https://jobeax.com/link/f5FQc9SxrVvnQKkz ideal candidate will have strong expertise in Databricks, PySpark, Delta Lake, Unity Catalog, Azure Databricks, ...
... and deploy high-performance data pipelines using PySpark, Databricks, and Azure Data Factory (ADF).- Optimize and maintain SQL queries, stored procedures, and ETL processes for large-scale data processing.- Work with Azure cloud services (Azure Data Lake, Azure Synapse, Blob Storage, etc.) to build scalable data solutions.- ...
... architectural roadmap activities.- Drive adoption of modern data engineering and cloud-native best https://jobeax.com/link/6wXuMF1RDT966PMt Skills:- Mandatory: Data Architecture, Databricks Lakehouse Architecture, Databricks Workflows / Lakeflow, Delta Lake, Unity Catalog, Databricks SQL, PySpark and Spark SQL, ETL/ELT design, Dimensional ...
Job Description :We are looking for an experienced Azure Data Engineer with strong expertise in Azure Data Factory, Azure Databricks, PySpark, and Snowflake to design, develop, and maintain scalable data https://jobeax.com/link/hEiNRx4W5ZDbjxHN ideal candidate should have hands-on experience in ETL development, big data ...