Azure Databricks Architect - ETL/PySpark in Bangalore,… - Jobeax
Vacancy description
Azure Databricks Architect - ETL/PySpark in Bangalore, भारत
SP Staffing Servcies
Bangalore, भारत
Work Location : Pan IndiaTotal Experience : 8 - 20 yearsRole Overview :Design and implement end-to-end data architecture using Azure Databricks and Microsoft Azure https://jobeax.com/link/v8qrFLRTZgSSorDg Responsibilities :- Architect scalable data lakehouse solutions using Delta Lake and medallion architecture (Bronze, Silver, Gold).- Define data ingestion, transformation, integration, and orchestration strategies.- Develop and optimize complex data pipelines using PySpark, Spark SQL, and Python.- Design and implement ETL/ELT pipelines using Azure Data Factory and Azure Databricks.- Implement data governance, security, access control, and compliance using Unity Catalog, RBAC, and Azure security services.- Optimize Spark workloads, cluster configurations, job performance, and cloud costs.- Integrate Databricks with Azure Data Lake Storage Gen2, Azure Synapse Analytics, and other enterprise data sources.- Establish CI/CD, deployment automation, monitoring, and data quality standards. (ref:hirist.tech)
... quality, security, and reliability.- Mentor engineers and collaborate with analytics, BI, and business https://jobeax.com/link/b98mNqN2dpm2gqBN Skills : - Azure Databricks- PySpark and Python- SQL and query optimization- Azure Data Factory (ADF)- Delta Lake- ETL/ELT pipeline development- Performance tuning and data architecture- ...
... scalable ETL / ELT pipelines using Azure Databricks, PySpark, SQL and Azure services.- Integrate data from SAP S/4HANA and other enterprise applications into Azure Data Lake.- Develop curated data layers using Delta Lake and lakehouse architecture principles.- Implement incremental data processing, pipeline orchestration, ...
... DevOps/GitLab), Git- Unity Catalog, Data Governance & SecurityResponsibilities : - Design, build, and optimize scalable batch and real-time data pipelines on Azure.- Develop ETL/ELT workflows using Databricks and Azure Data Factory.- Implement Lakehouse architectures using Delta Lake and Medallion (Bronze-Silver-Gold) layers.- ...
... Azure Databricks, PySpark, and Spark SQL.- Hands-on experience in Azure Data Factory (ADF) and pipeline orchestration.- Strong understanding of Apache Spark architecture, including RDDs and DataFrames.- Experience in designing and developing scalable data pipelines and ETL/ELT workflows.- Strong SQL skills, including complex ...
... are looking for an experienced Databricks Data Architect to lead the architecture, design, and implementation of enterprise-scale data platforms using the Databricks Lakehouse https://jobeax.com/link/f5FQc9SxrVvnQKkz ideal candidate will have strong expertise in Databricks, PySpark, Delta Lake, Unity Catalog, Azure Databricks, ...
... and deploy high-performance data pipelines using PySpark, Databricks, and Azure Data Factory (ADF).- Optimize and maintain SQL queries, stored procedures, and ETL processes for large-scale data processing.- Work with Azure cloud services (Azure Data Lake, Azure Synapse, Blob Storage, etc.) to build scalable data solutions.- ...
... architectural roadmap activities.- Drive adoption of modern data engineering and cloud-native best https://jobeax.com/link/6wXuMF1RDT966PMt Skills:- Mandatory: Data Architecture, Databricks Lakehouse Architecture, Databricks Workflows / Lakeflow, Delta Lake, Unity Catalog, Databricks SQL, PySpark and Spark SQL, ETL/ELT design, Dimensional ...
Job Description :We are looking for an experienced Azure Data Engineer with strong expertise in Azure Data Factory, Azure Databricks, PySpark, and Snowflake to design, develop, and maintain scalable data https://jobeax.com/link/hEiNRx4W5ZDbjxHN ideal candidate should have hands-on experience in ETL development, big data ...
... experience in Data Engineering, ETL/ELT development, Data Warehousing, and Azure Data Platform services. The ideal candidate should have strong expertise in Azure Data Factory (ADF), Azure Databricks, Azure Synapse, ADLS, Informatica PowerCenter, PySpark, and SQL to build scalable data integration and analytics solutions. ...