Azure Databricks Architect - ETL/PySpark in Bangalore, भारत

Work Location : Pan IndiaTotal Experience : 8 - 20 yearsRole Overview :Design and implement end-to-end data architecture using Azure Databricks and Microsoft Azure https://jobeax.com/link/v8qrFLRTZgSSorDg Responsibilities :- Architect scalable data lakehouse solutions using Delta Lake and medallion architecture (Bronze, Silver, Gold).- Define data ingestion, transformation, integration, and orchestration strategies.- Develop and optimize complex data pipelines using PySpark, Spark SQL, and Python.- Design and implement ETL/ELT pipelines using Azure Data Factory and Azure Databricks.- Implement data governance, security, access control, and compliance using Unity Catalog, RBAC, and Azure security services.- Optimize Spark workloads, cluster configurations, job performance, and cloud costs.- Integrate Databricks with Azure Data Lake Storage Gen2, Azure Synapse Analytics, and other enterprise data sources.- Establish CI/CD, deployment automation, monitoring, and data quality standards. (ref:hirist.tech)