Azure Databricks Engineer - Python/PySpark in Bangalore, भारत

Key Responsibilities :- Design, develop, and maintain scalable data pipelines using Azure Databricks and PySpark.- Build and optimize large-scale data processing solutions using Python and Spark.- Develop robust ETL/ELT pipelines for batch and near-real-time data processing.- Work with Azure Data Lake Storage (ADLS) and other Azure data services.- Implement data transformation, cleansing, validation, and enrichment processes.- Optimize Spark jobs, Databricks notebooks, clusters, and SQL queries for performance and cost efficiency.- Develop reusable and maintainable data engineering frameworks and components.- Implement data quality, governance, security, and monitoring practices.- Work with structured and unstructured datasets across multiple data sources.- Collaborate with solution architects and engineering teams to translate business requirements into technical solutions.- Troubleshoot production issues and perform root-cause analysis.- Implement CI/CD processes for Databricks code and data pipelines.- Follow engineering best practices around version control, testing, documentation, and deployment.- Mentor junior and mid-level data engineers and contribute to technical https://jobeax.com/link/tUNKhTSumiAniKVx Skills & Experience :- 8+ years of experience in Data Engineering, Big Data, or related technologies.- Strong hands-on experience with Azure Databricks.- Excellent knowledge of PySpark and Apache Spark.- Strong programming experience in Python.- Strong experience developing ETL/ELT data pipelines.- Hands-on experience with Azure Data Lake Storage (ADLS Gen2).- Good knowledge of Azure data services and cloud architecture.- Strong SQL skills and experience working with relational and analytical databases.- Experience with Delta Lake and Delta Tables.- Strong understanding of data modeling, data warehousing, and distributed data processing.- Experience with performance tuning and optimization of Spark/Databricks workloads. (ref:hirist.tech)