Job Description – Lead/Senior Databricks Engineer

Experience: 8+ Years

Location: Pune / Hyderabad (Hybrid)

Employment Type: Full-Time

Role Summary

We are looking for a skilled Data Engineer with strong experience in Databricks and Azure Cloud to design, develop, and optimize scalable data pipelines and modern data platforms. The ideal candidate should have expertise in big data technologies, ETL development, and cloud-based data engineering.

Key Responsibilities

Design and develop scalable ETL/ELT pipelines using Databricks and PySpark .
Build and optimize data ingestion frameworks from multiple data sources.
Develop data pipelines using Azure Data Factory (ADF) .
Work with Azure Data Lake Storage (ADLS Gen2) for storing and processing large datasets.
Implement Delta Lake and Lakehouse architecture concepts.
Optimize Spark jobs for performance and scalability.
Develop data transformations using SQL and PySpark.
Collaborate with data architects, analysts, and business stakeholders.
Implement CI/CD pipelines and deployment automation.
Ensure data quality, security, and governance standards.

Required Skills

Strong hands-on experience with Databricks
Proficiency in PySpark and Spark SQL
SSIS, Essbase, SQL Server, Data Warehouse, Stored Procedure, Views Triggers
Experience with Microsoft Azure
Good knowledge of:

Azure Data Factory (ADF)
Azure Data Lake Storage (ADLS)
Delta Lake
Azure Synapse Analytics

Strong SQL and data modeling skills
Experience with Git and CI/CD processes
Knowledge of performance tuning and optimization

Preferred Skills

Experience with:

Streaming using Spark Structured Streaming
Kafka/Event Hub
Data Warehousing concepts
DevOps and Infrastructure as Code

Knowledge of:

Data Governance
Data Quality frameworks
Medallion Architecture (Bronze, Silver, Gold)

Educational Qualification

Bachelor's degree in Computer Science, Information Technology, or related field.
Relevant Azure or Databricks certifications are preferred.

Nice to Have Certifications

Databricks Certified Data Engineer Associate
Microsoft Certified Azure Data Engineer Associate

Requirements

Required Skills

Strong hands-on experience with Databricks
Proficiency in PySpark and Spark SQL
SSIS, Essbase, SQL Server, Data Warehouse, Stored Procedure, Views Triggers
Experience with Microsoft Azure
Good knowledge of:

Azure Data Factory (ADF)
Azure Data Lake Storage (ADLS)
Delta Lake
Azure Synapse Analytics

Strong SQL and data modeling skills
Experience with Git and CI/CD processes
Knowledge of performance tuning and optimization

Databricks Engineer

Start with your email