We are looking for a skilled Data Engineer with strong experience in Databricks and Azure Cloud to design, develop, and optimize scalable data pipelines and modern data platforms. The ideal candidate should have expertise in big data technologies, ETL development, and cloud-based data engineering.
Key Responsibilities
Design and develop scalable ETL/ELT pipelines using Databricks and PySpark .
Build and optimize data ingestion frameworks from multiple data sources.
Develop data pipelines using Azure Data Factory (ADF) .
Work with Azure Data Lake Storage (ADLS Gen2) for storing and processing large datasets.
Implement Delta Lake and Lakehouse architecture concepts.
Optimize Spark jobs for performance and scalability.
Develop data transformations using SQL and PySpark.
Collaborate with data architects, analysts, and business stakeholders.
Implement CI/CD pipelines and deployment automation.
Ensure data quality, security, and governance standards.
Required Skills
Strong hands-on experience with Databricks
Proficiency in PySpark and Spark SQL
SSIS, Essbase, SQL Server, Data Warehouse, Stored Procedure, Views Triggers
Experience with Microsoft Azure
Good knowledge of:
Azure Data Factory (ADF)
Azure Data Lake Storage (ADLS)
Delta Lake
Azure Synapse Analytics
Strong SQL and data modeling skills
Experience with Git and CI/CD processes
Knowledge of performance tuning and optimization
Preferred Skills
Experience with:
Streaming using Spark Structured Streaming
Kafka/Event Hub
Data Warehousing concepts
DevOps and Infrastructure as Code
Knowledge of:
Data Governance
Data Quality frameworks
Medallion Architecture (Bronze, Silver, Gold)
Educational Qualification
Bachelor's degree in Computer Science, Information Technology, or related field.
Relevant Azure or Databricks certifications are preferred.
Nice to Have Certifications
Databricks Certified Data Engineer Associate
Microsoft Certified Azure Data Engineer Associate
Requirements
Required Skills
Strong hands-on experience with Databricks
Proficiency in PySpark and Spark SQL
SSIS, Essbase, SQL Server, Data Warehouse, Stored Procedure, Views Triggers
Job Summary : We are seeking a Senior Databricks Engineer with 6 to 10 years of data engineering experience to support a large-scale analytics modernization and refactoring initiative. This role requires a highly independent engineer who can operate effectively with high-level direction, decompose ambiguous objectives into ...
Job Summary : We are looking for an experienced Lead Databricks Engineer with strong expertise in Azure Databricks, PySpark, Python, SQL, Azure Data Factory (ADF), and Delta Lake to develop and optimize scalable data engineering https://jobeax.com/link/XnflZD2dGSZYgXFG Responsibilities : - Design, develop, and maintain ...
Role Overview : We are seeking a hands-on Databricks Platform Engineer to provide offshore operational support, platform administration, automation, monitoring, and continuous improvement for an enterprise Databricks Lakehouse platform on https://jobeax.com/link/bGlnCchIlVM0MSjQ engineer will work closely with the onshore ...
Role Overview : We are seeking a hands-on Databricks Platform Engineer to provide offshore operational support, platform administration, automation, monitoring, and continuous improvement for an enterprise Databricks Lakehouse platform on https://jobeax.com/link/bGlnCchIlVM0MSjQ engineer will work closely with the onshore ...
... capability and contribute to certification and enablement plans for engineers joining the https://jobeax.com/link/nJVhOsAxVRxcpcPH Candidate :- Strong hands-on Databricks Architect/Databricks Engineer Profile with end-to-end build-and-solution capability and Databricks professional https://jobeax.com/link/fcwhEFe0vmiWsaxL Experience ...
Role Overview : We are looking for an experienced Databricks Platform Engineer to manage, maintain, secure, and optimize enterprise-scale Databricks environments hosted on AWS. The ideal candidate will have strong expertise in Databricks administration, Unity Catalog, AWS infrastructure, Terraform automation, platform operations, ...
Role Overview : We are looking for an experienced Databricks Platform Engineer to manage, maintain, secure, and optimize enterprise-scale Databricks environments hosted on AWS. The ideal candidate will have strong expertise in Databricks administration, Unity Catalog, AWS infrastructure, Terraform automation, platform operations, ...
... We are seeking a hands-on Databricks Platform Engineer to support the administration, operation, monitoring, and continuous improvement of our enterprise Databricks platform. The ideal candidate will have experience with Databricks administration, data engineering concepts, platform observability, and technical troubleshooting. ...
... https://jobeax.com/link/QvUCu2JBmJS3gG3G Modernization & Data Engineering : - Participate in Databricks modernization initiatives on AWS.- Support migration of legacy SAS workloads to Databricks, Python or PySpark.- Develop and optimize data pipelines using Databricks, Spark, SQL and Python.- Collaborate with architecture teams to evaluate workloads ...