Vacancy description
Databricks Architect/Manager - Pharmaceutical/Life Sciences Domain in Bangalore, भारत
ipivot
Bangalore, भारत
Hyderabad & Bengaluru - (1 days at office required in a week).We are looking for an experienced, hands-on Databricks Architect and Data Engineering Manager with 10+ years of experience in designing and delivering scalable data engineering https://jobeax.com/link/5WM6jVJPxXxkaX5M ideal candidate should be a strong individual contributor and technical lead, capable of driving solution architecture, developing reusable Python-based frameworks, designing complex data models, and leading a team of Data https://jobeax.com/link/D2R7aDvMFh9bTP8V Technical Skills & Responsibilities :Azure Databricks Architecture : - Strong hands-on experience with Azure Databricks Lakehouse Architecture, Delta Lake, Unity Catalog, Databricks Workflows/Jobs, and Databricks Asset Bundles (DAB).- Experience designing scalable, secure, and production-ready Databricks https://jobeax.com/link/eVrw94oIwWZMKogD Python & PySpark : - Expert-level proficiency in Python, PySpark, and SQL, with the ability to independently develop, review, troubleshoot, and optimize complex data engineering solutions.- Must be comfortable contributing directly to code https://jobeax.com/link/Ae4zhZuTlbq61pha Python Framework Development : - Proven experience architecting and developing modular, reusable, configuration-driven Python frameworks for data ingestion, transformation, validation, orchestration, and automation.- Strong understanding of metadata-driven design, configurable business rules, extensible components, and reusable execution https://jobeax.com/link/3QmBXTAwu1vALPhM Modeling & Architecture : - Strong experience in conceptual, logical, and physical data modeling, relational and dimensional modeling, schema design, source-to-target mappings, CDC, SCD Type 2, and historical data processing.- Ability to independently design complex data models and develop SQL DDL https://jobeax.com/link/Gj0aQPWHWnoIAEPu Data Engineering : - Experience designing and implementing Bronze, Silver, and Gold data architectures, Delta tables, scalable ETL/ELT pipelines, incremental processing, and enterprise data transformation frameworks.Real-Time & Batch Data Processing : - Hands-on experience building batch and streaming ingestion pipelines using Spark Structured Streaming, Auto Loader, Azure Event Hubs, Delta Change Data Feed (CDF), and related https://jobeax.com/link/g2fj9cg4giLlZJKg Reliability & Data Quality : - Ability to architect production-grade pipelines with robust exception handling, retries, replay, reconciliation, audit logging, data quality validations, and failure recovery https://jobeax.com/link/febNqiy2dmIGGEd7 & Cost Optimization : - Strong experience optimizing Spark workloads, SQL queries, Delta tables, cluster configurations, job execution, and overall Databricks compute https://jobeax.com/link/vpl71I8jlvFAspjD & Engineering Best Practices : - Experience with Git, CI/CD, Databricks Asset Bundles, automated deployments, unit testing, code quality standards, and reusable engineering https://jobeax.com/link/LsGEhTdo6AXLzIRx / Life Sciences Domain : - Good understanding of pharmaceutical industry data and business processes, preferably across Commercial, Sales, Marketing, Patient Services, Clinical, or R&D domains.- Exposure to pharma datasets, data integration challenges, industry-specific business rules, and data governance https://jobeax.com/link/jd7RJPTFkCOnxWn2 Leadership & Team Management : - Ability to lead and mentor Data Engineers, provide technical direction, conduct code reviews, define engineering standards, resolve complex technical challenges, and ensure high-quality project https://jobeax.com/link/1d6UM3meOskpWr8G Design & Stakeholder Collaboration : - Ability to translate functional and business requirements into scalable technical architectures, prepare solution design documents, define integration patterns, and collaborate with business stakeholders, architects, DevOps, QA, and engineering https://jobeax.com/link/b98mNqN2dpm2gqBN Qualifications :- 10+ years of overall IT experience, with significant experience in Data Engineering and Data Architecture.- Strong hands-on experience in Azure Databricks, Python, PySpark, and SQL.- Proven experience architecting and developing reusable, metadata-driven Python frameworks.- Strong expertise in enterprise data modeling and complex data pipeline design.- Experience leading Data Engineering teams while remaining actively involved in hands-on development.- Working knowledge of Pharmaceutical / Life Sciences data and business processes. (ref:hirist.tech)