Antino - Senior Data Engineer - Databricks/Azure in Pune, भारत

Role Overview :We are seeking a Senior Data Engineer to drive cloud data modernization and build scalable, AI-ready data platforms. This role emphasizes expertise in Databricks, PySpark, Azure Data Factory, Logic Apps, and Airflow, with a strong focus on orchestration, pipeline reliability, and end-to-end data workflow https://jobeax.com/link/8l1HKAPsCuIcTvHp Responsibilities :- Design, build, and maintain scalable data pipelines using Databricks (PySpark), ADF, Logic Apps, and Airflow- Develop and manage end-to-end orchestration frameworks integrating Airflow DAGs with ADF and Logic Apps- Implement advanced workflow orchestration patterns (event-driven, micro-batch, hybrid scheduling)- Ensure pipeline dependency management, execution reliability, and operational excellence- Build high-performance ETL/ELT pipelines using Databricks and Delta Lake architecture- Implement data observability, monitoring, and alerting mechanisms across workflows- Optimize pipelines and workflows for performance, scalability, and cost efficiency- Integrate pipelines with Azure services such as ADLS Gen2, Blob Storage, and event triggers- Implement CI/CD for pipelines and workflows using GitHub Actions or equivalent- Ensure data quality, governance, and security compliance- Collaborate with data science teams for ML/GenAI data readiness- Mentor junior engineers and drive best practices for orchestration and pipeline https://jobeax.com/link/4TPJYhFtX2QN1Vr3 Qualifications :- 7+ years of experience in data engineering- Highly proficient in Databricks (PySpark, Delta Lake), Azure Data Factory (ADF), Azure Logic Apps, and Apache Airflow- Exposure to AI/ML and Llm data pipelines and MLflow- Strong programming skills in Python and advanced SQL- Experience in building orchestrated data platforms with multi-tool integration- Strong understanding of ETL/ELT patterns and orchestration frameworks- Experience in cloud-native data architectures (Azure preferred)- Hands-on experience with monitoring, logging, and pipeline reliability (ref:hirist.tech)