Azure Data Engineer - Python/Databricks in Chennai, भारत

Azure Data EngineerKey Responsibilities:- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF).- Build and optimize data processing workflows using Azure Databricks and Python.- Develop ETL/ELT processes to extract, transform, and load data from multiple sources.- Work with structured and unstructured data to support analytics and reporting requirements.- Implement data transformation, cleansing, validation, and integration processes.- Optimize data pipelines and Spark jobs for performance, scalability, and cost efficiency.- Integrate data from various sources into Azure-based data platforms.- Ensure data quality, consistency, reliability, and security across data workflows.- Monitor and troubleshoot data pipelines, jobs, and production issues.- Collaborate with data analysts, data scientists, architects, and business stakeholders to understand data requirements.- Follow best practices for data governance, access management, documentation, and deployment.- Support automated deployment and version control processes for data engineering https://jobeax.com/link/qVQYcn8zRo6FsrUo Skills and Qualifications:- 5 - 10 years of relevant experience in data engineering or related technical roles.- Strong hands-on experience with Microsoft Azure data services.- Proficiency in Python for data processing and automation.- Hands-on experience with Azure Data Factory (ADF) for building and managing data pipelines.- Strong experience with Azure Databricks for data transformation and processing.- Understanding of ETL/ELT concepts, data integration, and data warehousing.- Knowledge of SQL and database concepts.- Experience with data pipeline monitoring, troubleshooting, and performance optimization.- Strong analytical, problem-solving, and communication skills. (ref:hirist.tech)