Apexon - Lead Data Engineer - Azure/Big Data Platform in Bangalore, भारत

We are looking for a Data Engineer with 6 - 12 years of experience in building and supporting scalable data engineering solutions across Azure and Big Data https://jobeax.com/link/UuxHBYkbQjke5gQz ideal candidate should have strong hands-on experience with Azure Databricks, PySpark, Hadoop, Talend, Delta Lake, and Azure Data Factory (ADF).The role will involve developing, integrating, and optimizing data pipelines that support analytics, reporting, and AI/ML initiatives, while working closely with data architects, engineers, and business https://jobeax.com/link/YhqoS1DeSF1mXc3U Responsibilities :- Develop and maintain scalable data pipelines using Azure Databricks and PySpark.- Build data transformation and processing solutions using Delta Lake and Lakehouse architecture.- Develop and optimize ETL/ELT pipelines using Talend for large-scale data ingestion and transformation.- Work with Hadoop ecosystem technologies for distributed data processing and storage.- Integrate data from multiple sources including databases, APIs, files, and streaming platforms.- Build and manage data orchestration workflows using Azure Data Factory (ADF).- Optimize Spark jobs, Databricks clusters, and data pipelines for performance and cost efficiency.- Implement data quality, validation, security, and governance practices.- Collaborate with architects, business stakeholders, and data teams to understand requirements and deliver technical solutions.- Participate in code reviews, technical discussions, and implementation of engineering best practices.- Work with DevOps teams on CI/CD and deployment automation for data engineering solutions.- Troubleshoot and resolve data pipeline and production issues.Must-Have Skills :- 6 - 12 years of experience in Data Engineering / Big Data Engineering.- Strong hands-on experience with Azure Databricks and PySpark.- Strong experience with Hadoop and its ecosystem.- Hands-on experience with Talend for ETL/data integration.- Good experience with Delta Lake / Lakehouse architecture.- Strong experience with Azure Data Factory (ADF).- Strong SQL and data transformation skills.- Experience working with large-scale datasets and distributed data processing.- Understanding of cloud-based data engineering and data integration concepts.- Exposure to CI/CD and DevOps practices. (ref:hirist.tech)