Senior Data Engineer - Databricks & AI Solutions in Chennai, India

Role Overview:Seeking a highly skilled Senior Data Engineer with deep expertise in Databricks, Real-Time Data Processing, Lakehouse Architecture, and AI-driven solutions. The role involves designing, developing, and supporting scalable data platforms and streaming pipelines, leveraging modern Databricks capabilities such as Lakeflow, Unity Catalog, and Generative AI https://jobeax.com/link/86DscMukVX8p2VNe Responsibilities:- Design, develop, and maintain end-to-end streaming and batch data pipelines using Databricks and PySpark.- Build, optimize, and manage Delta Lake tables within Unity Catalog.- Develop and support Lakeflow pipelines with incremental processing, CDC, and data quality validations.- Integrate enterprise systems into the Databricks Lakehouse.- Ensure data quality, governance, security, and operational excellence across platforms.- Monitor, troubleshoot, and optimize workflows for reliability and performance.- Collaborate with stakeholders to deliver scalable, business-aligned solutions.- Build and support AI-powered data solutions leveraging LLMs, RAG, and vector search.- Implement CI/CD best practices and maintain deployment pipelines.- Drive deliverables independently, proactively resolving challenges with minimal https://jobeax.com/link/d98s5MNtpPjtjH4I Stack & Requirements :- Hands-on experience with Databricks and modern Lakehouse architecture.- Expertise in PySpark, Spark Structured Streaming, SQL, NoSQL databases, Delta Lake.- Proficiency with Unity Catalog for governance and security.- Experience building AI agents, copilots, and generative AI solutions using LLMs, RAG, and vector search.- Experience with Git-based source control, CI/CD pipelines, Databricks Repos, and Databricks Asset https://jobeax.com/link/7XVLZmR00vDKuYTN Qualifications :- BE/BTech/ME/MTech/MCA only. (ref:hirist.tech)