Eucloid - Lead Data Engineer in Chennai, भारत

Job Description :We are looking for a Lead Data Engineer to join our Data Platform team and take ownership of designing, developing, and scaling modern data platforms for enterprise https://jobeax.com/link/iiI55QEBsS5uFv8R role involves leading data engineering initiatives across technology evaluation, architecture, pipeline design, data integration, performance optimization, and platform https://jobeax.com/link/xr65F3iaCgiH6SkB ideal candidate will be responsible for translating complex business and technical requirements into scalable data solutions while providing technical direction to data engineering https://jobeax.com/link/3IVZ3ngA7H22gp6w role requires strong expertise in cloud data platforms, distributed data processing, data architecture, ETL/ELT, and data quality, along with the ability to work closely with architects, analysts, application teams, and business https://jobeax.com/link/YhqoS1DeSF1mXc3U Responsibilities :- Lead the architecture and design of scalable, secure, and high-performance data engineering solutions for complex business requirements.- Own end-to-end data platform initiatives, from technology evaluation and solution design to implementation, deployment, and production support.- Define data ingestion, transformation, processing, storage, and consumption strategies across enterprise data ecosystems.- Design and optimize large-scale batch and streaming data pipelines integrating data from multiple internal and external sources.- Establish reusable data engineering frameworks, standards, design patterns, and best practices across projects.- Lead the development of cloud-based data platforms leveraging AWS services such as S3, Redshift, EMR, Athena, Kinesis, and Lambda.- Drive the design and implementation of modern data architectures using technologies such as Databricks and Snowflake.- Provide technical leadership for ETL/ELT development, data warehousing, data lake, and data lakehouse initiatives.- Design robust mechanisms for handling schema evolution, schema drift, data dependencies, and source-system changes.- Establish data quality, validation, reconciliation, monitoring, and observability frameworks to ensure reliability of critical data pipelines.- Define strategies for pipeline performance optimization, scalability, availability, and cost efficiency.- Lead integration of data from supplier systems, landing zones, APIs, databases, files, and other heterogeneous sources.- Evaluate and recommend appropriate technologies, tools, and frameworks based on scalability, performance, maintainability, and business requirements.- Design workflow orchestration and dependency management using tools such as Airflow, Azkaban, or Luigi.- Guide implementation of ETL solutions using platforms such as Talend, Informatica, or equivalent technologies.- Work with NoSQL and relational databases to design appropriate data storage and processing solutions.- Establish engineering practices around version control, CI/CD, testing, deployment, monitoring, and production support.- Lead root-cause analysis for complex data platform and pipeline issues and drive permanent solutions.- Collaborate with software engineers, solution architects, database architects, data scientists, analysts, and business stakeholders.- Mentor data engineers and provide technical guidance on architecture, coding standards, design reviews, and engineering best practices.- Review technical designs and code to ensure adherence to scalability, reliability, security, and maintainability standards.- Drive continuous improvement and modernization of existing data platforms and engineering processes.- Ensure technical documentation, architecture decisions, data flows, and operational processes are maintained https://jobeax.com/link/3qFj8GYvD4BVGxx7 Expertise :- 7+ years of experience in Data Engineering, Data Platforms, Big Data, or related technology roles.- Strong hands-on experience designing and implementing enterprise-scale data engineering solutions.- Strong expertise in AWS data services such as S3, Redshift, EMR, Athena, Kinesis, and Lambda.- Strong understanding of data warehousing, data lakes, data lakehouse architecture, ETL/ELT, and distributed data processing.- Experience working with Databricks and/or Snowflake is highly preferred.- Strong experience building and optimizing large-scale data pipelines and data processing frameworks.- Experience with workflow orchestration tools such as Airflow, Azkaban, or Luigi.- Good experience with ETL tools such as Talend, Informatica, or equivalent platforms.- Experience working with relational and NoSQL databases.- Strong understanding of data modelling, metadata management, data lineage, dependency management, and data quality.- Experience handling schema evolution, schema drift, data reconciliation, and data validation.- Strong understanding of pipeline monitoring, production support, performance tuning, and operational reliability.- Experience with CI/CD, Git, automated testing, and deployment practices for data engineering environments.- Strong understanding of cloud-native data architecture and scalable distributed https://jobeax.com/link/7vfq0o2YhszrmAf5 & Stakeholder Management :- Ability to lead technical discussions and drive architecture and design decisions.- Strong problem-solving skills with the ability to resolve complex data engineering challenges.- Experience mentoring and guiding data engineering teams.- Ability to conduct design and code reviews and establish engineering standards.- Strong communication skills with the ability to explain complex technical concepts to both technical and non-technical stakeholders.- Ability to work independently and take ownership of large-scale data engineering initiatives.- Strong stakeholder management skills with experience working directly with clients and cross-functional teams (ref:hirist.tech)