Back to list

Vacancy description

Lead Software Engineer - Data Engineer in Chennai, भारत

Caterpillar

  • On-siteWork from the office
  • Full-timeStandard weekly hours

Chennai, भारत

Responsibilities

  • Lead and mentor a team of data engineers and platform developers.
  • Architect scalable and secure AWS-based data platforms.
  • Design and develop batch and near-real-time ingestion pipelines for structured and unstructured data.
  • Build and optimize Snowflake data platforms, including data modeling, governance, metadata management, performance, and cost optimization.
  • Use AWS services to implement end-to-end data pipelines.
  • Establish engineering best practices, code quality, testing, version control, CI/CD standards, and reusable ingestion and transformation frameworks.
  • Drive Agile delivery, automation, continuous improvement, scalability, resiliency, and operational readiness.
  • Partner with product managers, subject matter experts, data science and AI teams, and business stakeholders to deliver scalable data solutions.
  • Operationalize ML models and analytics workflows and support real-time analytics, edge AI, and AI-enabled data engineering use cases.

Requirements

  • 10+ years of experience in data engineering or data platform roles.
  • Strong experience with the AWS data ecosystem, including S3, Glue, Lambda, EMR, and Redshift.
  • Deep expertise in Snowflake architecture, optimization, and data modeling.
  • Strong programming skills in Python and SQL.
  • Extensive experience with data ingestion pipelines and ETL/ELT frameworks.
  • Experience with real-time streaming technologies such as Kafka and Spark Streaming.
  • Experience with CI/CD and engineering tools including GitHub, Jenkins, AWS CloudFormation, and Git.
  • Understanding of distributed systems, scalable architectures, software engineering principles, testing, and design patterns.
  • Experience working with Agile teams and leading engineering teams at scale.
  • Bachelor’s or master’s degree in Computer Science or Engineering.
  • Preferred experience with graph databases such as Neo4j and Neptune; vector databases such as Milvus and OpenSearch; and the NVIDIA ecosystem, including RAPIDS, cuDF, cuML, and cuGraph.
  • Preferred experience integrating AI/ML pipelines or GenAI workflows.

Benefits

  • The position requires working onsite five days per week.
  • Relocation is available.
  • The posting is open from November 4, 2026 through November 12, 2026.

Similar vacancies

More like this role

  • IT
... your software engineering career while building impactful technology solutions. Grow your skills and make a difference with a collaborative team. As a Senior Lead Engineer at JPMorgan Chase within the Chief Technology Office team, you will design, develop, and deliver innovative software solutions. You will collaborate ...

Bangalore, भारत

  • Full-time
  • IT
... Experience with Git, CI/CD pipelines, and DevOps practices - Knowledge of Linux environments - Experience with Jira and ITSM tooling - Exposure to cloud-based data platforms Candidate Profile - Medior-level engineer capable of working independently on standard tasks - Able to create and resolve customers' requirements - ...

Bangalore, भारत

  • Full-time
  • IT
Responsibilities - Design, development and testing of components / modules in TOP (Trade Open Platform) involving Spark, Java, Hive and related big-data technologies in a datalake architecture - Contribute to the design, development and deployment of new features & new components in Azure public cloud - Contribute to the ...

Bangalore, भारत

  • Full-time
  • IT
Senior software engineering role at NIKE in Karnataka. Needs 7–10 years and an https://jobeax.com/link/MPrGoo1PSJpZ7d0e, https://jobeax.com/link/2DENLGgQM7whDhaE or B.E.. What you'll do - Lead the design of consumption layers, APIs, and pipelines that BI Experience teams build on — establishing patterns, contracts, and ...

भारत

  • Full-time
  • IT
Senior software engineering role at JPMorgan Chase in Bengaluru. Needs 5+ years. What you'll do - Use domain modeling techniques to build best-in-class business products - Structure software for ease of understanding, testing, and evolution - Build scalable solutions that avoid single points of failure What you need - Java ...

Bangalore, भारत

  • Full-time
  • IT
Senior software engineering role at Danaher in Bengaluru. Needs a https://jobeax.com/link/2DENLGgQM7whDhaE or B.E.. What you'll do - Lead the technical execution of new product development and Product Care projects , ensuring delivery against business, quality, schedule, and technical objectives - Provide technical leadership ...

Bangalore, भारत

  • Full-time
  • IT
... networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Lead Software Engineer Corporate Security Responsibility All activities involving access to Mastercard assets, information, and networks comes with an inherent risk ...

Pune, भारत

  • Full-time
  • IT
... experience with data governance, RLS/ABAC, ontology/semantic layers, dbt, agentic/LLM-powered analytics layers, or large-scale BI platforms. WHAT YOU'LL WORK ON As a Lead Software Engineer in Marketplace Data Engineering, you will own the technical direction of a data product squad serving Nike Direct, Wholesale Demand, Inventory, ...

भारत

  • Full-time
... you'll do - Design, develop, and maintain robust, scalable, and efficient data pipelines on Google Cloud Platform (GCP) - Work extensively with BigQuery (BQ) for data warehousing, query optimization, and large-scale data analysis - Build and manage big data processing workflows using Dataproc (Hadoop/Spark ecosystem on GCP) ...

Chennai, भारत

  • On-site
  • Full-time
  • Contract
... transaction is a sophisticated technology ecosystem that processes billions of payment events with speed, resilience, and precision. As a Lead Data Engineer within the Data Collection & Engineering (DC&E) organisation, you will play a critical role in designing, building, and scaling Mastercard's next-generation data platforms and ...

Hyderabad, भारत