Vacancy description

Lead Data Engineer in Pune, भारत

Mastercard

  • HybridMix of office and remote
  • Full-timeStandard weekly hours

Pune, भारत

Our Purpose Mastercard powers economies and empowers people in 200+ countries and territories worldwide. Together with our customers, we’re helping build a sustainable economy where everyone can prosper. We support a wide range of digital payments choices, making transactions secure, simple, smart and accessible. Our technology and innovation, partnerships and networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Lead Data Engineer Position Overview Have you ever wanted to be part of something BIG? Now is the time to make an immediate impact at a leading global technology company, Mastercard. This role is part of the AI&DPE Data Engineering platform team, responsible for building and evolving large scale backend data systems, real time and batch pipelines, and AI enabled services that power analytics, decisioning, and automation across the organization. This is a backend engineering role, focused on Scala/Python/Java, distributed data platforms (Cloudera/Spark), and cloud based architectures, with a strong emphasis on AI agent creation and intelligent automation. Visualization tools (e.g., Qlik) are consumers of the platform, not the core focus of this role. You will work with massive transactional datasets, modern big data and cloud platforms, and AI driven workflows to transform how Mastercard processes, enriches, and operationalizes data at global scale. PRIMARY RESPONSIBILITIES Backend & Data Platform Engineering

  • Design, develop, and maintain backend services and data pipelines using Scala, Python and Java
  • Build and optimize batch and streaming workloads on Cloudera Data Platform (CDP) using Spark
  • Work with Cloudera Manager to support platform configuration, monitoring, performance tuning, and operational stability
  • Design and implement high quality datamarts and curated datasets with strong emphasis on data integrity, performance, and reliability
AI Agents & Intelligent Automation
  • Design, build, and integrate AI powered agents that operate to support:
  • Anomaly detection and operational intelligence
  • workflow automation
  • Apply generative AI driven, or rule based agents to reduce manual effort and improve scalability across backend systems
Cloud & Modern Data Architecture
  • Build and support data and compute workloads in AWS environments
  • Leverage Databricks for large scale data processing, advanced analytics
  • Contribute to cloud native and hybrid architectures integrating on prem and cloud platforms
Integration & Downstream Enablement
  • Enable downstream consumers (analytics, visualization, reporting tools such as Qlik) through well designed, reliable backend data interfaces
  • Partner with analytics, fraud, and business teams to ensure backend systems meet evolving needs without compromising platform stability
Documentation & Collaboration
  • Create clear technical documentation, including architecture diagrams, data flows, and design specifications
  • Participate in Agile/Scrum ceremonies and cross functional design reviews
  • Mentor and upskill team members in backend engineering, big data, and AI agent concepts
KNOWLEDGE AND SKILL REQUIREMENTS Required
  • BS/BA degree in Computer Science, Engineering, Information Systems, or related field
  • Strong handson experience with Scala,Python/Java in backend or data intensive systems
  • Experience working with Cloudera Data Platform (CDP) and Spark
  • Familiarity with Cloudera Manager for cluster administration, monitoring, or troubleshooting
  • Strong understanding of data modeling concepts, distributed systems, and large scale data processing
  • Excellent problem solving skills and ability to work independently in complex environments
GOOD TO HAVE / STRONGLY PREFERRED
  • Hands on experience building, integrating, or supporting AI driven agents or intelligent automation solutions
  • Experience with Databricks for data engineering or ML workloads
  • Experience working in AWS (e.g., S3, EC2, EMR, Glue, Lambda, IAM, or equivalent services)
  • Knowledge of streaming and big‑data technologies:
  • Kafka
  • Hadoop ecosystem
  • Hive/Impala
  • Exposure to model monitoring, or AI platform enablement
  • Experience with ETL tools such as Informatica
  • Experience working in Agile / Scrum teams within large enterprises
Corporate Security Responsibility All activities involving access to Mastercard assets, information, and networks comes with an inherent risk to the organization and, therefore, it is expected that every person working for, or on behalf of, Mastercard is responsible for information security and must:
  • Abide by Mastercard’s security policies and practices;
  • Ensure the confidentiality and integrity of the information being accessed;
  • Report any suspected information security violation or breach, and
  • Complete all periodic mandatory security trainings in accordance with Mastercard’s guidelines.

Similar vacancies

More like this role

  • Hybrid
... organization. What will you do? - You will be the lead data engineer on our greenfield data platform and key contributor to its success. - Embed AI in all phases of data lifecycle. - Design, develop and own robust data pipelines, ensuring optimal performance, scalability, and maintainability. - Design and implement Data Lake, ...

Gurgaon, भारत

Referrals Only

Lead Data Engineer

  • Hybrid
  • Full-time
Lead data engineers at Thoughtworks develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. They might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to ...

Bangalore, भारत

  • On-site
  • Full-time
  • Contract
... transaction is a sophisticated technology ecosystem that processes billions of payment events with speed, resilience, and precision. As a Lead Data Engineer within the Data Collection & Engineering (DC&E) organisation, you will play a critical role in designing, building, and scaling Mastercard's next-generation data platforms and ...

Hyderabad, भारत

Dr Reddy's Laboratories Limited

Lead Data Engineer

  • Full-time
  • Contract

24k

... and diversity in our organization. " Job Description Role Summary We are looking for a Lead Data Engineer to design, build, and modernize DRL's cloud-native data platforms on Google Cloud. The role goes beyond pipeline development: the candidate will own data architecture and modeling, lead the move from legacy ETL platforms ...

Hyderabad, भारत

Calfus Technologies India Pvt Ltd

Calfus - Lead Data Engineer

About the Role : As a Lead Data Engineer, you'll own the design and delivery of our cloud data platforms on AWS and Azure. You'll architect lakehouse solutions built on Medallion Architecture, set the data modelling standards, and lead a team of data engineers delivering reliable, scalable pipelines. This is a hands-on ...

Pune, भारत

  • Full-time
... Looking For A seasoned Data Engineering expert who can drive the adoption of modern data platforms and provide technical thought leadership to clients across cloud data engineering, data architecture, and enterprise data management. This role requires deep expertise in building scalable, high-performance data engineering solutions ...

Pune, भारत

  • Full-time
Leadership data science role at Capco in Pune. Needs 10+ years. What you'll do - Develop high quality, secure and scalable data pipelines using spark, Scala/ python on Hadoop or object storage - Leverage new technologies and approaches to innovate with increasingly large data sets - Drive automation and efficiency in Data ...

Pune, भारत

... perspective gives us a competitive advantage. Position: Data Engineer Location: Pune, Maharashtra Job Description The Data semantic solution team is looking for a Big Data Engineer to drive our mission to unlock potential of data assets by consistently innovating, eliminating friction in how users access data from its Big Data repositories ...

Pune, भारत

  • On-site
  • Full-time
... platforms for batch and real-time data processing - Build scalable data ingestion and processing frameworks to handle structured, semi-structured, and unstructured datasets - Solve complex business problems by leveraging data engineering best practices and advanced technologies What you need - SQL - ETL - Python - Spark - Azure ...

Pune, भारत

Kalpavruksh Systems Private Limited

Senior Data Architect/Lead Data Engineer

Role Overview : We are seeking an exceptional Senior Data Architect / Lead Data Engineer who combines deep architectural thinking with strong hands-on engineering capabilities. This is not a documentation-only architecture role. The ideal candidate can define the target architecture, make critical technology decisions, ...

भारत