Vacancy description

Senior Data Engineer in Bangalore, भारत

Luxoft

  • Full-timeStandard weekly hours

Bangalore, भारत

This is a strategic data engineering engagement with our client to architect and plan the migration of their entire data processing and ETL estate from Matillion to AWS Glue — a foundational shift in how one of the world's largest financial market infrastructure companies handles its data pipelines.
During this Mobilisation Phase, you'll work jointly with client's engineering teams to reverse-engineer the existing landscape, design the target-state architecture across every layer (infrastructure, data processing, workflows, dependencies, and operating model), and build the detailed delivery blueprint that will greenlight the full-scale migration.
The work is technically rich and highly collaborative: you'll review and validate job inventories spanning hundreds of ETL workflows, define reusable migration patterns and templates, design a validation and reconciliation framework, run a proof-of-concept to stress-test the approach, and navigate client's rigorous internal governance — from Architectural Significance Assessments through Architectural Review Boards to a formal Gate 1 decision.
This is the kind of engagement where your recommendations directly shape a multi-phase, multi-million-pound programme: the target-state framework you produce here becomes the blueprint that a larger delivery team will execute against. Perfect opportunity for combining deep data engineering knowledge with architecture leadership, stakeholder influence, and structured delivery planning inside a Tier 1 financial services environment

Responsibilities

Design, build, and maintain ETL/ELT pipelines on AWS

Migrate existing data pipelines and workloads from legacy/on-prem systems to AWS

Develop and optimize data models for data warehouse/data lake (Redshift, S3)

Build and orchestrate data workflows (Glue, Step Functions, Lambda, EMR)

Implement data quality checks, validation, and reconciliation processes

Optimize pipeline performance and manage compute/storage costs

Ensure data security and access control (IAM, KMS, encryption)

Monitor and troubleshoot pipeline failures (CloudWatch, logging)

Collaborate with data analysts, BI developers, and architects on data requirements

Document data pipelines, architecture, and operational runbooks

Skills

Must have

5+ years of experience

Hands-on experience building data pipelines on AWS (Glue, EMR, Lambda, Step Functions)

Strong SQL and experience with data warehousing (Redshift, dimensional modeling)

Proficiency in Python or Scala for data engineering

Experience with S3-based data lake architecture (partitioning, cataloging, formats like Parquet)

Experience migrating data pipelines from on-prem or other cloud platforms

Understanding of data governance, security, and access control on AWS

Experience with CI/CD for data pipelines

Nice to have

AWS certification (Data Analytics Specialty or Data Engineer Associate)

Experience with streaming data (Kinesis, MSK/Kafka)

Familiarity with Infrastructure as Code (Terraform/CloudFormation)

Experience with orchestration tools (Airflow, dbt)

Experience in financial domain

Other

Languages

English: C2 Proficient

Seniority

Senior

Similar vacancies

More like this role

Merrill Communications

Senior Data Engineer

  • Full-time
... Spark (PySpark) – Strong experience in building distributed data pipelines . - Python – Proficiency in writing optimized and maintainable Python code for data engineering. - Unity Catalog – Hands-on experience implementing data governance, access controls, and lineage tracking . - SQL – Strong knowledge of SQL for data transformations ...

Chennai, भारत

GE HealthCare seeks a Senior Data Engineer to design and implement data engineering solutions, ETL pipelines, and analytics infrastructure. The role involves data profiling, metadata management, and automation using cloud and low-code tools to support data science outcomes. Responsibilities - Design and build automations ...

Bangalore, भारत

  • On-site
  • Full-time
... your world, creating more fulfilling work and projects for our people and clients. For more information, visit [ Job Description We’re looking for a skilled Senior Data Engineer with strong experience in distributed systems, Python/Scala, and modern data engineering tools to help design and implement an end-to-end data ...

Hyderabad, भारत

  • Full-time
Job Summary Valorem Reply is looking for a Data Engineer with experience building and contributing to the design of database systems, both normalized transactional systems and dimensional reporting systems. Strong experience with SQL Server as a database engine, as well as Microsoft and Databricks technology experience ...

Ernakulam, भारत

  • Remote
  • Hybrid
  • Full-time
  • Contract
... perfect your skills in a field filled with exciting challenges and opportunities. About the role As part of the Big Data B2B program, OBS set up a shared Big Data platform and a Data Lake for Use cases exploration and industrialization. We are looking for Senior Data Engineer 5-8 years of experience in building Data Pipelines ...

Gurgaon, भारत

  • On-site
  • Contract
... is a sophisticated technology ecosystem that processes billions of payment events with speed, resilience, and precision. As a Senior Data Engineer within the Data Collection & Engineering (DC&E) organisation, you will play a critical role in designing, building, and scaling Mastercard's next-generation data platforms and ...

Hyderabad, भारत

  • Hybrid
... innovation, and large-scale data engineering? If so, we have an exciting opportunity for you to make a meaningful impact on a global scale. About the Role As a Senior Data Engineer , you will be at the heart of our ambition to become a truly data-driven organisation. You will design and deliver best-in-class data engineering ...

Bangalore, भारत

  • Full-time
Mid-level data science role at KreditBee in Bengaluru. Needs 2–4 years and a https://jobeax.com/link/2DENLGgQM7whDhaE or B.E.. What you'll do - ⁠ ⁠Develop various RestFul APIs & Integrate with various data sources - ⁠ ⁠Develop tools & techniques related to ETL - ⁠ ⁠Exposure in AWS or Other Cloud Platform is advantageous ...

Bangalore, भारत

  • Full-time
  • Contract
Senior Data Engineer POSITION PURPOSE: - The Data & Analytics organization enables Bayer’s Corporate Functions—including Finance, Supply Chain & Logistics, HR, Procurement, Legal, and Sustainability—to become more insight and data-driven. Our mission is to deliver a robust data foundation and FAIR—Findable, Accessible, ...

Bangalore, भारत

  • Full-time
  • IT
... networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Senior Data Engineer Overview We are the global technology company behind the world’s fastest payments processing network. We are a vehicle for commerce, a connection ...

Pune, भारत