Back to list

Vacancy description

Lead Software Engineer - Data Ingestion in Bangalore, भारत

Société Générale

  • Full-timeStandard weekly hours

Bangalore, भारत

Data Ingestion Specialist Role Summary We are looking for a Medior Data Ingestion Specialist to support development, operation, and maintenance of data ingestion platform orchestrated by Apache Airflow. The role focuses on onboarding and maintaining data integrations from multiple source systems, troubleshooting operational issues, and supporting release and governance processes Key Responsibilities Data Integration & Pipeline Development

  • Configure and maintain data ingestion pipelines
  • Develop and support Python-based ingestion components
  • Integrate data:
  • Relational databases (Oracle, PostgreSQL, MS SQL, MySQL, DB2, etc)
  • Apache Kafka
  • AWS S3, flat files
  • Elasticsearch, Hive, Impala, hBase
  • Troubleshoot data flow, connectivity, and integration issues
  • Ingestion to Databricks, Databricks loader
Platform Operations
  • Support incident, problem, and change management activities
  • Maintain Python libraries and dependencies
  • Create ETL/ELT pipelines focused on source ingestion
  • Monitor pipeline execution and resolve operational issues
  • Assist with platform upgrades, patching, and maintenance activities
Release & Governance
  • Support Dev/Test/Prod deployment processes
  • Review configuration and template changes
  • Ensure compliance with governance, security, and operational standards
  • Maintain technical and operational documentation
Collaboration
  • Work closely with architects, engineering teams, and platform owners
  • Collaborate with security, audit, and infrastructure teams
  • Participate in release planning and communication activities
Nice-to-Have Skills
  • Knowledge of ETL Python frameworks (e.g. Meltano)
  • Experience with other ETL/ELT tools
  • Experience with Git, CI/CD pipelines, and DevOps practices
  • Knowledge of Linux environments
  • Experience with Jira and ITSM tooling
  • Exposure to cloud-based data platforms
Candidate Profile
  • Medior-level engineer capable of working independently on standard tasks
  • Able to create and resolve customers' requirements
  • Able to analyse integration issues and identify root causes
  • Comfortable working in an enterprise environment with governance and security requirements
  • Good communication skills and ability to collaborate with multiple stakeholders

Similar vacancies

More like this role

  • IT
... ServiceNow, SharePoint), databases (e.g., SQL Server), cloud storage, message queues, and local files. In addition to being an important part of Lakeflow and Data Engineering, Connect is also a key platform capability. Every surface in Databricks (Dashboards, Notebooks, SQL, AI) requires ingestion capabilities and the lead for ...

Bangalore, भारत

  • IT
... ServiceNow, SharePoint), databases (e.g., SQL Server), cloud storage, message queues, and local files. In addition to being an important part of Lakeflow and Data Engineering, Connect is also a key platform capability. Every surface in Databricks (Dashboards, Notebooks, SQL, AI) requires ingestion capabilities and the lead for ...

Bangalore, भारत

  • On-site
  • Full-time

Rs 16 lakh p.a.

Summary We are seeking an experienced Senior Data Ingestion Engineer to architect and manage robust data transfer solutions within our IT Services division. This pivotal role focuses on integrating diverse data sources, including SaaS platforms, cloud environments, and on-premises systems, to ensure seamless and reliable ...

भारत

  • On-site
  • Full-time
  • IT
Responsibilities - Lead and mentor a team of data engineers and platform developers. - Architect scalable and secure AWS-based data platforms. - Design and develop batch and near-real-time ingestion pipelines for structured and unstructured data. - Build and optimize Snowflake data platforms, including data modeling, governance, ...

Chennai, भारत

  • IT
... your software engineering career while building impactful technology solutions. Grow your skills and make a difference with a collaborative team. As a Senior Lead Engineer at JPMorgan Chase within the Chief Technology Office team, you will design, develop, and deliver innovative software solutions. You will collaborate ...

Bangalore, भारत

  • Remote
  • Hybrid
  • On-site
  • Full-time

Rs 5 lakh p.a.

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Data Ingestion Engineer (Fivetran) based in India. We are seeking an experienced Senior Data Ingestion Engineer to design, implement, and manage reliable data integration solutions across ...

भारत

  • Full-time
  • IT
Responsibilities - Design, development and testing of components / modules in TOP (Trade Open Platform) involving Spark, Java, Hive and related big-data technologies in a datalake architecture - Contribute to the design, development and deployment of new features & new components in Azure public cloud - Contribute to the ...

Bangalore, भारत

  • Full-time
  • IT
Senior software engineering role at NIKE in Karnataka. Needs 7–10 years and an https://jobeax.com/link/MPrGoo1PSJpZ7d0e, https://jobeax.com/link/2DENLGgQM7whDhaE or B.E.. What you'll do - Lead the design of consumption layers, APIs, and pipelines that BI Experience teams build on — establishing patterns, contracts, and ...

भारत

  • Full-time
  • IT
Senior software engineering role at JPMorgan Chase in Bengaluru. Needs 5+ years. What you'll do - Use domain modeling techniques to build best-in-class business products - Structure software for ease of understanding, testing, and evolution - Build scalable solutions that avoid single points of failure What you need - Java ...

Bangalore, भारत

  • Full-time
  • IT
Senior software engineering role at Danaher in Bengaluru. Needs a https://jobeax.com/link/2DENLGgQM7whDhaE or B.E.. What you'll do - Lead the technical execution of new product development and Product Care projects , ensuring delivery against business, quality, schedule, and technical objectives - Provide technical leadership ...

Bangalore, भारत