Back to list

Vacancy description

Lead Software Engineer - Data Ingestion in Bangalore, भारत

Société Générale

  • Full-timeStandard weekly hours

Bangalore, भारत

Data Ingestion Specialist Role Summary We are looking for a Medior Data Ingestion Specialist to support development, operation, and maintenance of data ingestion platform orchestrated by Apache Airflow. The role focuses on onboarding and maintaining data integrations from multiple source systems, troubleshooting operational issues, and supporting release and governance processes Key Responsibilities Data Integration & Pipeline Development

  • Configure and maintain data ingestion pipelines
  • Develop and support Python-based ingestion components
  • Integrate data:
  • Relational databases (Oracle, PostgreSQL, MS SQL, MySQL, DB2, etc)
  • Apache Kafka
  • AWS S3, flat files
  • Elasticsearch, Hive, Impala, hBase
  • Troubleshoot data flow, connectivity, and integration issues
  • Ingestion to Databricks, Databricks loader
Platform Operations
  • Support incident, problem, and change management activities
  • Maintain Python libraries and dependencies
  • Create ETL/ELT pipelines focused on source ingestion
  • Monitor pipeline execution and resolve operational issues
  • Assist with platform upgrades, patching, and maintenance activities
Release & Governance
  • Support Dev/Test/Prod deployment processes
  • Review configuration and template changes
  • Ensure compliance with governance, security, and operational standards
  • Maintain technical and operational documentation
Collaboration
  • Work closely with architects, engineering teams, and platform owners
  • Collaborate with security, audit, and infrastructure teams
  • Participate in release planning and communication activities
Nice-to-Have Skills
  • Knowledge of ETL Python frameworks (e.g. Meltano)
  • Experience with other ETL/ELT tools
  • Experience with Git, CI/CD pipelines, and DevOps practices
  • Knowledge of Linux environments
  • Experience with Jira and ITSM tooling
  • Exposure to cloud-based data platforms
Candidate Profile
  • Medior-level engineer capable of working independently on standard tasks
  • Able to create and resolve customers' requirements
  • Able to analyse integration issues and identify root causes
  • Comfortable working in an enterprise environment with governance and security requirements
  • Good communication skills and ability to collaborate with multiple stakeholders

Similar vacancies

More like this role

  • IT
... ServiceNow, SharePoint), databases (e.g., SQL Server), cloud storage, message queues, and local files. In addition to being an important part of Lakeflow and Data Engineering, Connect is also a key platform capability. Every surface in Databricks (Dashboards, Notebooks, SQL, AI) requires ingestion capabilities and the lead for ...

Bangalore, भारत

  • IT
... ServiceNow, SharePoint), databases (e.g., SQL Server), cloud storage, message queues, and local files. In addition to being an important part of Lakeflow and Data Engineering, Connect is also a key platform capability. Every surface in Databricks (Dashboards, Notebooks, SQL, AI) requires ingestion capabilities and the lead for ...

Bangalore, भारत

  • On-site
  • Full-time

Rs 16 lakh p.a.

Summary We are seeking an experienced Senior Data Ingestion Engineer to architect and manage robust data transfer solutions within our IT Services division. This pivotal role focuses on integrating diverse data sources, including SaaS platforms, cloud environments, and on-premises systems, to ensure seamless and reliable ...

भारत

  • On-site
  • Full-time
  • IT
Responsibilities - Lead and mentor a team of data engineers and platform developers. - Architect scalable and secure AWS-based data platforms. - Design and develop batch and near-real-time ingestion pipelines for structured and unstructured data. - Build and optimize Snowflake data platforms, including data modeling, governance, ...

Chennai, भारत

  • IT
... your software engineering career while building impactful technology solutions. Grow your skills and make a difference with a collaborative team. As a Senior Lead Engineer at JPMorgan Chase within the Chief Technology Office team, you will design, develop, and deliver innovative software solutions. You will collaborate ...

Bangalore, भारत

  • Remote
  • Hybrid
  • On-site
  • Full-time

Rs 5 lakh p.a.

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Data Ingestion Engineer (Fivetran) based in India. We are seeking an experienced Senior Data Ingestion Engineer to design, implement, and manage reliable data integration solutions across ...

भारत

... inclusion, ownership, continuous learning, and engineering excellence. Required qualifications, capabilities, and skills - Formal training or certification on software engineering concepts and 5+ years applied experience in Python software development and architecture. - Hands-on practical experience in software, data, or ...

Bombay, भारत

  • Full-time
  • IT
Responsibilities - Design, development and testing of components / modules in TOP (Trade Open Platform) involving Spark, Java, Hive and related big-data technologies in a datalake architecture - Contribute to the design, development and deployment of new features & new components in Azure public cloud - Contribute to the ...

Bangalore, भारत

Senior software engineering role at Nomura in Mumbai. What you'll do - NXTConnect and Core Gateways are our flagship solutions that provides HFT/Hedge Fund Clients with low latency Direct Market Access (DMA) and connectivity to Algo… - This is an individual contributor position within Connectivity IT team - In this role, ...

Bombay, भारत

  • Full-time
  • IT
... networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Lead Software engineer All About Us MasterCard is a technology company in the global payments business. We connect consumers, financial institutions, merchants, governments, ...

Pune, भारत