Back to list

Vacancy description

Lead Software Engineer - Data Ingestion in Bangalore, भारत

Société Générale

  • Full-timeStandard weekly hours

Bangalore, भारत

Data Ingestion Specialist Role Summary We are looking for a Medior Data Ingestion Specialist to support development, operation, and maintenance of data ingestion platform orchestrated by Apache Airflow. The role focuses on onboarding and maintaining data integrations from multiple source systems, troubleshooting operational issues, and supporting release and governance processes Key Responsibilities Data Integration & Pipeline Development

  • Configure and maintain data ingestion pipelines
  • Develop and support Python-based ingestion components
  • Integrate data:
  • Relational databases (Oracle, PostgreSQL, MS SQL, MySQL, DB2, etc)
  • Apache Kafka
  • AWS S3, flat files
  • Elasticsearch, Hive, Impala, hBase
  • Troubleshoot data flow, connectivity, and integration issues
  • Ingestion to Databricks, Databricks loader
Platform Operations
  • Support incident, problem, and change management activities
  • Maintain Python libraries and dependencies
  • Create ETL/ELT pipelines focused on source ingestion
  • Monitor pipeline execution and resolve operational issues
  • Assist with platform upgrades, patching, and maintenance activities
Release & Governance
  • Support Dev/Test/Prod deployment processes
  • Review configuration and template changes
  • Ensure compliance with governance, security, and operational standards
  • Maintain technical and operational documentation
Collaboration
  • Work closely with architects, engineering teams, and platform owners
  • Collaborate with security, audit, and infrastructure teams
  • Participate in release planning and communication activities
Nice-to-Have Skills
  • Knowledge of ETL Python frameworks (e.g. Meltano)
  • Experience with other ETL/ELT tools
  • Experience with Git, CI/CD pipelines, and DevOps practices
  • Knowledge of Linux environments
  • Experience with Jira and ITSM tooling
  • Exposure to cloud-based data platforms
Candidate Profile
  • Medior-level engineer capable of working independently on standard tasks
  • Able to create and resolve customers' requirements
  • Able to analyse integration issues and identify root causes
  • Comfortable working in an enterprise environment with governance and security requirements
  • Good communication skills and ability to collaborate with multiple stakeholders
Loading similar vacancies…