Vacancy description

Senior Python Engineer - Data Pipelines - Hybrid in Hyderabad, भारत

TekPioneers - A TekGence Company

  • HybridMix of office and remote
  • Full-timeStandard weekly hours

Hyderabad, भारत

This is a hybrid role based in Hyderabad, India.

Role Summary

Seeking a hands-on senior Python engineer to own data pipelines end to end (ingestion, transformation, orchestration, quality, and delivery) at enterprise scale. Pipelines feed analytics, search, and GenAI/Agentic AI platforms.

Responsibilities

- Design, build, and own production data pipelines in Python (batch and near-real-time).

- Operate orchestrated workflows (AWS Step Functions, Airflow, Dagster, or Prefect) with retries, idempotency, and backfills.

- Model data at scale: partitioning, Parquet/Avro/Iceberg, schema evolution, CDC.

- Establish data quality and observability (Great Expectations/Pandera, SLAs, lineage, alerting).

- Build document-processing and RAG retrieval pipelines (chunking, embeddings, vector and hybrid search).

- Build serving APIs with FastAPI, Flask, or Django REST.

- Set Python engineering standards; mentor junior engineers; liaise with business stakeholders.

Qualifications

- 8–12 years of software engineering, 5–6+ years building production data pipelines in Python.

- Expert modern Python (3.10+), typing, asyncio/concurrency, pandas/Polars, PyArrow, PySpark/Dask/Ray.

- Strong SQL and data modeling; Snowflake, Redshift, Iceberg/Delta, dbt.

- Strong AWS: Lambda, Step Functions, S3, Glue, EMR, IAM, CloudWatch; IaC (CDK/Terraform).

- pytest, packaging (uv/Poetry), CI/CD.

- Strongly preferred: RAG, AWS Bedrock, LangChain/LlamaIndex, agentic frameworks, MCP servers, GenAI evaluation.

- Nice to have: Kafka/Kinesis/Flink streaming, C#/.NET, regulated-industry experience.

Similar vacancies

More like this role

Intraedge Technologies Ltd.

Senior Data Engineer - ETL Pipelines

About the Role:We are looking for an experienced Senior Data Engineer with strong expertise in designing, developing, and optimizing enterprise-scale ETL/ELT pipelines and data integration https://jobeax.com/link/XnflZD2dGSZYgXFG Responsibilities:- Design, develop, and maintain scalable ETL/ELT pipelines for extracting, ...

Pune, भारत

Job Summary : We are looking for an experienced Senior Data Engineer to design, develop and maintain scalable data pipelines, data platforms and cloud-based data infrastructure. The ideal candidate should have strong expertise in data engineering, ETL/ELT processes, big data technologies, cloud platforms and data https://jobeax.com/link/17NlwZ4oSJJaCzVO ...

Calcutta, भारत

  • Full-time
... role at Worley in Bengaluru. Needs 10+ years and a https://jobeax.com/link/2DENLGgQM7whDhaE or B.E.. What you'll do - Perform design activities for offshore pipelines in all phases of the project (from concept design to execution phase) in collaboration within the pipeline team across… - Assist Line Manager/ Lead Engineer ...

Bangalore, भारत

  • Full-time
... and AI-enabled delivery, we support customers in navigating complexity as they meet today's needs and transition to more sustainable solutions. The Role As a Senior Marine Pipelines Engineer with Worley, you will work closely with our existing team to deliver projects for our clients while continuing to develop your skills ...

Bangalore, भारत

... Skills :- Data Engineering, AI-Augmented Development, Python, SQL, ETL, ELT, Apache Airflow, Snowflake, dbt, Databricks, Apache Spark, PySpark, Data Pipelines, Data Modelling, Data Warehousing, Data Integration, REST APIs, Kafka, AWS, Azure, BigQuery, Git, CI/CD, GitHub Copilot, Cursor, Claude Code, Data Quality, Data Governance ...

Chennai, भारत

  • Full-time
... efficient data engineering solutions using our platform Cognite Data Fusion.- Drive and manage integrations, extractions, data modeling, and analysis using Cognite data connectors and SQL, Python/Java and Rest APIs.- Create custom data models for data discovery, mapping, and cleansing.- Collaborate with data scientists, project ...

Bangalore, भारत

GANIT BUSINESS SOLUTIONS PRIVATE LIMITED

Ganit - Senior Data Engineer - Python/Spark

  • On-site
... and action, translating data into valuable insights and strategies. We work with businesses to create state-of-the-art solutions that push the boundaries of data https://jobeax.com/link/tIh845hdjJOTUnWw & Responsibilities:- Data Engineering Excellence: Design and implement data pipelines using formats like JSON, Parquet, ...

Gurgaon, भारत

BNP Paribas India Solutions Pvt. Ltd.

BNP Paribas ISPL - Senior Data Engineer - Python

... technology ecosystem and provides end-to-end technology platform that enables the bank's anti-money-laundering (AML) https://jobeax.com/link/8PycUj1GMlj0NwXn a Senior Data Engineer you will design, build and operate high-performance, Python services that ingest, enrich, analyze and surface AML-relevant https://jobeax.com/link/VqRHKsBGCmTUM5K6 ...

Chennai, भारत

  • IT

$80 USD

... engineering practices (testing, observability, documentation). Must have • 5-7 years of data engineering in production: building, deploying, and operating pipelines that other teams depend on daily. Analyst, BI-developer, or drag-and-drop ETL-tool-only experience does not count toward this. • Strong Python (3.x): pandas/pyarrow, ...

Hyderabad, भारत

BNP Paribas India Solutions Pvt. Ltd.

BNP Paribas ISPL - Senior Data Engineer - Python

... technology ecosystem and provides end-to-end technology platform that enables the bank's anti-money-laundering (AML) https://jobeax.com/link/8PycUj1GMlj0NwXn a Senior Data Engineer you will design, build and operate high-performance, Python services that ingest, enrich, analyze and surface AML-relevant https://jobeax.com/link/VqRHKsBGCmTUM5K6 ...

Bombay, भारत