Back to list

Vacancy description

Senior Python Engineer - Data Pipelines - Hybrid in Hyderabad, भारत

TekPioneers - A TekGence Company

  • HybridMix of office and remote
  • Full-timeStandard weekly hours

Hyderabad, भारत

This is a hybrid role based in Hyderabad, India.

Role Summary

Seeking a hands-on senior Python engineer to own data pipelines end to end (ingestion, transformation, orchestration, quality, and delivery) at enterprise scale. Pipelines feed analytics, search, and GenAI/Agentic AI platforms.

Responsibilities

- Design, build, and own production data pipelines in Python (batch and near-real-time).

- Operate orchestrated workflows (AWS Step Functions, Airflow, Dagster, or Prefect) with retries, idempotency, and backfills.

- Model data at scale: partitioning, Parquet/Avro/Iceberg, schema evolution, CDC.

- Establish data quality and observability (Great Expectations/Pandera, SLAs, lineage, alerting).

- Build document-processing and RAG retrieval pipelines (chunking, embeddings, vector and hybrid search).

- Build serving APIs with FastAPI, Flask, or Django REST.

- Set Python engineering standards; mentor junior engineers; liaise with business stakeholders.

Qualifications

- 8–12 years of software engineering, 5–6+ years building production data pipelines in Python.

- Expert modern Python (3.10+), typing, asyncio/concurrency, pandas/Polars, PyArrow, PySpark/Dask/Ray.

- Strong SQL and data modeling; Snowflake, Redshift, Iceberg/Delta, dbt.

- Strong AWS: Lambda, Step Functions, S3, Glue, EMR, IAM, CloudWatch; IaC (CDK/Terraform).

- pytest, packaging (uv/Poetry), CI/CD.

- Strongly preferred: RAG, AWS Bedrock, LangChain/LlamaIndex, agentic frameworks, MCP servers, GenAI evaluation.

- Nice to have: Kafka/Kinesis/Flink streaming, C#/.NET, regulated-industry experience.

Similar vacancies

More like this role

Intraedge Technologies Ltd.

Senior Data Engineer - ETL Pipelines

About the Role:We are looking for an experienced Senior Data Engineer with strong expertise in designing, developing, and optimizing enterprise-scale ETL/ELT pipelines and data integration https://jobeax.com/link/XnflZD2dGSZYgXFG Responsibilities:- Design, develop, and maintain scalable ETL/ELT pipelines for extracting, ...

Pune, भारत

Youth4 Talent Solutions and Services

Senior Data Engineer - Python/Big Data

... up-and-coming data management and software engineering technologies into existing data structures.- Develop set processes for data integration, data mining, data modeling, and data production.- Research new uses for existing data.- Collaborate with members of your team (e.g., Business Analysts, the IT team, data scientists) ...

Bangalore, भारत

Job Summary : We are looking for an experienced Senior Data Engineer to design, develop and maintain scalable data pipelines, data platforms and cloud-based data infrastructure. The ideal candidate should have strong expertise in data engineering, ETL/ELT processes, big data technologies, cloud platforms and data https://jobeax.com/link/17NlwZ4oSJJaCzVO ...

Calcutta, भारत

  • Full-time
... role at Worley in Bengaluru. Needs 10+ years and a https://jobeax.com/link/2DENLGgQM7whDhaE or B.E.. What you'll do - Perform design activities for offshore pipelines in all phases of the project (from concept design to execution phase) in collaboration within the pipeline team across… - Assist Line Manager/ Lead Engineer ...

Bangalore, भारत

... Skills:Data Engineering:- Python, PySpark, SQL (Advanced), Data Modeling, ETL/ELT Development, Data Warehousing ConceptsBig Data Technologies:- Apache Spark, Databricks, Hadoop Ecosystem, Delta Lake, Apache Iceberg, Apache HudiData Integration & Streaming:- Apache Kafka, Apache Flink, Apache Airflow, Azure Data Factory, ...

Thiruvananthapuram, भारत

... Skills :- Data Engineering, AI-Augmented Development, Python, SQL, ETL, ELT, Apache Airflow, Snowflake, dbt, Databricks, Apache Spark, PySpark, Data Pipelines, Data Modelling, Data Warehousing, Data Integration, REST APIs, Kafka, AWS, Azure, BigQuery, Git, CI/CD, GitHub Copilot, Cursor, Claude Code, Data Quality, Data Governance ...

Chennai, भारत

  • Full-time
... efficient data engineering solutions using our platform Cognite Data Fusion.- Drive and manage integrations, extractions, data modeling, and analysis using Cognite data connectors and SQL, Python/Java and Rest APIs.- Create custom data models for data discovery, mapping, and cleansing.- Collaborate with data scientists, project ...

Bangalore, भारत

GANIT BUSINESS SOLUTIONS PRIVATE LIMITED

Ganit - Senior Data Engineer - Python/Spark

  • On-site
... and action, translating data into valuable insights and strategies. We work with businesses to create state-of-the-art solutions that push the boundaries of data https://jobeax.com/link/tIh845hdjJOTUnWw & Responsibilities:- Data Engineering Excellence: Design and implement data pipelines using formats like JSON, Parquet, ...

Gurgaon, भारत

BNP Paribas India Solutions Pvt. Ltd.

BNP Paribas ISPL - Senior Data Engineer - Python

... technology ecosystem and provides end-to-end technology platform that enables the bank's anti-money-laundering (AML) https://jobeax.com/link/8PycUj1GMlj0NwXn a Senior Data Engineer you will design, build and operate high-performance, Python services that ingest, enrich, analyze and surface AML-relevant https://jobeax.com/link/VqRHKsBGCmTUM5K6 ...

Chennai, भारत

  • IT

$80 USD

... engineering practices (testing, observability, documentation). Must have • 5-7 years of data engineering in production: building, deploying, and operating pipelines that other teams depend on daily. Analyst, BI-developer, or drag-and-drop ETL-tool-only experience does not count toward this. • Strong Python (3.x): pandas/pyarrow, ...

Hyderabad, भारत