Senior Data Engineer - AWS, DevOps in Chennai, India - Jobeax
Vacancy description
Senior Data Engineer - AWS, DevOps in Chennai, India
talentgigs
India, Chennai
Senior Data Engineer - AWS, DevOps in Chennai, India is listed on Jobeax. Browse 30,000+ vacancies available.
Data Engineer – Python & Pyspark (AWS) Exp: 6+ yrs Location : Chennai Client Place Guindy Work Mode : All 5 days
Role Overview We're seeking a Senior Data Engineer with strong hands-on expertise in Python, PySpark, and AWS to design, build, and operate scalable data pipelines for analytics and operational workloads. You will own end-to-end data engineering — from ingestion to delivery — across batch and near-real-time patterns, with a focus on pipeline reliability, data quality, and cloud-native engineering. This is an engineering role — you will build and own production systems, not just develop queries or reports. Individual contributor expected to influence architecture, drive technical decisions, and mentor peers without formal people management responsibilities.
Key Responsibilities Design, build, and maintain scalable ETL/ELT pipelines using Python and PySpark to ingest data from files, APIs, RDBMS, NoSQL, and message queues into AWS data stores Develop and optimize PySpark jobs for large-scale data processing, transformation, and aggregation on AWS EMR or Databricks Build and manage AWS-native data workflows using S3, Glue, Lambda, Redshift, and Step Functions; manage scheduling, dependencies, retries, and SLAs Write production-grade Python for data transformations, validations, orchestration, and API integrations — not notebook-level scripting Implement robust data quality controls: schema validation, deduplication, referential integrity checks, and anomaly detection Collaborate on data modeling (star/snowflake, data vault, medallion architecture) and define data contracts and lineage Build CI/CD for data pipelines (tests, linting, packaging) using GitHub Actions or AWS CodePipeline; maintain documentation and runbooks Troubleshoot and remediate production issues; drive continuous improvement in performance, reliability, and cost Required Qualifications
6–8 years of hands-on data engineering experience with a strong focus on Python, PySpark, and AWS
Production-grade Python skills — modular code, logging, error handling, REST/GraphQL API integration; not scripting or notebook-only work
Advanced SQL skills — complex joins, window functions, query optimization; Oracle or PostgreSQL experience is a plus
Solid understanding of ETL/ELT patterns, data modeling, and data quality frameworks
Experience with workflow orchestration: Apache Airflow, AWS Step Functions, or equivalent
CI/CD experience: GitHub Actions, AWS CodePipeline, Azure DevOps, or GitLab CI
Strong engineering mindset — you build it, you own it, you operate it
Nice-to-Have
Experience with Kafka, Kinesis, or Event Hubs for event streaming Familiarity with dbt for ELT and data testing Great Expectations or Deequ for data quality Docker / Kubernetes for containerized data services Knowledge of data security patterns: PII masking, tokenization, column/row-level security Informatica, ODI, or SSIS exposure Education & Certifications
Bachelor's/Master's in Computer Science, Engineering, or equivalent AWS Certified Data Engineer, AWS Solutions Architect, Databricks, or Snowflake certifications are a plus
Position: Senior Data Engineer Location: Pune Job Responsibilities Design, develop, and maintain scalable data pipelines using AWS services. Design and implement scalable, secure, and resilient MLOps architecture across cloud and on-prem environments Define best practices for model lifecycle management, including versioning, ...
... specifications. Assist with data modelling and optimization tasks. Document the data platform architecture, processes, and best practices. Stay up to date with the latest trends and technologies in full stack development, data engineering, and DevOps. Proactively suggest improvements and innovations for the data platform.
... least one certification • Databricks Certified Data Engineer Associate OR Databricks Certified Data Engineer Professional Additional Certifications (Preferred) - Databricks Certified Associate Developer for Apache Spark - Cloud platform certifications (Azure Data Engineer Associate, AWS Certified Data Analytics, or Google Cloud ...
... Vault / AWS Secrets Manager. Pipeline Reliability & DevOps Set up monitoring, logging, and alerting frameworks for pipeline health. Drive test-driven data engineering (unit, integration, regression tests). Use Azure DevOps and GitHub for CI/CD automation and version control. Collaborate with Data Engineering, Data Architects, ...
... interfaces to analyze, transform and prepare data, to ship trusted insights at enterprise scale. Don’t miss the rocket ship—join Prophecy and build the next data revolution. This is a high-impact opportunity to be a senior DevOps engineer in a fast-growing startup, based in Prophecy’s India engineering center. You will ...
Role - DevOps Engineer Location - Chennai Mode - Work from office Experience - 5+ Years Employment - Full-time Industries - Banking, Payments, FinTech, IT Products Note: Candidates having experience with US banking sector with US payment domains are preferred. About the role - We are looking for a Senior DevOps Engineer ...
Job description Job Description (JD) - Senior Associate DevOps Assistant (Grade 2) The Senior Associate DevOps Assistant (Grade 2) supports the DevOps team in automating, streamlining, and enhancing the development and deployment pipelines. This role involves assisting in managing infrastructure, automating software deployment, ...
... companies that need to ship enterprise integrations at scale. We're reimagining integrations with AI agents that work like business consultants and integration engineers, building them 10x faster, in hours instead of days. The payoff: integrations stop being a bottleneck that costs you deals and burns margin, and become a growth ...
... testing of the platform takes place automatically. The individual is expected to collaborate and operate in parallel with the DevOps Engineers and Application Engineers. Mentoring and training other team members will be an expectation from this role. Responsibilities: The Senior Cloud Services Engineer utilizes software that ...
... cloud platform (AWS), with proven hands-on experience in Terraform, Docker, and Kubernetes. We are not looking for a 'server administrator.' Can challenge poor engineering practices Thinks in terms of reliability engineering, not firefighting For most SaaS companies, DevOps becomes a support function. A Weak DevOps Team Creates ...
We are looking for an experienced Senior Software Engineer / Developer with strong hands-on expertise in AWS cloud services, microservices, REST API development, and Node.js. The ideal candidate will design, develop, deploy, and support scalable, secure, and high-performance cloud-native applications on AWS. Exposure of ...
... experiences across the growth journey, from the first marketing touchpoint to personalized engagement moments like Daily Streak notifications. As a Senior Software Engineer , you will help transform GDP from a data mover into a real-time decisioning platform by operationalizing ML models, strengthening observability and data trust, ...
... installations, we ensure cost-efficient delivery of services, upgrades and spare parts in order to secure availability and predictability to our customers. DevOps Engineer with Kongsberg Maritime India. We are looking for software professionals who can join our team and work closely with team leads and peer developers/testers ...
... Implement cutting-edge AI integrations and intelligent features that enhance the developer experience and provide smart insights Cloud Infrastructure : Leverage AWS services to build resilient, performant solutions Cross-functional Collaboration : Work closely with frontend engineers, DevOps, and stakeholder teams to deliver ...
... code - Experience mentoring engineers through code review, pair programming, or technical guidance Experience with spec-driven, agentic AI workflows where engineers delegate autonomous tasks to AI tools and provide guidance and oversight Knowledge of NoSQL data stores, such as MongoDB or DynamoDB Experience with AWS, Azure ...
... but your past experience doesn't align perfectly with every single qualification in the job description, nobody's perfect – and we encourage you to apply. The Senior Devops Engineer is responsible for developing deployment framework and managing deployment automation strategies. This is a hands-on leadership role and individual ...