Back to list

Vacancy description

#111 - Data Engineer in Hyderabad, भारत

Lifted, an Upwork Company™

  • RemoteWork from anywhere
  • ContractFixed-term or freelance

Hyderabad, भारत

Job Description



We are seeking a hands-on Data Engineer to support a large-scale data platform modernization initiative. You will migrate approximately 1,000 datasets to Snowflake, adapt existing pipelines, and help deliver reliable orchestration with Apache Airflow 3. The architecture and technical approach are already defined, with the role focused on implementation and migration delivery through March 2027.

 

Enterprise experience strongly preferred.

 

Key Responsibilities

- Migrate pipelines from Apache Airflow 2 to Apache Airflow 3.

- Migrate Confluent connections between cloud regions.

- Migrate pipelines from Hive to Iceberg using Snowpark Connect.

- Adapt existing data pipelines to Snowflake while maintaining data quality and reliability.

- Support workflow scheduling and monitoring in Apache Airflow.

- Partner with the team to execute time-sensitive data migrations.

Qualifications



Must-Have Skills

- 2+ years of experience.

- Strong software engineering fundamentals.

- Hands-on Python development experience.

- Advanced SQL, including complex joins, CTEs, window functions, and performance tuning.

- Snowflake data modeling.

- Apache Airflow experience developing and testing pipelines in Airflow and Python.

- Apache Kafka experience.

- Apache Spark experience, including hands-on PySpark.

- Experience with Iceberg and Parquet data formats.

- AWS experience, particularly S3 data lakes, partitioning, and lifecycle policies.

- Testing strategy experience and the ability to debug and test code hands-on.

- At least 2 years of industry experience working with large-scale datasets, including Airflow and Spark.

- Strong communication skills.

- Ability to work independently.

 

Nice-to-Have Skills

- Hands-on Snowflake migration experience, including data warehouse modernization and performance and cost optimization.

- Apache Airflow migration experience, including version upgrades, executor changes, or platform migrations.

- Experience using AI tools for development.

Additional Information



Required Tools & Platforms

- Python and SQL.

- Snowflake, Apache Airflow, Apache Kafka, and Apache Spark/PySpark.

- Iceberg and Parquet.

- AWS S3.

- The migration environment also includes Confluent, Hive, and Snowpark Connect.

 

Location, Time & Engagement

- Open to candidates in eligible EMEA and APAC locations.

- Coverage until 12:00 PM Pacific Time is required.

- Remote contract engagement for 40 hours per week.

- Expected engagement through March 31, 2027; start date to be confirmed.

- Two openings are available.

- This is not currently a contract-to-hire opportunity.

Apply to vacancy

Similar vacancies

More like this role

Lifted, an Upwork Company™

#111 - Data Engineer

  • Remote
  • Full-time
  • Contract
We are seeking a hands-on Data Engineer to support a large-scale data platform modernization initiative. You will migrate approximately 1,000 datasets to Snowflake, adapt existing pipelines, and help deliver reliable orchestration with Apache Airflow 3. The architecture and technical approach are already defined, with the ...

भारत

Chalhoub Group

Data Engineer (Data Intelligence)

  • Remote
  • Hybrid
... drift. • Document model implementations, experiments, and test results for reproducibility. • Collaborate with senior engineers on model integration and with data engineering teams on data readiness. What you’ll need to succeed • Minimum 3 years of experience in machine learning or AI engineering. • Proficiency in Python ...

Bangalore, भारत

Data Analyst / Data Engineer

  • Remote
  • Full-time
  • Data Analytics
Data Analyst / Data Engineer – SQL, BI & Data Analytics | Remote Position Type: Full-Time, Remote Working Hours: U.S. Business Hours About the Role At Pavago, one of our clients is hiring a Data Analyst / Data Engineer to transform business data into reliable reporting, actionable insights, and scalable data systems. Depending ...

India, भारत

Chalhoub Group

Data Engineer

  • Remote
  • Hybrid
... Net Zero by 2040. What you'll be doing Build, maintain, and monitor automated data pipelines on the Group's cloud data platform, transforming raw transactional data into reporting schemas and developing BI dashboards to deliver reliable, analytics-ready data. • Build, maintain, and monitor automated ETL/ELT data pipelines ...

Bangalore, भारत

Omnicom Media

Data Engineer

  • Full-time

$47 USD

... experienced Data Engineer to manage in-progress and upcoming data infrastructure projects. The candidate will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The ideal candidate is an experienced data pipeline ...

Pune, भारत

ScaleupAlly

Data Engineer

  • Full-time
Purpose of Job We are looking for a Senior Data Engineer who is equally strong on the data side (modelling, data quality, SQL, understanding the business) and the engineering side (Python, PySpark, CI/CD, performance, reliability). You will own the design and delivery of production data platforms from requirements to production ...

Noida, भारत

Take-Two Interactive Software, Inc.

Data Engineer

  • Full-time
... scalable data pipelines and third-party API integrations. - Work with business teams and analysts to interpret business rules and derive insights. - Implement data-quality processes, monitoring, and alerting to maintain accurate and available production data. - Analyze and troubleshoot data issues and assist with their resolution. ...

Bangalore, भारत

OSI Digital

Data Engineer

  • Full-time
... Data Frames using PySpark • Conduct complex data analysis and report on results. • Combine raw information from different sources. • Explore ways to enhance data quality and reliability. Required Skills: • 4-5 years of data engineering experience loading, analyzing and interpreting data. • Solid experience with AWS Glue, ...

Hyderabad, भारत

gradera

Data Engineer

  • Full-time
... measurable business outcomes Overview We are seeking skilled Data Engineers to join our Data & Digital Twin Foundation team. You will design, build, and maintain data pipelines that power digital twin platforms, real-time operational systems, and AI/ML workloads. Working closely with data architects, simulation engineers, and ...

Hyderabad, भारत

Vrinda International

DATA ENGINEER

  • Full-time

Rs 17 lakh p.a.

... analysis and reporting. - Implement and monitor data quality frameworks to ensure the integrity, consistency, and reliability of all data assets. - Optimize database queries and data models to enhance system performance and reduce latency. - Engineer end-to-end ETL and ELT workflows to facilitate seamless data movement ...

Delhi, भारत