Senior Data Engineer : Big Data in Gurugram, India is listed on Jobeax. Browse 30,000+ vacancies available.
Senior Data Engineer – GCP Experience: 5+ Years Role: Senior Data Engineer
Domain: Cloud Data Engineering
Platform: Google Cloud Platform (GCP)
Role Summary We are seeking a Senior Data Engineer with 5+ years of experience in designing, developing, and maintaining scalable, cloud-based data solutions. The ideal candidate will have strong hands-on expertise in SQL, Python, PySpark, and Google Cloud Platform (GCP) .
The candidate will be responsible for building enterprise-grade ETL/ELT pipelines , implementing scalable batch and real-time data solutions, optimizing data processing workloads, and ensuring high standards of data quality, reliability, and performance.
You will collaborate closely with Product Owners, Engineering teams, Business SMEs, and other stakeholders to understand requirements and deliver robust, scalable, and high-quality data products.
Mandatory Skills
Python – Advanced
SQL – Advanced
PySpark
Google Cloud Platform (GCP)
BigQuery
Cloud Composer / Apache Airflow
Dataproc
Pub/Sub
Google Cloud Storage (GCS)
ETL / ELT Development
Batch & Streaming Data Processing
Data Pipeline Development & Optimization
Git / GitHub
CI/CD
Concepts
Key Responsibilities
Design, develop, and maintain scalable ETL/ELT data pipelines using Python and PySpark.
Build and optimize cloud-native data solutions on Google Cloud Platform.
Develop optimized SQL queries, transformation logic, and data models in BigQuery.
Design and maintain workflow orchestration using Cloud Composer / Apache Airflow .
Develop, optimize, and troubleshoot Spark applications running on Dataproc .
Design and implement event-driven data ingestion pipelines using Google Pub/Sub.
Build and manage both batch and real-time data processing pipelines .
Develop data solutions for Data Warehouses and Data Lakes .
Implement data validation, reconciliation, exception handling, monitoring, and data quality controls.
Optimize pipeline performance, query execution, resource utilization, and processing costs.
Develop reusable data engineering frameworks, utilities, and components to improve engineering productivity.
Implement logging, monitoring, alerting, and troubleshooting mechanisms for production data pipelines.
Participate in code reviews and follow established engineering, coding, and development best practices .
Create and maintain comprehensive technical documentation for data pipelines, workflows, data models, and architecture.
Collaborate with Product Owners, Business SMEs, Engineering teams, and other stakeholders to translate business requirements into scalable technical solutions.
Google Cloud & Data Engineering Expertise Hands-on experience with Google's data ecosystem, including:
BigQuery
Dataproc
Dataflow
Pub/Sub
BigTable
Cloud Spanner
Cloud SQL
AlloyDB
Google Cloud Storage
Cloud Composer / Apache Airflow
Cloud Scheduler
Experience designing and implementing batch and real-time data pipelines , data migration solutions, and scalable data-layer architectures across the GCP ecosystem is highly desirable.
Data Warehouse & Data Modeling Strong understanding of:
Data Warehouse Architecture
Data Lake vs. Data Warehouse
Star Schema
Snowflake Schema
Fact & Dimension Modeling
Slowly Changing Dimensions (SCD)
Partitioning & Clustering
Data Layer Design
Data Transformation & Aggregation
Data Quality & Governance
Big Data & Open-Source Technologies Experience with one or more of the following:
Apache Spark
PySpark / Python
Spark / Scala
Apache Hadoop
Apache Beam
Apache Airflow
dbt
DevOps & Engineering Practices
Git / GitHub
CI/CD
Docker
Version Control
Code Reviews
Automated Testing
Logging & Monitoring
Production Support & Troubleshooting
Preferred Qualifications
Experience working on enterprise-scale cloud migration initiatives .
Experience developing reusable data engineering frameworks and utilities .
Strong understanding of data governance and data quality practices .
Exposure to AI-assisted development tools , such as GitHub Copilot.
Experience with both batch and streaming data architectures .
Strong analytical, problem-solving, and troubleshooting skills.
Excellent communication and stakeholder management skills.
Ability to work effectively with cross-functional Product, Engineering, and Business teams .
Experience Requirements
- 5+ years of professional experience in Data Engineering .
- Strong hands-on experience with SQL, Python, and PySpark .
- Proven experience developing production-grade data pipelines .
- Hands-on experience with Google Cloud Platform (GCP) and its data engineering services.
- Experience designing scalable solutions for Data Warehouses, Data Lakes, batch processing, and real-time streaming .
Ideal Candidate The ideal candidate is a hands-on Senior Data Engineer who can independently design and deliver scalable data solutions using Python, SQL, PySpark, and GCP , with strong expertise in BigQuery, Dataproc, Cloud Composer/Airflow, Pub/Sub, and Cloud Storage .
The candidate should combine strong technical expertise with a solid understanding of data architecture, data modeling, data quality, cloud engineering, and production operations .