Capco
GCP Data Engineer
- Cloud
Pune, भारत
Vacancy description
Luminite Services
India, भारत
Data EngineerExperience: 6+ years in data engineering, ETL/ELT, analytics engineering, data warehouse development, or big data pipelines; minimum 4 years on GCP cloud data platforms https://jobeax.com/link/bPH40YQHlBdyx8oe we're looking for:- Strong SQL and Python.- Data modelling, warehousing concepts, partitions, clustering, indexing, optimization.- ETL/ELT pipeline design and production support.- Batch and streaming pipeline fundamentals.- Orchestration using Airflow, Composer, Control-M, ADF, dbt, or equivalent.- Big data processing using Spark, PySpark, Beam, Dataflow, Dataproc, Databricks, EMR, or equivalent.- Cloud storage and data lake concepts.- Data quality, reconciliation, schema validation, lineage, monitoring, and alerting.- CI/CD for data pipelines and code versioning.- Ability to explain scale scenarios, not just tool https://jobeax.com/link/w20nDoSA2f7Mp1HD Skills:- Python and advanced SQL.- Data quality checks, reconciliation, alerting, retries, and backfills.- Production pipeline monitoring and https://jobeax.com/link/3mvkfheMPtV9f85w Skills:- Mandatory: BigQuery: datasets, tables, views, partitioning, clustering, SQL optimisation, cost awareness.- Option 1: Dataflow / Apache Beam or equivalent streaming/batch framework.- Option 2: Dataproc / PySpark or strong Spark experience.- Pub/Sub or equivalent event streaming.- Cloud Composer / Airflow DAG https://jobeax.com/link/iDCqPrhlKH5MkOB3 to have:- Dataform, dbt, CI/CD for data.- BigQuery ML.- Dataflow templates.- Looker / LookML basics.- Terraform for data infrastructure. (ref:hirist.tech)
Similar vacancies
Capco
Pune, भारत
Capco Technologies Pvt Ltd
Bangalore, भारत
Artefact
Pune, भारत
Cephas Consultancy Services Private Limited
Chennai, भारत
Pune, भारत
OSI Digital
Hyderabad, भारत
Hyderabad, भारत
Pune, भारत
$14.68 USD
Bangalore, भारत
Ernst & Young
Bangalore, भारत