AWS / GCP Data Engineer in Bengaluru, India is listed on Jobeax. Browse 30,000+ vacancies available.
GCP Data Engineer
Mandatory Skills
Relevant work experience between 3 to 7 years.
Proficient Experience on designing, building and operationalizing large-scale enterprise data solutions using at least four GCP services among Data Flow, Data Proc, Pub Sub, BigQuery, Cloud Functions, Composer, GCS
Proficient hands-on programming experience in Spark/Scala (python/java)
Proficient in building production level ETL/ELT data pipelines from data ingestion to consumption
Data Engineering knowledge (such as Data Lake, Data warehouse - Redshift/Hive/Snowflake, Integration, Migration)
Excellent communicator (written and verbal formal and informal)
Experience using software version control tools (Git/Bitbucket/code commit).
Flexible and proactive/self-motivated working style with strong personal ownership of problem resolution.
Ability to multi-task under pressure and work independently with minimal supervision.
Must be a team player and enjoy working in a cooperative and collaborative team environment.
Desired Skills and Experience (Good to Have):
GCP Certification preferred.
Additional Cloud experience in AWS or Azure preferred.
Qualification
~ Completed undergraduate degree with outstanding academic credentials (preferably a technical undergrad degree e.g. Computer Science, Engineering)
... Python. Develop and optimize large-scale data transformations using PySpark. Ensure data quality, reliability, and performance of data pipelines. GCP BigQuery Data Ingestion & ETL/ELT Data Pipeline Orchestration Python PySpark Strong SQL and data engineering fundamentals hands-on experience in GCP-based data platforms.
... on-premises to cloud Spark code optimization and Medallion Architecture. Strong knowledge of Databricks and its components, including Delta Live Table (DLT) pipeline implementations. Familiarity with AWS services (experience with additional cloud platforms like GCP or Azure is a plus). AWS/GCP/Azure Data Engineer Certification.
... experiences. Contribute to AI-enabled features and use AI development tools responsibly to improve coding, testing, research, documentation, and delivery. Apply sound engineering practices across architecture, performance, security, testing, debugging, and maintainability. Deploy and operate services using AWS and CI/CD workflows, ...
... understanding of how to write geo/spatial queries. Version Control: Basic understanding of tools like Git/GitHub. Cloud Basics: Familiarity with cloud platforms (AWS, GCP, or Azure). OS: Comfortable with basic Linux/Unix command-line operations. Portfolio: Any personal or academic projects showcasing data pipelines or data processing.
... managing this platform, utilizing leading technologies like Databricks, Delta Lake, Spark, PySpark, Scala, and the AWS suite. We are actively seeking skilled Data Engineers to join our team and contribute to scaling our platform across the organization. Create and manage robust data ingestion pipelines leveraging Databricks, ...
... Work Type: Full Time At VIDA we're building the future of digital identity. As a Data Engineer you'll work across the stack—from platform and infrastructure to data pipelines and end-user tooling. You'll help us modernize our batch ETLs and lead our push into streaming and real-time fraud detection. We run on AWS and make ...
... architects, analysts, developers, and business stakeholders to understand data requirements.- Ensure data quality, security, governance, and best practices across data engineering https://jobeax.com/link/Gpaa4oyRTNYXQU0V Stack:- Python, PySpark, SQL, AWS Glue, AWS Glue Data Catalog, AWS Lambda, Amazon S3, AWS Step Functions, ...
... team and contribute to building and maintaining applications that power the Dun & Bradstreet Contact Pipeline. This role involves working closely with senior data engineers to optimize data ingestion processes, analyze performance bottlenecks, and implement improvements. You will also create metrics to monitor application ...
We are seeking a Data Engineer to join our team. The role will involve building and maintaining data pipelines and collaborating closely with stakeholders. Key responsibilities Design and develop scalable data pipelines using AWS services. Implement ETL processes with AWS Glue, Lambda, and DBT. Work with PySpark and SQL ...
... similar). Experience in building data pipelines or ETL processes. Familiarity with REST APIs and backend frameworks (Flask/Django is a plus). Understanding of data structures and algorithms. Good To Have Experience with big data tools (Spark, Hadoop). Knowledge of cloud platforms (AWS, Azure, or GCP). Exposure to machine ...
... understanding of Apache Airflow and ability to design and manage complex DAGs. - Solid SQL skills and familiarity with data warehouse platforms (e.g., Snowflake, Redshift, BigQuery). - Familiarity with version control tools (Git), CI/CD pipelines, and Agile methodologies. - Exposure to cloud environments like AWS, GCP, or Azure
... data quality, reliability, and performance across data systems Required Skills: - Scala (must-have), python - Apache Spark - Strong coding fundamentals and engineering principles - Azure or on-premises Hadoop ecosystem experience - 5+ years of experience in Data Engineering - Python * SQL * AWS / Azure / GCP Preferred (Bonus) ...
... approach using AWS and some GCP. Develop and optimize ETL and ELT pipelines while working with Data Analysts Design and implement scalable, automated processes for data extraction, processing, and analysis within a Data Mesh architecture Act as a subject matter expert for data technologies and solutions Improve efficiency in ...
... Snowflake, Airflow and dbt You are comfortable and have expertise in data engineering tooling such as Jira, git, buildkite, Terraform, and containers as well as GCP suite, terraform kubernetes, cloud functions You understand standard ETL patterns, modern data warehousing ideas such as data mesh or data vaulting, and data ...
Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Python (Programming ...
... Senior Data Engineer with strong hands-on experience building and supporting data solutions on AWS . The role requires solid experience with Amazon Redshift, S3, AWS Glue, Amazon AppFlow, Lambda, and SQL . You will work on data ingestion, transformation, data pipelines, data warehouse solutions, and integrations across different ...
... Data Factory, Databricks, Synapse, dbt (any two – Mandatory). Data Warehousing: Azure SQL Server/Redshift/Big Query/Databricks/Snowflake (Anyone - Mandatory). Data Visualization: Looker, Power BI, Tableau (Basic understanding to support stakeholder queries). Cloud: Azure (Mandatory), AWS or GCP (Good to have). SQL and Scripting: ...
... entity resolution and record linkage. Strong SQL and Python programming skills. Experience with ETL/ELT pipeline development. Familiarity with cloud platforms like AWS, Azure, or GCP. Knowledge of data engineering best practices and scalable architectures. Experience with Apache Spark or PySpark. Knowledge of Docker, Kubernetes, ...