AWS / GCP Data Engineer in Bengaluru, India

• Hybrid

... Python. Develop and optimize large-scale data transformations using PySpark. Ensure data quality, reliability, and performance of data pipelines. GCP BigQuery Data Ingestion & ETL/ELT Data Pipeline Orchestration Python PySpark Strong SQL and data engineering fundamentals hands-on experience in GCP-based data platforms.

... pipelines to facilitate seamless data flow across systems and platforms. - Database Management: Managing databases, ensuring their integrity, and implementing data storage and retrieval solutions. - Cloud Services Integration: Leveraging cloud services such as MS Azure, GCP, and AWS to architect and deploy scalable data ...

• On-site

... on-premises to cloud Spark code optimization and Medallion Architecture. Strong knowledge of Databricks and its components, including Delta Live Table (DLT) pipeline implementations. Familiarity with AWS services (experience with additional cloud platforms like GCP or Azure is a plus). AWS/GCP/Azure Data Engineer Certification.

• Full-time

... Work Type: Full Time At VIDA we're building the future of digital identity. As a Data Engineer you'll work across the stack—from platform and infrastructure to data pipelines and end-user tooling. You'll help us modernize our batch ETLs and lead our push into streaming and real-time fraud detection. We run on AWS and make ...

• Hybrid

... architects, analysts, developers, and business stakeholders to understand data requirements.- Ensure data quality, security, governance, and best practices across data engineering https://jobeax.com/link/Gpaa4oyRTNYXQU0V Stack:- Python, PySpark, SQL, AWS Glue, AWS Glue Data Catalog, AWS Lambda, Amazon S3, AWS Step Functions, ...

... team and contribute to building and maintaining applications that power the Dun & Bradstreet Contact Pipeline. This role involves working closely with senior data engineers to optimize data ingestion processes, analyze performance bottlenecks, and implement improvements. You will also create metrics to monitor application ...

We are seeking a Data Engineer to join our team. The role will involve building and maintaining data pipelines and collaborating closely with stakeholders. Key responsibilities Design and develop scalable data pipelines using AWS services. Implement ETL processes with AWS Glue, Lambda, and DBT. Work with PySpark and SQL ...

... similar). Experience in building data pipelines or ETL processes. Familiarity with REST APIs and backend frameworks (Flask/Django is a plus). Understanding of data structures and algorithms. Good To Have Experience with big data tools (Spark, Hadoop). Knowledge of cloud platforms (AWS, Azure, or GCP). Exposure to machine ...

... understanding of Apache Airflow and ability to design and manage complex DAGs. - Solid SQL skills and familiarity with data warehouse platforms (e.g., Snowflake, Redshift, BigQuery). - Familiarity with version control tools (Git), CI/CD pipelines, and Agile methodologies. - Exposure to cloud environments like AWS, GCP, or Azure

• On-site

... data quality, reliability, and performance across data systems Required Skills: - Scala (must-have), python - Apache Spark - Strong coding fundamentals and engineering principles - Azure or on-premises Hadoop ecosystem experience - 5+ years of experience in Data Engineering - Python * SQL * AWS / Azure / GCP Preferred (Bonus) ...

Rs 2.5 - 4.5 lakhs p.a.

... approach using AWS and some GCP. Develop and optimize ETL and ELT pipelines while working with Data Analysts Design and implement scalable, automated processes for data extraction, processing, and analysis within a Data Mesh architecture Act as a subject matter expert for data technologies and solutions Improve efficiency in ...

$180 USD

... Snowflake, Airflow and dbt You are comfortable and have expertise in data engineering tooling such as Jira, git, buildkite, Terraform, and containers as well as GCP suite, terraform kubernetes, cloud functions You understand standard ETL patterns, modern data warehousing ideas such as data mesh or data vaulting, and data ...

• Full-time

Project Role : Data Engineer Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems. Must have skills : Python (Programming ...

... experience in data engineering. - Proficiency in: Programming languages - Python, Java, SQL, Spark SQL. - Data technologies - Hadoop, PySpark, NoSQL databases. - Data visualization tools - Qliksense, Tableau, Power BI - Cloud platorms - Azure Data Factory, Azure Databricks, AWS NovintiX is a fast-growing engineering and digital ...

$180 USD

... Snowflake, Airflow and dbt You are comfortable and have expertise in data engineering tooling such as Jira, git, buildkite, Terraform, and containers as well as GCP suite, terraform kubernetes, cloud functions You understand standard ETL patterns, modern data warehousing ideas such as data mesh or data vaulting, and data ...

Senior Engineer, DevOps Primary Skills ~ GCP Cloud Build, GCP Deploy, Cloud Run, Kubernetes, GCP Cloud CI/CD Pipeline, GCP Spinnaker Specialization ~ GCP DevOps Engineering: Engineer Job requirements ~ Experience Range: With at least 2 years of hands-on experience in GCP DevOps engineering, including CI/CD pipeline management ...

• Remote
• Contract

... entity resolution and record linkage. Strong SQL and Python programming skills. Experience with ETL/ELT pipeline development. Familiarity with cloud platforms like AWS, Azure, or GCP. Knowledge of data engineering best practices and scalable architectures. Experience with Apache Spark or PySpark. Knowledge of Docker, Kubernetes, ...

Data Engineer – Python & Pyspark (AWS) Exp: 6+ yrs Location : Chennai Client Place Guindy Work Mode : All 5 days Role Overview We're seeking a Senior Data Engineer with strong hands-on expertise in Python, PySpark, and AWS to design, build, and operate scalable data pipelines for analytics and operational workloads. You ...

... processes, optimising data delivery and re-designing infrastructure for greater scalability.- Collaborate with stakeholders to ensure seamless integration of data with internal data marts, enhancing advanced reporting.- Setup and maintain data ingestion, streaming, scheduling, and job monitoring automation using AWS services.- ...