We are looking for an experienced Data Engineer to design, develop, and manage scalable cloud-native data engineering solutions on AWS. The role involves building automated ETL/ELT pipelines, real-time data processing frameworks, and governed data platforms.
Key Responsibilities:
Build and maintain scalable data pipelines using AWS, Python, SQL, and Databricks.
Work with Snowflake, Apache Airflow, Spark, Kafka, and AWS Kinesis for data processing and orchestration.
Implement containerisation and deployment using Docker and Kubernetes.
Manage CI/CD pipelines and infrastructure automation using Terraform, Git, and Jenkins.
Ensure data quality, governance, monitoring, and performance optimisation.
... Use SQL for data querying, validation, and data modeling, as required. 3. Data Engineering Fundamentals - Design, develop, and maintain ETL/ELT pipelines for data ingestion and transformation. - Build efficient data models and optimize data processing workflows. - Work with structured and unstructured data from multiple ...
... Engineer (Platform Engineering Focus) The Opportunity: The Platform Engineer builds, automates, and supports the shared cloud platform services that enable reliable data, analytics, and AI delivery. This hands-on role works with Senior Platform Engineers, Data Engineers, Analytics Engineers, Security, and Architecture teams to ...
... deployment. Prepare user training documents and related frameworks to ensure smooth adoption new applications and minimize transition risks. Mandatory Skills: DataBricks - Data Engineering. Experience: 5-8 Years. Reinvent your world. We are building a modern Wipro. We are an end-to-end digital transformation partner with ...
... design discussions, and grow your skills in distributed data processing in a creative and fast-moving environment. You will build and maintain reliable, scalable data pipelines and contribute to the data systems that support marketing across our Labels. Working under the technical direction of senior engineers, you'll turn ...
... Net Zero by 2040. What you'll be doing Build, maintain, and monitor automated data pipelines on the Group's cloud data platform, transforming raw transactional data into reporting schemas and developing BI dashboards to deliver reliable, analytics-ready data. • Build, maintain, and monitor automated ETL/ELT data pipelines ...
... scalable data pipelines and third-party API integrations. - Work with business teams and analysts to interpret business rules and derive insights. - Implement data-quality processes, monitoring, and alerting to maintain accurate and available production data. - Analyze and troubleshoot data issues and assist with their resolution. ...
... effective solutions - Ensure adherence to best practices related to data governance, quality, and security across all data pipelines - Automate and optimize data processing tasks to improve reliability and reduce manual intervention - Drive adoption of emerging big data technologies, cloud engineering, and automation practices ...
... and analytics. - Implement data quality checks, validation frameworks, monitoring, error handling, and pipeline performance improvements. - Collaborate with Data Analysts, Data Scientists, Product Teams, and Business Stakeholders to understand requirements and deliver scalable data solutions. - Follow data engineering ...
... execution hints to improve performance. - IBM DataStage experience preferred. - Able to write SQL code sufficient for most business requirements for pulling data from sources, applying rules to the data, and stocking target data - Proven track record in troubleshooting ETL jobs and addressing production issues like performance ...
... participate in design reviews, and help keep our schemas and contracts sane as the system grows What You'll Bring Must have - 1–3 years of professional software engineering experience, with meaningful exposure to data-intensive systems - Strong programming skills in Python, Java, or Scala, and the ability to write production-quality, ...
Baxter seeks a Data Engineer to develop and support ETL and data transformation jobs using PySpark and IBM DataStage in AWS. The role involves building new pipelines, enhancing existing systems, performing code reviews, and providing technical support for a global data warehouse. Responsibilities - Develop new ETL and data ...