We are looking for an experienced Data Engineer to design, develop, and manage scalable cloud-native data engineering solutions on AWS. The role involves building automated ETL/ELT pipelines, real-time data processing frameworks, and governed data platforms.
Key Responsibilities:
Build and maintain scalable data pipelines using AWS, Python, SQL, and Databricks.
Work with Snowflake, Apache Airflow, Spark, Kafka, and AWS Kinesis for data processing and orchestration.
Implement containerisation and deployment using Docker and Kubernetes.
Manage CI/CD pipelines and infrastructure automation using Terraform, Git, and Jenkins.
Ensure data quality, governance, monitoring, and performance optimisation.
... deployment. Prepare user training documents and related frameworks to ensure smooth adoption new applications and minimize transition risks. Mandatory Skills: DataBricks - Data Engineering. Experience: 5-8 Years. Reinvent your world. We are building a modern Wipro. We are an end-to-end digital transformation partner with ...
... design discussions, and grow your skills in distributed data processing in a creative and fast-moving environment. You will build and maintain reliable, scalable data pipelines and contribute to the data systems that support marketing across our Labels. Working under the technical direction of senior engineers, you'll turn ...
... Net Zero by 2040. What you'll be doing Build, maintain, and monitor automated data pipelines on the Group's cloud data platform, transforming raw transactional data into reporting schemas and developing BI dashboards to deliver reliable, analytics-ready data. • Build, maintain, and monitor automated ETL/ELT data pipelines ...
... scalable data pipelines and third-party API integrations. - Work with business teams and analysts to interpret business rules and derive insights. - Implement data-quality processes, monitoring, and alerting to maintain accurate and available production data. - Analyze and troubleshoot data issues and assist with their resolution. ...
... effective solutions - Ensure adherence to best practices related to data governance, quality, and security across all data pipelines - Automate and optimize data processing tasks to improve reliability and reduce manual intervention - Drive adoption of emerging big data technologies, cloud engineering, and automation practices ...
... and analytics. - Implement data quality checks, validation frameworks, monitoring, error handling, and pipeline performance improvements. - Collaborate with Data Analysts, Data Scientists, Product Teams, and Business Stakeholders to understand requirements and deliver scalable data solutions. - Follow data engineering ...
... execution hints to improve performance. - IBM DataStage experience preferred. - Able to write SQL code sufficient for most business requirements for pulling data from sources, applying rules to the data, and stocking target data - Proven track record in troubleshooting ETL jobs and addressing production issues like performance ...
... participate in design reviews, and help keep our schemas and contracts sane as the system grows What You'll Bring Must have - 1–3 years of professional software engineering experience, with meaningful exposure to data-intensive systems - Strong programming skills in Python, Java, or Scala, and the ability to write production-quality, ...
Baxter seeks a Data Engineer to develop and support ETL and data transformation jobs using PySpark and IBM DataStage in AWS. The role involves building new pipelines, enhancing existing systems, performing code reviews, and providing technical support for a global data warehouse. Responsibilities - Develop new ETL and data ...
... industry's most diverse portfolio: high-speed trains, metros, trams, maintenance services and integrated systems. Could you be the full-time Industrialization Data Engineer (Master Data Specialist) in Bangalore we’re looking for? Your Role, Your Impact Take on a new challenge and apply your data management and process optimization ...
... agreements (SLAs), data contracts, query and warehouse tuning, and data quality checks to keep the platform reliable and cost-efficient. - Partner with Analytics Engineers, Data Analysts, Data Scientists, Data Governance, and business partners so data lands in the shape they need for downstream use. - Use AI coding agents such ...