Data Engineer - Bangalore in Bangalore, India - Jobeax
Vacancy description
Data Engineer - Bangalore in Bangalore, भारत
Duobridge Talent Solution
Rs 15 - 18 lakhs p.a.
Bangalore, भारत
Job Description:
We are looking for an experienced Data Engineer to design, develop, and manage scalable cloud-native data engineering solutions on AWS. The role involves building automated ETL/ELT pipelines, real-time data processing frameworks, and governed data platforms.
Key Responsibilities:
Build and maintain scalable data pipelines using AWS, Python, SQL, and Databricks.
Work with Snowflake, Apache Airflow, Spark, Kafka, and AWS Kinesis for data processing and orchestration.
Implement containerisation and deployment using Docker and Kubernetes.
Manage CI/CD pipelines and infrastructure automation using Terraform, Git, and Jenkins.
Ensure data quality, governance, monitoring, and performance optimisation.
... scalable data pipelines and third-party API integrations. - Work with business teams and analysts to interpret business rules and derive insights. - Implement data-quality processes, monitoring, and alerting to maintain accurate and available production data. - Analyze and troubleshoot data issues and assist with their resolution. ...
Baxter seeks a Data Engineer to develop and support ETL and data transformation jobs using PySpark and IBM DataStage in AWS. The role involves building new pipelines, enhancing existing systems, performing code reviews, and providing technical support for a global data warehouse. Responsibilities - Develop new ETL and data ...
... participate in design reviews, and help keep our schemas and contracts sane as the system grows What You'll Bring Must have - 1–3 years of professional software engineering experience, with meaningful exposure to data-intensive systems - Strong programming skills in Python, Java, or Scala, and the ability to write production-quality, ...
... execution hints to improve performance. - IBM DataStage experience preferred. - Able to write SQL code sufficient for most business requirements for pulling data from sources, applying rules to the data, and stocking target data - Proven track record in troubleshooting ETL jobs and addressing production issues like performance ...
... and analytics. - Implement data quality checks, validation frameworks, monitoring, error handling, and pipeline performance improvements. - Collaborate with Data Analysts, Data Scientists, Product Teams, and Business Stakeholders to understand requirements and deliver scalable data solutions. - Follow data engineering ...
... effective solutions - Ensure adherence to best practices related to data governance, quality, and security across all data pipelines - Automate and optimize data processing tasks to improve reliability and reduce manual intervention - Drive adoption of emerging big data technologies, cloud engineering, and automation practices ...
... immigration processing fees, etc.). Follow the LINK to understand more about recruitment scams in the name of ExxonMobil. Functional Skills Applied Software Engineering for Data Data Architecture & Strategy Data Integration Data Management Data Visualization Data Modeling Machine Learning Nothing herein is intended to override ...
... across raw, curated, and consumption layers. - Collaborate with data engineers and architects to define data inputs, transformations, outputs, dependencies, and data flows. - Define data quality standards, validation rules, business logic, metadata requirements, and data control mechanisms. - Ensure data requirements address ...
... immigration processing fees, etc.). Follow the LINK to understand more about recruitment scams in the name of ExxonMobil. Functional Skills Applied Software Engineering for Data Data Architecture & Strategy Data Integration Data Management Data Visualization Data Modeling Machine Learning Nothing herein is intended to override ...
... dollar portfolios. What are we looking for? We're in search of a Data Engineer who brings a passion for innovation and problem-solving, not limited to traditional data engineering. This role is for those with deep expertise in building large scale data pipelines, SQL optimization, and cloud infrastructure, excited to design ...