... ensuring that data pipelines are robust, efficient, and scalable Part of a cross-disciplinary team, working closely with other data engineers, software engineers, data scientists, data managers and business partners. Implements and maintains reliable and scalable data infrastructure to move, process and serve data. Writes, deploys ...
... Development Develop, test, and optimize batch and stream-processing pipelines to handle real-time IoT telematics, state-of-charge tracking, and transactional business data. Build efficient ingestion mechanisms from internal systems, IoT hardware modules, APIs, and third-party operational platforms using languages like Go, Rust, ...
... affordable healthcare. Build and manage digital analytics datasets and reporting solutions Develop data pipelines and analytical reporting frameworks Support data validation, UAT, process improvements, and analytics initiatives. Experience & Qualification: Python Power BI & Dashboard Development Data Modeling & Data Validation ...
Consultant Data Engineer Databricks | Azure | PySpark | Spark SQL - Bengaluru / Hyderabad / GurugramLooking for an experienced Data Engineer with strong expertise in Azure Databricks, PySpark, Spark SQL, Azure Data Factory (ADF), ADLS, ETL, and SQL. You will design scalable data pipelines, build modern Lakehouse solutions, ...
... packaging Develop and operationalize production-grade ML solutions with strong focus on reproducibility, maintainability, scalability, and measurable business impact Partner with data engineers and software engineers to integrate models into applications, APIs, workflows, and downstream business systems Design and implement MLOps ...
... data and the decision they support. Quality-check the team's work and guide models to production through cross-team collaboration. Lead project delivery across timelines, scope and team allocation, often across parallel engagements. Mentor and manage a team of data scientists, and keep yourself and the team current with advancements ...
... and deliver critical projects on time. - Your use of this page is subject to Employ's Privacy Notice and Cookie Policy, which governs the processing of visitor data on this platform. We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or ...
... lifecycle. Design and build scalable backend services, REST APIs, integrations, and reusable software components using Python and, where relevant TypeScript. Develop data ingestion, transformation, service-to-service communication, and automation capabilities that support reliable product experiences. Contribute to AI-enabled features ...
... application development to join our team. The role involves managing development of FastAPI based applications to run large data modeling, and leveraging the Databricks platform. The ideal candidate will have a robust background in backend development, Data science model basics, and cloud-based solutions to drive data-driven ...
... and troubleshoot automated data workflows using Apache Airflow. Stream Processing: Collaborate with the team to build and monitor real-time or near-real-time data streaming applications using Apache Flink. Coding & Debugging: Write clean, readable, and well-documented code in Python. Continuous Learning: Actively participate ...
... or similar real-time streaming technologies. - Strong command of SQL and experience with data modelling, dimensional modelling, partitioning, and large-scale data processing. - Experience with Airflow, MWAA, Step Functions , or comparable workflow orchestration tools. - Strong understanding of batch and real-time data processing ...
... data ingestion pipelines leveraging Databricks, Airflow, Kafka, and Terraform. Ensure high performance, reliability, and efficiency by optimizing large-scale data pipelines. Develop data processing workflows using Databricks, Delta Lake, and Spark technologies. Maintain and improve the Data Lakehouse, utilizing Unity Catalog ...
... Discovery is part of the job — expect to map what is actually there before you design against it. Who This Role Is Not For We would rather be direct than waste your time. This role is probably not a fit if: - Your data engineering experience is entirely on managed cloud services and you have never sized, deployed or operated infrastructure ...
... responsibilities include: - Create and maintain detailed data management plans and documentation, ensuring compliance with industry standards and regulatory requirements. - Participate in the identification and resolution of data discrepancies and issues, working to streamline data flow and enhance data quality throughout the study lifecycle. ...
Senior Data Engineer – Data Pipelines & Cloud Databases Location: Pune | Experience: 4+ years | Type: Full-Time Role Overview Design, build, manage enterprise data pipelines on Azure and Databricks. Own schema design, API development, and data infrastructure for analytics and intelligence products. Key Responsibilities ...
... disparate sources.- Optimize Snowflake database performance by managing compute resources, clustering keys, and materialized views to reduce latency for end-users.- Partner with business stakeholders to define data requirements, ensuring that the data warehouse architecture aligns with long-term strategic business objectives.- ...
Note: This job role is part of MetLife's Hack4Job India (a hiring hackathon). Only shortlisted candidates will be invited. Department: Global Technology Role Overview MetLife is seeking an experienced Data Engineer to drive our digital and AI transformation journey. This role focuses on building modern data platforms, enhancing ...
... workloads from legacy data warehouse or Synapse environments to a Databricks Lakehouse Ability to re-implement governance and control frameworks natively in Databricks (audit logging, reconciliation, DQ checks) Experience with Delta Lake features: MERGE, CDC, time travel, schema enforcement Data Engineering & Development ...
... logic (e.g., dbt), including testing, documentation, and data quality. Terraform) and contribute to CI/CD and deployment practices. Partner with Engineering, and Data Science to deliver low-latency signals and features for real-time decisioning. Solid engineering foundation with 5+ years of relevant experience in data engineering ...