... Engineer will own the end-to-end operationalisation of machine learning, large language model (LLM), and agentic AI workloads on the Bajaj Finance Enterprise Data Platform — a 5PB+ medallion lakehouse built on Azure Databricks and Unity Catalog. This role sits at the intersection of data engineering, model lifecycle management, ...
... new technologies and approaches to innovate with increasingly large data sets. Drive automation and efficiency in Data ingestion, data movement and data access workflows by innovation and collaboration. Understand, implement and enforce Software development standards and engineering principles in the Big Data space. Work ...
... batch jobs, database replication & Kafka- Set up and maintain monitoring/observability for data pipelines- Support ML pipelines, Airflow workflows and production data science infrastructureIDEAL PROFILE:Looking for candidates with strong hands-on experience in:- Data Operations- Production Data Engineering- Data Platform Support- ...
... for critical database incidents. Establish governance frameworks, operational procedures, standards, and best practices for database management. Mentor junior database administrators and provide technical leadership across database initiatives. Expertise You'll Bring: - 10-15 years of experience as a SQL Server Database ...
Supports work flow and solutions; trouble shoots user errors and supports reporting capabilities. Utilizes system monitoring utilities to monitor system availability. Extracts and compiles data system monitoring data to create availability scorecards and reports Perform research, systems analysis and design and makes recommendations ...
... years of experience • Minimum 3 years of experience in design, build and deployment of Ab Initio-based applications • Expertise in handling complex large-scale Data Lake and Warehouse environments • Hands-on experience writing complex SQL queries, exporting and importing large amounts of data using utilities • Excellent verbal ...
... pipelines in cloud or modern data platforms. - 4+ years of hands-on experience with Databricks or similar cloud-based lakehouse platforms supporting enterprise data lakes, data warehouses, and business intelligence - 2+ years of experience using dbt (or similar transformation frameworks) including model structuring, tests, ...
... ensuring that data pipelines are robust, efficient, and scalable Part of a cross-disciplinary team, working closely with other data engineers, software engineers, data scientists, data managers and business partners. Implements and maintains reliable and scalable data infrastructure to move, process and serve data. Writes, deploys ...
Administers, analyzes, and prioritizes systems issues and negotiates a course of action for resolution. Supports work flow and solutions; trouble shoots user errors and supports reporting capabilities. Utilizes system monitoring utilities to monitor system availability. Extracts and compiles data system monitoring data ...
... affordable healthcare. Build and manage digital analytics datasets and reporting solutions Develop data pipelines and analytical reporting frameworks Support data validation, UAT, process improvements, and analytics initiatives. Experience & Qualification: Python Power BI & Dashboard Development Data Modeling & Data Validation ...
... Databricks platform. The ideal candidate will have a robust background in backend development, Data science model basics, and cloud-based solutions to drive data-driven decision-making and innovation within our portfolio. Key skills - Python, FastAPI Framework, REST API design, Pydantic models, Async programming, Modular ...
... design, API development, and data infrastructure for analytics and intelligence products. Key Responsibilities Data Pipeline Development Implement robust, scalable data pipelines using Microsoft Azure and Databricks stack Build reusable data pipeline components and frameworks Design and optimize data workflows for performance ...
... enhancing data storage and access, and ensuring seamless data consumption through APIs. The ideal candidate will work with Azure Cloud technologies to build robust data pipelines, data lakes, and marts to support business analysts and data scientists. Key Responsibilities Modern Data Platform Development: Build data lake components ...
... configurations, and cost optimization strategies.- Proven ability to write complex, highly optimized SQL queries and stored procedures to handle large-scale data transformations.- Strong experience in building and maintaining robust ETL/ELT workflows using modern data engineering tools and frameworks.- Exceptional communication ...
... data pipelines and ETL workflows on AWS.- Develop efficient data processing solutions using python and pySpark.- Write complex and optimized SQL queries for data extraction, transformation, and analysis.- Build and manage ETL pipelines using AWS Glue.- Work with AWS Glue Data Catalog for metadata management and data discovery.- ...
... with stakeholders. Key responsibilities Design and develop scalable data pipelines using AWS services. Implement ETL processes with AWS Glue, Lambda, and DBT. Work with PySpark and SQL to transform and cleanse data. Configure and manage Airflow workflows and Amazon EMR clusters. Integrate data services with API Gateway and ...
... have legal authorization to work in the country where this role is based on the first day of employment. Address client inquiries related to Addepar's portfolio data feeds and on general data product functionalities within established SLAs. Manage and complete requests from internal Data teams that require client outreach ...
... orchestration and automation of data workflows. Ensure the reliability, scalability, and efficiency of data pipelines for ingestion, transformation, and storage. Work with cross-functional teams to understand data needs and deliver high-quality solutions. Troubleshoot and resolve data pipeline issues in production environments. ...
... Own end-to-end delivery of data products—from raw source data ingestion through transformation to governed, consumption-ready datasets. Collaborate with the Data Platform team on pipeline integration, CI/CD workflows, and adherence to shared coding and deployment standards. - Data Quality & Analysis: Implement data quality ...
... data engineering, and DevOps automation. The ideal candidate will manage and optimize Cloudera environments, build CI/CD pipelines, and support enterprise-scale data processing workloads. Key Responsibilities Administer and support Cloudera CDP/CDH platforms, including HDFS, Hive, Spark, YARN, Hue, and CDE. Develop, deploy, ...