... trails. Establish and maintain Feature Store — curated, reusable feature sets across credit risk, fraud, customer propensity, and collections models — ensuring data freshness, SLA adherence, and lineage traceability. Operationalise Databricks Model Serving (serverless + provisioned endpoints) and Mosaic AI for scalable, low-latency ...
... scalable data pipelines using spark, Scala/ python on Hadoop or object storage. Leverage new technologies and approaches to innovate with increasingly large data sets. Drive automation and efficiency in Data ingestion, data movement and data access workflows by innovation and collaboration. Understand, implement and enforce ...
... batch jobs, database replication & Kafka- Set up and maintain monitoring/observability for data pipelines- Support ML pipelines, Airflow workflows and production data science infrastructureIDEAL PROFILE:Looking for candidates with strong hands-on experience in:- Data Operations- Production Data Engineering- Data Platform Support- ...
... years of experience • Minimum 3 years of experience in design, build and deployment of Ab Initio-based applications • Expertise in handling complex large-scale Data Lake and Warehouse environments • Hands-on experience writing complex SQL queries, exporting and importing large amounts of data using utilities • Excellent verbal ...
... business stakeholders to: - Define and maintain enterprise data models and domain boundaries using traditional and AI-enabled methods - Design and govern reusable data assets and data products - Establish standards and patterns for robust, scalable data pipelines - Ensure data is discoverable, trustworthy, and fit-for-purpose ...
... preparing data for prescriptive and predictive modeling. Data engineers also develop data set processes for data modeling, mining, and production, integrate new data management technologies and software engineering tools into existing structures, and collaborate with data scientists and analysts to ensure data accuracy and ...
... affordable healthcare. Build and manage digital analytics datasets and reporting solutions Develop data pipelines and analytical reporting frameworks Support data validation, UAT, process improvements, and analytics initiatives. Experience & Qualification: Python Power BI & Dashboard Development Data Modeling & Data Validation ...
... DevOps. Develop reusable, modular, and maintainable code following software engineering best practices and coding standard. Optimize application performance, scalability, and reliability for high-volume data processing workloads. Implement automated testing, including unit test cases using pytest. Collaborate with front end ...
... pipelines on Azure and Databricks. Own schema design, API development, and data infrastructure for analytics and intelligence products. Key Responsibilities Data Pipeline Development Implement robust, scalable data pipelines using Microsoft Azure and Databricks stack Build reusable data pipeline components and frameworks ...
... build robust data pipelines, data lakes, and marts to support business analysts and data scientists. Key Responsibilities Modern Data Platform Development: Build data lake components on cloud-based platforms Design and develop data marts for business analysts and data scientists Data Engineering & Pipelines: Design data pipelines ...
... responsible for designing robust data models and optimizing our Snowflake-based data warehousing environment to support complex business intelligence needs. You will collaborate closely with cross-functional product teams, data scientists, and senior stakeholders to translate intricate business requirements into scalable, high-performance ...
... optimized SQL queries for data extraction, transformation, and analysis.- Build and manage ETL pipelines using AWS Glue.- Work with AWS Glue Data Catalog for metadata management and data discovery.- Design and implement serverless data processing solutions using AWS Lambda.- Use Amazon S3 for scalable data storage and data ...
We are seeking a Data Engineer to join our team. The role will involve building and maintaining data pipelines and collaborating closely with stakeholders. Key responsibilities Design and develop scalable data pipelines using AWS services. Implement ETL processes with AWS Glue, Lambda, and DBT. Work with PySpark and SQL ...
... inquiries related to Addepar's portfolio data feeds and on general data product functionalities within established SLAs. Manage and complete requests from internal Data teams that require client outreach and/or action to resolve data verification issues. Investigate client reported bugs and data processing issues, and triage ...
... and data processing logic using Java. Build and maintain DAGs in Apache Airflow for orchestration and automation of data workflows. Ensure the reliability, scalability, and efficiency of data pipelines for ingestion, transformation, and storage. Work with cross-functional teams to understand data needs and deliver high-quality ...
... Own end-to-end delivery of data products—from raw source data ingestion through transformation to governed, consumption-ready datasets. Collaborate with the Data Platform team on pipeline integration, CI/CD workflows, and adherence to shared coding and deployment standards. - Data Quality & Analysis: Implement data quality ...
... MAKE AN IMPACT Job Title: Data Engineer _DEPS Experience: 3–5 Years Location: Bangalore and Pune (Hybrid – Client Office) Job Summary: Seeking a skilled Senior Data DevOps Engineer having experience in Cloudera platforms, data engineering, and DevOps automation. The ideal candidate will manage and optimize Cloudera environments, ...
... deploying complex data architectures within Microsoft Fabric and the broader Azure ecosystem.- Advanced proficiency in Python and SQL for building sophisticated data processing pipelines and complex analytical models.- Strong background in data engineering principles, including ETL/ELT design, data modeling, and performance ...
... Docker, Cloud-Native Infrastructure, CI/CD pipelines. Industry Context: Data engineering experience in Banking Risk, Retail Products, Cards, Mortgage, Deposits, or Wealth Management. Assumed Requirements / Certifications: Databricks Certified Data Engineer, AWS Certified Data Analytics, or Azure Data Engineer Associate.
Data Pipeline Development & Operations - Design, build, and operate scalable and reliable data pipelines on the Databricks platform - Develop end-to-end data workflows from ingestion through transformation to consumption - Implement robust error handling, monitoring, and alerting mechanisms - Ensure data pipeline reliability, ...