... lifecycle. Design and build scalable backend services, REST APIs, integrations, and reusable software components using Python and, where relevant TypeScript. Develop data ingestion, transformation, service-to-service communication, and automation capabilities that support reliable product experiences. Contribute to AI-enabled features ...
... (Extract, Transform, Load) scripts to move data across various systems. Database Management: Write basic SQL queries, create tables, and interact with PostgreSQL databases to support data storage and retrieval. Data Orchestration: Help schedule, monitor, and troubleshoot automated data workflows using Apache Airflow. Stream ...
... data ingestion pipelines leveraging Databricks, Airflow, Kafka, and Terraform. Ensure high performance, reliability, and efficiency by optimizing large-scale data pipelines. Develop data processing workflows using Databricks, Delta Lake, and Spark technologies. Maintain and improve the Data Lakehouse, utilizing Unity Catalog ...
... whether the foundation can be trusted. About Algo8 Algo8 is an industrial and manufacturing AI firm. We work inside plants and R&D centres — with engineers, scientists and plant teams — building the data foundations and AI capabilities that enable industrial organisations to make better decisions. Our work is hands-on and ...
... management plans and documentation, ensuring compliance with industry standards and regulatory requirements. - Participate in the identification and resolution of data discrepancies and issues, working to streamline data flow and enhance data quality throughout the study lifecycle. - Collaborate closely with cross-functional ...
Senior Data Engineer – Data Pipelines & Cloud Databases Location: Pune | Experience: 4+ years | Type: Full-Time Role Overview Design, build, manage enterprise data pipelines on Azure and Databricks. Own schema design, API development, and data infrastructure for analytics and intelligence products. Key Responsibilities ...
... build robust data pipelines, data lakes, and marts to support business analysts and data scientists. Key Responsibilities Modern Data Platform Development: Build data lake components on cloud-based platforms Design and develop data marts for business analysts and data scientists Data Engineering & Pipelines: Design data pipelines ...
... and drive the evolution of our analytical ecosystem. In this role, you will be responsible for designing robust data models and optimizing our Snowflake-based data warehousing environment to support complex business intelligence needs. You will collaborate closely with cross-functional product teams, data scientists, and ...
Job Description – Senior Data Engineer / Platform Re-Engineering Lead (Azure Synapse & Databricks Migration) Role Overview We are looking for a highly experienced Senior Data Engineer to lead the re-engineering of an existing enterprise data platform built on Azure Synapse Analytics. The role requires deep technical seniority ...
... and lead our push into streaming and real-time fraud detection. We run on AWS and make extensive use of Scala Spark, dbt, and Terraform. Build and expose the data lake/lakehouse so teams can understand business and product performance and make better decisions. Design and evolve a data platform that enables scientists and ...
... optimized SQL queries for data extraction, transformation, and analysis.- Build and manage ETL pipelines using AWS Glue.- Work with AWS Glue Data Catalog for metadata management and data discovery.- Design and implement serverless data processing solutions using AWS Lambda.- Use Amazon S3 for scalable data storage and data ...
... techniques. Hands-on experience integrating and optimizing Starburst/Trino, including connecting Starburst from Lambda and Glue ETL jobs. Experience with NoSQL databases such as DynamoDB, MongoDB. Experience working with data formats including Avro, Parquet, JSON, XML, and CSV. Comfortable challenging your peers and leadership ...
... details related to data engineering or a relevant https://jobeax.com/link/cwwOL3BLJkve6nkU Skills :- Azure Databricks, Python, pyspark, sql, ADF- Proficiency in data pipeline design and implementation- Strong understanding of data factory and orchestration toolsPreferred Skills:- Familiarity with advanced data processing techniques- ...
... & Data Engineering (AWS) Strong handson experience with AWS data services, including: Amazon S3, AWS Glue, Athena, Redshift Experience designing cloudnative data lakes and data warehouse architectures on AWS Deep understanding of batch and streaming data pipelines Experience building scalable, faulttolerant data ingestion ...
... work independently and collaboratively to solve complex data challenges and drive continuous improvements across our data engineering practices. 4+ years of data engineering experience, building and managing large-scale data pipelines - Strong proficiency in Python, PySpark, Spark, Databricks, Delta Lake, PowerBI and SQL, ...
Senior Data Engineer Starts Immediately Competitive salary Competitive salary Experience We are looking for a Senior Data Engineer to join our team. In this role, you will design, build, and optimize scalable data pipelines on the Databricks Lakehouse Platform. You will partner with data science, analytics, and business ...
... capture the most granular data. Align the engineering and product teams on the Data Granularity and accuracy requirements. Build Alerts to detect bugs in the data pipeline. Perform data analysis to test hypothesis and uncover factor influencing metric variations Build Feature Store for Data Science implementations Overall ...
... ~ Manage budgets related to data sourcing, annotation operations, and vendor contracts. 8+ years of overall experience in GIS, remote sensing, or geospatial data operations; Strong hands-on knowledge of satellite imagery sourcing (optical, SAR, multispectral) and remote sensing data pipelines. - Proven experience managing ...
Role OverviewWe are seeking a high-caliber Data Migration Consultant Developer with proven expertise in Guidewire Data Migration methodology and tools . The ideal candidate must hold a Mammoth Certification (Data Migration Track) and bring extensive, hands-on experience in migrating client data from legacy Policy systems ...