... data ingestion pipelines leveraging Databricks, Airflow, Kafka, and Terraform. Ensure high performance, reliability, and efficiency by optimizing large-scale data pipelines. Develop data processing workflows using Databricks, Delta Lake, and Spark technologies. Maintain and improve the Data Lakehouse, utilizing Unity Catalog ...
... non-technical stakeholders why data is or is not ready, and that conversation matters as much as the pipeline. Useful, Not Essential - Manufacturing or industrial data experience - Kafka, Debezium or other change-data-capture tooling - Metadata catalogue and governance tooling - Experience defining acceptance criteria on a client ...
... management plans and documentation, ensuring compliance with industry standards and regulatory requirements. - Participate in the identification and resolution of data discrepancies and issues, working to streamline data flow and enhance data quality throughout the study lifecycle. - Collaborate closely with cross-functional ...
Senior Data Engineer – Data Pipelines & Cloud Databases Location: Pune | Experience: 4+ years | Type: Full-Time Role Overview Design, build, manage enterprise data pipelines on Azure and Databricks. Own schema design, API development, and data infrastructure for analytics and intelligence products. Key Responsibilities ...
... opportunity for a Data Scientist to join a new and rapidly growing intraday team. In this role you will partner with our close-knit team of quantitative researchers, data engineers, technologists and data sourcing colleagues to research, engineer, and validate quantitative signals derived from high-frequency equity market data ...
... build robust data pipelines, data lakes, and marts to support business analysts and data scientists. Key Responsibilities Modern Data Platform Development: Build data lake components on cloud-based platforms Design and develop data marts for business analysts and data scientists Data Engineering & Pipelines: Design data pipelines ...
Role Overview : We are seeking a seasoned Data Engineering professional to spearhead our data architecture initiatives and drive the evolution of our analytical ecosystem. In this role, you will be responsible for designing robust data models and optimizing our Snowflake-based data warehousing environment to support complex ...
Data Science Data Engineer Work Type: Full Time At VIDA we're building the future of digital identity. As a Data Engineer you'll work across the stack—from platform and infrastructure to data pipelines and end-user tooling. You'll help us modernize our batch ETLs and lead our push into streaming and real-time fraud detection. ...
... optimized SQL queries for data extraction, transformation, and analysis.- Build and manage ETL pipelines using AWS Glue.- Work with AWS Glue Data Catalog for metadata management and data discovery.- Design and implement serverless data processing solutions using AWS Lambda.- Use Amazon S3 for scalable data storage and data ...
... techniques. Hands-on experience integrating and optimizing Starburst/Trino, including connecting Starburst from Lambda and Glue ETL jobs. Experience with NoSQL databases such as DynamoDB, MongoDB. Experience working with data formats including Avro, Parquet, JSON, XML, and CSV. Comfortable challenging your peers and leadership ...
About the Company :We are a leading technology firm dedicated to innovation and excellence in data https://jobeax.com/link/fc637Lq7iXaUjBEJ mission is to empower businesses through advanced data engineering and analytics, fostering a culture of collaboration and continuous https://jobeax.com/link/yeka7e9uOhXV2ngJ the Role ...
... & Data Engineering (AWS) Strong handson experience with AWS data services, including: Amazon S3, AWS Glue, Athena, Redshift Experience designing cloudnative data lakes and data warehouse architectures on AWS Deep understanding of batch and streaming data pipelines Experience building scalable, faulttolerant data ingestion ...
Senior Data Engineer Starts Immediately Competitive salary Competitive salary Experience We are looking for a Senior Data Engineer to join our team. In this role, you will design, build, and optimize scalable data pipelines on the Databricks Lakehouse Platform. You will partner with data science, analytics, and business ...
... capture the most granular data. Align the engineering and product teams on the Data Granularity and accuracy requirements. Build Alerts to detect bugs in the data pipeline. Perform data analysis to test hypothesis and uncover factor influencing metric variations Build Feature Store for Data Science implementations Overall ...
... America's Top GreenTech Companies for the 3rd year in a row. Deloitte Technology Fast 500™ ranked AiDASH No. AiDash is looking for an experienced Manager – AI Data Ops to lead a team responsible for sourcing, processing, and annotating satellite and remote sensing imagery that powers our AI/ML models. This role blends hands-on ...
... stakeholders, including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs. Keep our data separated and secure across national boundaries through multiple data centres and AWS regions. Create data tools for analytics and data scientist team members ...
Role OverviewWe are seeking a high-caliber Data Migration Consultant Developer with proven expertise in Guidewire Data Migration methodology and tools . The ideal candidate must hold a Mammoth Certification (Data Migration Track) and bring extensive, hands-on experience in migrating client data from legacy Policy systems ...
... BigQuery, Cloud Functions, Composer, GCS Proficient hands-on programming experience in Spark/Scala (python/java) Proficient in building production level ETL/ELT data pipelines from data ingestion to consumption Data Engineering knowledge (such as Data Lake, Data warehouse - Redshift/Hive/Snowflake, Integration, Migration) ...
... https://jobeax.com/link/MJvWAZuK638tHuBb RESPONSIBILITIES : Tech Stack & Skills : - Experience in model development using Python/PySpark libraries. Development on Databricks or Dataiku DSS is a plus.- Strong experience on Spark with Scala/Python/Java.- Proficiency in building, training, and evaluating state-of-the-art machine ...