... robust and scalable ETL/ELT pipelines using Apache Spark and Delta Lake to transform raw (bronze) data into trusted curated (silver) and analytics-ready (gold) data layers. Apply schema evolution and data versioning to support agile data development. Platform Integration & Data Ingestion: Connect and ingest data from enterprise ...
... hands-on experience in data science.- Strong proficiency in Python and libraries such as NumPy, pandas, scikit-learn, TensorFlow/PyTorch.- Expertise in SQL for data extraction, transformation, and analysis.- Proven experience with PySpark or distributed data processing frameworks.- Deep understanding of machine learning and ...
... Kinesis.- Modern Data Stack : Design and scale a unified Lakehouse/Warehouse (e.g., Databricks, Snowflake) to handle both streaming telemetry and complex enterprise data (ERP, CRM, MES).- Data Pipelines : Build automated, resilient ETL/ELT workflows using tools like Apache Airflow, dbt, and PySpark to ensure high data quality ...
... strategies. Develop and maintain systems using Python, SQL, PySpark, and Django for front-end development. Work directly with clients and end-users and address their data needs Utilize databases and tools including and not limited to, Postgres, Redshift, Airflow, and MongoDB to support our data ecosystem. Leverage AI frameworks ...
... in shared data. Lead data governance and compliance activities - Support the implementation of data governance policies, standards, and practices across the data platform. - Maintain metadata, data lineage, and business classifications — working with tooling such as Microsoft Purview to ensure the data catalogue is accurate ...
... workflows for real-world use cases. Mindrift is looking for highly skilled Senior Python Data Scraping Engineers to join the Tendem project and drive specialized data scraping workflows for real-world applications. In this role, you'll apply your expertise in web scraping, data extraction, and data processing to deliver accurate, ...
... workflows for real-world use cases. Mindrift is looking for highly skilled Senior Python Data Scraping Engineers to join the Tendem project and drive specialized data scraping workflows for real-world applications. In this role, you'll apply your expertise in web scraping, data extraction, and data processing to deliver accurate, ...
... requirements and technical execution, ensuring high-quality solution delivery Mentor junior scientists on coding standards, experimental design, and model ethics Audit data quality and process integrity across the entire modelling lifecycle Mentor and guide junior data scientist Stay updated with the latest trends in data science ...
... identification, documenting data sources, methodologies, and quality findings with recommendations for improvement. Designing and building data pipelines to automate data movement and processing. Apply advanced analytical techniques to large datasets to uncover trends and correlations, develop validated logical data models, and ...
... build, and optimize high-throughput data pipelines using Python, R, and PostgreSQL to handle market and execution data. - Build and manage robust, scheduled data workflows using tools like Airflow/Dagster to ensure 24/7 data availability for global markets. - Monitor pipeline health, debug data discrepancies in logs, and ...
... secure and high performance data platforms and architectures Lead the development of batch and real time data processing solutions using modern cloud and big data technologies Define data engineering standards best practices and governance frameworks Drive architectural decisions for data lakes lakehouses data warehouses ...
... documentation for data enablement & quality processes. - Liaise closely with Labcorp Data Insight colleagues to ensure feedback loops are in place around any data issues. - Assist in the development and implementation of Data Quality initiatives, including data quality checks, data cleaning, data enrichment, and normalization ...
... the enterprise data lake, ensuring seamless flow and quality. Enhance cloud data infrastructure on AWS or Azure for scalability and performance. Utilize big data tools and frameworks to optimize data acquisition and preparation. Build architectures to move data to/from data lakes and data warehouses for advanced analytics. ...
... and processing - Experience with big data and real-time processing technologies, including Apache Spark, Hadoop, Kafka, and Amazon Kinesis - Experience with data modelling and database design across Structured Query Language and NoSQL platforms - Understanding of data governance, data quality, and security best practices ...
... huge datasets and have experience with the organization and curation of data for analytics. You have a strategic and long-term view on architecture of advanced data eco systems. You are experienced in building efficient and scalable data services and have the ability to integrate data systems with AWS tools and services to ...
... helping enterprises solve complex data, AI and cloud transformation challenges. Founded in 1991, Impetus has grown into a global engineering organization serving data-driven enterprises and Fortune 100/500 clients. Its capabilities span cloud engineering, data engineering, enterprise AI, application modernization, DevOps and ...
... storage. Build and maintain medical imaging data repositories, ensuring seamless access, query optimization, and compliance with healthcare regulations. Implement data processing workflows for clinical imaging data (CT, MRI) to extract, standardize, and structure metadata. Develop scalable solutions for medical imaging data ...
... and improve data quality. Monitor pipeline reliability, data availability and performance. What we're looking for - 5+ years of experience in Data Engineering, Data Operations or a related field. - Strong experience with GCP and Azure data platforms. - Hands-on experience in data pipelines, ETL, databases and data warehousing. ...
... exceptional customer https://jobeax.com/link/Pbn4lQC1L4k8wFDg team is passionate about innovation and collaboration, and we are looking for a talented Senior Data Scientist to join our team and work on impactful, high-scale https://jobeax.com/link/95FjFQQxFCFXlCQv You'll Do :Role Summary :As a Senior Data Scientist, you'll ...