We are seeking an experienced Data Engineer to architect, build, and maintain scalable data infrastructure within the IT Services sector. In this pivotal role, you will drive the design of robust data warehousing solutions and ensure seamless data modeling to support critical business analytics. Your expertise will be essential in managing cloud-native data pipelines, overseeing high-quality data migration strategies, and optimizing large-scale processing workflows. This position offers the opportunity to leverage cutting-edge Google Cloud technologies to transform raw data into actionable insights, directly contributing to the organization's data-driven decision-making capabilities and operational efficiency.
Responsibilities
Design and implement comprehensive data warehousing architectures and advanced data modeling strategies to meet evolving business needs.
Develop and maintain efficient version control workflows using Git and GitHub, integrating continuous integration and continuous deployment (CI/CD) pipelines for automated testing and deployment.
Construct and optimize complex data processing pipelines utilizing Google Dataflow and Apache Beam for real-time and batch data transformation.
Establish rigorous data quality frameworks and lead strategic cloud migration initiatives to ensure data integrity and security during transitions.
Manage and scale data storage and processing environments using BigQuery and Dataproc, ensuring high performance and cost-effectiveness.
Orchestrate scheduled workflows and dependency management through Cloud Composer (Apache Airflow) to guarantee reliable data delivery.
Implement event-driven data ingestion and storage solutions leveraging Pub/Sub and Cloud Storage to handle high-volume data streams.
Requirements
Requirements:
Experience: A minimum of 15 to 20 years of professional experience in data engineering and related fields.
Core Competencies: Proven expertise in Data Warehousing and sophisticated Data Modeling techniques.
Version Control & DevOps: Strong proficiency with Git/GitHub and the implementation of CI/CD practices.
Big Data Processing: Hands-on experience with Google Dataflow and Apache Beam for scalable data processing.
Data Governance: Demonstrated ability in ensuring Data Quality standards and executing successful Cloud Migration projects.
Cloud Technologies: In-depth knowledge of Google BigQuery and Dataproc for analytics and processing.
Workflow Orchestration: Skilled in utilizing Cloud Composer (Apache Airflow) for workflow management.
Messaging & Storage: Familiarity with Pub/Sub and Cloud Storage for data ingestion and archival.
... scalable ETL/ELT pipelines for processing structured and unstructured data - Build data ingestion and transformation workflows using Python, SQL, and relevant data engineering frameworks - Extract, clean, validate, normalize, and transform data from APIs, databases, files, and other sources What you need - Python - Spark ...
... & Data Platform. You’ll design and maintain reliable, secure, and scalable data pipelines and models that power a new generation of reporting, insights, and data-driven products across the StarRez ecosystem. This role sits at the intersection of data, product, and engineering. You will work closely with Data Analysts, ...
Data science role at Priceline Technology India in Mumbai. Hybrid. What you'll do - Participate in cloud migration and modernizing current data systems - Oversee data systems that serve millions of events a day - Maintain and design early warning and analytics systems using cutting-edge big data technologies Job details ...
... architecture and pipelines that adhere to ETL principles and business goals - Define data requirements, gather and mine large-scale structured and unstructured data, and validate data using various data tools in the Big Data Environment - Support Data Scientists in data sourcing and preparation to visualize data and synthesize ...
... We’re confident enough to move fast and talented enough not to break things. Are you a Data Engineer looking to make a real impact? We are looking for a founding Data Engineer to architect the modern data stack that powers our Customer Success organization. In this role, you will move beyond simple data maintenance to design ...
Summary We are seeking an experienced Data Engineer to join our IT Services division. In this pivotal role, you will design, build, and maintain robust data pipelines that facilitate seamless information flow across our Salesforce ecosystem and data warehouse. You will be instrumental in ensuring data integrity, optimizing ...
Role Overview We are looking for a skilled Data Engineer with strong analytical, data interpretation, and problem-solving abilities. The ideal candidate will have hands-on experience with SQL, Python, Excel, and data visualization tools to analyze large datasets and generate meaningful business insights. Key Responsibilities ...
... About the Role As a Data Engineer, you will be responsible for designing, building, and maintaining robust and scalable data pipelines. You will work with large datasets, ensuring data quality, and enabling data-driven decision-making across the organization. This role requires a strong understanding of data warehousing concepts, ...
... channels. This initiative requires a robust on-premises data infrastructure to support distributed computing, large-scale data ingestion, and advanced analytics. The Data Engineer will be responsible for building and maintaining resilient pipelines and data systems that feed into MMO models, ensuring data quality, governance, and ...
About the Job Position : Role : Data Engineer - Databricks & PySparkExperience : 3 - 6 yearsLocation : Gurgaon, IndiaWork Mode : Work from Office (Mon - Fri)Shift : 2 : 00 PM - 11 : 00 PM ISTMantriAI is hiring for a leading global analytics and advisory firm, helping banks and financial institutions drive revenue growth, ...
... choice. Data Engineer POSITION PURPOSE: As a Data Engineer, you are developing and operating reliable data pipelines that enable timely access to high-quality data assets. You are part of a dynamic team of technical experts, working hand-in-hand with subject matter experts, data product managers, semantic experts and data ...