We are seeking a skilled Data Engineer to design, build, and maintain robust data infrastructure within the IT Services sector. This pivotal role focuses on architecting scalable data solutions using Microsoft Azure technologies to ensure seamless data flow, integrity, and accessibility. The successful candidate will play a critical part in transforming raw data into actionable insights, supporting strategic business decisions through advanced data warehousing and analytics capabilities.
Responsibilities
Design and implement scalable data storage and processing architectures utilizing Azure Data Lake Storage (ADLS) and Delta Lake.
Develop and optimize complex ETL pipelines using Azure Data Factory (ADF) and PySpark to ensure efficient data movement and transformation.
Build and manage data models and warehouses, leveraging Azure Synapse Analytics for high-performance querying and reporting.
Construct and maintain data pipelines that integrate diverse data sources, ensuring accuracy and consistency across the ecosystem.
Write and refine Python and SQL scripts to automate data processes, perform data quality checks, and support analytical workflows.
Collaborate with cross-functional teams to define data requirements and deliver solutions that align with organizational goals.
Requirements
Requirements:
Possess 4 to 9 years of professional experience in data engineering or a related technical field.
Demonstrate deep proficiency with Azure cloud services, specifically Azure Data Lake Storage (ADLS) and Azure Synapse Analytics.
Show strong expertise in data modeling, data warehousing concepts, and the implementation of Delta Lake architecture.
Be highly skilled in developing ETL pipelines using Azure Data Factory (ADF) and executing big data processing with PySpark.
Have a solid command of Python and SQL for data manipulation, scripting, and query optimization.
Experience working within Databricks environments for collaborative data analysis and engineering is essential.
Target compensation range is approximately ₹24 LPA, commensurate with experience and skill level.
... BI & DAX- Azure Databricks- Data IntegrationGood to Have :- Microsoft Fabric / Azure Data Engineer / Power BI certifications- Azure DevOps & Git- Snowflake- Databricks- Knowledge of Data Governance and Cloud Data PlatformsCandidate Profile :- 3 - 5 years of relevant experience in Data Engineering, BI, Analytics Engineering ...
... and SSRS, Streaming technologies (Fabric Real-Time Intelligence, Event Hubs, Kafka), Power BI and DAX.Certifications:- Preferred: Microsoft Certified: Fabric Data Engineer Associate (DP-700), Fabric Analytics Engineer Associate (DP-600), Azure Data Engineer Associate (DP-203), Databricks Certified Data Engineer Professional, ...
... sets that meet functional / non-functional business requirements - Build the infrastructure required for optimal extrssaction, transformation What you need - Azure - SQL - Python - ETL - Databricks - Azure Data Factory Good to have - Spark - NoSQL - Root cause analysis - Data structures Job details - Location: Remote, anywhere ...
... Bachelor's or master's degree in computer science, data engineering, or a related field, along with 4+ year work experience in data engineering and cloud platforms. - Azure and Databricks: Strong proficiency in Azure services such as Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Data Lake Storage, and Azure Synapse ...
... Workflows or Azure Data Factory . - Monitor and troubleshoot failures in job execution and cluster performance . - Collaborate with Stakeholders - Work with Data Analysts, Scientists, and Business Teams to understand requirements. - Translate business needs into scalable data engineering solutions . - API expertise - Ability ...
... Bachelor's or Master's degree, ideally in Computer Science, Information Systems or a related field.- Proven work experience as an Analytics Engineer or Data Engineer with either Azure Synapse or Databricks (Spark/PySpark, Delta Lake, Unity Catalog etc).- Strong SQL and Python (or equivalent language) skills for data manipulation ...
... a Data Architect or Senior Data Engineer/Lead designing and implementing enterprise data warehouses and modern data platforms.- 8 - 12 years of experience in Data Engineering/Architecture, with at least 5+ years in Azure & Databricks ecosystem.- Hands-on expertise with Azure services: Azure Synapse Analytics (Dedicated ...
... of the data engineering team and working on in-house products in for Data Foundation Collaborating closely with Data Engineering Leads and managers and other DataOps teams. This role involves hands-on development work and will directly report to Data Engineering Lead. Must have skills: - Azure Databricks, Unity Catalog, ...
... models, SQL queries, XML data models. Extensive experience with software test automation tools and working in fast-paced Agile environments. Experience with Azure Data Factory for pipeline execution and analysis. Experience with Azure Databricks to access data from Azure Data Lakes. Experience with Azure Notebooks Experience ...