Senior Data Engineer - Azure Databricks in Hyderabad, भारत
US TECHNOLOGY INTERNATIONAL PRIVATE LIMITED
Hyderabad, भारत
Job Overview : We are seeking a highly experienced Senior Data Engineer to design, develop, and optimize scalable data pipelines in a cloud-based https://jobeax.com/link/2rce0yCTCWx0P6oA ideal candidate will have deep expertise in PySpark, SQL, Azure Databricks, and experience with either AWS or GCP.A strong foundation in data warehousing, ELT/ETL processes, and dimensional modeling (Kimball/star schema) is essential for this https://jobeax.com/link/ZNQqU5E5annp3g9v Requirements : - Strong proficiency in PySpark and SQL for data transformation and pipeline development.- Experience working in Azure Databricks or equivalent Spark-based cloud platforms.- Practical knowledge of cloud data environments - Azure, AWS, or GCP.- Solid understanding of data warehousing concepts, including Kimball methodology and star/snowflake schema design.- Proven experience designing and maintaining ETL/ELT pipelines in production.- Familiarity with version control (e.g., Git), CI/CD practices, and data pipeline orchestration tools (e.g., Airflow, Azure Data Factory).- Experience in the financial domain or understanding of investment banking https://jobeax.com/link/C0Mci7VFqmf0Rl7S Experience : 7 to 9 years. (ref:hirist.tech)
... improve data processing efficiency.- Implement real-time and streaming data solutions using technologies such as Spark Streaming and Kafka.- Work with cloud-native data services and integrate Databricks with Azure or AWS environments.- Collaborate with architects, data engineers, and cross-functional teams to deliver robust data ...
Azure Data EngineerKey Responsibilities:- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF).- Build and optimize data processing workflows using Azure Databricks and Python.- Develop ETL/ELT processes to extract, transform, and load data from multiple sources.- Work with structured and ...
... optimisation and marketing mix modelling. - Monitor pipeline health, data quality and cluster cost, and tune them as needed. Must-Have Skills - 2–4 years of hands-on data engineering experience, including at least 1.5 years on Databricks. - Strong skills in PySpark, SQL and Python. - Experience with Delta Lake, including MERGE/upserts, ...
... and query optimization.- Hands-on experience with Azure Databricks for data engineering and transformation workloads.- Experience building and orchestrating data pipelines using Azure Data Factory (ADF).- Strong understanding of data engineering principles, ETL/ELT processes, data transformation, and pipeline orchestration.- ...
... independence and autonomy while contributing to the technical maturity of the organization. Qualifications Qualifications & Skills - Experience: 5+ years of Data Engineering experience, including 4+ years designing and building Databricks data pipelines is essential. While Azure cloud experience is preferred, we are happy ...
... Governance: Experience implementing data quality frameworks, data observability, and Unity Catalog-based https://jobeax.com/link/jNu8rE2vHzGnMJ64 Qualifications:- Databricks Certified Data Engineer Professional or Associate; AWS Certified Data Engineer or Azure Data Engineer Associate.- Experience delivering data platforms across ...
... a highly skilled and experienced Senior Data Engineer (AWS/Databricks) with 5+ years of hands-on experience in designing, developing, and optimizing scalable data engineering solutions. The ideal candidate should have strong expertise in Python, SQL, PySpark, AWS, Databricks, ETL/ELT pipelines, and cloud-based data platforms ...
... seeking a skilled Senior Data Engineer with expertise in Databricks to join our dynamic data team. The ideal candidate will design build and maintain scalable data pipelines and architectures to support our organizations data-driven initiatives and leverage Databricks to process large-scale datasets optimize data workflows ...
Databricks Forward Deployed Engineer :Role Overview :The Databricks Forward Deployed Engineer (FDE) is a senior client-facing engineer embedded with enterprise clients to design, build, and optimise production-grade data platforms on Databricks. The role requires deep Spark and Databricks technical expertise combined with ...
... Experience & Qualifications • 5–7 years of overall experience in Data Engineering / ETL development. • Minimum 3+ years of hands-on experience in Microsoft Azure Data Engineering. • Strong hands-on experience in: o Azure Data Factory (ADF) o Azure Databricks o Azure Data Lake Storage (ADLS) o Azure SQL Database o Azure Synapse ...