Senior Data Engineer - Azure Databricks in Hyderabad, भारत
US TECHNOLOGY INTERNATIONAL PRIVATE LIMITED
Hyderabad, भारत
Job Overview : We are seeking a highly experienced Senior Data Engineer to design, develop, and optimize scalable data pipelines in a cloud-based https://jobeax.com/link/2rce0yCTCWx0P6oA ideal candidate will have deep expertise in PySpark, SQL, Azure Databricks, and experience with either AWS or GCP.A strong foundation in data warehousing, ELT/ETL processes, and dimensional modeling (Kimball/star schema) is essential for this https://jobeax.com/link/ZNQqU5E5annp3g9v Requirements : - Strong proficiency in PySpark and SQL for data transformation and pipeline development.- Experience working in Azure Databricks or equivalent Spark-based cloud platforms.- Practical knowledge of cloud data environments - Azure, AWS, or GCP.- Solid understanding of data warehousing concepts, including Kimball methodology and star/snowflake schema design.- Proven experience designing and maintaining ETL/ELT pipelines in production.- Familiarity with version control (e.g., Git), CI/CD practices, and data pipeline orchestration tools (e.g., Airflow, Azure Data Factory).- Experience in the financial domain or understanding of investment banking https://jobeax.com/link/C0Mci7VFqmf0Rl7S Experience : 7 to 9 years. (ref:hirist.tech)
... requirements into scalable, production-grade data pipelines and Lakehouse https://jobeax.com/link/JJKfuHPEsmrFPG6l Responsibilities : - Data Pipeline & Lakehouse Engineering : Design, implement, and maintain high-performance batch and real-time streaming data pipelines using Python, PySpark, and Spark SQL on Azure Databricks ...
About the Job : We are looking for a skilled Senior Data Engineer with 7+ years of experience in designing, building, and maintaining scalable data https://jobeax.com/link/hEiNRx4W5ZDbjxHN ideal candidate will have strong expertise in PySpark, SQL, Databricks, Azure, and Azure Data Factory (ADF).You will be responsible ...
... scalable ETL/ELT pipelines to integrate disparate data sources, ensuring high data quality and availability for downstream analytics.- Develop and optimize complex data models within Snowflake and Databricks environments to support high-performance reporting and business intelligence requirements.- Collaborate with engineering ...
Azure Data EngineerKey Responsibilities:- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF).- Build and optimize data processing workflows using Azure Databricks and Python.- Develop ETL/ELT processes to extract, transform, and load data from multiple sources.- Work with structured and ...
... Delta Lake concepts (Time travel, ACID transactions).- Familiarity with CI/CD concepts and deploying code using Azure DevOps or GitHub Actions.- Prior experience working in an Agile/Scrum environment.- Certifications : Azure Data Fundamentals (DP 900) or Databricks Certified Data Engineer Associate. (ref:hirist.tech) []
... Skills and Experience : - 6 - 12 years of relevant professional experience, including strong hands-on experience in Azure Data Engineering.- Strong proficiency in Azure Databricks, Python, PySpark and SQL.- Experience with Azure Data Lake Storage (ADLS).- Hands-on experience with Azure Data Factory (ADF) or Databricks Workflows.- ...
... optimisation and marketing mix modelling. - Monitor pipeline health, data quality and cluster cost, and tune them as needed. Must-Have Skills - 2–4 years of hands-on data engineering experience, including at least 1.5 years on Databricks. - Strong skills in PySpark, SQL and Python. - Experience with Delta Lake, including MERGE/upserts, ...
... and query optimization.- Hands-on experience with Azure Databricks for data engineering and transformation workloads.- Experience building and orchestrating data pipelines using Azure Data Factory (ADF).- Strong understanding of data engineering principles, ETL/ELT processes, data transformation, and pipeline orchestration.- ...
... and query optimization.- Hands-on experience with Azure Databricks for data engineering and transformation workloads.- Experience building and orchestrating data pipelines using Azure Data Factory (ADF).- Strong understanding of data engineering principles, ETL/ELT processes, data transformation, and pipeline orchestration.- ...
... in Databricks, PySpark, Delta Lake, Unity Catalog, Azure Databricks, SQL, and dimensional data modeling. The candidate will act as the technical owner of the data platform, providing architectural direction, establishing engineering standards, mentoring teams, and ensuring scalable and high-performance data https://jobeax.com/link/XnflZD2dGSZYgXFG ...