Senior Data Engineer - Azure Databricks in Hyderabad, भारत
US TECHNOLOGY INTERNATIONAL PRIVATE LIMITED
Hyderabad, भारत
Job Overview : We are seeking a highly experienced Senior Data Engineer to design, develop, and optimize scalable data pipelines in a cloud-based https://jobeax.com/link/2rce0yCTCWx0P6oA ideal candidate will have deep expertise in PySpark, SQL, Azure Databricks, and experience with either AWS or GCP.A strong foundation in data warehousing, ELT/ETL processes, and dimensional modeling (Kimball/star schema) is essential for this https://jobeax.com/link/ZNQqU5E5annp3g9v Requirements : - Strong proficiency in PySpark and SQL for data transformation and pipeline development.- Experience working in Azure Databricks or equivalent Spark-based cloud platforms.- Practical knowledge of cloud data environments - Azure, AWS, or GCP.- Solid understanding of data warehousing concepts, including Kimball methodology and star/snowflake schema design.- Proven experience designing and maintaining ETL/ELT pipelines in production.- Familiarity with version control (e.g., Git), CI/CD practices, and data pipeline orchestration tools (e.g., Airflow, Azure Data Factory).- Experience in the financial domain or understanding of investment banking https://jobeax.com/link/C0Mci7VFqmf0Rl7S Experience : 7 to 9 years. (ref:hirist.tech)
... requirements into scalable, production-grade data pipelines and Lakehouse https://jobeax.com/link/JJKfuHPEsmrFPG6l Responsibilities : - Data Pipeline & Lakehouse Engineering : Design, implement, and maintain high-performance batch and real-time streaming data pipelines using Python, PySpark, and Spark SQL on Azure Databricks ...
About the Job : We are looking for a skilled Senior Data Engineer with 7+ years of experience in designing, building, and maintaining scalable data https://jobeax.com/link/hEiNRx4W5ZDbjxHN ideal candidate will have strong expertise in PySpark, SQL, Databricks, Azure, and Azure Data Factory (ADF).You will be responsible ...
... scalable ETL/ELT pipelines to integrate disparate data sources, ensuring high data quality and availability for downstream analytics.- Develop and optimize complex data models within Snowflake and Databricks environments to support high-performance reporting and business intelligence requirements.- Collaborate with engineering ...
Azure Data EngineerKey Responsibilities:- Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF).- Build and optimize data processing workflows using Azure Databricks and Python.- Develop ETL/ELT processes to extract, transform, and load data from multiple sources.- Work with structured and ...
... Delta Lake concepts (Time travel, ACID transactions).- Familiarity with CI/CD concepts and deploying code using Azure DevOps or GitHub Actions.- Prior experience working in an Agile/Scrum environment.- Certifications : Azure Data Fundamentals (DP 900) or Databricks Certified Data Engineer Associate. (ref:hirist.tech) []
... Skills and Experience : - 6 - 12 years of relevant professional experience, including strong hands-on experience in Azure Data Engineering.- Strong proficiency in Azure Databricks, Python, PySpark and SQL.- Experience with Azure Data Lake Storage (ADLS).- Hands-on experience with Azure Data Factory (ADF) or Databricks Workflows.- ...
... optimisation and marketing mix modelling. - Monitor pipeline health, data quality and cluster cost, and tune them as needed. Must-Have Skills - 2–4 years of hands-on data engineering experience, including at least 1.5 years on Databricks. - Strong skills in PySpark, SQL and Python. - Experience with Delta Lake, including MERGE/upserts, ...
... and query optimization.- Hands-on experience with Azure Databricks for data engineering and transformation workloads.- Experience building and orchestrating data pipelines using Azure Data Factory (ADF).- Strong understanding of data engineering principles, ETL/ELT processes, data transformation, and pipeline orchestration.- ...
... and query optimization.- Hands-on experience with Azure Databricks for data engineering and transformation workloads.- Experience building and orchestrating data pipelines using Azure Data Factory (ADF).- Strong understanding of data engineering principles, ETL/ELT processes, data transformation, and pipeline orchestration.- ...
... independence and autonomy while contributing to the technical maturity of the organization. Qualifications Qualifications & Skills - Experience: 5+ years of Data Engineering experience, including 4+ years designing and building Databricks data pipelines is essential. While Azure cloud experience is preferred, we are happy ...