Back to list

Vacancy description

RandomTrees - Senior Data Engineer - PySpark/Azure Databricks in Hyderabad, भारत

RANDOMTREES PRIVATE LIMITED

Hyderabad, भारत

About the Job : We are looking for a skilled Senior Data Engineer with 7+ years of experience in designing, building, and maintaining scalable data https://jobeax.com/link/hEiNRx4W5ZDbjxHN ideal candidate will have strong expertise in PySpark, SQL, Databricks, Azure, and Azure Data Factory (ADF).You will be responsible for developing efficient data solutions, optimizing workflows, and ensuring data reliability to support business https://jobeax.com/link/nEC7Pw3JSppzjDvo Responsibilities : - Design, develop, and deploy high-performance data pipelines using PySpark, Databricks, and Azure Data Factory (ADF).- Optimize and maintain SQL queries, stored procedures, and ETL processes for large-scale data processing.- Work with Azure cloud services (Azure Data Lake, Azure Synapse, Blob Storage, etc.) to build scalable data solutions.- Implement data integration, transformation, and warehousing solutions in Databricks.- Collaborate with data scientists, analysts, and business teams to understand data requirements and deliver actionable insights.- Ensure data quality, governance, and security best practices across all pipelines.- Troubleshoot and resolve data-related issues in production environments.- Automate workflows and improve efficiency using CI/CD pipelines.- Mentor junior engineers and provide technical https://jobeax.com/link/1N9F4ujaOnSRMas2 Skills & Qualifications : - 7+ years of hands-on experience in Data Engineering.- Strong expertise in PySpark for large-scale data processing.- Proficiency in SQL (query optimization, stored procedures, performance tuning).- Hands-on experience with Azure cloud services (ADF, Azure Data Lake, Synapse, Blob Storage, etc.)- Experience with Databricks (Delta Lake, Spark optimization, notebook workflows).- Knowledge of data modeling, ETL/ELT processes, and data warehousing.- Familiarity with Python, Shell scripting, and automation.- Understanding of CI/CD pipelines and DevOps practices (Git, Azure DevOps).- Strong problem-solving skills and ability to work in a fast-paced environment. (ref:hirist.tech)

Similar vacancies

More like this role

RANDOMTREES PRIVATE LIMITED

RandomTrees - AWS DevOps Engineer

  • Remote
  • IT
  • DevOps
Role Overview : We are looking for an experienced AWS DevOps Engineer with strong hands-on expertise in Terraform, AWS infrastructure provisioning, GitLab CI/CD, and DevSecOps practices. The ideal candidate must have practical experience designing and provisioning AWS infrastructure using Terraform, developing reusable ...

Hyderabad, भारत

... Query, etc.- Proficient in at least one programming language used in data engineering, such as Python (or Scala/Rust/Java).- Has strong execution knowledge of Data Modeling (star schema, snowflake schema, fact vs dimension tables).- Proficient in Spark and related applications like Databricks, GCP DataProc, AWS Glue, EMR, ...

Hyderabad, भारत

  • Full-time
Mid-level data science role at KreditBee in Bengaluru. Needs 2–4 years and a https://jobeax.com/link/2DENLGgQM7whDhaE or B.E.. What you'll do - ⁠ ⁠Develop various RestFul APIs & Integrate with various data sources - ⁠ ⁠Develop tools & techniques related to ETL - ⁠ ⁠Exposure in AWS or Other Cloud Platform is advantageous ...

Bangalore, भारत

  • On-site
  • Full-time
... your world, creating more fulfilling work and projects for our people and clients. For more information, visit [ Job Description We’re looking for a skilled Senior Data Engineer with strong experience in distributed systems, Python/Scala, and modern data engineering tools to help design and implement an end-to-end data ...

Hyderabad, भारत

  • Full-time
  • Contract
... YOU ARE: - At least five years of experience in data engineering, DataOps, DevOps, or a comparable technical role. - Advanced hands-on experience with Python, PySpark, SQL, Git, and Databricks. - Strong experience with the Databricks ecosystem, including Delta Lake, Unity Catalog, and Databricks Serverless SQL. - Proven experience ...

Bangalore, भारत

  • Full-time
... migrating data pipelines from on-prem or other cloud platforms Understanding of data governance, security, and access control on AWS Experience with CI/CD for data pipelines Nice to have AWS certification (Data Analytics Specialty or Data Engineer Associate) Experience with streaming data (Kinesis, MSK/Kafka) Familiarity ...

Bangalore, भारत

Merrill Communications

Senior Data Engineer

  • Full-time
... expertise - Ability to pull data from a wide variety of APIs using different strategies and methods Required Skills & Experience: - Azure Databricks & Apache Spark (PySpark) – Strong experience in building distributed data pipelines . - Python – Proficiency in writing optimized and maintainable Python code for data engineering. ...

Chennai, भारत

  • Remote
  • Hybrid
  • Full-time
  • Contract
... store raw, curated, and enriched data (often structured into a Bronze, - Silver, and Gold medallion architecture). - Data Processing & Transformation: Uses Azure Databricks (powered by Apache Spark) for heavy big-data engineering, machine learning, and stream-batch processing. - Data Warehousing & Analytics: Uses Azure Synapse ...

Gurgaon, भारत

  • Full-time
  • IT
... networks combine to deliver a unique set of products and services that help people, businesses and governments realize their greatest potential. Title and Summary Senior Data Engineer Overview We are the global technology company behind the world’s fastest payments processing network. We are a vehicle for commerce, a connection ...

Pune, भारत

  • Full-time
... experience with Databricks, including Unity Catalog, Databricks SQL, Workflows, and Delta Sharing. - Proficiency in Python and Apache Spark. - Develop and manage Databricks notebooks for data transformation, exploration, and model deployment. - Expertise in Microsoft Azure services, including Azure SQL, Azure Data Factory (ADF), ...

Ernakulam, भारत

  • On-site
  • Contract
... is a sophisticated technology ecosystem that processes billions of payment events with speed, resilience, and precision. As a Senior Data Engineer within the Data Collection & Engineering (DC&E) organisation, you will play a critical role in designing, building, and scaling Mastercard's next-generation data platforms and ...

Hyderabad, भारत