Back to list

Vacancy description

InfoBeans Technologies - Data Engineer - Azure Databricks in Indhur, भारत

InfoBeans

Indhur, भारत

About the Role :We are looking for an experienced Data Engineer with strong hands-on expertise in Python, PySpark, SQL, and cloud-based data engineering. The role involves designing and developing scalable data processing solutions, building reusable frameworks, optimizing data transformation pipelines, and delivering reliable data solutions using Azure Databricks and Azure Data Factory (ADF).Key Responsibilities :- Design, develop, and maintain scalable data processing solutions using Python and PySpark.- Write complex SQL queries for data extraction, transformation, validation, reconciliation, and analysis.- Build and maintain robust data pipelines and workflows using Azure Databricks and Azure Data Factory (ADF).- Develop reusable, modular, and scalable code frameworks to support efficient data engineering.- Design and implement data transformations and processing logic for large datasets.- Develop solutions from scratch, following clean coding practices, reusable design patterns, and comprehensive documentation standards.- Optimize existing code, Spark jobs, SQL queries, and data pipelines to improve performance, scalability, and resource utilization.- Troubleshoot data pipeline failures, processing bottlenecks, and data quality issues.- Collaborate with data engineers, analysts, architects, and business stakeholders.- Ensure data accuracy, consistency, integrity, and reliability throughout the data processing lifecycle.- Participate in technical design discussions, code reviews, and continuous improvement https://jobeax.com/link/GTNZKF2kxpVUFhTn Skills & Experience : - 10 - 13 years of experience in data engineering, data development, or related software engineering roles.- Strong hands-on proficiency in Python, including core programming concepts, data structures, modular design, and reusable code development.- Strong experience with PySpark for distributed data processing and large-scale data transformations.- Advanced SQL skills, including complex queries, joins, aggregations, analytical functions, and query optimization.- Hands-on experience with Azure Databricks for data engineering and transformation workloads.- Experience building and orchestrating data pipelines using Azure Data Factory (ADF).- Strong understanding of data engineering principles, ETL/ELT processes, data transformation, and pipeline orchestration.- Experience designing scalable and reusable data processing frameworks.- Experience working with Git, code reviews, testing, and deployment practices.- Excellent analytical, communication, and cross-functional collaboration skills. (ref:hirist.tech)

Similar vacancies

More like this role

... including temporal data history and party/role modelling patterns.- Reverse - engineer legacy SQL Server environments and develop strategies to decouple monolithic databases into services using strangler - style architecture patterns.- Architect and lead migrations from SQL Server virtual machine environments to Azure managed ...

Pune, भारत

  • On-site
  • IT
  • DevOps
Role Overview:We are looking for an experienced Senior DevOps Engineer with strong expertise in Jenkins, CI/CD pipeline management, Linux system administration, and on-premises infrastructure. The role involves managing and optimizing build and deployment environments, automating operational processes, troubleshooting complex ...

Pune, भारत

... Understanding, Microsoft AI Builder, or similar IDP platforms.- Generative AI and Prompt https://jobeax.com/link/sqogeXiFsYETeD28 Competencies :Mandatory :- Prompt Engineering- Generative AI & Large Language Models (LLMs)- Intelligent Document Processing (IDP)- OCR Technologies- Data Validation & Quality Assurance, Process Analysis ...

Chennai, भारत

  • Fullstack
... developmentKey Technologies : Python, FastAPI, https://jobeax.com/link/Z30sZYC3jcRenVso, React, TypeScript, SQLJob Summary : We are looking for a hands-on Full Stack Engineer with strong experience in Python backend development and modern frontend technologies, particularly FastAPI, https://jobeax.com/link/Z30sZYC3jcRenVso, React, ...

Hyderabad, भारत

... Designer.- Develop and support API-based integrations using REST and SOAP APIs.- Implement integrations across various enterprise systems using JSON, XML, SFTP, and database connectivity.- Design and implement integration workflows involving relational databases and SQL-based operations.- Configure and manage API authentication ...

Indhur, भारत

... Experience with Python-based API frameworks (FastAPI, Django REST Framework).- Understanding of CI/CD pipelines (Jenkins, GitHub Actions, GitLab CI).- Knowledge of databases (SQL/NoSQL) and testing frameworks (Jest, Enzyme, PyTest).- Experience with Cloud platforms (AWS/Azure) and containerization (Docker/Kubernetes). (ref:hirist.tech) ...

Pune, भारत

Responsibilities : - Develop highly scalable front-end applications using React JS, AG Grid, TypeScript, HTML5 and CSS3.- Deliver to production on Azure using CI/CD (GitHub Actions) and integrated Azure services.- Collaborate and join with backend engineers to enhance Python/Django APIs and deliver end-to-end features.- ...

Pune, भारत

  • On-site
  • IT
  • DevOps
... 10+ years of hands-on experience in IBM FileNet Content Manager (P8) and Red Hat OpenShift container platform.- Ingestion & App Servers : Experience with IBM Datacap and Oracle WebLogic Server.- DevOps & Source Control : Proficiency with Git/GitHub, GitHub Actions or Jenkins, and JFrog Artifactory.- Databases & Messaging ...

Chennai, भारत

... extraction, transformation, loading, data reconciliation, and source-to-target validation.- Familiarity with data warehousing concepts, dimensional modelling, and data processing workflows.- Experience working with one or more database or data platforms such as Oracle, Denodo, Azure Databricks, or SQL Server.- Experience designing ...

Pune, भारत

  • Backend
... https://jobeax.com/link/JpsOC0VrTgitAlOy :- Lead the development of backend micro services using Django.- Design and implement scalable and secure APIs.- Integrate Azure Cloud services for application deployment and management.- Utilize Azure Databricks for big data processing and analytics.- Implement data processing pipelines ...

Indhur, भारत