PYTHON PYSPARK DEVELOPER in Hyderabad, India - Jobeax
Vacancy description
PYTHON PYSPARK DEVELOPER in Hyderabad, भारत
Vrinda International
Full-timeStandard weekly hours
Rs 20 - 25 lakhs p.a.
Hyderabad, भारत
Python & PySpark Developer
Summary
We are seeking an experienced Python and PySpark Developer to join our IT Services team. In this pivotal role, you will architect and implement robust data solutions, focusing on the design, development, and optimization of complex data pipelines. You will play a critical part in transforming raw data into actionable insights by leveraging cloud-native technologies and advanced processing frameworks. This position requires a deep understanding of data engineering principles to ensure high availability, scalability, and efficiency across our data infrastructure.
Responsibilities
Design, build, and maintain scalable ETL and ELT pipelines to facilitate seamless data movement and transformation.
Develop high-performance data processing applications using Python and PySpark to handle large-scale datasets.
Integrate and manage various AWS data services to support cloud-based data operations and storage requirements.
Create and optimize SQL queries for data extraction, manipulation, and reporting within complex database environments.
Implement and maintain REST APIs to enable efficient data integration between disparate systems and applications.
Utilize cloud technologies to deploy and manage data solutions, ensuring security and compliance with industry standards.
Collaborate with cross-functional teams to define data requirements and deliver data engineering solutions that meet business objectives.
Requirements
Requirements:
Experience: 6 to 9 years of professional experience in data engineering or related software development roles.
Core Programming: Proficiency in Python and Scala for developing data-intensive applications.
Big Data Technologies: Strong expertise in PySpark for distributed data processing and analytics.
Data Engineering: Demonstrated ability to design and implement ETL/ELT pipelines and data integration strategies.
Cloud & Infrastructure: Hands-on experience with AWS data services and general cloud technologies.
Database Skills: Advanced knowledge of SQL and relational database management systems.
API Development: Competence in building and consuming REST APIs for data connectivity.
Industry Context: Familiarity with the IT Services sector and its specific data handling challenges.
... Functions , Performance Optimization ,Spark Architecture,Salting ,Column Pruning Must have experience in leading a Development Team Proven experience as a Core Python developer Good understanding of Gen AI Experience developing Classes, OOPS, exception handling, parallel processing Strong knowledge of DB connectivity, data ...
Job Overview : We are seeking an experienced Python Developer to join our team. The ideal candidate will have a strong background in backend development, cloud technologies, and big data processing. This role involves leading technical projects, mentoring junior developers, and ensuring the delivery of high-quality https://jobeax.com/link/JpsOC0VrTgitAlOy ...
We are seeking an experienced AWS Glue Python Developer to design, develop, and support scalable cloud-based inbound and outbound file based data integration and processing solutions. The ideal candidate will have strong expertise in Python, AWS Glue, PySpark, and serverless AWS services. The candidate will work with business ...
About the Role :We are looking for an experienced Python Full-Stack Developer with strong expertise in Core Python, backend development, SQL, and modern web technologies. The role involves building scalable applications, developing APIs, integrating cloud services, and supporting data-driven https://jobeax.com/link/XnflZD2dGSZYgXFG ...
... Degree or equivalency (CS, CE, CIS, IS, MIS, or engineering discipline) or equivalent work experience.- Excellent coding skills in Python or Scala (preferably Python).- Must have implemented at least 4 projects end-to-end in Databricks.- Strong skills in Azure Data Factory, Azure Databricks, Python, and Pyspark.- Expert with ...
... Camel routes and integration services.- Build and support API integrations with internal and external systems.- Develop ETL and data ingestion pipelines using PySpark and Python.- Create scalable healthcare interoperability solutions utilizing HL7 FHIR standards.- Consume and publish REST APIs across enterprise platforms.- ...
... Power BI | DAX | SQL | Power Query | Data Modeling | ETL/ELT | Azure Data Factory | Microsoft Fabric | Data Warehousing | Lakehouse | SQL Server/Azure SQL | Python | Spark/PySpark | Delta Lake | Git | Power BI Service Duties - Develop and maintain advanced Power BI dashboards and reports using DAX, Power Query, data modeling, ...
Role : Data Engineer Location : ChennaiExperience : 5+ yearsInterview : FridayMust-Have Skills : - Python- PySpark- AWS- SQL- Azure Data Factory (ADF)- dbtKey Responsibilities : - Develop and maintain data pipelines and ETL/ELT processes.- Build data solutions using Python and PySpark.- Work with AWS cloud services for ...
... operating pipelines that other teams depend on daily. Analyst, BI-developer, or drag-and-drop ETL-tool-only experience does not count toward this. • Strong Python (3.x): pandas/pyarrow, packaging, virtual envs, writing testable modules and shared libraries — not just notebooks. • Apache Spark / PySpark at scale: DataFrame ...