Back to list

Vacancy description

Infosys - PySpark Developer - ETL Pipelines in Hyderabad, भारत

infosys ltd

Hyderabad, भारत

Key Responsibilities:- Design, develop, and maintain scalable ETL pipelines using PySpark.- Build robust data ingestion frameworks for processing data from multiple source systems.- Develop efficient batch and near real-time data processing workflows.- Perform data extraction, transformation, cleansing, validation, and loading activities.- Optimize PySpark jobs for performance, memory utilization, and execution time.- Develop reusable data transformation components and utility functions.- Write complex SQL queries for data analysis, validation, and reconciliation.- Process large datasets using distributed computing techniques.- Collaborate with business teams to understand data requirements and translate them into technical solutions.- Develop data quality checks and validation frameworks to ensure data accuracy and consistency.- Troubleshoot ETL failures, production issues, and data discrepancies.- Perform root cause analysis and implement permanent fixes for recurring issues.- Integrate data from relational databases, cloud storage, APIs, and flat files.- Support production deployments and monitor ETL job execution.- Participate in code reviews and ensure adherence to coding standards and best practices.- Create technical documentation, data mappings, and process flow diagrams.- Work closely with DevOps and infrastructure teams to improve deployment automation and operational efficiency.- Contribute to continuous improvement initiatives for data engineering https://jobeax.com/link/UuZ31CweuYCeQSOI Skills:- Experience with Azure Databricks or similar Spark-based data platforms.- Understanding of distributed computing and big data processing.- Familiarity with cloud platforms such as Azure, AWS, or GCP.- Experience with data pipeline orchestration tools.- Knowledge of dimensional data modeling and data warehouse architecture.- Experience working with large-scale enterprise data platforms.- Exposure to DevOps practices and automated deployment pipelines. (ref:hirist.tech)

Similar vacancies

More like this role

Edgeverve Systems Limited

Infosys - C++ Developer

We are looking for an experienced C++ Developer to design, develop, optimize, and maintain high-performance software https://jobeax.com/link/uTkIfl1tAHKRWQPa ideal candidate will have strong expertise in C++ programming, object-oriented design, data structures, algorithms, and system-level development, with the ability ...

Chennai, भारत

Capgemini

ETL Developer

  • Full-time
Senior data science role at Capgemini in Chennai. Needs 6+ years. What you'll do - Design, develop, and maintain ETL pipelines for data ingestion, transformation, and loading - Develop and optimize SQL queries for data processing and reporting requirements - Build and maintain data transformation jobs using Python and ETL ...

Chennai, भारत

PeoplePlus

ETL Developer

  • Full-time

Rs 4 - 7 lakhs p.a.

Junior data science role at PeoplePlus in Mumbai. Needs 1–3 years. Pays ₹4–7 LPA. What you'll do - Build and Operate Data Pipelines: Develop and maintain robust and scalable data pipelines to ensure the efficient and reliable flow of data within the organization What you need - ETL - SQL - Spark - Hadoop - Python - Data ...

Bombay, भारत

  • Full-time
  • Data Analytics
... Analyst or in a similar role, preferably in financial services, regulatory reporting, or liquidity risk. - Demonstrated expertise with Hadoop, Spark including PySpark and Scala Spark, Kafka, and cloud-based data platforms such as AWS S3 and Databricks. - Experience with data modeling tools and techniques. - Experience with ...

Pune, भारत

Match Me Resume Match | Citi

Big Data pyspark developer

  • Full-time
Mid-level data science role at Match Me Resume Match | Citi in Pune. Needs 3–5 years.

Pune, भारत

... Product Managers, Data Scientists, and AdOps teams in an Agile/Scrum environment to translate ambiguous business requirements into scalable, production-grade ETL/ELT data pipelines and dimensional data models.- Design and maintain end-to-end cloud Lakehouse architectures on Python using PySpark, Databricks, and Delta Lake, ...

Chennai, भारत

PradeepIT Consulting Services

Full Stack Angular Developer - Infosys

  • Full-time
  • Frontend
Senior frontend role at PradeepIT Consulting Services in Pune. Needs 4+ years. com/link/JSd7UYCSg572jJDY Job details - Location: Pune - Experience: 4–8 years - Education: Any graduate - Employment type: Full-time - Languages: English [{"name": "full_time", "text": "Full time", "categoryName": "Type of employment"}]

Pune, भारत

Squircle IT Consulting Services

MDM and ETL Developer

  • Full-time
... strong understanding of business processes for functions such as Sales, Finance, and Marketing in order to be able to better support the MDM needs What you need - ETL - SQL - Informatica - Data warehousing - Data modelling Job details - Location: Bengaluru - Experience: 5–7 years - Education: Any graduate - Employment type: ...

Bangalore, भारत

  • IT
  • Fullstack
About the Role : We are seeking a Senior Python Full Stack Developer to lead the design and development of complex web applications and scalable software https://jobeax.com/link/jUnnPoqbQaS40ptC Experience : 10 - 14 yearsKey Responsibilities : - Lead the architecture, design, development, and maintenance of full stack applications.- ...

India, भारत

  • IT
  • Fullstack
Role Overview :We are looking for a Principal Python Full Stack Developer to define the technical architecture and engineering strategy for enterprise-scale web applications and digital https://jobeax.com/link/UuxHBYkbQjke5gQz role involves guiding backend and frontend architecture, establishing engineering standards, driving ...

India, भारत

Job Description:We are looking for an experienced ETL Developer with strong hands-on experience in designing, developing, and maintaining data integration and ETL/ELT https://jobeax.com/link/7yDwjVoYn8HZ8pTw Responsibilities:- Develop and maintain ETL/ELT workflows and data pipelines.- Perform data extraction, transformation, ...

Bangalore, भारत