Back to list

Vacancy description

PySpark/Big Data Developer in Bangalore, भारत

L A Consultancy

Bangalore, भारत

BE | BTech | MCA | OR equivalent degree in Computer Science, IT or a related field candidates having 5 to 10 years of experience in Python, PySpark, SQL Unix + Big Data (Hadoop + Hive), Airflow, Datawarehousing, Data https://jobeax.com/link/jJwPeXOR5xzgNWwu skills :- Strong hands-on experience in Python Development.- Extensive experience with PySpark / Apache Spark.- Expertise in Data Engineering, SQL Unix + Big Data (Hadoop + Hive), Airflow, Datawarehousing, Data modelling and data processing frameworks.- Strong knowledge of SQL and database concepts.- Experience working with large-scale datasets in distributed environments.- Understanding of Data Warehousing concepts and dimensional data modeling.- Exposure to cloud platforms such as Azure, AWS, or GCP.- Experience with version control tools such as Git.- Experience with Kubernetes, Java or python + Spark SQL, Delta Lake.- Knowledge of Airflow, ADF, or other orchestration tools.- Experience in CI/CD implementation.- Familiarity with SAP BW/4HANA, SAP S/4HANA, or SAP Banking https://jobeax.com/link/RTFVePHul1yv5RJz Skills :- Exposure to SAP FSDM/FPSL.- Exposure to SAP Financial Services Data Management (FSDM).- Understanding of SAP FSPL (Financial Services Package for Liquidity) + Big Data (Hadoop + Hive) processes and data structures.- Knowledge of financial services, banking, risk, liquidity, or regulatory reporting domains is https://jobeax.com/link/mxQSVu7ep23Mr2Gv & Responsibilities :- Partner closely with Product Managers, Data Scientists, and AdOps teams in an Agile/Scrum environment to translate ambiguous business requirements into scalable, production-grade ETL/ELT data pipelines and dimensional data models.- Design and maintain end-to-end cloud Lakehouse architectures on Python using PySpark, Databricks, and Delta Lake, establishing standardized frameworks for automated data ingestion, schema enforcement, and data reconciliation.- Manage production workflow orchestration using Apache Airflow, building automated failure recovery, log monitoring, and SLA-driven alerting to guarantee 99.9% data pipeline uptime and reliable downstream analytics delivery.- Mentored junior data engineers through pair programming and code reviews; established team-wide engineering best practices around PySpark performance tuning, Git version control, and CI/CD pipelines.- Conducted routine cluster cost optimizations and query tuning across cloud infrastructure, adjusting Spark driver/executor configurations, dynamic allocation, and storage layouts to maximize compute efficiency. (ref:hirist.tech)

Similar vacancies

More like this role

... Airflow, Datawarehousing, Data https://jobeax.com/link/jJwPeXOR5xzgNWwu skills :- Strong hands-on experience in Python Development.- Extensive experience with PySpark / Apache Spark.- Expertise in Data Engineering, SQL Unix + Big Data (Hadoop + Hive), Airflow, Datawarehousing, Data modelling and data processing frameworks.- ...

Chennai, भारत

  • Full-time
  • Data Analytics
... reporting, or liquidity risk. - Demonstrated expertise with Hadoop, Spark including PySpark and Scala Spark, Kafka, and cloud-based data platforms such as AWS S3 and Databricks. - Experience with data modeling tools and techniques. - Experience with Python or R for data analysis is highly desirable. - Experience preparing data ...

Pune, भारत

Match Me Resume Match | Citi

Big Data pyspark developer

  • Full-time
... solutions and improvements - Conduct testing and debugging, utilize script tools, and write basic code for design specifications What you need - Spark - AWS - Data modelling - Hadoop - Kafka - Databricks Good to have - Machine learning - Python - R - Scala Job details - Location: Pune - Experience: 3–5 years - Education: ...

Pune, भारत

... related field. - 5+ years of IT experience with5+ years in Data Engineering and Big Data development. - Strong hands-on experience with PySpark and Spark-based data processing. - Experience developing data pipelines on Cloudera Data Platform (CDP) . - Strong knowledge of Apache Iceberg table architecture and data lake concepts. ...

Bangalore, भारत

  • Full-time
Mid-level data science role at Citi in Pune. Needs 4+ years. What you'll do - Responsible for participation in the establishment and implementation of new or revised application systems and programs in coordination with the Technology team What you need - Spark - SQL - Python - Hadoop - ETL - Linux Good to have - FastAPI ...

Pune, भारत

  • Backend
... services using Django.- Design and implement scalable and secure APIs.- Integrate Azure Cloud services for application deployment and management.- Utilize Azure Databricks for big data processing and analytics.- Implement data processing pipelines using PySpark.- Collaborate with front-end developers, product managers, and ...

Indhur, भारत

  • IT
  • Fullstack
... development practices.- Design and implement scalable and secure APIs.- Integrate Azure Cloud services for application deployment and management.- Utilize Azure Databricks for big data processing and analytics.- Implement data processing pipelines using PySpark.- Collaborate with front-end developers, product managers, and ...

Indhur, भारत

Persistent Systems

Developer

  • Hybrid
  • Full-time
  • Backend

$436 USD - $1654.4 USD

... are seeking a skilled Abinitio-Python Developer with strong expertise in Python PySpark Advanced SQL and Big Data technologies to design develop and optimize data processing applications. The ideal candidate will have hands-on experience in data engineering ETL development cloud platforms and large-scale data processing ...

Bangalore, भारत

PradeepIT Consulting Services

Big Data Developer

  • Full-time
Senior data science role at PradeepIT Consulting Services in Bengaluru. Needs 7–10 years. What you'll do - Design, develop, and implement highly scalable and distributed big data - Architect HBase schemas and data models to accommodate evolving business - Develop complex Hive queries and data processing pipelines to transform ...

Bangalore, भारत

  • Remote
  • Full-time
Senior software engineering role at Cognyte. Needs 6–9 years and a B.Sc. Remote. What you'll do - Design, develop, test, and maintain high-quality BigData systems and services - Take ownership of features and components aligned with your experience level - Collaborate with cross-functional teams to deliver scalable and ...

Pune, भारत