Back to list

Vacancy description

Data Engineer with Scala and Spark/ Pyspark in Bangalore, भारत

Capco

  • HybridMix of office and remote
  • Full-timeStandard weekly hours

Bangalore, भारत

About Us Capco, a Wipro company, is a global technology and management consulting firm. Awarded with Consultancy of the year in the British Bank Award and has been ranked Top 100 Best Companies for Women in India 2022 by Avtar & Seramount. With our presence across 32 cities across globe, we support 100+ clients across banking, financial and Energy sectors. We are recognized for our deep transformation execution and delivery.

WHY JOIN CAPCO?

You will work on engaging projects with the largest international and local banks, insurance companies, payment service providers and other key players in the industry. The projects that will transform the financial services industry.

MAKE AN IMPACT

Innovative thinking, delivery excellence and thought leadership to help our clients transform their business. Together with our clients and industry partners, we deliver disruptive work that is changing energy and financial services.

#BEYOURSELFATWORK

Capco has a tolerant, open culture that values diversity, inclusivity, and creativity.

CAREER ADVANCEMENT

With no forced hierarchy at Capco, everyone has the opportunity to grow as we grow, taking their career into their own hands.

DIVERSITY & INCLUSION

We believe that diversity of people and perspective gives us a competitive advantage.

MAKE AN IMPACT

Job Title: Data Engineer Position: Data Engineer Experience: 4+ Years Work Mode: Hybrid (Capco Office)

Key Responsibilities

  • Design, develop, and optimize large-scale data processing applications using Scala, Apache Spark, and Java .
  • Build and maintain high-performance data pipelines capable of processing 80–90 million records daily with a focus on scalability, reliability, and efficiency.
  • Develop and integrate Native APIs and data services to support business-critical applications and analytics platforms.
  • Collaborate with cross-functional teams to gather requirements and deliver robust data engineering solutions.
  • Optimize Spark jobs, data workflows, and distributed computing processes to ensure high throughput and low latency.
  • Implement best practices for data quality, monitoring, governance, and operational excellence.
  • Troubleshoot and resolve performance bottlenecks across data processing and ingestion pipelines.
  • Participate in code reviews and contribute to engineering standards, architecture decisions, and continuous improvement initiatives.

Required Skills & Experience

  • Strong hands-on experience in Scala , Apache Spark , and Java .
  • Proven experience building and supporting large-scale distributed data processing systems handling tens of millions of records daily.
  • Expertise in developing and consuming Native APIs and microservices.
  • Strong understanding of Spark architecture, performance tuning, partitioning, caching, and optimization techniques.
  • Experience with data modeling, ETL/ELT processes, and large-scale batch and streaming data pipelines.
  • Solid understanding of distributed systems, concurrency, and high-volume data processing.
  • Experience with SQL and relational/non-relational databases.
  • Strong debugging, analytical, and problem-solving skills.

Preferred Qualifications

  • Experience working in enterprise-scale data environments within Financial Services, Payments, or FinTech domains.
  • Exposure to cloud platforms (AWS, Azure, or GCP) and containerized deployments.
  • Familiarity with Kafka, Airflow, Hadoop ecosystem, or similar big data technologies.
  • Experience with CI/CD pipelines, DevOps practices, and Agile methodologies.

What We're Looking For

  • Engineers passionate about solving complex data challenges at scale.
  • Professionals who can design highly performant systems capable of processing and transforming massive datasets efficiently.
  • Team players who thrive in fast-paced, collaborative environments and take ownership of end-to-end delivery.

Similar vacancies

More like this role

  • Hybrid
  • Full-time
... daily with a focus on scalability, reliability, and efficiency. - Develop and integrate Native APIs and data services to support business-critical applications and analytics platforms. - Collaborate with cross-functional teams to gather requirements and deliver robust data engineering solutions. - Optimize Spark jobs, data ...

Hyderabad, भारत

  • Full-time
... Chennai, Hyderabad and 1 more city. What you'll do - A experienced Scala Developer responsible for building Spark-based data pipelines, performing advanced data engineering tasks, managing big data workflows in HDFS - This position offers an exciting opportunity to contribute to high-impact data engineering projects within a ...

Hyderabad, भारत

... Big Data Engineer – Scala We are looking for an experienced Big Data Engineer with strong expertise in Scala and Apache Spark to design, develop, and optimize scalable data-processing solutions. The ideal candidate should have hands-on experience with distributed data platforms, large-scale ETL/data pipelines, and big data ...

Pune, भारत

... Big Data Engineer – Scala We are looking for an experienced Big Data Engineer with strong expertise in Scala and Apache Spark to design, develop, and optimize scalable data-processing solutions. The ideal candidate should have hands-on experience with distributed data platforms, large-scale ETL/data pipelines, and big data ...

Pune, भारत

  • On-site

Rs 10 - 19 lakhs p.a.

... looking for an experienced Big Data Applications Engineer with strong software development skills and hands-on experience in AWS, distributed data processing, and API development. Key Responsibilities: 1. Design, develop, test, and deploy scalable Big Data applications using Java, Scala, Python, and Spark. 2. Build and ...

Hyderabad, भारत

Zorba Consulting India

AWS Big Data Engineer

Rs 10 - 19 lakhs p.a.

... Design, develop, test, and deploy scalable Big Data applications and data engineering solutions on AWS. Develop API-driven backend services to manage, monitor, and govern large-scale batch data processing applications. Build and maintain data pipelines using Apache Spark, Python, Scala, SQL, Hive, and Hadoop. Work with AWS ...

Hyderabad, भारत

  • Full-time
... Pune. What you'll do - Design, develop, and maintain scalable Big Data applications and data pipelines - Develop high-performance data-processing solutions using Scala and Apache Spark - Build and optimize ETL/ELT pipelines for large volumes of structured and unstructured data What you need - Scala - SQL - ETL - Spark - Hadoop ...

Pune, भारत

Zorba Consulting India

Big Data Engineer

  • On-site

Rs 10 - 18 lakhs p.a.

... an experienced Big Data Engineer with strong software engineering and distributed data processing skills. The ideal candidate should have hands-on experience with Java, Apache Spark, Python/Scala, and the Hadoop ecosystem to build scalable, reliable data solutions. Roles & Responsibilities Analyze complex systems and collaboratively ...

Hyderabad, भारत

  • Full-time
... experienced Senior Data Engineer with strong expertise in Apache Kafka, Apache Spark, Scala/PySpark, SQL, and modern data engineering technologies . The ideal candidate should have hands-on experience designing, developing, optimizing, and maintaining scalable data pipelines, including batch and real-time streaming solutions. ...

Pune, भारत

  • Full-time
... experience in Data Engineering. - Strong programming skills in Python and Scala . - Hands-on experience with Databricks and Apache Spark. - Expertise in Spark SQL, DataFrames, Delta Lake , and Databricks Workflows. - Strong SQL development and query optimization experience. - Experience with ETL/ELT frameworks and data integration. ...

Pune, भारत