Back to list

Vacancy description

Senior Data Engineer in Chennai, भारत

Merrill Communications

  • Full-timeStandard weekly hours

Chennai, भारत

Job Description: Responsibilities:

  • Develop & Optimize Data Pipelines
  • Build, test, and maintain ETL/ELT data pipelines using Azure Databricks & Apache Spark (PySpark) .
  • Optimize performance and cost-efficiency of Spark jobs.
  • Ensure data quality through validation, monitoring, and alerting mechanisms.
  • Understand cluster types, configuration, and use-case for serverless
  • Implement Unity Catalog for Data Governance
  • Design and enforce access control policies using Unity Catalog.
  • Manage data lineage, auditing, and metadata governance .
  • Enable secure data sharing across teams and external stakeholders.
  • Integrate with Cloud Data Platforms
  • Work with Azure Data Lake Storage / Azure Blob Storage/ Azure Event Hub to integrate Databricks with cloud-based data lakes, data warehouses, and event streams .
  • Implement Delta Lake for scalable, ACID-compliant storage.
  • Automate & Orchestrate Workflows
  • Develop CI/CD pipelines for data workflows using Azure Databricks Workflows or Azure Data Factory .
  • Monitor and troubleshoot failures in job execution and cluster performance .
  • Collaborate with Stakeholders
  • Work with Data Analysts, Scientists, and Business Teams to understand requirements.
  • Translate business needs into scalable data engineering solutions .
  • API expertise
  • Ability to pull data from a wide variety of APIs using different strategies and methods
Required Skills & Experience:
  • Azure Databricks & Apache Spark (PySpark) – Strong experience in building distributed data pipelines .
  • Python – Proficiency in writing optimized and maintainable Python code for data engineering.
  • Unity Catalog – Hands-on experience implementing data governance, access controls, and lineage tracking .
  • SQL – Strong knowledge of SQL for data transformations and optimizations.
  • Delta Lake – Understanding of time travel, schema evolution, and performance tuning .
  • Workflow Orchestration – Experience with Azure Databricks Jobs or Azure Data Factory .
  • CI/CD & Infrastructure as Code (IaC) – Familiarity with Databricks CLI, Databricks DABs, and DevOps principles .
  • Security & Compliance – Knowledge of IAM, role-based access control (RBAC), and encryption .
Preferred Qualifications:
  • Experience with MLflow for model tracking & deployment in Databricks.
  • Familiarity with streaming technologies (Kafka, Delta Live Tables, Azure Event Hub, Azure Event Grid).
  • Hands-on experience with dbt (Data Build Tool) for modular ETL development.
  • Certification in Databricks, Azure is a plus.
  • Experience with Azure Databricks Lakehouse connectors for SalesForce and SQL Server
  • Experience with Azure Synapse Link for Dynamics, dataverse
  • Familiarity with other data pipeline strategies, like Azure Functions, Fabric, ADF, etc
Soft Skills:
  • Strong problem-solving and debugging skills.
  • Ability to work independently and in teams .
  • Excellent communication and documentation skills.
Apply to vacancy

Similar vacancies

More like this role

NorthBay Solutions

Senior Data Engineer

  • On-site
  • Full-time
... Onsite Required Experience: 610 Years Job Summary We are seeking an experienced Senior Data Engineer to design develop and manage modern data platforms scalable data pipelines and cloud-based analytics solutions. The ideal candidate will have strong hands-on expertise in Microsoft Azure Data Services large-scale data processing ...

भारत

Luxoft

Senior Data Engineer

  • Full-time
... migrating data pipelines from on-prem or other cloud platforms Understanding of data governance, security, and access control on AWS Experience with CI/CD for data pipelines Nice to have AWS certification (Data Analytics Specialty or Data Engineer Associate) Experience with streaming data (Kinesis, MSK/Kafka) Familiarity ...

Bangalore, भारत

Persistent Systems

Senior Data Engineer

  • Full-time

$436 USD - $1654.4 USD

... (PE) Services PEAK MatrixAssessment 2026andSoftware Product Engineering PEAK MatrixAssessment 2026. AboutPosition: We are looking for a highly skilled Senior Data Engineer with expertise in SQL Cloud Data Platforms Data Architecture and Data Integration to design and build scalable secure and high-performance data solutions. ...

Bangalore, भारत

Ge Healthcare

Senior Data Engineer

GE HealthCare seeks a Senior Data Engineer to design and implement data engineering solutions, ETL pipelines, and analytics infrastructure. The role involves data profiling, metadata management, and automation using cloud and low-code tools to support data science outcomes. Responsibilities - Design and build automations ...

Bangalore, भारत

Blend360

Senior Data Engineer

  • On-site
  • Full-time
... your world, creating more fulfilling work and projects for our people and clients. For more information, visit [ Job Description We’re looking for a skilled Senior Data Engineer with strong experience in distributed systems, Python/Scala, and modern data engineering tools to help design and implement an end-to-end data ...

Hyderabad, भारत

Reply

Senior Data Engineer

  • Full-time
Job Summary Valorem Reply is looking for a Data Engineer with experience building and contributing to the design of database systems, both normalized transactional systems and dimensional reporting systems. Strong experience with SQL Server as a database engine, as well as Microsoft and Databricks technology experience ...

Ernakulam, भारत

ORANGE

Senior Data Engineer

  • Remote
  • Hybrid
  • Full-time
  • Contract
... perfect your skills in a field filled with exciting challenges and opportunities. About the role As part of the Big Data B2B program OBS set up a shared Big Data platform and a Data Lake for Use cases exploration and industrialization. We are looking for Senior Data Engineer 5-8 years of experience in building Data Pipelines ...

Gurgaon, भारत

ORANGE

Senior Data Engineer

  • Remote
  • Hybrid
  • Full-time
  • Contract
... perfect your skills in a field filled with exciting challenges and opportunities. About the role As part of the Big Data B2B program, OBS set up a shared Big Data platform and a Data Lake for Use cases exploration and industrialization. We are looking for Senior Data Engineer 5-8 years of experience in building Data Pipelines ...

Gurgaon, भारत

Mastercard

Senior Data Engineer

  • On-site
  • Contract
... is a sophisticated technology ecosystem that processes billions of payment events with speed, resilience, and precision. As a Senior Data Engineer within the Data Collection & Engineering (DC&E) organisation, you will play a critical role in designing, building, and scaling Mastercard's next-generation data platforms and ...

Hyderabad, भारत

Swiss Re

Senior Data Engineer

  • Hybrid
... innovation, and large-scale data engineering? If so, we have an exciting opportunity for you to make a meaningful impact on a global scale. About the Role As a Senior Data Engineer , you will be at the heart of our ambition to become a truly data-driven organisation. You will design and deliver best-in-class data engineering ...

Bangalore, भारत