Back to list

Vacancy description

Datavail Infotech - Senior Data Engineer - Azure Data Factory & Databricks in Bangalore, भारत

Datavail

Bangalore, भारत

Senior Data Engineer - Azure Data Factory, Microsoft Fabric & https://jobeax.com/link/F7hq9A2NG049yICW is hiring a Senior Data Engineer to join our high-performing team building and supporting modern data platforms across a wide range of clients and https://jobeax.com/link/1eUjct1h9VTVITOE are looking for a senior-level data engineer who operates as a trusted technical advisor, someone who goes beyond building pipelines to turn business requirements into sound technical designs, lead platform migrations, and set engineering standards for the https://jobeax.com/link/GAQw2xZfHsdWGuLm ideal candidate will bring hands-on depth across Azure Data Factory, Microsoft Fabric and Databricks, mentor junior engineers, and own production support for client workloads with a blend of engineering rigor and consultative https://jobeax.com/link/g6CfXTUYsouKM6V8 is a full-time position with full medical https://jobeax.com/link/3pPCGloDELEEHsuu Data Factory, Microsoft Fabric & Databricks Expertise:- Metadata-driven ADF pipelines: designing reusable, parameterized pipelines for batch and incremental loads.- Hybrid ingestion & integration runtimes: ingesting from on-prem databases, files, REST APIs, SaaS applications and cloud storage.- ADF orchestration & monitoring: schedule, tumbling window and event-based triggers; orchestrating Databricks notebooks and jobs.- Fabric Lakehouse & Warehouse design: medallion (Bronze/Silver/Gold) architecture on OneLake; Warehouse objects in T-SQL.- Fabric data engineering: Data Pipelines and Dataflows Gen2 for ingestion and transformation; PySpark and Spark SQL.- Semantic modeling & capacity management: star-schema and Direct Lake semantic models for high-performance Power BI reporting.- Databricks development: scalable processing in PySpark, Spark SQL and Python; Databricks Workflows/Jobs.- Delta Lake design & Spark performance tuning: MERGE/upsert, SCD Type 1 and Type 2, schema evolution and Time Travel.- Unity Catalog & platform configuration: catalogs, schemas, access controls, lineage and external locations.- CI/CD across platforms: ADF source control with Git; Fabric Git integration; Databricks Asset https://jobeax.com/link/q6t1ng61NYN61aP4 this role, you will:- Serve as a trusted technical advisor to clients, turning business requirements into technical designs.- Lead migrations from legacy ETL (SSIS, Informatica, on-prem SQL) and from Azure Synapse to Microsoft Fabric and Databricks.- Design, develop and implement scalable data pipelines using Microsoft Fabric Data Factory, Azure Data Factory and Azure Databricks.- Set up security across all platforms using Key Vault, managed identities, RBAC, and row and column-level security.- Build ETL/ELT workflows to ingest data from databases, ERP and CRM systems.- Develop and maintain enterprise data warehouses and subject-area data marts.- Provide production support for Azure, Fabric and Databricks workloads.- Collaborate directly with client technical teams and stakeholders.- Participate in the on-call rotation to support critical production issues.- Design and implement data quality, validation and reconciliation checks.- Proactively monitor pipeline performance and platform costs.- Build reusable frameworks and templates for ingestion, logging and auditing.- Run discovery and onboarding for new client environments.- Lead data platform modernization roadmaps.- Partner with BI developers and analysts to deliver reliable, well-modeled data for Power BI reporting.- Coordinate with offshore and onshore team https://jobeax.com/link/u68N17RBB1wSbmoY Qualifications:- 12+ years of overall experience, including 6+ years of hands-on experience with Azure Data Factory and Databricks, and 2+ years of experience delivering production workloads using Microsoft Fabric.- Expert-level PySpark and Spark SQL writing and optimizing large-scale transformations.- Advanced SQL/T-SQL complex joins, window functions, CTEs, MERGE logic, and query tuning.- Strong Python for data engineering modular, testable code, reusable libraries, and unit testing with pytest.- Built and supported metadata/config-driven ADF frameworks in production.- Hands-on Delta Lake implementation, MERGE/upsert, SCD Type 1 and Type 2.- Production experience with Auto Loader, Structured Streaming or Delta Live Tables.- Hands-on Unity Catalog administration.- Built Fabric Lakehouses/Warehouses end to end.- Dimensional data modeling like star and snowflake schemas.- Azure platform skills like ADLS Gen2, Azure SQL, Key Vault, managed identities.- CI/CD implementation - Git branching strategies, Azure DevOps YAML pipelines or GitHub Actions.- Infrastructure as Code with Terraform or Bicep.- Data quality and observability implementation.- Proven track record migrating SSIS, Informatica or Synapse workloads.- Hands-on production support experience.- Clear verbal and written communication https://jobeax.com/link/ZitjVqvb84V4AOyQ to Have:- Snowflake, SSIS and SSRS, Streaming technologies (Fabric Real-Time Intelligence, Event Hubs, Kafka), Power BI and DAX.Certifications:- Preferred: Microsoft Certified: Fabric Data Engineer Associate (DP-700), Fabric Analytics Engineer Associate (DP-600), Azure Data Engineer Associate (DP-203), Databricks Certified Data Engineer Professional, or SnowPro Core. (ref:hirist.tech)

Similar vacancies

More like this role

  • Remote
JOB OVERVIEW:OSI Group is looking for a technically skilled, customer-focused Presales Engineer to support its growing IT infrastructure, networking and cybersecurity https://jobeax.com/link/muaVYoA9oiUKJZHd will work closely with the sales, technical, vendor and delivery teams to understand customer requirements, design ...

भारत

Mid-level data science role at Progress in Bengaluru. Needs 4+ years and a https://jobeax.com/link/2DENLGgQM7whDhaE, B.E. or B.Sc. Hybrid. What you'll do - Develop and manage data models, identity resolution rules, and segmentation frameworks - Develop and implement data normalization processes to ensure consistent, accurate, ...

Bangalore, भारत

... identity resolution rules, and segmentation frameworks - Develop and implement data normalization processes to ensure consistent, accurate, and usable customer data - Maintain comprehensive documentation of platform configurations, operational workflows What you need - Data warehousing - Machine learning - ETL - Data privacy ...

Bangalore, भारत

... Factory, AWS Glue, GCP Composer, etc.- Working knowledge of one or more OLTP databases (Postgres, MySQL, SQL Server, etc.).- Working knowledge of one or more Data Warehouse like Snowflake, Redshift, Azure Synapse, Hive, Big Query, etc.- Proficient in at least one programming language used in data engineering, such as Python ...

Hyderabad, भारत

... managers and other DataOps teams. This role involves hands-on development work and will directly report to Data Engineering Lead. Must have skills: - Azure Databricks, Unity Catalog, Delta live tables - Data Modelling & Architecture - ETL pipeline design - Expert in Python, Pyspark and SQL - Azure Data Factory - Azure DevOps ...

Hyderabad, भारत

Rs 10 - 20 lakhs p.a.

... next-gen cloud data platform - Design and implement scalable data models and ETL/ELT pipelines using Databricks, Azure Data Factory and AWS Glue - Conduct deep data analysis and build insightful dashboards and visualisations for internal and client stakeholders What you need - SQL - AWS - Python - Databricks - Azure Data ...

Calcutta, भारत

Procter & Gamble

Senior Data Engineer

... Bachelor's or master's degree in computer science, data engineering, or a related field, along with 4+ year work experience in data engineering and cloud platforms. - Azure and Databricks: Strong proficiency in Azure services such as Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Data Lake Storage, and Azure Synapse ...

Hyderabad, भारत

... Java, Scala, or Golang. Big Data & Analytics Concepts: Data modeling, star schemas, slowly changing dimensions (SCD), Apache Beam, and dbt. Education & Experience: Typically a Bachelor’s or Master’s degree in Computer Science or Engineering, Nice to have Banking Other Languages English: C2 Proficient Seniority Senior []

Pune, भारत

Procter & Gamble

Senior Data Engineer

... LangChain or equivalent) and context engineering for data-facing agents - Experience with orchestration and cloud data integration tools such as Airflow and Azure Data Factory. - Hands-on experience with CI/CD pipelines, unit testing, API integrations, and modern software engineering practices across Agile and DevOps environments. ...

Bombay, भारत

  • Full-time
Leadership data science role at Commvault in Bengaluru. Needs 8+ years and a https://jobeax.com/link/wOcGjnRrrRFUZzB4, https://jobeax.com/link/2DENLGgQM7whDhaE or B.E.. What you'll do - Platform Architecture & Engineering - Design and implement secure, scalable platform services for Databricks, Azure Data Lake, Microsoft ...

Bangalore, भारत