Data Engineer with SQL in Pune, India is listed on Jobeax. Browse 30,000+ vacancies available.
Senior Data Engineer – Data Pipelines & Cloud Databases Location: Pune | Experience: 4+ years | Type: Full-Time Role Overview Design, build, manage enterprise data pipelines on Azure and Databricks. Own schema design, API development, and data infrastructure for analytics and intelligence products. Key Responsibilities
Data Pipeline Development Implement robust, scalable data pipelines using Microsoft Azure and Databricks stack Build reusable data pipeline components and frameworks Design and optimize data workflows for performance and reliability Monitor pipeline health and implement automated alerting Database Architecture Design relational and non-relational database schemas MongoDB: schema design, aggregation pipelines, indexing, sharding, replica sets, performance tuning Azure Cosmos DB: multi-model access, partitioning, consistency levels, throughput management (RU/s), global distribution Azure Cosmos DB Gremlin API: graph data modeling, traversals, vertex/edge design, relationship analytics API Development Build RESTful APIs using FastAPI with async endpoint design Implement Pydantic models, middleware, dependency injection, background tasks Generate and maintain auto-generated OpenAPI/Swagger documentation Configure Azure API Management (APIM) for API gateway and security Project & Stakeholder Management Collect progress updates from squads regularly Consolidate updates into weekly project status reports Develop L3-level project plans with task details, milestones, dependencies Participate in early-stage design and feature definition Communicate complex data insights to non-technical stakeholders Collaboration & Integration Work across multiple engineering teams on prototype integration Support integration of proven prototypes into core intelligence products Strong team collaboration and cross-functional communication Knowledge sharing and documentation Required Experience: Data Engineering & Databases
4+ years data engineering or data platform experience
API & Backend Development FastAPI proficiency: async endpoints, Pydantic models, middleware, dependency injection Background tasks and job scheduling RESTful API design and best practices OpenAPI/Swagger documentation Cloud Platform Azure fundamentals and hands-on experience Azure Data Factory or Databricks for ETL/ELT Azure Cosmos DB multi-region setup Azure API Management (APIM) configuration Data Pipeline Skills ETL/ELT pipeline design and development Data quality validation and monitoring Schema design for analytics and reporting Performance optimization and scalability
Preferred Experience Databricks Delta Lake experience Azure Synapse Analytics Python for data engineering Spark SQL optimization Real-time data streaming Data governance and metadata management Agile/Scrum development model CI/CD pipeline experience
... and detail-oriented Associate Data Engineer to join our growing data team. If you are a recent graduate passionate about data and eager to build a career in data engineering, this is the perfect opportunity for you! In this role, you will work closely with senior engineers to build, maintain, and optimize data pipelines. ...
... Minimum 7 years as a Data Engineer or similar role. Hands-on experience with Databricks, Delta Lake, Spark, and Scala. Proven ability to design, build, and operate Data Lakes or Data Warehouses. Proficiency with Data Orchestration tools (Airflow, Dagster, Prefect). Familiarity with Change Data Capture tools (Canal, Debezium, ...
... real-time decisioning. Solid engineering foundation with 5+ years of relevant experience in data engineering and/or distributed systems. Hands-on work building data pipelines (batch and/or streaming) using at least one distributed framework (e.g., Comfortable with SQL and a transformation framework (e.g., Familiarity with ...
... Bachelor's degree (preferably in Computer Science, Mathematics, Data Science, or a related field). - 3-6 years of overall experience in software development and data analysis. - Hands-on experience in Python for application development. - Strong proficiency in SQL for data analysis and query generation. - Experience with hosted ...
... AWS Glue, Lambda, and DBT. Work with PySpark and SQL to transform and cleanse data. Configure and manage Airflow workflows and Amazon EMR clusters. Integrate data services with API Gateway and REST/Soap APIs. Required qualifications - 3 to 5 years of experience in data engineering. - Proficient in AWS S3, Glue, Lambda, ...
... scalable data engineering and data warehouse solutions while managing data schemas, SQL query tuning, and code reviews. Who You Are: - 5+ years of experience in Data Engineering, with strong knowledge of Data Platforms and Data Warehousing - 2+ years of hands-on experience with SSAS Tabular Models - Experience designing data ...
... teams including data analysts and engineers. Write reusable, efficient, and well-documented code. Integrate APIs and third-party services. Troubleshoot and debug data-related issues. Required Skills Strong proficiency in Python. Experience with data libraries like Pandas, NumPy. Hands-on experience with SQL and relational databases. ...
... Copilot with prompt engineering to increase engineering velocity. Enforce data governance, risk management policies, and regulatory compliance standards across all data platforms. Preferred Candidate Profile Work Experience: 8+ years in large-scale application development with 5+ years in a Python and PySpark Data Engineering ...
We are looking for a Data Engineer to join our team and play a key role in building, optimizing, and managing data pipelines. You will work closely with Data Analysts and Business Functions to ensure seamless data processing, reporting, and dashboarding. Our tech stack includes BigQuery, Snowflake, Airflow, Stitch/Fivetran, ...
... - Strong proficiency in Python, SQL, and Spark/PySpark.- Hands-on experience with Azure Data Factory, Azure Databricks, and/or Synapse Analytics.- Experience with ETL/ELT frameworks and data integration tools.- Good understanding of data engineering concepts and data pipeline architecture.- Experience with data transformation, ...
... experience with Python, SQL, and PySpark. Strong understanding of unit testing, test automation, debugging, and code quality best practices. Good understanding of data engineering fundamentals and advanced concepts. Experience building scalable and fault-tolerant data pipelines. Strong knowledge of CI/CD, automated testing, ...
... Technical Bachelor/Master's Degree with 4+ years of experience across Data Engineering (Data Pipelining, Warehousing, ETL Tools etc.) Extensive hands-on experience with data engineering techniques and Python & SQL Strong working knowledge of Snowflake, Airflow and dbt You are comfortable and have expertise in data engineering ...
... Professional & Technical Skills: Must To Have Skills: Proficiency in Python (Programming Language). Strong understanding of data structures and algorithms. Experience with ETL tools and data integration techniques. Familiarity with cloud platforms such as AWS or Azure. Knowledge of database management systems and SQL. Additional ...
... understood with data Compile and analyze data related to business' issues Develop clear visualizations to convey complicated data in a straightforward fashion Qualifications - Bachelor's or Master's degree in Statistics or Applied Mathematics or equivalent experience - 1 - 2 years' Data Analysis experience - Proficient in SQL
... reusable, scalable, and maintainable Python code. Write complex and optimized SQL queries , stored procedures, joins, subqueries, and CTEs. Work with relational databases such as PostgreSQL, MySQL, Oracle, or SQL Server . Develop and integrate REST APIs and backend services. Troubleshoot application and database-related issues ...
... reusable, scalable, and maintainable Python code. Write complex and optimized SQL queries , stored procedures, joins, subqueries, and CTEs. Work with relational databases such as PostgreSQL, MySQL, Oracle, or SQL Server . Develop and integrate REST APIs and backend services. Troubleshoot application and database-related issues ...
... expectations are met through disciplined implementation (access controls, lineage, and quality standards) 5+ years building and operating production data pipelines (data engineering—not primarily BI/analytics or data science) - Strong SQL plus strong data modeling skills (dimensional and/or lakehouse modeling) - Hands-on Databricks ...