We are seeking a highly skilled and experienced Python/PySpark Lead to drive the design, development, and delivery of large-scale data engineering solutions. The ideal candidate should possess strong technical expertise in Python and PySpark, coupled with proven leadership capabilities to manage project deliverables, mentor development teams, and engage effectively with business and technology stakeholders
Responsibilities
Lead the end-to-end delivery of Python/PySpark-based solutions, ensuring adherence to project timelines and quality standards.
Provide technical leadership and guidance to development teams across design, development, testing, and deployment phases.
Collaborate with business stakeholders, project managers, and technical teams to understand requirements and translate them into scalable solutions.
Drive stakeholder communication by providing regular project updates, highlighting risks, dependencies, and mitigation plans.
Perform hands-on development and code reviews to ensure adherence to coding standards, best practices, and performance optimization techniques.
Foster collaboration, accountability, and a high-performance culture within the team.
Support estimation, planning, resource coordination, and delivery governance activities.
Ensure production stability, resiliency, and continuous improvement of data processing frameworks.
Qualifications
10+ years of overall IT experience with strong expertise in Python and PySpark.
Extensive experience in designing and developing scalable data processing and data transformation solutions.
Strong knowledge of Spark architecture, distributed computing, performance tuning, and optimization.
Experience working with Hadoop ecosystem components, Hive, SQL, and large-scale data platforms.
Strong understanding of software engineering principles, design patterns, and Agile delivery methodologies.
Excellent problem-solving, debugging, and analytical skills.
Proven experience in leading technical teams and driving project execution.
Excellent verbal and written communication skills with demonstrated stakeholder management experience.
Preferred Qualifications
Experience in Banking and Financial Markets domain is an added advantage.
Exposure to cloud-based data platforms such as Databricks, AWS, or Azure is desirable.
Experience in mentoring and developing high-performing engineering teams.
... with business objectives, ensuring solutions are both technically sound and strategically relevant. - Author, review, and approve technical requirements and architectural designs, and lead process re-engineering efforts to deliver cost-effective, high-impact business solution - Drives team adoption of enterprise-authorized ...
Technical Lead / Data Architect Azure | 8+ Years | BangaloreHiring an experienced Technical Lead / Data Architect to drive end-to-end technical architecture and delivery of cloud and data engineering projects. This is an excellent opportunity for professionals with strong expertise in Azure, Azure Databricks, Python, Data ...
... Architecture.- Strong hands-on experience in Azure Databricks, Python, PySpark, and SQL.- Proven experience architecting and developing reusable, metadata-driven Python frameworks.- Strong expertise in enterprise data modeling and complex data pipeline design.- Experience leading Data Engineering teams while remaining actively ...
... decision-making. Design and manage complex data pipelines and analytics solutions to address business challenges in the Insurance domain Develop scalable workflows using Python and PySpark to handle large datasets efficiently Optimize querying and data manipulation through expertise in SQL, with a preference for Spark SQL Architect ...
... decision-making. Design and manage complex data pipelines and analytics solutions to address business challenges in the Insurance domain Develop scalable workflows using Python and PySpark to handle large datasets efficiently Optimize querying and data manipulation through expertise in SQL, with a preference for Spark SQL Architect ...
... implementation lifecycle and define target-state architecture and roadmap.- Review technical designs and code developed by team members.- Guide developers on PySpark optimization, Databricks best practices, and data modeling.- Lead technical discussions with client architects and business stakeholders.- Ensure scalability, ...
... best practices.- Lead technical decisions related to scalability, performance, reliability, and security.- Design and build scalable ETL/ELT pipelines using PySpark on Databricks.- Architect Bronze, Silver, and Gold data transformation frameworks.- Design and manage Delta Lake storage architecture, including optimization ...
... to lead enterprise data modernization initiatives using Microsoft Fabric and Azure Data Platform technologies. This customer-facing role combines solution architecture, technical leadership, pre-sales support, and delivery https://jobeax.com/link/TkHlAUE0pvOY4b2m :- Lead architecture workshops and stakeholder discussions- ...
ROLE SUMMARY: We are looking for an AI Solution Architect who can take an enterprise AI initiative from vision to execution. You will shape AI strategy with C-level stakeholders, architect cloud-native GenAI and ML platforms on Azure and Databricks, and lead cross-functional teams across data science, MLOps and product ...