We are seeking a highly skilled and experienced Python/PySpark Lead to drive the design, development, and delivery of large-scale data engineering solutions. The ideal candidate should possess strong technical expertise in Python and PySpark, coupled with proven leadership capabilities to manage project deliverables, mentor development teams, and engage effectively with business and technology stakeholders
Responsibilities
Lead the end-to-end delivery of Python/PySpark-based solutions, ensuring adherence to project timelines and quality standards.
Provide technical leadership and guidance to development teams across design, development, testing, and deployment phases.
Collaborate with business stakeholders, project managers, and technical teams to understand requirements and translate them into scalable solutions.
Drive stakeholder communication by providing regular project updates, highlighting risks, dependencies, and mitigation plans.
Perform hands-on development and code reviews to ensure adherence to coding standards, best practices, and performance optimization techniques.
Foster collaboration, accountability, and a high-performance culture within the team.
Support estimation, planning, resource coordination, and delivery governance activities.
Ensure production stability, resiliency, and continuous improvement of data processing frameworks.
Qualifications
10+ years of overall IT experience with strong expertise in Python and PySpark.
Extensive experience in designing and developing scalable data processing and data transformation solutions.
Strong knowledge of Spark architecture, distributed computing, performance tuning, and optimization.
Experience working with Hadoop ecosystem components, Hive, SQL, and large-scale data platforms.
Strong understanding of software engineering principles, design patterns, and Agile delivery methodologies.
Excellent problem-solving, debugging, and analytical skills.
Proven experience in leading technical teams and driving project execution.
Excellent verbal and written communication skills with demonstrated stakeholder management experience.
Preferred Qualifications
Experience in Banking and Financial Markets domain is an added advantage.
Exposure to cloud-based data platforms such as Databricks, AWS, or Azure is desirable.
Experience in mentoring and developing high-performing engineering teams.
... with business objectives, ensuring solutions are both technically sound and strategically relevant. - Author, review, and approve technical requirements and architectural designs, and lead process re-engineering efforts to deliver cost-effective, high-impact business solution - Drives team adoption of enterprise-authorized ...
Technical Lead / Data Architect Azure | 8+ Years | BangaloreHiring an experienced Technical Lead / Data Architect to drive end-to-end technical architecture and delivery of cloud and data engineering projects. This is an excellent opportunity for professionals with strong expertise in Azure, Azure Databricks, Python, Data ...
... Architecture.- Strong hands-on experience in Azure Databricks, Python, PySpark, and SQL.- Proven experience architecting and developing reusable, metadata-driven Python frameworks.- Strong expertise in enterprise data modeling and complex data pipeline design.- Experience leading Data Engineering teams while remaining actively ...
... implementation lifecycle and define target-state architecture and roadmap.- Review technical designs and code developed by team members.- Guide developers on PySpark optimization, Databricks best practices, and data modeling.- Lead technical discussions with client architects and business stakeholders.- Ensure scalability, ...
ROLE SUMMARY: We are looking for an AI Solution Architect who can take an enterprise AI initiative from vision to execution. You will shape AI strategy with C-level stakeholders, architect cloud-native GenAI and ML platforms on Azure and Databricks, and lead cross-functional teams across data science, MLOps and product ...
... Engineering, or a related discipline; masters degree is preferred.- Strong expertise in designing modern data architectures, including Lakehouse, medallion architecture, data modeling, data warehousing, and scalable ingestion and transformation frameworks.- Deep technical proficiency in Databricks, PySpark, Python, SQL, ...
... Engineering and Architecture. You must have led at least 23 large-scale migrations from legacy to modern cloud platforms.- Technical Depth : - Expert-level Python/PySpark and SQL.- Deep expertise in Snowflake or Databricks (Certifications like SnowPro or Databricks Architect are highly preferred).- Familiarity with "Traditional" ...
... experience designing and implementing Generative AI solutions.- Strong expertise in RAG, Agentic AI, LLM orchestration, prompt engineering, and AI application architecture.- Strong programming skills in at least one major language used for AI/GenAI development, preferably Python, PySpark, or Java.- Hands-on experience integrating ...
... finance users to define benchmark queries and validate numbers during UAT and parallel runs. Report progress risks and decisions to stakeholders and delivery leadership. Required skills Area Databricks Unity Catalog Fine-grained security Delta Lake Performance PySpark and SQL Python engineering Azure DevOps Data modelling ...