Infometry - Senior Data Engineer - SQL/Python in Bangalore, भारत

Job Summary : We are seeking an experienced Senior Data Engineer with strong expertise in Databricks, Apache Airflow, SQL, and Python to design and optimize data pipelines, ETL workflows, and cloud-based data solutions. The ideal candidate will have deep experience in big data processing, cloud-based analytics, and automation to drive efficient and scalable data engineering https://jobeax.com/link/XnflZD2dGSZYgXFG Responsibilities : - Design, develop, and optimize data pipelines using Databricks and Apache Airflow.- Implement PySpark-based transformations and processing in Databricks for handling large-scale data.- Develop and maintain SQL-based data pipelines, ensuring performance tuning and optimization.- Create Python scripts for automation, data transformation, and API-based data ingestion.- Work with Airflow DAGs to schedule and orchestrate data workflows efficiently.- Optimize data lake and data warehouse performance for scalability and reliability.- Integrate data pipelines with cloud platforms (AWS, Azure, or GCP) and various data storage solutions.- Ensure adherence to data security, governance, and compliance https://jobeax.com/link/NKlt3YnZFxTxA2qx Skills & Qualifications : - 8-9 years of experience in Data Engineering or related fields.- Strong expertise in Databricks (PySpark, Delta Lake, DBSQL).- Proficiency in Apache Airflow for scheduling and orchestrating workflows.- Advanced SQL skills for data extraction, transformation, and performance tuning.- Strong programming skills in Python (pandas, NumPy, PySpark, APIs).- Experience with big data technologies and distributed computing.- Hands-on experience with cloud platforms (AWS / Azure / GCP).- Expertise in data warehousing and data modeling concepts.- Understanding of CI/CD pipelines, version control (Git)- Experience with data governance, security, and compliance best practices.- Excellent troubleshooting, debugging, and performance optimization skills. (ref:hirist.tech)