Vacancy description
Vision Excel Career Solutions
India, Hyderabad
AWS Data Architect - Python/Apache Airflow in Hyderabad, India is listed on Jobeax. Browse 30,000+ vacancies available.
Role : AWS Data Architect Job Description :Candidate should possess a deep understanding of big data technologies, cloud services, and data architecture, with a proven track record of leading data-driven projects to successful https://jobeax.com/link/CEKJnNMNtDL19sum Responsibilities :- Lead a team of data engineers, providing technical guidance and mentorship. - Develop and execute a strategic roadmap for data processing, storage, and analytics in alignment with organizational goals.- Design, implement, and maintain robust data pipelines using Python and Airflow, ensuring efficient data flow and transformation for analytical and operational purposes.- Utilize AWS services, including S3 for data storage, Glue and EMR for data processing, and orchestrate data workflows that are scalable, reliable, and secure.- Implement real-time data processing solutions using Kafka, SQS, and Event Bridge, addressing high-volume data ingestion and streaming needs.- Oversee the integration of diverse systems and data sources through AppFlow, APIs, and other integration tools, ensuring seamless data exchange and connectivity.- Lead the development of data warehousing solutions, applying best practices in data modelling to support efficient data storage, retrieval, and analysis.- Continuously monitor, optimize, and troubleshoot data pipelines and infrastructure, ensuring optimal performance and scalability.- Ensure adherence to data governance, privacy, and security policies, implementing measures to protect sensitive data and comply with regulatory https://jobeax.com/link/COQih4IcZKKIeDR1 :- Bachelor's or master's degree in computer science, Engineering, or a related field.- 8-10 years of experience in data engineering, with at least 3 years in a leadership role.- Proficient in Python programming and experience with Airflow for workflow management.- Strong expertise in AWS cloud services, particularly in data storage, processing, and analytics (S3, Glue, EMR, etc.).- Experience with real-time streaming technologies like Kafka, SQS, and Event Bridge.- Solid understanding of API based integrations and familiarity with integration tools such as AppFlow.- Deep knowledge of data warehousing concepts, data modelling techniques, and experience in implementing largescale data warehousing solutions.- Demonstrated ability to lead and mentor technical teams, with excellent communication and project management skills. (ref:hirist.tech)
Similar vacancies
BlackBox
... and troubleshoot automated data workflows using Apache Airflow. Stream Processing: Collaborate with the team to build and monitor real-time or near-real-time data streaming applications using Apache Flink. Coding & Debugging: Write clean, readable, and well-documented code in Python. Continuous Learning: Actively participate ...
India, Mumbai
talentgigs
... complex joins, window functions, query optimization; Oracle or PostgreSQL experience is a plus - Solid understanding of ETL/ELT patterns, data modeling, and data quality frameworks - Experience with workflow orchestration: Apache Airflow, AWS Step Functions, or equivalent - CI/CD experience: GitHub Actions, AWS CodePipeline, ...
India, Chennai
Weekday AI
... for an experienced Data Engineer to design, develop, and maintain scalable data platforms and pipelines using AWS, Apache Spark, Python, SQL, Kafka, and modern data engineering frameworks . The role will focus on building reliable data solutions for high-volume batch and real-time workloads, including data ingestion, transformation, ...
India, Mumbai
Accenture
... frameworks for building scalable data pipelines. Understanding of Big Data architectures, data storage, and data processing concepts. Familiarity with cloud-native data storage solutions such as S3, Blob Storage, BigQuery, or Redshift. Experience with data orchestration tools like Apache Airflow or similar. Knowledge of data ...
India, Bengaluru
Databricks
20k
... community. 12+ years in data engineering, data science, technical architecture, or a similar pre-sales / consulting role. - 8+ years hands-on experience with Big Data and AI technologies, including Apache Spark™, data engineering, data science, and modern AI/ML workloads. - Strong hands-on architectural experience — able to ...
India, Bengaluru
Questhiring
... Storage Programming: Python, SQL Data Processing: Dataflow, Apache Beam, Dataproc, Spark/PySpark Orchestration: Cloud Composer, Apache Airflow Transformation: Dataform, dbt Engineering: Git, CI/CD, Automated Testing, Monitoring Data Engineering: ETL/ELT, Batch Processing, Data Modelling, Data Quality, Data Contracts, Metadata ...
India, Haryāna
srs consulting inc
... understanding of Apache Airflow and ability to design and manage complex DAGs. - Solid SQL skills and familiarity with data warehouse platforms (e.g., Snowflake, Redshift, BigQuery). - Familiarity with version control tools (Git), CI/CD pipelines, and Agile methodologies. - Exposure to cloud environments like AWS, GCP, or Azure
India, Pune
Accenture
... scalable solutions while balancing long-term architecture goals with project delivery needs. Key Skills: Data anonymization, privacy-by-design, and clinical trial data concepts Python, R, Apache Airflow, Kafka, REST/gRPC integrations Data formats: CSV, Parquet, SAS, RDS AWS (EKS, S3, IAM, DocumentDB) Solution/Data Architecture, ...
India, Rāniganj
... Dataflow, Pub/Sub, and Airflow (Composer). BigQuery: Strong SQL skills with expertise in query optimization, partitioning, clustering, and cost management. Airflow (Composer): Proven ability to build, schedule, and monitor ETL/ELT pipelines using Apache Airflow. Programming: Proficiency in Python or Java for data processing, ...
India, Mannady
Funic Tech
... scalable ETL/ELT data pipelines.- Build and manage distributed and event-driven architectures for large-scale data processing.- Develop backend services and data infrastructure using Python.- Implement DAG-based workflow orchestration using Airflow, Prefect, or Dagster.- Work with AWS services including Lambda, S3, ECS/EKS, ...
India, Pune
... Requirements : - 4 - 6 years of experience in data engineering or a closely related field, with strong hands-on production experience.- Expert-level SQL and strong Python skills, with experience writing production-grade, maintainable, and well-tested code.- Strong hands-on experience with Apache Airflow or a comparable workflow ...
India, Gurugram
... at least 2 years in a technical leadership role overseeing data engineering or data platform teams. - Strong proficiency in big data technologies, including Apache Spark/PySpark, Apache Airflow and Apache Kafka. - Experience with programming languages such as Scala and Python for developing robust data pipelines. - In-depth ...
India, Bengaluru
calfus inc.
... Engineering: Architect Medallion (Bronze/Silver/Gold) or equivalent layered data platform designs using Delta Lake; own Unity Catalog governance, access control, and data lineage. Data Integration: Oversee ETL/ELT pipeline development and orchestration (Apache Airflow, ADF), including legacy migration support (e.g., SSIS, Informatica, ...
India, Bengaluru
Rs 5 - 30 lakhs p.a.
... JavaScript. - Knowledge of database systems (e.g., PostgreSQL, MySQL, MongoDB). - Familiarity with version control tools like Git. - Understanding of RESTful APIs and web services integration. - Experience with cloud services (e.g., AWS, Azure) is a plus. - Strong analytical and problem-solving skills, with attention to detail.
India, Rāniganj
varun malhotra designs
... position sizing, max daily loss limits, and 'kill-switch' functionality. Backtesting & Logging: Create a framework to test strategies against historical NSE data and maintain detailed logs for audit and debugging. Required Skills & Qualifications Language Proficiency: Strong expertise in Python (preferred for its libraries ...
India, Delhi
synapse xtl
... maintain Python applications for data extraction, transformation, automation, and reporting. Build and optimize data pipelines and ETL processes to handle large datasets from multiple sources. Develop reusable Python modules to streamline analytics and reporting workflows. Integrate APIs, databases, and external data sources ...
India, Indore
... using Terraform, Kubernetes, and AWS services such as RDS, MSK, and EC2. Deploy applications into AWS environments. Develop and maintain scripts and tools using Python to automate common tasks. Deploy and manage PostgreSQL and other databases to ensure high availability and data consistency. Deploy and manage Kafka clusters ...
India, Chennai
Vehant Technologies
Rs 5 - 8 lakhs p.a.
... or equivalent with a strong software architecture mindset and adept in documentation of work Strong understanding and experience in real-time process control, data flow and multi-threaded process control architecture over a heterogeneous compute hardware architecture involving CPU and GPU and networked computers. Software ...
India, Bengaluru
... GCP data engineer- Pyspark- Dataflow- Dataproc- BigQuery- AirflowKey Responsibilities :- Design, develop, and optimize endtoend data pipelines using PySpark, Dataflow, and Dataproc- Implement data ingestion, transformation, and loading processes into BigQuery- Orchestrate workflows and schedule jobs with Apache Airflow- ...
India, Pune