Vacancy description
Vision Excel Career Solutions
India, Hyderabad
AWS Data Architect - Python/Apache Airflow in Hyderabad, India is listed on Jobeax. Browse 30,000+ vacancies available.
Role : AWS Data Architect Job Description :Candidate should possess a deep understanding of big data technologies, cloud services, and data architecture, with a proven track record of leading data-driven projects to successful https://jobeax.com/link/CEKJnNMNtDL19sum Responsibilities :- Lead a team of data engineers, providing technical guidance and mentorship. - Develop and execute a strategic roadmap for data processing, storage, and analytics in alignment with organizational goals.- Design, implement, and maintain robust data pipelines using Python and Airflow, ensuring efficient data flow and transformation for analytical and operational purposes.- Utilize AWS services, including S3 for data storage, Glue and EMR for data processing, and orchestrate data workflows that are scalable, reliable, and secure.- Implement real-time data processing solutions using Kafka, SQS, and Event Bridge, addressing high-volume data ingestion and streaming needs.- Oversee the integration of diverse systems and data sources through AppFlow, APIs, and other integration tools, ensuring seamless data exchange and connectivity.- Lead the development of data warehousing solutions, applying best practices in data modelling to support efficient data storage, retrieval, and analysis.- Continuously monitor, optimize, and troubleshoot data pipelines and infrastructure, ensuring optimal performance and scalability.- Ensure adherence to data governance, privacy, and security policies, implementing measures to protect sensitive data and comply with regulatory https://jobeax.com/link/COQih4IcZKKIeDR1 :- Bachelor's or master's degree in computer science, Engineering, or a related field.- 8-10 years of experience in data engineering, with at least 3 years in a leadership role.- Proficient in Python programming and experience with Airflow for workflow management.- Strong expertise in AWS cloud services, particularly in data storage, processing, and analytics (S3, Glue, EMR, etc.).- Experience with real-time streaming technologies like Kafka, SQS, and Event Bridge.- Solid understanding of API based integrations and familiarity with integration tools such as AppFlow.- Deep knowledge of data warehousing concepts, data modelling techniques, and experience in implementing largescale data warehousing solutions.- Demonstrated ability to lead and mentor technical teams, with excellent communication and project management skills. (ref:hirist.tech)
Similar vacancies
talentgigs
... complex joins, window functions, query optimization; Oracle or PostgreSQL experience is a plus - Solid understanding of ETL/ELT patterns, data modeling, and data quality frameworks - Experience with workflow orchestration: Apache Airflow, AWS Step Functions, or equivalent - CI/CD experience: GitHub Actions, AWS CodePipeline, ...
India, Chennai
... Strong understanding of cloud platforms (AWS, GCP, or Azure). Preferred Experience with Data Engineering concepts, ELT/ETL pipelines, and workflow orchestration (Airflow/Dagster). Good to have experience with dbt, Snowflake, Databricks, Apache Beam, or Apache Flink. Good to have experience with building AI/ML or RAG infrastructure. ...
India, Ahmedabad
Accenture
... frameworks for building scalable data pipelines. Understanding of Big Data architectures, data storage, and data processing concepts. Familiarity with cloud-native data storage solutions such as S3, Blob Storage, BigQuery, or Redshift. Experience with data orchestration tools like Apache Airflow or similar. Knowledge of data ...
India, Bengaluru
mindbrain
... Apache Airflow Cloud Scheduler Experience designing and implementing batch and real-time data pipelines , data migration solutions, and scalable data-layer architectures across the GCP ecosystem is highly desirable. Data Warehouse & Data Modeling Strong understanding of: Data Warehouse Architecture Data Lake vs. Data Warehouse ...
India, Gurugram
RIVER MOBILITY PRIVATE LIMITED
... scooter features).Data Engineering & Infrastructure : - IoT Telemetry Pipelines : Architect low-latency, high-throughput streaming pipelines to ingest real-time data from vehicle sensors (VCU, BMS), mobile apps, and charging infrastructure using MQTT, Apache Kafka, or AWS Kinesis.- Modern Data Stack : Design and scale a unified ...
India, Bengaluru
Questhiring
... Storage Programming: Python, SQL Data Processing: Dataflow, Apache Beam, Dataproc, Spark/PySpark Orchestration: Cloud Composer, Apache Airflow Transformation: Dataform, dbt Engineering: Git, CI/CD, Automated Testing, Monitoring Data Engineering: ETL/ELT, Batch Processing, Data Modelling, Data Quality, Data Contracts, Metadata ...
India, Haryāna
srs consulting inc
... understanding of Apache Airflow and ability to design and manage complex DAGs. - Solid SQL skills and familiarity with data warehouse platforms (e.g., Snowflake, Redshift, BigQuery). - Familiarity with version control tools (Git), CI/CD pipelines, and Agile methodologies. - Exposure to cloud environments like AWS, GCP, or Azure
India, Pune
Commonwealth Bank
... writing efficient queries, database management, performance tuning, and handling complex data transformations. - Strong experience in designing and managing data orchestration workflows using Apache Airflow . - Solid understanding of streaming data processing and event-driven architectures . - Experience in CI/CD pipeline ...
India, Bengaluru
... Dataflow, Pub/Sub, and Airflow (Composer). BigQuery: Strong SQL skills with expertise in query optimization, partitioning, clustering, and cost management. Airflow (Composer): Proven ability to build, schedule, and monitor ETL/ELT pipelines using Apache Airflow. Programming: Proficiency in Python or Java for data processing, ...
India, Mannady
Funic Tech
... scalable ETL/ELT data pipelines.- Build and manage distributed and event-driven architectures for large-scale data processing.- Develop backend services and data infrastructure using Python.- Implement DAG-based workflow orchestration using Airflow, Prefect, or Dagster.- Work with AWS services including Lambda, S3, ECS/EKS, ...
India, Pune
MapOut Digital Solutions Inc
Rs 2.5 - 4.5 lakhs p.a.
... dbt, Apache Airflow, Fivetran, AWS, Git, and Looker Strong SQL development skills with experience in query optimization and performance tuning Experience in data ingestion techniques using custom pipelines or SaaS tools like Fivetran Expertise in data modeling and optimization of existing or new data models Experience ...
India, Bengaluru
... at least 2 years in a technical leadership role overseeing data engineering or data platform teams. - Strong proficiency in big data technologies, including Apache Spark/PySpark, Apache Airflow and Apache Kafka. - Experience with programming languages such as Scala and Python for developing robust data pipelines. - In-depth ...
India, Bengaluru
Talent Aspire
Rs 5 - 11 lakhs p.a.
... At least 2 years in data analysis and mining techniques to aid in descriptive and diagnostic insights Extensive hands-on experience with Postgres, Redshift, Airflow, MongoDB, and real-time data workflows. Technical Skills : Expertise in Python, SQL, and PySpark Strong background in software architecture and scalable development ...
India
varun malhotra designs
... position sizing, max daily loss limits, and 'kill-switch' functionality. Backtesting & Logging: Create a framework to test strategies against historical NSE data and maintain detailed logs for audit and debugging. Required Skills & Qualifications Language Proficiency: Strong expertise in Python (preferred for its libraries ...
India, Delhi
synapse xtl
... maintain Python applications for data extraction, transformation, automation, and reporting. Build and optimize data pipelines and ETL processes to handle large datasets from multiple sources. Develop reusable Python modules to streamline analytics and reporting workflows. Integrate APIs, databases, and external data sources ...
India, Indore
... using Terraform, Kubernetes, and AWS services such as RDS, MSK, and EC2. Deploy applications into AWS environments. Develop and maintain scripts and tools using Python to automate common tasks. Deploy and manage PostgreSQL and other databases to ensure high availability and data consistency. Deploy and manage Kafka clusters ...
India, Chennai
Vehant Technologies
Rs 5 - 8 lakhs p.a.
... or equivalent with a strong software architecture mindset and adept in documentation of work Strong understanding and experience in real-time process control, data flow and multi-threaded process control architecture over a heterogeneous compute hardware architecture involving CPU and GPU and networked computers. Software ...
India, Bengaluru
Orcapod Consulting Services Private Limited
... expertise in designing and deploying complex Generative AI systems, with a deep understanding of LLM integration and prompt engineering.- Proven ability to architect and scale microservices-based applications using modern cloud-native patterns on AWS.- Strong proficiency in building and managing REST Web APIs, ensuring ...
India, Pune
Orcapod Consulting Services Private Limited
... expertise in designing and deploying complex Generative AI systems, with a deep understanding of LLM integration and prompt engineering.- Proven ability to architect and scale microservices-based applications using modern cloud-native patterns on AWS.- Strong proficiency in building and managing REST Web APIs, ensuring ...
India, Bengaluru