Vacancy description
Avisoft
Noida, भारत
Job Overview:We are looking for a Data Engineering Manager with 10 - 15 years of experience to lead data engineering initiatives, build scalable data pipelines, manage data lake architecture, and enable real-time analytics and AI use cases. The role will work closely with application, ML, analytics, and business teams while mentoring data engineers and driving modern engineering https://jobeax.com/link/TtkK7BHPbancvnFq Responsibilities:- Lead real-time pipelines using Kafka/MSK, Debezium CDC, and Apache Pinot.- Design and maintain Java/Python-based frameworks for ingestion, validation, transformation, and replication.- Design and manage data lakes using Apache Iceberg, AWS Glue, Athena, and S3.- Define data modeling standards across Pinot, Redshift, and warehouse layers.- Lead orchestration and workflow platforms using Apache Airflow and Superset.- Drive cloud-native deployments using ECS/EKS, Docker, Terraform/CDK, and CI/CD.- Support ML, analytics, product, and AI initiatives with reliable data solutions.- Ensure data quality, observability, lineage, governance, and security across the data stack.- Mentor data engineers and promote modern architecture and engineering best https://jobeax.com/link/wJIgo45kO3tINlTo Skills:- Real-Time Data: Kafka/MSK, Debezium, Event-Driven Architecture, CDC- Data Platforms: Apache Pinot, Redshift, RocksDB/NoSQL, Apache Iceberg, Athena, AWS Glue- Programming: Java, Python- Orchestration: Apache Airflow, Superset- AWS: S3, IAM, Lambda, Glue, ECS/EKS, CloudWatch- Data Architecture: Data Lakes, Streaming & Batch Processing, Data Modeling, Parquet, Avro- DevOps: Docker, Terraform/CDK, CI/CD, GitHub ActionsGood to Have:- dbt, Trino/Presto, ClickHouse, or Druid.- Data security, encryption at rest/in transit, GDPR/PCI compliance.- CloudFormation and advanced DevOps practices.- Experience with scalable cloud-native data architectures. (ref:hirist.tech)