Platform Engineer - Kafka/ClickHouse in Bangalore, भारत

Core Responsibility Summary:Deploys, operates, and evolves the Kafka - Flink - ClickHouse platform on Kubernetes across multiple factory data https://jobeax.com/link/OL1ucwj5Tz3jE3fQ the platform is reliable, observable, performant, and ready for data engineering teams to build on top https://jobeax.com/link/V2x8g0JTJoGLl1fz & Container Platform:- Deep Kubernetes expertise (workload types, scheduling, resource requests/limits, namespaces, RBAC).- Kubernetes operators for stateful systems - Strimzi (Kafka), Flink Kubernetes Operator, ClickHouse Operator (Altinity).- Helm chart authoring and management (not just consumption - writing and maintaining charts).- StatefulSet management, persistent volume claims, storage class configuration.- Multi-cluster and multi-site Kubernetes deployments.- Node affinity, taints/tolerations, pod disruption budgets for workload isolation.- Kubernetes networking: services, ingress, network policies, DNS.- Secrets management (Kubernetes secrets, Vault integration, or equivalent).Platform Component Expertise :Kafka :- Kafka cluster sizing, broker configuration, topic design (partitions, replication, retention).- Strimzi operator - KafkaNodePool, KafkaTopic, KafkaUser, KafkaMirrorMaker2 CRDs.- Kafka Connect cluster deployment and connector lifecycle management.- Kafka security: TLS, mTLS, SASL, ACLs.- Kafka upgrade and rolling restart procedures with zero data https://jobeax.com/link/jXQDzjpxxeErRaIo :- Flink cluster deployment modes on Kubernetes (Application mode, Session mode).- Flink Kubernetes Operator - FlinkDeployment, FlinkSessionJob CRDs.- Checkpointing and savepoint management (S3 or distributed storage backends).- Flink HA configuration (Kubernetes-native HA or ZooKeeper).- Job submission, monitoring, and restarts without data https://jobeax.com/link/ywQ3GirDBjtFAW17 :- ClickHouse cluster deployment with Altinity operator or ClickHouse Operator.- Shard and replica topology design for multi-factory deployments.- Storage configuration: tiered storage, disk policies, S3-backed cold storage.- Backup and restore strategies.- ClickHouse Keeper vs ZooKeeper for https://jobeax.com/link/5YkiqVUeKzzV2H61 & Monitoring :- Full observability stack deployment: Prometheus, Grafana, Alertmanager.- Kafka metrics: broker JMX metrics, consumer lag (Burrow or Cruise Control), under-replicated partitions.- Flink metrics: checkpoint duration, backpressure, restart frequency, throughput/latency per operator.- ClickHouse metrics: query latency, merge backlog, insert rate, replication lag, memory/disk pressure.- Kubernetes cluster metrics: node utilization, pod restarts, OOMKill events, PVC utilization.- SLO/SLI definition and alerting for platform uptime and data freshness.- Distributed tracing for pipeline end-to-end latency visibility.- Log aggregation (EFK stack, Loki, or equivalent) for platform component https://jobeax.com/link/TWkBSj8aIsq01ji8 Testing & Capacity Planning:- Load testing frameworks for streaming platforms (Gatling, k6, custom Kafka producer harnesses).- Designing load test scenarios that simulate factory production volumes and burst conditions.- Benchmarking Kafka throughput (MB/s, msg/s) under varying partition counts and consumer configurations.- Flink job stress testing - measuring backpressure onset, checkpoint interval degradation under load.- ClickHouse ingestion benchmarking (insert throughput, concurrent query performance under load).- Capacity modeling: translating factory data volumes and growth projections into infrastructure sizing.- Breaking point analysis - identifying bottlenecks before production traffic hits https://jobeax.com/link/zmcg1CeumRDpk77e & Multi-Tenancy:- Namespace-level isolation for multiple factory deployments on shared Kubernetes clusters.- Kafka topic and consumer group isolation strategies across factory sites and teams.- Resource quotas and LimitRanges to prevent noisy-neighbor problems between factory workloads.- Network policy enforcement to isolate factory data plane traffic.- Separate Kafka clusters vs shared clusters with namespace isolation - trade-off analysis and implementation.- Per-factory ClickHouse cluster deployment vs shared cluster with database/user https://jobeax.com/link/HBZa0TOVmto8WmS0 & GitOps for Platform:- GitOps-driven platform deployment: ArgoCD or Flux for Kubernetes manifest management.- Helm chart versioning and promotion across environments (dev - staging - factory-prod).- Operator upgrade management and CRD migration procedures.- Automated smoke tests post-deployment to validate platform health.- Change management for platform updates in production factory https://jobeax.com/link/k2cL8sNJZafkhSDj & Compliance:- mTLS and certificate management (cert-manager on Kubernetes).- Kafka ACL management at scale.- Network segmentation between factory OT and IT data flows.- Image scanning and supply chain security for platform container images.- Audit logging for platform access and configuration changes.AI-Assisted Development:- Using AI coding assistants (GitHub Copilot, Cursor, or equivalent) to accelerate Helm chart authoring, operator configuration, and runbook generation.- AI-assisted troubleshooting - using LLMs to diagnose Kubernetes events, Flink exceptions, and Kafka error logs faster.- Generating observability dashboards and alert rules with AI assistance.- Critical review of AI-generated infrastructure code before applying to production. (ref:hirist.tech)