Kafka Platform Engineer

About Wissen Technology

Wissen Technology is a specialized consulting firm offering advanced solutions to leading companies in Banking & Finance, Telecom, and Healthcare sectors. Since its establishment in 2015 as part of the Wissen Group, it has expanded significantly with offices worldwide including the US, UK, India, Australia, Mexico, and Canada. The company employs over 1700 professionals and is recognized for rapid growth, top AI/ML vendor status, and delivering high-value projects to Fortune 500 firms.

Key Responsibilities

• Oversee the full lifecycle of Kafka clusters including provisioning, tuning configurations, performing upgrades, and decommissioning old clusters.
• Deploy and maintain Kafka installations on Kubernetes using Confluent for Kubernetes (CFK).
• Continuously monitor cluster performance indicators such as consumer lag, broker status, and topic throughput, and establish alerting protocols.
• Design and manage disaster recovery mechanisms as well as replication strategies across multiple regions.
• Implement and manage Kafka security measures like access control lists (ACLs) and conduct periodic access reviews.
• Diagnose and resolve issues related to performance bottlenecks, lagging consumers, and broker failures.

Required Skills and Experience

• A minimum of 8 years’ experience in messaging or streaming platform engineering with at least 5 years focused specifically on Apache Kafka platform engineering.
• Hands-on experience administering self-managed Kafka clusters in KRaft mode on-premises.
• Expertise in Confluent for Kubernetes (operator handling, custom resource definitions).
• Proficiency with Kafka Connect including managing connector lifecycle, single message transforms (SMTs), and error handling mechanisms.
• Strong understanding of Confluent Schema Registry and managing schema evolution and compatibility.
• Solid knowledge of Kafka security protocols such as TLS/mTLS, SASL authentication, and ACL configuration and troubleshooting.
• Programming skills in Java or Scala, with Python used for scripting and tooling support.
• Experience with monitoring and observability tools including Prometheus, Grafana, and OpenTelemetry.

Desirable Skills

• Familiarity with Apache Flink for advanced event processing alongside Kafka Streams.
• Experience building change data capture (CDC) pipelines into Kafka.
• Operational knowledge of Mirror Maker 2 or Confluent Replicator in multi-data-center environments.
• Understanding of event-driven architecture concepts and data mesh principles.
• Ability to develop and maintain infrastructure-as-code for Kafka platform resources.
• Integrating Kafka platform modifications with continuous integration/continuous deployment (CI/CD) pipelines targeting topics and schemas.

Eligibility and Education

Candidates must hold a B.Tech or B.E. degree in any discipline.