Streaming Platform Engineer - Confluent Kafka Platform

About the Role

Seeking a seasoned Streaming Platform Engineer based in Dubai, UAE with over 6 years of professional experience, specializing in administration and production support of the Confluent Kafka Platform.

Key Responsibilities

• Oversee Kafka cluster performance, availability, and capacity, implementing scaling and tuning as needed.
• Manage installations, upgrades, patches, and version control of the Confluent Kafka platform as well as Apache Flink.
• Administer Kafka components including topics, partitions, consumer groups, and data retention policies.
• Ensure Kafka clusters maintain high availability, disaster recovery, and fault tolerance standards.
• Implement security measures such as ACLs, authentication, and encryption protocols.
• Diagnose and resolve issues involving producers, consumers, and brokers.
• Work collaboratively with development and DevOps teams to maintain streaming data pipelines.
• Document configurations, operational procedures, and best practices thoroughly.

Qualifications and Experience

• Bachelor's or Master's degree in Computer Science, Information Technology, Computer Engineering, Telecommunication Engineering, or related field.
• At least 4 years of hands-on experience managing the Confluent Kafka Platform, with 3-4 years focused on platform administration.
• Strong understanding of Kafka ecosystem components including brokers, topics, partitions, consumer groups, and connectors.
• Experience designing, developing, and operating Kafka-based data pipelines, including multiple source Confluent Connectors.
• Knowledge of distributed systems and platform performance tuning.
• Familiarity with cloud computing environments such as AWS or Azure is advantageous.
• Strong scripting skills with Ansible, Bash, and Python related to platform administration.
• Experience or exposure of approximately 1 year with Apache Flink installation and upgrades is preferred.
• Proven problem-solving capability and aptitude to troubleshoot complex distributed systems.

Additional Information

Candidates must have direct, robust experience with the outlined responsibilities; applications lacking this will not be considered. The position is based in Dubai and requires onsite presence. Please note this is a full-time role with an emphasis on strong technical and operational expertise in streaming platform technology management.