Senior Elasticsearch Observability Engineer

Job Overview

We are seeking a seasoned Senior Elasticsearch Observability Engineer who possesses extensive practical expertise in handling Elasticsearch clusters and observability infrastructures in live production settings. This role primarily emphasizes Elasticsearch management, ensuring cluster scalability, security enforcement, advanced observability, leveraging Elastic ML functionalities, automation procedures, and seamless integration with corporate systems.

Key Responsibilities

• Architect, implement, and maintain Elasticsearch clusters across development, staging, and production environments.
• Devise and apply strategies for cluster scaling that involve shard distribution, sizing of nodes, and managing Index Lifecycle Management (ILM) policies.
• Serve as the main point for technical escalation regarding Elasticsearch matters within various teams.
• Enforce security measures on Elasticsearch environments including hardening practices.
• Oversee Role-Based Access Control (RBAC) and manage TLS certificates alongside associated security setups.
• Set up Service Level Objective (SLO) monitoring systems in Kibana Observability and incorporate SLO violation alerts into incident handling processes.
• Connect observability frameworks with multiple enterprise solutions and prepare alert response documentation to ensure standardized operational practices.
• Deploy and sustain Elastic ML features such as outlier detection, regression, classification, and utilize the Data Frame Analytics API for enhanced analytics.
• Engage with container orchestration tools like Kubernetes, perform Linux/Unix system administration, and apply knowledge of networking basics and storage performance tuning.
• Create automation and operational scripts utilizing Python, Bash, or equivalent scripting languages.

Required Qualifications and Skills

• Over 5 years of proficient experience in administrating Elasticsearch clusters in production environments.
• Deep understanding and experience with Elasticsearch and Kibana Observability suites.
• Proven skills in cluster scalability, shard management, node sizing, and ILM implementation.
• Strong grasp of Elasticsearch security protocols, RBAC, and TLS management.
• Experience using Kubernetes for orchestration.
• Proficiency in Linux/Unix systems administration.
• Fundamental knowledge of networking and storage optimization.
• Competency in scripting languages such as Python and Bash.
• Excellent problem-solving aptitude and capability to handle complex technical escalations.

Preferred Expertise

• Fluency in Arabic is desirable.
• Possession of Elastic Certified Engineer or Elastic Certified Analyst credentials.
• Experience working with Elastic Cloud on Kubernetes (ECK) Operator.
• Familiarity with Oracle Cloud Infrastructure (OCI).
• Understanding of large-scale data pipeline technologies including Apache Kafka, Fluentd, and Vector.
• Experience managing observability and log management systems handling petabyte-scale data.