Machine Learning Engineer – Computer Vision

About Xantory

Xantory develops a vertical-farming control platform managing every aspect of crop cycles, from climate regulation to irrigation and dosing, with data capture from cloud to rack-level controllers. While the system records extensive sensor data and grow stages, it currently lacks visual plant analysis capabilities.

Role Overview

The Machine Learning Engineer – Computer Vision, as the team’s first ML engineer, will create vision-based monitoring by implementing cameras, building models to interpret crop visual data, and integrating these insights into alerts, records, and control mechanisms. The position focuses on real-world production deployment with live crops and involves combining new visual data with existing sensor and yield datasets.

Key Responsibilities

• Design the imaging architecture for racks and trays, including selecting cameras, mounting, lighting compatible with grow lights, scheduling captures, and defining annotation standards to ensure dataset reliability.
• Create the farm’s first labeled image datasets directly from on-site crops through data collection efforts.
• Develop, train, and deploy machine learning models aimed at plant detection, counting, growth stage identification, canopy assessment, stress and disease detection, and readiness for harvest.
• Select appropriate model approaches such as classification, detection, segmentation, and anomaly detection, rigorously validating them against new crop and rack data.
• Implement edge inference solutions on devices like Raspberry Pi and NVIDIA Jetson, optimizing for balance between accuracy, latency, and hardware constraints, alongside server-level deployments.
• Correlate computer vision results with time-series sensor data (temperature, humidity, CO₂, PPFD, pH, EC, flow, actuator logs, stage boundaries, and yields) to evaluate growth outcomes relative to recipes.
• Analyze and compare growth cycle durations and yields per rack to improve recipe effectiveness based on empirical data.
• Integrate machine learning models seamlessly within the existing platform infrastructure, including technologies such as Rust, PostgreSQL, Redpanda, and Kubernetes, with results displayed in the Sentinel interface.
• Maintain version control over datasets, experiments, and models; continuously monitor model performance and retrain to address degradation based on production feedback.
• Define and manage data interfaces between the ML models and backend, edge, and frontend engineering teams ensuring smooth collaboration.

Qualifications & Experience

• At least 3 years of practical experience in computer vision using frameworks like PyTorch or TensorFlow, including ownership of vision systems from data collection through operational deployment.
• Experience constructing image datasets from scratch with attention to quality control, class balance, and managing small or evolving datasets.
• Hands-on knowledge of camera systems and video processing pipelines, including managing exposure, color fidelity under artificial grow lighting, and camera calibration.
• Familiarity with horticulture, controlled environment agriculture, plant phenotyping, or industrial quality inspection is a plus.
• Experience with multispectral or near-infrared (NIR) imaging is advantageous.

Skills & Competencies

• Strong foundation in Python programming and machine learning principles, including robust validation procedures to prevent data leakage.
• Comfortable working within a production engineering setup using source control, code reviews, testing, containerization, and collaborative services.
• Experience with edge inference technologies such as ONNX, TensorRT, and deployments on NVIDIA Jetson or Raspberry Pi platforms is preferred.
• Knowledge of MLOps tools including experiment tracking, model registries, and monitoring frameworks is beneficial.
• Familiarity with Rust or Go programming languages, MQTT protocol, and Kubernetes deployment environments is an advantage.
• Fluent communication skills in English, capable of explaining model limitations to growers and technical details to backend engineers.

Performance Metrics

• Number of vision models successfully deployed in production on active racks and speed from dataset creation to first deployment.
• Model accuracy in detection, counting, and growth stage recognition validated against unseen crops and environments.
• Efficiency and reliability of edge inference on devices in terms of latency, uptime, and resource consumption.
• Quality and comprehensiveness of labeled datasets covering various crop types and growth phases with consistent annotations.
• Ability to identify and address model performance degradation promptly through retraining cycles informed by production data.
• Impact on farming operations, including adoption of alerts and records by growers and improvements in yield or cycle length based on recipe insights.

Work Location

This role requires on-site presence in Dubai with regular engagement inside the farm environment.