Staff Software Engineer - Machine Learning (Search)

About the Role

The Applied AI team at Databricks is driving the next generation of AI and machine learning-enabled products. Our customers continuously generate diverse assets including tables, notebooks, dashboards, data rooms, pipelines, SQL queries, and ML models, sometimes reaching hundreds of millions in volume. Ensuring users can efficiently locate these assets is crucial to their productivity.

We are searching for a Staff Engineer to lead the enhancement of our Search Quality capabilities. In 2025, our priorities include boosting search result ranking, enriching query understanding, establishing robust evaluation mechanisms, and expanding asset coverage to support seamless search experiences at scale.

Key Responsibilities

• Lead the creation and deployment of machine learning-driven search and relevance models integrated within Databricks' product suite.
• Design and build automated ML and NLP workflows for tasks such as data preprocessing, query interpretation and rewriting, ranking and retrieval, and model assessment to facilitate fast experimentation and iteration.
• Partner closely with product managers and interdisciplinary teams to spearhead technology-driven initiatives that align with strategic business priorities and product development plans related to search and discovery.
• Help establish a dependable framework for assessing improvements in search ranking both offline and in live environments.

Candidate Profile

• Bachelor's degree or higher (master's or PhD preferred) in Computer Science or a related discipline.
• More than 10 years of experience building production-scale search relevance systems or contributing significantly in relevant research roles.
• Hands-on experience utilizing large language models (LLMs) to enhance search relevance.
• Expertise in one or more areas such as query understanding, natural language processing, text mining, recommendation systems, personalization, discovery, or conversational AI.
• Strong foundations in computer science principles.
• History of meaningful contributions to widely-used open-source projects.

Why Join Databricks?

At Databricks, we are pioneering AI solutions that transform how users engage with data and products. You will have a unique opportunity to influence the evolution of AI-powered products, work with state-of-the-art generative models, and collaborate with top AI and ML professionals globally.

If you are passionate about advancing practical AI applications and pushing innovation boundaries, we encourage you to apply.

About Databricks

Databricks is a leading Data and AI company supporting over 20,000 organizations worldwide, including prominent brands like adidas, AT&T, Bayer, Mastercard, and Unilever, as well as 70% of Fortune 500 companies. We provide a unified platform comprising Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, and Unity Catalog. Our headquarters are in San Francisco, with more than 30 offices globally.

Benefits

Databricks offers a comprehensive range of benefits tailored to meet diverse employee needs. Detailed benefit information is available per region.

Diversity and Inclusion

Our company values a diverse and inclusive workplace that enables everyone to thrive. Our hiring processes ensure equal opportunity regardless of age, color, disability, ethnicity, family or marital status, gender identity or expression, language, nationality, physical or mental ability, political beliefs, race, religion, sexual orientation, socio-economic status, veteran status, or other protected traits.

Compliance Note

For roles requiring access to export-controlled technology or source code, the employer retains discretion to seek necessary U.S. government licenses and may choose not to proceed with applications based solely on this factor.