Staff Software Engineer - Search Platform

About the Role

The Search team at Databricks plays a pivotal role in developing AI and ML-driven products that empower users to efficiently discover valuable assets such as tables, notebooks, dashboards, data pipelines, SQL queries, and machine learning models. Our ongoing mission is to refine and scale our search capabilities, focusing on elevating search ranking, enhancing query understanding, and expanding asset coverage.

Key Responsibilities

• Lead the design, development, and deployment of machine learning models and relevant systems that enhance search relevancy within Databricks' product ecosystem.
• Develop automated pipelines leveraging ML and NLP techniques for preprocessing data, query parsing and rewriting, retrieval, ranking, and evaluation to facilitate fast experimentation and iterations.
• Partner closely with product management and interdisciplinary teams to initiate technology-driven projects that unlock new business approaches and shape the roadmap for search and discovery features.
• Create robust evaluation frameworks for testing search ranking improvements in both offline and live environments.

Candidate Profile

• Educational background: Bachelor’s degree in Computer Science or a related discipline, with a preference for advanced degrees (Master’s or PhD).
• Extensive experience: Over a decade working on large-scale search relevance systems, either in production environments or impactful research settings.
• Proven expertise applying large language models (LLM) to enhance search relevance.
• Strong experience in one or multiple domains including query understanding, natural language processing (NLP), text mining, recommendation engines, personalized content delivery, discovery tools, and conversational AI.
• Solid foundation in computer science principles and practices.
• Active contribution to prominent, widely-used open-source projects is highly valued.

Why Join Databricks?

Join us in pioneering advanced AI solutions that redefine user engagement with data. Work alongside a world-class team of AI/ML professionals and influence the future direction of AI-powered products using cutting-edge technologies.

Company Overview

Databricks is a leader in Data and AI technology, trusted by over 20,000 organizations globally, including many Fortune 500 companies. The company offers a unified platform featuring products such as Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, and Unity Catalog.

Benefits and Inclusion

Databricks provides comprehensive benefits tailored to support our diverse workforce. We are committed to fostering an inclusive environment where all individuals can thrive. We practice equal employment opportunity without discrimination based on any protected categories.

Compliance Note

Positions requiring access to export-controlled technology or source code may be subject to U.S. government licensing decisions, and this may affect application progression.