About semantic search and how it works

Guide has begun to use semantic search as a way to generate the most accurate search results possible based on the intent and context of user search queries. Unlike search methods that find literal matches to keywords, semantic search captures the meaning of search queries, helping end users and agents search for and locate content without prior knowledge of the exact keywords to use.

This article contains the following sections:

• What is semantic search?
• How semantic search works
• How Guide uses semantic search to improve the search experience
• Understanding the rollout plan
• How to check if your help center is enabled for semantic search

What is semantic search?

Semantic search uses artificial intelligence (AI) to process and understand the full meaning and intent of language used in search queries, similar to the way a human would. Specifically, machine learning (ML) and natural language processing (NLP) technologies work together in semantic search to help the search engine understand the user’s intent when they submit a search query. Understanding what the user is actually looking for, regardless of the keywords they use, helps the search engine return and rank the most relevant results.

For example, with semantic search, end users and agents can ask natural language questions instead of worrying about which keywords to use in order to get the best results. New agents can ask “how do I start using Guide?” instead of trying to think of the best keywords to use or the sequence of keywords to use in their query. With semantic search, they can enter their question in the way that makes sense to them, and still see the most relevant articles at the top of the results.

With semantic search, Guide is taking another step towards providing agents and admins a high-deflection help center with an intuitive search experience.

How semantic search works

Guide search has historically relied on keyword matches between queries and content (for example, articles, community posts, or external records). However, it didn’t capture the “meaning,” or semantics, of a query. Semantic search, on the other hand, deploys natural language processing to understand the content of the search query and content. This helps the search engine identify complex patterns that are otherwise missed. In many cases, semantic search is able to capture similarity even when there is not a strong word overlap between the query and indexed content.

Relying on language models, semantic search can translate queries and help center articles into vectors (a numerical representation), and measure the distance between them. Articles closer in the vector space are considered to be more similar by the language model. The language model is trained on many text examples, and through these examples has learned how to accurately interpret the meaning of text accurately.

When semantic search is used, the search engine boosts the relevance of semantically better matches. This pushes the most relevant content, based on intent and context of the search query, to the top of the search results. Each time a search is performed, the search results are ranked not only based on keyword matches but also boosted by semantic matches.

How Guide uses semantic search to improve the search experience

Semantic search offers a more intuitive search experience that lets help center visitors search using their choice of terms. Since Guide has begun to incorporate semantic search, we’ve observed an increase in search quality metrics such as mean reciprocal rank (MRR) and click through rate (CTR). Specifically, we found that MRR improves 7% on average with this first iteration of semantic search for English Zendesk help centers. We’re also seeing more searches with clicks.

By improving the search relevance, and ranking the most relevant results on top, semantic search produces the following help center improvements:

• End users can find the information they are looking for more easily, increasing the deflection power of your help center.
• Agents can be more efficient as they can find answers more quickly.

The degree of impact for semantic search depends on the search behavior of the user. Semantic search has particularly positive benefits for longer search queries, but still improves relevance for all types of search.

Understanding the rollout plan

As we expand our AI model to cover more search channels, we will continue to roll out semantic search into additional areas.

Specifically, the semantic search rollout will be phased into content type, language, and search channel. See the following tables for the current and planned rollout of semantic search based on these parameters.

Rollout by content type

Content type	Coverage status
Articles	Covered
Community posts	Covered
External content	Planned

Rollout by help center search type

Help center search type	Coverage status
Two-pane search	Covered
Unified search	Covered
Federated search	Planned

Rollout by search channel

Search channel	Coverage status
Help center search engine results page (SERP)	Covered
Instant search	Planned
Request from article suggestions	Planned
Unified search API	Planned
Articles search API	Will not be supported
Posts search API	Will not be supported
Web Widget help center search	Planned
Knowledge in Agent Workspace	Planned

Rollout by language

Language	Coverage status
Arabic	Planned
Bulgarian	Not planned
Catalan	Not planned
Croatian	Not planned
Czech	Planned
Danish	Planned
Dutch	Planned
English (all variants)	Covered
Finnish	Not planned
French (all variants)	Planned
German	Planned
Greek	Not planned
Hebrew	Not planned
Hindi	Not planned
Hungarian	Not planned
Icelandic	Not planned
Indonesian	Not planned
Italian	Planned
Japanese	Planned
Korean	Planned
Latvian	Not planned
Lithuanian	Not planned
Norwegian	Planned
Persian	Not planned
Polish	Planned
Portuguese (Brazil)	Planned
Romanian	Planned
Russian	Planned
Serbian	Not planned
Simplified Chinese	Planned
Slovak	Not planned
Slovenian	Not planned
Spanish	Planned
Swedish	Planned
Thai	Not planned
Traditional Chinese	Not planned
Turkish	Planned
Ukrainian	Not planned
Urdu	Not planned
Vietnamese	Not planned

Note: Some languages may not be possible to support with semantic search if there is not sufficient data available to train the model.

How to check if your help center is enabled for semantic search

We’ve begun our semantic search implementation for Guide customers. As the rollout continues, we will continue to expand the feature to all Guide customers. You can use this procedure to determine whether your help center is currently enabled for semantic search.

To check whether your help center is enabled for semantic search

1. Perform a search in your help center.
2. On the search results page, open Inspect element in your browser.

   For example, in Chrome, right-click the search results page, select Inspect from the menu, then click the Elements tab.

   View full size

3. Select the <head> tag, then examine the guide_search/semantic_search_enabled element.

   If the element is set to "false" (as shown in the image below), semantic search is not yet enabled in your help center. If the element is set to "true," then semantic search is enabled.