- The search crawler lets you implement federated search without developer resources. You can set up multiple crawlers in your help center to crawl and index different content in the same or different websites.
- The Federated Search API is available if you have developer resources. This REST API that lets you ingest records of your external content into the Zendesk search indexes to implement federated search in your help center.
After you set up federated search, you need to configure the content that you want to include in help center search results (see Including external content in your help center search results).
Setting up the search crawler for federated search for your help center
The search crawler lets you implement federated search in your help center without developer resources. You can set up multiple crawlers in your help center to crawl and index different content in the same or different websites.
To set up the search crawler for federated search
-
In Knowledge admin, click Settings (
) in the sidebar.
- Click Search settings.
- Under Crawlers, click Manage.
- Click Add Crawler.
- In Name this crawler, enter the following:
- Name that you want to assign to the crawler. This is an internal name that identifies your search crawler on the crawler management list.
-
Owner who is the Knowledge admin responsible for crawler
maintenance and troubleshooting. By default, the crawler owner is
the user creating the crawler, however you can change this name to
any Knowledge admin.
Crawler owners receive email notifications both when the crawler runs successfully and when there are error notifications, such as problems with domain verification, processing the sitemap, or crawling pages.
- In Add the website you want to crawl, configure the following:
- Website URL - Enter the URL of the website that you want to crawl.
- I confirm that I have permission to crawl this website - Read the information under this checkbox, and then select to confirm you have permission to crawl this website.
- In Add a sitemap, in Sitemap URL, enter the URL for the
sitemap you want the crawler to use when crawling your site.
The sitemap must follow the sitemaps XML protocol and contain a list of all pages within the site that you want to crawl. The sitemap can be the standard sitemap containing all the pages of the site or it can be a dedicated sitemap that lists the pages that you want to crawl. All sitemaps must be hosted on the domain that the crawler is configured to crawl. The search crawler does not support sitemap indexes.
You can set up multiple crawlers on the same site that each use different sitemaps defining the pages you want the search crawler to crawl.
- In Add filters to help people find this content, configure the source and type filters used to
filter search results by your end users. Source refers to the origin of the
external content, such as a forum, issue tracker, or learning management
system. Type refers to the kind of content, such as blog post, tech note, or
bug report.
- Source - Click the arrow, then select a source from the list or select + Create new source to add a name that describes where this content lives.
- Type - Click the arrow, then select a type from the list or select + Create new type to add a name that describes what kind of content this is.
- Click Finish. The search crawler is created and pending. Within 24 hours the crawler will verify ownership of the domain and then fetch and parse the specified sitemap. Once the sitemap processing succeeds, the crawler begins to crawl the pages and index its content. If the crawler fails either during domain verification or while processing the sitemap, the crawler owner will receive an email notification with troubleshooting tips to help resolve the issue. The crawler will try again in 24 hours.Note: Zendesk/External-Content is the user agent for the search crawler. To prevent the crawler from failing due to a firewall blocking requests, whitelist (or allow-list) Zendesk/External-Content.
After you set up the search crawler, you need to select the content that you want to include and exclude in help center search results. See Including external content in your help center search results.
You can also include external content in search results in the knowledge section of the context panel for agents. See Configuring Knowledge in the context panel.
Using the API to configure federated search for your help center
Your developers can set up federated search in your help center using the Federated Search API. This method requires that your developers build and maintain a middleware layer to integrate the service or site that hosts the external content and the help center.
- Build your own integration with the Zendesk REST API then ingest the content you want to show up in your search results. See the Federated Search API reference documentation.
After you set up federated search, you need to select the content that you want to include and exclude in help center search results. See Including external content in your help center search results.
You can also include external content in search results in the knowledge section of the context panel for agents. See Configuring Knowledge in the context panel.