How to control AI Assist’s data ingestion using an XML sitemap

The instructions below are for desktops and laptops only.

AI Assist can learn about your products and services by crawling and gathering information from your website. To ensure it accesses only the most relevant and up-to-date content, you can guide its data ingestion using an XML sitemap.

By specifying which pages AI Assist should crawl, you can prevent outdated or irrelevant information from influencing its responses, leading to more accurate and helpful interactions with your customers.

It’s written in Extensible Markup Language (XML) and provides the URL of each page along with other information, such as the last modified date and frequency of updates.

Here’s an example of a sitemap:

Here’s what the key tags mean:

• <loc> (required)
  Specifies the page’s URL. AI Assist uses this to identify which pages to crawl.
  
  

• <lastmod> (optional)
  Indicates when the page was last updated.
  
  

• <changefreq> (optional)
  Suggests how often the content changes (e.g., daily, weekly).
  
  

• <priority> (optional)
  Represents the page’s importance on a scale from 0.0 to 1.0.
  
  

Note: AI Assist does not refer to <changefreq> and <priority> when ingesting your website data.

For more details on XML sitemaps and tags, see this guide: Understanding XML Sitemaps

Using this example, you can prepare an XML sitemap for AI Assist to crawl the essential pages of your website.

1. Once you’ve created a sitemap, save it as an XML file.
   For example: sitemap.xml
   
   

2. Upload the XML file to a public directory on your website.
   For example: https://www.mysite.com/folder/sitemap.xml

Next, let’s see how to add your sitemap to AI Assist’s data source.

1. Go to the Website tab on your AI Agent’s Data Source page.

3. Enter the direct link to your sitemap, and click Add.

AI Assist will begin crawling the URLs listed in your sitemap. The process may take a few hours, depending on the number of pages.

After ingestion, use the built-in test environment in your AI Assist settings to ensure it retrieves information correctly and provides accurate responses.

Best practices

• Maintain the sitemap
  Regularly update your sitemap to reflect new or modified content. Review it at least once a month, especially if your website changes frequently.
  
  

• Exclude irrelevant pages
  Avoid including duplicate or unnecessary information that could confuse AI Assist.
  
  

• Validate the sitemap
  Use tools like Google’s XML Sitemap Validator to check for errors.

By following these steps, you ensure that AI Assist accesses only the most relevant and up-to-date information from your website, enhancing its ability to assist your customers effectively.

If you have feedback about this article, or if you need more help:

• Click the green live chat icon

• Schedule a call with us

• Visit our community