How to control AI Assist’s data ingestion using an XML sitemap

The instructions below are for desktops and laptops only.

You can use an XML sitemap to control AI Assist’s website data ingestion. Using a sitemap, you can choose which articles, FAQs, or other content should be included.

This prevents irrelevant or outdated content from being included in your AI agent’s responses, leading to more useful interactions and higher customer satisfaction.

In this guide, we cover the basics of an XML sitemap and how to upload it to AI Assist’s data source.

An XML sitemap is a file that lists your website’s pages, making it easier for web crawlers to crawl your site effectively.

It’s written in Extensible Markup Language (XML) and provides the URL of each page along with other information, such as last modified date and frequency of updates.

Here’s an example of a sitemap:

Here’s what the tags mean:

• <loc> (compulsory)
  This tag contains the page’s URL. AI Assist uses this to crawl specific pages on your website.
  
  

• <lastmod> (optional)
  When the page was last updated.
  
  

• <changefreq> (optional)
  How often the page’s content is updated (e.g., daily, weekly, monthly).
  
  

• <priority> (optional)
  The page’s importance (0.0 to 1.0, with 1.0 being the highest).
  
  

See this guide to learn more about XML sitemaps and tags.

Using this example, you can prepare an XML sitemap for AI Assist to crawl the essential pages of your website.

1. Once you’ve created a sitemap, save it as an XML file.
   For example: sitemap.xml
   
   

2. Upload the XML file to a public directory on your website.
   For example: https://www.mysite.com/folder/sitemap.xml

Next, let’s see how to add your sitemap to AI Assist’s data source.

1. Go to the Website tab on your AI Agent’s Data Source page.

3. Enter the direct link to your sitemap, and click Add.

AI Assist will then crawl and gather content from the URLs in your sitemap. This process typically takes a few hours, but larger sitemaps may take longer.

After ingestion, use the built-in test environment to ensure AI Assist retrieves information correctly and provides accurate responses.

Best practices

• Maintain the sitemap
  Update it regularly to include new or changed content. We recommend reviewing your sitemap at least once a month, especially if your website changes frequently.
  
  

• Exclude irrelevant pages
  Don’t include duplicate or unnecessary information.
  
  

• Validate the sitemap
  Use tools like Google’s XML Sitemap Validator to find errors.

Following these steps will help ensure that AI Assist gets only the most relevant information from your website.

If you have feedback about this article, or if you need more help:

• Click the green live chat icon

• Schedule a call with us

• Visit our community