sitemap.xml so crawlers find every pageA sitemap is a plain XML file listing every URL you want crawled. AI and search crawlers don't always find pages by following links — a sitemap hands them the complete list directly, so nothing important gets missed. It's the cheapest way to make sure every page you've written is discoverable.
Put this at https://yourdomain.com/sitemap.xml and list one
<url> per page:
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url><loc>https://yourdomain.com/</loc></url>
<url><loc>https://yourdomain.com/about</loc></url>
<url><loc>https://yourdomain.com/blog/first-post</loc></url>
</urlset>
Add one line to robots.txt so crawlers find the sitemap automatically:
Sitemap: https://yourdomain.com/sitemap.xml
Most platforms generate a sitemap for you: Next.js, Astro, Hugo and WordPress
(via Yoast or Rank Math) all output sitemap.xml automatically — just
enable it. For a static site, a build-time generator keeps it in sync as you add
pages, so it never goes stale.