<div class="prose prose-hyva max-w-none"><p>Which content automatically ends up in the generated sitemap, which doesn't, and how custom module content like our Tutorial section can be added afterwards.</p><p>Out of the box, Magento ships with exactly four "providers" that together determine the content of every generated <em>sitemap.xml</em>: the store root URL itself, categories, products, and CMS pages. Each provider is its own PHP class (a so-called "item provider") that gets asked, on every generation run: "which URLs should go into the sitemap for this store?" Anything that doesn't belong to these four types – such as a fully custom module like our Tutorial section – stays invisible to the sitemap until someone registers a custom provider for it.</p>
<h3>Categories: all or nothing</h3>
<p>Every category with <strong>Enable Category = Yes</strong> automatically ends up in the sitemap – Magento has NO individual toggle to "exclude this one category from the sitemap". Even <strong>Include in Menu = No</strong> (hiding a category from navigation) has zero effect on sitemap inclusion – the two settings are completely independent. If you really want to keep a specific category out of the sitemap, your only practical options are <strong>Enable Category = No</strong> (which also makes it unreachable in the store itself) or a targeted exclusion in <code>robots.txt</code>.</p>
<h3>Products: visible AND enabled</h3>
<p>Products are filtered on two criteria: <strong>Enable Product = Yes</strong> (status) AND <strong>Visibility</strong> other than "Not Visible Individually" (i.e. "Catalog", "Search", or "Catalog, Search"). A disabled product never shows up; an enabled product set to "Not Visible Individually" (e.g. a single variation of a configurable product) doesn't either – only the parent, visible main product gets listed. Here too, there's no dedicated "include in sitemap" checkbox on the product itself.</p>
<h3>CMS pages</h3>
<p>Active CMS pages (<strong>Enable Page = Yes</strong>) get included automatically – the same all-or-nothing logic as categories.</p>
<h3>Setting priority and change frequency</h3>
<p>Under <em>Stores > Configuration > Catalog > XML Sitemap</em>, you set <strong>Frequency</strong> (how often the content typically changes – purely a hint to search engines, not a technical guarantee) and <strong>Priority</strong> (0.0 to 1.0, relative importance within your own sitemap) SEPARATELY for categories, products, and CMS pages. In the same area, <strong>Product Images Include Policy</strong> controls whether and which product images get added as extra <code><image></code> entries in the sitemap.</p>
<img loading="lazy" src="{{media url=wysiwyg/handbuch/1kapitel/priorit_t6kategorie.png}}" alt="PrioritätundÄnderungshäufigkeit" />
<h3>What does NOT get included automatically</h3>
<p>Any content that runs through its own module and isn't a category, product, or CMS page stays invisible to the sitemap – until a custom item provider gets registered for it. Two concrete examples from this project:</p>
<ul>
<li><strong>Magefan Blog:</strong> already ships its own item provider (registered via its own <code>di.xml</code>) – blog posts and the blog index therefore already appear in the sitemap automatically, without us having to build anything for it.</li>
<li><strong>Our Tutorial section (Mironsoft_Tutorial):</strong> series and chapters under <em>/tutorial/...</em> are a completely custom data structure (own database tables, own router) – until recently, they were therefore entirely absent from the sitemap. We've retrofitted this: a new class (<code>Mironsoft\Tutorial\Model\Sitemap\TutorialItemProvider</code>) registers itself with Magento's sitemap mechanism via <code>di.xml</code> and supplies every active series and chapter URL per store on every generation run. For you as an editor, this means: every new tutorial series and every new chapter automatically shows up in the sitemap on the next sitemap run – with no manual extra step.</li>
</ul>
<h3>Submitting the sitemap to Google</h3>
<p>The generated sitemap alone isn't enough – Google needs to know about it. In <strong>Google Search Console</strong> (requires a free Google account; create and verify a property for your domain) under <em>Sitemaps</em>, enter the URL of your sitemap file (e.g. <code>https://yourdomain.com/sitemap_en.xml</code>). Google then crawls it on its own on a regular basis – resubmitting manually after every sitemap update is NOT necessary.</p>
<p><strong>Practical tip:</strong> after major content additions (like a new Handbuch or Tutorial series), check under <em>Sitemaps</em> in Search Console whether the "discovered URLs" count increased as expected – if it doesn't, that usually points to a generation cron job that hasn't run yet, or an error in the relevant item provider.</p>
<h3>Automatic entry in robots.txt</h3>
<p>Under <em>Stores > Configuration > Catalog > XML Sitemap > Search Engine Submission Settings</em>, setting <strong>Enable Submission to Robots.txt</strong> to "Yes" automatically adds a <code>Sitemap:</code> line to robots.txt once the sitemap has been generated. This helps not only Google (which you've registered via Search Console anyway) but also other search engines like Bing, which primarily discover sitemaps through robots.txt.</p>
<img loading="lazy" src="{{media url=wysiwyg/handbuch/1kapitel/searchenginesubmissions6kategorie.png}}" alt="AutomatischerEintrag" />
<h3>Monitoring scheduled generation</h3>
<p>On the <strong>Generation Settings</strong> tab of a given sitemap, you set <strong>Enabled</strong>, <strong>Start Time</strong>, and <strong>Frequency</strong> (Daily/Weekly/Monthly) for automatic regeneration via cron job. You can additionally configure <strong>Error Email Recipient</strong>, <strong>Error Email Sender</strong>, and <strong>Error Email Template</strong> – if generation fails, you get notified instead of only noticing weeks later that the sitemap content is outdated.</p>
<img loading="lazy" src="{{media url=wysiwyg/handbuch/1kapitel/generationsettings6kategorie.png}}" alt="GeplanteGenerierung" />
<h3>Separate sitemaps per store view/language</h3>
<p>Use <strong>Add Sitemap</strong> to create a separate file per store view (e.g. <code>sitemap_de.xml</code> for the German store view and <code>sitemap_en.xml</code> for the English one) – each then contains only the URLs of its respective language version. This matters for multilingual stores, so search engines crawl the correct, language-specific URLs and URLs from a different store view don't accidentally end up in the wrong sitemap.</p>
<h3>Size limits and automatic splitting</h3>
<p>The <strong>Maximum No of URLs Per File</strong> and <strong>Maximum File Size</strong> fields (also under the XML sitemap configuration) limit how many URLs and how much data a single sitemap file may contain – per the official sitemap protocol, at most 50,000 URLs or 50 MB per file. If your store exceeds these limits, Magento automatically splits the sitemap into multiple files and additionally generates a parent sitemap index file that references all the individual files – with no extra effort required on your part.</p>
<img loading="lazy" src="{{media url=wysiwyg/handbuch/1kapitel/sitemapfile6kategorie.png}}" alt="Größenlimits" />
</div>