Differences

This shows you the differences between two versions of the page.

--- administration:search_engine_optimization [2021/04/16 10:51] – [Sitemaps] emaijala
+++ administration:search_engine_optimization [2024/05/08 09:00] – Add link to Tracking Record Changes page mtrojan
@@ Line 48: / Line 48: @@
 A VuFind site can easily have hundreds of thousands, or even millions, of pages in its sitemaps -- it's all dependent on the number of records in your index. Search engines will take a significant amount of time to crawl all of these pages, and even more time to detect changes. Be aware that publishing a sitemap will not instantly lead to full visibility of all of your content.
+Note that the effect of your settings on crawling budgets is dependent on each indexer. e.g. [[https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap|Google]] will ignore the **changefreq** information completely and prefer the **lastmod** information instead if present, which will be available in VuFind 10.X if [[indexing:tracking_record_changes|Tracking Record Changes]] is enabled.
+If you want to adjust the frequency for other bots, it is recommended to rather reduce this setting if you have a high amount of records, else the indexers might take months to crawl all of your pages and might waste the crawl budget by downloading the same sites over and over again even if there are no changes at all.
 === Using Search Engine Tools ===
@@ Line 63: / Line 67: @@
 [[administration:robots.txt]] - Recommendations for restricting search engine access to avoid confusing results and/or unnecessary server load.
+===== Related Video =====
+You can learn more about sitemaps through the [[videos:sitemaps_and_web_indexing|Sitemaps and Web Indexing]] video.
 ---- struct data ----
+properties.Page Owner :
 ----