For certain types of functionality, VuFind® needs to be aware not only of the content of the records in its index, but also the times at which these records were first indexed and most recently changed. By understanding the history of its own records, VuFind® can provide RSS feeds that show recent changes, and it can export its contents using the popular OAI-PMH protocol.

Keeping track of the information needed to support these features adds some overhead – more data in your MySQL database, and slower indexing times. Because not everyone needs the features, they are disabled by default. This page explains how to activate them when you need them.

The most important thing that you need to do in order to enable VuFind®'s change tracking is to edit the local MARC mappings file and uncomment the lines that fill in the “first_indexed” and “last_indexed” fields of your Solr index. This usually consists of:

1. Copy import/marc_local.properties into the import subdirectory of your local_settings_directory (usually $VUFIND_HOME/local/import), if you don't already have a local copy of marc_local.properties.

2. Uncomment the *_indexed lines in the local copy of marc_local.properties. The example lines assume that your record IDs come from the 001 field; if you use IDs in a different place, change 001 to an appropriate pattern (e.g. 999c for Koha).

If you are indexing non-MARC records, some change tracking facilities also exist in the XSLT index tool. Depending on your data source, you may need to customize your XSLT and/or properties file to implement support.

After making the necessary configuration changes, you will need to reindex all of your records. This will take a long time, as it will be generating a database so that VuFind® can correctly populate these fields in the future. “first_indexed” contains the time that the record was first indexed by VuFind®, while “last_indexed” contains the time that the record was most recently changed. For consistency and appropriate OAI-PMH behavior, it is STRONGLY RECOMMENDED that all times be stored in UTC; VuFind®'s provided tools will already do this, but if you build your own custom indexer, you should conform to this standard. VuFind® stores information in its database so that even if you upgrade Solr and destroy your entire index, it will still be able to rebuild the correct “first_indexed” and “last_indexed” values. In the future, when you reindex existing records, the process should be faster than the first time around, since for records that have not actually changed, VuFind® will not have to do much extra database work.

SolrMarc may require more memory in order to update the database and the Solr index at the same time. If you get error messages about running out of heap space during the index process, try editing import-marc.sh or import-marc.bat (depending on your platform) to change the Java memory settings. See the performance page for more details on Java memory tuning.

See the OAI-PMH Server page for details.

When record change tracking is enabled, RSS feeds will be more effective. Without tracking, RSS dates are based on publication dates, which may or may not correspond with the dates that items were added to your collection. With change tracking turned on, RSS feeds will be sorted by index date and will display index dates within the feed. This means that patrons can truly track searches in order to be notified of new additions within their chosen areas of interest.

Note that you can customize some of the RSS behavior in VuFind® – see the [RSS] section of searches.ini.

By default, VuFind® tries to use the ILS driver to power the “new items” search. However, not all ILS drivers support this functionality. If you turn on record change tracking, you can modify the method setting to “solr” in the [NewItem] section of searches.ini to gain new item searches without an ILS dependency.

This section contains answers to frequently asked questions about the technical details of the change-tracking implementation.

No, the change tracking system will only update dates in its internal database if it encounters a record ID it has never seen before, or if the record itself has changed. If you index the same MARC file over and over again, that record will only be recorded in the database once. However, if you edit the MARC record and export a new version, when you index the changed record, VuFind® will update the “last indexed” value to reflect that something is different.

This behavior is dependent on records containing last-changed dates. In the case of MARC records, VuFind® checks the 005 and 008 fields for modification dates. Note that if no dates are found, VuFind® will always assume that the record has changed as a precaution; thus, it is important to ensure that your ILS is including one of these fields in exported records if you want reliable change tracking. In the case of XSLT records, the custom PHP methods used by the XSLT expect a modification date parameter, and you should be sure to extract it from an appropriate place in your XML to ensure consistent behavior.

If you remove records from your ILS, you should inform VuFind® about the deleted records so it can update its change tracking database. This enables the system to correctly report deleted records through OAI-PMH, and it also allows the system to treat records as new if they later are added again after being previously deleted. You can use the util/deletes command line utility for this purpose.

For systems that have no easy way of reporting deleted records, a common solution is to simply delete VuFind®'s index, and then reindex from a fresh full export. If you do this, the deleted records will be removed from VuFind®'s index correctly, so normal search behavior will work as expected. However, the change tracking system will not be aware that records have been removed, and the now-missing records will not be reported as deleted through OAI-PMH (or any other feature that might rely on delete tracking functionality). This may or may not be a problem for you, depending on your use case.