About Features Downloads Getting Started Documentation Events Support GitHub

Site Tools


indexing:dspace

How to index DSpace with VuFind

1. Make sure OAI-PMH is turned on and properly indexed in DSpace

To retrieve records from DSpace, you will need to use the OAI-PMH protocol.

In newer versions of DSpace, OAI-PMH should be enabled by default; however, in DSpace 3.x and earlier, some additional configuration was needed.

You can check whether the service is enabled by visiting {DSpace Base URL}/oai in your web browser. If the service is turned on, you should see something similar to this:

Default Context. This is the default context of the DSpace OAI-PMH data provider. Identify. List Sets. List Metadata Formats. List Identifiers. List Records.

Before harvesting data from DSpace to VuFind, you should check that DSpace's OAI (Open Archives Initiative) Solr indexing is working correctly. For this, in the above screen click the List Records button, and you should see something similar to:

 List of Records. Results fetched: 1. Identifier oai:localhost:123456789/3. Last Modified 2020-09-12 08:49:34. Sets. Metadata.

In the above image, if you click Metadata, it should provide you with an oai_dc-formatted metadata response.

Fixing Missing Indexing

If metadata does not display correctly through the server (for example, if you receive an “Error No matches for the query” message), this is probably a sign that DSpace's OAI Solr index has not been updated correctly. DSpace's Solr does not index metadata automatically when it is imported through a manual submission process or batch imported using SIP or AIP. To overcome this limitation, the Data Provider Repository (DPR) administrator must run a process from the command line to correctly update the index. First, switch to the “bin” subdirectory of your DSpace installation, then run:

./dspace oai import -o -c

The meaning of parameters –o and –c is as follows:

  • -o Optimize index after indexing
  • -c Clears the Solr index before indexing (it will import all items again)

2. Import records into VuFind using OAI-PMH harvest

These steps use VuFind's OAI-PMH harvest tool and XML indexing tool. You can follow the links to learn more about the tools.

  1. Modify $VUFIND_LOCAL_DIR/harvest/oai.ini
    [DSpace]
    url=http://yourdspacehostname/oai/request
    metadataPrefix=oai_dc
    idSearch[]="/^oai:yourdspacehostname:/"
    idReplace[]="ir-"
    idSearch[]="/\//"
    idReplace[]="-"
    injectDate="datestamp"
    injectId="identifier"
    dateGranularity=auto
    harvestedIdLog=harvest.log
  2. Run these commands:
    cd $VUFIND_HOME/harvest
    php harvest_oai.php
    ./batch-import-xsl.sh DSpace dspace.properties

Troubleshooting

If you receive an error message during harvesting, you may need to rebuild the OAI indexes on your DSpace server. Log into that system and run these commands:

<path to dspace directory>/bin/dspace oai clean-cache
<path to dspace directory>/bin/dspace oai import -c

After that processing completes, retry the harvest process on the VuFind server as described above.

3. Customize Import Rules (optional)

If you wish to customize the way your records are ingested, see the indexing XML page for details. The instructions above use the example dspace.properties and dspace.xsl files that ship with VuFind. You can copy these into appropriate subdirectories of your $VUFIND_LOCAL_DIR directory and modify them as needed to change the way data is indexed.

:!: If you change import rules, note that you will need to remove your $VUFIND_LOCAL_DIR/harvest/DSpace directory, re-harvest the records, and repeat the indexing process in step 2 above.

4. Customize Record Display (optional)

By default, VuFind does not include any DSpace-specific display logic; records indexed from DSpace are displayed using the standard “SolrDefault” record driver and templates. However, the default import setup marks DSpace records with a record_format value of “dspace” which means that you can create a custom record driver named SolrDspace in order to create custom DSpace-only display options. See displaying a custom field for some examples of record display customization.

indexing/dspace.txt · Last modified: 2020/09/21 15:38 by demiankatz