Warning: This page has not been updated in over over a year and may be outdated or deprecated.
indexing:dspace
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
importing_records:how_to_index_dspace_with_vufind [2015/12/14 16:31] – [How to index DSpace with VuFind] demiankatz | indexing:dspace [2023/06/15 11:04] – [2. Import records into VuFind using OAI-PMH harvest] demiankatz | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== How to index DSpace with VuFind ====== | ====== How to index DSpace with VuFind ====== | ||
- | These are the instructions used by the Naval Postgraduate School | + | ===== 1. Make sure OAI-PMH is turned on and properly indexed |
- | :!: ** These instructions were written for VuFind 1.x; VuFind 2.x includes a DSpace | + | To retrieve records from DSpace, |
- | ===== Steps ===== | + | In newer versions of DSpace, OAI-PMH should be enabled by default; however, in DSpace 3.x and earlier, some [[indexing: |
- | OAI must be enabled | + | You can check whether the service is enabled |
- | - Modify the DSpace server config in **nginx.conf** on the DSpace server:<code>Location /oai/ { | + | |
- | Proxy_set_header X-Forwarded-Host $host; | + | |
- | Proxy_set_header X-Forwarded-Server $host; | + | |
- | Proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; | + | |
- | + | ||
- | Proxy_pass http:// | + | |
- | Proxy_redirect http:// | + | |
- | Proxy_buffering off; | + | {{ : |
- | Proxy_store off; | + | |
- | Proxy_connect_timeout 120; | + | Before harvesting data from DSpace |
- | Proxy_send_timeout 120; | + | |
- | Proxy_read_timeout 120; | + | |
- | } | + | |
- | </ | + | |
- | - Modify the **server.xml** for the appropriate DSpace Tomcat instance | + | |
- | Reloadable=" | + | |
- | allowLinking=" | + | |
- | - Modify | + | |
- | harvest.includerestricted.oai = true | + | |
- | harvester.autoStart = true | + | |
- | ...</ | + | |
- | Then you may proceed to import the OAI feed into VuFind: | + | {{ :indexing:dspace-records.jpg? |
- | - Modify **$VUFIND_HOME/ | + | |
- | - Modify **$VUFIND_HOME/ | + | |
- | | + | |
- | - Modify **$VUFIND_HOME/ | + | |
- | - cd $VUFIND_HOME/harvest | + | |
- | - php harvest_oai.php | + | |
- | | + | |
- | - ../vufind.sh restart | + | |
- | ===== Required Files ===== | + | In the above image, if you click Metadata, it should provide you with an oai_dc-formatted metadata response. |
- | ==== oai.ini ==== | + | ==== Fixing Missing Indexing ==== |
- | < | + | |
- | [DSpace] | + | If metadata does not display correctly through the server (for example, if you receive an "Error No matches for the query" message), this is probably a sign that DSpace' |
+ | |||
+ | <code bash> | ||
+ | ./ | ||
+ | </ | ||
+ | |||
+ | The meaning of parameters –o and –c is as follows: | ||
+ | |||
+ | * -o Optimize index after indexing | ||
+ | * -c Clears the Solr index before indexing (it will import all items again) | ||
+ | |||
+ | ===== 2. Import records into VuFind using OAI-PMH harvest ===== | ||
+ | |||
+ | These steps use VuFind' | ||
+ | |||
+ | - Modify **$VUFIND_LOCAL_DIR/ | ||
url=http:// | url=http:// | ||
metadataPrefix=oai_dc | metadataPrefix=oai_dc | ||
Line 57: | Line 45: | ||
dateGranularity=auto | dateGranularity=auto | ||
harvestedIdLog=harvest.log | harvestedIdLog=harvest.log | ||
- | </ | + | combineRecords=true</ |
+ | - Run these commands:< | ||
+ | php harvest_oai.php | ||
+ | ./ | ||
+ | |||
+ | ==== Troubleshooting ==== | ||
+ | If you receive an error message during harvesting, you may need to rebuild the OAI indexes on your DSpace server. Log into that system and run these commands: | ||
- | ==== dspace.properties ==== | ||
< | < | ||
- | [General] | + | <path to dspace directory>/ |
- | xslt = dspace.xsl | + | <path to dspace |
- | custom_class[] = VuFind | + | |
- | + | ||
- | [Parameters] | + | |
- | institution = " | + | |
- | collection = " | + | |
</ | </ | ||
- | ==== dspace.xsl ==== | + | After that processing completes, retry the harvest process |
- | < | + | |
- | <!-- available fields are defined in solr/ | + | |
- | < | + | |
- | xmlns: | + | |
- | xmlns: | + | |
- | xmlns: | + | |
- | xmlns: | + | |
- | xmlns: | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | <!-- ID --> | + | |
- | <!-- Important: This relies | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | <!-- RECORDTYPE --> | + | ===== 3. Customize Import Rules (optional) ===== |
- | <field name=" | + | |
- | <!-- FULLRECORD --> | + | If you wish to customize the way your records are ingested, see the [[indexing:xml|indexing XML]] page for details. The instructions above use the example [[https://github.com/ |
- | <!-- disabled for now; records are so large that they cause memory problems! | + | |
- | <field name=" | + | |
- | <xsl:copy-of select=" | + | |
- | </field> | + | |
- | | + | |
- | <!-- ALLFIELDS --> | + | :!: If you change import rules, note that you will need to remove your $VUFIND_LOCAL_DIR/harvest/DSpace directory, re-harvest the records, and repeat the indexing process |
- | <field name=" | + | |
- | <xsl:value-of select=" | + | |
- | </ | + | |
- | + | ||
- | <!-- INSTITUTION --> | + | |
- | <field name=" | + | |
- | <xsl:value-of select=" | + | |
- | </ | + | |
- | + | ||
- | <!-- COLLECTION --> | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | + | ||
- | <!-- LANGUAGE --> | + | |
- | <xsl:if test="// | + | |
- | < | + | |
- | <xsl:if test=" | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | <!-- FORMAT --> | + | |
- | <!-- populating the format field with dc.type instead, see TYPE below. | + | |
- | | + | |
- | in addition to the dynamic ones extracted from the record. | + | |
- | <field name=" | + | |
- | --> | + | |
- | + | ||
- | <!-- SUBJECT --> | + | |
- | <xsl:if test="// | + | |
- | <xsl:for-each select="// | + | |
- | <xsl:if test=" | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | <!-- DESCRIPTION --> | + | |
- | <xsl:if test="// | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | <!-- ADVISOR / CONTRIBUTOR --> | + | |
- | <xsl:if test="// | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | <!-- TYPE --> | + | |
- | <xsl:if test="// | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | <!-- AUTHOR --> | + | |
- | <xsl:if test="// | + | |
- | < | + | |
- | <xsl:if test=" | + | |
- | <!-- author is not a multi-valued field, so we'll put | + | |
- | first value there and subsequent values | + | |
- | | + | |
- | <xsl:if test=" | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | <xsl:if test=" | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | <!-- TITLE --> | + | |
- | <xsl:if test="// | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | <!-- PUBLISHER --> | + | |
- | <xsl:if test="// | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | <!-- PUBLISHDATE --> | + | |
- | <xsl:if test="// | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | <field name=" | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | <!-- URL --> | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | < | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | </ | + | |
- | + | ||
- | ==== DspaceRecord.php ==== | + | |
- | < | + | |
- | <?php | + | |
- | require_once ' | + | |
- | class DspaceRecord extends IndexRecord { | + | |
- | | + | |
- | global $interface; | + | |
- | $template = parent :: getSearchResult (); | + | |
- | $interface -> assign (' | + | |
- | $interface -> assign (' | + | |
- | $interface -> assign (' | + | |
- | $interface -> assign (' | + | |
- | return $template; | + | |
- | } | + | |
- | } | + | |
- | ?> | + | |
- | </ | + | |
+ | ===== 4. Customize Record Display (optional) ===== | ||
+ | By default, VuFind does not include any DSpace-specific display logic; records indexed from DSpace are displayed using the standard " | ||
---- struct data ---- | ---- struct data ---- | ||
+ | properties.Page Owner : | ||
---- | ---- | ||
indexing/dspace.txt · Last modified: 2024/08/28 13:35 by demiankatz