Table of Contents
MARC Export Notes
Exporting records from the ILS system is key for the use of VuFind. To make this process as easy as possible for you, we have started consolidating the documentation on how to do this for various known ILS systems.
If you have experience working with a particular system, please consider taking a moment to add notes that might help others get started more quickly. If you need help that is not available on this page, feel free to ask on one of the mailing lists listed on the support page.
Voyager includes a marcexport program designed for exporting MARC data. Details of its use can be found in the “Bulk Export of MARC Records” chapter of the Voyager Technical documentation, which you can download from SupportWeb (login required).
Additionally, some open source alternatives to marcexport are listed at Michael Doran's page.
You may also want to take a look at the Automation page for sample scripts using marcexport.
Loading MFHDs with Bibs
Voyager offers the option of exporting holdings information along with bibliographic information. This can be useful for setting up location-based faceting, but it requires some extra configuration to achieve.
After adding the appropriate flag to your marcexport call to include the MFHDs, you need to do a few things on the VuFind side:
1. Tell SolrMarc to combine the bibs and MFHDs by adding these lines to your import.properties file (found in the import subdirectory of either your VuFind home or local settings directory, depending on your version of VuFind):
marc.combine_records = 852 marc.combine_records.left_field = 001 marc.combine_records.right_field = 004
More details on the exact meanings of the various marc.combine_records settings can be found in the SolrMarc documentation.
NOTE: SolrMarc expects holdings records to immediately follow matching bibliographic records. If you have holdings records shared by multiple bibliographic records (a rare situation caused when items are bound together but cataloged separately) you may have some problems during import.
2. Tell SolrMarc to index your locations to an appropriate field and map Voyager's internal codes using a translation map by adding this line to marc_local.properties (found in the same place as import.properties):
collection = 852b, locations.properties(locations)
Note: “collection” is just one of several possible fields that you might want to use for indexing locations; “building” and “institution” might also be appropriate. The best option depends on your institutional needs.
3. Create the translation map file referenced in step 2 – this goes in the translation_maps subdirectory of your import directory and will need to be customized to your institution's needs; sample lines below:
locations.MAIN = Main Stacks locations.MICRO = Micro Collection locations.MUSIC = Music CDs
- choose data to export via p-ret-01
- if you don’t export everything, you need to run p-ret-10 afterwards (for example if you don’t want to export records which items are lost/missing/canceled = unavailable)
- export data via p-print-03
- since Aleph v20 you can export directly to MARC binary mode (XML; necessary for UTF8)
- in earlier versions you need a tool to convert the data (for example: yaz for SLES 11 http://software.opensuse.org/search?q=yaz&baseproject=SUSE%3ASLE-11%3ASP1&lang=de&exclude_debug=true)
- export options (were recommended to me):
- Fix-Routine: MABUS
MABUS fix_doc_mab FILE=tab_fix_mabaseq MABUS fix_doc_convit FILE=tab_fix_convit_copy_fld MABUS fix_doc_convit FILE=tab_fix_mab2usm
It is probable that the MABUS routine in tab_fix is superflous for MARC-Libraries (this is an example for those who work with MAB-Format).
- Expand-Routine: PRINT-REC
PRINT-REC expand_doc_mab PRINT-REC expand_doc_bib_aut_700 PRINT-REC expand_doc_index_from_z403 PRINT-REC expand_doc_bib_z403 CONF=expand_doc_bib_z403,TAG=991 PRINT-REC expand_doc_bib_z30 CONF=expand_doc_bib_z30_vufind,TAG=852 PRINT-REC expand_doc_bib_avail
If you use MARC-format, the tab_expand of the PRINT-REC the folowing configurations may not be relevant to you.
PRINT-REC expand_doc_mab PRINT-REC expand_doc_bib_aut_700
- If you use ADAM (catalogue enrichment), you need to configure „z403“ (no information received, since we don’t use it)
- Character Conversion: none
Further Information I received:
- Some libraries use the PST, AVA, OWN, DEL, and STA fields for the export
- To update the index you can use the field z07p to determine the items to export and insert into the index (for example new items)
- RESTful API:
Notes from Václav Rosecký:
For export process we use procedure print-03. If you need incremental update, you must combine print-03 with ret-01. Procedure ret-01 returns modified system numbers and you can use this list as the input for print-03. Export format is ISO 2709 MARC and you should not need any additional data processing in SolrMarc if you have one bibliographic base. If you have two or more bibliographic bases, you must modify index scripts in vufind to change record ID (we have two bibliographic bases, so our id in vufind is base + sysno separated by '-', eg. MZK03-000001479).
Before exporting your records, review your export tables to make sure that you are exporting all of the fields that you want and none of the fields that you don’t want. In particular, make sure that item and order record information containing potentially confidential data is not exported. Manual page 101563 has information about how to view export tables; to change your export tables, you must submit a service request.
If you want to export 970 fields containing table of contents information from a third-party vendor such as Backstage Library Works, you must get your vendor to contact Innovative and give permission before Innovative will change your export table to include 970 fields.
Data Exchange (see manual page 106000) is the easiest way to export the records used for the initial load. Although the file of records that is created will have an .out extension, it is really in .mrc format; it is safe to simply change the extension.
A script for daily exports of new items is available on the IUG Clearinghouse under the title “Create List of New Items automated.”
- Shrew - Tools to fully automate the export of bibliographic and item records from your Innovative Interfaces ILS; see also the Sierra version (though both of these code bases have been archived and may no longer be fully functional).
Koha has a cgi-bin/tools/export.pl file which allows you to export to MARC (or xml) format.
To run that manually, you will need to set the path for PERL5LIB (or whatever your perl lib variable is), and set the variable that points to the Koha configuration file. They will look something like:
export PERL5LIB=/usr/share/koha/lib:$PERL5LIB export KOHA_CONF=/etc/koha/sites/library/koha-conf.xml
To do the export, you will want to do something like this (the paths may differ)
/usr/share/koha/bin/export_records.pl --format=xml --filename="koha.xml"
Alternatively, you can use OAI-PMH or the web interface to export records; see Exporting Records from Koha for details.
Once you have exported the koha records, you will need to import the records into VuFind.
Copy your marc_local.properties from vufind/install to the vufind/local/install directory. make the changes to this file that you need to
#collection = "" institution = "My University" #building = "Library A" id = 999c, first
Koha needs to use the id = 999c for Vufind to properly tie the two together. The id= field is not in every version of the marc_local.properties file, so you may need to add that manually.
Now you are ready to run the import.
See the How to index Koha with VuFind page for a full end-to-end example.
See this page for details.
Note: These comments were written in reference to VuFind 1.0RC1 and Horizon 7.3.4 but should apply to other versions of both packages as well.
Horizon provides two methods for exporting MARC records: either from the Horizon client itself or using a command line utility. Both methods make use of an export target that needs to be set up before exporting can take place. To verify that appropriate export targets exist, launch the Horizon client and navigate to Cataloging → Export → Batch Bib Export, where there may already be targets listed that should be examined to see if they're suitable for creating MARC records that can be imported into VuFind. You may decide it's easier just to create new export targets if the existing ones are complex or should not be edited.
To create an export target that doesn't include holdings data, navigate to Batch Bib Export and in the “Code Lookup: Export Target” dialog box, choose “Add”, which will open a new “Edit: Export Target” dialog box. In the “Target” field enter an arbitrary name such “noitem” and then a description such as “Don't export item information”. All remaining fields may be left blank; click “Close” and answer “Yes” to save the new target.
Now that an export target has been created, the actual MARC records can be exported using either the Horizon client or from the command line as mentioned earlier. To export from the Horizon client, navigate to Cataloging → Export → Batch Bib Export and choose the export target you want to use. A dialog box for entering a SQL statement appears. An example statement might be “select bib# from bib_control where bib# >= 1000 and bib# < 2000”. Click OK and choose where the export file should be saved.
Exporting MARC records from the command line requires a utility called MarcOut.exe, available from SirsiDynix at the Client Care website (login required). Make sure to download the one that matches your version of Horizon. A list of switches for the utility is available by typing “marcout /?” on the command line. An example for specifying the mandatory export target as well as a range of bib#s might be:
marcout /sDatabaseServer /uDatabaseUser /pDatabasePassword /dDatabaseName /mexported.mrc /b1000 /e1999 /xnoitem /y
When checking item status, VuFind queries your Horizon database's item table using the bib#s associated with the imported marc records. That means you have to make sure that the MARC records you import have a tag somewhere containing the bib#. If your bibliographic records have no MARC tag containing the bib#, a new tag can be created during the export process by using MarcOut's “/q” switch. For example, to place the bib# in the 999a tag, append “/q999” to your MarcOut command line. Regardless of whether you already have the bib# in a MARC tag or need to create it during the export process, you have to make sure that the tag containing the bib# is correct in VuFind's import/marc.properties file before importing. In the example of using the 999a tag, it would require changing the line “id = 001, first” to “id = 999a”.
Notes currently unavailable – please help us fill in the documentation!