Warning: This page has not been updated in over over a year and may be outdated or deprecated.
videos:oai-pmh_server_and_harvest_functionality
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
videos:oai-pmh_server_and_harvest_functionality [2023/04/24 21:58] – [Transcript] crhallberg | videos:oai-pmh_server_and_harvest_functionality [2023/04/26 13:34] (current) – crhallberg | ||
---|---|---|---|
Line 1: | Line 1: | ||
====== Video 6: OAI-PMH Server and Harvest Functionality ====== | ====== Video 6: OAI-PMH Server and Harvest Functionality ====== | ||
- | The sixth VuFind | + | The sixth VuFind® |
Video is available as an [[https:// | Video is available as an [[https:// | ||
Line 13: | Line 13: | ||
===== Transcript ===== | ===== Transcript ===== | ||
- | // This is a raw machine-generated transcript; it has been partially cleaned up, but more work needs to be done on the later parts of the text. // | + | Hello and welcome to this VuFind tutorial video, in which I am going to talk about how VuFind uses the OAI-PMH protocol to both share and receive records. |
- | Hello and welcome to this VuFind | + | OAI-PMH |
- | tutorial video, in which I am going to | + | |
- | talk about how VuFind uses the | + | |
- | OAI-PMH protocol | + | |
- | receive | + | |
- | OAI-PMH | + | First of all I am going to show you how you can turn on VuFind' |
- | archives initiative protocol for | + | |
- | metadata harvesting | + | |
- | supported and widely used method of | + | |
- | sharing xml metadata between systems. | + | |
- | It supports not just harvesting entire | + | |
- | collections of metadata but also doing | + | |
- | incremental harvests so you can get only | + | |
- | things | + | |
- | prior harvest, and it can also address | + | |
- | deleted records | + | |
- | has been removed from an upstream system. | + | |
- | The protocol always supports Dublin core | + | |
- | metadata but it also can support any | + | |
- | kind of XML format as well. The server | + | |
- | and client are both able to deal with | + | |
- | the same standard. | + | |
- | First of all I am | + | That' |
- | going to show you how you can turn on | + | |
- | VuFind' | + | |
- | to the command line and I'm going to | + | |
- | edit my local config.ini file and | + | |
- | you'll see that | + | |
- | in the default configuration that comes | + | |
- | with VuFind the entire [OAI] section is | + | |
- | commented out, so by deleting this | + | |
- | semicolon | + | |
- | header I have now activated my OAI-PMH | + | |
- | server. | + | |
- | That's all I need to do to turn on | + | There are also some settings related |
- | the basic functionality but there are a | + | |
- | few things here that I would probably | + | |
- | also want to do like give the name, and | + | |
- | you can set a separate administrative | + | |
- | email for your OAI server | + | |
- | it will use the default email address. | + | |
- | There are also some settings related | + | There is another important step though that you have to take before |
- | OAI servers can divide a collection into | + | |
- | specific sets. You can use a Solr field like a | + | |
- | facet for defining sets or you can | + | |
- | specify particular named sets with | + | |
- | particular queries associated with them | + | |
- | if you want to allow people | + | |
- | specific subsets | + | |
- | if you just leave all this stuff | + | |
- | commented out then set functionality | + | |
- | will be disabled and people will only be | + | |
- | able to harvest | + | |
- | There is another important step though | + | To turn this on you just need to uncomment a couple of lines in the default marc_local.properties file, so I'm going to bring that up. This is the same file that we've worked on. You can see here near the top there are two lines, first_indexed and last_indexed, |
- | that you have to take before you can use | + | |
- | OAI-PMH server capabilities | + | |
- | and that is to turn on record | + | |
- | change tracking | + | |
- | protocol needs to know the history | + | |
- | when everything in your system was | + | |
- | created or changed so that it can do | + | |
- | incremental updates. VuFind needs to track | + | |
- | more information at index time so that | + | |
- | the server has the information | + | |
- | needs. By default, VuFind does not | + | |
- | track record | + | |
- | doing so makes the index process slower, | + | |
- | but if you do turn this on you not only | + | |
- | get the benefit | + | |
- | OAI-PMH server but you also gain access | + | |
- | to some other functionality that | + | |
- | otherwise won't work including RSS feeds | + | |
- | that are sorted based on actual | + | |
- | record | + | |
- | use Solr-based new record searching where you can | + | |
- | actually limit your search by how | + | |
- | recently records were added to the index. | + | |
- | To turn this on you just need to | + | Of course, simply making a change |
- | uncomment a couple of lines in the | + | |
- | default | + | |
- | I'm going to bring that up. This is | + | |
- | the same file that we've worked on. | + | |
- | You can see here near the top there are | + | |
- | two lines, first_indexed | + | |
- | and just by uncommenting these I turn on | + | |
- | change tracking. The difference between | + | |
- | these two fields is that the first_indexed | + | |
- | field will contain the date of | + | |
- | the first time a particular record ID | + | |
- | was indexed into the system and the last_indexed | + | |
- | date will contain the most | + | |
- | recent time that record changed, so when | + | |
- | you index a record for the first time | + | |
- | first_indexed and last_indexed will be set, | + | |
- | but if that record gets revised over | + | |
- | time last_indexed will change to reflect | + | |
- | those changes but first_indexed will | + | |
- | always stay the same so you know the age | + | |
- | of the overall record as well as the | + | |
- | date of its most recent change and this | + | |
- | is sort of the minimum amount of | + | |
- | information needed to implement OAI-PMH. | + | |
- | Of course, simply making a change | + | Of course |
- | marc_local.properties file is not enough. I | + | |
- | also index all of my records and just in | + | |
- | keeping with past demos I'm going to | + | |
- | index 3 of the sample MARC record files | + | |
- | included with VuFind: journals.mrc, | + | |
- | geo.mrc and authoritybibs.mrc. | + | |
- | Of course I've showed you how to turn on | + | If you go to your VuFind URL with /oai on the end of it you will get to a convenient page that shows you all of the verbs supported by the OAI-PMH protocol. It lets you test them out on your instance, so for example, the most simple thing you can do is just say " |
- | change tracking for MARC records. At some | + | |
- | point in the future we'll also index XML. | + | |
- | When we get that far you can also turn on change | + | |
- | tracking there, it's just done in a | + | |
- | different way. For now we've got our | + | |
- | index updated the way we need it to be. | + | |
- | We have the OAI server functionality | + | |
- | turned on in config.ini, so I'm going to | + | |
- | switch over to a web browser and show | + | |
- | you how this works. | + | |
- | + | ||
- | If you go to your | + | |
- | VuFind URL with /oai on the end | + | |
- | of it you will get to a convenient page | + | |
- | that shows you all of the verbs | + | |
- | supported by the OAI-PMH protocol. It lets | + | |
- | you test them out on your instance, so | + | |
- | for example, the most simple thing you | + | |
- | can do is just say " | + | |
- | dump out basic information about the | + | |
- | server and as you can see that the " | + | |
- | repo" repository name I put into config.ini | + | |
- | comes through here. | + | |
Of course, much more interesting is finding out what kind of metadata formats are supported by an OAI-PMH server. As I mentioned before, they always support Dublin Core but different formats may be supported by different servers so in a view case I'm just going to give one of the records and the index and find out what formats are supported. | Of course, much more interesting is finding out what kind of metadata formats are supported by an OAI-PMH server. As I mentioned before, they always support Dublin Core but different formats may be supported by different servers so in a view case I'm just going to give one of the records and the index and find out what formats are supported. | ||
Line 180: | Line 56: | ||
So, that's all I wanted to show this month. This will become much more interesting when we talk about ingesting XML because you can harvest with OAI and then load a whole directory of records into VuFind. We will look at that next time. In the meantime, I also just wanted to quickly mention that if you want to do this OAI-PMH harvesting without having to install all of VuFind, it has actually been split out into a separate project called VuFind Harvest. So, you can just check out VuFind Harvest and run a simplified version of the script without having to carry the whole way to VuFind around with you. And, I will include a link to that project in the notes with the video. That's all for now. Thank you, and I will provide more information next month. | So, that's all I wanted to show this month. This will become much more interesting when we talk about ingesting XML because you can harvest with OAI and then load a whole directory of records into VuFind. We will look at that next time. In the meantime, I also just wanted to quickly mention that if you want to do this OAI-PMH harvesting without having to install all of VuFind, it has actually been split out into a separate project called VuFind Harvest. So, you can just check out VuFind Harvest and run a simplified version of the script without having to carry the whole way to VuFind around with you. And, I will include a link to that project in the notes with the video. That's all for now. Thank you, and I will provide more information next month. | ||
+ | |||
+ | //This is an edited version of an automated transcript. Apologies for any errors.// | ||
---- struct data ---- | ---- struct data ---- | ||
properties.Page Owner : | properties.Page Owner : | ||
---- | ---- | ||
videos/oai-pmh_server_and_harvest_functionality.1682373515.txt.gz · Last modified: 2023/04/24 21:58 by crhallberg