About Features Downloads Getting Started Documentation Events Support GitHub

Love VuFind®? Consider becoming a financial supporter. Your support helps build a better VuFind®!

Site Tools


Warning: This page has not been updated in over over a year and may be outdated or deprecated.
development:howtos:connecting_a_new_external_data_source

This is an old revision of the document!


Connecting a New External Data Source

VuFind includes support for several external services in addition to the core Solr index. However, it may sometimes be necessary to build support for a new service. This page will help take you through the process.

Initial Decisions

There are a few things to decide before implementing any code.

  • What URLs do you want to use for searches and record views? This will affect the names of the controllers you create. (i.e. http://localhost/vufind/WorldCat/Search, http://localhost/vufind/WorldCatRecord/12345)
  • What “source” value do you want VuFind to use for identifying records from your new source? This value will be saved in the database and passed in some URLs. It should be short, unique and alphanumeric (i.e. “WorldCat”, “Summon”).
  • What name do you want to use for your family of search classes; this should almost always be the same as your “source” value.
  • What name do you want to use for your record driver? Again, this should probably be consistent with the “source” value.

Step-by-Step Building Instructions

If you haven't already, you may want to read the Data Model / Key Concepts page for background information on how VuFind's pieces fit together.

Once you are ready to write code, you can follow these steps to get a new module added to VuFind. Obviously, this is just a recommendation – some of these steps can be taken out of order without any problems.

Note that for examples below, we will assume that you have chosen Source, search class and record driver names of “Sample.” Just replace “Sample” with your actual choice.

  1. Build/Find Connection Library - Before you can integrate a new service into VuFind, you need to be able to talk to it from within PHP. It's possible that the service provides its own library, which may save you some time. Otherwise, you'll have to build something. If you can solve your service-specific connection problems prior to VuFind integration, then you won't have as many different problems to debug during integration.
  2. Implement Record Driver - The record driver object is responsible for extracting key pieces of data about a record from the data returned by your chosen service. To create it, follow these steps:
    1. In the library/VF/RecordDriver directory, create a file with an appropriate name containing the appropriate class – i.e. Sample.php containing a class called VF_RecordDriver_Sample.
    2. Decide which existing record driver class to extend. If the records you are working with are very distinctive, you will probably want to extend VF_RecordDriver_Base and build custom templates. However, in many cases, records will contain bibliographic data that is similar enough to VuFind's default Solr records to allow you to extend VF_RecordDriver_SolrDefault (or one of its subclasses) and thus recycle many of VuFind's default built-in templates. This approach is recommended whenever it is possible.
    3. Implement key methods:
      • Constructor - The constructor is fed data directly from the external service and is responsible for storing it within the object. This data will be used by all other methods to retrieve specific details. The nature of the data being passed in is determined by your search results class (see below); you can use whatever is most convenient for your situation. The constructor should also set the $this→resourceSource property to the source value you chose earlier (i.e. 'Sample') and, if applicable, the $this→recordIni property to an .ini file containing record-related settings (if you want to take advantage of related record modules).
      • getUniqueId() - This method returns the record's unique identifier. All VuFind records must have a unique identifier of some sort. Note that this unique ID is combined with the “source” value when VuFind does lookups, so you do not have to worry about IDs from one service colliding with IDs from another service.
      • getBreadcrumb() - This method returns a string used for identifying the record in breadcrumbs, page titles, etc. Normally it will be a short form title or something similar.
      • getRecordRoute() - This returns the name of the Zend Router route used to build URLs for accessing individual records. You will define this route in a later step. You can use any name you like, but the standard format is 'samplerecord'.
      • …and the rest - If you are extending VF_RecordDriver_SolrDefault and your saved data is not structured exactly like a Solr response, you should override the public get methods of that class to return information from your chosen format. (If the data is unavailable for a given method, you don't have to override it – the base code will handle missing data appropriately). If you are extending VF_RecordDriver_Base, you should add public get methods for any data elements you will need to display in your templates. When possible, try to name your methods consistently with other record drivers (see the Record Driver Method Master List) since this will improve your chances of being able to reuse certain VuFind components.
      • In most cases, you may want to adjust other methods dealing with things like tabs to display in record view or supported export formats, but these details can probably wait until after the first phase of development.
  3. Implement Search Classes - Create a new directory under library/VF/Search named for the search class ID you picked earlier. In this new directory, you will create three files:
    1. Options.php - This will contain the VF_Search_Sample_Options class, which must extend VF_Search_Base_Options. See the existing VF_Search_Solr_Options or VF_Search_WorldCat_Options classes for some examples of how this works. These are the most important methods to implement in this class:
      • Constructor - The constructor tells the other search classes which .ini files to use for search/facet settings and sets up initial options by reading them in from those .ini files.
      • getSearchAction() - This returns an array of information necessary to access the search action for your new service. By default, the array should contain a 'controller' element matching your chosen controller name (i.e. 'Sample') and an 'action' element of 'Results' (unless you want to rename the search results action in your controller).
      • getAdvancedSearchAction() - This is just like getSearchAction, except it points to the Advanced Search form. By default, this should be an array containing a 'controller' element matching your chosen controller name (i.e. 'Sample') and an 'action' element of 'Advanced'. If you do not wish to support advanced searches, simply return false.
    2. Params.php - This will contain the VF_Search_Sample_Params class, which must extend VF_Search_Base_Params. Unless you need to do special parameter processing or add new parameters not supported by the base class, you are not required to implement any methods here – you can just extend with an empty class. You'll probably end up adding methods here eventually, but for the initial implementation it is nice to leave this empty – one less thing to worry about!
    3. Results.php - This will contain the VF_Search_Sample_Results class, which must extend VF_Search_Base_Results. This is the most important and complex search class, and looking at existing examples like VF_Search_Solr_Results and VF_Search_WorldCat_Results may help you understand it better. You will have to implement several methods:
      • performSearch() - This method reads the search parameters from the parameters and options objects, uses them to perform a search against your chosen service, and uses the results of the search to populate two object properties: $this→resultTotal, the total number of search results found, and $this→results, an array of record driver objects representing the current page of results requested by the user.
      • getFacetList() - This method returns facet options related to the current search. You may need to store extra values in performSearch() to allow the list to be generated. It is possible that getFacetList will be called before a search has been performed – in this case, you should call the performAndProcessSearch method to fill in all the details. (See VF_Search_Solr_Results for an example of this).
      • getRecord() - This static method takes an ID value and returns a record driver representing the requested record. If the record does not exist, it should throw a VF_Exception_RecordMissing object.
      • getRecords() - This static method takes an array of IDs and returns an array of record drivers representing the records. It may throw a VF_Exception_RecordMissing object if appropriate. This method is implemented for you in the VF_Search_Base_Result class, so you do not have to write your own – but if there is an efficient way to request multiple IDs at the same time, writing your own method will improve VuFind's performance; the default base class method simply calls getRecord() over and over from a loop.
  4. Create Controllers - In order to provide web access to your new code, you will need to create two new controllers in the application/controllers folder:
    1. Search Controller - This should have a name like SampleController and should extend VF_Controller_Search. This controller will handle displaying and processing search forms. At a bare minimum, it should contain an init() method which sets the $this→searchClassId property to the appropriate value (i.e. 'Sample') and calls parent::init(). You can look at the VF_Controller_Search class for other available options. You can also define controller actions if you wish, but this is not necessary – by default, you will have Home, Results and Advanced actions inherited from the base class.
    2. Record Controller - This should have a name like SamplerecordController and should extend VF_Controller_Record. This controller will handle displaying records and performing record-related tasks (export, save, etc.). At a bare minimum, it should contain an init() method which sets the $this→searchClassId property to the appropriate value (i.e. 'Sample') and calls parent::init().
  5. Set Up Record Route - You'll need to edit the _initRoutes() method of application/Bootstrap.php to include a route that points to your new record controller. Be sure you use the same name that you used in your record driver's getRecordRoute() method. When creating the route, you can use the existing 'worldcatrecord' as a model and simply replace the WorldCat-specific route and controller names.
  6. Set Up Templates - Now that the models and controllers are set up, you need some views.
    1. Search Templates - Within your chosen theme, create a templates/sample directory (replacing 'sample' with a lowercased version of your search controller's name; you may need to insert dashes if your name includes CamelCase). In this directory, you need to create one template for each action: advanced.phtml, home.phtml and search.phtml. These templates can simply wrap around the default search templates, overriding a few settings if necessary. See the existing templates in templates/world-cat for the simplest possible example.
    2. Record Templates - If your record driver extends VF_RecordDriver_SolrDefault, you may not need to create any record templates at all – the defaults should work for you. However, if you built a custom driver, you will need to create an appropriately-named directory under templates/RecordDriver in your chosen theme. Look at the existing templates in templates/RecordDriver/SolrDefault to see which filenames you need to create and what sort of content they should contain (note that some are optional and some filenames are dynamically generated based on other record driver options – i.e. the tab-*.phtml files, which are driven by allowed tab options).
development/howtos/connecting_a_new_external_data_source.1330711852.txt.gz · Last modified: 2014/06/13 13:13 (external edit)