About Features Downloads Getting Started Documentation Events Support GitHub

Love VuFind®? Consider becoming a financial supporter. Your support helps build a better VuFind®!

Site Tools


Warning: This page has not been updated in over over a year and may be outdated or deprecated.
development:howtos:connecting_a_new_external_data_source

This is an old revision of the document!


Connecting a New External Data Source

VuFind includes support for several external services in addition to the core Solr index. However, it may sometimes be necessary to build support for a new service. This page will help take you through the process.

Initial Decisions

There are a few things to decide before implementing any code.

  • What URLs do you want to use for searches and record views? This will affect the names of the routes and controllers you create. (i.e. http://localhost/vufind/WorldCat/Search, http://localhost/vufind/WorldCatRecord/12345)
  • What “source” value do you want VuFind to use for identifying records from your new source? This value will be saved in the database and passed in some URLs. It should be short, unique and alphanumeric (i.e. “WorldCat”, “Summon”).
  • What name do you want to use for your family of search classes; this should almost always be the same as your “source” value.
  • What name do you want to use for your record driver? Again, this should usually be consistent with the “source” value.

Step-by-Step Building Instructions

If you haven't already, you may want to read the Data Model / Key Concepts page for background information on how VuFind's pieces fit together.

Once you are ready to write code, you can follow these steps to get a new module added to VuFind. Obviously, this is just a recommendation – some of these steps can be taken out of order without any problems.

Note that for examples below, we will assume that you have chosen Source, search class and record driver names of “Sample.” Just replace “Sample” with your actual choice.

The examples also assume that you are adding to core VuFind code. You also have the option of building your classes inside a custom module; to do this, you just need to create all new classes within your module's namespace and register them in your module's config/module.config.php instead of inside the main VuFind module.

  1. Build/Find Connection Library - Before you can integrate a new service into VuFind, you need to be able to talk to it from within PHP. It's possible that the service provides its own library, which may save you some time. Otherwise, you'll have to build something. If you can solve your service-specific connection problems prior to VuFind integration, then you won't have as many different problems to debug during integration.
  2. Implement Record Driver - The record driver object is responsible for extracting key pieces of data about a record from the data returned by your chosen service. To create it, follow these steps:
    1. Create a record driver plugin for your data source – i.e. module/VuFind/src/VuFind/RecordDriver/Sample.php containing a class called \VuFind\RecordDriver\Sample.
    2. Decide which existing record driver class to extend. If the records you are working with are very distinctive, you will probably want to extend \VuFind\RecordDriver\AbstractBase and build custom templates. However, in many cases, records will contain bibliographic data that is similar enough to VuFind's default Solr records to allow you to extend \VuFind\RecordDriver\SolrDefault (or one of its subclasses) and thus recycle many of VuFind's default built-in templates. This approach is recommended whenever it is possible.
    3. Implement key methods:
      • Constructor - The constructor should set the $this→resourceSource property to the source value you chose earlier (i.e. 'Sample') and, if applicable, the $this→recordIni property to an .ini file containing record-related settings (if you want to take advantage of related record modules).
      • setRawData() - This method is fed data directly from the external service and is responsible for storing it within the object. This data will be used by all other methods to retrieve specific details. The nature of the data being passed in is determined by your search results class (see below); you can use whatever is most convenient for your situation. \VuFind\RecordDriver\AbstractBase contains a default implementation of this method which stores the data in the $this→fields property; in many cases, you can use this default, but you may want to modify this method if additional processing is necessary.
      • getUniqueId() - This method returns the record's unique identifier. All VuFind records must have a unique identifier of some sort. Note that this unique ID is combined with the “source” value when VuFind does lookups, so you do not have to worry about IDs from one service colliding with IDs from another service.
      • getBreadcrumb() - This method returns a string used for identifying the record in breadcrumbs, page titles, etc. Normally it will be a short form title or something similar.
      • …and the rest - If you are extending \VuFind\RecordDriver\SolrDefault and your saved data is not structured exactly like a Solr response, you should override the public get methods of that class to return information from your chosen format. (If the data is unavailable for a given method, you don't have to override it – the base code will handle missing data appropriately). If you are extending \VuFind\RecordDriver\AbstractBase, you should add public get methods for any data elements you will need to display in your templates. When possible, try to name your methods consistently with other record drivers (see the Record Driver Method Master List) since this will improve your chances of being able to reuse certain VuFind components.
      • In most cases, you may want to adjust other methods dealing with things like tabs to display in record view or supported export formats, but these details can probably wait until after the first phase of development.
  3. Register Record Driver - Add your new record driver to the invokables array of the recorddriver_plugin_manager section of module/VuFind/config/module.config.php.
  4. Implement Search Classes - Create a new directory under module/VuFind/src/VuFind/Search named for the search class ID you picked earlier. In this new directory, you will create three files:
    1. Options.php - This will contain the \VuFind\Search\Sample\Options class, which must extend \VuFind\Search\Base\Options. See the existing \VuFind\Search\Solr\Options or \VuFind\Search\WorldCat\Options classes for some examples of how this works. These are the most important methods to implement in this class:
      • Constructor - The constructor tells the other search classes which .ini files to use for search/facet settings and sets up initial options by reading them in from those .ini files. (You may at this time wish to establish a Sample.ini file – you can use config/vufind/Summon.ini and config/vufind/WorldCat.ini for inspiration).
      • getSearchAction() - This returns the name of the route used to access the search action for your new service. This will normally be something like 'sample-search.'
      • getAdvancedSearchAction() - This is just like getSearchAction, except it points to the Advanced Search form. This will normally be something like 'sample-advanced.' If you do not wish to support advanced searches, simply return false.
    2. Params.php - This will contain the \VuFind\Search\Sample\Params class, which must extend \VuFind\Search\Base\Params. Unless you need to do special parameter processing or add new parameters not supported by the base class, you are not required to implement any methods here – you can just extend with an empty class. You'll probably end up adding methods here eventually, but for the initial implementation it is nice to leave this empty – one less thing to worry about!
    3. Results.php - This will contain the \VuFind\Search\Sample\Results class, which must extend \VuFind\Search\Base\Results. This is the most important and complex search class, and looking at existing examples like \VuFind\Search\Solr\Results and \VuFind\Search\WorldCat\Results may help you understand it better. You will have to implement several methods:
      • performSearch() - This method reads the search parameters from the parameters and options objects, uses them to perform a search against your chosen service, and uses the results of the search to populate two object properties: $this→resultTotal, the total number of search results found, and $this→results, an array of record driver objects representing the current page of results requested by the user.
      • getFacetList() - This method returns facet options related to the current search. You may need to store extra values in performSearch() to allow the list to be generated. It is possible that getFacetList will be called before a search has been performed – in this case, you should call the performAndProcessSearch method to fill in all the details. (See \VuFind\Search\Solr\Results for an example of this).
      • getRecord() - This method takes an ID value and returns a record driver representing the requested record. If the record does not exist, it should throw a \VuFind\Exception\RecordMissing object.
      • getRecords() - This static method takes an array of IDs and returns an array of record drivers representing the records. It may throw a \VuFind\Exception\RecordMissing object if appropriate. This method is implemented for you in the \VuFind\Search\Base\Result class, so you do not have to write your own – but if there is an efficient way to request multiple IDs at the same time, writing your own method will improve VuFind's performance; the default base class method simply calls getRecord() repeatedly from a loop.
  5. Register Search Classes - Add your new search objects' namespace to the namespaces_by_id array of the ['vufind']['search_manager'] section of module/VuFind/config/module.config.php.
  6. Create Controllers - In order to provide web access to your new code, you will need to create two new controllers in the module/VuFind/src/VuFind/Controller folder:
    1. Search Controller - This should have a name like \VuFind\Controller\SampleController and should extend \VuFind\Controller\AbstractSearch. This controller will handle displaying and processing search forms. At a bare minimum, it should contain a constructor which sets the $this→searchClassId property to the appropriate value (i.e. 'Sample') and calls the parent constructor. You can look at the \VuFind\Controller\AbstractSearch class for other available options. You can also define controller actions if you wish, but this is not necessary – by default, you will have Home, Results and Advanced actions inherited from the base class.
    2. Record Controller - This should have a name like \VuFind\Controller\SamplerecordController and should extend \VuFind\Controller\AbstractRecord. This controller will handle displaying records and performing record-related tasks (export, save, etc.). At a bare minimum, it should contain a constructor which sets the $this→searchClassId property to the appropriate value (i.e. 'Sample') and calls the parent constructor.
  7. Register Controllers - Now that your controllers are created, you should register them in the invokables array of the controller section of module/VuFind/config/module.config.php. The entries should look like 'sample' ⇒ 'VuFind\Controller\SampleController' and 'samplerecord' ⇒ 'VuFind\Controller\SamplerecordController'.
  8. Set Up Routes - You'll need to edit the module/VuFind/config/module.config.php configuration file to set up all the routes related to your new controllers. The configuration is designed to make it easy to add new routes:
    1. In the $recordRoutes array, add an association between the route name (which should be your source value concatenated with the word “record”) and the route's controller name (as registered in controller invokables above). For example: 'samplerecord' ⇒ 'samplerecordcontroller'
    2. In the $staticRoutes array, add any necessary search-related routes… for example, 'Sample/Home', 'Sample/Search' and 'Sample/Advanced'.
    3. Note that, if you are using a custom module, you may need to copy some extra route-building logic from the main VuFind configuration, since these convenience route-building arrays are not present by default in empty modules.
  9. Set Up Templates - Now that the models and controllers are set up, you need some views.
    1. Search Templates - Within your chosen theme, create a templates/sample directory (replacing 'sample' with a lowercased version of your search controller's name). In this directory, you need to create one template for each action: advanced.phtml, home.phtml and search.phtml. These templates can simply wrap around the default search templates, overriding a few settings if necessary. See the existing templates in templates/worldcat for the simplest possible example.
    2. Record Templates - If your record driver extends \VuFind\RecordDriver\SolrDefault, you may not need to create any record templates at all – the defaults should work for you. However, if you built a custom driver, you will need to create an appropriately-named directory under templates/RecordDriver in your chosen theme. Look at the existing templates in templates/RecordDriver/SolrDefault to see which filenames you need to create and what sort of content they should contain (note that some are optional and some filenames are dynamically generated based on other record driver options – i.e. the tab-*.phtml files, which are driven by allowed tab options).
development/howtos/connecting_a_new_external_data_source.1357910431.txt.gz · Last modified: 2014/06/13 13:13 (external edit)