VuFind Documentation

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Connecting a New External Data Source ======

VuFind includes support for several external services in addition to the core Solr index.  However, it may sometimes be necessary to build support for a new service.  This page will help take you through the process.

===== Initial Decisions =====

There are a few things to decide before implementing any code.

  * What URLs do you want to use for searches and record views?  This will affect the names of the routes and controllers you create. (i.e. http://localhost/vufind/WorldCat/Search, http://localhost/vufind/WorldCatRecord/12345)
  * What "source" value do you want VuFind to use for identifying records from your new source?  This value will be saved in the database and passed in some URLs.  It should be short, unique and alphanumeric (i.e. "WorldCat", "Summon").
  * What name do you want to use for your family of search classes; this should almost always be the same as your "source" value.
  * What name do you want to use for your record driver?  Again, this should usually be consistent with the "source" value.

===== Step-by-Step Building Instructions =====

If you haven't already, you may want to read the [[development:architecture:data_model_key_concepts|Data Model / Key Concepts]] page for background information on how VuFind's pieces fit together.

Once you are ready to write code, you can follow these steps to get a new module added to VuFind.  Obviously, this is just a recommendation -- some of these steps can be taken out of order without any problems.

Note that for examples below, we will assume that you have chosen Source, search class and record driver names of "Sample."  Just replace "Sample" with your actual choice.

The examples also assume that you are adding to core VuFind code.  You also have the option of building your classes inside a [[development:architecture:customizing_vufind#modules|custom module]]; to do this, you just need to create all new classes within your module's namespace and register them in your module's config/module.config.php instead of inside the main VuFind module.

  - **Build Search Connector/Backend** - Before you can integrate a new service into VuFind, you need to be able to talk to it from within PHP.  It's possible that the service provides its own library, which may save you some time.  Whether or not there is a pre-existing library, you should implement a search backend for VuFind's [[development:architecture:search_service|search service]].  This consists of a few steps:
    - Create a backend implementing \VuFindSearch\Backend\BackendInterface (and possibly also some of the feature interfaces found in the \VuFindSearch\Feature namespace, if appropriate).  The purpose of this is to provide low-level functionality for performing searches and retrieving records.  Several example backend implementations can be found in the VuFindSearch module -- you should be able to use these as models during development.  In some cases (as in connecting to a custom Solr core) you may be able to use existing code but simply configure it differently.
    - Create a factory to construct your backend in the context of VuFind; some example factories can be found in the \VuFind\Search\Factory namespace.
    - Register your factory in the ['vufind']['plugin_managers']['search_backend']['factories'] section of module.config.php (either by modifying the VuFind module's configuration, or that of your own local custom module).  The name of the registered service should match the "source" value you chose above.
  - **Implement Record Driver** - The record driver object is responsible for extracting key pieces of data about a record from the data returned by your chosen service.  To create it, follow these steps:
    - Create a [[development:plugins:record_drivers|record driver plugin]] for your data source -- i.e. module/VuFind/src/VuFind/RecordDriver/Sample.php containing a class called \VuFind\RecordDriver\Sample.
    - Decide which existing record driver class to extend.  If the records you are working with are very distinctive, you will probably want to extend \VuFind\RecordDriver\AbstractBase and build custom templates.  However, in many cases, records will contain bibliographic data that is similar enough to VuFind's default Solr records to allow you to extend \VuFind\RecordDriver\SolrDefault (or one of its subclasses) and thus recycle many of VuFind's default built-in templates.  This approach is recommended whenever it is possible.
    - Implement key methods:
      * //setRawData()// - This method is fed data directly from the external service (usually by a class implementing \VuFindSearch\Response\RecordCollectionFactoryInterface and set up by the service factory described above) and is responsible for storing it within the object. This data will be used by all other methods to retrieve specific details. The nature of the data being passed in is determined by your search results class (see below); you can use whatever is most convenient for your situation.  \VuFind\RecordDriver\AbstractBase contains a default implementation of this method which stores the data in the $this->fields property; in many cases, you can use this default, but you may want to modify this method if additional processing is necessary.
      * //getUniqueId()// - This method returns the record's unique identifier.  All VuFind records must have a unique identifier of some sort.  Note that this unique ID is combined with the "source" value when VuFind does lookups, so you do not have to worry about IDs from one service colliding with IDs from another service.
      * //getBreadcrumb()// - This method returns a string used for identifying the record in breadcrumbs, page titles, etc.  Normally it will be a short form title or something similar.
      * ...and the rest - If you are extending \VuFind\RecordDriver\SolrDefault and your saved data is not structured exactly like a Solr response, you should override the public get methods of that class to return information from your chosen format.  (If the data is unavailable for a given method, you don't have to override it -- the base code will handle missing data appropriately).  If you are extending \VuFind\RecordDriver\AbstractBase, you should add public get methods for any data elements you will need to display in your templates.  When possible, try to name your methods consistently with other record drivers (see the [[development:architecture:record_driver_method_master_list|Record Driver Method Master List]]) since this will improve your chances of being able to reuse certain VuFind components.  
      * In most cases, you may want to adjust other methods dealing with things like tabs to display in record view or supported export formats, but these details can probably wait until after the first phase of development.
  - **Register Record Driver** - Add your new record driver to the recorddriver_plugin_manager section of module/VuFind/config/module.config.php; you will often want to set up a factory function to pass configurations to the record driver's constructor -- see existing driver configurations (in [[https://github.com/vufind-org/vufind/blob/dev/module/VuFind/src/VuFind/RecordDriver/PluginManager.php|VuFind\RecordDriver\PluginManager]]) for some examples. Be sure that the name you register the driver under matches the name expected by your backend factory (and see the [[development:plugins:record_drivers]] page for some notes on registering custom Solr record drivers).
  - **Implement Search Classes** - Create a new directory under module/VuFind/src/VuFind/Search named for the search class ID you picked earlier.  In this new directory, you will create three files:
    - //Options.php// - This will contain the \VuFind\Search\Sample\Options class, which must extend \VuFind\Search\Base\Options.  See the existing \VuFind\Search\Solr\Options or \VuFind\Search\WorldCat\Options classes for some examples of how this works.  These are the most important methods to implement in this class:
      * //Constructor// -  The constructor tells the other search classes which .ini files to use for search/facet settings and sets up initial options by reading them in from those .ini files.  (You may at this time wish to establish a Sample.ini file -- you can use config/vufind/Summon.ini and config/vufind/WorldCat.ini for inspiration).
      * //getSearchAction()// - This returns the name of the route used to access the search action for your new service.  This will normally be something like 'sample-search.'
      * //getAdvancedSearchAction()// - This is just like getSearchAction, except it points to the Advanced Search form.  This will normally be something like 'sample-advanced.'  If you do not wish to support advanced searches, simply return false.
    - //Params.php// - This will contain the \VuFind\Search\Sample\Params class, which must extend \VuFind\Search\Base\Params.  Unless you need to do special parameter processing or add new parameters not supported by the base class, you are not required to implement any methods here -- you can just extend with an empty class.  You'll probably end up adding methods here eventually, but for the initial implementation it is nice to leave this empty -- one less thing to worry about!
    - //Results.php// - This will contain the \VuFind\Search\Sample\Results class, which must extend \VuFind\Search\Base\Results.  This class interacts with your backend to expose results to VuFind, and looking at existing examples like \VuFind\Search\Solr\Results and \VuFind\Search\WorldCat\Results may help you understand it better.  You will have to implement several methods:
      * //performSearch()// - This method reads the search parameters from the parameters and options objects, uses them to perform a search against your chosen service, and uses the results of the search to populate two object properties: $this->resultTotal, the total number of search results found, and $this->results, an array of record driver objects representing the current page of results requested by the user.
      * //getFacetList()// - This method returns facet options related to the current search.  You may need to store extra values in performSearch() to allow the list to be generated.  It is possible that getFacetList will be called before a search has been performed -- in this case, you should call the performAndProcessSearch method to fill in all the details.  (See \VuFind\Search\Solr\Results for an example of this).
  - **Register Search Classes** - You may not have to register your search classes at all -- if they live in the VuFind\Search namespace and have no special constructor parameters, the default factory will take care of building them for you.  However, if you have special needs, you will need to register the classes in the ['vufind']['plugin_managers']['search_options'], ['vufind']['plugin_managers']['search_params'] and/or ['vufind']['plugin_managers']['search_results'] sections of module/VuFind/config/module.config.php.
  - **Create Controllers** - In order to provide web access to your new code, you will need to create two new controllers in the module/VuFind/src/VuFind/Controller folder:
    - **Search Controller** - This should have a name like \VuFind\Controller\SampleController and should extend \VuFind\Controller\AbstractSearch.  This controller will handle displaying and processing search forms.  At a bare minimum, it should contain a constructor which sets the $this->searchClassId property to the appropriate value (i.e. 'Sample') and calls the parent constructor.  You can look at the \VuFind\Controller\AbstractSearch class for other available options.  You can also define controller actions if you wish, but this is not necessary -- by default, you will have Home, Results and Advanced actions inherited from the base class.
    - **Record Controller** - This should have a name like \VuFind\Controller\SamplerecordController and should extend \VuFind\Controller\AbstractRecord.  This controller will handle displaying records and performing record-related tasks (export, save, etc.).  At a bare minimum, it should contain a constructor which sets the $this->searchClassId property to the appropriate value (i.e. 'Sample') and calls the parent constructor.
  - **Register Controllers** - Now that your controllers are created, you should register them in the invokables array of the controller section of module/VuFind/config/module.config.php.  The entries should look like 'sample' => 'VuFind\Controller\SampleController' and 'samplerecord' => 'VuFind\Controller\SamplerecordController'.
  - **Set Up Routes** - You'll need to edit the module/VuFind/config/module.config.php configuration file to set up all the routes related to your new controllers.  The configuration is designed to make it easy to add new routes:
    - In the $recordRoutes array, add an association between the route name (which should be your source value concatenated with the word "record") and the route's controller name (as registered in controller invokables above).  For example: 'samplerecord' => 'samplerecordcontroller'
    - In the $staticRoutes array, add any necessary search-related routes... for example, 'Sample/Home', 'Sample/Search' and 'Sample/Advanced'.
    - Note that, if you are using a [[development:architecture:customizing_vufind#modules|custom module]], you may need to copy some extra route-building logic from the main VuFind configuration, since these convenience route-building arrays are not present by default in empty modules.
  - ** Set Up Templates** - Now that the models and controllers are set up, you need some views.
    - //Search Templates// - Within your chosen theme, create a templates/sample directory (replacing 'sample' with a lowercased version of your search controller's name).  In this directory, you need to create one template for each action: advanced.phtml, home.phtml and search.phtml.  These templates can simply wrap around the default search templates, overriding a few settings if necessary.  See the existing templates in templates/worldcat for the simplest possible example.
    - //Record Templates// - If your record driver extends \VuFind\RecordDriver\SolrDefault, you may not need to create any record templates at all -- the defaults should work for you.  However, if you built a custom driver, you will need to create an appropriately-named directory under templates/RecordDriver in your chosen theme.  Look at the existing templates in templates/RecordDriver/SolrDefault to see which filenames you need to create and what sort of content they should contain (note that some are optional and some filenames are dynamically generated based on other record driver options -- i.e. the tab-*.phtml files, which are driven by allowed tab options).
VuFind Documentation

Site Tools

Page Tools
