Table of Contents
This page is aimed at developers who want to test that their changes have not broken existing VuFind functionality or who are interested in creating standard tests for new VuFind components. If you are interested in testing performance rather than functionality, see the Testing Performance page instead.
The test modules provided with VuFind use the PHPUnit testing framework. The framework can be installed using PEAR (see details on the project's homepage). Once installed, you will have a phpunit command line tool that you can use to run tests.
These tests were designed for use with VuFind's continuous integration system. As such, some tests create and destroy data. Although some safety mechanisms exist to reduce the chances of accidental data loss, THE TESTS SHOULD NEVER BE RUN ON A PRODUCTION SYSTEM.
The testing process was developed under Linux; this is the recommended platform for VuFind testing, and different procedures may need to be developed for other platforms like Windows.
The easiest way to run VuFind's tests is with the help of the Phing build tool. VuFind comes with a build.xml file that Phing can use to automate tasks.
Recommended setup procedure
1.) Create a fresh copy of VuFind (i.e. git clone the repository to a fresh directory, then run “composer install”)
2.) Create a phing.sh script in the root of your fresh VuFind install to automatically pass important parameters to Phing (see build.xml for other parameters that may be set here with the -D parameter):
#!/bin/sh $VUFIND_HOME/vendor/bin/phing -Dmysqlrootpass=mypasswd $*
Running tests after setup
Follow these steps to run tests. Keep in mind that testing will create a test Solr index listening on port 8080. This may cause port conflicts if you test on a server that is already running applications on that port – plan accordingly.
1.) ./phing.sh startup
This command will start up an instance of VuFind containing test data that may be used by some of the tests.
2.) ./phing.sh phpunit
(the phpunit command will run tests and generate report data for use by continuous integration; phpunitfast will run tests more quickly by skipping reports)
3.) ./phing.sh shutdown
This command will turn off the VuFind test instance created by step 1.
The Faster Version
If you don't want to run integration tests using a running VuFind, you can simply execute:
The integration tests will be skipped, but all unit tests not depending on a running VuFind instance will still run.
Running Just One Test
Sometimes you want to run a specific test without the rest of the suite. You can use the phpunit_extra_params parameter for this:
./phing.sh phpunitfast -Dphpunit_extra_params=path/to/your/test.php
Browser Automation Tests with Mink
Some of VuFind's tests are designed to use Mink to automate a browser. These tests will be skipped unless you have properly configured your environment so the tests have a VuFind instance they can drive.
If you wish to run this portion of the test suite, revise your phing.sh script (described above) to include some additional variables:
- extra_shutdown_cleanup - an extra command to run as part of the shutdown process; it's usually a good idea to change the ownership of the local/cache directory here, since running tests will cause some files to be owned by Apache, and that will interfere with subsequent file deletion.
- apacheconfdir - a directory from which Apache will auto-load configurations; we need this to start up a new VuFind instance by injecting a file
- apachectl - the command to restart Apache; you'll probably need to prefix this with sudo in order to make it work when running tests as a different user
- mysqlrootpass - the MySQL root password (needed for building VuFind's database)
- vufindurl - the URL where VuFind will be accessed (defaults to http://localhost/vufind if omitted)
Here's an example script from an Ubuntu environment:
#!/bin/sh phing -Dextra_shutdown_cleanup="sudo chown -R myusername:mygroup /path/to/vufind/local/cache" -Dapacheconfdir="/etc/apache2/conf-enabled" -Dapachectl="sudo /etc/init.d/apache2" -Dmysqlrootpass=myrootpasswd -Dvufindurl=http://localhost/vufind_test $*
Prior to running your tests, you should download the Selenium server .jar file from here and then run it with “java -jar [downloaded file]”
When running in a continuous integration environment, you should use Xvfb to create a virtual frame buffer so that the browser driven by Selenium can run without errors. However, for manually-executed tests, you can watch the browser executing the tests, which can be useful for troubleshooting purposes.
If you don't want to monitor test results or watch the Selenium server jar's output, you can have Phing automatically start up/shut down Selenium and Xvfb by passing the -Dseleniumjar=[downloaded file] option when you startup and shutdown your test instance.
If you want to use a browser other than Firefox with Selenium, use the -Dselenium_browser=[name] setting to switch. For example, you can use -Dselenium_browser=chrome (though this requires you to have ChromeDriver installed on your system).
When running tests in a windowed environment, it's a good idea to open a couple of Terminal windows – that way you can run the Selenium server in one window (to watch it receive commands) while running your tests themselves in another (to see failures, etc.).
Note: Prior to VuFind 3.1, the VuFind team used Zombie.js as their primary Mink driver; however, due to limitations with Zombie.js, Selenium is now the only supported driver for testing.
VuFind unit tests (which check the functionality of individual components) can be found in the module/VuFind/tests/unit-tests directory of your VuFind installation.
VuFind integration tests (which check the interaction of components in a real running system) can be found in the module/VuFind/tests/integration-tests directory of your VuFind installation.
VuFind support modules (VuFindHttp, etc.) have their own tests directories, but these are configured to be run automatically as part of VuFind's main test suite.
The layout of the each test directory is designed to mirror the VuFind module's library (module/VuFind/src/VuFind). Tests should be placed in a directory that corresponds with the component being tested. For example, the unit tests for the \VuFind\Date\Converter class are found in “module/VuFind/tests/unit-tests/Date/ConverterTest.php”.
The VuFindTest namespace (code found in module/VuFind/src/VuFindTest) contains several classes which may be useful in writing new tests – a fake record driver in VuFindTest\RecordDriver and some base test classes containing convenience methods in VuFindTest\Unit.
When working on integration tests, be aware that the full framework configuration is not loaded. Instead, VuFindTest\Unit\TestCase sets up a fake service manager to provide key services. If you add a dependency on a new service, it will not be made available automatically – it will need to be added to the base test class. This is not ideal and the integration test strategy may be improved in a future version of VuFind.
Mink Browser Automation
If you want to write browser automation tests, you should extend \VuFindTest\Unit\MinkTestCase. This class will automatically set up the Mink/Zombie.js environment for you. It also provides a changeConfigs() method which can be used to reconfigure the running VuFind instance by modifying its .ini files. Any changes made by this method will be automatically rolled back at the end of the test, so you can test a variety of configurations in a single test class.
Some example Mink tests can be found in module/VuFind/tests/integration-tests/src/VuFindTest/Mink/.
Create a directory in your local module mimicking the current structure of the vufind tests. Then all you have to do is require the class you are testing. If you figure out a way to autoload local module classes for testing, please let us know!
Prior to vufind 2.2 this will require changes to phpunit.xml. See commit 6e2b138.