Continuous Integration

VuFind uses continuous integration to monitor the quality of the code base and allow early detection of problems. This is a work in progress, and this page exists to help keep track of tools and ideas. You can help by writing new unit tests and by fixing problems identified within our Jenkins instance.

The following technologies are currently installed as part of our continuous integration setup.

Jenkins (formerly named Hudson) is a popular open source continuous integration server.

Resources:

• Jenkins Home Page
• PHP + Hudson Article
• VuFind's Jenkins Server

GitHub Actions and Travis CI are hosted continuous integration services with strong GitHub integration. While VuFind uses Jenkins for its primary CI functionality due to richer reporting capabilities, it also uses GitHub Actions or Travis as a first line of defense for detecting test breaks and style problems in pull requests. (VuFind used Travis for many years, but switched to GitHub Actions when Travis reduced its support for open source development).

Phing is a build tool designed as the PHP-based answer to Apache Ant. It can be integrated with Jenkins just as Maven can. We currently use Phing to set up our Selenium test environment and to generate several of Jenkins's reports.

Resources:

• Phing Home Page

PHPUnit is a unit testing framework for PHP. See the Unit Tests page for more details.

We use Mink (a browser automation abstraction library) with either Chrome in headless mode or Selenium (a browser automation library) for building automated browser-based tests. This is useful for integration testing, although complex tests can sometimes take a very long time to execute. See this wiki page for more details.

Resources:

• Mink Home Page
• Selenium Home Page

Here are quick notes on how to set up Jenkins with VuFind. These notes were made when setting up the environment on a CentOS 6.5 server with Jetty 9, so they may need to be adjusted for other distributions and versions.

Install Jenkins using commands suitable for the distribution, e.g.

  sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat/jenkins.repo
  sudo rpm --import http://pkg.jenkins-ci.org/redhat/jenkins-ci.org.key
  yum install jenkins

Install Jetty along the lines of http://techtraits.com/jetty9install/ but get the latest version and add the following to the beginning of /etc/init.d/jetty:

  JETTY_HOME=/opt/jetty
  JETTY_USER=jetty
  JETTY_ARGS="jetty.port=8081 jetty.host=127.0.0.1"
  JETTY_LOGS=/var/log/jetty/

Create log directory:

  mkdir /var/log/jetty
  chown jetty /var/log/jetty

Link Jenkins to Jetty's webapps:

  ln -s /usr/lib/jenkins/jenkins.war /opt/jetty/webapps/
  

Add to /opt/jetty/etc/jetty.xml (change the path as necessary):

  <Call class="java.lang.System" name="setProperty">
    <Arg>JENKINS_HOME</Arg>
    <Arg>/data/jenkins</Arg>
  </Call>

Add security context /opt/jetty/webapps/jenkins.xml to jetty:

  <Configure class="org.eclipse.jetty.webapp.WebAppContext">
    <Set name="contextPath">/jenkins</Set>
    <Set name="war"><SystemProperty name="jetty.home" default="."/>/webapps/jenkins.war</Set>
    <Get name="securityHandler">
      <Set name="loginService">
        <New class="org.eclipse.jetty.security.HashLoginService">
          <Set name="name">Jenkins Realm</Set>
          <Set name="config"><SystemProperty name="jetty.home" default="."/>/etc/realm.properties</Set>
        </New>
      </Set>
    </Get>
  </Configure>

Setup directory permissions (change the path as necessary):

  
  chown jetty:adm /data/jenkins

Set up reverse proxy e.g. from Apache if necessary (see https://wiki.jenkins-ci.org/display/JENKINS/Running+Jenkins+behind+Apache)

Install ImageMagick development package (e.g. ImageMagick-devel).

Install Composer.

• It used to be necessary to install a large number of tools to get Continuous Integration working, but VuFind's composer.json now does all of that work for you. As long as Composer is installed in a location accessible to Jenkins, the rest should happen automatically.

Install GraphViz (optional)

• GraphViz can be used to draw some graphs as part of the PhpDocumentor process; however, it doesn't always work properly. If you encounter errors during documentation generation, it is actually better to leave it out altogether.

Start Jetty, point your browser to http://host:port/jenkins and configure Jenkins security etc. (not documented here).

Install the following plugins to Jenkins:

• GitHub
• Phing
• Checkstyle
• PMD
• DRY
• Clover
• JDepend
• Test Results Analyzer

These are sample values, modify accordingly:

• Project name: VuFind
• GitHub project: https://github.com/your_org/your_repo
• Source Code Management/Git:
  • Repository URL: https://github.com/your_org/your_repo
• Build triggers:
  • [x] Build when a change is pushed to GitHub (note that you need to configure the hooks: https://wiki.jenkins-ci.org/display/JENKINS/GitHub+Plugin#GitHubPlugin-TriggerabuildwhenachangeispushedtoGitHub)
• Execute shell [this must run BEFORE Phing]:
  • composer install
• Build / Invoke Phing targets:
  • Phing version: Be sure to set up a custom Phing installation in Manage Jenkins / Configure System. This should have a PHP Command setting of “$WORKSPACE/vendor/bin/phing” to run the version of Phing installed by Composer.
  • Targets: main
  • If you need to customize the build file, click Advanced… and enter your custom build file name in “Phing Build File”
• In Post-build Actions, add all the publishers:
  • Checkstyle (Checkstyle results: /tmp/build/vufind/reports/checkstyle.xml)
  • PMD (PMD results: /tmp/build/vufind/reports/phpmd.xml)
    • duplicate code (/tmp/build/vufind/reports/pmd-cpd.xml)
    • Clover coverage (/tmp/build/vufind/reports/coverage)
    • Junit (/tmp/build/vufind/reports/phpunit.xml)
    • Javadoc (/tmp/build/vufind/apidocs)
    • JDepend (/tmp/build/vufind/reports/jdepend.xml)