| Both sides previous revisionPrevious revisionNext revision | Previous revision |
| development:testing:manual_testing [2023/04/12 16:43] – ↷ Page name changed from development:testing:checklist to development:testing:manual_testing demiankatz | development:testing:manual_testing [2024/04/08 17:27] (current) – demiankatz |
|---|
| ====== Testing Checklist ====== | ====== Manual Testing ====== |
| |
| In any application, adding new features can sometimes break old ones. The intent of this page is to list key VuFind features in checklist format so that periodic reviews can be made to ensure that everything is functioning as expected. This list is aimed at high-level testing by a human using a web browser; for automated testing, see the [[development:testing:unit_tests|Unit Tests]] page. | VuFind® uses a set of [[development:testing:unit_tests|Unit Tests]] to check for problems as the software develops and evolves. Automated tests are generally the preferred way to test software, since they are automatic and repeatable. However, sometimes humans can detect problems in areas where automated tests cannot (like aesthetic changes), and sometimes manual testing is useful as a lead-up to developing automated tests. This page contains notes intended to be helpful for manual testers. |
| |
| Please add your own notes to this page if you add or encounter features that you think may get lost in the shuffle without occasional testing. This list may become very long, but it is useful to have a reminder. | |
| |
| :!: This page is fairly old and hasn't been updated in a while; in most cases, the [[https://github.com/vufind-org/vufind/tree/dev/module/VuFind/tests/integration-tests/src/VuFindTest/Mink|Mink-based browser automation tests]] are better than testing by hand, and building new test classes is preferable to adding notes here. However, this page is still being retained for reference for the moment, at least until all of the listed tasks are covered by automated tests. | |
| ===== General Considerations ===== | ===== General Considerations ===== |
| |
| |
| ==== Browser Testing ==== | ==== Browser Testing ==== |
| |
| * Test the interface in multiple browsers to check for display inconsistencies and Javascript incompatibilities. We are most concerned with compatibility with the current generation of browsers, but compatibility with earlier versions is desirable when possible. Recommended browsers to test: | It is a good idea to test the interface in multiple browsers to check for display inconsistencies and Javascript incompatibilities. We are most concerned with compatibility with the current generation of browsers, but compatibility with earlier versions is desirable when possible. Recommended browsers to test: |
| * Firefox | |
| * Internet Explorer | * Firefox |
| * Chrome | * Edge |
| * Safari | * Chrome |
| * Opera | * Safari |
| | * Opera |
| | |
| | The automated test suite typically runs in Chrome, so doing manual testing in other browsers is a good way to potentially uncover browser-specific issues. |
| |
| ==== Input Testing ==== | ==== Input Testing ==== |
| | |
| | Some general patterns to follow when testing new features: |
| |
| * Test fields stored in the database for SQL injection vulnerabilities. | * Test fields stored in the database for SQL injection vulnerabilities. |
| * Test fields where input is stored and displayed back for XSS vulnerabilities and missing HTML/URL encoding. A good test is "<script>alert('test');</script>" -- if there is an XSS vulnerability, this will cause a pop-up to appear! | * Test fields where input is stored and displayed back for XSS vulnerabilities and missing HTML/URL encoding. A good test is "<script>alert('test');</script>" -- if there is an XSS vulnerability, this will cause a pop-up to appear! |
| |
| ===== Specific Modules ===== | If you find something that breaks, please don't just fix it -- also build an automated test to prevent regressions! |
| | |
| | ===== Testing Integrations ===== |
| | |
| | VuFind®'s test suite is not able to connect to third-party systems. While it can simulate these interactions for testing purposes, if third-party systems actually change their interfaces, the test suite will not be able to detect problems. There is no way around this -- we do not own copies of every Integrated Library System ever published, or have subscriptions to every third-party resource. Therefore, user testing of integrations that they have access to is valuable to ensure that our code remains up-to-date and relevant. |
| | |
| | Some areas of code that benefit from this type of testing: |
| | |
| | * Export Targets (e.g. EndNote, RefWorks, etc.) |
| | * ILS Drivers (e.g. Alma, Aleph, etc.) |
| | * Search Backends (e.g. EDS, Summon, WorldCat, etc.) |
| | * SMTP integration (e.g. sending emails via various services) |
| | * SMS integration (e.g. sending texts via various services) |
| | |
| | If you test and discover that something is not working, please open a [[https://vufind.org/jira|JIRA Ticket]] to report the problem. |
| | |
| | ===== Feature Testing ===== |
| | |
| | VuFind® has a lot of optional features which are not immediately visible. This section of the page provides quick steps for turning on optional features so that you can test and evaluate them. This is intended to be used in combination with VuFind®'s standard test environment, which you can learn how to set up by watching the [[videos:testing_1|Setting Up a Test Environment video]] or reading the [[development:testing:unit_tests|unit testing]] page. |
| | |
| | ==== Combined Search ==== |
| | |
| | If you want to test [[configuration:combining_search_types#multi-column_search|multi-column search]], you will need to customize combined.ini, since the default configuration includes examples that are unlikely to work in most environments without additional configuration. For testing purposes, you can override the third-party service examples with the Search2 backend to simulate results. Just create a local copy of combined.ini and change the [Summon] and [EDS] section headers to [Search2:Summon] and [Search2:EDS] respectively. When you perform a search, all of the columns will show results from the Solr index, and there is likely to be overlap between the columns, but for the purpose of testing multi-column functionality, this should not matter too much. |
| | |
| | ==== Hierarchical Facets ==== |
| | |
| | VuFind® supports indexing data in a specially-formatted way to enable facets to be displayed as a hierarchical tree. The test suite sample data includes a hierarchical facet field which is disabled by default but can be turned on for testing by following these steps: |
| | |
| | - In facets.ini, at the top of the [Results] section, add ''hierarchical_facet_str_mv = Hierarchy'' |
| | - Also in facets.ini, in the [SpecialFacets] section, add ''hierarchical[] = hierarchical_facet_str_mv'' |
| | - Now, in the user interface, you should see the facet tree displayed at the top of the sidebar facets in search results |
| | - To test "exclude facets" you can add ''exclude = *'' or ''exclude = hierarchical_facet_str_mv'' to the [Results_Settings] section of facets.ini. It is recommended to test both with and without this setting. |
| | - To test "OR facets" you can add ''orFacets = *'' or ''orFacets = hierarchical_facet_str_mv'' to the [Results_Settings] section of facets.ini. It is recommended to test both with and without this setting. |
| | - To test a hierarchical facet on the home screen, add ''hierarchical_facet_str_mv = Hierarchy'' at the top of the [HomePage] section of facets.ini. This should make the tree appear at the /Search/Home URL of your installation. |
| |
| ==== Admin Module ==== | ==== Library Cards ==== |
| |
| * Test Statistics feature. | VuFind®'s "library cards" feature allows a single user account to be linked to multiple ILS accounts, for scenarios where (for example) a single VuFind® instance is connected to multiple libraries, and users might have accounts in multiple locations. The Demo driver (which simulates an ILS using entirely fake data) can be used for testing library cards. Follow these steps: |
| * Does statistics collection enable/disable appropriately based on config.ini setting? | |
| * Does browser identification work correctly? | |
| * Are record views counted? | |
| * Are searches counted? | |
| * Are no-hit searches counted? | |
| |
| ==== Author Module ==== | - In config.ini, in the [Catalog] section, set driver to "Demo" and library_cards to "true". |
| | - In the VuFind® web interface, create a user account (or log in to an existing account). |
| | - On the "Your Account" page, click on "Library Cards" in the side menu. |
| | - You can now create multiple named library cards with different login credentials; by default, the Demo driver accepts all credentials, so you can make up whatever usernames and passwords you like. |
| | - Now if you go to the "Profile" page, you should see a drop-down menu of library cards at the top of the "Library Catalog Profile" section of the page. When you switch the values in the drop-down menu, you should see the accounts associated with the different cards. The "First Name" field will include the username that you provided as evidence that the data is actually being switched. |
| | - All other features related to the ILS (holds, fines, etc., etc.) should also display the library card selection control. |
| |
| * Test sorting and paging of results. | ==== Merging Record Data When IDs Change ==== |
| |
| ==== Browse Module ==== | VuFind® includes a feature to merge locally-stored data like tags and comments when a service reports that a record's unique identifier has changed. This can occur in some third-party APIs and can also be useful when migrating to a new ILS (since you can index your old IDs in a "previous ID" field and turn on the associated [[development:plugins:fallback_record_loaders|fallback record loader]] to trigger automatic deduplication). |
| |
| * Test all paths through the browse boxes. | When a changed ID is found, an automatic process of reconciling and deduplicating saved data occurs. These steps, combined with data in the default test data, can allow you to confirm that this process works correctly. |
| * Test that result counts are consistent between browse view and search results screen when following links to results. | |
| |
| ==== MyResearch Module ==== | Here are step-by-step instructions (examples assume you are running your test environment at http://localhost/vufind_test -- substitute your own base path as needed): |
| |
| * Test that ILS data is properly pulled down and displayed (checked out items, etc.). | - Go to ''<nowiki>http://localhost/vufind_test/Record/vtls000000329</nowiki>'' in a standard test environment, create an account, and save the record to your favorites. You can also add comments, record-level tags or other data if you wish to make the test case more complex. |
| * Test that list management functionality works correctly. | - Log in to your database on the server and run ''update resource set record_id='(IeDuNL)1048' where record_id='vtls000000329';'' -- this will change the record you just created so that its ID matches the simulated "old ID" value we will be using for the record instead of the id value, which will let us simulate a record ID change. |
| | - Now refresh ''<nowiki>http://localhost/vufind_test/Record/vtls000000329</nowiki>'' in your browser, and repeat saving the record and adding other user data that you wish to test merging. |
| | - In searches.ini, uncomment the fallback_id_field setting and set it to ctrlnum. This will put the system into a mode where, if it tries to do a lookup by ID and can't find a match in the regular ID field, it will fall back and check the ctrlnum field... and if it finds a match, it will then merge any existing data associated with the ctrlnum value into the associated id value. |
| | - Now go to ''<nowiki>http://localhost/vufind_test/Record/(IeDuNL)1048</nowiki>'' -- you should see exactly the same record that you saw in the other link, and this should trigger a data merge in the database. |
| | - Now you can check that everything merged together correctly -- all of the tags, favorites, comments and notes you created in steps 1 and 3 should be combined together in the record view. |
| |
| ==== Record Module ==== | ==== Transaction History ==== |
| |
| * Test "Cite This" functionality. | VuFind® allows users to view (and in some cases purge) their transaction history, but this functionality is disabled by default. To see how it behaves, follow these steps: |
| * Test "Text This" functionality. | |
| * Test "Email This" functionality. | |
| * Test RefWorks/EndNote/Zotero interface. | |
| * Test "Add to Favorites" functionality. | |
| |
| ==== Search Module ==== | - In config.ini, in the [Catalog] section, set driver to "Demo". |
| | - In Demo.ini, in the [TransactionHistory] section, set enabled to true. |
| | - After creating an account and logging in, a new option should now be available in the account side menu to allow access to transaction history. The Demo driver will populate this with simulated data (and you can log out and back in again to generate a new set of simulated data). |
| |
| * Test sorting and paging of results. | |
| * Test that all special searches (author, title, etc.) are actually searching the correct set of index fields. | |
| * Test that adding and removing facet filters from the result set works correctly. | |
| * Doing an author search should display a box of author facets at the top of the results screen with links to the Author module. | |
| ---- struct data ---- | ---- struct data ---- |
| properties.Page Owner : | properties.Page Owner : |
| ---- | ---- |
| |
