The eighteenth VuFind® instructional video continues the discussion of testing, explaining how VuFind's test suite can use a real browser to test functionality with the help of the Mink library.

Video is available as an mp4 download or through YouTube.

1. Unit Testing Documentation
2. Mink Documentation
3. Code Developed During the Video

This is a machine-generated transcript and will be corrected as time permits.

hello and welcome to the third viewfinder video on testing uh last time we talked about unit testing and this time we're going to talk about integration testing actually making sure that the whole system works when put together with the help of automated browser testing and we're going to do automated browser testing with the help of mink mink is a library produced to support the b hat project b hat is a behavior driven development testing framework that i personally don't use but fortunately they shared their low-level browser integration as a separate project that can be integrated with all kinds of testing and so the viewfind project takes advantage of this to allow browser automation from within our php unit suite uh today i'm going to walk through some of our mink code and show you the process i use for writing new mink tests so perhaps you can help us increase our coverage by adding to our overall integration suite so let's start with a quick tour of the code if you go into the view find module of the code under the test directory there's an integration tests directory and under there there is a mink subdirectory with a whole bunch of tests in it let's start by looking at basic test which is one of our simpler test cases and just like any other php unit test you're going to see that this is just a class whose name ends in test which includes a whole bunch of methods that begin with test so the first test here uh is called test home page and this just checks that uh the word viewfind appears when we open the viewfind home page in a web browser so let's look at how this works first of all let me highlight the fact that basic test extends viewfind test integration mink test case which is a base class that includes a whole bunch of viewfind specific helper methods wrapped around mink code which make common tasks a little bit more convenient it helps make the tests less verbose and saves you some time coding so we'll look at that in a little more detail later but first let's get into this test so first of all there's this uh called a get mink session and this sets up mink and gives us a session object which is what allows us to control the browser that we're using for testing the session object has a visit method which we can send a url to in order to navigate to a particular page so again this is taking advantage of another uh helper method in the base class called get viewfind url which gives us the base url of the current running test environment and then just goes to the search home route under that base url uh once a the session has visited a page we can call the getpage method to get hold of a page object which represents the content of the page and from there we can then make assertions about that content and interact with the page in other ways so as i mentioned this first test is really simple we're just getting all the content of the page and asserting that somewhere in that content is the word viewfind very simple test so let's look at a slightly more complex one this is test ajax status and what this does is test that the ajax routines that fill in uh item availability are working correctly and it does this by just navigating to search results and then checking that expected values are present so let's step through this starts off exactly the same as the last test we get the mink session we go to the home page we grab the page object so we can interact but now we do a little bit more first of all we call find css which is another help helper method from the base class what this does is it takes a page object that we want to search and actually it doesn't have to be a page object it could be any page element it searches within that element for a css selector in this case we're looking for the search input box by its id search form underscore look for and then if the selector cannot be found it will throw an exception otherwise it's going to return a reference to an object matching that selector so in this case we get back the search box object and then on that object we call set value which allows us to actually interact with the page and fill in that text box with a search query in this case we're just retrieving one of the specific test records that were indexed as part of the setup of the test environment then we use another helper method from the base class click css which actually just calls find css and then clicks on the thing that it finds so in this case we're grabbing the primary button on the page which in this instance should be the submit button for the search form and we're clicking it we then call another helper method this one's called snooze and all this does is wait for a second uh because browser-based testing is kind of timing dependent a lot of these tests have to pause now and then to make sure that the browser has time to do all of the things that we want it to do in this case clicking the button is going to cause the next page to load and then on the next page it needs to make the ajax call and fill in some details we just want to be sure that all has happened before we make any assertions about what's going on so again after the snooze we finish off our test with some assertions by default viewfinder is set up with a sample driver which always returns this call number of a1234567 and a location of third floor main library so we just want to make sure that those values appear in the places where we expect them to so since we did a search for a single record we can be confident that there's only going to be one call number and one location element on the resulting screen so we just use find css to retrieve those elements and get text to pull out their content as plain text make assertions about it and we've tested that ajax works because if the ajax call failed these values that are loaded through the ios driver call wouldn't be there uh and there are a few more tests here i'm not going to go through all of them in great detail but they use the same basic elements uh as as before uh so you can read them on your own time if you want to learn a little bit more having sort of introduced what these test classes look like i also wanted to talk about some shared characteristics of all of these tests they're all designed in a way where tests early in the suite can potentially impact tests later in the suite so for example there are a lot of tests where the first test will create a user account and then subsequent tests will log into that account and make use of it and that's okay that's something that we do by convention uh in our test suites it's useful to be able to test a number of things that depend on one another however every test class is designed to stand alone so if a test creates data in the database like user accounts it is also designed to get rid of all of that data when it finishes running that way it's possible to run any of these test classes in isolation if you want to test a particular feature without having to worry that it has some kind of a dependency on some other test it keeps things simple and isolated so those are the two design principles to keep in mind it's okay to have side effects within a test uh in a test class but it is not okay for test classes to interact with each other because they should all stand alone uh and many of these tests actually check and make sure that they're being run in a clean empty environment uh and will fail if they find say existing user data in the database that just helps keep things predictable so now that we've looked at an existing test class we've talked about some general principles uh let's go ahead and look at that uh base mink test case class uh just to see what that looks like that's in the viewfind module source directory under viewfind test integration link test case and as you can see this is an abstract class it's only meant to be extended when we build specific tests and it builds upon the standard php unit framework test case it also uses a couple of feature traits that may also be shared by other tests the auto retry trait which allows it to look for retry annotations in your test comments and if it finds a numeric retry trait or rather annotation it will retry tests that fail a few times and this is useful for the mink test because as i mentioned sometimes because they're so timing dependent something like a network hiccup will cause a test to randomly fail but if you just try it over it will succeed so the auto retry trait prevents arbitrary circumstances from making your test suite fail also used here is the live detection trait and this detects whether you've used thing to set up a test environment if there's no test environment active we can't run mink tests we need to be able to detect that and fail the tests immediately if there's no environment to test with uh and i'm not going to go through all the contents of this file i just want to highlight a few useful helpers many of which i already showed you when i went through the basic test here's one that's great uh change configs will take uh an array of configuration settings that we want to change and actually rewrite the any files in the test environment for the duration of the test this way if you want to test a feature that's not turned on by default you can change the configuration dynamically and test the feature and this is designed in such a way that at the end of each test all the configurations you changed will revert back to their default versions so this automatically lets you test things within the context of a single test and not have to worry about later side effects and this is used in quite a few places here is the snooze method i mentioned that just waits for a certain period of time you'll notice that there's an environment variable here called if you find snooze multiplier by default snooze is just going to wait one second but you can set this environment variable to change the amount of time it waits for so you could use a fractional value if you want to make it faster if you're confident that your environment is just going to respond immediately or you can make it a larger number if you're working in a resource constrained environment and you want to slow down the tests to be completely sure that they pass and then you know there are many things here i'm not going to go through all of them but you might want to review this on your own time just to see if there's anything here that might help you here's the find css method i mentioned that uh waits to see if things exist and then select them and will fail if if the selector fails to match anything click css and so forth there's a lot of useful stuff in here uh it saves you some time to extend this class instead of reinventing the wheel the other thing is at the beginning of this i showed you that we leverage a couple of feature traits that viewfind provides but these two are not the only ones if you look in the feature directory inside the viewfind test namespace you'll see that we have quite a few traits that you can mix into your tests that will give you useful common functionality that might save you some time of particular interest for mink testing are the live database trait which gives you access to the real my sequel database if you want to create or destroy data and the live not the live detection trait the um user creation trait which allows you to quickly create users by filling in the create account form and log in with them so if you have a test that requires you to create users and log in and out this trait just does those things for you so you don't have to reinvent that code so obviously this is no replacement for a mink tutorial but hopefully it shows you that if you look at the existing tests and these traits in base class you can get a feel for how the testing works and you should be able to make some tests of your own so for now let me show you the process i go through to write a new test if you recently had a new feature added where public lists now show an icon to indicate that they're public and also display a message on some screens just to make it more obvious to users which of their lists are public and which are private we don't currently have any test coverage for this new feature so i'm going to work through writing that test just to show you how you can approach this so first of all i know that there's an existing test suite that applies to this kind of functionality the favorites test which has all kinds of tests related to favorites and favorite lists and this as i said we can have tests where the tests all build upon each other uh so this already creates some lists and populates them it even makes one of them public so what i think makes the most sense is to just add a test to this existing test suite which uh checks that once we have public and private lists in place they display the way we expect them to so if i just search the code for public list i find a comment here that there's a test called test email public list and if i look through this code i can see that one of the things that it's doing is making a list public so what i think i want to do is just add another test following this one that will test that the the icon appears where i expect it to etc uh also note that this test has some dependencies documented through the um depends annotation which is how we show when one of our tests relies on side effects of an earlier test so before i can actually start writing the code of my test it makes sense for me to do a little bit of reconnaissance and collect some relevant information that i'm going to need while i do my my test building so the easiest way to do that is to just log into my test instance of view find and play around a little bit before i started recording this video i already ran the startup task so i have a running test instance here in localhost you find underscore test so the quickest way to find out what i need is i'm just going to create an account and i'm going to do a blank search and i'm just going to create a couple of lists i'll call one open which will be my public list and i'll call one closed which will be my private list and so now i'm just going to go into my account screen here and you can see this is exactly what i'm looking to test my public list has an icon next to it my private list does not so i just need to collect a little bit of information about what i am testing for so i open up my handy browser console by hitting f12 and i'm just going to use my element inspector tool to take a look at what exactly is going on here so i can see that all of my lists here have a class of user list link on them which is going to be a useful selector for extracting information from the page object in my test and if i open up this second link the open one i can see that it has an icon which is an i with the class of f a f a globe on it so i take note of these two things and this is going to help me when i write my test and also if i click on these lists if i go to the public list i see that it says public list right here up at the top i'm going to inspect that as well and so this is just a strong element with the fa globe icon inside it so again that's a distinctive selector that i should be able to test for and i can see that if i click over to my private list that public indicator is not present at the top so i think this little investigation gives me a strategy all i need to do to test that this is working the way i want it to uh is to go to the user account screen look at the list of lists uh check that the public ones have the public icon the private ones do not and then i can click into a public list make sure that it shows that public message and also click into a private list and verify that it does not have the public message i think that covers all of our needs here so now i just need to write that as code however during the course of this testing i've created some data in the database and that's going to prevent my tests from running because as i mentioned before a lot of the tests want to verify that they have a clean and empty environment before they run so i am very going very quickly going to clean up after myself i'm just going to go to the command line and log into my viewfind test database and i can just run delete from user that wipes out my account and all associated data now i'm all set to run my test once i have written it but of course now the hard part is actually writing the test so obviously i'm not going to do all of this development live while i'm recording a video i did some of this work in advance uh so i am just going to type out the code and explain it as i go but before i do that let me talk a little bit about how i approached this um first of all it can be a little frustrating to add a test to a long and complicated test suite like this one because it takes three to five minutes to run so every time you write your code and you want to test it you have to sit through the whole test running except that you don't actually have to sit through the whole thing as i noticed here the the test that my test is going to depend upon only depends on two other specific tests and they in turn may have some dependencies but there are also a number of tests here that are irrelevant to what i'm doing i find it's it's sometimes helpful uh during testing to just put a z in front of the function name of all of the tests that i don't want to run because php unit will only run methods that start with test so this effectively disables them so when i'm developing i'll actually z out almost all of my tests and then just re-enable the ones that i need to get to where i need to be it saves a lot of time while you're iterating through uh there are also some more formal ways that you can do this within the php unit framework um there is a filter option you can send to phpunit that will allow you to run just one specific method but that's not helpful if your method has dependencies on others there's also a group option where you can add a group annotation like group my group to particular tests and then when you run php unit you can specify minus minus group and a group name and it will only run tests in that group so there are a variety of ways you can control which tests you run when you're iterating through your test writing but i tend to just do the manual z out the unwanted tests approach in any case let me show you what i came up with after i went through that whole process so since my test uh relies on data that was set up by test email public list and some of the earlier methods uh i'm just going to write it immediately after that test uh you'll notice that the next test is called test bulk delete and it actually deletes some of the data that i'm going to be working with so the order of the tests is very important in this example so i always start my test with the comment explaining what the test does so in this case test that public list indicator appears as expected and i'm going to state my dependencies here so this depends on test email public list as i already mentioned uh it also depends on test add record to favorites login which is another earlier test that creates some lists and i learned as i was testing that i would need that now i'm going to actually create my test function we'll call it test public list indicator might as well be nice and specific can also be fancy and give it a void return value annotation since it does not return anything and so i'm actually going to start this by using a helper method that's defined earlier in this class this go to user account so this simply logs in as a user and goes to the account screen uh the function is defined up above i'm not going to review it now in the interest of time but just take my word for it that this is going to save us the effort of writing all the code to get to the screen where we want to start doing our testing and it just returns this helpful page object which gives us access to the contents of the user account screen now i'm going to write a comment explaining what i'm going to do collect data about the user list links on the page we are checking for expected descriptions and icons and we'll want urls so we can visit links individually in other words what i'm going to do is first i'm going to extract all of the user list link elements from the page so first of all note that i am using page find all which is actually part of mink so i just want to find all elements within the page that we got by accessing the user account the first parameter here tells what kind of method i'm using to find elements i'm using a css selector you could also do things like search by the text within the element or or other things but css selectors are are commonly the most useful thing and then i'm just grabbing user list link which is the css selector i came up with earlier when i was examining the page so this is going to give me an array of all of the links on the page and i'm just going to collect some information out of those links so that i can make assertions about it later so i'm going to initialize a couple of arrays a data array which will describe all the links i find and an hrefs array which will collect all the links because we're going to want to go to some of these individual pages and check them for the public message so now i iterate through all my links using it for each loop and i just pull out the data i want so each element of my data array is going to be another array which has two elements i'm going to pull the text out of the link so i can just use link get text which is another mink method it just returns the textual context of content of an element and i'm going to get a count of how many icons are inside the link so i'm just going to do count link find all again i'm going to use a css selector i dot fa globe so this is the other thing that i figured out while i was inspecting the page earlier so as you can see uh the way mink works is it every element is searchable and can return more elements so i got my page at the beginning on the page i found all my links each one of these links is an element which has the same interface as the page itself so i can search within them so i'm just looping through all my links getting their text and then searching all of them for icons and counting how many i find so this is going to give me a handy array that summarizes all of my links what's their text how many icons do they have and while i'm in here i'm also going to add the the href target of each of these links to my array so i can do that by just saying link get attribute href and again get attribute is just part of the mink framework makes it easy to pull attribute values out of elements so now what do i expect i'm going to create an expected data array which i'm going to use in my assertion and this is just what i expect i should have and i learned these values by running the test and seeing what what is done i'm sparing you watching me do that so i'm not getting these values by magic i got these by actually running the test and seeing what the other tests created but the important thing is that the previous tests have created three links three lists um two of them are private and one of them is public so the result i expect from all of my work above is that i have an array with these three named lists the first two have no icons the third one which is public and even has public list in its textual description has one icon so now all i need to do is make an assertion that this is the case assert equals expected data data so this just says the result of this loop should be exactly this stuff and now the last thing to do is to check those two list pages uh for the presence or absence of the icon so first of all the future list should not be public so what i'm going to do is i'm going to click on the link to this list the way i'm doing this is with a fancy css selector so as i mentioned i created this array of urls up here as i loop through the links i know that the future list is the first one because it's the first one in my list of expected data here and i know that this is a private list because my icon count is zero so i just take the first url and i use that to select the link that has that as an href and click on it then because it might take a moment to process i'm going to snooze and then i'm going to assert that there are no strong uh strong elements with globe icons in them on this page so page find all css strong i dot fa globe so this just searches the page for strong elements with globes in them counts them asserts that there aren't any and finally i'm going to copy and paste this code because i only need to change it slightly from my last test the test list should be public so fortunately because this menu of list is on every single page i can be confident that this selector is going to work even though we've moved to a new page i just want to click now on the third list instead of the first list and indexing starts at zero so that's hrefs2 and i want to assert that there is going to be exactly one globe here rather than zero and that's my test so now all that's left to do is run it and make sure that it passes so i'm going to go to the terminal and i'm going to use the chrome test mode script that i uh created in our first testing video to start up an instance of chrome that i can actually watch running while the tests run so there that's ready to go i'm going to source my test environment script to make sure all of my variables are correct because i'm going to use viewfind home now and so now i just run thing.sh php unit faster if anything fails i want it to stop immediately and i'm going to use the minus d php unit extra param switch to tell phpunit exactly which test to run which in this case is going to be viewfind home module viewfind tests integration tests source viewfind test make favorites test so this will run just the test i've modified we'll have to wait a moment because it's going to run the whole suite which includes uh all of the existing tests as well as the one i've just added this is why i mentioned earlier when i was developing the test i disabled a number of these to save some time but as you can see we've created an account we're starting to build some lists and test other features of them so now i'll just sit back and watch as the magic happens now we've created i believe all three of the lists that are going to get tested just doing some things within the various lists now we're creating a second user account because some of the tests check that you know user two can access public lists from user one but not private lists so forth we can do some fast forwarding during the video editing over this portion here we're making a list public so our test is coming up very soon we just have to get through this emailing test all right i believe this is it so we're logging in we're switching between lists and i think we're done it went very by very quickly but it appears that uh all of the new tests have passed so we just have to wait for the remaining tests to finish up and we're done all tests have passed we have successfully written a new integration test i realized this was a whirlwind tour and there's a lot more you could learn but i hope this has been a helpful introduction and i'd love to see some new tests contributed especially if you plan on adding new functionality thanks for taking the time to watch and i'll have more content soon talk to you later