Differences

This shows you the differences between two versions of the page.

--- videos:testing_3 [2021/10/22 14:25] – created demiankatz
+++ videos:testing_3 [2021/10/22 18:59] (current) – [Transcript] demiankatz
@@ Line 1: / Line 1: @@
-====== Video 16: Testing, Part 3: Integration Testing with Mink ======
+====== Video 18: Testing, Part 3: Integration Testing with Mink ======
-The eighteenth VuFind instructional video continues the discussion of testing, explaining how VuFind's test suite can use a real browser to test functionality with the help of the [[https://mink.behat.org/en/latest/|Mink]] library.
+The eighteenth VuFind® instructional video continues the discussion of testing, explaining how VuFind's test suite can use a real browser to test functionality with the help of the [[https://mink.behat.org/en/latest/|Mink]] library.
 Video is available as an [[https://vufind.org/video/Testing3.mp4|mp4 download]] or through [[https://www.youtube.com/watch?v=r-zVSg7FAPI&feature=youtu.be|YouTube]].
@@ Line 9: / Line 9: @@
   - [[development:testing:unit_tests|Unit Testing Documentation]]
   - [[https://mink.behat.org/en/latest/|Mink Documentation]]
+  - [[https://github.com/vufind-org/vufind/commit/cdd7ca133e301a8f70c43d7e0a0f0010a7d9af0a#diff-d4ab68ec7841636ccd1a77171f410ed66b9d6d52e5befe4a3bfb1518becbc987|Code Developed During the Video]]
 ===== Transcript =====
@@ Line 14: / Line 15: @@
 // This is a machine-generated transcript and will be corrected as time permits. //
+ hello and welcome to the third
+viewfinder video on testing
+uh last time we talked about unit
+testing and this time we're going to
+talk about integration testing actually
+making sure that the whole system works
+when put together
+with the help of automated browser
+testing
+and we're going to do automated browser
+testing with the help of mink
+mink is a library produced to support
+the b hat project b hat is a behavior
+driven development testing framework
+that i personally don't use but
+fortunately they shared their low-level
+browser integration as a separate
+project that can be integrated with all
+kinds of testing
+and so the viewfind project takes
+advantage of this
+to allow browser automation from within
+our php unit suite uh today i'm going to
+walk through some of our mink code and
+show you the process i use for writing
+new mink tests so perhaps you can help
+us increase our coverage by adding to
+our overall integration suite
+so let's start with a quick tour of the
+code
+if you go into the view find module of
+the code
+under the test directory there's an
+integration tests directory and under
+there
+there is a
+mink subdirectory with a whole bunch of
+tests in it
+let's start by looking at basic test
+which is one of our simpler test cases
+and just like any other php unit test
+you're going to see that this is just a
+class whose name ends in test which
+includes a whole bunch of methods that
+begin with test
+so
+the first test here uh is called test
+home page and this just checks that uh
+the word viewfind appears when we open
+the viewfind home page
+in a web browser
+so let's look at how this works
+first of all let me highlight the fact
+that basic test extends
+viewfind test integration mink test case
+which is a
+base class that includes a whole bunch
+of viewfind specific helper methods
+wrapped around mink code
+which make common tasks a little bit
+more convenient
+it helps make the tests less verbose
+and saves you some time coding so we'll
+look at that in a little more detail
+later but first let's get into this test
+so first of all there's this uh called a
+get mink session and this sets up mink
+and gives us a session object which is
+what allows us to control the browser
+that we're using for testing
+the session object has a visit method
+which we can send a url to
+in order to navigate to a particular
+page
+so again this is taking advantage of
+another uh helper method in the base
+class called get viewfind url which
+gives us the base url of the current
+running test environment
+and then just goes to the search home
+route under that base url
+uh once a the session has visited a page
+we can call the getpage method to get
+hold of a page object which represents
+the content of the page
+and from there we can then make
+assertions about that content and
+interact with the page in other ways
+so as i mentioned this first test is
+really simple
+we're just
+getting all the content of the page
+and asserting that
+somewhere in that content is the word
+viewfind
+very simple test
+so let's look at a slightly more complex
+one this is test ajax status
+and what this does is test that the
+ajax routines that fill in uh item
+availability are working correctly
+and it does this by just navigating to
+search results and then checking that
+expected values are present so let's
+step through this starts off exactly the
+same as the last test
+we get the mink session we go to the
+home page we grab the page object so we
+can interact
+but now we do a little bit more
+first of all we call find css which is
+another help helper method from the base
+class
+what this does is it takes a page object
+that we want to search and actually it
+doesn't have to be a page object it
+could be any page element
+it searches within that element for a
+css selector
+in this case we're looking for the
+search input box by its id search form
+underscore look for
+and then if the selector cannot be found
+it will throw an exception
+otherwise it's going to return a
+reference to an object matching that
+selector
+so in this case we get back
+the search box object
+and then on that object we call set
+value which allows us to actually
+interact with the page and fill in that
+text box with a search query in this
+case we're just retrieving one of the
+specific test records that were indexed
+as part of the setup of the test
+environment
+then we use another helper method from
+the base class click css
+which actually just calls find css and
+then clicks on the thing that it finds
+so in this case we're grabbing the
+primary button on the page which in this
+instance should be the submit button for
+the search form and we're clicking it
+we then call another helper method this
+one's called snooze
+and all this does is wait for a second
+uh because browser-based testing is kind
+of timing dependent a lot of these tests
+have to pause now and then to make sure
+that the browser has time to do all of
+the things that we want it to do
+in this case clicking the button is
+going to cause the next page to load
+and then on the next page
+it needs to make the ajax call and fill
+in some details we just want to be sure
+that all has happened before we make any
+assertions about what's going on
+so again after the snooze we finish off
+our test with some assertions
+by default viewfinder is set up with a
+sample driver which always returns this
+call number of a1234567
+and a location of third floor main
+library so
+we just want to make sure that those
+values appear in the places where we
+expect them to
+so since we did a search for a single
+record
+we can be confident that there's only
+going to be one call number and one
+location element
+on the resulting screen
+so we just use find css
+to retrieve those elements and get text
+to pull out their content as plain text
+make assertions about it and we've
+tested that ajax works because if the
+ajax call failed these values that are
+loaded through the ios driver call
+wouldn't be there
+uh and there are a few more tests here
+i'm not going to go through all of them
+in great detail but they use the same
+basic elements
+uh as
+as before
+uh so you can read them on your own time
+if you want to learn a little bit
+more
+having sort of introduced what these
+test classes look like i also wanted to
+talk about
+some shared characteristics of all of
+these tests
+they're all designed in a way
+where
+tests early in the suite can potentially
+impact tests later in the suite so for
+example there are a lot of tests where
+the first test
+will create a user account
+and then subsequent tests will log into
+that account and make use of it
+and that's okay that's something that we
+do by convention
+uh in our test suites it's useful to be
+able to test a number of things that
+depend on one another
+however
+every test class is designed to stand
+alone
+so if a test creates data in the
+database like user accounts it is also
+designed to get rid of all of that data
+when it finishes running
+that way it's possible to run any of
+these test classes in isolation if you
+want to test a particular feature
+without having to worry that it has some
+kind of a dependency on some other test
+it keeps things simple and isolated
+so those are the two design principles
+to keep in mind it's okay to have side
+effects within a test
+uh
+in a test class but it is not okay for
+test classes to interact with each other
+because they should all stand alone
+uh and many of these tests actually
+check and make sure that they're being
+run in a clean empty environment uh and
+will fail if they find say existing user
+data in the database
+that just helps keep things predictable
+so now that we've looked at
+an existing test class we've talked
+about some general principles uh let's
+go ahead and look at that uh base mink
+test case class
+uh just to see what that looks like
+that's in the viewfind module source
+directory under viewfind test
+integration
+link test case
+and as you can see this is an abstract
+class it's only meant to be extended
+when we build specific tests
+and it builds upon the standard php unit
+framework test case
+it also uses a couple of feature traits
+that may also be shared by other tests
+the auto retry trait
+which allows it to look for retry
+annotations in your test comments
+and if it finds a numeric retry
+trait
+or rather annotation
+it will
+retry tests that fail a few times
+and this is useful for the mink test
+because as i mentioned sometimes because
+they're so timing dependent something
+like a network hiccup will cause a test
+to randomly fail but if you just try it
+over it will succeed
+so the auto retry trait
+prevents arbitrary circumstances from
+making your test suite fail
+also used here is the live detection
+trait
+and this detects whether you've used
+thing to set up a test environment if
+there's no test environment active we
+can't run mink tests
+we need to be able to detect that and
+fail the tests immediately if there's no
+environment to test with
+uh and i'm not going to go through all
+the contents of this file i just want to
+highlight a few useful
+helpers
+many of which i already showed you when
+i went through the basic test
+here's one that's great uh change
+configs
+will take uh an array of configuration
+settings that we want to change
+and actually
+rewrite the any files in the test
+environment for the duration of the test
+this way if you want to test a feature
+that's not turned on by default
+you can change the configuration
+dynamically and test the feature
+and this is designed in such a way that
+at the end of each test all the
+configurations you changed will revert
+back to their default versions so this
+automatically lets you test things
+within the context of a single test and
+not have to worry about later side
+effects
+and this is used in quite a few places
+here is the snooze method i mentioned
+that just waits for a certain period of
+time
+you'll notice that there's an
+environment variable here called if you
+find snooze multiplier
+by default snooze is just going to wait
+one second
+but you can set this environment
+variable to change the amount of time it
+waits for
+so you could use a fractional value if
+you want to make it faster if you're
+confident that your environment is just
+going to respond immediately
+or you can make it a larger number if
+you're working in a resource constrained
+environment and you want to slow down
+the tests to be completely sure that
+they pass
+and then
+you know there are many things here
+i'm not going to go through all of them
+but you might want to review this on
+your own time just to see if there's
+anything here that might help you
+here's the find css method i mentioned
+that
+uh
+waits to see if things exist and then
+select them and will
+fail if
+if the selector fails to match anything
+click css
+and so forth
+there's a lot of useful stuff in here
+uh it saves you some time to extend this
+class instead of reinventing the wheel
+the other thing
+is at the beginning of this i showed you
+that we
+leverage a couple of
+feature traits
+that viewfind provides but these two are
+not the only ones
+if you look in the
+feature directory
+inside the viewfind test namespace
+you'll see that we have quite a few
+traits that you can mix into your tests
+that will give you useful common
+functionality
+that might save you some time
+of particular interest for mink testing
+are the live database trait which gives
+you access to the real my sequel
+database if you want to create or
+destroy data
+and the live not the live detection
+trait the um
+user creation trait
+which allows you to
+quickly
+create users by filling in the create
+account form
+and log in with them so if you have a
+test that requires you to create users
+and log in and out
+this trait just does those things for
+you so you don't have to reinvent that
+code
+so obviously
+this is no replacement for a mink
+tutorial
+but hopefully it shows you that if you
+look at the existing tests and these
+traits in base class
+you can get a feel for how the testing
+works and you should be able to make
+some tests of your own
+so
+for now let me show you the process i go
+through to write a new test
+if you recently had a new feature added
+where public lists now show an icon to
+indicate that they're public
+and also display a message on some
+screens just to make it more obvious to
+users which of their lists are public
+and which are private
+we don't currently have any test
+coverage for this new feature so i'm
+going to work through writing that test
+just to show you how you can approach
+this
+so first of all
+i know that there's an existing test
+suite that
+applies to this kind of functionality
+the favorites test which has all kinds
+of tests related to favorites and
+favorite lists
+and
+this as i said we can have tests where
+the tests all build upon each other
+uh so this already creates some lists
+and populates them it even makes one of
+them public
+so what i think makes the most sense is
+to just add a test to this existing test
+suite
+which uh checks that once we have public
+and private lists in place
+they display the way we expect them to
+so if i just search the code for public
+list
+i find a comment here
+that there's a test called test email
+public list
+and if i look through this code i can
+see that one of the things that it's
+doing
+is making a list public
+so
+what i think i want to do is just add
+another test following this one
+that will test
+that the the icon appears where i expect
+it to etc
+uh also note
+that this test has some dependencies
+documented through the um
+depends annotation
+which is how we show when one of our
+tests relies on side effects of an
+earlier test
+so
+before i can actually start writing the
+code of my test
+it makes sense for me to
+do a little bit of reconnaissance and
+collect some relevant information that
+i'm going to need
+while i do my my test building
+so
+the easiest way to do that is to just
+log into my test instance of view find
+and play around a little bit
+before i started recording this video i
+already ran the startup task so i have a
+running test instance
+here in
+localhost
+you find underscore test
+so the quickest way to find out what i
+need is i'm just going to create
+an account
+and
+i'm going to do a blank search
+and i'm just going to create
+a couple of lists i'll call one open
+which will be my public list
+and i'll call one closed which will be
+my private list
+and so now i'm just going to go into my
+account screen here and you can see this
+is exactly what i'm looking to test my
+public list
+has an icon next to it my private list
+does not
+so i just need to collect a little bit
+of information about what i am testing
+for
+so i open up my handy browser console by
+hitting f12
+and i'm just going to use my element
+inspector tool
+to take a look at what exactly is going
+on here so i can see that all of my
+lists here have a class of user list
+link on them which is going to be a
+useful selector for extracting
+information from the page object in my
+test
+and if i open up this
+second link the open one
+i can see that it has an icon
+which is an i with the class of f a f a
+globe on it so i take note of these two
+things
+and this is going to help me when i
+write my test
+and also if i click on these lists if i
+go to the public list
+i see that it says public list right
+here up at the top
+i'm going to inspect that as well
+and so this is just a strong element
+with the fa globe icon inside it so
+again that's a distinctive selector that
+i should be able to test for
+and i can see that if i click over to my
+private list
+that public
+indicator is not present at the top
+so i think this little investigation
+gives me a strategy all i need to do to
+test that this is working the way i want
+it to
+uh is to go to the user account screen
+look at the list of lists
+uh check that the public ones have the
+public icon the private ones do not
+and then i can click into a public list
+make sure that it shows that public
+message and also click into a private
+list and
+verify that it does not have the public
+message i think that covers all of our
+needs here so now i just need to write
+that as code
+however
+during the course of this testing i've
+created some data in the database
+and that's going to prevent my tests
+from running because as i mentioned
+before a lot of the tests want to verify
+that they have a clean and empty
+environment
+before they run so
+i am very going very quickly going to
+clean up after myself i'm just going to
+go to the command line
+and log into my viewfind test database
+and i can just run delete from user that
+wipes out my account and all associated
+data
+now i'm all set to run my test once i
+have written it
+but of course now the hard part is
+actually
+writing the test
+so obviously i'm not going to do all of
+this development live while i'm
+recording a video i did some of this
+work in advance
+uh so i am just going to type out the
+code and explain it as i go
+but before i do that let me talk a
+little bit about how i approached this
+um
+first of all it can be a little
+frustrating to add a test to a long and
+complicated test suite like this one
+because it takes three to five minutes
+to run so every time you write your code
+and you want to test it
+you have to sit through the whole test
+running
+except that you don't actually have to
+sit through the whole thing
+as i noticed here
+the the test that my test is going to
+depend upon
+only depends on two other specific tests
+and they in turn may have some
+dependencies but there are also a number
+of tests here that are irrelevant to
+what i'm doing
+i find it's it's sometimes helpful uh
+during testing to just put a z in front
+of the
+function name of all of the tests that i
+don't want to run
+because php unit will only run methods
+that start with test so this effectively
+disables them
+so when i'm developing i'll actually
+z out almost all of my tests and then
+just re-enable the ones that i need to
+get to where i need to be
+it saves a lot of time while you're
+iterating through
+uh there are also some more formal ways
+that you can do this within the php unit
+framework
+um there is a filter option you can send
+to phpunit that will allow you to run
+just one specific method
+but that's not helpful if your method
+has dependencies on others
+there's also a group option
+where you can add a group annotation
+like
+group my group
+to particular tests and then when you
+run php unit you can specify minus minus
+group and a group name and it will only
+run tests in that group so there are a
+variety of ways you can control which
+tests you run
+when you're iterating through your test
+writing
+but i tend to just do the manual z out
+the unwanted tests approach
+in any case
+let me show you what i came up with
+after i went through that whole
+process
+so since my test uh relies on data that
+was set up by
+test email public list and some of the
+earlier methods
+uh i'm just going to write it
+immediately after that test uh you'll
+notice that the next test is called test
+bulk delete and it actually deletes some
+of the data that i'm going to be working
+with so the order of the tests is very
+important in this example
+so i always
+start my test with the comment
+explaining what the test does so in this
+case test that public list indicator
+appears as expected
+and i'm going to state my dependencies
+here so
+this depends on test email public list
+as i already mentioned
+uh it also depends on test add record
+to favorites
+login which is another earlier test that
+creates some lists
+and i learned as i was testing that i
+would need that
+now i'm going to actually create my
+test function we'll call it test public
+list indicator might as well be
+nice and specific
+can also be fancy and give it a void
+return value annotation since it does
+not return anything
+and so
+i'm actually going to start this by
+using a helper method
+that's defined earlier in this class
+this
+go to user account
+so this simply logs in as a user and
+goes to the account screen uh the
+function is defined up above i'm not
+going to review it now in the interest
+of time but just take my word for it
+that this is going to save us the effort
+of writing all the code to get to the
+screen where we want to start doing our
+testing
+and it just returns this helpful page
+object which gives us access to the
+contents of the user account screen
+now i'm going to write a comment
+explaining what i'm going to do
+collect data about the user list links
+on the page
+we are checking
+for
+expected descriptions
+and icons and we'll want urls so we can
+visit links
+individually
+in other words what i'm going to do
+is first i'm going to extract
+all of the
+user
+list link elements from the page
+so
+first of all
+note that i am using
+page find all which is actually
+part of mink
+so i just want to find all elements
+within the page that we got by accessing
+the user account the first parameter
+here tells what kind of method i'm using
+to find elements i'm using a css
+selector you could also do things like
+search by the text within the element or
+or other things but css selectors are
+are commonly
+the most useful thing
+and then i'm just grabbing user list
+link which is the css selector i came up
+with earlier when i was examining the
+page
+so this is going to give me
+an array of all of the links on the page
+and i'm just going to collect some
+information out of those links so that i
+can make assertions about it later
+so i'm going to initialize a couple of
+arrays
+a data array which will describe all the
+links i find and an hrefs array which
+will collect all the links because we're
+going to want to go to some of these
+individual pages and check them for
+the public message
+so now i iterate through all my links
+using it for each loop
+and i just pull out the data i want so
+each element of my data array is going
+to be another array which has two
+elements i'm going to pull the text
+out of the link so i can just use
+link get text which is another mink
+method it just returns the textual
+context of content of an element
+and i'm going to get a count of how many
+icons are inside the link
+so i'm just going to do count link
+find all
+again i'm going to use a css selector
+i dot fa globe so this is the other
+thing that i figured out while i was
+inspecting the page earlier
+so as you can see
+uh the way mink works is it
+every element is searchable and can
+return more elements so i got my page at
+the beginning
+on the page i found all my links each
+one of these links is an element which
+has the same interface as the page
+itself so i can search within them
+so i'm just looping through all my links
+getting their text and then searching
+all of them for icons and counting how
+many i find so this is going to give me
+a handy array that summarizes all of my
+links what's their text how many icons
+do they have
+and while i'm in here i'm also going to
+add the the href target of each of these
+links to my array
+so i can do that by just saying link get
+attribute
+href and again get attribute is just
+part of the mink framework makes it easy
+to pull attribute values out of elements
+so now
+what do i expect
+i'm going to create an expected data
+array which i'm going to use in my
+assertion
+and this is just what i expect i should
+have
+and i learned these values by running
+the test and seeing what what is done
+i'm sparing you watching me do that so
+i'm not getting these values by magic i
+got these by actually running the test
+and
+seeing what the other tests created
+but
+the important thing
+is that the previous tests have created
+three links three lists
+um
+two of them are private and one of them
+is public
+so
+the result i expect from all of my work
+above is that i have an array
+with
+these three named lists the first two
+have no icons the third one which is
+public and even has public list in its
+textual description has one icon
+so now all i need to do is make an
+assertion that this is the case assert
+equals
+expected data data so
+this just says the result of this loop
+should be exactly this stuff
+and now the last thing to do
+is to check those two list pages
+uh for the presence or absence of the
+icon so first of all the future list
+should not be public
+so what i'm going to do is i'm going to
+click
+on
+the link
+to
+this
+list
+the way i'm doing this
+is with a fancy css selector
+so as i mentioned i created this array
+of urls up here
+as i loop through the links
+i know that the future list is the first
+one because it's the first one in my
+list of expected data here and i know
+that this is a private list because my
+icon count is zero
+so
+i just take the first
+url and i use that to select the link
+that has that as an href and click on it
+then because it might take a moment
+to process i'm going to snooze
+and then i'm going to
+assert
+that
+there are no strong uh
+strong elements with globe icons in them
+on this page so page
+find all
+css
+strong i dot fa globe
+so this just
+searches the page for strong elements
+with globes in them counts them asserts
+that there aren't any
+and
+finally i'm going to copy and paste this
+code because i only need to change it
+slightly from my last test
+the test list
+should be public
+so
+fortunately because this menu of list is
+on every single page
+i can be confident that this selector is
+going to work even though we've moved to
+a new page
+i just want to click now
+on
+the third list instead of the first list
+and indexing starts at zero so that's
+hrefs2
+and i want to assert that there is going
+to be exactly one globe here rather than
+zero
+and that's my test
+so now all that's left to do is run it
+and make sure that it passes
+so i'm going to go to the terminal
+and i'm going to use the chrome test
+mode script that i uh created in our
+first testing video to start up an
+instance of chrome that i can actually
+watch running while the tests run
+so there that's ready to go
+i'm going to
+source my test environment script to
+make sure all of my variables are
+correct because i'm going to use
+viewfind home now
+and so now i just run thing.sh
+php unit faster if anything fails i want
+it to stop immediately
+and i'm going to use the minus d php
+unit extra param switch to tell phpunit
+exactly which test to run
+which in this case is going to be
+viewfind home module
+viewfind tests integration tests
+source viewfind test make
+favorites test
+so this will run just the test i've
+modified we'll have to wait a moment
+because it's going to run the whole
+suite which includes
+uh all of the existing tests as well as
+the one i've just added
+this is why i mentioned earlier when i
+was developing the test i disabled a
+number of these to save some time but as
+you can see we've created an account
+we're starting to
+build some lists and test other features
+of them
+so now i'll just sit back and watch as
+the magic happens
+now we've created i believe all three of
+the lists that are going to get tested
+just doing some things within the
+various lists
+now we're creating a second user account
+because some of the tests
+check that you know user two can access
+public lists from user one but not
+private lists so forth
+we can do some fast forwarding during
+the video editing over this portion
+here we're making a list public so our
+test is coming up very soon
+we just have to get through this
+emailing test all right i believe this
+is it
+so we're logging in
+we're switching between lists
+and i think we're done it went very by
+very quickly
+but it appears that uh all of the new
+tests have passed so we just have to
+wait for the remaining tests to finish
+up
+and we're done all tests have passed we
+have successfully written a new
+integration test
+i realized this was a whirlwind tour and
+there's a lot more you could learn but i
+hope this has been a helpful
+introduction and i'd love to see some
+new tests contributed especially if you
+plan on adding new functionality
+thanks for taking the time to watch and
+i'll have more content soon
+talk to you later
 ---- struct data ----
+properties.Page Owner :
 ----