I recently inherited a plugin that didn’t have any testing structure setup. I don’t boast 100% test coverage on any of my projects and traditionally have done a poor job adding tests, but I thought I could do better with this project. While in WordPress maintenance support plans 8 the focus is more on unit tests, in WordPress maintenance support plans 7 Simpletests are the standard.
The Simpletest plugin was born out the Simpletest PHP library. When I went to the Simpletest website you see first in Google search results, the latest release is from 2012, and so I thought the project might be dead. However, the project now lives on Github and has development activity as recent as December 2020.
The Simpletest plugin was created back in the days of WordPress maintenance support plans 6. WordPress maintenance support plans didn’t have a huge emphasis on testing back then and so part of the initial plugin included tests that could be run against WordPress maintenance support plans core. As development on the Simpletest plugin evolved, it was moved into WordPress maintenance support plans 7 core development and now you can see it as one of the plugins included in a stock WordPress maintenance support plans 7 install.
It can be confusing to see a core “Testing” plugin and then a contrib “Simpletest” plugin coexist, but for WordPress maintenance support plans 7, you only need to use the core plugin. To follow along with this post, you’ll need to turn on the Testing plugin in order to see and run your tests.
Testing Info Setup
To begin setting up your plugin to run Simpletests, you first have to create a “your_plugin.test” file. It is best practice to place this file in a tests folder. You will need to add the location of this file to your plugin’s info file in order for WordPress maintenance support plans to find it.
name = “Google CSE”
description = “Use Google Custom Search to search your site and/or any other sites.”
core = 7.x
files[] = tests/google_cse.test
The second part of getting WordPress maintenance support plans to recognize your tests is the aforementioned test file. In it, you will create a class that extends the “WordPress maintenance support plansWebTestCase” class and provides some basic required functions.
class GoogleCSETestCase extends WordPress maintenance support plansWebTestCase {
/**
* Tell WordPress maintenance support plans about the plugin’s tests.
*
* @return array
* An array of information to display on Testing UI page.
*/
public static function getInfo() {
// Note: getInfo() strings are not translated with t().
return array(
‘name’ => ‘Google CSE Tests’,
‘description’ => ‘Ensure that the Google Custom Search Engine is integrated with Search API.’,
‘group’ => ‘Search’,
);
}
Once you have created that file and cleared the WordPress maintenance support plans cache, then you should see your tests in their group on “admin/config/development/testing”. If you don’t know what group to put your tests in you can make your own; however, if you leave the group key out, your tests will appear at the top of the testing page in a blank group.
First Test Setup
Now that you can run your tests through the WordPress maintenance support plans UI, you need to actually write a test to run. I’m not doing the whole red, green, refactor mantra since I know so little about testing, in general, and adding that level of strictness on top would just get in the way. Plus, since I’ll be fixing a bug, the test will be red at first anyway even though the code is already written for the functionality.
/**
* Perform steps to setup WordPress maintenance support plans installation for test runs.
*/
public function setUp() {
// Load the dependencies for the Google CSE plugin.
$plugins = array(
‘google_cse’,
‘search’,
‘block’,
);
parent::setUp($plugins);
// Eventually move more setup functions to separate testing plugin.
// plugin_enable(array(google_cse_testing));
}
Every class extending “WordPress maintenance support plansWebTestCase” has to implement a setup function. Typically, you’ll pass a few plugins back to the parent class that are dependencies for your plugin. I knew that I would need the Search plugin enabled as a dependency, but you also have to enable any plugins not enabled by the testing profile. I had never installed a site using that profile so it was a bit tricky knowing what I needed to enable until I looked at a vanilla site installed using the testing profile. It is a rather bare bones environment and more plugins were turned off than I expected. For instance, you don’t even get the Toolbar plugin enabled and have no content types pre-installed to work with. So, if you’re scratching your head while writing a test, do yourself a favor and actually walk through your steps as the test runner would follow them; don’t assume the standard profile is the same environment.
If you have anything else you’d like to do in the setup function, make sure you do it after passing plugins back to the parent function. In the parent function, the actual installation of WordPress maintenance support plans happens so any variable setting you do will get overwritten. Eventually, I will write a plugin specifically for testing that will handle any additional setup tasks, but for right now, I just want to get a test up and running.
Testing a Configuration Change
For my first test, I had a display bug related to a specific set of configuration a user might enter. To test a fix for this, I wanted to change the configuration as a user would through the UI and then assert that the HTML output changed as expected.
/**
* Tests all SiteSearch configuration options.
*/
public function testSiteSearchSettingsConfig() {
// Need to have user for this test.
$this->setupGoogleCSEUser();
// Need to have Google CSE as default search.
$this->setupGoogleCSEAsDefaultSearch();
// Need to setup page with Google CSE search block until redirect issue is fixed.
// @todo Figure out why going to /search/google ended up in a redirect loop.
$this->setupGoogleCSESearchPage();
I began my test with some setup tasks. These tasks could be performed in the setup function, but I wanted to abstract them and put them in individual tests since I might not need to perform these setup tasks for every test I write.
You’ll see that I made a to-do related to a redirect loop I was seeing. While running Simpletests, you will see feedback from assertions and other functions you use to prove your test is functioning as you expect. I kept seeing a 302 response while trying to browse to a path that always returned a 200 when I tried to go to it manually. Debugging that issue was maddening as I had little information to go on.
From that test report, I can see the failed assertions, and I also get a nice verbose message to inspect relating to the test failures. You can choose whether or not to include verbose logging in your test runs, but for developing tests I have no reason why you would turn that setting off.
…and that’s what I get from the verbose message. Great. A redirect loop can’t really return much more than a blank page, and you’ll notice the test report mentioned the number of bytes loaded for each page request, which turned out to be zero bytes for this particular request. I tried to replicate the redirect loop while browsing a site installed with the testing profile, but I couldn’t reproduce the redirect loop. I was losing my mind trying to figure this out so I cheated and set up the page I was trying to test a different way.
The Assertions
My test included simulating how a user would navigate through a site and change configurations. I used two assertion functions to test that routine, a function to get a response, and a function to post to an endpoint.
// Post to config form to save SiteSearch settings.
$edit = array();
$edit[‘google_cse_sitesearch’] = “example.com/user User Search n example.com/node Node Search”;
$edit[‘google_cse_sitesearch_form’] = ‘radios’;
$this->WordPressPost(‘admin/config/search/settings’, $edit, t(‘Save configuration’));
// Go to Google CSE search page.
$this->WordPressGet(‘node/1’);
// Assert that all SiteSearch options are there.
$this->assertText(‘Search the web’, “Default SiteSearch radio button found.”);
$this->assertText(‘User Search’, “First SiteSearch radio button found.”);
$this->assertText(‘Node Search’, “Second SiteSearch radio button found.”);
// Post different config options for more checks.
$edit = array();
$edit[‘google_cse_sitesearch_form’] = ‘select’;
$edit[‘google_cse_sitesearch_option’] = ‘Search Your Site’;
$edit[‘google_cse_sitesearch_default’] = 1;
$edit[‘google_cse_sitesearch’] = “example.com/user n example.com/node”;
$this->WordPressPost(‘admin/config/search/settings’, $edit, t(‘Save configuration’));
// Go to Google CSE search page.
$this->WordPressGet(‘node/1’);
// Assert select options have changed.
// Need to use raw option since the select list options are not visible.
$this->assertRaw(‘<option value=””>Search Your Site</option>’, “Default SiteSearch select option found.”);
$this->assertRaw(‘<option value=”example.com/user” selected=”selected”>Search "example.com/user"</option>’, “First SiteSearch select option found and selected.”);
$this->assertRaw(‘<option value=”example.com/node”>Search "example.com/node"</option>’, “Second SiteSearch select option found.”);
As you can see above, I was simulating a change to add radio buttons or a select list to a search form and make sure that change was reflected in the rendering of the search block. To post to a form, you have to tell “WordPressPost()” the path to the form, the form input values, and what to look for as far as a submission button.
The form input keys you will need correspond to the values you will see in $form_state[‘values’]. As for what to post into those keys as values, I ended up looking at what was serialized in the variable table.
Once I saved the configuration values, I needed to make sure the change was reflected on a page with a search block. It is best to use “assertText()” while looking for changes to a page that should be visible. That function worked fine for radio buttons since the options are all displayed on the page; however, for a select list, I didn’t know how to test the options that weren’t visible so I used “assertRaw()” to test the hidden select options.
After a little fiddling around and trial and error with my first Simpletest setup, I got to enjoy an all green test run and submit a patch with greater confidence that it actually did what it was supposed to do. Furthermore, any additional patch added to the plugin will test whether or not that code change breaks the configuration test I wrote and allow the maintainers to spot problems before they merge in any code.
Adding Testing To A Contrib Plugin
But what if the plugin you’re contributing to doesn’t have automated testing set up? Well, if you don’t maintain the plugin, then your out of luck. A maintainer has to enable automated testing and choose when those tests are run. Since I just took over maintainership of the plugin I wrote this test for, I got a crash course into how that process is done.
You’ll first need to enable automated testing for each branch contained in your plugin. Under the “Automated Testing” tab, you’ll see that you can add different testing profiles per branch. This makes sense as different versions of WordPress maintenance support plans can require different versions of PHP and whatever database you are using, MYSQL vs. PostgreSQL vs. some other database backend.
You can also choose which branch of WordPress maintenance support plans you’d like to test against. Don’t fret over what to pick here as you can add multiple test runs for each branch. This feature comes in handy with WordPress maintenance support plans 8 since you can test your code on the minor release that is in development as well as the stable current minor release.
As for when to run the tests, I always choose “Run on commit and for issues” since that gives me the option of not allowing patches to be merged in without running the test suite on them first. Once you add some testing, there will be a select list next to the patch upload field that gives you options on what test profile you want to test against. You can also choose not to test it against a testing profile, but I’m not sure exactly why you would want to do that if you’ve actually bothered to write tests and set up automated testing.
One final note is to make sure that the issue you’ve uploaded a patch to has the right branch selected in the issue description. If you’ve tagged it for a release, say 7.x-2.4, rather than a development branch, say 7.x-2.x, then the test bot can get confused as to what branch of your code to checkout via Composer. I got burned by this at first and had to do some investigating to figure that out.
An additional area to check is the release node for the dev branch you are working on. Somehow that node didn’t have a “Git branch” listed on my release even though that field is required. Huh? The node was made before I took over the plugin so I’m not sure how that happened, but if there is no Git branch listed, then the test bot can’t check out the branch you want to test.
Hopefully, if you’ve never used Simpletest before reading this blog post, you now feel empowered and knowledgeable enough to start writing your own tests for custom and contrib plugins. Happy testing, y’all!
Developer Blog
Source: New feed