Lately I’ve been experimenting with integrating YSlow into our existing Selenium test suites. For those not familar with YSlow, its a Firefox add-on developed by Yahoo which provides page performance analysis. An excellent tool if web testing is a part of your repertoire.
The Selenium / YSlow integration concept itself isn’t new. There’s a great post on Adam Goucher’s blog discussing the idea and while I wasn’t keen on tying it to our CI system, the idea of getting page performance data at the completion of our automated tests sounded great.
In terms of setup, its relatively painless. The solution is based around the free Yslow firefox addon and the (opensource) ShowSlow reporting server. Given that there isn’t a lot of information available on the setup process, here’s a brief overview:
ShowSlow Server Setup
1. Download and install Apache, PHP and MySQL (essentials will do).
2. Once all of the above are setup, checkout the ShowSlow project source from here and host it via Apache.
3. You’ll also need to create a new database in MySQL and run in tables.sql (from the ShowSlow directory) to create all the relevant tables for ShowSlow.
4. Edit config.php and supply your database credentials.
5. Load the ShowSlow site in your browser on your webserver and click on the “configuring your yslow” link – you’ll need the values from this page later.
YSlow Client Setup
1. Create a custom Firefox profile on your Selenium RC machines (using firefox.exe -ProfileManager). The profile needs to be setup with the following attributes (at minimum):
- YSlow add-on installed and setup to run perf analysis on page load (set in Yslow options).
- In about:config, set “extensions.yslow.optinBeacon” to true and “extensions.yslow.beaconUrl” to the value from your “configuring your yslow” page.
2. On each Selenium RC box (the machine you’ll use to run automated tests), you’ll need to setup Selenium to use your new custom profile (using DseleniumArgs=”-firefoxProfileTemplate ” argument). In order to achieve this, modify the Selenium RC launch command to include the path to the new Firefox profile you just created. For example:
java -jar selenium-server.jar -firefoxProfileTemplate “C:/SELENI~2/SELENI~1″
NOTE: You need to provide the profile path using the DOS short name format. For more info on finding your firefox profile directory, see: http://support.mozilla.com/en-US/kb/profiles
Once you’ve completed the setup, running your automated tests should result in some YSlow performance statistics being sent to your ShowSlow server.
It’s worth noting that there are some downsides to this setup. Firstly, its Firefox only (currently) so any tests run against other browsers wont record any results with ShowSlow. And finally, it records page performance data for every page that your browser loads. This means that you’ll get performance data for URL’s that you don’t necessarily care about like those used by Selenium and those with variable query strings. While you can’t explicitly tell YSlow to not record performance data for these URL’s, you can modify the code within ShowSlow to prevent these URL’s from contaminating your results (eg./ I’m currently stripping some variable data out of certain URL’s as well as filtering the ShowSlow ouput).
If you’re already using Selenium as the basis for your automated testing, I’d definitely recommend tying it in with YSlow. With a little extra effort up front, it’ll add ongoing value to your automated tests.
UPDATE: It seem that 2.0b6 is the last that works correctly with the current version of ShowSlow. Version 2.0.1 of Yslow seems to rely on POST’s to send the full set of performance data to the beacon server. Version 2.0.1 will still work, you’ll just notice that the details page has a “0″ value for all of the Yslow Breakdown results.