How I shaved 50% off a single suite run – a write-up

Here is another one of those ‘reports’ I’ll send out to clients when I’ve done a bunch of work. There was some non-relevant-to-this-post stuff at the end which is why it ends somewhat abruptly.

<email>

So I looked the ‘photodashboard’ scripts earlier today. The first step is of course to run them and see whats going on.

35 minutes?!?

That’s a little insane for for 20 scripts. So I went digging.

The first thing I did was remove the 404 that happens at each script. This was put in originally since there could be things like server.client.com/install_a and server.client.com/install_b and there was some bouncing to the right (or wrong!) place happening. If we’re running on a machine that has that in the future would just should include the install_whatever in the ‘base_url’ rather than try and build it using the ‘prefix’. I didn’t time the savings of it, but I would guess it would be around 2 minutes for the run.

Next I looked at the actual scripts and noticed something. It is extremely useful to include a specific setUp method in a class when having to do common setup tasks for the test methods (that is of course what it is for). What is less useful is when that setUp method does a login and then 50% of the tests immediately logout. If you find yourself in this situation, create a separate class — one for navigating to a page with authentication, and one without. Which is what I did and got it down to 21 minutes. Yes, over 10 minutes saved by this simple thing.

That is still to long. So I looked at being able to authenticate outside of the browser since that is slow since it involves a couple page loads. Using firebug I was able to see that the process of authenticating is just doing a POST with the username and password and when successful using the JSESSIONID cookie that is returned. With this I knew I could do this out of the browser and so have not modified the login_as method to include an optional browserless parameter. When this is set it will, outside of the browser, authenticate and then set the cookie on the se browser instance as if we had done it through the browser.

And with that we are down to 17 minutes. Which is still way too long, but that is the real speed of the app which means server.client.com is either massively underpowered or ‘application’ has some serious performance problems.

Or both.

</email>

This illustrates a bit of my process for cleaning up scripts.

  • Run the scripts
  • Read the scripts
  • Mark the failures
  • Fix failures individually
  • Fix scripting smells
  • Run the scripts

Some important bits to glean from this that you can apply to your projects.

  • Deal with environment differences at the config level, not the script level. Scripts should be environment neutral.
  • If you find yourself undoing something that was done in the setup phase of the script, it should be in a different class with a different setup.
  • Do everything that you can outside of the browser [if it doesn’t matter for the script]

Comments 2