Unit testing for speed?
I was just thinking about performance testing, and wondered if writing a unit test would be sensible. If a functional requirement is, for example, "Import all of the data in the crime scene XML file into the database within 0.5 seconds", a test could be written to:
Although the idea seems like it would work, it feels like a wrong thing to do. Has anyone else done this successfully, or can anyone give me an actual reason why this is wrong (other than "it just feels like it")?
- Get the time into startTime
- Call the ImportCrimeSceneData method
- Get the time into endTime
- Assert endTime < startTime + 0.5 seconds
Although the idea seems like it would work, it feels like a wrong thing to do. Has anyone else done this successfully, or can anyone give me an actual reason why this is wrong (other than "it just feels like it")?
4 Comments:
In my opinion there are too many outside factors that come into play when testing performance. A Unit Test may pass on one machine and fail on another, despite having exactly the same code.
Thanks, that's a good reason.
I guess that could be levelled against the functional specification too. If it doesn't say what machine configuration the half second applies to, then it can't be measured properly, and therefore can't be signed off.
If it does say what machine to run on, then I suppose it could be tested on a similar machine: in this case, the database might be on a networked server, which makes the measurement doubly unreliable.
I think it's a good idea in some situations. Recommended reading: Continuous Performance Testing with jUnitPerf. I ported jUnitPerf to Python. See Performance Testing with pyUnitPerf.
Thanks Grig, that's a really interesting article, and pretty much what I had in mind. I've read a few of Mike Clark's things, and they're usually pretty good.
Post a Comment