Don’t forget to turn of the debugger when doing performance testing

Recently our dotless lead tester Gert discovered a little bug in the dotless minifier that made the minifier “loose” css expressions that where not followed by a semicolon.

The fix to this was in theory quite simple, simply treat all closing braces } as semicolons so expressions are still terminated. Doing so then led to one new cause for problems: ; }.

Semicolons followed by a closing brace call the ExpressionBuilder with a empty value, the ExpressionBuilder can now either throw an exception or return null. I decided to try throwing exceptions first.

Returning null somehow felt less elegant and I did not consider the performance impact of throwing an exception. So after implementing I decided to test this and wrote a little benchmark console program to test my theories.

In Shock I watched the exception case to be 10 times slower than simply doing a null check. Hastily I reverted the exception check and started implementing the null-check in the dotless code when I remembered the debugger.

Doh! How could I run a performance test with the debugger attached. Ran the program without the debugger: exceptions where only twice as slow as the null check. Not too bad, but still not fast enough for dotless.

So, here some wisdom: Always make sure you run without the debugger when testing performance. Exceptions while debugging are roughly 10 times more expensive than without debugger.