The screen resolution and the Ranorex Studio debugger affect the execution speed of the test run.
Ranorex Studio is built on top of the excellent open source SharpDevelop IDE, which is designed to mimic the Visual Studio IDE. This gives Ranorex many advanced capabilities, including the ability to debug and step through code modules and recording modules. However, the debugger also adds some overhead and slows down the test runs.
The Ranorex team is already working on making it easier to selectively attach or detach the debugger as necessary. Expect upcoming versions to improve this area.
This is post #1 in a series of posts on optimizing Ranorex test run performance.
The following table shows the test run duration on my system at its min and max resolutions, and with and without the debugger. For more info on the system setup, go back to the introductory post.
|Screen Resolution||Is the Debugger Attached?||Test Run Duration in Minutes|
As you can see, running in a lower resolution takes less time. In this case, it saves 0.6 minutes. That’s not such a bad ratio, considering that there are 6 times as many pixels on the screen, but it is surprising that the resolution has any effect.
The debugger has a more significant effect on the time. In both resolutions, running with the debugger adds a full minute to the test run duration (4.2 vs. 3.2, or 3.6 vs. 2.6).
Running in a lower resolution and without the debugger, in this case, takes 1.6 minutes less. It’s nearly 40% faster!
So when you’re running your tests as part of an automated build on a test machine, try to run at the lowest resolution that makes sense for your app and your tests. Usually automated builds do not attach a debugger.
Things are a bit trickier on your work machine. If you’re like me, you probably won’t lower the resolution just to run your tests, even if it will improve the speed by a few percentage points. But unless you’re writing a lot of code in code modules or recording modules, you usually don’t need the debugger. At the moment, you can run your tests without the debugger, but only in some situations. In case it isn’t obvious, any breakpoints you set will be ignored when running without the debugger.
- When you’re running a full test suite, avoid pressing the the RUN button in the test suite tab. It attaches the debugger before starting the test run. To run the test suite without the debugger, press Ctrl+F5, click the black triangle (with the tooltip “Run the program without attached debugger”), or select the “Debug > Run without debugger” menu item.
- If you want to run just part of the test suite, modify the test run configuration. Simply check or uncheck the relevant folders and test cases before running without the debugger. You can also save your test run configuration to re-run it in the future.
- If you want to use any of the context (right-click) menu “Run” commands on the test suite, there is no way to run them without the debugger. This usually isn’t a problem because you’re running just a subset, but it is good to keep in mind. The same goes for the PLAY command on the recording module tab’s toolbar, or the other context menu “Play” commands.
This is all easily reversible. If you ever do need to debug code modules or code actions in recoding modules, add your breakpoints and then press F5, click the green arrow, or press the RUN button.