Clover.NET Alpha Version 0.1 Alpha (10-May-2004)

Welcome to the latest Clover.NET alpha snapshot.

Installation

The zip file for Clover.NET contains three installers. One installer, VSIP Interop Assembly Redist.msi, is provided by Microsoft and includes a number of interop assemblies to expose core Visual Studio functionality to managed code. This must be installed before the Clover VS.NET plugin will be functional.

The two remaining are for different Clover installs. One installs the Clover.NET command line tools, whilst the other also installs the Clover.NET visual studio plugin.

Using the Clover Visual Studio.NET Plugin

The Clover.NET plugin works with Visual Studio.NET 2003.

The Clover.NET install includes a samples directory with a very basic sample project, Sample. The Sample dll project comes with a Sample.sln solution file. The following steps show how to Clover this project with the VS.NET plugin

1. Launch Visual Studio.NET by opening the solution file. You should see the Clover logo in the VS.NET splash screen.
2. Adjust the nunit framework reference in the Tests project to your local NUnit install, if necessary.
3. Open the Clover options tool window by selecting the menu Tools->Clover->Options. You can resize or dock this window as you would any other Tool Window in VS.NET. The options window consists of a tool bar and a properties pane.
4. The toolbar icons have the following functions

   	A toggle button to control display of coverage information in the VS.NET editor
   	Clover and build the current solution
   	load the latest coverage information
   	Clean coverage information (Not implemented)

5. Click the Clover build button to Clover and build the Sample project. Please check the output window to ensure that all has gone smoothly. You should see something like "Clover: 2 projects, 2 Clovered, 2 built, 0 failed"
6. Now open the Coverage View window from the menu View->Other Windows->Coverage View. This window has three components - a toolbar at the top which is identical to the options window, a coverage tree view, which will be empty at this time and a panel at the bottom for metrics and coverage information.
7. Open the SampleClass.cs file in the VS.NET editor. Click the sync button to load the latest coverage information - of which there will not be any at this point in time. Note that the tree view in the Coverage View Window is now populated and shows coverage of 0.00%
8. Click on the display toggle to display the current coverage state in the editor window. the whole of the negate method should be red. If you hover over the method, a tooltip will indicate that the method has not been entered. If you expand the tree view in the Coverage Window the icon next to the negate method node is red indicating the method has not been entered.
9. If you wish to change the colour of the coverage display, you can select the Tools->Options... menu and then Fonts and Colors in the dialog. In the Display items list, there are three entries for Clover
   • Clover: method not covered
   • Clover: statement not covered
   • Clover: conditional not covered
   You can change the color used for any of these items.
10. Use nunit-gui to run the Clovered Tests. Navigate to the CloverBuild\Tests\bin\Debug\Tests.dll. Run the tests and quite nUnit.
11. Return to the IDE and click the sync button again. The display in the editor window will update to show the statements not executed. The Coverage Tree view will now show the coverage achieved. Note the icon for the negate method is now green indicating the method has been entered.

Clovering with Command Line Tools

The following steps show how to use Clover.NET's command line tools to Clover the Sample project mentioned above.

1. Open the solution file in Visual Studio 2003. It consists of two projects - the Sample DLL and a NUnit based Test suite.
2. Adjust the nunit framework reference in the Tests project to your local NUnit install, if necessary.
3. Build this solution now. It should produce a Sample.dll and a Tests DLL.
4. Launch the NUnit-gui and load Tests.dll in the Tests project. Run the test and hopefully you will see all Green.
5. The next step is to Clover the code. Copy the whole Sample directory and call the copy InstSample - this is going to hold the instrumented code.
6. Clover the code with the following command
   CloverInstr -i sample.cdb -s Sample -d InstSample
7. The instrumentation will have instrumented the source code and generated a coverage database for the project in the sample.cdb file. If the source were contained in sub-directories, use the -r switch to recursively scan for source files.
8. Open the sample.csproj project file in the InstSample directory.
9. Before you will be able to build the instrumented project, you must add a reference to the CloverRuntime.dll to the project. Right click on the References section of the project and select "Add Reference ...". Browse to the CloverRuntime DLL in the Clover.NET installation directory.
10. Now build the clovered sample project
11. Copy the clovered sample.dll over the non-clovered version in the Tests\bin\debug directory. Additionally, copy the Clover DLL into the Tests\bin\debug directory.
12. Run NUnit-gui again and execute the tests. To have Clover write out its coverage database you must either close the project in NUnit-gui or simply quit. The test results should be the same, but Clover.NET will have created a coverage recording beside the sample.cdb file, starting with sample.cdb and ending with a set of hex digits
13. With a coverage recording, you can now generate a coverage report. Use the following command
    HtmlReporter -i sample.cdb -o report -t "Test Report"

At the end of the above process there will be an HTML coverage report in the "report" directory. That's all there is to Clovering your projects. While this is a simple project, the same principles can be used for more complex projects. An alternative it to add the Clovered source code as a separate project in your main Visual Studio solution and build alongside the non-Clovered code.

Notes and Known Issues

Visual Studio Plugin

Clovering and building takes place on the main UI thread. For a large project, the UI may be unresponsive while this occurs.

When running a Clover build, the Clover plugin spawns an external instance of Visual Studio (devenv) to do the build. This is done to ensure this build does not interfere with the state of the current solution. If build errors, as opposed to Clovering errors, occur, the build will finish with the task list but the task list may be empty, since the external errors are not currently parsed. If this is the case, please refer to the build output window for more information.

You can generate an HTML report of VS.NET Clovered projects using the command line tool. The collected data is the same regardless of how the coverage was generated. Conversely, you can view coverage in the VS.NET IDE for projects built and run using the command line tools

Using csUnit

Clover.NET hooks the AppDomain Unload event to trigger writing out coverage information. The csUnit test runner will not always unload an assembly when it quits, resulting in no coverage information being written. A workaround is to rerun the unit test. This causes csUnit to Unload the currently AppDomain and assembly before starting the next test run. This unloading is sufficient to write out the coverage information. In fact the second test run can be stopped straight away as the coverage information from the first run will have been saved. A bug has been reported to the csUnit project.

Preprocessor support

Clover.NET parses preprocessor directives but does not act upon them. This may affect Clover.NET's ability to parse a source file with preprocessor directives. If, ignoring the directives, the resulting source file is syntactically correct, Clover.NET will instrument the code correctly. Otherwise, Clover.NET will report a syntax error. The use of file level #if ... #endif constructs should, therefore, be OK. More complete preprocessor support is planned.

Debugging issues

The instrumenter supports three debugging command line options

• --verbose - verbose logging
• --debug - debug level logging
• --trace - parse state tracing

The verbose setting gives you more information on what the instrumenter is doing as it goes. This should help locate any files that cause problems. If you do encounter an issue where the Instrumenter does not correctly instrument your code or crashes, please rerun the instrumenter, firstly with debug and then trace settings. For example, on the sample project

If you want to instrument a single file, use the following example as a guide:

Please report such errors and include the debug.txt and trace.txt files. If you can include the source files as well, that would be appreciated. Only the source files which cause problems are required.

Note: Please be aware that the trace operation generates extremely large output as it tracks the parser states.

Feedback and Issues

We are interested in all feedback. Please feel free to express you thoughts on any aspect of Clover.NET. If you run into a problem, please try to let us know as much as you can. We are particularly interested when instrumentation generates code which fails to compile. If you can send us the offending code, that would be great. We realize you cannot always send in code, for whatever reasons, so in these cases we would appreciate as much information as you are comfortable sending.

For any issues, just email clover-support@thecortex.net with the details.

We have also created a set of Clover.NET fora on our forum system. You can use this to discuss issues with the current alpha. The Clover.NET fora are available here