NCover FAQ

If you have questions that this document does not address, contact Peter Waldschmidt or try the NCover Forums.

1. What is code coverage analysis?

A code coverage analyzer monitors your code at runtime and records information about which lines of code were executed. NCover shows each sequence point in your application along with the number of times that point was executed. Sequence points are generated by the compiler and stored in the debug information (.pdb) files. A sequence point basically corresponds to a single program statement (often a line of code) in your high-level language.

2. Why would I want to do code coverage analysis?

Unit test suites are often used as a quality tool during the development process to keep the codebase stable as it changes and expands. Tools such as NUnit are often used to run and report on the test suites. However, when implementing unit testing in your build process, you have no way of knowing how much of your code the unit tests are actually testing. This is where code coverage comes in. You can run NUnit within NCover and use the code coverage report to determine which code was not tested by that particular test suite.

3. What versions of the CLR does NCover support?

NCover 1.5.x requires the .NET framework version 2.0.50727 to be installed; however, the application being profiled can be written against any shipping version of the framework. NCover has been tested profiling coverage of .NET 2.0, .NET 1.1 and .NET 1.0 applications.

4. Which version of NCover should I install?

If you have the .NET 2.0 framework installed on your machine then you should use the latest NCover version available. NCover as of version 1.5 can profile .NET 2.0, 1.1 and 1.0 applications.

For development teams who do not have the .NET framework 2.0 installed but do have the .NET framework version 1.1.4322, you can try NCover 1.3.3. Note however that this version is no longer supported as it has a number of known issues and limitations.

5. What is the command line syntax for NCover?

Here is the usage info from the NCover command line (for NCover versions from 1.5.6 only):

NCover.Console [<command line> [<command args>]]
               [//svc <service name>]
               [//iis]
               [//a <assembly list>]
               [//w <working directory>]
               [//ea <exclusion list>]
               [//reg]
               [//x <xml output file>]
               [//s [<settings file>]] [//r [<settings file>]]
               [//v] [//q] 
               [//l <log file>]

//svc  For profiling windows services
//iis  For profiling web applications

//a    List of assemblies to profile separated by semi-colons i.e. "MyAssembly1;MyAssembly2". Do not include paths or suffixes.
//w    Working directory for profiled application 
//ea   List of attributes marking classes or methods to exclude from coverage 

//reg  Register profiler temporarily for user. (helps with xcopy deployment) 
//x    Specify coverage output file. (default: coverage.xml).
//pm   Specify name of process to profile (i.e. myapp.exe)

//s    Save settings to a file (defaults: NCover.Settings) 
//r    Use settings file, overriding other settings (default: NCover.Settings) 

//l    Specify profiler log file (default: coverage.log).
//q    No logging (quiet) 
//v    Enable verbose logging (show instrumented code)

<command line> - This argument specifies the command-line of the .NET application you want to analyze. Any command line arguments not starting with // will be passed through to that application. NCover will profile the running application until it has exited. See below for examples.
//svc - This option is an alternative to the <command line> for profiling windows services, which cannot be run directly as executables. NCover will start the service (stopping it first if already running) and profile coverage until the windows service is stopped.
//iis - This option is an alternative to the <command line> for profiling web applications. NCover will start the IISAdmin and W3C services (stopping first if currently running) and profile coverage until the IISAdmin service is stopped.
//a - This command-line argument specifies the assemblies that you want to analyze. NCover can only analyze assemblies that have .pdb files included with them. If you do not specify the //a argument, NCover will attempt to analyze every loaded assembly that has debug information available. Note that the assembly name arguments are the module name within the assembly, not the physical file name. e.g. "MyAssembly" rather than "MyAssembly.dll".
//w - If the application being profiled requires the working directory to be set to something other than the current directory you are executing the command line from then you can override it with this argument.
//ea - You can choose to exclude classes and methods from coverage statistics by defining .NET attribute(s) and applying it to the affected code. When using this argument you must specify the full type namespace of these attribute(s) separated by semi-colons. See below for an example.
//reg - NCover requires a COM registration of the CoverLib.dll assembly containing the profiler, which is performed automatically by the default .msi installation. If you require an xcopy style deployment of NCover like many other .NET tools, then you can use this argument which will temporarily register the profiler while performing coverage. This feature was added in NCover 1.5.6.
//x - The output of NCover is an xml file (example below). Use this argument to specify an alternate filename to "coverage.xml" in the current directory.
//pm - This setting tells NCover to ignore processes that don't have the specified process module name. This is the name of the executable (i.e. myapp.exe). This setting is useful in cases, where your NCover command spawns a series of child processes. Using this setting will help NCover determine which process to profile.
//s - You may find it more convenient to use a settings file rather than specifying a long list of command line arguments for running NCover. If you get the NCover command line working as you would like it and then use the //s argument it will save the required arguments as an xml file that can then be used by the //r argument below.
//r - For use when you have used //s to construct an NCover settings file containing your command line arguments. e.g. "ncover.console.exe //r NCover.Settings"
//l - The coverage log file can provide an insight if the desired coverage output is not obtained. Useful information you may find to assist you includes which assemblies were loaded by NCover, their file paths and which of those it found the .pdb build symbols for. Use this argument to specify an alternative log file name or location to coverage.log in the current directory.
//q - Suppresses writing the coverage.log file.
//v - This command-line argument makes the profiler emit all the original IL and modified IL instructions to the coverage log. This is useful for debugging purposes. Beware that this can make your coverage log file very large!

6. Does NCover required a special compilation step for my code?

No. Some code coverage tools change your source code and force you to recompile it into a special build. NCover is designed to work on shipping code. NCover uses the .NET Framework profiling API to monitor your code. It does require build symbols, but can be run on release code without any modifications.

7. How does NCover work?

NCover uses the .NET Framework profiler API to monitor an application's execution. When a method is loaded by the CLR, NCover retrieves the IL and replaces it with instrumented IL code. NCover does not change your original IL code, it simply inserts new code to update a visit counter at each sequence point. Upon request, (usually after the .NET process has shut down) the profiler outputs statistics to the coverage file.

8. What is the output of NCover?

NCover generally writes out three files after analysis completes.

Coverage.log - This file is a log of the events and messages from the profiler during the analysis process. Most of the time, error messages are recorded in this log. If you enable verbose logging, the coverage log will contain disassembly of the original and instrumented IL code. Verbose logging is not recommended for normal use.
Coverage.xml - This file is the analysis output of NCover. You can see an example of the output below.
Coverage.xsl - This file is a simple XML transformation that makes the XML output easily readable.

Example XML output

<method class="NCoverTest.ClassLoaded" name="HasDeadCode">
    <seqpnt document="C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs"
            column="13" line="48" endcolumn="58" endline="48" visitcount="1" /> 
    <seqpnt document="C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs" 
            column="13" line="49" endcolumn="22" endline="49" visitcount="1" /> 
    <seqpnt document="C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs" 
            column="17" line="50" endcolumn="24" endline="50" visitcount="1" /> 
    <seqpnt document="C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs" 
            column="13" line="51" endcolumn="48" endline="51" visitcount="0" /> 
    <seqpnt document="C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs" 
            column="9" line="52"  endcolumn="10" endline="52" visitcount="0" /> 
</method>

Example transformed output

NCoverTest.ClassLoaded.HasDeadCode

Visit Count	Line	Column	End Line	End Column	Document
1	48	13	48	58	C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs
1	49	13	49	22	C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs
1	50	17	50	24	C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs
0	51	13	51	48	C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs
0	52	9	52	10	C:\Dev\Utilities\ncover\NCoverTest\NCoverTest.cs

Suggested usages of the coverage.xml output are to display it in the NCoverExplorer gui with the source code highlighted, to generate html reports, or to include it in your continuous build server reports such as CruiseControl.Net. For more information on these options see below in the FAQ.

9. How do I use coverage exclusions?

First you should define an attribute to markup your excluded code with. You will likely want to put this in a common assembly to make it reusable, or indeed within a "CommonAssemblyInfo.cs" that you include in all your application assemblies.

namespace MyNamespace {
    class CoverageExcludeAttribute : System.Attribute { }
}

Apply the attribute to the C# classes and/or methods you wish to mark as excluded from code coverage statistics:

    [CoverageExclude]
    private void SomeMethodToExclude() {}

Finally, ensure you pass the full qualified attribute information in the NCover command line:

    NCover.Console MyApplication.exe //ea MyNamespace.CoverageExcludeAttribute

Note that if you are using the TestDriven.Net VS.Net add-in to "Test with Coverage" it will automatically pass through "//ea CoverageExcludeAttribute" which you should define without a namespace like above. For further information refer to this blog entry.

10. Examples

Coverage while running a simple executable until it exits:

    NCover.Console MyApplication.exe

Coverage while running all the unit tests in an assembly using NUnit, profiling all loaded assemblies with .pdb build symbols:

    NCover.Console nunit-console.exe MyApplication.Tests.dll

Coverage of only a subset of loaded assemblies while running unit tests:

    NCover.Console nunit-console.exe MyApplication.Tests.dll //a MyApplication.Core;MyApplication.Utilities

Coverage of a windows service. Stop the service to generate the coverage output:

    NCover.Console //svc MyServiceName

Coverage of an ASP.Net application. Stop the IIS service to generate the coverage output:

    NCover.Console //iis

11. Where can I get help or support?

Your best approach is to browse the NCover forums as well as the blog by the author Peter Waldschmidt. If you cannot find a similar issue mentioned feel free to post your query and perhaps someone can help.

12. How do I "xcopy deploy" NCover like my other build tools?

Many developers prefer to have their build tools such as NUnit, NAnt etc stored in source control in a Tools folder along with the source code. This ensures that a new developer can obtain and build the application without having to install additional tools on their own machines.

NCover can also be deployed in this fashion. However the one gotcha with NCover versus other tools is that the profiler within CoverLib.dll must be COM registered on the local machine before you execute it. Prior to NCover 1.5.6 this was usually achieved as part of your build script, which would call regsvr32 with the path to the CoverLib.dll in your Tools folder. Alternatively the <ncover> NAnt and MSBuild tasks described below will do this for you. As of NCover 1.5.6 you can also use the //reg option in the command line arguments which will temporarily register the profiler. Note that the //reg option will not work for IIS or Windows Service profiling unless you are running NCover under the same Windows login account as the IIS worker process, or your Windows Service.

13. How do I see my source code highlighted with the coverage results?

NCoverExplorer is a gui and console-based .NET application developed by Grant Drake. NCoverExplorer parses the coverage.xml files output from NCover and displays the results integrated with your source code. It also includes a number of additional features to merge, filter, sort and generate html reports. The console version is designed to be used as part of an automated build process. The support forums for NCoverExplorer are located with the NCover ones at http://ncover.org/.

14. How do I run NCover from within the Visual Studio.Net IDE?

The TestDriven.Net add-in by Jamie Cansdale offers a right-click capability within the IDE to execute your unit tests with code coverage. The results of the NCover code coverage are displayed with the bundled NCoverExplorer gui for analysis and reporting.

15. How do I run NCover from a NAnt or MSBuild task?

You can use an <exec> task with NAnt or an <Exec> task with MSBuild. Alternatively you may want to use the custom <ncover> task for NAnt or <NCover> task for MSBuild developed by Grant Drake for a more developer friendly syntax. The source code, compiled assemblies and documentation are located in the NCoverExplorer.Extras.zip available from http://ncoverexplorer.org/.

16. How do I include NCover output in my CruiseControl.Net build reports?

CruiseControl.Net is a continuous integration build server which offers web-based reporting of the outputs of a build such as unit test results and code coverage reporting. The default CruiseControl.Net installation includes a basic stylesheet which works in combination with the standard coverage.xml formatted output. So all you need to do is include the execution of NCover as part of your build, then add a CruiseControl.Net merge file publisher task to integrate the coverage.xml results into the build output.

An improvement on the above to display more attractive and powerful reports as well as minimize the build log size is to use NCoverExplorer. The NCoverExplorer.Console.exe is designed to produce a more concise xml report summary that is combined with an alternate xsl stylesheet for CruiseControl.Net. You can find more information and screenshots in this blog entry - all the necessary tasks, examples and documentation are located within NCoverExplorer.Extras.zip available from http://ncoverexplorer.org/.

17. How do I merge multiple NCover coverage.xml results?

You can can use NCoverExplorer to merge the results of multiple coverage runs. For more information refer to this blog entry.

18. Troubleshooting: Why is my coverage.xml file empty?

If using the command-line, did you COM register CoverLib.dll (or use the //reg option from NCover 1.5.6)?
Did you generate build symbol files (.pdbs) for the profiled application?
If using the //a option, did you correctly list just the assembly names without paths or .dll suffixes?

19. Troubleshooting: I have coverage.xml output but my XYZ assembly is not included in it?

NCover will only profile loaded assemblies - did your code execution path while under coverage force that assembly to be loaded (e.g. by loading a type or calling a method in that assembly)?
Did you generate build symbol files (.pdb files) for the missing assembly?
If using the //a option, did you correctly list the assembly names including the one that is missing?
Can you see information about the assembly being loaded within the coverage.log? Is the correct assembly being loaded (check the path) - if you have a version in the GAC it may possibly prevent the .pdb file from being loaded.
If using the NCoverExplorer gui, have you got a coverage exclusion defined which is hiding it from the display?

20. Troubleshooting: After running NCover my coverage.log says "Failed to load symbols for module XYZ"?

This message means that no .pdb build symbol file was found for that assembly so it cannot be profiled for code coverage. If that assembly is part of the .NET framework for instance like System.Data.dll, then this is an expected message and should not cause concern.
If however the assembly belongs to your application, did you generate the build symbol files (.pdb files) for it?

21. Troubleshooting: I get a "Profiled process terminated. Profiler connection not established" message?

If using the command-line, did you COM register CoverLib.dll (or use the //reg option from NCover 1.5.6)?
Are you running Windows XP 64-bit? You may want to take a look at this thread

22. Troubleshooting: My coverage exclusions are not working?

Have you put the full namespace type name to the exclusion including the Attribute suffix in the //ea argument? See the "How do I use coverage exclusions?" question above.