Blog: Code Coverage and Functional Coverage – What’s the Difference?

Having come from a design background and working on many projects for 14 years…
—

It is interesting for me to see things from a different perspective in my new role as Applications Specialist at FirstEDA. My primary focus as an Applications Specialist is to use my design experience to help engineers get the most from the tools and solutions they purchase from us.

In this blog, I am exploring the difference between Code Coverage and Functional Coverage with particular focus on using Aldec’s Riviera-PRO with SystemVerilog constructs.

How do you know that you have fully tested your design? This is a question that every verification engineer seeks to answer.

Historically, companies have been focussed more on Code Coverage than Functional Coverage and to many of us, the difference between them may not be crystal clear.

Code Coverage is mainly described as simulator collected metrics on code that has been run. While it identifies what has not been run, it cannot identify the code that wasn’t implemented as per the specification. By nature, Code Coverage analyses the structural correctness of the code, not the functional correctness. Sometimes the coverage goal is hard to express in terms of behaviour; it may be easier to specify using values of design variables and that is why we need Functional Coverage.

Functional Coverage checks the correctness of the design by collecting values (or sets of values) of design variables during the simulation. It is a user-written code that observes the execution of the test plan. It ensures that a test did what was intended, particularly with randomisation. When the test plan has been executed, testing can be declared as complete.

Combining Code Coverage and Functional Coverage provides us with an interesting overall set of testing metrics for our design. Code Coverage can be used as a catch-all to indicate whether something got missed in Functional Coverage. Therefore, we can safely say that:

Test Done = 100% Functional Coverage + 100% Code Coverage

Historically, most tools gave information about Code Coverage by using some additional switches on the compiler and simulator, on the other hand, Functional Coverage relied on completeness of the test plan and was more or less manual.

The good news is that Functional Coverage is no longer manual. There are language constructs available both in OSVVM (in VHDL) and SystemVerilog, that allow the users to define it as well as to measure it.

Here we will focus on SystemVerilog constructs for defining Functional Coverage, specifically in the context of simulation. FirstEDA also represents OneSpin and their class-leading formal solutions. Formal is another approach to determining precise coverage metrics and is complimentary to simulation. A topic for another day!

Covergroup Coverage is a form of Functional Coverage that calculates SystemVerilog coverage model statistics. It is a user-defined metric that measures the percentage of design specification that has been examined by running the simulation session. It verifies interesting and relevant design features (listed within the verification plan), which have been observed using the generated stimuli. In contrast with Assertion Coverage, which is used to verify design features over time, Covergroup Coverage is focused on covering relevant values that are accepted during the entire simulation. However, it requires manual specification of coverage objects and instantiation within the testbench code. Furthermore, it is used for high-level verification of complex design and is available in Aldec’s Riviera-PRO tool.

SystemVerilog allows the user to describe Item Point Coverage or Item Coverage using coverpoints which enables the user to track the relationship with a single object. For each coverpoint, bins can be created to organise the possible signal values into meaningful categories. It also allows users to track the relationship between the various objects by defining Cross Coverage. It is specified using the cross construct.

Below is an example where I have used Aldec’s Riviera-PRO to look at Functional Coverage using SystemVerilog.

The Functional Code Coverage model APB_Cov_Collector is written as a Class that is described in APB_Monitor.

In the APB_Component, the APB_Cov_Collector is instantiated using the new() constructor and connected to an instance of APB_Monitor using mon_chan as shown below:

Covergroup can be in a module, interface, program or class. Here it has been described in a class:

Associating the covergroup with a clock event is also a good way to trigger the coverage sampling. Sampling can also be done by explicitly calling the .sample() method in procedural code as shown below. This is used when coverage sampling is required based on some calculations, rather than events.

{{cta(‘a165d89c-433a-4f58-9cf4-a656ccded90d’,’justifycenter’)}}
The SystemVerilog also defines a set of options for the coverage. Each covergroup contains options for configuration which allows for customisation. They control the behaviour of the covergroup, coverpoint and cross. In the covergroup example shown above, some options have been used such as option.per_instance which has been set to ‘1’. By default, coverage is collected across all the instances of a covergroup. Setting it to ‘1’ ensures that each instance coverage is collected separately for each instance in which the new() constructor is called.

There is another option called auto_bin_max that has been set to ‘3’, as shown above, which will determine the number of bins that are created for the data signal in the absence of a defined number of bins. In this example, option.name has been defined as ‘APB_Coverage’ in the absence of which, the tool would have given it any name. Other useful options allow the user to change the weighting of the covergroup instance, specify the target coverage goal percentage and the required number of hits for each bin.

The cross between addr and dir, indicated by cross_addr_x_dir in the code snippet above, records the correlation between the transaction address region and direction of transfer, which could be used to access how well the memory map is tested. By using ignore_bins, the bin showing the number of writes to ROM has been excluded from coverage analysis, making the possibility of achieving 100% Coverage realistic. The cross between addr and data is also useful. By looking at the number of hits of data values for each category of an address area, the user can judge if the stimulus was reasonably diverse. This can also help them decide if they need to constrain the stimulus further to achieve better coverage results.

After Riviera-PRO has finished running a simulation, it will generate HTML and text reports detailing the Coverage Analysis of the design. The HTML report for this example is shown below. It contains all the information of CoverGroup, CoverPoints and the bins defined in the Functional Coverage model above. This report is well organised and easy to read with coloured highlights and it contains all the information of the number of hits, the goal (the coverage percentage) and the coverage status of the CoverGroup as a hierarchical tree.

To conclude, I have demonstrated how effective Functional Coverage is in providing verification engineers with control & visibility, making their work more predictable. This applies equally when running both directed tests and constrained random. In conjunction with Code Coverage, Functional Coverage provides deterministic confidence that overall project coverage goals are being met and ideally bettered. A comprehensive documentation can be generated to report the results – essential for efficient peer review and external audit, where required.

If you are looking to improve your verification solution then you can read about our FPGA methodology solutions. Alternatively, you can request a demo with one of our Applications Specialists who will be happy to discuss your interests and answer your questions.