Importance of Constrained Random Verification Approach

Dear All,

As a verification Engineers we must know what technique should be used in Test Bench development to verify IP, FPGA or any ASIC/SoC Verification.

I have been hearing many ideas, techniques and approaches on Directed Testing as well as Constrained Random Verification. Here I would like to share some advantages of CRV (Constrained Random Verification) over Directed Testing/Verification. Let us try to understand in brief on both approaches

First, let me give you an idea on 'What is Directed Testing?'

Directed Verification Environment with a sets of directed tests is extremely time-consuming and difficult to maintain. Directed tests only cover conditions that have been anticipated by the verification team, This can lead to costly re-spins and still there are changes of missing market windows which is extremely painful for any semiconductor company.

In directed Verification testing, Engineers spends good amount of time to understand the functionality of Design and identify different verification scenarios to cover functionality. Once they are done with identifying scenarios, they start defining directed test bench architecture. Traditionally verification IP (VIP) works in a directed test environment by acting on specific testbench commands such as read, write or some other commands to generate transactions for specific protocol testing. This type of directed testing is used to verify that an interface behaves as expected in response to valid/invalid transaction. Bigger risk with this type of testing is that directed tests only test for predicted behavior. So some time it leads to extremely costly bugs found in silicon which they missed during the scenario identification phase !!

Constrained Random Verification Methodology gives effective method to achieve coverage goals faster and most importantly it helps in finding corner case problem. Advantage is, Engineers does not have to write many test cases, smaller set of constrained-random scenario with few full random test scenario are good enough to fulfill coverage goals (functional as well as code coverage).

Based on my experience and understanding, usually people follows layered architecture in constrained random verification. (For better understanding of layered architecture, click on Gopi's Blog or Read VMM User manual by Synopsys) where you will see Test layer controls over whole verification environment and component. Mostly this control will be given to user. So user can run same test suites with different configuration if require to achieve coverage goal. In constrained random approach, scoreboards are used to verify that data has successfully reached to its destination while monitors snoops the interfaces to provide coverage information. New or revised constraints focus verification on the uncovered scenarios to meet coverage goal. As verification progresses the simulation tool identifies the best seeds which are then retained as regression tests to create a set of scenarios, constraints and seeds. In this approach of verification, you will be having less number of test cases which is enough to achieve coverage goals. I have observed one best usage of Directed tests in random verification, Here I am describing the best usage place of directed tests.

Always use directed tests after regression cycles of random verification, Random verification regression cycles gives some corner scenarios which are always left in coverage and you can always identify from functional and code coverage analysis. So identify those kind of scenario and write directed tests with specific constraint to cover specific scenario. This way coverage will be achieved !

Constrained Random Verification is very popular now a days because of so many reason, I have tried to capture couple of differences and advantages over both techniques.

Hope this post adds better understanding over constrained random verification technique.

Note : Special Thanks to Mr Gopi Krishna allowing me to use his webpages as references and posting my couple of interesting posts on his website http://www.testbench.in/links.html. My Coverage Post, My Pass/Fail Message Post

Comments, suggestion and questions are always welcome !

Enjoy!

3 comments:

rakesh sachdev said...

Hi Ankit,

Weather,Its a pre-silicon or post-silicon enviornment testing/verification/validation is never considered complete til number of bugs/week goes to zero with functional coverage is exhausted.

Directed random testing can be mixed with constrained random testing by writing testbenches which targets the perticular functional or new design feature closely with randomized vectors defined by seeds to generated randomized scenarios.Directed and Random tests can be chosen on seed basis even.

Today,Directed tests can run on simulators to get the expected results stored in memory image with next runs on models to get the actual image.Then,final comparisons can be in order to generate test pass/fail flags.

Although,directed tests are pretty tedious and does not generate the unintended test-cases but pretty much required to verify functionality of all new features with corner-cases on radar!!!

Even directed tests can be partly randomized through test seeds with patterns,timings of signals and pipeline features can be randomized.Mix and Match of it with VMM in pre-silicon will yeild the best results.

I am open to further discussion here.

Regards,
Rakesh Sachdev

Ankit Gopani said...

Hi Rakesh,

Thanks for reading my blogpost and providing your view on it !

Well, You are absolutely correct. Even I have seen most of the project where verification closer happens with directed testing to cover corener scnearions and to fulfil the code and functional coverage expectation.

Sometime random seeds are not enough to cover some super corner scenario where engineer has to identify the left/uncovered part of code and then need to identify directed scneario to hit those part!!

Appreciate your view ! Thanks for reading and keep writing !

Enjoy
ASIC With Ankit

sriram said...

I am reading your post from the beginning, it was so interesting to read & I feel thanks to you for posting such a good blog, keep updates regularly.

software testing course in chennai