Verification of a processor consumes nearly 60-65% of the average processor development cycle. It is almost impossible to completely verify a processor given the large exponential solution space. It order to verify a processor, one does not only need to check the compliance of the processor with the ISA spec, but also verify the various ISA independent micro-architectural states and circuits like: hazard detection, operand forwarding, branch predictors, cache optimizations. While one could easily develop a bottom-up approach to verify create a verification suite of test-benches for each module, but even that would require analyzing and identifying all possible inputs and scenarios when the module is plugged in to larger system. While, the bottom-up approach is a needed task, it should be limited to verifying the base functionality only to keep the turn-around time small.
A processor verification team, typically writes a series of software routines in assembly or C, which excite and test specific aspects of the processor. Considering the large state-space of the processor, manually writing these tests to cover even 80% of the states would take significant amount of time, thereby increasing the time to market. An approach to reduce this effort is to use random program generators for the processor. While these generators do not guarantee to provide full coverage, but a lot of base-case scenarios can easily be covered and the user is burdened with writing only a few specific directed test cases for the processor system.
In order to verify the C-class core, we have used multiple random program generators:
(1) RISC-V Torture [RISC-V Torture 2012]:This is a random assembly program generator from UC Berkely. A user can specify the total size of the program and also define the distribution count of various instruction extensions within the program.
(2) CSMITH [Yang et al. 2011]: This is random C-program generator from the University of Utah.
CSMITH can generate program which are illegal and cannot be ported to RISC-V, and thus care needs to be taken to ensure valid programs are only used to test the processor.
(3) AAPG (Automated Assembly Program Generator):This is a Python based SHAKTI developed tool, which can generate directed random assembly programs. The following section provides more details about the tool and its usage in verifying the C-class processor.
Fig. 4. Verification scheme using random program generators
Each of the above generators are used to generate programs which are then run on the processor model platform. The correctness of execution is checked by comparing the architectural state of the processor after every instruction with that of a golden instruction set simulator. Figure-4 shows how one can perform verification using these tools.
5.1 AAPG: Automated Assembly Program Generator
AAPG is a more robust tool as compared to its contemporaries. The tool is entirely developed in python and provides the user a much more finer control over generation as compared with other tools. Some of the features include:
(1) Nested Loops: The user can specify the depth of nested loops, size of each loop and the number of such nested instances. The user can also specify the instruction distribution within a loop. These parameters can be used to validate the branch predictors.
(2) Number of Forward Jumps: The user can specify an approximate number of forward and backward jumps.
(3) Recursive calls: AAPG provides a recursive template which can be configured in the number of calls and the type of instructions within each call.
(4) Cache Thrashing sequences: AAPG also provides a i-cache and d-cache thrashing template where the user only needs to provide certain parameters of the cache, such as number of ways, number of sets, etc.
(5) Data Access distribution: The user can specify which section of the data segment should be accessed how many number of times.
(6) Exception Generation: AAPG allows the user to specify the kinds of exceptions that need to be generated and also provide a place-holder to custom trap handler routines.