symbioticeda / riscv-formal Goto Github PK

The readme says:

Please open an issue if you kown of other RISC-V formal verification projects I should link to in this section.

Well... there's "Kami: A Framework for (RISC-V) HW Verification", as presented at the 4th RISC-V workshop.

Hey Clifford,

I build a small RiscV processor to get a bit of practice in the topic. The core is working but I want to verify that the core is really ISA conform. For this purpose, is modified your Picorv32 example to work with my core. The tests are starting and working but the checks for the lb, lbu, lh and lhu do always fail with the assertion assert(spec_rd_wdata == rd_wdata) . By looking at the trace files I do not get why.

The spec says that the read mask for a lb for example should be 0b1000. From my understanding the read mask should be 0b0001 since RiscV is said to be a little-endian machine. With the readmask of 0b1000 the spec says hat for a memory read data of 0x00000001 the write back value should be 0x00000000.

Can you explain me why or point me in the direction of the point I miss?

Attached is the trace file of the missing test
insn_lb_ch0.zip

Hello, I have carefully watched these docs and the introduction video for the risc-v formal framework and learned about the use of symbiyosys in order to know how this framework works. However, when I read the SBY file in the check part of the framework, I am confused about the skip in options . Considering it's a BMC process, I wonder why there is no assert error in these skipped steps? I appreciate for your early reply, thanks！

[options]
mode bmc
expect pass,fail
append 0
depth 21
skip 20

[engines]
smtbmc boolector

Hi there

I noticed that the insn_lw_ch0 check has no notion of a memory bus error. That is, it only specifies that a trap should be raised iff the address is wrongly aligned, or the misa_ok check fails.

Is there a recommended way of handling this in cores which raise a memory bus error? My current plan is to assume bus responses do not fault, and test that scenario by other means.

Cheers,
Ben

base: rvfi_reg_check.sv:19: ERROR: syntax error, unexpected TOK_RAND

We attempted to run csrw_check on all CSRs that are supported both by cv32e40x and by riscv_formal.
This did not work with MISA, because it is WARL and we have it as read-only.
I was not able to find an explanation in the docs/ directory, and the code seems to indicate that it is currently irreconcilable.
Is it the intention of the csrw_check that such registers shall not be attempted to be checked here, is it possible to configure the test differently so it works, or is it a feature that could be added to riscv_formal?

What license is riscv-formal released under?

Took a clean clone of the repo. Followed the quickstart guide to get going. Unfortunately the verification of the picorv32 results in two failed checks; csrw_mcycle_ch0 & csrw_minstret_ch0. While the quickstart guide doesn't indicate that the picorv32 should pass all of the checks, I have assumed that would be the case. I have pasted the logfile output below, of the mcycle check:

SBY 21:34:50 [csrw_mcycle_ch0] engine_0: smtbmc boolector
SBY 21:34:50 [csrw_mcycle_ch0] base: starting process "cd csrw_mcycle_ch0/src; yosys -ql ../model/design.log ../model/design.ys"
SBY 21:34:53 [csrw_mcycle_ch0] base: finished (returncode=0)
SBY 21:34:53 [csrw_mcycle_ch0] smt2: starting process "cd csrw_mcycle_ch0/model; yosys -ql design_smt2.log design_smt2.ys"
SBY 21:34:53 [csrw_mcycle_ch0] smt2: finished (returncode=0)
SBY 21:34:53 [csrw_mcycle_ch0] engine_0: starting process "cd csrw_mcycle_ch0; yosys-smtbmc -s boolector --presat --unroll --vlogtb-top wrapper.uut --noprogress -t 30:31 --append 0 --dump-vcd engine_0/trace.vcd --dump-vlogtb engine_0/trace_tb.v --dump-smtc engine_0/trace.smtc model/design_smt2.smt2"
SBY 21:34:53 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Solver: boolector
SBY 21:34:53 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Skipping step 0..
SBY 21:34:53 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Skipping step 1..
SBY 21:34:53 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Skipping step 2..
SBY 21:34:53 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Skipping step 3..
SBY 21:34:53 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Skipping step 4..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Skipping step 5..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Skipping step 6..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Skipping step 7..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Skipping step 8..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Skipping step 9..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:00  Skipping step 10..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:01  Skipping step 11..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:01  Skipping step 12..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:01  Skipping step 13..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:01  Skipping step 14..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:01  Skipping step 15..
SBY 21:34:54 [csrw_mcycle_ch0] engine_0: ##   0:00:01  Skipping step 16..
SBY 21:34:55 [csrw_mcycle_ch0] engine_0: ##   0:00:01  Skipping step 17..
SBY 21:34:55 [csrw_mcycle_ch0] engine_0: ##   0:00:01  Skipping step 18..
SBY 21:34:55 [csrw_mcycle_ch0] engine_0: ##   0:00:01  Skipping step 19..
SBY 21:34:55 [csrw_mcycle_ch0] engine_0: ##   0:00:01  Skipping step 20..
SBY 21:34:55 [csrw_mcycle_ch0] engine_0: ##   0:00:01  Skipping step 21..
SBY 21:34:55 [csrw_mcycle_ch0] engine_0: ##   0:00:02  Skipping step 22..
SBY 21:34:55 [csrw_mcycle_ch0] engine_0: ##   0:00:02  Skipping step 23..
SBY 21:34:55 [csrw_mcycle_ch0] engine_0: ##   0:00:02  Skipping step 24..
SBY 21:34:55 [csrw_mcycle_ch0] engine_0: ##   0:00:02  Skipping step 25..
SBY 21:34:55 [csrw_mcycle_ch0] engine_0: ##   0:00:02  Skipping step 26..
SBY 21:34:56 [csrw_mcycle_ch0] engine_0: ##   0:00:02  Skipping step 27..
SBY 21:34:56 [csrw_mcycle_ch0] engine_0: ##   0:00:02  Skipping step 28..
SBY 21:34:56 [csrw_mcycle_ch0] engine_0: ##   0:00:02  Skipping step 29..
SBY 21:34:56 [csrw_mcycle_ch0] engine_0: ##   0:00:02  Checking assumptions in step 30..
SBY 21:34:58 [csrw_mcycle_ch0] engine_0: ##   0:00:05  Assumptions are unsatisfiable!
SBY 21:34:58 [csrw_mcycle_ch0] engine_0: ##   0:00:05  Status: PREUNSAT
SBY 21:34:58 [csrw_mcycle_ch0] engine_0: finished (returncode=1)
SBY 21:34:58 [csrw_mcycle_ch0] engine_0: Status returned by engine: ERROR
SBY 21:34:58 [csrw_mcycle_ch0] summary: Elapsed clock time [H:MM:SS (secs)]: 0:00:08 (8)
SBY 21:34:58 [csrw_mcycle_ch0] summary: Elapsed process time [H:MM:SS (secs)]: 0:00:08 (8)
SBY 21:34:58 [csrw_mcycle_ch0] summary: engine_0 (smtbmc boolector) returned ERROR
SBY 21:34:58 [csrw_mcycle_ch0] DONE (ERROR, rc=16)

Did I miss something? Is there something missing in the instructions? Or is it something else entirely?

Thanks in advance, for any help.

The current RISC-V specification work in Sail (now Sail 2) is here:
https://github.com/rems-project/sail/tree/sail2/riscv

While trying to figure out why bugs deliberately added to the register file or bypass logic of my CPU where not caught by the register checks, I noticed that the register checks don't seem to be checking anything.

More specifically, I tried the following (on picorv32):

• Forcing rvfi_rsx_rdata to fixed values;
• Forcing rvfi_rd_wdata to a fixed value;
• Adding assert(0) on line 40 of rvfi_reg_check.sv.

In all cases, the register checks would pass just fine.

The assert(0) was added inside the if (check) test and when added instead in the else branch of that test, the check did fail. It seems therefore that the check input never becomes true which causes all checks to be disabled.

The function print_rewrite_file in monitor/generate.py has no check for RISCV_FORMAL_CSR_MISA which results in the ifdef RISCV_FORMAL_CSR_MISA line to be printed to the output.

Hi,

If I've read this thread correctly;

riscv/riscv-isa-manual#269
@kasanovic says 'Misaligned address trap should be taken precisely, implying rd would not be updated.'

This appears to be not what the RVFI checkers are expecting when rvfi_trap is raised on an unaligned branch, as they barf on mismatching rd_addr/rd_wdata (if the DUT has indeed squashed the writeback to RD of the faulting JAL/JALR instructions).

Any thoughts?

In the I-type (shift variation) instruction format (e.g. insns/insn_slli.v), the insn_funct6 field is defined as follows:

wire [6:0] insn_funct6 = rvfi_insn[31:26];

That is, while insn_funct6 is declared as a 7-bit field, it is assigned a 6-bit value. Also, in line 46 of the same file, we have insn_funct6 == 6'b 000000 as a subexpression which further suggests that insn_funct6 is treated as a 6-bit value for all intents and purposes.

I suppose this is unintentional, or am I missing something?

I'm setting a region of memory to be invalid using RISCV_FORMAL_VALIDADDR. This works as expected for instruction fetch (after fixing some bugs in my processor). Unfortunately I'm still failing some load instruction checks. The example trace it gave me for the test I looked at had a load from an invalid address with spec_trap = 0.

It looks like the following lines in rvfi_insn_check.sv
https://github.com/cliffordwolf/riscv-formal/blob/master/checks/rvfi_insn_check.sv#L104-L105
should be something like this:

        wire mem_access_fault = spec_valid && (spec_rs1_addr == rs1_addr) &&
                ((spec_mem_rmask && !mem_pma_r) || (spec_mem_wmask && !mem_pma_w)
                || ((spec_mem_rmask || spec_mem_wmask) && !`rvformal_addr_valid(spec_mem_addr)));

I tried this out on my processor and all the checks pass now.

I was going through the verification procedure and found that we need to create a genchecks.cfg file. How do I figure out how to set the values in this file?

When I run the insn_div test (without defining RISCV_FORMAL_ALTOPS) it always stops with an error after retiring any division by 0 instruction. After examining the counter trace I noticed that the signal rvfi_testbench.checker_inst.insn_spec.spec_rd_wdata is equal to 1 instead of -1.

I believe the problem is this line:

I have already read the description of RISCV_FORMAL_ALTOPS here but I think this define should also be listed in the Configuration Macros doc.

By the way, thanks for this great project. I have already caught some bugs in my implementation since I started using this framework.

I have been working with the riscv-formal for a couple of months now.
Arithmetic, load and store checks are passing but I want to ask you what would be the procedure for a JAL?, at the end of the pipe I have 2 PC values without importance when JAL comes.

I see the correct PC in the pipe.

But how could I tell that the data is not ready yet? if I put the jump trigger signal in the valid the output is warmup!

I am working with a SIIO 5-stage pipeline core.

Thank you for a great tool.

When running monitor/generate.py with default arguments, wire ialign16 is not generated in the output of any instruction module. It appears that print_rewrite_file cannot handle the nested `ifdefs present in the various insn_*.v files.

I'm a bit confused whether this is an issue with riscv-formal or if I'm just holding it wrong. The issue I see is that the reg test fails because of a mismatch for rvfi_rs1_rdata. I think the problem is that the rs1 data is read from the wrong register. It's an optimization, which is fine because we never care about the data in rs1 dugin CSR*I instruction. But it seems like riscv-formal needs to have correct rs1 data here. Is that the intention, or can we relax the requirement in this case?

Hi Clifford,
I was testing my core in the last few days and I've notice some problems when testing the compressed ISA, my IP is able to pass in all checks but not in the c.j and c.jal instructions, then when I saw the error log, it was displayed that fails in the assertion of trap. Looking deeper in the VCD I understood the problem, that's related to the LSB being set in the jump address, so then my IP raised a synchronous trap what it was not expected by the rvfi_insn_check.v. Commenting the code segment like the image below of a trap generation shows that the IP pass in all the tests. Could you clarify me that verify LSB it's wrong on these instructions?

It seems that there is a updated and actively developed version at https://github.com/YosysHQ/riscv-formal, so is this repo discontinued?

hi
riscv-formal can only verify RV32I and RV64I? how about the privileged spec, can it be used to verify it?
thanks
Peter

Dear developers, I followed the quickstart guide document to setup my environments(installed latest git versions via git-clone-make or AUR xxx-git package), and in a fresh repository clone I got these error when make -C checks:

Below shows my yosys and boolector version.
When running make -C checks again, seems other parts of the test is being run, and sometimes I get similar error while other times it showed PASS. Just picking out two other errors I met:

SBY 22:42:58 [csrw_mcycle_ch0] base: starting process "cd csrw_mcycle_ch0/src; yosys -ql ../model/design.log ../model/design.ys"
SBY 22:42:58 [csrw_mcycle_ch0] base: ERROR: Assert `ifdef_already_satisfied' failed in frontends/verilog/preproc.cc:758.
SBY 22:42:58 [csrw_mcycle_ch0] base: finished (returncode=1)
SBY 22:42:58 [csrw_mcycle_ch0] base: job failed. ERROR.

SBY 22:42:56 [cover] base: starting process "cd cover/src; yosys -ql ../model/design.log ../model/design.ys"
SBY 22:42:56 [cover] base: ERROR: Module `picorv32' referenced in module `rvfi_wrapper' in cell `uut' does not have a port named 'rvfi_csr_minstret_wdata'.
SBY 22:42:56 [cover] base: finished (returncode=1)
SBY 22:42:56 [cover] base: job failed. ERROR.

What would my problem be in this situation?

Peter Gu

When running formal verification of an instruction on a risc-v core, not all assertions in the rvfi_insn_check.sv file are relevant, and can cause a lot of noise in the form of unmet related covers. A good example of this is rs2-related assertions when checking a I-type instruction, as there is no rs2-field, the preconditions can not be met, which shows as an error in the formal verification run.

I propose that these assertions are filtered by ifdef/generate based on instruction type, so the assertion only gets generated if the assertion is relevant for the instruction that is being checked.

Hi,
The Ibex core has mechanisms to handle data memory transactions returning error responses. The RVFI spec details that rvfi_trap should be used to indicate memory access violations, but is there any way to signal a load/store error through the interface? Right now, I can view all memory transactions through the RVFI, but I have no way of knowing whether the transaction is actually valid, or has raised an error response.
Thanks!

Hi,

The proposed Zce extension has various instructions that have multiple side effects. Currently such instructions cannot be modeled with RVFI. An example are the PUSH and POP instructions, see https://github.com/riscv/riscv-code-size-reduction/blob/master/ISA%20proposals/Huawei/Zce_spec.adoc#pushpoppopret. Such instructions can perform multiple loads/stores, update multiple register file registers and optionally perform a jump.

The following could maybe all be changed into arrays to support multiple register file writes and multiple loads/stores per instruction:

output [NRET *    5 - 1 : 0] rvfi_rd_addr
output [NRET * XLEN - 1 : 0] rvfi_rd_wdata

output [NRET * XLEN   - 1 : 0] rvfi_mem_addr
output [NRET * XLEN/8 - 1 : 0] rvfi_mem_rmask
output [NRET * XLEN/8 - 1 : 0] rvfi_mem_wmask
output [NRET * XLEN   - 1 : 0] rvfi_mem_rdata
output [NRET * XLEN   - 1 : 0] rvfi_mem_wdata

Are there any plans to add assertions for Zce and for the B extension (which would require an additional register file read port)?

On https://github.com/SymbioticEDA/riscv-formal there's a Table of Contents that gives good information, but I have a new RISC-V CPU I want to try to test and need to know the procedure to create a folder and whatever else to be able to test it. This would be a good webpage addition to have for those of us wanting to test new CPUs. I finally found some info on page 15 of http://www.clifford.at/papers/2017/riscv-formal/slides.pdf. It gives a little info and says "Write a genchecks.cfg config file for the new core–See cores/picorv32/ and cores/rocket/ for examples" Problem is, I can't find the genchecks.cfg file in either folder. More info on what should go in genchecks.cfg would be appreciated.

While attempting to add support for a RV32IC core, I have encountered Yosys error messages while attempting to run tests.

Unfortunately, I cannot include the code to reproduce, so I was wondering if you might be able to shed some light on the meaning of the error (which is the first error with no preceding errors or warnings).

make: *** [insn_addi_ch0/PASS] Error 3
SBY [pc_bwd_ch0] smt2: ERROR: Assert `sig.size() == 1 && sig.chunks().size() == 1' failed in ./kernel/rtlil.h:1301.
SBY [pc_bwd_ch0] smt2: finished (returncode=1)
SBY [pc_bwd_ch0] smt2: job failed. ERROR.
SBY [pc_bwd_ch0] summary: Elapsed clock time [H:MM:SS (secs)]: 0:00:15 (15)
SBY [pc_bwd_ch0] summary: Elapsed process time [H:MM:SS (secs)]: 0:00:15 (15)
SBY [pc_bwd_ch0] DONE (ERROR, rc=3)

The documentation mentions that

Some arithmetic operations (such as multiplication and division) are beyond to practical capabilities of even modern hardware model checkers.

However, there are checks implemented for the MUL/DIV family of instruction (e.g., insn_mul.v), and their implementations seem sensible.

So I was wondering what "beyond the practical capabilities" exactly means. Do we just need to be more patient with the model checker (and maybe increase the depth for insn checks) or is it really useless?

Hi there

I think there are some mismatches between riscv-formal and the spec (v20190305-Base-Ratification) with the C extension decoding.

• Some C instructions require that operand fields take a specific value in order to be correctly decoded.
• This means that either rs1/rd != 0, or rs2 != 0`.
• A subset of the RV32 C instructions in riscv-formal do not perform these check on the operand fields.

The following instructions are missing operand-field value checks for rd/rs1!=0 in their spec_valid signal assignment:

• c.li
• c.lui
• c.addi
• c.slli
• c.mv
• c.add

The checks are missing both in the generated code insns/insn_*.v and in the generator code insns/generate.py.

If you agree this is a problem, I'm happy submitting a PR to fix it, though what that would do to cores currently using / passing the checks I can't say.

RISC-V formal verification project: https://github.com/mrLSD/riscv-fs

under active development

Hello,

I have a few questions regarding the verification of the rocket core as I am not able to run it successfully.

00f6ff5 made a switch to a non public repo?

If using the alternate repo from freechipsproject, yosys must be compiled with verific support, which is not possible with only the evaluation license?

If I go back to dbb2d9b the check csrw_mcycle_ch1 fails with Warmup failed!. I did not investigate further as it might not make sense to work on an old state.

Am I wrong to assume that with out a verific license and access to the sifive repo, it is not possible to run the verification for the rocket core?

Thanks!

Following the exercise 1, no vcd trace is created.

Hi Clifford,
first thank you for this project, seems a really good way to handle new open source designs as the community grows!. So, I'm trying to get started with it inserting the RVFI in my core but I have some simple questions due to briefly understand that are:

1. How you should write the code segment like *_FAIRNESS macros, is this something to emulate memory behavior?
2. Is there some doc related how can I configure the numbers inside checks.cfg in depth segment?
   like:

insn              20
reg       15    30

I understand that it refers to cycles of test but why some times you have 2 ou even 3 numbers defined at....should be something like:
type_of_checking reset_cycles trigger_cycle checker_cycle
?
I know that are quite basic things due to the project but it's not so close to me how to setup it.
Best regards, Anderson

Hi

in the description of the interface, rvfi_valid is said to indicate the following

1. When the core retires an instruction, it asserts the rvfi_valid
2. The signals are only valid during such a cycle and can be driven to arbitrary values in a cycle in which rvfi_valid is not asserted.

later

rvfi_trap must be set for an instruction that cannot be decoded as a legal instruction, such as 0x00000000.

The purpose if the rvfi_trap is to indicate an exception, an instruction which causes an exception, by definition does not retire.
But this contradicts the previous statement, a good example is the misaligned LD/ST

If I have the following instruction (assuming the implementation does not support misaligned LD/ST)
lw x1, x0(1)

This will take an exception, and X1 will not update as the instruction does not complete (retire)
How can this be described in the interface, as it should be the following
rvfi_valid=0, rvfi_trap=1, ....

But this would be ignored as rvfi_valid is 0

If rvfi_valid indicates a retirement, and rvfi_trap indicates an exception - then these two signals must surely be mutually exclusive ?

I tried the spike test in tests/spike, and it failed on the first instruction (add).
I cloned this repo just before sending this issue so i have the latest version.
I've seen that there is also an issue with the picorv sample?

In some cores, such as CV32E40P, the processor will retire an instruction before knowing that it was the last instruction prior to a halt being executed. A simple example is when the last instruction is a branch, the next instruction has not been retrieved and an external halt request is executed.

Can you change the rvfi_halt definition to be set on the first instruction that is part of the debug handler (similar to rvfi_intr).

Also, can you expand on the statement “This signal enables verification of liveness properties”?

I have been playing with riscv-formal for checking one of our small processors, and I came across a few issues where the documentation does not appear to match what the checks are actually doing. Our small processor supports a bit less than the bare minimum of RV32I (It's actually a bare minimal RV32E processor that I temporarily increased the number of registers to 32 registers just for testing with riscv-formal - as a result, its basically just an RV32I processor without counters). It supports interrupts and exceptions, but for the purposes of these tests, I have hardwired the interrupt inputs to 0. I'm currently running the same checks as picorv except I changed the isa to rv32i and I'm skipping liveness for now.

1. rvfi_intr - According to rvfi.md: "rvfi_intr must be set for the first instruction that is part of a trap handler, i.e. an instruction that has a rvfi_pc_rdata that does not match the rvfi_pc_wdata of the previous instruction.". Looking at the checks, it seems like rvfi_intr is ignored. My implementation sets rvfi_pc_wdata to the start of the trap handler for each instruction that causes a fault and it passes all the checks that are done for picorv (except liveness - I'm skipping it for now).

2. rvfi_rs1_addr and rvfi_rs2_addr - According to rvfi.md: "For an instruction that reads no rs1/rs2 register, this output can have an arbitrary value." Our processor always assumes rs1 is insn[19:15] and rs2 is insn[24:20]. I tried keeping those values for rvfi_rs1_addr and rvfi_rs2_addr, but I failed checks for addi, lui, and other similar instructions that don't use rs1 and/or rs2. I ended up changing rvfi_rsX_addr and rvfi_rsX_rdata to 0 if the instruction doesn't use rsX, and with that change I passed all the tests. This is consistent with what I saw when looking at the instruction check system verilog.

Anyways, I'm really enjoying using with riscv-formal. Thanks!

Hi Clifford,

I'm attempting to integrate a rocket chip core to riscv-formal.
I've run "python3 ../../checks/genchecks.py" successfully.
When I run "make -C checks -j2", the initial test runs for about 2 minutes, console output below.
Any help with the errors below would be greatly appreciated.

Kind Regards,
Ciaran

Console Output
SBY [causal_ch0] engine_0: ## 115 0:01:55 Warmup failed!
SBY [causal_ch0] engine_0: ## 115 0:01:55 Status: FAILED (!)
SBY [causal_ch0] engine_0: finished (returncode=1)
SBY [causal_ch0] engine_0: Status returned by engine: FAIL
SBY [causal_ch0] summary: Elapsed clock time [H:MM:SS (secs)]: 0:02:03 (123)
SBY [causal_ch0] summary: Elapsed process time [H:MM:SS (secs)]: 0:02:08 (128)
SBY [causal_ch0] summary: engine_0 (smtbmc --presat boolector) returned FAIL
SBY [causal_ch0] summary: counterexample trace: causal_ch0/engine_0/trace.vcd
SBY [causal_ch0] DONE (FAIL, rc=2)
make: *** [causal_ch0/PASS] Error 2
make: *** Waiting for unfinished jobs....
SBY [causal_ch1] engine_0: ## 117 0:01:57 Checking assumptions in step 36..
SBY [causal_ch1] engine_0: ## 117 0:01:57 Warmup failed!
SBY [causal_ch1] engine_0: ## 117 0:01:57 Status: FAILED (!)
SBY [causal_ch1] engine_0: finished (returncode=1)
SBY [causal_ch1] engine_0: Status returned by engine: FAIL
SBY [causal_ch1] summary: Elapsed clock time [H:MM:SS (secs)]: 0:02:05 (125)
SBY [causal_ch1] summary: Elapsed process time [H:MM:SS (secs)]: 0:02:09 (129)
SBY [causal_ch1] summary: engine_0 (smtbmc --presat boolector) returned FAIL
SBY [causal_ch1] summary: counterexample trace: causal_ch1/engine_0/trace.vcd
SBY [causal_ch1] DONE (FAIL, rc=2)
make: *** [causal_ch1/PASS] Error 2
make: Leaving directory `/home/user/riscv-formal/cores/MV/checks'
user@user:~/riscv-formal/cores/MV$ make -C checks -j2
SBY [causal_ch0] engine_0: ## 115 0:01:55 Warmup failed!
SBY [causal_ch0] engine_0: ## 115 0:01:55 Status: FAILED (!)
SBY [causal_ch0] engine_0: finished (returncode=1)
SBY [causal_ch0] engine_0: Status returned by engine: FAIL
SBY [causal_ch0] summary: Elapsed clock time [H:MM:SS (secs)]: 0:02:03 (123)
SBY [causal_ch0] summary: Elapsed process time [H:MM:SS (secs)]: 0:02:08 (128)
SBY [causal_ch0] summary: engine_0 (smtbmc --presat boolector) returned FAIL
SBY [causal_ch0] summary: counterexample trace: causal_ch0/engine_0/trace.vcd
SBY [causal_ch0] DONE (FAIL, rc=2)
make: *** [causal_ch0/PASS] Error 2
make: *** Waiting for unfinished jobs....
SBY [causal_ch1] engine_0: ## 117 0:01:57 Checking assumptions in step 36..
SBY [causal_ch1] engine_0: ## 117 0:01:57 Warmup failed!
SBY [causal_ch1] engine_0: ## 117 0:01:57 Status: FAILED (!)
SBY [causal_ch1] engine_0: finished (returncode=1)
SBY [causal_ch1] engine_0: Status returned by engine: FAIL
SBY [causal_ch1] summary: Elapsed clock time [H:MM:SS (secs)]: 0:02:05 (125)
SBY [causal_ch1] summary: Elapsed process time [H:MM:SS (secs)]: 0:02:09 (129)
SBY [causal_ch1] summary: engine_0 (smtbmc --presat boolector) returned FAIL
SBY [causal_ch1] summary: counterexample trace: causal_ch1/engine_0/trace.vcd
SBY [causal_ch1] DONE (FAIL, rc=2)
make: *** [causal_ch1/PASS] Error 2

It seems the checker is wrong as both picorv32 and cv32e40x get failing assertions and the traces look weird.
Otherwise, I might be wrong and have misunderstood something here.
If anyone else currently see reg_check working fine then please let me know.

Brief:
Register checks should execute (at least) 2 instructions, then check that the last one reads correctly that which the first one wrote.
What I observe is that the register read is arbitrary (undriven) and does not match the register read.
Example: Write rd_wdata='hABCD to rd_addr=1, and then CEX because a read from rs1_addr=2 (note: different than rd) does not have matching data rs1_rdata != 'hABCD.

Evidence 1

picorv32 see failure when running the reg check.

Fresh build/install of yosys/symbiyosys/boolector from:
https://symbiyosys.readthedocs.io/en/latest/install.html
Followed the README in cores/picorv32 to wget the source and launch checks.
Failure reported in "reg" checks.
NB! It was not able to parse by default, so I changed the const rand reg macro to just reg. (this was not changed for cv32e40x using a different formal tool).

"logfile.txt" line 63:

SBY 15:21:37 [reg_ch0] engine_0: ##   0:00:13  Assert failed in rvfi_testbench: rvfi_testbench.sv:39.17-57.3|rvfi_reg_check.sv:43.71-44.59

"src/rvfi_reg_check.sv" line 44:

                                        assert(register_shadow == rvfi_rs2_rdata[`RISCV_FORMAL_CHANNEL_IDX*`RISCV_FORMAL_XLEN +: `RISCV_FORMAL_XLEN]);

The rs2 check fails.

This might also be a bug in picorv32, or it might be a bug in my methodology.

Evidence 2

cv32e40x see failure when running the reg check, and the checker seems to do an arbitrary comparison.

A similar problem is observed here. There is a CEX because the register_shadow was set for a different register than the one being read later (rs1).

It is also possible to get a trace where only 1 instruction retires, but that seems counter to the purpose of the test. I would think assumes should be in place to not permit this.

Thoughts

I don't fully understand the code,
but it seems that nothing is driving register_index and that it should maybe be stored similar to how register_shadow is stored.

Extra info - Possibly a similar bug in the PC checks

There is a similar issue in pc_fwd and pc_bwd where the expected PC can be stored once, then one or more instructions may follow, and finally the check happens but the expected PC was not updated by the intermediate instructions so the comparison is done against an outdated expectation and gets an error

The RVFI spec states:

The signal rvfi_halt must be set when the instruction is the last instruction that the core retires before halting execution.

It is unclear when this signal is supposed to be used. When is a core supposed to 'halt execution'? Is this related to WFI and/or maybe to debug entry?