CHI (Arm) near/far atomic operation causes deadlock about gem5

[yichungchen84]

@ivanaamit the issue is solved. Thanks.

[tiagormk]

@ivanaamit, @yichungchen84, there are some fixes that are required so the Atomic operation is properly forwarded from the L3 to the HNF. I'm planning to have a look at that, so maybe we could keep this issue open and will keep you guys posted.

[andysan]

Ping @tiagormk.

[vsoriap]

Hi @yichungchen84, could you attach the config.ini and config.json you use.

Note that the far AMO feature is intended to be used only with three levels of cache. However, near AMO should work as the old version 23.0.0.

[yichungchen84]

Hi @vsoriap

Sorry that due to the IT restriction, I cannot share the config,ini/json files. I will see if I can share the configuration files (CHI_config.py, etc).

My configuration has private L1, private L2, shared L3, and SLC. I try to set all cache controllers "alloc_on_atomic = false". I assume it should turn off AMO feature (am I correct?), but it still has deadlocks with "alloc_on_atomic = false".

Would you mind to let me know where I can look into source code of far AMO feature/policy, so I can configure it to support my cache hierarchy? Thanks.

[vsoriap]

Okay, I see that you have 4 levels of cache. We never tested that configuration, and you may encounter some problems. How L3 and SLC have the flags is_HN configured?

alloc_on_atomic is not a flag that turns off AMO. AMOs are instructions supported in the RISC-V and ARM architecture. You cannot switch them off. What you can do is remove them from your binary.

What alloc_on_atomic really encodes is if the HNF needs to allocate a cache line or not if an Atomic Transaction arrives at the HNF and the cache line is not present. If true, the HNF requests the cache line to the SNF, performs the operation, and stores it in the local cache associated with the HNF (in your case, I assume it's the SLC). If false, the HNF requests the cache line to the SNF, performs the atomic operation, and then writes the updated value back to SNF.

If you want to simulate AMOs as they were executed in gem5 version 23.0.0. you need to set policy_type to 0 (Execute all AMO in the L1).

You can check the AMO code here

[yichungchen84]

hi @vsoriap

Thanks for the info. It is really helpful.

My configuration: L3 is_HN = false, SLC is_HN = true.

I tried another configuration L1 private, L2 shared, and SLC. This works without the issue. I guess 4-level cache it the problem.

I will try the policy_type = 0 later on.

[ivanaamit]

Hi @yichungchen84, it appears that this issue has been resolved. However, I wanted to double-check before closing it. Thanks.

[yichungchen84]

hi @ivanaamit

Thanks. The issue is resolved. I was wondering whether it will support 4-level cache int the future?

[yichungchen84]

Thanks, @tiagormk . Let me know if I can be of any help.