ibm-capi / pslse Goto Github PK

Hi

Code in the top-level libcxl folder is intended for HW deployment. This is now redundant and instead we should be using https://github.com/ibm-capi/libcxl.git. Can we remove this libcxl folder to avoid confusion?

Thanks

afu_driver/src/Makefile determines the VPI_USER_H_DIR variable automatically when Cadence NCSIM is installed. If not, it will fail with the message to set VPI_USER_H_DIR.
Change request: Add automatic setting when Xilinx tools are installed.

Proposed changes (diff):
diff --git a/afu_driver/src/Makefile b/afu_driver/src/Makefile
index f1d54c2..0e015fb 100644
--- a/afu_driver/src/Makefile
+++ b/afu_driver/src/Makefile
@@ -12,6 +12,13 @@ ifndef VPI_USER_H_DIR
endif
endif

+# if VPI_USER_H_DIR is still not set, try to use Xilinx install dir path
+ifndef VPI_USER_H_DIR
+ ifdef XILINX_VIVADO
+ VPI_USER_H_DIR=$(XILINX_VIVADO)/data/xsim/include
+ endif
+endif
+
# if VPI_USER_H_DIR is STILL not set, print an error message
ifndef VPI_USER_H_DIR
$(error Must set VPI_USER_H_DIR to point to include path for the simulator)

https://github.com/ibm-capi/libcxl.git support's new functions like
cxl_afu_host_thread_wait()
cxl_work_get_tid()
cxl_work_enable_wait()
cxl_work_disable_wait()

Are you planing to get this implemented into PSLSE ?

ibcxl.c: In function '_psl_loop':
libcxl.c:1127: warning: unused variable 'op2'
libcxl.c:1127: warning: unused variable 'op1'
libcxl.c:1123: warning: unused variable 'function_code'
libcxl.c:1123: warning: unused variable 'op_size'
libcxl.c: In function '_pslse_connect':
libcxl.c:1499: error: 'PSLSE_VERSION_MAJOR' undeclared (first use in this function)
libcxl.c:1499: error: (Each undeclared identifier is reported only once
libcxl.c:1499: error: for each function it appears in.)
libcxl.c:1500: error: 'PSLSE_VERSION_MINOR' undeclared (first use in this function)
make: *** [libcxl.o] Error 1

would be great if those could get fixed. They might hide real issues.

Hey,

We are having an issue with the new dynamic credits feature.

More often then not it returns 0 credits. So if we submit a handful of requests and then wait for all the credits to return then we will more often than not deadlock. All the responses would return 0 credits and no credits get returned unless we submit more commands. It's rare for the AFU to see that the PSL is idle and it will certainly never happen unless we are submitting new commands.

The change also makes no sense to me. Does the real hardware actually behave this way? Why would it complete a command and not have space for a new command? If this is the case it needs a separate way to return credits when they become available and no new tags have been submitted.

Thanks,

Logan

bash-4.1$ ldd ../../pslse/libcxl/libcxl.so
linux-vdso.so.1 => (0x00007fffb81d7000)
libc.so.6 => /lib64/libc.so.6 (0x00002b228488b000)
/lib64/ld-linux-x86-64.so.2 (0x00000030df800000)
bash-4.1$

This enforces that all applications using this must do a -pthread to avoid problems during linkage.
Better is this:

bash-4.1$ ldd libcxl.so
linux-vdso.so.1 => (0x00007fff551d8000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b12ca601000)
libc.so.6 => /lib64/libc.so.6 (0x00002b12ca81f000)
/lib64/ld-linux-x86-64.so.2 (0x00000030df800000)
bash-4.1$

Using this:

diff --git a/libcxl/Makefile b/libcxl/Makefile
index a956dc6..46f17be 100644
--- a/libcxl/Makefile
+++ b/libcxl/Makefile
@@ -4,6 +4,7 @@ include Makefile.vars
include Makefile.rules

OBJS = libcxl.o debug.o utils.o
+LDLIBS = -pthread

all: libcxl.so libcxl.a

@@ -17,7 +18,7 @@ endif
libcxl.o: $(COMMON_DIR)/misc/cxl.h

libcxl.so: $(OBJS)

•   $(call Q,CC, $(CC) $(CFLAGS) -shared $^ -o $@, $@) -Wl,--version-script symver.map
  

•   $(call Q,CC, $(CC) $(CFLAGS) -shared $^ -o $@ $(LDLIBS), $@) -Wl,--version-script symver.map
  

libcxl.a: $(OBJS)
$(call Q,AR, $(AR) rcs $@ $^, $@)
bash-4.1$

Hi,

It turned out (by experiment ...) that reading FIRs via MMIO fails once the AFU reported the error.
We mapped our FIRs into the errinfo area in the AFU configuration space and saw that we get the FIRs even after the AFU noticed the error. I therefore like to change our FIR checking code to poll the FIRs via the error buffer and not via MMIO anymore. For this I would need real and not fake versions of the functions above.
Can you give me an outlook when you will have some time to get this done?
By the way I saw in Helens (?) git, that she has already cr_device, cr_vendor() functions which appear more real than my fake functions. Are those ready to be tried out in the afudirected branch some day?

Thanks

Frank

Howdy,

I'm getting an issue where the ah_csize of my AFU output is getting dropped to 0 in PSLSE. Here's some of the debug output that shows this issue.

DEBUG:afu0.0:JOB code=0x90 ea=0x0000000001e77080
DEBUG:afu0.0:JOB running
DEBUG:afu0.0:COMMAND tag=0xff code=0x0a00 size=0x80 abt=0 cch=0x0000
DEBUG:afu0.0:COMMAND tag=0xff addr=0x0000000001e77080
WARNING:AFU issued command with invalid size 0!
DEBUG:afu0.0:RESPONSE tag=0xff code=0x8

I run into this on on both the memcopy example from ibm developerworks and my test repo at https://github.com/KennethWilke/capi-tinker

Sometimes this bug doesn't occur, it seems to happen less often with the debug build.

I'll try to find out more if I can

there is a report of a couple of itag anomalies. first, there is a case where an xlate request is issued, but does not get an itag in response. second, there is a case where an itag is issued a second time while the afu has not yet used the first.

Hi,

We are having trouble with commit 01b571f.

The new active tag flag gets set but only gets cleared on initialization. So we can never reuse a tag even if it has correctly finished processing.

Also, we weren't aware that having duplicate tags in process at the same time was an issue. We've changed our RTL to avoid this.

Logan

Hi,

I think that too much data is printed out in default mode. I suggest that this is reduced a bit and made potentially configurable: best at runtime 2nd best compile time.

So this here already reduces the output quite a bit:

diff --git a/afu_driver/src/afu_driver.c b/afu_driver/src/afu_driver.c
index 0eaa5db..59d6da9 100644
--- a/afu_driver/src/afu_driver.c
+++ b/afu_driver/src/afu_driver.c
@@ -672,10 +672,12 @@ void psl_bfm(const svLogic       ha_pclock,               // used as pclock on PLI
          *ha_mmdatapar_top = (event.mmio_wdata_parity) & 0x1;          // 2016/05/11: UMA: checking whether ensuring bval is set always to 0
          *ha_mmcfg_top = event.mmio_afudescaccess;
          *ha_mmval_top = 1;
+#if 0
          printf("%08lld: ", (long long) c_sim_time);
          printf("MMIO rnw=%d dw=%d addr=0x%08x data=0x%016llx\n",
                     event.mmio_read, event.mmio_double, event.mmio_address,
                     (long long)event.mmio_wdata);
+#endif
          cl_mmio = CLOCK_EDGE_DELAY;
          event.mmio_valid = 0;
         }      

Please consider putting this behind e.g. #ifdef DEBUG or alike.

Thanks

Frank

The get_signal64() function is sometimes reading a wrong value due to an overflow in this line:

*data += (uint64_t) value.value.vector[0].aval;

As it is a bit-level concatenation I replaced it by:

*data |= (((uint64_t) value.value.vector[0].aval) & 0x00000000ffffffff);

This resolves the problem which might be machine/OS/compiler related.
We have been observing this on x84 Intel, RHEL 6.4, gcc 4.4.7

Hello,

I am raising this issue, cause the PSLSE CAPI 1.0 doesn't adhere to the Translation ordering ahX_cabt input to the PSL, it assumes a perfect scenario all the time. ahX_cabt is an important way to control the behavior and performance of the PSL as stated by the CAPI 1.0 document. I am extremely interested in the interaction of the AFU with the PSL cache And don't want to use the prefetching "TOUCH_I" commands for all cases. Hence, when we have a PAGED response or FAULT from the PSL, for example, the AFU needs to handle this properly with the PSL. Instead, we can only generate PAGED responses but the interaction with the PSLSE matters (there are no flushes and RESTART commands doesn't affect the PSLSE)

Thanks a lot in advance...!!

haver@ubuntu-on-mrmac:/GenWQE/pslse/libcxl$ make clean all
rm -f .[od] ../common/.[od] gmon.out libcxl.so libcxl.a ../common/misc/cxl.h
[CC] libcxl.o
In file included from libcxl.c:32:0:
libcxl.h:21:22: fatal error: misc/cxl.h: No such file or directory
compilation terminated.
Makefile.rules:22: recipe for target 'libcxl.o' failed
make: *** [libcxl.o] Error 1
haver@ubuntu-on-mrmac:/GenWQE/pslse/libcxl$ make clean all
rm -f .[od] ../common/.[od] gmon.out libcxl.so libcxl.a ../common/misc/cxl.h
[CURL ../common/misc/cxl.h]
[CC] libcxl.o
[CC] libcxl.d
[SED] libcxl.d
[CC] debug.o
[CC] debug.d
[SED] debug.d
[CC] utils.o
[CC] utils.d
[SED] utils.d
[CC] libcxl.so
[AR] libcxl.a
haver@ubuntu-on-mrmac:~/GenWQE/pslse/libcxl$

A user has requested that a version of pslse be created that prohibits the use of commands and interfaces that are not supported by the psl9d implementation. psl9d essentially only supports the dma command and data interfaces. The user code to the specification interface only to discover later that the psl hardware had limited support

I notice that the MMIO read and write functions behave different from the libcxl implementations.

I am mapping the MMIO registers using:
cxl_mmio_map(afu_h, CXL_MMIO_LITTLE_ENDIAN)

When reading a BE value (on hardware) using cxl_mmio_read64(afu_h, 0x18, &trc_r) (of the pslse) I get a BE value in the variable trc_r.
If using the cxl_mmio_read64() function from libcxl I get a little endian version of the BE value on the hardware. Thus I need to swap the bytes.

The behavior applied to cxl_mmio_write64() functions.

Does the PSLSE capture the cache behavior of CAPI and Power 8? Please let me know your opinions.

We created a pthread which was supposed to poll a hardware register. Once we did not need this thread anymore, we canceled it by doing a pthread_cancel(). That caused in some cases deadlocks, due to a lock in a subsequent call to mmio_read/write which required the lock which was not unlocked.

To circumvent the problem we used

pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &oldstate);
do something ...
cxl_mmio_read32()
...
pthread_setcancelstate(oldstate, NULL);

Please review and fix functions which acquire locks and which are likely to be used in multithreaded code which can be stopped with pthread_cancel().

Thanks

Frank

We recently found a case where we called cxl_afu_free() while some operations were still in progress. This was a timeout issue where we did not see completion after a certain amount of time. So as result we liked to release the afu, print an error message and go out.

Unfortunately libcxl was not willing to let us go and ended up looping in the following code:

	while (afu->attached)	/*infinite loop */
		_delay_1ms();

Since the code which seemed to be responsible to modify afu->attached was not working as expected, we think that it waited for an ACK from the hardware which was not really coming due to incomplete implementation, the pslse variant of libcxl was stuck and therefore our application too. Only gdb attaching to the hanging process helped to figure that out.

So could you please have a look in your code where there are loops without an emergency exit condition and consider fixing those up, such we do not see those hang conditions anymore? I am aware that this is an error condition but still thing we should handle it as smooth as possible. Thanks.

I found the following in the CAPI2.0 docu:
Read buffer latency. This bus is a static indicator of the access latency of the read
buffer. It must not change while there are commands that have been submitted on
the command interface that have not been acknowledged on the response interface.
It is sampled continuously. However, after a reset, the PSL assumes this is a
constant and that it is static for any particular AFU.
0 Data is ready the cycle after ha_brvalid is asserted.
1 Data is ready the second cycle after ha_brvalid is asserted.
2 Data is ready the third cycle after ha_brvalid is asserted.
All other values are illegal

But in pslse/cmd.c I found this:
case PSL_COMMAND_WRITE_MI: /*fall through */
case PSL_COMMAND_WRITE_MS: /*fall through */
case PSL_COMMAND_WRITE_NA: /*fall through */
case PSL_COMMAND_WRITE_INJ: /*fall through */
if (!(latency % 2) || (latency > 3))
error_msg("Write with invalid br_lat=%d", latency);
_add_write(cmd, handle, tag, command, abort, addr, size,
unlock);
break;

The result is that br_lat equal 2 is not working!

i don't know where VPI_USER_H_DIR is and how to set it

when i run top.v in the afu_driver/verilog ,i found the ");" is extra in 21 line .When i delete it ,i can simulate it
in vivado succeffully.please check it.thank you

This issue is open to address Joerg Kayser's request to add more debug info/error checking for cases where AFU designs exceed the spec limit of 8 (each) outstanding DMA_rd or DMA_wr requests.
The afu_driver code isn't checking the return code from calls to psl_interface, so it's not a quick fix to force a hard stop in case of error. However, psl_interface code can be made to track # of outstanding DMA requests, and printout error messages indicating when limits have been exceeded. In that case, simulation will stop once the afu disconnects from the socket (which is does when afu_driver can no longer pass on incoming DMA requests).

Hi Helena,
from your subset of the trace I cannot see, when any architectural limit is exceeded.

But if we all agree, that exceeding the maximum of pending requests is a hard protocol violation and finally screws the whole application, we should urgently push out a limit enforcement. This can be a clear ERROR message or even a hard PSLSE stop, but not just forgetting to serve this request.

Please add an appropriate msg and/or HALT to the code to better find this violation of the queue limit.
Thanks

Mit freundlichen Grüßen / Kind regards

Joerg Kayser
D/6833 Unit Verification, IBM Systems & Technology Group
e-mail: [email protected] Phone: +49-7031-16-2445

I would like to know what happens, when I have an open connection to pslse in my application and then someone kills pslse for whatever reason. Will my application hang or will there be error codes returned from the libcxl functions I am using?

I noticed cxl_get_caia_version and cxl_get_psl_revision are currently omitted from symver.map

Hi

Currently only brlat values of 1 and 3 are supported in PSL. However PSLSE has no issues with the value of 2, even though this causes HW failure. Can the simulation raise an exception if an invlaid value of brlat is applied on the PSL interface?

Thanks

Stephen

When compiling the master branch of PSLSE, we see compile errors in libcxl. The QUICKSTART notice shows new environment variables PSL8 and PSL9 to be set, but there is no information in README. Also according to QUICKSTART, the PSL8 and PSL9 should not be set both to 1. This cries for a better solution

• a hint in Makefile, if compile fails due to variables not set
• or a default, if the user specifies no environment
• one environment variable (PSL = P8 / P9) to prevent invalid settings
• a hint in README to read QUICKSTART

and finally (unrelated to the above)

• one Makefile in PSLSE_ROOT to call the subdirectory Makefiles in afu_driver/src, libcxl, debug and pslse

Request made to mask off 3 upper address bits (reduce from 10 bits to 7 bits).
Without the upper 3 bits it would be:

1. 64B on address 128: laddr = 0, data left aligned
2. 64B on address 192: laddr = 64, data left aligned
3. 64B on address 256: laddr = 0, data left aligned..

PSLSE is working well with REORDER_PERCENT: 0, but if I use other values (REORDER_PERCENT: 80,90) it hangs. I see in the waveviewer only one response for all the commands! Below two short parts of the debug.lst:

GOOD ONE:
afu0.0,0:CMD: New tag=0x00 code=0x0a00
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x00
afu0.0,0:CMD: New tag=0x01 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x00
afu0.0:MMIO: Ack
afu0.0:CMD: Buffer Write request tag=0x00
afu0.0,0:CMD: Update tag=0x00 resp=0x00
afu0.0,0:CMD: New tag=0x02 code=0x0a00
afu0.0,0:SOCKET OUT: MMIO ACK
afu0.0,0:MMIO: Return
afu0.0:CMD: Response tag=0x00
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x01
afu0.0,0:CMD: New tag=0x03 code=0x0a00
afu0.0,0:SOCKET IN: READ32
afu0.0,0:MMIO: Added Read32 Address=0x003c00
afu0.0,0:CMD: New tag=0x04 code=0x0a00
afu0.0:MMIO: Sent Read32 Address=0x803c00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x01
afu0.0:CMD: Buffer Write request tag=0x01
afu0.0,0:CMD: Update tag=0x01 resp=0x00
afu0.0,0:CMD: New tag=0x05 code=0x0a00
afu0.0:CMD: Response tag=0x01
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x02
afu0.0,0:CMD: New tag=0x06 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x02
afu0.0:CMD: Buffer Write request tag=0x02
afu0.0,0:CMD: Update tag=0x02 resp=0x00
afu0.0,0:CMD: New tag=0x07 code=0x0a00
afu0.0:CMD: Response tag=0x02
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x03
afu0.0,0:CMD: New tag=0x08 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x03
afu0.0:CMD: Buffer Write request tag=0x03
afu0.0,0:CMD: Update tag=0x03 resp=0x00
afu0.0,0:CMD: New tag=0x09 code=0x0a00
afu0.0:CMD: Response tag=0x03
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x04
afu0.0,0:CMD: New tag=0x0a code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x04
afu0.0:CMD: Buffer Write request tag=0x04
afu0.0,0:CMD: Update tag=0x04 resp=0x00
afu0.0,0:CMD: New tag=0x0b code=0x0a00
afu0.0:CMD: Response tag=0x04
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x05
afu0.0,0:CMD: New tag=0x0c code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x05
afu0.0:CMD: Buffer Write request tag=0x05
afu0.0,0:CMD: Update tag=0x05 resp=0x00
afu0.0,0:CMD: New tag=0x0d code=0x0a00
afu0.0:CMD: Response tag=0x05
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x06
afu0.0,0:CMD: New tag=0x0e code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x06
afu0.0:CMD: Buffer Write request tag=0x06
afu0.0,0:CMD: Update tag=0x06 resp=0x00
afu0.0,0:CMD: New tag=0x0f code=0x0a00
afu0.0:CMD: Response tag=0x06
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x07
afu0.0,0:CMD: New tag=0x10 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x07
afu0.0:MMIO: Ack
afu0.0:CMD: Buffer Write request tag=0x07
afu0.0,0:CMD: Update tag=0x07 resp=0x00
afu0.0,0:CMD: New tag=0x11 code=0x0a00
afu0.0,0:SOCKET OUT: MMIO ACK
afu0.0,0:MMIO: Return
afu0.0:CMD: Response tag=0x07
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x08
afu0.0,0:CMD: New tag=0x12 code=0x0a00
afu0.0,0:SOCKET IN: READ32
afu0.0,0:MMIO: Added Read32 Address=0x003c00
afu0.0,0:CMD: New tag=0x13 code=0x0a00
afu0.0:MMIO: Sent Read32 Address=0x803c00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x08
afu0.0:CMD: Buffer Write request tag=0x08
afu0.0,0:CMD: Update tag=0x08 resp=0x00
afu0.0,0:CMD: New tag=0x14 code=0x0a00
afu0.0:CMD: Response tag=0x08
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x09
afu0.0,0:CMD: New tag=0x15 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x09
afu0.0:CMD: Buffer Write request tag=0x09
afu0.0,0:CMD: Update tag=0x09 resp=0x00
afu0.0,0:CMD: New tag=0x16 code=0x0a00
afu0.0:CMD: Response tag=0x09
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x0a
afu0.0,0:CMD: New tag=0x17 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x0a
afu0.0:CMD: Buffer Write request tag=0x0a
afu0.0,0:CMD: Update tag=0x0a resp=0x00
afu0.0,0:CMD: New tag=0x18 code=0x0a00
afu0.0:CMD: Response tag=0x0a
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x0b
afu0.0,0:CMD: New tag=0x19 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x0b
afu0.0:CMD: Buffer Write request tag=0x0b
afu0.0,0:CMD: Update tag=0x0b resp=0x00
afu0.0,0:CMD: New tag=0x1a code=0x0a00
afu0.0:CMD: Response tag=0x0b
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x0c
afu0.0,0:CMD: New tag=0x1b code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x0c
afu0.0:CMD: Buffer Write request tag=0x0c
afu0.0,0:CMD: Update tag=0x0c resp=0x00
afu0.0,0:CMD: New tag=0x1c code=0x0a00
afu0.0:CMD: Response tag=0x0c
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x0d
afu0.0,0:CMD: New tag=0x1d code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x0d
afu0.0:CMD: Buffer Write request tag=0x0d
afu0.0,0:CMD: Update tag=0x0d resp=0x00
afu0.0,0:CMD: New tag=0x1e code=0x0a00
afu0.0:CMD: Response tag=0x0d

HANG:
afu0.0,0:CMD: New tag=0x00 code=0x0a00
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x00
afu0.0,0:CMD: New tag=0x01 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x00
afu0.0:MMIO: Ack
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x01
afu0.0,0:CMD: New tag=0x02 code=0x0a00
afu0.0,0:SOCKET OUT: MMIO ACK
afu0.0,0:MMIO: Return
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x01
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x02
afu0.0,0:CMD: New tag=0x03 code=0x0a00
afu0.0,0:CMD: New tag=0x04 code=0x0a00
afu0.0,0:SOCKET IN: READ32
afu0.0,0:MMIO: Added Read32 Address=0x003c00
afu0.0,0:CMD: New tag=0x05 code=0x0a00
afu0.0:MMIO: Sent Read32 Address=0x803c00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x02
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x05
afu0.0,0:CMD: New tag=0x06 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x05
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x06
afu0.0,0:CMD: New tag=0x07 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x06
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x07
afu0.0,0:CMD: New tag=0x08 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x07
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x08
afu0.0,0:CMD: New tag=0x09 code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x08
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x09
afu0.0,0:CMD: New tag=0x0a code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x09
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x0a
afu0.0,0:CMD: New tag=0x0b code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x0a
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x0b
afu0.0,0:CMD: New tag=0x0c code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x0b
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x0c
afu0.0,0:CMD: New tag=0x0d code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x0c
afu0.0,0:SOCKET OUT: READ
afu0.0,0:CMD: Client Request tag=0x0d
afu0.0,0:CMD: New tag=0x0e code=0x0a00
afu0.0,0:SOCKET IN: MEM ACK
afu0.0,0:CMD: Client Return tag=0x0d

I was working on a project that had a poweron reset asserted for the first 120ns of the simulation. This caused the RTL core to ignore the reset command issued by PSLSE and created a deadlock.

I added the following lines in top.v as a workaround:

288: ha_pclock <= 0;
289: // $afu_init;
290: #120ns;
292: psl_bfm_init();

403: always @ (negedge ha_pclock) begin
404: if($time > 120ns)
405: get_simuation_error(simulationError);

Could a simulation delay function be added to PSLSE?

Thanks,

Ken

It appears that we presently only track a single interrupt up through the system, and until that interrupt is handled (via cxl_read_event), that all subsequent interrupts get coalesced to the current one. The actual system only coalesces interrupts of the same source, so multiple interrupts can be available. Need to see if we can improve our modeling of the interrupt flow.

Hey,

It seems you are missed including a file:

libcxl.h:21:22: error: misc/cxl.h: No such file or directory

(More errors follow but presumably they would be fixed by the inclusion of this file.)

Thanks,

Logan

PLI has been swapped out in favour of DPI which is a SystemVerilog construct

hello,

The PSLSE generates the data to read (write buffer) with a valid signal haX_bwvalid and I noticed for each cache line we assume the worst-case scenario which are two valid signals for 128Bytes request. I was wondering shouldn't we make this haX_bwvalid signal be random (1,2,3...etc) this makes the simulation more complete for the PSLSE. as it states in the document these valid signals could be multiple (1,2,3...etc) and driven back to back.

I know CAPI 1.0 might seem outdated but since this is the only version that supports caches, it is important for us to use.

Thanks again!

Current pslse only supports DMA operations with addresses that are 128B aligned. For addresses that are 64B aligned, first completion cycle must not cross 128B boundary. CAPI-SNAP test case to be used for testing: ./run_sim -explore -vv -app ../actions/hdl_example/sw/snap_example -arg "-a2 -S0 -B1? -A64 -t500" where ? can be 2, 3, 4, 5, 6,7. 8 (B= # of 64B blocks of data, -A64 indicates 64B address alignment)

Hi, developers, thank you for your checking. It seems the lock lock is not released correctly when psl_list == NULL and (listen_fd = _start_server()) < 0?

Best,

I am using the latest PSLSE code for P8. I have set "export PSL8=1" and see a lot of this warning!!! WHY CAIA2?
WARNING!! ah_brlat has a value other than what is supported on CAIA2. Current value=0x03

Hello,

I am trying to use this in conjunction with https://github.com/sbates130272/capi-textswap, and when I was trying to configure, I ran into an error. I fixed the error by adding "#include <stdio.h>" to the libcxl.h in pslse/libcxl.

Thanks,

Luis E. P.

Howdy folks,

I have been running with a much older PSLSE for a while (originally pulled the repo down in January), and the flow I used was to dynamically link my userspace app the the libcxl.so library built from within the libcxl/ directory. This shared library seems to have been commented out of the Makefiles back in Septemeber and merged into the master branch shortly after I started working with CAPI. Is there a reason the shared library was removed?

I liked the shared library since I could just point the dynamic linker to the legit libcxl or the pslse version. Is the recommendation now to statically link this library?

Thanks!
Kenneth

In the develop branch please don't call srand.

We call srand in our executables based on a command line argument. This helps us reproduce and fix bugs. If you call srand in libcxl.c then it overrides our srand call and makes it impossible to have consistent runs.

If you must do this, then go back to using SEED in the pslse.parms file and let us disable it with a -1 value or something like that.

During my Jenkins build runs I got:

+ rm -rf pslse
+ git clone https://github.com/ibm-capi/pslse pslse
Cloning into 'pslse'...
+ make -C pslse/libcxl
make: Entering directory `/var/lib/jenkins/workspace/SNAP_IMG_KU3_Sponge/pslse/libcxl'
 [CURL ../common/misc/cxl.h]	
 [CC]	 libcxl.o
In file included from libcxl.c:32:
libcxl.h:21:22: error: misc/cxl.h: No such file or directory
In file included from libcxl.c:32:
libcxl.h:104: error: expected declaration specifiers or '...' before '__u64'
libcxl.h:105: error: expected declaration specifiers or '...' before '__s16'
libcxl.h:106: error: expected declaration specifiers or '...' before '__u64'
libcxl.h:107: error: expected declaration specifiers or '...' before '__u64'
libcxl.h:108: error: expected declaration specifiers or '...' before '__s16'
libcxl.h:109: error: expected declaration specifiers or '...' before '__u64'
libcxl.h:186: warning: 'struct cxl_event' declared inside parameter list
libcxl.h:188: warning: 'struct cxl_event' declared inside parameter list
libcxl.c: In function '_handle_dsi':
libcxl.c:111: error: dereferencing pointer to incomplete type

I tried:

git clone https://github.com/ibm-capi/pslse pslse
(cd pslse/libcxl ;
 sed -e 's|http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/include/uapi/misc/cxl.h|https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/include/uapi/misc/cxl.h|g' < Makefile > Makefile.new
)
# ( cd pslse ; git checkout afudirected )
make -C pslse/libcxl -f Makefile.new

that helped for now, but might not be the final solution ... please have a look.

Regards

Frank

parity bytes appear to be swapped coming out of afu_driver. This causes intermittant parity errors depending on the actual parity data.