I porting coremark to AndroidPlatform.
The CPU has 8 core
The result is that:
gts6lwifi:/data/local/tmp $ ./libcoremark
2K performance run parameters for coremark.
CoreMark Size : 666
Total ticks : 16352
Total time (secs): 16.352000
Iterations/Sec : 18346.379648
Iterations : 300000
Compiler version : Android (7211189, based on r416183) Clang 12.0.4 (https://android.googlesource.com/toolchain/llvm-project c935d99d7cf2016289302412d708641d52d2f7ee)
Compiler flags : -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -march=armv7-a -mthumb -Wformat -Werror=format-security -fexceptions -O2
Memory location : Please put data memory location here
(e.g. code in flash, data on heap etc)
seedcrc : 0xe9f5
[0]crclist : 0xe714
[0]crcmatrix : 0x1fd7
[0]crcstate : 0x8e3a
[0]crcfinal : 0xcc42
Correct operation validated. See README.md for run and reporting rules.
CoreMark 1.0 : 18346.379648 / Android (7211189, based on r416183) Clang 12.0.4 (https://android.googlesource.com/toolchain/llvm-project c935d99d7cf2016289302412d708641d52d2f7ee) -DANDROID -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -D_FORTIFY_SOURCE=2 -march=armv7-a -mthumb -Wformat -Werror=format-security -fexceptions -O2 / Heap

I want to know this result is a single core score or muti core score ?

Hello:

What is the point of -DMULTITHREAD? I've tried running the benchmark on my system with this flag set to 1,2,4,16 and getting the same results.
However, the more complex underlined question is how CoreMarks takes into account multiple cores? There is nothing about it in the README or the EEMBC whitepaper.

Thank you.

When building and running coremark.exe with gcc on redhat 8.2, sometimes I will get the error of "ERROR! Must execute for at least 10 secs for a valid result!" since the total run time is less than 10 secs.
What does this message mean? Why does the test need to run for at least 10 secs? Could you please help to explain, thanks a lot!

make command

make CC="gcc" PORT_CFLAGS="-O0 -g"

the output of binary

$ ./coremark.exe  0x0 0x0 0x66 0 7 1 2000
2K performance run parameters for coremark.
CoreMark Size    : 666
Total ticks      : 14733
Total time (secs): 14.733000
Iterations/Sec   : 4072.490328
Iterations       : 60000
Compiler version : GCC10.2.0
Compiler flags   : -O0 -g    -lrt
Memory location  : Please put data memory location here
                        (e.g. code in flash, data on heap etc)
seedcrc          : 0xe9f5
[0]crclist       : 0xe714
[0]crcmatrix     : 0x1fd7
[0]crcstate      : 0x8e3a
[0]crcfinal      : 0xbd59
Correct operation validated. See readme.txt for run and reporting rules.
CoreMark 1.0 : 4072.490328 / GCC10.2.0 -O0 -g    -lrt / Heap

$ ./coremark.exe  0x3415 0x3415 0x66 0 7 1 2000
2K validation run parameters for coremark.
CoreMark Size    : 666
Total ticks      : 9786
Total time (secs): 9.786000
Iterations/Sec   : 4087.471899
ERROR! Must execute for at least 10 secs for a valid result!
Iterations       : 40000
Compiler version : GCC10.2.0
Compiler flags   : -O0 -g    -lrt
Memory location  : Please put data memory location here
                        (e.g. code in flash, data on heap etc)
seedcrc          : 0x18f2
[0]crclist       : 0xe3c1
[0]crcmatrix     : 0x0747
[0]crcstate      : 0x8d84
[0]crcfinal      : 0xbe81
Errors detected

We can easily demonstrate this bug on linux64. Modify linux64/core_portme.mak to add ITERATIONS=123456:

diff --git a/linux64/core_portme.mak b/linux64/core_portme.mak
index 5cfabee..4c7f3f5 100755
--- a/linux64/core_portme.mak
+++ b/linux64/core_portme.mak
@@ -138,3 +138,4 @@ MKDIR = mkdir -p
 # FLAG: PERL
 # Define perl executable to calculate the geomean if running separate.
 PERL=/usr/bin/perl
+ITERATIONS=123456

Then make to compile and run. The log files show that (on my system) it ran 400000 iterations, not 123456 as requested.

The problem is the dollar parenthesis around ITERATIONS in the main Makefile, line 50. After applying the patch from Pull Request #12 , the testcase works as expected, running 123456 iterations.

In core_list_init, we use

list_head *core_list_init(ee_u32 blksize, list_head *memblock, ee_s16 seed) {
	/* calculated pointers for the list */
	ee_u32 per_item=16+sizeof(struct list_data_s);
	ee_u32 size=(blksize/per_item)-2; /* to accomodate systems with 64b pointers, and make sure same code is executed, set max list elements */
	list_head *memblock_end=memblock+size;
	list_data *datablock=(list_data *)(memblock_end);
	list_data *datablock_end=datablock+size;

IIUC, per_item is calculated as 16B for list_head + sizeof struct list_data_s. size = capacity - 2 to provide space for head and tail; However, the value of size being used to compute memblock_end means we can only store capacity-2 elements even including head and tail; This also means the last 2 loop iterations of the code below would silently discard the data since we would be past memblock_end when inserting

	/* create a fake items for the list head and tail */
	list->next=NULL;
	list->info=datablock;
	list->info->idx=0x0000;
	list->info->data16=(ee_s16)0x8080;
	memblock++;
	datablock++;
	info.idx=0x7fff;
	info.data16=(ee_s16)0xffff;
	core_list_insert_new(list,&info,&memblock,&datablock,memblock_end,datablock_end);
	
	/* then insert size items */
	for (i=0; i<size; i++) {
		ee_u16 datpat=((ee_u16)(seed^i) & 0xf);
		ee_u16 dat=(datpat<<3) | (i&0x7); /* alternate between algorithms */
		info.data16=(dat<<8) | dat;		/* fill the data with actual data and upper bits with rebuild value */
		core_list_insert_new(list,&info,&memblock,&datablock,memblock_end,datablock_end);
	}

This isn't a functional bug since core_list_insert_new is robust to the two drops and we don't use the pointer returned by that function in list_init.

I don't understand if this is actually intentional. However, I thought I should bring it up just in case my understanding was incorrect.

When attempting to run all-threads on a Ryzen 7 3700 system:

user@box:~/coremark$ make XCFLAGS="-DMULTITHREAD=16 -DUSE_PTHREAD"
[...]
/usr/bin/ld: /tmp/cc1D8hXe.o: in function `core_start_parallel':
core_portme.c:(.text+0x199): undefined reference to `pthread_create'
/usr/bin/ld: /tmp/cc1D8hXe.o: in function `core_stop_parallel':
core_portme.c:(.text+0x1d0): undefined reference to `pthread_join'
collect2: error: ld returned 1 exit status

How to calculate the coremark score from the logs which is generated as a result of coremark run. Can you emphasize more on that because I am able to run the coremark on the embedded device succesfully but not able to infer the score like which variables I need to consider to calculate the coremark/MHz score.

For E.g. what would the coremark score for the following output and how do we calculate the coremark/MHz value for reporting and inference purposes?
CoreMark 1.0 : 6508.490622 / GCC3.4.4 -O2 / Heap

In the source code, like core_matrix.c, the index variable of a loop always uses a 32bits variable(ee_u32), but I think it should use the standard size_t, because it can reduce displacement on 64 bits machine which doesn't supply 32 bits registers:

void
matrix_mul_vect(ee_u32 N, MATRES *C, MATDAT *A, MATDAT *B)
{
    ee_u32 i, j;
    for (i = 0; i < N; i++)
    {
        C[i] = 0;
        for (j = 0; j < N; j++)
        {
            C[i] += (MATRES)A[i * N + j] * (MATRES)B[j];
        }
    }
}

I run the command make and this is the error:
Makefile:44: *** missing separator. Stop.
Line 44 : $(error PLEASE define PORT_DIR! (e.g. make PORT_DIR=simple))

Hello, I noticed that there is no type definition provided for ee_f16 inside the project (although it is used to define type MATDAT ). core_portme.h, which contains definitions for most of the ee_~ types doesn't contain definition for ee_f16. Are benchmark users expected to provide their own implementation and definition for ee_f16 to run the benchmark with ee_f16 ? Thank you 🤓

I.e. in

void matrix_add_const(ee_u32 N, MATDAT *A, MATDAT val) {
	ee_u32 i,j;
	for (i=0; i<N; i++) {
		for (j=0; j<N; j++) {
			A[i*N+j] += val;
		}
	}
}

since unsigned types are allowed to wrap/overflow this code is not the equivalent of some abstract A[i][j] when pointers are 64-bit (where is no guarantee that i*N + j doesn't wrap). Solution: either use signed type or size_t.

Note, that it becomes a correctness issue, not a performance one, if N is allowed to be > 2^16.

Unlike C++, this function isn't recognised as a valid function prototype - it may take 0 or n arguments. To avoid warnings, this line should be replaced with

ee_u8 check_data_types(void);

In core_main.c, the results of time_in_secs is printed using the %d format specifier, which handles arguments of type int.
However type secs_ret is type ee_u32, which is not int-sized on 16-bit platforms (for which I'm working on an LLVM port :) ).
I got lucky in that my platform is little endian, so the low order 16-bits that will be read by %d are the ones that I want, and there aren't any arguments after the problematic %d's.

Three ways off the top of my head to fix this:

1. Cast to int before the format specifier.
2. Use the C99 format specifier macros for uint32_t.
3. Create an EEMBC-specific version of the above macro and add it to the port headers.

coremark.h

/* ultithread specific */

assume this is intended to mean multithread specific

If thread create fails, it fails silently:

Error code should be checked.

make -j8 all
/bin/sh: 1: [[: not found
/bin/sh: 1: [[: not found
Makefile:45: *** PLEASE define PORT_DIR! (e.g. make PORT_DIR=simple). Stop.

i am using linux 32 bit and in make file its defined as PORT_DIR=Linux but still i am unable to buil in eclipse neon. please help

Hi

I am trying to run coremark benchmark on rasberry pi board , I ported the coremark to rasberrypi But when i run the code I am getting "segmentation fault" error . Board details are mentioned below

execution steps:

cd coremark
make linux
./coremark.exe

Board Details:
Linux pi64 4.19.66-v8-fc5826fb999e-p4-bis+ #2 SMP PREEMPT Fri Aug 16 13:58:31 GMT 2019 aarch64 GNU/Linux

Error :
"Segmentation fault"

I wanted to know what could be the reason .
If anyone know how to resolve the issue ,help me to resolve .

Regards
Megha

Building with -Wall and -Wextra creates the following warnings:

cc -O2 -Imacos -Iposix -I. -DFLAGS_STR=\""-O2   "\" -DITERATIONS=0 -Wall -Wextra  core_list_join.c core_main.c core_matrix.c core_state.c core_util.c posix/core_portme.c -o ./coremark.exe 
posix/core_portme.c:209:38: warning: unused parameter 'argc' [-Wunused-parameter]
portable_init(core_portable *p, int *argc, char *argv[])
                                     ^
posix/core_portme.c:209:50: warning: unused parameter 'argv' [-Wunused-parameter]
portable_init(core_portable *p, int *argc, char *argv[])
                                                 ^
lib/coremark/core_list_join.c: In function 'core_bench_list':
lib/coremark/core_list_join.c:445:60: warning: 'info.data16' may be used uninitialized in this function [-Wmaybe-uninitialized]
  445 |         while (list && ((list->info->data16 & 0xff) != info->data16))
      |                                                        ~~~~^~~~~~~~
lib/coremark/core_list_join.c:167:16: note: 'info.data16' was declared here
  167 |     list_data  info;
      |                ^~~~
4 warnings generated.

Tested with: riscv64-unknown-elf-gcc (GCC) 11.1.0

Hey Guys:
Thanks for this helpful benchmark! It's way more easy to read and port then Drystone.
Anywhere, when I tried to port to our bare mental RISC-V core, I found following issues may need to fixed:

size_t is missing

I believed the 'size_t' is defined in <stddef.h> which is Linux only? for bare mental cores, we need to add a definition for that. My work around is adding following line to barebones/core_portme.h

#define size_t long

CLOCKS_PER_SEC is missing

Also need to add the definition to barebones/core_portme.h

PORT_OBJS is missing

We have defined PORT_SRCS in barebones/core_portme.mak, but did not define PORT_OBJS which is also referenced in top Makefile.

HAS_FLOAT macros check.

In coremark/barebones/core_portme.h we have following configuration switches:

/* Configuration : HAS_FLOAT 
	Define to 1 if the platform supports floating point.
*/
#ifndef HAS_FLOAT 
#define HAS_FLOAT 1
#endif

But we only checks if the HAS_FLOAT defined or not, which is not right because the macros is always defined, but with different values.

We may need to check the macro value other than only check the definition in barebones/ee_printf.c:

#if(HAS_FLOAT==1)
...
#endif

After the aforementioned work around, I was able to compile & link the barebone porting with RISC-V toolchains (https://github.com/ShawnLess/coremark).

When i run this command in terminal to run the coremark:
gcc -O2 -o coremark.exe core_list_join.c core_main.c core_matrix.c core_state.c core_util.c simple/core_portme.c -DPERFORMANCE_RUN=1 -DITERATIONS=1000
Please let us know in which directory we have to run this command that you have mention in Readme file.

Reported:

If the seedcrc doesn’t match a known type then total_errors=-1, However if the call to check_data_types()then fails then we could end up with a count of zero and a pass.

	switch (seedcrc) { /* test known output for common seeds */
		case 0x8a02: /* seed1=0, seed2=0, seed3=0x66, size 2000 per algorithm */
			known_id=0;
			ee_printf("6k performance run parameters for coremark.\n");
			break;
		case 0x7b05: /*  seed1=0x3415, seed2=0x3415, seed3=0x66, size 2000 per algorithm */
			known_id=1;
			ee_printf("6k validation run parameters for coremark.\n");
			break;
		case 0x4eaf: /* seed1=0x8, seed2=0x8, seed3=0x8, size 400 per algorithm */
			known_id=2;
			ee_printf("Profile generation run parameters for coremark.\n");
			break;
		case 0xe9f5: /* seed1=0, seed2=0, seed3=0x66, size 666 per algorithm */
			known_id=3;
			ee_printf("2K performance run parameters for coremark.\n");
			break;
		case 0x18f2: /*  seed1=0x3415, seed2=0x3415, seed3=0x66, size 666 per algorithm */
			known_id=4;
			ee_printf("2K validation run parameters for coremark.\n");
			break;
		default:
			total_errors=-1;
			break;
	}
	if (known_id>=0) {
		for (i=0 ; i<default_num_contexts; i++) {
			results[i].err=0;
			if ((results[i].execs & ID_LIST) && 
				(results[i].crclist!=list_known_crc[known_id])) {
				ee_printf("[%u]ERROR! list crc 0x%04x - should be 0x%04x\n",i,results[i].crclist,list_known_crc[known_id]);
				results[i].err++;
			}
			if ((results[i].execs & ID_MATRIX) &&
				(results[i].crcmatrix!=matrix_known_crc[known_id])) {
				ee_printf("[%u]ERROR! matrix crc 0x%04x - should be 0x%04x\n",i,results[i].crcmatrix,matrix_known_crc[known_id]);
				results[i].err++;
			}
			if ((results[i].execs & ID_STATE) &&
				(results[i].crcstate!=state_known_crc[known_id])) {
				ee_printf("[%u]ERROR! state crc 0x%04x - should be 0x%04x\n",i,results[i].crcstate,state_known_crc[known_id]);
				results[i].err++;
			}
			total_errors+=results[i].err;
		}
	}
	total_errors+=check_data_types();

On a Raspberry Pi 4B running 32bit Raspbian log1 contains:

ERROR! Please define ee_ptr_int to a type that holds a pointer!
2K performance run parameters for coremark.
ERROR: ee_ptr_int is not a datatype that holds an int pointer!
ERROR: Please modify the datatypes in core_portme.h!
CoreMark Size    : 666 
Total ticks      : 18173
Total time (secs): 18.173000
Iterations/Sec   : 11005.337589
Iterations       : 200000
Compiler version : GCC8.3.0
Compiler flags   : -O2 -DPERFORMANCE_RUN=1  -lrt
Memory location  : Please put data memory location here
                        (e.g. code in flash, data on heap etc)
seedcrc          : 0xe9f5
[0]crclist       : 0xe714
[0]crcmatrix     : 0x1fd7
[0]crcstate      : 0x8e3a
[0]crcfinal      : 0x4983
Errors detected

The current test driver is based on string (usually stdout and argv) I/O and is not quite convenient to call from more restrictive environments such as iOS. A graphics environment might also want to display the results without too much string manipulation. A more "struct-ured" (pardon my pun) API taking a struct input (multithread, seed) and a struct output (anything printed) might be more adequate and portable. core_results is 90% there already.

Could I port coremark for Aurix Tricore TC3xx? and what I need to do so?

Hey,

I have been running CoreMark with a simple RISC-V 32-bit core single issue in-order pipeline and the results are quite impressive as it shown below. When comparing with other cores it seems that this one outperform most of the others which does not make much sense due its simple microarchitecture when in comparison with designs like Aria/CV32E40P. What are the places to check if the CoreMark port for this architecture is correct or not?

2K performance run parameters for coremark.
CoreMark Size    : 666
Total ticks      : 667898007
Total time (secs): 13
Iterations/Sec   : 230
Iterations       : 3000
Compiler version : riscv-none-embed-gcc (xPack GNU RISC-V Embedded GCC x86_64) 10.2.0
Compiler flags   : -O0 -g -march=rv32i -mabi=ilp32 -Wall -Wno-unused -ffreestanding --specs=nano.specs -DPRINTF_DISABLE_SUPPORT_FLOAT -DPRINTF_DISABLE_SUPPORT_EXPONENTIAL -DPRINTF_DISABLE_SUPPORT_LONG_LONG -DREAL_UART -Wall -Wno-main -DPERFORMANCE_RUN=1  -O0 -g
Memory location  : STACK
seedcrc          : 0xe9f5
[0]crclist       : 0xe714
[0]crcmatrix     : 0x1fd7
[0]crcstate      : 0x8e3a
[0]crcfinal      : 0xcc42
Correct operation validated. See README.md for run and reporting rules.

In the core_portme.c I am using this

CORETIMETYPE cpu_get_cycle(void) {

  union {
    uint64_t uint64;
    uint32_t uint32[sizeof(uint64_t)/sizeof(uint32_t)];
  } cycles;

  register uint32_t tmp1, tmp2, tmp3;
  while(1) {
    tmp1 = read_csr(0xC80);
    tmp2 = read_csr(0xC00);
    tmp3 = read_csr(0xC80);
    if (tmp1 == tmp3) {
      break;
    }
  }

  cycles.uint32[0] = tmp2;
  cycles.uint32[1] = tmp3;

  return cycles.uint64;
}
/* Define : TIMER_RES_DIVIDER
        Divider to trade off timer resolution and total time that can be
   measured.

        Use lower values to increase resolution, but make sure that overflow
   does not occur. If there are issues with the return value overflowing,
   increase this value.
        */
#define CLOCKS_PER_SEC             50000000
#define GETMYTIME(_t)              (*_t = cpu_get_cycle())
#define MYTIMEDIFF(fin, ini)       ((fin) - (ini))
#define TIMER_RES_DIVIDER          1
#define SAMPLE_TIME_IMPLEMENTATION 1
#define EE_TICKS_PER_SEC           (CLOCKS_PER_SEC / TIMER_RES_DIVIDER)

Hello, I noticed that type ee_f32 is defined as double in core_portme.h, but double represents 64bits in both rv32 and rv64 per riscv-spec: https://riscv.org/wp-content/uploads/2015/01/riscv-calling.pdf.
Could you help me understand why the definition is like this? Thank you 🤓

Dear,

I want to test the score using Coremark's test code for an MCU with RAM = 2KB.
How should I deploy it?
How to calculate the coremark score for the test results?
Currently my test results are as follows.

CoreMark Size : 170
Total ticks : 15673
Total time (secs):
Iterations/Sec :
Iterations : 5000
Compiler version : GCC10.3.1
Compiler flags : -Os
Memory location : STACK
seedcrc : 0xd967
[0]crclist : 0x11ee
[0]crcmatrix : 0x66cc
[0]crcstate : 0x5cd9
[0]crcfinal : 0x0a03
Cannot validate operation for these seed values, please compare with results on a known platform.

Thanks for your help in advance.
BR,
Cpantherk

Hi,

I am a newbie, trying to use coremark benchmarks for Raspberry Pi 4. I have never run any benchmarks without GUI interface, so I am having difficulties using the 'run' command. Can you give a small example, on how to use the 'run' command? I have googled it and I did not find any helpful examples. I may be missing something simple, as everyone else seems to be getting it (as there aren't much examples). This is not really an issue, but I could not find, where to ask for help.

Thank You.

I added some print in the "main" function (after the "iterate" function and the total_time calculation, for sure), then I found that the coremark changed.

Is that reasonable ?
I thought it should not impact the coremark since I did not change the iterate process at all.

Thanks.