vberlier / narwhal Goto Github PK

View Code? Open in Web Editor NEW

115.0 8.0 6.0 633 KB

A progressive test framework for C.

License: MIT License

Makefile 3.03% C 96.97%

c test-framework tests fixtures parametrized-tests mocking

narwhal's People

Contributors

Stargazers

Watchers

Forkers

na1pir leehaipeng srabon444 thelightway zeta1999

narwhal's Issues

String mismatch marker

Hello!

Adding a marker at the first character mismatch when comparing string would make it easier to identify where the string differs.

Current error:

Location: main.c:5
Time:     0.07ms
Error:    Assertion "1234567890" == "1234567790" failed.
          First argument "1234567890" is not equal to "1234567790".

Requested error:

Location: main.c:5
Time:     0.07ms
Error:    Assertion "1234567890" == "1234567790" failed.
          First argument "1234567890" is not equal to "1234567790".
                                 ^                            ^

Sample program:

#include <unicorn/unicorn.h>

TEST(test_1)
{
    ASSERT_EQ("1234567890", "1234567790");
}

int main()
{
    return RUN_TESTS(
        test_1
    );
}

Output captured data on failure

Hello!

The program at the end of the issue produces the output "Current output" with the latest unicorn implementation. It would be helpful to output the captured data as well, as seen in "Requested output".

Current output:

    test_1 failed:
...
        Output:
         1 |  Before capture!
         2 |  After capture!
         3 |  

    test_2 failed:
...
        Output:
         1 |  Before capture!
         2 |

Requested output:

    test_1 failed:
...
        Output:
         1 |  Before capture!
         2 |  In capture!
         3 |  After capture!
         4 |  

    test_2 failed:
...
        Output:
         1 |  Before capture!
         2 |  In capture!
         3 |

Sample program:

#include <unicorn/unicorn.h>

TEST(test_1)
{
    printf("Before capture!\n");
    
    CAPTURE_OUTPUT(output) {
        printf("In capture!\n");
    }

    printf("After capture!\n");

    ASSERT(0);
}

TEST(test_2)
{
    printf("Before capture!\n");
    
    CAPTURE_OUTPUT(output) {
        printf("In capture!\n");
        ASSERT(0);
    }

    printf("After capture!\n");
}

int main()
{
    return RUN_TESTS(
        test_1,
        test_2
    );
}

How to use a GDB with Narwhal

Hello!

I can't figure out how to use GDB to debug a test case that crashes. I can set a breakpoint at _narwhal_test_function_<mytestcase>, but GDB never halts there. I've tried set follow-fork-mode child and other alternatives I found searching the web, but nothings seems to work.

This is probably not a problem with Narwhal itself, but my poor GDB knowledge. Anyhow, it would probably help others as well if there is a short guide on how to use GDB with Narwhal.

Crash with char * as NULL

Hello!

Calling ASSERT_EQ(char *, char *) with one of both parameters as NULL fails as below when the address sanitizer enabled. I suggest just comparing the pointers if one or both are NULL.

Test program:

#include "narwhal.h"

TEST(test)
{
    ASSERT_EQ(NULL, NULL);
    printf("void OK\n");

    ASSERT_EQ((char *)NULL, (char *)NULL);
    printf("char OK\n");
}

Build and run:

$ cc -fsanitize=address main.c narwhal.c && ./a.out

Running tests...

     1 - 1 |  F

Test results:

    FAIL test (16.81ms)

Failing tests:

    test failed:

        Location: main.c:3
        Time:     16.81ms
        Error:    Test process exited unexpectedly.

        Output:

         1 |  void OK
         2 |  AddressSanitizer:DEADLYSIGNAL
         3 |  =================================================================
         4 |  ==12194==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x7f872d3772cc bp 0x7ffd9ac4f790 sp 0x7ffd9ac4eef0 T0)
         5 |  ==12194==The signal is caused by a READ memory access.
         6 |  ==12194==Hint: address points to the zero page.
         7 |      #0 0x7f872d3772cb  (/lib/x86_64-linux-gnu/libasan.so.5+0xd52cb)
         8 |      #1 0x56064af942f3 in narwhal_check_string_equal (/home/erik/workspace/ntest/a.out+0x122f3)
         9 |      #2 0x56064af86769 in _narwhal_test_function_test (/home/erik/workspace/ntest/a.out+0x4769)
        10 |      #3 0x56064af8990a in execute_test_function (/home/erik/workspace/ntest/a.out+0x790a)
        11 |      #4 0x56064af8a438 in narwhal_run_test (/home/erik/workspace/ntest/a.out+0x8438)
        12 |      #5 0x56064af92c97 in narwhal_test_session_run_test (/home/erik/workspace/ntest/a.out+0x10c97)
        13 |      #6 0x56064af92d10 in narwhal_test_session_run_parameterized_test (/home/erik/workspace/ntest/a.out+0x10d10)
        14 |      #7 0x56064af93061 in narwhal_test_session_run_test_group (/home/erik/workspace/ntest/a.out+0x11061)
        15 |      #8 0x56064af937cf in narwhal_run_root_group (/home/erik/workspace/ntest/a.out+0x117cf)
        16 |      #9 0x56064af8e8e7 in main (/home/erik/workspace/ntest/a.out+0xc8e7)
        17 |      #10 0x7f872d0ddb6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)
        18 |      #11 0x56064af86489 in _start (/home/erik/workspace/ntest/a.out+0x4489)
        19 |  
        20 |  AddressSanitizer can not provide additional info.
        21 |  SUMMARY: AddressSanitizer: SEGV (/lib/x86_64-linux-gnu/libasan.so.5+0xd52cb) 
        22 |  ==12194==ABORTING
        23 |  

Tests: 1 failed, 0 passed, 1 total
Time:  17.37ms

Assert memory

Hello!

Sometimes I want to assert that two buffers contains the same data. I could not find any such assert-macro in narwhal, would it be possible to add one?

TEST(foo)
{
    uint8_t buf1[3];
    uint8_t buf2[10];

    buf1[0]= 's';
    buf1[1]= 's';
    buf1[2]= 's';
    buf2[0]= 's';
    buf2[1]= 't';
    buf2[2]= 's';

    ASSERT_MEMORY(buf1, buf2, sizeof(buf1));
}

The output could look like this:

foo failed:

    Location: main.c:13
    Time:     0.12ms
    Error:    Assertion memcmp(buf1, buf2, sizeof(buf1)) == 0 failed.
              See diff for details.

    11 |      buf2[2] = 's'.
    12 |
  > 13 |      ASSERT_MEMORY(buf1, buf2, sizeof(buf1));
    14 |  }
    15 |

    Diff:

   - 1 | 00000000  73 73 73                                          |sss|
   + 1 | 00000000  73 74 73                                          |sts|

Capture output until

I want to test a shell that runs in a separate thread. Today I do it like this:

CAPTURE_OUTPUT(output) {
  // Start the shell.
  // Write to shell stdin.
  usleep(50000);
}

ASSERT_SUBSTRING(output, "OK!\n");

The sleep is needed to give the shell thread some time to execute before the output is checked. It's not very pretty as the test may fail if the CPU is busy with other tasks.

I was thinking about something like this instead:

CAPTURE_OUTPUT_UNTIL("OK\n") {
  // Start the shell.
  // Write to shell stdin.
}

Would it be possible to add something similar to this?

Show stdout and stderr output regardless of test result

Hello,

I often want to see all stdout and stderr output regardless of the test result. There are two reasons; I like printf-debugging and I want to see the output from the project's logging module. Would it be possible to add support for it somehow? Maybe write the output to a file, maybe to the terminal, maybe something else. Maybe a command line argument to enable it?

<time.h> not included in amalgamated headers/source

Attempting to compile my tests and running into this error...

I can fix within my system including <time.h> in the header and source within /dist but thats just a workaround and not an actual solution.

Question: Mocking

Hello!

Are there any plans on adding support for mocking?

Thanks!

vberlier / narwhal Goto Github PK

narwhal's People

Contributors

Stargazers

Watchers

Forkers

narwhal's Issues

String mismatch marker

Output captured data on failure

How to use a GDB with Narwhal

Crash with char * as NULL

Assert memory

Capture output until

Show stdout and stderr output regardless of test result

<time.h> not included in amalgamated headers/source

Question: Mocking

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

Jobs