marcpage / libernet Goto Github PK

There are several TODO or @todo tags in the source. These should be moved to github issues.

Currently SymmetricEncrypt and Hash use macOS specific API and can use the openssl API and both can be tested on macOS. AsymmetricEncrypt should also have macOS APIs available and should be the default and would allow for code to work without openssl and cross-test between openssl and macOS APIs

• Create a new RSA key-pair, given a number of bits
• Serialize RSA public key
• Deserialize RSA public key
• Serialize RSA private key
• Deserialize RSA private key
• Sign text
• Verify signed text
• Public key encrypt
• Private key decrypt
• Get public key from private key
• Test openssl public serialize -> macOS deserialize
• Test openssl private serialize -> macOS deserialize
• Test macOS public serialize -> openssl deserialize
• Test macOS private serialize -> openssl deserialize
• Test openssl sign -> macOS verify
• Test macOS sign -> openssl verify
• Test openssl encrypt -> macOS decrypt
• Test macOS encrypt -> openssl decrypt

In tests/HTTP_test.cpp add code that exercises a URL query that has a name, but no value:

http://my.url.com/test.html?name
http://my.url.com/test.html?name=
http://my.url.com/test.html?name=value&nombre
http://my.url.com/test.html?name=value&nombre=

Path should have methods to read and change the read/write/execute for user/group/everyone for a file (chmod).

FAILED: Exception: OpenSSL Error (EVP_CipherFinal_ex(context, outBuffer, &bytesWritten)): error:06065064:digital envelope routines:EVP_DecryptFinal_ex:bad decrypt File: ../os/SymmetricEncrypt.h Line: 216

Steps to reproduce

• Docker container using latest alpine linux
• Install the latest packages with apk add: git, g++, clang, make, doxygen, cppcheck, sqlite-dev, openssl-dev, zlib-dev, and compiler-rt-static.
• Create a symlink at /usr/lib/clang/9.0.0/libclang_rt.profile-x86_64.a to /usr/lib/clang/9.0.0/lib/linux/libclang_rt.profile-x86_64.a
• git clone https://github.com/marcpage/os.git
• make TEST=SymmetricEncrypt test

Versions

• alpine 3.11.6
• git (2.24.3-r0)
• g++ (9.2.0-r4)
• binutils (2.33.1-r0)
• clang (9.0.0-r1)
• make (4.2.1-r2)
• doxygen (1.8.16-r1)
• cppcheck (1.89-r1)
• sqlite-dev (3.30.1-r2)
• openssl-dev (1.1.1g-r0)
• zlib-dev (1.2.11-r3)
• compiler-rt-static (9.0.0-r0)
• ca-certificates (20191127-r1)
• expat (2.2.9-r1)
• gcc (9.2.0-r4)
• gmp (6.1.2-r1)
• isl (0.18-r0)
• libatomic (9.2.0-r4)
• libc-dev (0.7.2-r0)
• libcurl (7.67.0-r0)
• libffi (3.2.1-r6)
• libgcc (9.2.0-r4)
• libgomp (9.2.0-r4)
• libstdc++ (9.2.0-r4)
• libxml2 (2.9.10-r3)
• llvm9-libs (9.0.0-r1)
• mpc1 (1.1.0-r1)
• mpfr4 (4.0.2-r1)
• musl-dev (1.1.24-r2)
• nghttp2-libs (1.40.0-r0)
• pcre (8.43-r0)
• pcre2 (10.34-r1)
• pkgconf (1.6.3-r0)
• sqlite-libs (3.30.1-r2)
• xz-libs (5.2.4-r0)

Get a stack backtrace apparently doesn't work in the linux environment. We need to figure out how to do this for linux.

Steps to reproduce

In BackTrace.h, remove the APPLE conditional around the following code:

#if defined(__APPLE__)
#include <execinfo.h> // not available on Linux?
#define BACKTRACE_AVAILABLE 1
#endif

• Docker container using latest alpine linux
• Install the latest packages with apk add: git, g++, clang, make, doxygen, cppcheck, sqlite-dev, openssl-dev, zlib-dev, and compiler-rt-static.
• Create a symlink at /usr/lib/clang/9.0.0/libclang_rt.profile-x86_64.a to /usr/lib/clang/9.0.0/lib/linux/libclang_rt.profile-x86_64.a
• git clone https://github.com/marcpage/os.git
• make TEST=Backtrace test

Versions

• alpine 3.11.6
• git (2.24.3-r0)
• g++ (9.2.0-r4)
• binutils (2.33.1-r0)
• clang (9.0.0-r1)
• make (4.2.1-r2)
• doxygen (1.8.16-r1)
• cppcheck (1.89-r1)
• sqlite-dev (3.30.1-r2)
• openssl-dev (1.1.1g-r0)
• zlib-dev (1.2.11-r3)
• compiler-rt-static (9.0.0-r0)
• ca-certificates (20191127-r1)
• expat (2.2.9-r1)
• gcc (9.2.0-r4)
• gmp (6.1.2-r1)
• isl (0.18-r0)
• libatomic (9.2.0-r4)
• libc-dev (0.7.2-r0)
• libcurl (7.67.0-r0)
• libffi (3.2.1-r6)
• libgcc (9.2.0-r4)
• libgomp (9.2.0-r4)
• libstdc++ (9.2.0-r4)
• libxml2 (2.9.10-r3)
• llvm9-libs (9.0.0-r1)
• mpc1 (1.1.0-r1)
• mpfr4 (4.0.2-r1)
• musl-dev (1.1.24-r2)
• nghttp2-libs (1.40.0-r0)
• pcre (8.43-r0)
• pcre2 (10.34-r1)
• pkgconf (1.6.3-r0)
• sqlite-libs (3.30.1-r2)
• xz-libs (5.2.4-r0)

We can parse the json format with much more certainty instead of interpreting the text output.

https://gcc.gnu.org/onlinedocs/gcc/Invoking-Gcov.html#Invoking-Gcov

In tests/HTTP_test.cpp add code that tests ResponseLine parsing of:

FTP 200 OK
FTP/ 200 OK
FTP/alpha 200 OK

Currently Socket is pretty well tested via tests/SocketServer_test.cpp. However it does not test:

size_t Socket::sendTo(const Address &,const std::string &,size_t, size_t, RoutingOptions,OutOfBandDataOptions)

I believe this method is only used in UDP transactions. So UDP tests would be the contents of this test file.

Implement the following method:

void copyAttributesTo(const Path&) const;

This will copy permissions, xattr, ACL, etc. whatever is appropriate for the platform.

This class in Address.h is not documented

The DateTime object does not handle mach_absolute_time/mach_timebase_info, add APPLE specific code that handles these types.

Need to add code for the case use_bundles, to test:

Library::Library(const char*)

and pass a bundle basename (ie CoreFoundation.framework, but pass CoreFoundation). Call a function in the framework or bundle.

Also test loading a name that doesn't have a framework or bundle.

Many of these methods will probably be tested with any UDP tests.

int SocketGeneric::descriptor();
void SocketGeneric::setOption(int,bool,int); // UDP will probably test
void SocketGeneric::broadcast(bool); // UDP will probably test
void SocketGeneric::bind(Address&); // UDP will probably test

Port makefile, tests/test.cpp, and code changes to get this up and building and running on Windows.

Create enums to handle oflags and mode or some other mechanism to abstract the system specific mode and oflags.

This method is not tested.

These methods are not tested in the test code

Add code to tests/Library_test.cpp to test the template function name lookup where the name does not exist in the library.

The os module issues that affect us are:

• locale issue when trying to standardize URL for the AddressHistory test
• aes256 encryption issue in Bundle, Data, and OwnerIdentity tests

_empty.wait_for can return a value other than timeout. If the while loop iterates again, the timeout should be less. We should measure the time wait_for executes and decrement timeoutInSeconds by that amount. If timeoutInSeconds is less than or equal to zero, then it is a timeout.

Currently we handle SO_BROADCAST, but we should handle more.

There is very little documentation in this file. Document every class, method, and member.

There is very little documentation in this file. Document every class, method, and member.

Current test coverage us through SocketServer_test.cpp. Write a file to specifically test addresses.

Add code to tests/JSON_test.cpp that will insert a value into a json ArrayType.

json::Value myArray(json::ArrayType);

myArray.insert(json::Value() = "2", 0); // 2
myArray.insert(json::Value() = "0", 0); // 0,2
myArray.insert(json::Value() = "1", 1); // 0,1,2
myArray.insert(json::Value() = "3", 3); // 0,1,2,3

dotest(myArray[0] == (json::Value() = "0"));
dotest(myArray[1] == (json::Value() = "1"));
dotest(myArray[2] == (json::Value() = "2"));
dotest(myArray[3] == (json::Value() = "3"));

Add a test to tests/HTTP_test.cpp that tests the const method that returns the info.

const Headers &Message<HeaderLine>::fields() const

Have a plug-in model to allow openssl and platform-specific implementation to coexist and be tested in parallel, like we did with Hash

There are very few doxygen comments. Document every class method and member.

msg::Exception now supports reporting the function. This should be supported by subclasses.

Once there is a test that tests UDP, these methods (bind, listen, accept) todos can be removed.

• Create src/os and move all headers from os into this directory.
• Create src/protocol and move all headers from protocol into this directory.
• Move headers to src/libernet
• Move os/tests/*.cpp to tests.
• Move protocol/tests/*.cpp to tests.
• Modify tests/test.cpp to work with this new directory layout.
• Move issues from os to libernet
• Move issues from protocol to libernet
• Update Makefile to build test.cpp from tests instead of ../os/tests
• Update Makefile to search in src//.h for todo items

SocketGeneric would benefit from turning socket signals into exceptions so we can handle error situations.

There is very little documentation in this file. Document every class, method, and member.

The method ignores the outBufferSize parameter and has a cryptic @todo that says "and probably fix". We should make sure we don't overwrite the bounds of the out buffer.

size_t _crypt(int,const void*,const void*,size_t,void *,size_t,const void*)

Similar to io::File read/write integer, using various endians.

template <class Int>
Int MemoryMappedFile::read(off_t &offset, Endian endian) const;
template <class Int>
void write(Int number, off_t &offset, Endian endian);

writes or returns Int of given size at that offset in the file and advances offset to just past the int.

FAIL: Exception not caught: locale::facet::_S_create_c_locale name not valid

I'm hoping this is just a container configuration issue.
I've looked into building and installing musl-locales, but didn't seem to work.

Steps to reproduce

• Docker container using latest alpine linux
• Install the latest packages with apk add: git, g++, clang, make, doxygen, cppcheck, sqlite-dev, openssl-dev, zlib-dev, and compiler-rt-static.
• Create a symlink at /usr/lib/clang/9.0.0/libclang_rt.profile-x86_64.a to /usr/lib/clang/9.0.0/lib/linux/libclang_rt.profile-x86_64.a
• git clone https://github.com/marcpage/os.git
• make TEST=Text test

Versions

• alpine 3.11.6
• git (2.24.3-r0)
• g++ (9.2.0-r4)
• binutils (2.33.1-r0)
• clang (9.0.0-r1)
• make (4.2.1-r2)
• doxygen (1.8.16-r1)
• cppcheck (1.89-r1)
• sqlite-dev (3.30.1-r2)
• openssl-dev (1.1.1g-r0)
• zlib-dev (1.2.11-r3)
• compiler-rt-static (9.0.0-r0)
• ca-certificates (20191127-r1)
• expat (2.2.9-r1)
• gcc (9.2.0-r4)
• gmp (6.1.2-r1)
• isl (0.18-r0)
• libatomic (9.2.0-r4)
• libc-dev (0.7.2-r0)
• libcurl (7.67.0-r0)
• libffi (3.2.1-r6)
• libgcc (9.2.0-r4)
• libgomp (9.2.0-r4)
• libstdc++ (9.2.0-r4)
• libxml2 (2.9.10-r3)
• llvm9-libs (9.0.0-r1)
• mpc1 (1.1.0-r1)
• mpfr4 (4.0.2-r1)
• musl-dev (1.1.24-r2)
• nghttp2-libs (1.40.0-r0)
• pcre (8.43-r0)
• pcre2 (10.34-r1)
• pkgconf (1.6.3-r0)
• sqlite-libs (3.30.1-r2)
• xz-libs (5.2.4-r0)

Code needs to be added to tests/DateTime_test.cpp that tests the following constructor:

DateTime::DateTime(int year, Month month, int day, int hour,CivilianHour ampm, int minutes, double secs,Location location)

When hour == 12 and ampm == AM.

When we are calculating the padding for AddressHistory so that it matches the hash of the address, we are unnecessarily converting from JSON to string every time. We could set the padding to a unique value that is not found elsewhere in the formatted string. We can then just search and replace the unique number with random numbers so we don't have to execute all that code to convert json to string for each iteration.

Maybe remove the typecast? Need to check both macOS and Linux and try different combinations to get rid of the warning.

../libernet/Bundle.h:262:65: warning: type qualifiers ignored on cast result type [-Wignored-qualifiers]

The following constructor currently is not tested in tests/HTTP_test.cpp:

Message<HeaderLine>::Message(const String &headers, String::size_type &after)

Add a test to tests/HTTP_test.cpp that tests the const method that returns the info.

const HeaderLine &Message<HeaderLine>::info() const

Add test code to tests/HTTP_test.cpp that tests the ReponseLine class parsing when there is no end of line character in the string

This may be a docker container configuration issue, or the path may be incorrect. The test may need to change to point to the correct install location for libz.

FAILED: EXCEPTION(libz.dylib:zlibVersion): (found) failed:Unable to open library: /usr/lib/libz.so File: ../os/Library.h Line: 227

Steps to reproduce

• Docker container using latest alpine linux
• Install the latest packages with apk add: git, g++, clang, make, doxygen, cppcheck, sqlite-dev, openssl-dev, zlib-dev, and compiler-rt-static.
• Create a symlink at /usr/lib/clang/9.0.0/libclang_rt.profile-x86_64.a to /usr/lib/clang/9.0.0/lib/linux/libclang_rt.profile-x86_64.a
• git clone https://github.com/marcpage/os.git
• make TEST=Library test

Versions

• alpine 3.11.6
• git (2.24.3-r0)
• g++ (9.2.0-r4)
• binutils (2.33.1-r0)
• clang (9.0.0-r1)
• make (4.2.1-r2)
• doxygen (1.8.16-r1)
• cppcheck (1.89-r1)
• sqlite-dev (3.30.1-r2)
• openssl-dev (1.1.1g-r0)
• zlib-dev (1.2.11-r3)
• compiler-rt-static (9.0.0-r0)
• ca-certificates (20191127-r1)
• expat (2.2.9-r1)
• gcc (9.2.0-r4)
• gmp (6.1.2-r1)
• isl (0.18-r0)
• libatomic (9.2.0-r4)
• libc-dev (0.7.2-r0)
• libcurl (7.67.0-r0)
• libffi (3.2.1-r6)
• libgcc (9.2.0-r4)
• libgomp (9.2.0-r4)
• libstdc++ (9.2.0-r4)
• libxml2 (2.9.10-r3)
• llvm9-libs (9.0.0-r1)
• mpc1 (1.1.0-r1)
• mpfr4 (4.0.2-r1)
• musl-dev (1.1.24-r2)
• nghttp2-libs (1.40.0-r0)
• pcre (8.43-r0)
• pcre2 (10.34-r1)
• pkgconf (1.6.3-r0)
• sqlite-libs (3.30.1-r2)
• xz-libs (5.2.4-r0)

This function is not covered in the test. Verify it works correctly.

The following cast operators are not tested. Add tests the verify they work correctly.

DateTime::operator time_t() const
DateTime::operator timeval() const
DateTime::operator const timespec() const
DateTime::operator double() const

Also add test code for:
timespec &DateTime::value(timespec &ts) const

Call and verify that copyContentsTo works correctly.

const Path &Path::copyContentsTo(const Path&) const

FAILED: Exception: [EINVAL (22): Invalid argument]: ep = ::readdir(dp) File: ../os/Path.h Line: 553

Steps to reproduce

• Docker container using latest alpine linux
• Install the latest packages with apk add: git, g++, clang, make, doxygen, cppcheck, sqlite-dev, openssl-dev, zlib-dev, and compiler-rt-static.
• Create a symlink at /usr/lib/clang/9.0.0/libclang_rt.profile-x86_64.a to /usr/lib/clang/9.0.0/lib/linux/libclang_rt.profile-x86_64.a
• git clone https://github.com/marcpage/os.git
• make TEST=Path test

Versions

• alpine 3.11.6
• git (2.24.3-r0)
• g++ (9.2.0-r4)
• binutils (2.33.1-r0)
• clang (9.0.0-r1)
• make (4.2.1-r2)
• doxygen (1.8.16-r1)
• cppcheck (1.89-r1)
• sqlite-dev (3.30.1-r2)
• openssl-dev (1.1.1g-r0)
• zlib-dev (1.2.11-r3)
• compiler-rt-static (9.0.0-r0)
• ca-certificates (20191127-r1)
• expat (2.2.9-r1)
• gcc (9.2.0-r4)
• gmp (6.1.2-r1)
• isl (0.18-r0)
• libatomic (9.2.0-r4)
• libc-dev (0.7.2-r0)
• libcurl (7.67.0-r0)
• libffi (3.2.1-r6)
• libgcc (9.2.0-r4)
• libgomp (9.2.0-r4)
• libstdc++ (9.2.0-r4)
• libxml2 (2.9.10-r3)
• llvm9-libs (9.0.0-r1)
• mpc1 (1.1.0-r1)
• mpfr4 (4.0.2-r1)
• musl-dev (1.1.24-r2)
• nghttp2-libs (1.40.0-r0)
• pcre (8.43-r0)
• pcre2 (10.34-r1)
• pkgconf (1.6.3-r0)
• sqlite-libs (3.30.1-r2)
• xz-libs (5.2.4-r0)

These two functions are not called in the test code. Make sure they work as expected

There is very little documentation in this file. Document every class, method, and member.