Compilation on MacOS about pgp-packet-library

[bysnack]

GSL is a dependency of the library and needs to be installed or the include path for it needs to be provided.

A working version of GSL is provided as a submodule, you could specify the include path for it but the easiest and recommended way to use the library is with CMake, #55 introduced a change in the CMake config script which allows you to use the library with a really short CMakeLists.txt script

# CMakeLists.txt
cmake_minimum_required(VERSION 3.13.0)
project(pgp-packet-example
        VERSION 0.1.1
        LANGUAGES CXX)
find_package(pgp-packet-packet REQUIRED)
add_executable(pgp-packet-example example.cpp)
target_link_libraries(pgp-packet-example pgp-packet)

and #56 introduces the examples directory, you could also run

cmake -B build && cmake --build build

Inside of that directory to compile the three examples.

[skaht]

Your point about not including the header path for the local GSL distribution was spot on, a late night oversight on my part. Anyhow, I had some compilation issues after the include paths were straightened out at my end. So I proceeded with a path forward consistent with you recommendations above:

Saw the pgp-packet-library was updated today. Downloaded the latest and recompiled in a MacOS Catalina Environment where the Homebrew package manager was used to load the latest boost, cryptopp, and libsodium packages.

% clang++ --version
Apple clang version 11.0.3 (clang-1103.0.32.62)
Target: x86_64-apple-darwin19.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Details at https://github.com/summitto/pgp-packet-library#building-the-library are sufficient assuming one changes into their local pgp-packet-library directory prior to executing:

1. % git submodule update --init
2. % cmake -B build && make -C build

Went into the newly added pgp-packet-library/examples directory where I saw the new CMakeLists.txt file that appears to have the ability to build key_from_raw_data.cpp. (I'm a Cmake newbie.)

Through some trial and error, I realized that the CMAKE_PREFIX_PATH environmental variable needed to be set to /Users/Scott/Projects/Libgcrypt/pgp-packet-library/cmake. Was unable to use pgp-packet_DIR as an environmental variable due to a "-" (dash) being part of the name instead of an "_" (underscore) recommended by the /Users/Scott/Projects/Libgcrypt/pgp-packet-library/examples/build/CMakeFiles/CMakeOutput.log log file.

% echo $CMAKE_PREFIX_PATH
/Users/Scott/Projects/Libgcrypt/pgp-packet-library/cmake

% cmake -B build
-- The CXX compiler identification is AppleClang 11.0.3.11030032
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at /Users/Scott/Projects/Libgcrypt/pgp-packet-library/cmake/pgp-packet-config.cmake:1 (include):
  include could not find load file:

    /Users/Scott/Projects/Libgcrypt/pgp-packet-library/cmake/pgp-packet-targets.cmake
Call Stack (most recent call first):
  CMakeLists.txt:7 (find_package)


-- Found Boost: /Users/Scott/usr2/local/lib/cmake/Boost-1.72.0/BoostConfig.cmake (found version "1.72.0")  
-- Found PkgConfig: /Users/Scott/usr2/local/bin/pkg-config (found version "0.29.2") 
-- Found Sodium: /Users/Scott/usr2/local/lib/libsodium.dylib  
-- Found CryptoPP: /Users/Scott/usr2/local/include (found version "8.1.0") 
-- Configuring incomplete, errors occurred!
See also "/Users/Scott/Projects/Libgcrypt/pgp-packet-library/examples/build/CMakeFiles/CMakeOutput.log".

/Users/Scott/Projects/Libgcrypt/pgp-packet-library/cmake/pgp-packet-targets.cmake does not exist:

% ls /Users/Scott/Projects/Libgcrypt/pgp-packet-library/cmake/pgp-packet-targets.cmake
ls: /Users/Scott/Projects/Libgcrypt/pgp-packet-library/cmake/pgp-packet-targets.cmake: No such file or directory

% ls -l /Users/Scott/Projects/Libgcrypt/pgp-packet-library/cmake
total 8
drwxr-xr-x   5 Scott  staff  160 Jun  4 15:19 ./
drwxr-xr-x  19 Scott  staff  608 Jun  4 15:24 ../
drwxr-xr-x   5 Scott  staff  160 Jun  4 15:19 Modules/
-rw-r--r--   1 Scott  staff  482 Jun  4 15:19 pgp-packet-config.cmake
drwxr-xr-x   4 Scott  staff  128 Jun  4 15:19 tests/

% ls -l /Users/Scott/Projects/Libgcrypt/pgp-packet-library/cmake/Modules/
total 24
drwxr-xr-x  5 Scott  staff   160 Jun  4 15:19 ./
drwxr-xr-x  5 Scott  staff   160 Jun  4 15:19 ../
-rw-r--r--  1 Scott  staff   851 Jun  4 15:19 CheckCXX17SourceRuns.cmake
-rw-r--r--  1 Scott  staff  1106 Jun  4 15:19 FindCryptoPP.cmake
-rw-r--r--  1 Scott  staff  1154 Jun  4 15:19 Findsodium.cmake

% ls -l /Users/Scott/Projects/Libgcrypt/pgp-packet-library/cmake/tests/
total 16
drwxr-xr-x  4 Scott  staff  128 Jun  4 15:19 ./
drwxr-xr-x  5 Scott  staff  160 Jun  4 15:19 ../
-rw-r--r--  1 Scott  staff  148 Jun  4 15:19 std_span_test.cpp
-rw-r--r--  1 Scott  staff  643 Jun  4 15:19 std_variant_test.cpp

[bysnack]

It seems that you didn't install the library, if you want the files to be found automatically you could just run

make -C build install

and then try to build the examples.

If you don't want to install the library you can also set the CMAKE_PREFIX_PATH to the build directory instead, an easy way to do this is to run this command inside of the examples directory

cmake -DCMAKE_PREFIX_PATH="../build/" .. -B build

This will allow CMake to search for the pgp-packet package in the directory it was built in.

Have in mind that this will use the library include interface since the headers were not installed under the pgp-packet directory in your include path and therefore you might need to slightly modify the header includes in the examples for them to work

...
+#include <packet.h>
-#include <pgp-packet/packet.h>
...

[skaht]

Thanks for the assist again. Got the 3 example executables to compile after 1st completing the install.

As a prior cybersecurity guy, I'm very reluctant to perform full up installs until I've established trust in using code. Hence, I usually try using code and headers from within build directories 1st.

After executing the key_from_raw_data executable the generated an output binary file called keyfile, I converted its binary content into a base64 encoded file called keyfile.asc and also prefixing it with the standard PGP header and postfixing it with a PGP trailer.

% cat keyfile.asc
-----BEGIN PGP PRIVATE KEY BLOCK-----

lFgEXPe1uxYJKwYBBAHaRw8BAQMEr1+yLGI+AwmQvDJwpVIkiMcORp1UAGTDE6sE
e32mXwQAAQDV9qCsUy+jsN7augXiO9bjC7naUZKikzPAXk+8b4tMYBHytCRBbm5l
IE9ueW1vdXMgPGFub255bW91c0BleGFtcGxlLm9yZz6IdAQTFggADwUCXPe1uwUJ
XPfDywIbAwAXFiEEQS2I/nlPAsGuwzrHZYdFQ3wZtZIXsgEA31nh+V/J/mtejfrh
SAcXFBVEnrH9ZB3cu6OwkO2FuAIBAN60LSTN+mVxTYE55HLk34gChunpNRZ7h+IQ
EbJINpcH
-----END PGP PRIVATE KEY BLOCK-----

That allows me to import the key into a keychain and also allows me to execute a crappy little tcsh script that I wrote last year after trying to decipher RFC 4880 and examining gpg source code. I will rewrite the script in Python this time around and plan to open source.

From the output of the the parse.csh script below, you can see I never figured out how to synthesize associated keygrip from an ed25519 public key and eight layers of gpg code just confused me. My conclusion after examining RFC 4880 from my myopic *.asc file analysis perspective, is that the specification is necessary, not sufficient, incomplete, and contains false information relative to the reference gpg implementation for PGP version 4 formatted packets.

% ./parse.csh keyfile.asc

9458045cf7b5bb16092b06010401da470f01010304af5fb22c623e030990bc3270a5522488c70e469d540064c313ab047b7da65f04000100d5f6a0ac532fa3b0dedaba05e23bd6e30bb9da5192a29333c05e4fbc6f8b4c6011f2b424416e6e65204f6e796d6f7573203c616e6f6e796d6f7573406578616d706c652e6f72673e887404131608000f05025cf7b5bb05095cf7c3cb021b030017162104412d88fe794f02c1aec33ac7658745437c19b59217b20100df59e1f95fc9fe6b5e8dfae14807171415449eb1fd641ddcbba3b090ed85b8020100deb42d24cdfa65714d8139e472e4df880286e9e935167b87e21011b248369707


1) ECC Secret-Key Packet (Tag 5): 0x94
   Raw Packet: 9458045cf7b5bb16092b06010401da470f01010304af5fb22c623e030990bc3270a5522488c70e469d540064c313ab047b7da65f04000100d5f6a0ac532fa3b0dedaba05e23bd6e30bb9da5192a29333c05e4fbc6f8b4c6011f2
   Packet size in bytes         : 90
   Packet body size in bytes    : 0x58 or 88
   Packet Version               : 0x04
             UNIX Epoch Time    : 0x5cf7b5bb  or  Wed Jun 5 08:29:47 EDT 2019
             Pubkey Algo index  : 0x16 -> EdDSA [RFC8032]
             OID Size in bytes  : 0x09 or 9
             Object Identifier  : 0x2b06010401da470f01 -> 1.3.6.1.4.1.11591.15.1, Ed25519
             KDF Size in bytes  : 0x01 or 1
             KDF Details        : 0x07 -> SHA2-256
             Canonical Prefix   : 0x04 -> Standard flag for uncompressed format
           ECC Public Key       : af5fb22c623e030990bc3270a5522488c70e469d540064c313ab047b7da65f04
           ECC Fingerprint Basis: 990033045cf7b5bb16092b06010401da470f01010304af5fb22c623e030990bc3270a5522488c70e469d540064c313ab047b7da65f04
           ECC Fingerprint      : 412d88fe794f02c1aec33ac7658745437c19b592
    BROKEN ECC Keygrip          : 5dfe4d1b2db7d6b3e716319251a1d877b878fb2c
           S2K Hash Header      : 0x00 -> Simple S2K
           S2K Hash Algorithm   : 0x01 -> SHA-1
           S2K Cipher Header    : 0x00 -> No S2K Cipher, No encryption, Cleartext
               Secret Key       : d5f6a0ac532fa3b0dedaba05e23bd6e30bb9da5192a29333c05e4fbc6f8b4c60
               Checksum         : 0x11f2

2) User ID Packet       (Tag 13): 0xb4
   Raw Packet: b424416e6e65204f6e796d6f7573203c616e6f6e796d6f7573406578616d706c652e6f72673e
   Packet size in bytes         : 38
   Packet body size in bytes    : 0x24 or 36
                 User ID String : Anne Onymous <[email protected]>

3) ECC Signature Packet  (Tag 2): 0x88
   Raw Packet: 887404131608000f05025cf7b5bb05095cf7c3cb021b030017162104412d88fe794f02c1aec33ac7658745437c19b59217b20100df59e1f95fc9fe6b5e8dfae14807171415449eb1fd641ddcbba3b090ed85b8020100deb42d24cdfa65714d8139e472e4df880286e9e935167b87e21011b248369707
   Packet size in bytes         : 118
   Packet body size in bytes    : 0x74 or 116
   Packet Version               : 0x04
                 Signature Type : 0x13 -> Positive certification of a User ID and Public-Key packet.
           Public Key Algorithm : 0x16 -> EdDSA [RFC8032]
                 Hash Algorithm : 0x08 -> SHA2-256 [FIPS180] or SHA256
              HASHED DATA COUNT : 0x000f or 15
   Subpacket #1 Length in bytes : 0x05 or 5
                 Subpacket Type : 0x02 -> Issuer Signature Creation Time - https://tools.ietf.org/html/draft-ietf-openpgp-rfc4880bis-08#section-5.2.3.4
                UNIX Epoch Time : 0x5cf7b5bb  or  Wed Jun 5 08:29:47 EDT 2019
   Subpacket #2 Length in bytes : 0x05 or 5
                 Subpacket Type : 0x09 -> Key Expiration Time - https://tools.ietf.org/html/draft-ietf-openpgp-rfc4880bis-08#section-5.2.3.6
                UNIX Epoch Time : 0x5cf7c3cb  or  Wed Jun 5 09:29:47 EDT 2019
   Subpacket #3 Length in bytes : 0x02 or 2
                 Subpacket Type : 0x1b -> Key Flags - https://tools.ietf.org/html/draft-ietf-openpgp-rfc4880bis-08#section-5.2.3.22
                    Hashed Data : 0x03
            UNHASHED DATA COUNT : 0x0017 or 23
   Subpacket #4 Length in bytes : 0x16 or 22
                 Subpacket Type : 0x21 -> Issuer Fingerprint - https://tools.ietf.org/html/draft-ietf-openpgp-rfc4880bis-08#section-5.2.3.28
                  Unhashed Data : 0x04412d88fe794f02c1aec33ac7658745437c19b592
          signedHashValuePrefix : 0x17b2
                   Delimiter #1 : 0x0100
         R Portion of Signature : df59e1f95fc9fe6b5e8dfae14807171415449eb1fd641ddcbba3b090ed85b802
                   Delimiter #2 : 0x0100
         S Portion of Signature : deb42d24cdfa65714d8139e472e4df880286e9e935167b87e21011b248369707

My next step is to successfully compile key_from_raw_data.cpp outside the pgp-packet-library build directories using very old fashion Makefiles.

FYI - The crappy little parse.csh script demonstrated above does have the ability to parse *.asc files containing both ed25519 and cv25519 keys including subkeys (Secret-Subkey Packet and complementary Signature Packet) to complement a master key or a master key stub and their associated signatures. So I will have the ability to test this library further... The to be developed *.asc Python PGP Packet parse script will complement https://github.com/ConradIrwin/gpg-decoder that is also incomplete.

[skaht]

Found the variables defined in pgp-packet-library/examples/build/CMakeFiles/key_from_raw_data.dir/flags.make to be important when one rolls their own Makefiles.

From a cybersecurity perspective, which approach segregate header file coupling the most?

1. CXX_INCLUDES = -isystem /Users/Scott/usr2/local/include -isystem /Users/Scott/usr2/local/include/pgp-packet/GSL/include
2. CXX_INCLUDES = -I$(INCLUDE_PGP_PACKET) -I$(INCLUDE_GSL) -I$(INCLUDE_CRYPTOPP) -I$(INCLUDE_LIBSODIUM) -I$(INCLUDE_BOOST)

#2

[skaht]

Above where the output of % ./parse.csh keyfile.asc is shown, the pgp-packet-library generated public key makes no sense.

1st assertion, RFC 4880bis specifies that RFC 8032 is used to compute the public key which I confirmed.

The example below yields identical results as the RFC 8032 test vector referenced immediately above:

% echo 9d61b19deffd5a60ba844af492ec2cc44449c5697b326919703bac031cae7f60 | ./25519
d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a

or alternatively

% echo 9d61b19deffd5a60ba844af492ec2cc44449c5697b326919703bac031cae7f60 | bx sha512 | cut -c 1-64 | clamp | ./sc_reduce32 | ./secret_key_to_public_key
d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a

To use libsodium to deterministically to calculate an RFC 8032 ed25519 public key from a specified private key the following approach can be applied:

 unsigned char  sk[  crypto_sign_ed25519_SECRETKEYBYTES] = { 0 },  // Zero-izing the binary sk is very important!
                pk[  crypto_sign_ed25519_PUBLICKEYBYTES],
                seed[crypto_sign_SEEDBYTES];

1. sodium_hex2bin(...) the 64 hexidecimal ASCII representation of the secret key to partially initialize sk
2. crypto_sign_ed25519_sk_to_seed( seed, sk ) to initialize the seed from the secret key
3. crypto_sign_seed_keypair( pk, sk, seed ) to finalize the initialization of sk and the public key pk
4. crypto_core_ed25519_is_valid_point( pk ) is optional but validates the pk is on the ed25519 curve

Anyhow, the correct pubic is independently calculated from the script below:

% echo d5f6a0ac532fa3b0dedaba05e23bd6e30bb9da5192a29333c05e4fbc6f8b4c60 | ./25519
de7e263408a32895953773d49138d598a319a916a1e1516e68e7fcc1947fc6ae

which does not match the public key extracted using the parse.csh keyfile.asc script above: af5fb22c623e030990bc3270a5522488c70e469d540064c313ab047b7da65f04

Most all the ed25519 *.asc files I've analyzed generated from gpg applications tend to have 0x40 for their prefix opposed to 0x04 that is set at https://github.com/summitto/pgp-packet-library/blob/master/examples/key_from_raw_data.cpp#L38.

https://tools.ietf.org/pdf/draft-koch-eddsa-for-openpgp-04.pdf complements RFC 4880bis09 concerning the 0x40 prefix.

% echo 1a8b1ff05ded48e18bf50166c664ab023ea70003d78d9e41f5758a91d850f8d2 | ./25519
3f098994bdd916ed4053197934e4a87c80733a1280d62f8010992e43ee3b2406

So look for 403f098994bdd916ed4053197934e4a87c80733a1280d62f8010992e43ee3b2406.

[bysnack]

Thanks for your research @skaht

After seeing what you did and doing some more investigation I realized that the key from raw data example was indeed wrong.

I updated the example in #57, the used keys should be correct now and the generated packet should be formatted according to RFC4880.

I suggest you give the new example a try.

[skaht]

@bysnack, thanks for assist:-) No problem sharing findings to improve this product. Will use the the updated pgp-packet-library package, and will continue to explore its PGP certificate synthesis capabilities to mimic prior PGP certificates/packets created by gpg.

BTW - The timing of your https://github.com/summitto/pgp-packet-library/blob/master/examples/key_from_raw_data.cpp update was perfect. Glad to see the pgp::signature_subpacket::issuer field initialization was added to the unhashed portion the signature_packet. Was struggling with how that field was to be initialized.

It is worth noting that public_key_data and secret_key_data variables would be easier for others to recognize by others if they were renamed master_ed25519_public_key and master_ed25519_secret_key. https://wiki.debian.org/Subkeys makes reference to master and subkey concepts.

Will be exercising pgp-packet-library's pgp::signature_subpacket::(features | exportable_certification | key_server_preferences | preferred_key_server | preferred_symmetric_algorithms | preferred_hash_algorithms | preferred_compression_algorithms ) subpackets/fields for the hashed portion the signature_packet, described further by https://tools.ietf.org/html/draft-ietf-openpgp-rfc4880bis-09#section-5.2.3.1 shortly.

[skaht]

Closing out this issue.

Continued exploration of pgp-packet-library has instead moved my interests on to #58 that will be the appropriate place to exercise pgp::signature_subpacket::(features | exportable_certification | key_server_preferences | preferred_key_server | preferred_symmetric_algorithms | preferred_hash_algorithms | preferred_compression_algorithms ).