Fail to pass signature verification about pyc2pa

[ethanwu155]

The reason hash is wrong is because extracted claim.json is incorrect. Currently extracted claim.json from testing data is (includes label and content):

{"label":"cai.claim","content":{"assertions":["self#jumbf=cai/cb.starling_1/cai.assertions/cai.location.broad?hl=mEiAiWeYKpcrzcEZUwo1l1J09GPElaDfnEarwHnJvlAh/oA","self#jumbf=cai/cb.starling_1/cai.assertions/cai.rights?hl=mEiDz9sNp/tzDru0T9PjM6InZBzjMLlg0XX6kYTbs8DtdJw","self#jumbf=cai/cb.starling_1/cai.assertions/cai.claim.thumbnail.jpg?hl=mEiD/j+T9FkU0KbQW4xMsc0ySxVsDyglHqRToq9/Fon6fkg","self#jumbf=cai/cb.starling_1/cai.assertions/cai.acquisition.thumbnail.jpg?hl=mEiD/j+T9FkU0KbQW4xMsc0ySxVsDyglHqRToq9/Fon6fkg","self#jumbf=cai/cb.starling_1/cai.assertions/adobe.asset.info?hl=mEiCm/aYUgYWjwC0emm90PWK2qxpbXxqC6/bMJMkKevY1LA","self#jumbf=cai/cb.starling_1/cai.assertions/starling.integrity?hl=mEiBX5zocwF840JwzsiC+LHPubE9MTn93t8utThK/kguMsg"],"asset_hashes":[{"length":"0x0000000000009959","name":"JFIF SOI-APP0","start":"0x0000000000000000","url":"","value":"EiAuxjtmax46cC2N3Y9aFmBO9Jfay8LEwJWzBUtZ0sUM8gA="},{"length":"0x000000000000027d","name":"JFIF APP1/XMP","start":"0x0000000000009959","url":"","value":"EiDjZifCgG2iKxcYeChKTOcWlJ9I/UC9/c5XFiJREqJFpwA="},{"length":"0x00000000000215e6","name":"JFIF DQT-EOI","start":"0x000000000000a90c","url":"","value":"EiArx031oA0N5KOEG6n9R/bJJFYJvmGlDoLtuwbRipLTKAA="}],"recorder":"Starling Capture using Numbers Protocol","signature":"self#jumbf=cai/cb.starling_1/cai.signature"}}

Signed claim should be:

{"assertions":["self#jumbf=cai/cb.starling_1/cai.assertions/cai.location.broad?hl=mEiAiWeYKpcrzcEZUwo1l1J09GPElaDfnEarwHnJvlAh/oA","self#jumbf=cai/cb.starling_1/cai.assertions/cai.rights?hl=mEiDz9sNp/tzDru0T9PjM6InZBzjMLlg0XX6kYTbs8DtdJw","self#jumbf=cai/cb.starling_1/cai.assertions/cai.claim.thumbnail.jpg?hl=mEiD/j+T9FkU0KbQW4xMsc0ySxVsDyglHqRToq9/Fon6fkg","self#jumbf=cai/cb.starling_1/cai.assertions/cai.acquisition.thumbnail.jpg?hl=mEiD/j+T9FkU0KbQW4xMsc0ySxVsDyglHqRToq9/Fon6fkg","self#jumbf=cai/cb.starling_1/cai.assertions/adobe.asset.info?hl=mEiCm/aYUgYWjwC0emm90PWK2qxpbXxqC6/bMJMkKevY1LA","self#jumbf=cai/cb.starling_1/cai.assertions/starling.integrity?hl=mEiBX5zocwF840JwzsiC+LHPubE9MTn93t8utThK/kguMsg"],"asset_hashes":[{"length":"0x0000000000009959","name":"JFIF SOI-APP0","start":"0x0000000000000000","url":"","value":"EiAuxjtmax46cC2N3Y9aFmBO9Jfay8LEwJWzBUtZ0sUM8gA="},{"length":"0x000000000000027d","name":"JFIF APP1/XMP","start":"0x0000000000009959","url":"","value":"EiDjZifCgG2iKxcYeChKTOcWlJ9I/UC9/c5XFiJREqJFpwA="},{"length":"0x00000000000215e6","name":"JFIF DQT-EOI","start":"0x000000000000a90c","url":"","value":"EiArx031oA0N5KOEG6n9R/bJJFYJvmGlDoLtuwbRipLTKAA="}],"recorder":"Starling Capture using Numbers Protocol","signature":"self#jumbf=cai/cb.starling_1/cai.signature"}

If using the correct claim contents:

$ python endesive-sign.py -v test/certificate.crt.pem test/claim.json test/claim-signature.der 
Verifying Signature
signature ok? True
hash ok? True
cert ok? True

[ethanwu155]

The above signature verification is for the cms signature implementation that is not the recommended digital signature implementation. I have added the following endesive CADES-B signature implementation in the utils folder. It includes endesive_sign.py script for signing and verifying. README also available with steps Adobe took to verify signature.

Summary:

General p12 and cert.pem

1. Generate Public and Private Key with the following:
   openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 7
openssl pkcs12 -export -out <filename>.p12 -inkey key.pem -in cert.pem

   Output will be the following:

   <filename>.p12
   

2. Generate certificate for verification with the following:
   openssl pkcs12 -in <filename>.p12 -out <filename>crt.pem -clcerts -nokeys

   Output will be the following:

   <filename>.crt.pem
   

3. Generate Signature with:

   python endesive-sign.py -s <p12> <claim JSON> <name of signature file.der>

4. Verify Signature with:

   python endesive-sign.py -v <crt.pem> <claim JSON> <signature file .der>

Sample Usage:

# claim json : starling.claim.json

# Generate p12 
# Will output file certificate.p12 (will be used to generate signature). set pass to 1234
$ openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 7
$ openssl pkcs12 -export -out certificate.p12 -inkey key.pem -in cert.pem

# Generate crt.pem
# Will output file certificate.crt.pem (will be used to verify signed) 
$ openssl pkcs12 -in certificate.p12 -out certificate.crt.pem -clcerts -nokeys


# Usage:
# Generating Signature:
$ python endesive-sign.py -s certificate.p12 starling.claim.json starling.der
Generating Signature

# Verifying Signature:
$ python endesive-sign.py -v certificate.crt.pem starling.claim.json starling.der
Verifying Signature
signature ok? True
hash ok? True
cert ok? True

Verifying using Adobe's Methodology

Requirements

• Extracted CMS signature to starling.der
• Extracted claim JSON to starling.claim.json (exact byte sequence inserted into image)

# Convert signature from DER to PEM encoding:
$ openssl pkcs7 -inform der -in starling.der -out starling.der.pkcs7

# Extract X.509 certificates from signature:
$ openssl pkcs7 -print_certs -in starling.der.pkcs7 -out starling.der.cert

# Verify CMS signature against detached data (claim):
$ openssl smime -verify -binary -inform der -in starling.der -content starling.claim.json -certfile starling.der.cert -noverify
{
    "assertions": [
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.location.precise?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMoBUAVbTEB7K",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.sensors?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMvY4QFEN3973",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.device?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMwEoBojZcUrZ",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.integrity?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMo3SG72sZg13"
    ],
    "asset_hashes": [
        {
            "start": "0x0000000000000000",
            "length": "0x0000000000009959",
            "name": "JFIF SOI-APP0",
            "url": "",
            "value": "EiAuxjtmax46cC2N3Y9aFmBO9Jfay8LEwJWzBUtZ0sUM8gA="
        },
        {
            "start": "0x0000000000009959",
            "length": "0x000000000000027d",
            "name": "JFIF APP1/XMP",
            "url": "",
            "value": "EiDjZifCgG2iKxcYeChKTOcWlJ9I/UC9/c5XFiJREqJFpwA="
        },
        {
            "start": "0x000000000000a90c",
            "length": "0x00000000000215e6",
            "name": "JFIF DQT-EOI",
            "url": "",
            "value": "EiArx031oA0N5KOEG6n9R/bJJFYJvmGlDoLtuwbRipLTKAA="
        }
    ],
    "recorder": "Starling Capture",
    "signature": "self#jumbf=cai/cb.starling_1/cai.signature"
Verification successful

[ethanwu155]

Added tutorial portion of signature-verification. Also was able to set passphrase for p12 to ''.

See Below:

# Export existing pkcs12 to pem file
$ openssl pkcs12 -in certificate.p12 -nodes -out temp.pem
Enter Import Password:
MAC verified OK

# convert pem back to p12 w/ no password
$ openssl pkcs12 -export -in temp.pem  -out unprotected.p12
Enter Export Password:
Verifying - Enter Export Password:

# remove temp certfiicate
$ rm temp.pem

# generate crt.pem
$ openssl pkcs12 -in unprotected.p12 -out unprotected.crt.pem -clcerts -nokeys
Enter Import Password:
MAC verified OK

# sign claim
$ python endesive-sign.py -s unprotected.p12 starling.claim.json unprotected.der
Generating Signature

# verify signature
$ python endesive-sign.py -v unprotected.crt.pem starling.claim.json unprotected.der 
Verifying Signature
signature ok? True
hash ok? True
cert ok? True

Adobe Verification works the same way:

$ openssl pkcs7 -inform der -in unprotected.der -out unprotected.der.pkcs7

$ openssl pkcs7 -print_certs -in unprotected.der.pkcs7 -out unprotected.der.cert

$ openssl smime -verify -binary -inform der -in unprotected.der -content starling.claim.json -certfile unprotected.der.cert -noverify
{
    "assertions": [
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.location.precise?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMoBUAVbTEB7K",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.sensors?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMvY4QFEN3973",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.device?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMwEoBojZcUrZ",
        "self#jumbf=cai/cb.starling_1/cai.assertions/starling.integrity?hl=z26ycANRgtWbqYX9cdsWD4rsTqz8RYHQArrq4CZJwZn1cxX73kTP6x3rRcBsUfMo3SG72sZg13"
    ],
    "asset_hashes": [
        {
            "start": "0x0000000000000000",
            "length": "0x0000000000009959",
            "name": "JFIF SOI-APP0",
            "url": "",
            "value": "EiAuxjtmax46cC2N3Y9aFmBO9Jfay8LEwJWzBUtZ0sUM8gA="
        },
        {
            "start": "0x0000000000009959",
            "length": "0x000000000000027d",
            "name": "JFIF APP1/XMP",
            "url": "",
            "value": "EiDjZifCgG2iKxcYeChKTOcWlJ9I/UC9/c5XFiJREqJFpwA="
        },
        {
            "start": "0x000000000000a90c",
            "length": "0x00000000000215e6",
            "name": "JFIF DQT-EOI",
            "url": "",
            "value": "EiArx031oA0N5KOEG6n9R/bJJFYJvmGlDoLtuwbRipLTKAA="
        }
    ],
    "recorder": "Starling Capture",
    "signature": "self#jumbf=cai/cb.starling_1/cai.signature"
Verification successful

CAI code has been modified to take no passphrase:

elif type_sig=='endesive':
                # load_key_and_certificates second parameter is password to decrypt the data. Can be set to None of PKCS12 is not encrypted
                # https://cryptography.io/en/latest/hazmat/primitives/asymmetric/serialization.html
                key = pkcs12.load_key_and_certificates(f.read(), b'', backends.default_backend())

[bafu]

@ethanwu155 I followed the steps but the hash check is False.

My steps and testing data (cicada.zip) are below. Can you help check it? Thanks!

$ endesive-sign.py -v certificate.crt.pem claim.json claim-signature.der
Verifying Signature
signature ok? True
hash ok? False
cert ok? True

[bafu]

When creating claim.json manually, you might need to remove the '0x0a' at the end of the file.

Check if there is 0x0a at the end of the file:

$ xxd claim.json
00000000: 7b22 6173 7365 7274 696f 6e73 223a 5b22  {"assertions":["
...
00000510: 7265 227d 0a                             re"}.

Remove 0x0a by Vim as example:

$ vim -b claim.json

# In Vim
:set noeol
:wq

Double check if there is 0x0a:

$ xxd claim.json
00000000: 7b22 6173 7365 7274 696f 6e73 223a 5b22  {"assertions":["
...
00000510: 7265 227d                                re"}

[bafu]

Closed by PR #18