Error with handshake process about spark-server

[jlkalberer]

Doh -- I didn't push. It's updated now.

[jlkalberer]

What I'm saying is that I implemented Electron support before they fixed that bug. In the past the Electron always sent ecc keys for UDP and TCP. Now that it's fixed, I can just use the RSA key code.

I'll try to find my electron and see if I can get this fixed tonight.

[jlkalberer]

Do you have something other than an electron to test with? I just want to make sure that it's a server issue and not with the setup process.

[jlkalberer]

It looks like you're doing everything correctly. I'll need to find my electron to debug this.

[jlkalberer]

I upgraded to the latest.

The reason it's happening to you is that you're using an Electron. Nobody really uses that and hosts their own cloud.

[jlkalberer]

What particle device are you using?
Did you configure the device to use TCP? <-- You need to manually add this flag for devices like electron
Did you run particle config myconfig before adding the device to your server? <-- I see a lot of weird errors when this isn't done

[xeanhort]

It's an electron, and yes! I did the step to change the profile (particle config myconfig) and the tcp step.

I had to change the command slightly to work tough: particle keys server default_key.pub.pem --host IP_ADDRESS --port 5683 --protocol tcp and also particle keys protocol --protocol tcp

[xeanhort]

By the way, I'm using the latest particle CLI version (2.3.0)

[jlkalberer]

There are two things I can think of

1. The server key was incorrectly set on the electron. I would retry this and make sure the file extension is correct.
2. Node v8.11.1 has a crypto bug or some behavior changed in the libraries

[xeanhort]

I did the whole process few times, but I can check again!
About node 8.11.1, I've used it because is the one used in this link. If you have an specific version of node/particleCLI I can use it and try again.

[jlkalberer]

Ok, I think I just need to use the original version of this - usrz/ec-key@23f4dd3

I'm guessing the dependencies are just out of date now.

[xeanhort]

I've changed package.json, deleted node_modules and executed yarn install again. The file yarn.lock was automatically updated with new versions of ec-key, bn.js and asn1.js but no luck connecting the device. The same Error: Failed to match tag: "objid" at: ["algorithmIdentifier"]["parameters"]

Is there anything I'm missing?

[jlkalberer]

I updated the dependency. You'll need to pull the latest version of this repo and yarn install --force to upgrade dependencies.

[xeanhort]

I've seen that spark-protocol repo was updated, but not this. Shouldn't this package.json be updated with the new ec-key ^0.0.4 too?

I can't update properly. It keeps downloading the spark-protocol without your last changes

[xeanhort]

Tried the with the updated version but still fails with the same error. Also I've moved to node v14.2.0 without any improvement :(

This is the "updated" error log:

2020-05-06T07:45:09.161Z] ERROR: Handshake.js/5216 on 8NEO6A5: Handshake failed (cache_key=_2, deviceID=XXXXXXXXXXXXXXXXXXXXXXXX, ip=::ffff:XX.XX.XX.XX)
Error: Failed to match tag: "objid" at: ["algorithmIdentifier"]["parameters"]
at DecoderBuffer.error (D:\spark-server\node_modules\asn1.js\lib\asn1\base\reporter.js:78:11)
at DERNode.decodeTag [as _decodeTag] (D:\spark-server\node_modules\asn1.js\lib\asn1\decoders\der.js:71:19)
at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:341:25)
at decodeChildren (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:378:15)
at Array.forEach ()
at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:375:22)
at decodeChildren (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:378:15)
at Array.forEach ()
at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:375:22)
at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:280:47)
at Generated.decode (D:\spark-server\node_modules\asn1.js\lib\asn1\decoders\der.js:28:20)
at Entity.decode (D:\spark-server\node_modules\asn1.js\lib\asn1\api.js:44:32)
at parseSpki (D:\spark-server\node_modules\ec-key\src\ec-key.js:158:27)
at parsePem (D:\spark-server\node_modules\ec-key\src\ec-key.js:184:12)
at new ECKey (D:\spark-server\node_modules\ec-key\src\ec-key.js:237:15)
at new DeviceKey (D:\spark-server\node_modules\spark-protocol\dist\lib\DeviceKey.js:32:25)
at DeviceKey.equals (D:\spark-server\node_modules\spark-protocol\dist\lib\DeviceKey.js:75:20)
at Handshake._callee5$ (D:\spark-server\node_modules\spark-protocol\dist\lib\Handshake.js:383:29)
at tryCatch (D:\spark-server\node_modules\regenerator-runtime\runtime.js:62:40)
at Generator.invoke [as _invoke] (D:\spark-server\node_modules\regenerator-runtime\runtime.js:296:22)
at Generator.prototype. [as next] (D:\spark-server\node_modules\regenerator-runtime\runtime.js:114:21)
at step (D:\spark-server\node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
at D:\spark-server\node_modules\babel-runtime\helpers\asyncToGenerator.js:28:13
[2020-05-06T07:45:09.161Z] INFO: Device.js/5216 on 8NEO6A5: Device disconnected (cache_key=_2, deviceID="", disconnectCounter=1)
[2020-05-06T07:45:09.163Z] ERROR: DeviceServer.js/5216 on 8NEO6A5: Device startup failed (deviceID=null)
Error: Failed to match tag: "objid" at: ["algorithmIdentifier"]["parameters"]
at DecoderBuffer.error (D:\spark-server\node_modules\asn1.js\lib\asn1\base\reporter.js:78:11)
at DERNode.decodeTag [as _decodeTag] (D:\spark-server\node_modules\asn1.js\lib\asn1\decoders\der.js:71:19)
at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:341:25)
at decodeChildren (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:378:15)
at Array.forEach ()
at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:375:22)
at decodeChildren (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:378:15)
at Array.forEach ()
at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:375:22)
at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:280:47)
at Generated.decode (D:\spark-server\node_modules\asn1.js\lib\asn1\decoders\der.js:28:20)
at Entity.decode (D:\spark-server\node_modules\asn1.js\lib\asn1\api.js:44:32)
at parseSpki (D:\spark-server\node_modules\ec-key\src\ec-key.js:158:27)
at parsePem (D:\spark-server\node_modules\ec-key\src\ec-key.js:184:12)
at new ECKey (D:\spark-server\node_modules\ec-key\src\ec-key.js:237:15)
at new DeviceKey (D:\spark-server\node_modules\spark-protocol\dist\lib\DeviceKey.js:32:25)
at DeviceKey.equals (D:\spark-server\node_modules\spark-protocol\dist\lib\DeviceKey.js:75:20)
at Handshake._callee5$ (D:\spark-server\node_modules\spark-protocol\dist\lib\Handshake.js:383:29)
at tryCatch (D:\spark-server\node_modules\regenerator-runtime\runtime.js:62:40)
at Generator.invoke [as _invoke] (D:\spark-server\node_modules\regenerator-runtime\runtime.js:296:22)
at Generator.prototype. [as next] (D:\spark-server\node_modules\regenerator-runtime\runtime.js:114:21)
at step (D:\spark-server\node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
at D:\spark-server\node_modules\babel-runtime\helpers\asyncToGenerator.js:28:13
[2020-05-06T07:45:13.036Z] ERROR: Handshake.js/5216 on 8NEO6A5: Handshake failed (cache_key=_1, deviceID=null, ip=::ffff:XX.XX.XX.XX)
Error: Handshake did not complete in 10 seconds
at Timeout._onTimeout (D:\spark-server\node_modules\spark-protocol\dist\lib\Handshake.js:236:23)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7)
[2020-05-06T07:45:13.037Z] INFO: Device.js/5216 on 8NEO6A5: Device disconnected (cache_key=_1, deviceID="", disconnectCounter=1)

[jlkalberer]

The only thing I can think of is that the key is improperly formatted.

Can you run a test script with ec-key which tries to parse the device's key? (I am slammed at work so I can't debug this)
var key = new ECKey(pem_key, 'pem');
I'm wondering if the format of the device key has changed in newer versions of Particle firmware. I use P1s for my fleet so I don't ever test this.

[xeanhort]

There the test with the two keys. The one in deviceKeys.db (also matches the generated XXXXX_rsa_new.pub.pem) and the one received from the device during handshake.

const ECKey = require('ec-key');

device_db_pem = "-----BEGIN PUBLIC KEY-----\nMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEaUXZhpYnCARKEx1FGFrOgu8Dgyb5\n//Drwxg2oR8LkP37MDj7ESmj78PdBqD6PeNmvBMQg5Z7NQ8saRDxX1h50g==\n-----END PUBLIC KEY-----\n"

device_sent_pem = "-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQBThDnRSoUGN20VitFNf4Kj1wWh\nIdiJvFJ4CPj2oAoGCCqGSM49AwEHoUQDQgAEaUXZhpYnCARKEx1FGFrOgu8Dgyb5\n//Drwxg2oR8LkP37MDj7ESmj78PdBqD6PeNmvBMQg5Z7NQ8saRDxX1h50v//////\n/////////////////wIDAQAB\n-----END PUBLIC KEY-----\n"

var key = new ECKey(device_db_pem, 'pem');
console.log(key)

var key2 = new ECKey(device_sent_pem, 'pem');
console.log(key2)
/* The second throws the error:
D:\spark-server>node test.js

D:\spark-server\node_modules\asn1.js\lib\asn1\base\reporter.js:84
    throw err;
    ^
ReporterError: Failed to match tag: "objid" at: ["algorithmIdentifier"]["parameters"]
    at DecoderBuffer.error (D:\spark-server\node_modules\asn1.js\lib\asn1\base\reporter.js:78:11)
    at DERNode.decodeTag [as _decodeTag] (D:\spark-server\node_modules\asn1.js\lib\asn1\decoders\der.js:71:19)
    at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:341:25)
    at decodeChildren (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:378:15)
    at Array.forEach (<anonymous>)
    at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:375:22)
    at decodeChildren (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:378:15)
    at Array.forEach (<anonymous>)
    at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:375:22)
    at DERNode.decode [as _decode] (D:\spark-server\node_modules\asn1.js\lib\asn1\base\node.js:280:47) {
  path: '["algorithmIdentifier"]["parameters"]'
}
*/

[jlkalberer]

Ok -- that's super helpful. It's definitely a different format. I'm wondering if it's just an RSA key instead of an EC key.

particle-iot/particle-cli#280

This makes me think that I don't need the ec-key parsing code since I'm not using UDP.

[xeanhort]

I'm not completely sure of how is this issue related because I always use --protocol tcp in all particlecli commands. Also, that issue is closed, so not sure how they finally implemented it.

[jlkalberer]

Let me know if this works -- I couldn't find my electron so I am just guessing at a fix.

[xeanhort]

It now seems to compare the keys properly but still the keys doesn't match:

07:36:51.287Z  INFO DeviceServer.js: New Connection
07:36:52.332Z ERROR Handshake.js: Handshake failed (cache_key=_31, deviceID=XXXXXXXXXXXXXXXXXXXXXXXX, ip=::ffff:XX.XX.XX.XX)
    Error: key passed to device during handshake doesn'tmatch saved public key: XXXXXXXXXXXXXXXXXXXXXXXX
        at Handshake._callee5$ (D:\spark-server\node_modules\spark-protocol\dist\lib\Handshake.js:388:21)
        at tryCatch (D:\spark-server\node_modules\regenerator-runtime\runtime.js:62:40)
        at Generator.invoke [as _invoke] (D:\spark-server\node_modules\regenerator-runtime\runtime.js:296:22)
        at Generator.next (D:\spark-server\node_modules\regenerator-runtime\runtime.js:114:21)
        at step (D:\spark-server\node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
        at D:\spark-server\node_modules\babel-runtime\helpers\asyncToGenerator.js:28:13
07:36:52.333Z  INFO Device.js: Device disconnected (cache_key=_31, deviceID="", disconnectCounter=1)
07:36:52.334Z ERROR DeviceServer.js: Device startup failed (deviceID=null)
    Error: key passed to device during handshake doesn'tmatch saved public key: XXXXXXXXXXXXXXXXXXXXXXXX
        at Handshake._callee5$ (D:\spark-server\node_modules\spark-protocol\dist\lib\Handshake.js:388:21)
        at tryCatch (D:\spark-server\node_modules\regenerator-runtime\runtime.js:62:40)
        at Generator.invoke [as _invoke] (D:\spark-server\node_modules\regenerator-runtime\runtime.js:296:22)
        at Generator.next (D:\spark-server\node_modules\regenerator-runtime\runtime.js:114:21)
        at step (D:\spark-server\node_modules\babel-runtime\helpers\asyncToGenerator.js:17:30)
        at D:\spark-server\node_modules\babel-runtime\helpers\asyncToGenerator.js:28:13

I've did the complete process with the updated code to re-generate the device/database keys and be sure that it's all set properly.

deviceProvidedPem="-----BEGIN PUBLIC KEY-----\nMIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQBxq9GblXxQs0tjD3uWXhQsieLx\nKdzewcasniQNoAoGCCqGSM49AwEHoUQDQgAECdp0rE3z+K9Dgi40Og+668Vvkaiz\nacsyDzUZ3kpMPnp+fphkUApsQ0CLGp4E3S/hQQYZKfgS0qUhMcLNcQEjDP//////\n/////////////////wIDAQAB\n-----END PUBLIC KEY-----"

Database: "algorithm":"ecc","key":"-----BEGIN PUBLIC KEY-----\nMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAECdp0rE3z+K9Dgi40Og+668Vvkaiz\nacsyDzUZ3kpMPnp+fphkUApsQ0CLGp4E3S/hQQYZKfgS0qUhMcLNcQEjDA==\n-----END PUBLIC KEY-----\n"

[xeanhort]

Yes! I've tried with Photon and it works!

But still not working with Electron. I also checked a Boron but it doesn't even connect, so might be an error of UDP/TCP.

[jlkalberer]

So the ecc shouldn't show up in the database anymore. Did you delete the database before retrying?

[xeanhort]

All devices have ecc key in the database (except photon). I'm pretty sure I've deleted it before the last update.
I can check again tomorrow to be sure.

[xeanhort]

I've did it again cleaning the database but it still shows ecc.

Database:
{"deviceID":"XXXXXXXXXXXXXXX","algorithm":"ecc","key":"-----BEGIN PUBLIC KEY-----\nMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEqDnXk6K0TP14HmWmLpePHvn8Avq0\nyI3dQ1/Q4suA5dnkhZvPoMiSErseQ1+vi2b67DiooM9qYx8fYdlzyzSuxg==\n-----END PUBLIC KEY-----\n","_id":"ahWDZpOx5ZlOnPrv"}

Keys generated in the computer: https://we.tl/t-C9NqPlanxq

Commands used:

• particle config xxxx apiUrl "http://custom_host.com:8080"
• particle config xxxx
• DFU
• particle keys server default_key.pub.pem --host custom_host.com --port 5683 --protocol tcp
• particle keys protocol --protocol tcp
• particle keys doctor XXXXXXXXXXXXXXXXXXXX

[jlkalberer]

I got my Electron set up this morning but did not hook it back up to a network so I just tested and verified that it would send an RSA key instead of an ECC key.

Try this:

particle keys new test_key --protocol tcp
particle keys load test_key.der
particle keys send XXXXXXXXXXXXXXXXXXXXXXXX test_key.pub.pem

I'm going to open an issue with Particle CLI but I doubt they will fix it on their end. particle keys doctor should respect the protocol

particle-iot/particle-cli#582

[xeanhort]

I executed your commands and it connected properly! RSA in the database and everything seems OK about the keys. But there's a problem during the protocol initialization:

2020-05-12T17:31:04.926Z] INFO: DeviceServer.js/4608 on 8NEO6A5: Connected Devices (devices=0, sockets=0)
[2020-05-12T17:31:10.530Z] INFO: DeviceServer.js/4608 on 8NEO6A5: New Connection
[2020-05-12T17:31:12.786Z] INFO: Device.js/4608 on 8NEO6A5: Connection attributes (particleProductId=10, platformId=10, productFirmwareVersion=65535, reservedFlags=6)
[2020-05-12T17:31:12.787Z] INFO: DeviceServer.js/4608 on 8NEO6A5: Connection (connectionID=1, deviceID=XXXXXXXXXXXXXXXXXXXXXXXX, remoteIPAddress=::ffff:XX.XX.XX.XX)
[2020-05-12T17:31:14.926Z] INFO: DeviceServer.js/4608 on 8NEO6A5: Connected Devices (devices=1, sockets=1)
[2020-05-12T17:31:24.927Z] INFO: DeviceServer.js/4608 on 8NEO6A5: Connected Devices (devices=1, sockets=1)
[2020-05-12T17:31:27.850Z] ERROR: /4608 on 8NEO6A5: completeProtocolInitialization (appHash=null, deviceID=XXXXXXXXXXXXXXXXXXXXXXXX, functions=null, ip=::ffff:XX.XX.XX.XX, lastHeard=null, ownerID=null, particleProductId=10, platformId=10, productFirmwareVersion=65535, registrar=null, reservedFlags=6, variables=null)
Error: Request timed out - Describe
at Timeout._onTimeout (D:\spark-server\node_modules\spark-protocol\dist\clients\Device.js:1118:26)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7)
[2020-05-12T17:31:27.850Z] ERROR: DeviceServer.js/4608 on 8NEO6A5: Connection Error (deviceID=XXXXXXXXXXXXXXXXXXXXXXXX)
Error: Request timed out - Describe
at Timeout._onTimeout (D:\spark-server\node_modules\spark-protocol\dist\clients\Device.js:1118:26)
at listOnTimeout (internal/timers.js:549:17)
at processTimers (internal/timers.js:492:7)
[2020-05-12T17:31:27.850Z] INFO: Device.js/4608 on 8NEO6A5: Device disconnected (cache_key=_1, deviceID=XXXXXXXXXXXXXXXXXXXXXXXX, duration=15.055, disconnectCounter=1)
[2020-05-12T17:31:27.851Z] WARN: DeviceServer.js/4608 on 8NEO6A5: Session ended for Device (connectionKey=_1, deviceID=XXXXXXXXXXXXXXXXXXXXXXXX, ownerID=null)
[2020-05-12T17:31:34.927Z] INFO: DeviceServer.js/4608 on 8NEO6A5: Connected Devices (devices=0, sockets=0)

I've done the process from the beginning but always happen the same. I tried DeviceOS 0.7.0 and 1.2.1.

[jlkalberer]

Well, that's progress!

For some reason it seems like the DESCRIBE message isn't being returned. I can't remember what that does off the top of my head but I'll check it out tonight or tomorrow morning.

[xeanhort]

Yes! Thank you for all your support!

BTW: Which version do you use for the ParticleCLI / DeviceOS? It seems weird that all this errors are happening only to me, :(

[jlkalberer]

So I connected my Electron to my server and don't see the Describe error. The device does disconnect every 30 seconds but that's expected because of #291

[xeanhort]

Maybe it's related to 1.2.1 version? Not sure about the Describe message but tell me if I can help somehow!

[jlkalberer]

Could be. My Electron is the latest.

Can you try flashing your Electron with tinker? Then try it with the latest system firmware.